mirror of
https://github.com/vale981/ray
synced 2025-03-06 10:31:39 -05:00
50d49a2d7a
Looking at past failures of dataset_shuffle_push_based_random_shuffle_1tb and when running it on my own, I noticed that raylets are killed because GCS was not able to respond to it in time. It seems at the beginning of the run, there is a huge CPU spike which starved GCS out of CPU. With the same spirit of adjusting workers to higher OOM scores, we can give workers higher niceness so they yield CPU to GCS, Raylet and other user processes. I ran dataset_shuffle_push_based_random_shuffle_1tb a few time which no longer sees raylet death because of GCS CPU starvation. But there are other issues making the test fail which I will continue to investigate. |
||
|---|---|---|
| .. | ||
| ray | ||
| requirements | ||
| asv.conf.json | ||
| build-wheel-macos-arm64.sh | ||
| build-wheel-macos.sh | ||
| build-wheel-manylinux2014.sh | ||
| build-wheel-windows.sh | ||
| MANIFEST.in | ||
| README-building-wheels.md | ||
| requirements.txt | ||
| requirements_linters.txt | ||
| requirements_ml_docker.txt | ||
| setup.py | ||