mirror of
https://github.com/vale981/ray
synced 2025-03-09 04:46:38 -04:00
460cf86858
This PR adds support for automatic block splitting on read and map transforms, to keep block size bounded to ~500MiB. This avoids potential OOM situations where a map task may consume too much intermediate Python heap memory, or too much object store shared memory for one block.
1 line
No EOL
45 KiB
XML
1 line
No EOL
45 KiB
XML
