3 commits

Author	SHA1	Message	Date
xwjiang2010	9af8f11191	Revert "[docs] Clean up doc structure (first part) (#21667)" (#21763) ... This reverts commit 38e46c9fb3.	2022-01-20 15:30:56 -08:00
Max Pumperla	38e46c9fb3	[docs] Clean up doc structure (first part) (#21667)	2022-01-20 16:19:04 +01:00
Eric Liang	460cf86858	Split blocks automatically into 500MB chunks on file read and transformation (#20235) ... This PR adds support for automatic block splitting on read and map transforms, to keep block size bounded to ~500MiB. This avoids potential OOM situations where a map task may consume too much intermediate Python heap memory, or too much object store shared memory for one block.	2021-11-15 22:25:11 -08:00