hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-05 18:11:42 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

[Test] Fix Dask on Ray 1 TB bug #22431 Open

Browse source 
Fixes a bug. It seems like not df is not working with dataframe
This commit is contained in:
SangBin Cho 2022-02-17 02:44:36 +09:00 • committed by GitHub
parent 331b71ea8d
commit 42361a1801
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
release/nightly_tests/dask_on_ray/dask_on_ray_sort.py
View file

@ -57,9 +57,10 @@ def load_dataset(client, data_dir, s3_bucket, nbytes, npartitions):
df = None
max_retry = 3
retry = 0
while not df and retry < max_retry:
while retry < max_retry:
try:
df = dd.read_parquet(filenames)
break
except FileNotFoundError as e:
print(f"Failed to load a file. {e}")
# Wait a little bit before retrying.