ray/python/ray
Takeshi Yoshimura e115545579
[ray client] enable ray.get with >2 sec timeout (#21883) (#22165)
Commit 2cf4c72 ("[ray client] Fix ctrl-c for ray.get() by setting a
short-server side timeout") introduced a short server-side timeout not
to block later operations.

However, the fix implicitly assumes that get() is complete within
MAX_BLOCKING_OPERATION_TIME_S (two seconds). This becomes a problem
when apps use heavy objects or limited network I/O bandwidth that
require more than two seconds to push all chunks. The current retry
logic needs to re-push from the beginning of chunks and block clients
with the infinite re-push.

I updated the logic to directly pass timeout if it is explicitly given.
Without timeout, it still uses MAX_BLOCKING_OPERATION_TIME_S for
polling with the short server-side timeout.
2022-04-25 13:06:52 -07:00
..
_private improve exponential backoff when connecting to the redis (#24150) 2022-04-25 16:10:24 +08:00
autoscaler [KubeRay] Fix autoscaling with GPUs and custom resources, with e2e tests (#23883) 2022-04-21 14:54:37 -07:00
cloudpickle Revert "Revert "[serialization] Enable debugging into pickle backend (#23854)"(#23877)" (#23878) 2022-04-14 11:07:54 -07:00
core Define common data structures with protobuf. (#5121) 2019-07-08 22:41:37 +08:00
data [Datasets] [Out-of-Band Serialization: 3/3] Add out-of-band serialization. (#23932) 2022-04-25 12:47:06 -07:00
experimental Fix the failure of sort_main in the case of num_cpus > 1 and not an integer (#24099) 2022-04-24 11:54:08 +08:00
includes Node affinity scheduling strategy (#23381) 2022-04-12 21:31:26 -07:00
internal [core] Spill at least the object fusion size instead of at most (#22750) 2022-04-05 10:57:42 -07:00
job_submission [jobs] Rename JobData -> JobInfo (#22499) 2022-02-22 16:18:16 -06:00
ml [air] Use checkpoint.as_directory() instead of cleaning up manually (#24113) 2022-04-23 14:52:30 +01:00
ray_operator [Lint] Cleanup incorrectly formatted strings (Part 3: components) (#23130) 2022-03-16 12:36:57 -07:00
scripts [Core] Ray logs API (1/n) (#23435) 2022-04-20 23:11:02 -07:00
serve [Serve] Fix deployment func no args called with python (#24096) 2022-04-25 11:15:44 -05:00
sgd [Train] Rename Ray SGD v2 to Ray Train (#19436) 2021-10-18 22:27:46 -07:00
streaming [CI] Format Python code with Black (#21975) 2022-01-29 18:41:57 -08:00
tests Revert "[grpc] Upgrade grpc to 1.45.2 (#24064)" (#24145) 2022-04-23 23:47:11 -07:00
train [train] Copy resources_per_worker to avoid modifying user input 2022-04-23 15:01:35 -07:00
tune Revert "Revert "[tune] Also interrupt training when SIGUSR1 received"" (#24101) 2022-04-22 11:27:38 +01:00
util [ray client] enable ray.get with >2 sec timeout (#21883) (#22165) 2022-04-25 13:06:52 -07:00
workers Add storage-based spilling backend (#23341) 2022-03-21 19:17:42 -07:00
workflow [CI] Create zip of ray session_latest/logs dir on test failure and upload to buildkite via /artifact-mount (#23783) 2022-04-22 09:48:53 +01:00
__init__.py [Core] remove Windows compatibility for Redis (#23991) 2022-04-19 09:16:47 -07:00
_raylet.pxd [client] Move Client implementation of ObjectRef/ActorRef to python (#22148) 2022-02-06 13:03:51 -08:00
_raylet.pyi [Core] Type check ObjectRef (#9856) 2020-08-11 10:38:29 -07:00
_raylet.pyx [core] Add task function name to ray.timeline profiling events (#23920) 2022-04-19 09:38:52 -07:00
actor.py [core] Fix DAG options (#23880) 2022-04-18 10:32:43 -07:00
client_builder.py [Lint] Cleanup incorrectly formatted strings (Part 4: tests, _private) (#23263) 2022-03-17 00:49:16 -07:00
cluster_utils.py [gcs] Remove use_gcs_for_bootstrap in core (python) and autoscaler (#23050) 2022-03-11 14:36:16 -08:00
cross_language.py [core] Simplify options handling [Part 1] (#23127) 2022-04-11 20:49:58 -07:00
dashboard [dashboard] Rename "new_dashboard" -> "dashboard" (#18214) 2021-09-15 11:17:15 -05:00
exceptions.py Node affinity scheduling strategy (#23381) 2022-04-12 21:31:26 -07:00
external_storage.py Add storage-based spilling backend (#23341) 2022-03-21 19:17:42 -07:00
job_config.py [runtime env] Support set timeout for runtime env setup (#23082) 2022-03-18 12:52:59 -05:00
nightly-wheels.yaml fix wheel links (#17973) 2021-08-23 13:43:34 -07:00
node.py [Core] Add node_name field to GcsNodeInfo (#23543) 2022-04-19 05:03:12 -07:00
py.typed Support ray task type checking (#9574) 2020-07-21 19:05:42 -07:00
ray_constants.py improve exponential backoff when connecting to the redis (#24150) 2022-04-25 16:10:24 +08:00
remote_function.py [core] Fix DAG options (#23880) 2022-04-18 10:32:43 -07:00
rllib [rllib] Try moving RLlib to top level dir (#5324) 2019-08-05 23:25:49 -07:00
runtime_context.py [docs] fix doctests and activate CI (#23418) 2022-03-24 17:04:02 -07:00
runtime_env.py Fix broken links in documentation and put linkcheck linter in place on CI (#23340) 2022-03-18 21:02:52 -07:00
serialization.py Node affinity scheduling strategy (#23381) 2022-04-12 21:31:26 -07:00
serialization_addons.py [CI] Format Python code with Black (#21975) 2022-01-29 18:41:57 -08:00
setup-dev.py [ml] Trainer implementation (#22969) 2022-03-15 20:35:54 -07:00
state.py [Core] Add node_name field to GcsNodeInfo (#23543) 2022-04-19 05:03:12 -07:00
types.py [data] Fix the ObjectRef type in the dataset docs (#17111) 2021-07-15 09:50:37 -07:00
utils.py [runtime env] Support set timeout for runtime env setup (#23082) 2022-03-18 12:52:59 -05:00
worker.py [runtime_env] Add the ability to inject a setup hook for customization of runtime_env on init (#24036) 2022-04-20 13:27:37 -07:00
worker.pyi [CI] Replace YAPF disables with Black disables (#21982) 2022-02-08 16:29:25 -08:00