Ian Rodney
f9daf7fa2c
[Connect] Separate Specific Server logs into different files ( #16534 )
...
* better debug logging
* log to separate files
* use getter/setter methods
* actually use type for ray.node.Node
* update docstring
* use public method
* add a test
* fix glob glob
* fix test again
2021-07-01 21:27:03 -07:00
Kai Fricke
521a05834d
[tune] Fix resume=REMOTE documentation and log more details ( #16709 )
2021-07-01 18:40:15 -07:00
Eric Liang
7eec197d31
[data] Support block metadata and lazy loading of blocks from the filesystem ( #16689 )
2021-07-01 17:25:20 -07:00
Siyuan (Ryans) Zhuang
b570606438
[Workflow] Ensure asynchronous execution of "workflow.run" & fix TODOs ( #16727 )
...
* initial commit
* code refactoring; non-blocking "workflow.run"
* Rename "workflow_root_dir" to "storage_url"
* readable workflow_id and step_id
* verbose logging
* remove unused code
* slugify
* storage -> storage_or_url
* fix comments
* storage_or_url -> storage
* get workflow outputs without extra object copy & transfer
* update tests
* tests workflow outputs resolving
2021-07-01 15:53:22 -07:00
Edward Oakes
ff37bdcbd9
[serve] Fix bazel test sizes now that we're on buildkite ( #16785 )
2021-07-01 16:27:31 -05:00
Chen Shen
34cd6a862a
[dataset] implements split ( #16765 )
2021-07-01 13:03:34 -07:00
Alex Wu
c27a02a807
[Dataset] read binary files ( #16717 )
...
* done
* done
* done
* .
* .
* lint
* .
Co-authored-by: Alex Wu <alex@anyscale.com>
2021-07-01 13:00:10 -07:00
Clark Zinzow
52da2cce68
[Dataset] Adds JSON, CSV, Pandas, and Dask IO layers, and adds the write side of the Parquet IO layer. ( #16724 )
2021-07-01 11:57:40 -07:00
Edward Oakes
a6051ea3dc
[serve] Rolling updates for user_config changes ( #15909 )
2021-07-01 13:31:59 -05:00
Clark Zinzow
096b38df8b
[Dataset] Miscellaneous hotfixes. ( #16730 )
2021-07-01 10:50:06 -07:00
Edward Oakes
691f0a1dd1
[serve] Remove unnecessary STOPPED ReplicaState ( #16783 )
2021-07-01 12:35:00 -05:00
Kai Fricke
6f48e4f566
[tune/wandb] Fix parameter lookup for sequences ( #16806 )
2021-07-01 15:22:25 +01:00
mwtian
7669708237
Create a wait_for_num_nodes() function, and use it in train_small ( #16784 )
2021-07-01 10:17:53 +01:00
Antoni Baum
2cec40d0f5
[tune] LightGBM integration callbacks ( #16713 )
2021-07-01 10:12:57 +01:00
Siyuan (Ryans) Zhuang
18232bb808
[Core][Serialization] Fix recursive resolve/deserialization ( #16794 )
...
* fix recursive resolve
* fix threading issues
2021-07-01 01:44:24 -07:00
Amog Kamsetty
2c17f4907e
[Tune] Add pbt_transformers Client test ( #16793 )
2021-06-30 23:10:16 -07:00
chenk008
06c7db7dca
[Core] Rename container option and ray-nest-container ( #16771 )
...
* rename container_option to container
* rename ray-nest-container to ray-worker-container
* lint
Co-authored-by: wuhua.ck <wuhua.ck@alibaba-inc.com>
2021-07-01 13:12:26 +08:00
SangBin Cho
0ad2dac3ca
[Dataset] Map batch implementation ( #16729 )
2021-06-30 21:51:04 -07:00
SangBin Cho
94149ef50c
[Test] Skip some flaky tests ( #16768 )
...
* done
* Add test gcs fault tolerance to the flaky test
* Addressed code review
* Addressed the last code review.
2021-06-30 20:48:09 -07:00
Dmitri Gekhtman
0cbeefdadf
[autoscaler] GPU=0 resource tweak ( #16761 )
...
* tweak
* use >, some fields don't admit an order
2021-06-30 16:51:25 -07:00
Ian Rodney
1f14f366fd
[Client] Make Client_Hook per-thread ( #16731 )
2021-06-30 11:48:55 -07:00
architkulkarni
3962ce286b
[Core] Bump override_environment_variables removal date from Ray 1.5 to 1.6 ( #16755 )
2021-06-30 09:31:08 -05:00
architkulkarni
077c04b06c
[Serve] Fix fastapi tutorial and update doc with deprecation warning ( #16759 )
2021-06-30 09:30:49 -05:00
SongGuyang
1d5c72ebde
[C++ API] rename 'ray_redis_address' to 'ray_address' ( #16772 )
2021-06-30 20:50:28 +08:00
Kai Fricke
4b6f8a1ece
[cli] use shutil.move in ray cluster-dump and allow passing of tempfile ( #16738 )
2021-06-30 07:49:03 +01:00
Amog Kamsetty
69507f53db
[Horovod] Add Horovod example ( #16742 )
...
* wip
* updates
* updates
* update
* formatting
* updates
* updates
* update
* fix
* add timeout
2021-06-29 19:15:15 -07:00
Simon Mo
2ac8a197db
[Serve] Copy FastAPI ResponseModel field ( #16760 )
2021-06-29 16:28:08 -07:00
Patrick Ames
cf8785b0e1
[docs] Note that ordering of objects returned is preserved for ray.get. ( #16763 )
2021-06-29 16:17:16 -07:00
Richard Liaw
bcb73ed58b
finished impl ( #16753 )
2021-06-29 14:37:42 -07:00
Amog Kamsetty
56068f8f81
Skip test_component_failures_2 on Windows ( #16745 )
2021-06-29 14:06:09 -07:00
Ian Rodney
b8f950775e
[Client] Keep client_mode for dumps_from_client ( #16732 )
2021-06-29 13:30:10 -07:00
Amog Kamsetty
c0560dadef
[Docker] Pin Tensorflow ( #16741 )
2021-06-29 11:14:46 -07:00
Dmitri Gekhtman
257d072d13
[kubernetes][release] K8s release test instructions ( #16662 )
2021-06-29 10:57:35 -07:00
chenk008
c318293d9f
[Core] start worker in container ( #16671 )
2021-06-29 10:12:47 -07:00
Ian Rodney
b3532cc2d1
[Client][Test] Avoid Port-Reuse to DeFlake ( #16697 )
...
Co-authored-by: mwtian <81660174+mwtian@users.noreply.github.com>
2021-06-28 23:54:06 -07:00
Ian Rodney
a9df1b7a67
[Test][Modin] Actually run test_modin ( #16719 )
2021-06-28 20:39:30 -07:00
SangBin Cho
804a867b3d
Revert revert OBOD pubsub PR ( #16487 )
...
* Revert "Revert "[Pubsub] Use a pubsub module for Ownership based object directory (#16407 )" (#16486 )"
This reverts commit b986938f0f
2021-06-28 20:30:31 -07:00
SongGuyang
41b9a5102b
[C++ worker] support build C++ worker during python setup ( #16636 )
2021-06-29 10:29:47 +08:00
Amog Kamsetty
322b9531f6
[SGD] Add __init__ file to tf.examples ( #16726 )
2021-06-28 19:23:22 -07:00
Ian Rodney
1a357a7e4f
[Client] Auto-Run ray.client().connect() ( #16259 )
2021-06-28 17:01:26 -07:00
Travis Addair
e5dfa4cfb9
[tune] Only use TBXLoggerCallback when torch is installed ( #16695 )
...
* [tune] Only use TBXLoggerCallback when torch is installed
* Fix lint
* fix
* Update python/ray/tune/utils/callback.py
Co-authored-by: Amog Kamsetty <amogkamsetty@yahoo.com>
Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>
2021-06-28 16:34:20 -07:00
Alex Wu
0811ae4231
Use the same worker id in python and C++ ( #16712 )
...
Co-authored-by: Alex <alex@anyscale.com>
2021-06-28 15:42:37 -07:00
architkulkarni
b9f6132c08
skip flaky conda env fixture on MacOS ( #16710 )
2021-06-28 09:38:17 -07:00
Tao Wang
38157a3166
[Core]support external redis address when starting ray processes ( #13170 )
...
* support external redis address when starting ray processes
* use a more general name
* add cli option
* handle some details
* fix set shards logic
* reuse --address instead of introduce a new one
* lint
* tiny
* lint and fix
2021-06-28 09:22:40 -07:00
Kai Fricke
04bfba1274
[tune] Move reporter detection to utility function ( #16673 )
...
Test failures seem unrelated
2021-06-28 12:55:05 +01:00
Amog Kamsetty
54ce8092ab
[Tune] Update transformers to 4.6.1 ( #16397 )
...
* add examples
* update dask docs
* add build file
* formatting
* fix ci command
* fix
* Update python/ray/util/dask/BUILD
* newline
* fix pytest fixtures
* fixes
* formatting
* fix shuffle example
* update
* dont log to wandb
2021-06-26 14:10:47 -07:00
Eric Liang
aa882ed52d
Make it more convenient to develop ray.data by setting RAY_EXPERIMENTAL_DATA_API=1 ( #16685 )
...
* make it convenient to import ray.data
* update
* Update python/ray/experimental/data/read_api.py
Co-authored-by: Alex Wu <itswu.alex@gmail.com>
Co-authored-by: Alex Wu <itswu.alex@gmail.com>
2021-06-26 09:17:30 -07:00
Eric Liang
6bfa97eed7
Check in the first iteration of an Arrow-based dataset api ( #16648 )
2021-06-25 18:45:13 -07:00
Dmitri Gekhtman
7b58ec9ae5
[autoscaler] rsync bootstrap flag ( #16667 )
2021-06-25 15:26:47 -07:00
Eric Liang
9b17c35bee
Fix PullManager handling of get requests and liveness issues ( #16394 )
2021-06-25 13:01:46 -07:00
