Commit graph

8026 commits

Author SHA1 Message Date
mwtian
dce13d3a81
Explicitly set protobuf dependency version to allow building ray with bazel 4.0.0 (#15756)
Java protobuf dependency version is made to be consistent as well.
2021-05-13 10:34:09 -07:00
SangBin Cho
259fcbd5bd
[Pubsub] Generalize the pubsub interface and adapt it for ref counting protocol (#15446)
* Add mock code first

* In the initial progress.

* Fix the number error

* In progress.

* in more pgoress.

* in progress.

* lint.

* Prototype done.

* Fix compilation bug.

* Now it is working with reference counting.

* Remove template.

* lint.

* Fixed issues.

* Fix reference count test.

* Reference count test passes now.

* Fixed the test array problem

* Addressed code review.

* lint.

* Addressed half of code review.

* Fix tests.

* Addressed the most critical issue.

* Make subscriber thread-safe.

* Revert "Make subscriber thread-safe."

This reverts commit 9a6a52197cfa8463ab60dfaae9530ad3c0ed8790.

* Fixed test failures. The only failure now is the asan failure.

* Reset test suites and see if it fixes the issue.

* Fix a flaky test

* Addressed code review.
2021-05-13 09:29:02 -07:00
architkulkarni
a0c1cfe034
[Core] Pass RuntimeEnv as opaque string in the task spec (#15658) 2021-05-13 10:32:00 -05:00
Simon Mo
d97ff80149
[Core] Fix global state keep initializing in busy loop (#15777) 2021-05-13 10:30:49 -05:00
SongGuyang
40b2face74
Fix std::atomic compiling error (#15781) 2021-05-13 10:27:45 -05:00
Tao Wang
19462e43d6
[large scale]use proxy to track gcs server address in core worker (#15714) 2021-05-13 19:26:01 +08:00
fcardoso75
c877da4c19
create_and_mmap_buffer() - In case CreateFileMapping() fails, GetLastError() return code is printed (#15773)
* Enabling all test cases on test_client.py

* Moving test_client.py to a large CI py_test_module_list

* Disabling test_client::test_remote_functions

* Divide Run CI script action into separete Build action and Test action

* Reverting test_client.py to separate work for different tickes

* Reverting python\ray\tests\BUILD to separate work for different tickets

* create_and_mmap_buffer() - In case CreateFileMapping() fails, GetLastError() return code is printed

* Addressed lint comments

Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
2021-05-13 00:31:33 -07:00
Richard Liaw
ec27f9cc6b
[tune] avoid system exit error (#15687)
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
2021-05-13 00:24:30 -07:00
Sven Mika
c4a3e1589b
[RLlib] CQL: Bug fixes and OPE example added to test and offline_rl.py example. (#15761) 2021-05-13 09:17:23 +02:00
Philipp Moritz
78d0ed3503
[Windows] Fix Test Log Upload (#15769) 2021-05-12 17:57:49 -07:00
Ian Rodney
cdf93930f3
Revert "[Core] Fix event loop instrumentation causing Java segfaults in tests. (#15349)" (#15727)
This reverts commit edb0d1b376.
2021-05-12 15:49:06 -07:00
SangBin Cho
2fc892166f
[Test] unskip global gc (#15767) 2021-05-12 15:45:59 -07:00
Ian Rodney
6e3ec98b72
[test] Clean-Up Test-Top-Level API (#15713) 2021-05-12 13:50:17 -07:00
mwtian
6a044f4f30
[Test] Ensure output params are initialized before calling IsPlasmaObjectPinnedOrSpilled() (#15758) 2021-05-12 10:22:35 -07:00
Dmitri Gekhtman
052d2acaee
[autoscaler][kubernetes] Restart after head failure, more consistent operator restart behavior. (#15655) 2021-05-12 11:49:11 -05:00
Sven Mika
16ddab49f5
[RLlib] Trainer._evaluate -> Trainer.evaluate; Also make evaluation possible w/o evaluation worker set. (#15591) 2021-05-12 12:16:00 +02:00
fyrestone
56c309416e
[Job submission] Basic job submission structure (#15103) 2021-05-12 15:08:20 +08:00
fcardoso75
fcf56fb970
Divide run ci script action (#15657)
* Enabling all test cases on test_client.py

* Moving test_client.py to a large CI py_test_module_list

* Disabling test_client::test_remote_functions

* Divide Run CI script action into separete Build action and Test action

* Reverting test_client.py to separate work for different tickes

* Reverting python\ray\tests\BUILD to separate work for different tickets
2021-05-11 21:39:18 -07:00
qicosmos
e4fb1c163e
[C++ Worker] Support overload functions for REAY_REMOTE (#14981)
Co-authored-by: qicosmos <chufeng.qy@antgroup.com>
2021-05-12 11:19:39 +08:00
Clark Zinzow
c1b7d6f115
Don't consider a worker to be idle if it has in-flight object pinning RPCs. (#15686) 2021-05-11 19:21:52 -07:00
Eric Liang
82d5b67521
Remove placement group log spam (#15747) 2021-05-11 17:08:06 -07:00
Edward Oakes
8c2ac84988
[serve] Don't filter logs by job_id when using Serve (#15691)
* dont filter logs

* modify worker
2021-05-11 15:54:44 -07:00
Eric Liang
cb59d30917
Drop profiling events if the GCS becomes backlogged (#15726) 2021-05-11 14:10:34 -07:00
Ashwin Hegde
4d8ed6dd5c
#13890 [new-dashboard] add object store memory column (#15697) 2021-05-11 15:36:16 -05:00
Simon Mo
4459a5a2b1
[Serve] Fix ASGI Lifespan (#15732) 2021-05-11 10:43:56 -07:00
qicosmos
4795048f1b
[C++ Worker] Support ray remote in one line (#14979)
Co-authored-by: qicosmos <chufeng.qy@antgroup.com>
Co-authored-by: Ian Rodney <ian.rodney@gmail.com>
2021-05-11 16:29:21 +08:00
Edward Oakes
bf6f1cfa9a
[serve] Clean up package ref (#15690) 2021-05-11 01:23:58 -07:00
mwtian
285885c86e
Mention MacOS in the instructions to build Ray from source (#15731) 2021-05-11 00:19:45 -07:00
Ian Rodney
0632fd65d6
[Test][Flaky] Mark test_trial_scheduler_pbt as flaky (#15729) 2021-05-10 23:50:37 -07:00
Ian Rodney
5d938514b5
[Tests] Re-enable disabled tests (#15723) 2021-05-10 22:26:24 -07:00
Ian Rodney
90ce25cb35
[dashboard] Avoid global min_workers (#15660) 2021-05-10 15:47:51 -07:00
Tom Dörr
3c99f1db4c
[Docs] Tune Contributors fix (#15719) 2021-05-10 12:22:47 -07:00
Tom Dörr
b5c03b6458
Fix Link (#15722) 2021-05-10 12:19:32 -07:00
Ian Rodney
c50490ccef
[Lint] Pin Prettier to 2.3.0 (#15721) 2021-05-10 11:46:29 -07:00
Ian Rodney
11b5c6c702
[HotFix][Lint] Fix Lint because of Prettier update (#15720) 2021-05-10 09:51:41 -07:00
SongGuyang
b8ff86adb9
Add objectStore stats to dashboard API. (#15677) 2021-05-10 11:32:14 -05:00
Sven Mika
a495759f06
[RLlib] Discussion 2022: PPO should auto-adjust rollout_fragment_length if other settings do not align with train_batch_size. (#15611) 2021-05-10 16:16:02 +02:00
Sven Mika
461d73ddf1
[RLlib] simple_optimizer should not be used by default for tf+MA. (#15365) 2021-05-10 16:10:44 +02:00
Sven Mika
46f6fa2361
[RLlib] Example script for restoring 1 agent (out of n) from a checkpoint (multi-agent). (#15540) 2021-05-10 16:09:05 +02:00
dependabot[bot]
0026cf1be6
[tune](deps): Bump mxnet in /python/requirements (#15705)
Bumps [mxnet](https://github.com/apache/incubator-mxnet) from 1.7.0.post1 to 1.8.0.post0.
- [Release notes](https://github.com/apache/incubator-mxnet/releases)
- [Changelog](https://github.com/apache/incubator-mxnet/blob/master/NEWS.md)
- [Commits](https://github.com/apache/incubator-mxnet/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-09 10:25:29 -07:00
Kai Fricke
2a4a862929
[tune] add max_concurrent option to BasicVariantGenerator (#15680) 2021-05-08 21:07:52 -07:00
dependabot[bot]
e1246ff0fb
[tune](deps): Bump matplotlib from 3.3.4 to 3.4.2 in /python/requirements (#15702)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>
2021-05-08 21:06:29 -07:00
Eric Liang
0dfd43c61b
Add nightly release test directory and add shuffle release test (#15671)
* update

* udpate

* update

* update

* update

* Adjust script/release test json

* remove

* update

* lint

Co-authored-by: Kai Fricke <kai@anyscale.com>
2021-05-08 14:21:55 -07:00
dependabot[bot]
e778168957
[tune](deps): Bump h5py from 3.1.0 to 3.2.1 in /python/requirements (#15694)
* [tune](deps): Bump h5py from 3.1.0 to 3.2.1 in /python/requirements

Bumps [h5py](https://github.com/h5py/h5py) from 3.1.0 to 3.2.1.
- [Release notes](https://github.com/h5py/h5py/releases)
- [Changelog](https://github.com/h5py/h5py/blob/master/docs/release_guide.rst)
- [Commits](https://github.com/h5py/h5py/compare/3.1.0...3.2.1)

Signed-off-by: dependabot[bot] <support@github.com>

* Update python/requirements/requirements_tune.txt

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>
2021-05-08 12:57:22 -07:00
Kai Fricke
8db2e5c23a
[release] Move xgboost tune small + microbenchmark release test to new release automation (#15619) 2021-05-08 20:38:39 +01:00
Eric Liang
996a002b00
Add prepopulate plasma memory flag for debugging (#15669)
* add prepopulate flag

* fix build

* warn
2021-05-07 15:17:31 -07:00
Clark Zinzow
edb0d1b376
[Core] Fix event loop instrumentation causing Java segfaults in tests. (#15349)
* Reenable event loop instrumentation.

* Take stats handle by copy in post() handler closure.

* Revert "Take stats handle by copy in post() handler closure."

This reverts commit e46777939bcc3bb4bb101e136e9d3348ea4ae1a1.
2021-05-07 15:01:00 -07:00
Eric Liang
0fb2b6560d
Remove unnecessary flaky branch of test_object_spilling_2 on OSX (#15673) 2021-05-07 11:59:38 -07:00
Ian Rodney
1d834bcbe3
[Client] Add gRPC Keep-Alive messages 2021-05-06 13:02:28 -07:00
Simon Mo
56c103bb7e
[Serve] Move the doc warning location (#15667) 2021-05-06 12:08:51 -07:00