Simon Mo
761b584814
[Serve] Improve buffering for simple cases ( #9485 )
2020-07-16 15:34:08 -07:00
Stephanie Wang
baf4be245d
Fix flaky test_actor_failures::test_actor_restart ( #9509 )
...
* Fix flaky test
* os exit
2020-07-16 10:48:33 -07:00
Ameer Haj Ali
1e46d4e29f
[Autoscaler] Making bootstrap config part of the node provider interface ( #9443 )
...
* supporting custom bootstrap config for external node providers
* bootstrap config
* renamed config to cluster_config
* lint
* remove 2 args from importer
* complete move of bootstrap to node_provider
* renamed provider_cls
* move imports outside functions
* lint
* Update python/ray/autoscaler/node_provider.py
Co-authored-by: Eric Liang <ekhliang@gmail.com>
* final fixes
* keeping lines to reduce diff
* lint
* lamba config
* filling in -> adding for lint
Co-authored-by: Ameer Haj Ali <ameerhajali@Ameers-MacBook-Pro.local>
Co-authored-by: Eric Liang <ekhliang@gmail.com>
2020-07-16 09:54:20 -07:00
SangBin Cho
41ad5de1c4
Fix broken test_raylet_info_endpoint ( #9511 )
2020-07-16 11:51:06 -05:00
mehrdadn
ac39e23145
Get rid of build shell scripts and move them to Python ( #6082 )
2020-07-16 11:26:47 -05:00
SangBin Cho
2f674728a6
[GCS Actor Management] Gcs actor management broken detached actor ( #9473 )
2020-07-16 15:41:18 +08:00
kisuke95
5e2571e214
release gil in global state accessor ( #9357 )
2020-07-16 11:21:10 +08:00
Stephanie Wang
4e81804cba
[core] Replace task resubmission in raylet with ownership protocol ( #9394 )
...
* Add intended worker ID to GetObjectStatus, tests
* Remove TaskID owner_id
* lint
* Add owner address to task args
* Make TaskArg a virtual class, remove multi args
* Set owner address for task args
* merge
* Fix tests
* Add ObjectRefs to task dependency manager, pass from task spec args
* tmp
* tmp
* Fix
* Add ownership info for task arguments
* Convert WaitForDirectActorCallArgs
* lint
* build
* update
* build
* java
* Move code
* build
* Revert "Fix Google log directory again (#9063 )"
This reverts commit 275da2e400
2020-07-15 14:55:51 -07:00
krfricke
5a40299d42
[tune] extend PTL template (GPU, typing fixes, tensorboard) ( #9451 )
...
Co-authored-by: Kai Fricke <kai@anyscale.com>
2020-07-15 10:30:20 -07:00
mehrdadn
aa8928fac2
Make more tests compatible with Windows ( #9303 )
2020-07-15 11:34:33 -05:00
mehrdadn
33e400998c
Fix name clash on Windows ( #9412 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-14 23:14:53 -07:00
Stephanie Wang
6d99aa34a5
[core] Handle out-of-order actor table notifications ( #9449 )
...
* Drop stale actor table notifications
* build
* Add num_restarts to disconnect handler
* Unit test and increment num_restarts on ALIVE, not RESTARTING
* Wait for pid to exit
2020-07-14 22:55:04 -07:00
Edward Oakes
7eafe646a9
Fix flaky test_object_manager.py ( #9472 )
2020-07-14 18:44:48 -05:00
mehrdadn
ca4f3b79db
Speedups for GitHub Actions ( #9343 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-14 14:51:51 -07:00
Michael Mui
e93cde8c66
[tune] Issue 8821: ExperimentAnalysis doesn't expand user ( #9461 )
2020-07-14 13:53:37 -07:00
SangBin Cho
539c51a003
[Core] Support GCS server port assignment. ( #8962 )
2020-07-14 11:49:56 -05:00
SangBin Cho
f6eb47fc1f
[Stats] metrics agent exporter ( #9361 )
2020-07-14 11:49:16 -05:00
Siyuan (Ryans) Zhuang
5b192842b5
Fix ObjectRef and ActorHandle serialization ( #9462 )
2020-07-14 09:42:32 -07:00
krfricke
deba082cb4
[tune] PyTorch CIFAR10 example ( #9338 )
...
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
Co-authored-by: Kai Fricke <kai@anyscale.com>
2020-07-13 23:16:05 -07:00
Max Fitton
222635b63f
Machine View Sorting / Grouping ( #9214 )
...
* Convert NodeInfo.tsx to a functional component
* Update NodeRowGroup to be a functional component
* lint
* Convert TotalRow to functional component.
* lint
* move node info over to using the sortable table head component. spacing is still a little wonky.
* Factor a NoewWorkerRow class out of NodeRowGroup that will be usable when grouping / ungrouping
* Compilation checkpoint, I factored the worker filtering logic out of node info into the reducer
* Add sort accessors for CPU
* Add sort accessors for Disk
* Add sort accessors for RAM
* add a table sort util for function based accessors (rather than flat attribute-based accessor)
* wip refactor node info features
* wip
* Rendering Checkpoint. I've refactored the features and how they are called to add sorting support. Also reworks the way error counts and log counts are passed to the front-end to remove some ugly logic
* wip
* wip
* wip
* Finish adding sorting and grouping of machine view
* lint
* fix bug in filtration of logs and errors by worker from recent refactor.
* Add export of Cluster Disk feature
* fix some merge issues
Co-authored-by: Max Fitton <max@semprehealth.com>
2020-07-13 20:45:17 -05:00
SangBin Cho
22b2e51152
Fix test-multi-node ( #9453 )
2020-07-13 20:44:27 -05:00
Vasily Litvinov
6ad13e0da8
Add ability to specify SOCKS proxy for SSH connections ( #8833 )
2020-07-13 16:10:07 -07:00
Henk Tillman
c7714ca575
GCP authentication using oauth tokens ( #9279 )
2020-07-13 14:36:40 -07:00
Ian Rodney
0085cf75d0
Allow --lru-evict to be passed into ray start ( #8959 )
2020-07-13 14:09:39 -07:00
Amog Kamsetty
4454d05bcf
[Tune] Trainable documentation fix ( #9448 )
2020-07-13 13:15:01 -07:00
Nicolaus93
b5a6c57295
[tune] handling nan values ( #9381 )
2020-07-12 17:08:36 -07:00
Henk Tillman
8c985dc797
Update conda and ray wheel on GCP images ( #9388 )
2020-07-12 12:12:27 -07:00
Ian Rodney
26fcda50e7
Pass run args to DockerCommandRunner ( #9411 )
2020-07-10 18:09:01 -07:00
Simon Mo
d4a5d09dab
[Serve] Merge router with HTTPProxy ( #9225 )
2020-07-10 13:52:48 -07:00
Hao Chen
d49dadf891
Change Python's ObjectID to ObjectRef ( #9353 )
2020-07-10 17:49:04 +08:00
Hao Chen
bed1be611e
Fix flaky test_dynres.py ( #9310 )
2020-07-10 10:34:23 +08:00
Patrick Ames
dc51b08c36
[autoscaler] Allow users to disable the cluster config cache ( #8117 )
...
* [autoscaler] Remove autoscaler config cache.
* [autoscaler] Add flag allowing users to explicitly disable the config cache.
2020-07-09 15:47:58 -07:00
Eric Liang
09b9b81ea4
[autoscaler] Move command runners into separate file and clean up interface. ( #9340 )
...
* cleanup
* wip
* fix imports
* fix lint
2020-07-09 15:40:56 -07:00
Zhuohan Li
8a76f4cbb5
[Core] put small objects in memory store ( #8972 )
...
* remove the put in memory store
* put small objects directly in memory store
* cast data type
* fix another place that uses Put to spill to plasma store
* fix multiple tests related to memory limits
* partially fix test_metrics
* remove not functioning codes
* fix core_worker_test
* refactor put to plasma codes
* add a flag for the new feature
* add flag to more places
* do a warmup round for the plasma store
* lint
* lint again
* fix warmup store
* Update _raylet.pyx
Co-authored-by: Eric Liang <ekhliang@gmail.com>
2020-07-09 15:39:40 -07:00
Alex Wu
34b85659d4
[Core] New scheduler fixes ( #9186 )
...
* .
* test_args passes
* .
* test_basic.py::test_many_fractional_resources causes ray to hang
* test_basic.py::test_many_fractional_resources causes ray to hang
* .
* .
* useful
* test_many_fractional_resources fails instead of hanging now :)
* Passes test_fractional_resources
* .
* .
* Some cleanup
* git is hard
* cleanup
* Fixed scheduling tests
* .
* .
2020-07-09 15:37:51 -07:00
Stephanie Wang
0389735d7a
[core] Pass owner address from the workers to the raylet ( #9299 )
...
* Add intended worker ID to GetObjectStatus, tests
* Remove TaskID owner_id
* lint
* Add owner address to task args
* Make TaskArg a virtual class, remove multi args
* Set owner address for task args
* merge
* Fix tests
* Add ObjectRefs to task dependency manager, pass from task spec args
* tmp
* tmp
* Fix
* Add ownership info for task arguments
* Convert WaitForDirectActorCallArgs
* lint
* build
* update
* build
* java
* Move code
* build
* Revert "Fix Google log directory again (#9063 )"
This reverts commit 275da2e400
2020-07-09 14:35:54 -07:00
fangfengbin
62fe403a57
Fix the bug that PollOwnerForActorOutOfScope's reply may be missing ( #9330 )
2020-07-09 17:33:52 +08:00
Kai Yang
fa1eadd61f
fix java worker log dir ( #9352 )
2020-07-09 10:11:03 +08:00
SangBin Cho
9f8ff2e3b1
[Core] GCS Actor Management Race Condition ( #9215 )
...
* GCS Actor management on by default.
* Fix travis config.
* Change condition.
* Finish the initial race condition fix.
* Lint.
* Refine the codebase.
* Finish the initial version
* Improve logic.
* Remove unnecessary log messages.
* Address code review.
* Add tests
* Revert the second race condition that doesn't happen anymore, handle some edge cases. add tests.
* Address the second race condition found.
* Addressed code review.
* Addressed code review.
* Run a new unit test only when gcs actor management is on.
2020-07-08 10:56:52 -07:00
Ian Rodney
9172f8c3a6
[core] Store Internal Config in GCS ( #8921 )
2020-07-08 11:22:08 -05:00
Sven Mika
4da0e542d5
[RLlib] DDPG and SAC eager support (preparation for tf2.x) ( #9204 )
2020-07-08 16:12:20 +02:00
Tao Wang
f395e48031
[GCS] Add integration test for actor info cleaning ( #8900 )
2020-07-08 10:25:08 +08:00
Max Fitton
ca54da66b6
Worker <> GPU Mapping Information in Dashboard ( #9163 )
2020-07-07 18:58:22 -05:00
yncxcw
4ba4110dec
[Core] Make worker_register_timeout_seconds configurable ( #9221 )
2020-07-07 18:27:15 -05:00
SangBin Cho
8f19f1eafb
[Core] Actor handle refactoring ( #8895 )
...
* Marking needed changes.
* Resolve basic dependencies.
* In progress.
* linting.
* In progress 2.
* Linting.
* Refactor done. Cleanup needed.
* Linting.
* Recover kill actor in core worker because it is used inside raylet
* Cleanup.
* Use unique pointer instead. Unit tests are broken now.
* Fix the upstream change.
* Addressed code review 1.
* Lint.
* Addressed code review 2.
* Fix weird github history.
* Lint.
* Linting using clang 7.0.
* Use a better check message.
* Revert cpp stuff.
* Fix weird linting errors.
* Manuall fix all lint issues.
* Update a newline.
* Refactor some interface.
* Addressed all code review.
* Addressed code review
2020-07-07 11:11:41 -07:00
Ian Rodney
f69d8b951e
[core] Disable flaky cancellation test ( #9325 )
2020-07-07 09:52:35 -07:00
Ian Rodney
a1e14380ce
[core] Switch Async Callback to C++ [WIP] ( #9228 )
...
Co-authored-by: simon-mo <simon.mo@hey.com>
2020-07-07 09:47:25 -07:00
Ian Rodney
079c1eaa5c
[autoscaler] Require keyword arguments ( #9256 )
2020-07-06 22:12:59 -07:00
Stephanie Wang
b42d6a1ddc
[core] Refactor task arguments and attach owner address ( #9152 )
...
* Add intended worker ID to GetObjectStatus, tests
* Remove TaskID owner_id
* lint
* Add owner address to task args
* Make TaskArg a virtual class, remove multi args
* Set owner address for task args
* merge
* Fix tests
* Fix
* build
* update
* build
* java
* Move code
* build
* Revert "Fix Google log directory again (#9063 )"
This reverts commit 275da2e400#9063 )""
This reverts commit 4a326fcb148ca09a35bc7de11d89df10edbb56e7.
* lint
2020-07-06 21:25:14 -07:00
Ian Rodney
6fecd3cfce
[autoscaler] Run initialization_commands without a persistent connection ( #9020 )
...
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
2020-07-06 16:34:59 -07:00
