hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-06 10:31:39 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
4038 commits 227 branches 67 tags 234 MiB
Commit graph

1028 commits

Author SHA1 Message Date
Philipp Moritz
1c5446851a
Use Plasma with LRU refreshing integrated (#6050) 2019-11-03 16:19:05 -08:00
Eric Liang
fb34928a2a
[minor] Perf optimizations for direct actor task submission (#6044) 
* merge optimizations

* fix

* fix memory err

* optimize

* fix tests

* fix serialization of method handles

* document weakref

* fix check

* bazel format

* disable on 2
 2019-11-01 14:41:14 -07:00
Eric Liang
eef4ad3bba
Report census view data as part of raylet node stats (#6060) 2019-11-01 14:26:09 -07:00
Simon Mo
7f5b3502da
Implement Detached Actor (#6036) 
* Arg propagation works

* Implement persistent actor

* Add doc

* Initialize is_persistent_

* Rename persistent->detached

* Address comment

* Make test passes

* Address comment

* Python2 compatiblity

* Fix naming, py2

* Lint
 2019-11-01 10:28:23 -07:00
Eric Liang
c86f945520
Support pass by ref args in for direct actor calls (#6040) 2019-10-31 16:55:10 -07:00
Edward Oakes
16e9dfd2e1
Exit workers when raylet dies unexpectedly (#6014) 2019-10-30 20:29:25 -07:00
Eric Liang
8ebba202df
[minor] Reduce perf overhead of object ref tracking (#6041) 2019-10-29 18:14:51 -07:00
Eric Liang
b89cac976a
Basic direct actor call support in Python (#5991) 2019-10-28 22:09:04 -07:00
Edward Oakes
c1418b04df Remove CoreWorkerObjectInterface (#6023) 2019-10-28 10:48:41 -07:00
Philipp Moritz
80c01617a3
Optimize python task execution (#6024) 2019-10-27 00:43:34 -07:00
Stephanie Wang
eb41c945a1 Add gRPC endpoint to raylet to expose metrics (#6005) 2019-10-26 16:37:39 -07:00
Eric Liang
a5523466a2
Enable memstore by default (#6003) 2019-10-25 21:59:12 -07:00
Edward Oakes
d4055d70e3
Remove CoreWorkerTaskExecutionInterface (#6009) 2019-10-25 16:33:44 -07:00
Edward Oakes
e6141a0b8b
Remove UsePush logic from raylet (#6015) 2019-10-25 14:52:19 -07:00
Edward Oakes
1ce521a7f3
Remove task context from python worker (#5987) 
Removes duplicated state between the python and C++ workers. Also cleans up the serialization codepaths a bit.
 2019-10-25 07:38:33 -07:00
Philipp Moritz
09d05bb3fa
Reduce actor submission python overhead (#5949) 2019-10-23 00:11:32 -07:00
Edward Oakes
02931e08f3
[core worker] Python core worker task execution (#5783) 
Executes tasks via the event loop in the C++ core worker. Also properly handles signals (including KeyboardInterrupt), so ctrl-C in a python interactive shell works now (if connecting to an existing cluster).
 2019-10-22 20:15:59 -07:00
Philipp Moritz
b6e7ed20ce
Fix random numbers on linux wheel build (#5975) 2019-10-22 17:52:12 -07:00
Edward Oakes
fc56872012
Send active object IDs to the raylet (#5803) 
* Send active object IDs to the raylet

* comment

* comments

* dedup

* signed int in config

* comments

* Remove object ID from monitor

* Fix test

* re-add check

* fix cast

* check if core worker

* Add comment

* Reservoir sampling

* Fix lint

* Pointer return

* tmp

* Fix merge

* Initialize object ids properly

* Fix lint
 2019-10-20 22:05:28 -07:00
Stephanie Wang
bc4a0de4da
Fix multiple drivers for named actors and add test (#5956) 2019-10-20 16:04:21 -07:00
Stephanie Wang
697f765efc
Refactor CoreWorker to remove TaskInterface (#5924) 
* Remove TaskInterface

* Remove Status return value

* Remove CActorHandle, some return values, TaskSubmitter

* lint

* doc

* doc

* fix build

* lint

* Return Status, guarded by annotation, fail tasks for RECONSTRUCTING actors

* fix

* move annotation

* revert

* Fix core worker test

* nits
 2019-10-18 00:03:57 -04:00
Stephanie Wang
3ac8592dcf
Remove actor handle IDs (#5889) 
* Remove actor handle ID from main ActorHandle constructor

* Set the actor caller ID when calling submit task instead of in the actor handle

* Remove ActorHandle::Fork, remove actor handle ID from protobuf

* Make inner actor handle const, remove new_actor_handles

* Move caller ID into the common task spec, start refactoring raylet

* Some fixes for forking actor handles

* Store ActorHandle state in CoreWorker, only expose actor ID to Python

* Remove some unused fields

* lint

* doc

* fix merge

* Remove ActorHandleID from python/cpp

* doc

* Fix core worker test

* Move actor table subscription to CoreWorker, reset actor handles on actor failure

* lint

* Remove GCS client from direct actor

* fix tests

* Fix

* Fix tests for raylet codepath

* Fix local mode

* Fix multithreaded test

* Fix AsyncSubscribe issue...

* doc

* fix serve

* Revert bazel
 2019-10-17 12:36:34 -04:00
Richard Liaw
20c0cdee4f
[autoscaler] Worker-Head termination + Better Scale-up message (#5909) 2019-10-14 10:37:50 -07:00
Edward Oakes
abbfe7392f
Bump dev version to 0.8.0.dev6 (#5906) 2019-10-14 11:36:13 +01:00
Philipp Moritz
1100556ba2
Fix linux wheel build (#5881) 2019-10-10 16:15:26 -07:00
Eric Liang
1a8ac3db46
Implement fair task queueing to prevent task starvation (#5851) 
* initial commit

* lint

* clarify

* add feature flag

* comment

* add timeout to test

* fix print

* comment

* use id for scheduling class

* lint

* dad warn

* flake
 2019-10-08 21:04:25 -07:00
Edward Oakes
08e4e3a153
[core worker] Submit Python actor tasks through core worker (#5750) 
* Submit actor tasks through core worker

* Fix java

* add comment

* Remove task builder

* Check negative

* Increase -> Increment

* pass by reference

* fix signal

* Clean up c++ actor handle

* more cleanup

* Clean up headers

* Fix unique_ptr construction

* Fix java

* Move profiling to c++

* dedup

* fix error

* comments

* fix java

* Fix tests

* wait for actor to exit

* Start after constructor

* ignore java build

* fix comment

* always init logging

* Fix logging

* fix logging issue

* shared_ptr for profiler

* DEBUG -> WARNING

* fix killed_ init

* Fix flaky checkpointing tests

* -v flag for tune tests

* Fix checkpoint test logic

* Fix exception matching

* timeout exception

* Fix test exception info

* Fix import

* fix build

* Fix test

* shared_ptr
 2019-10-07 15:42:19 -07:00
Edward Oakes
17c6835c3f
Just die on signal (#5842) 2019-10-03 18:21:21 -07:00
Si-Yuan
2fb7d7846f Initial implementation of Cython pickle5 support (#5725) 2019-10-03 09:20:26 -07:00
Edward Oakes
4e049232a8
shared_ptr (#5830) 2019-10-02 16:29:04 -07:00
Edward Oakes
963bbe8bbd
Move profiling to c++ (#5771) 
* Move profiling to c++

* comments

* Fix tests

* Start after constructor

* fix comment

* always init logging

* Fix logging

* fix logging issue

* shared_ptr for profiler

* DEBUG -> WARNING

* fix killed_ init

* Fix flaky checkpointing tests

* Fix checkpoint test logic

* Fix exception matching

* timeout exception

* Fix import

* fix build

* use boost::asio

* fix double const

* Properly reset async_wait

* remove SIGINT

* Change error message

* increase timeout

* small nits

* Don't trap on SIGINT

* -v for tune

* Fix test
 2019-10-01 10:06:25 -07:00
Edward Oakes
86610a30c9
[flaky test] Fix flaky checkpointing tests (#5791) 
* Fix flaky checkpointing tests

* Fix checkpoint test logic

* Fix exception matching

* timeout exception

* Fix import

* fix build
 2019-09-27 11:03:07 -07:00
Eric Liang
b5da32df78 Bump Ray version in documentation to dev5 (#5794) 2019-09-27 00:19:17 -07:00
Edward Oakes
8a33891a40
Include object size in full error (#5782) 2019-09-25 17:04:17 -07:00
Zhijun Fu
ea9376c9ce Fix flaky core worker tests because of race condition in gcs client subscription (#5735) 2019-09-24 22:47:38 +08:00
Edward Oakes
61e5d674be
Push driver task in core worker (#5752) 2019-09-23 10:53:55 -05:00
Philipp Moritz
f4deecb5ab Fix travis error in direct_actor_transport.cc (#5710) 2019-09-15 22:19:20 -07:00
Eric Liang
4bf7de084d Speed up TaskSpecification copy (#5709) 2019-09-15 19:57:34 -07:00
Eric Liang
4979b8c4d9
Ordered execution of tasks per actor handle (#5664) 2019-09-14 22:31:33 -07:00
Edward Oakes
a5d7de6aaf [core worker] Python core worker normal task submission (#5566) 2019-09-14 13:02:53 -07:00
Edward Oakes
07c4c6367a [core worker] Python core worker object interface (#5272) 2019-09-12 23:07:46 -07:00
Edward Oakes
0bf79cfbde Properly short circuit core worker Get() on exception (#5672) 2019-09-11 18:38:14 -07:00
Eric Liang
2fdefe19b7
Take into account queue length in autoscaling (#5684) 2019-09-11 11:31:35 -07:00
Kai Yang
ed761900f6 [Java] Support direct actor call in Java worker (#5504) 2019-09-09 14:29:20 +08:00
Kai Yang
d8f5804690 Support metadata for passing by value task arguments (#5527) 2019-09-08 11:07:48 +08:00
Kai Yang
732336fc4f [Java] Support multiple workers in Java worker process (#5505) 2019-09-07 22:52:05 +08:00
Edward Oakes
f38bb288e2 Clean up Wait() in the core worker (#5628) 2019-09-04 21:31:34 -07:00
Zhijun Fu
bb5609afb3 ignore object exists error for memory store provider (#5607) 2019-09-05 11:45:35 +08:00
micafan
8236936189 Fix code style in unit test of GCS. (#5634) 2019-09-04 19:36:44 +07:00
Edward Oakes
0c68b4cc30 Clean up Wait() and Get() in the core worker (#5556) 2019-09-03 14:45:15 -07:00