hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-10 05:16:49 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
5519 commits 227 branches 67 tags 234 MiB
Commit graph

1419 commits

Author SHA1 Message Date
Kai Yang
c3ef8581d2 [Java] fix UT segmentation fault on exit (#6455) 
* fix segmentation fault in Java test

* update comments

* address comments
 2019-12-15 17:52:34 +08:00
Kai Yang
9cc0ecc6ff Fix duplicated logging if log dir is not set (#6342) 2019-12-15 13:29:36 +08:00
ZhuSenlin
6c0531683f Add gcs server as well as the unit test (#6401) 2019-12-15 13:23:42 +08:00
Edward Oakes
e2b7459bfc
Fix worker exit cleanup (#6450) 
* working but ugly

* comments

* proper but hanging in grpc server destructor

* grpc server shutdown deadline

* fix disconnect

* lint

* shutdown_only in test

* replace shutdown
 2019-12-13 16:52:50 -08:00
Eric Liang
335dade1e6
Check worker id for all core worker RPCs (#6472) 
* check worker id

* fix test

* owner

* fix tests

* comments
 2019-12-13 10:15:56 -08:00
Eric Liang
eb6f3f86e5
Seed using multiple samples (#6471) 2019-12-12 21:41:19 -08:00
Stephanie Wang
c57dcc82d1 Port actor creation to use direct calls (#6375) 2019-12-12 19:50:51 -08:00
Eric Liang
5a5c94939f
[direct call] Retry failed tasks with delay (#6453) 
* retry failed tasks with delay

* set to 0 for direct tests
 2019-12-12 17:12:38 -08:00
Kai Yang
3adbe29450 fix core worker test hanging due to heartbeat is not working (#6416) 2019-12-12 18:16:28 +08:00
micafan
8c1520d18e [GCS] refactor the GCS Client Job Interface (#5503) 2019-12-12 16:57:32 +08:00
wanxing
64d8626d6d Optimize ray::LocalMemoryBuffer performance (#6384) 2019-12-11 21:49:52 -08:00
Eric Liang
58ac8639b9
Fix bad checks and race condition from actor_deaths and node_failures tests (#6411) 2019-12-11 14:47:24 -08:00
Edward Oakes
82f7dbc7a7
Increase TaskID size by 2 bytes, taken from JobID (#6425) 
* Increase TaskID size by 2 bytes, taken from JobID

* comments

* check max job id

* fix doc

* fix local mode
 2019-12-11 10:45:14 -08:00
Kai Yang
a131082767 fix startup worker process count for multi-threading (#6382) 2019-12-11 20:19:49 +08:00
Edward Oakes
044527adb8
Remove ref counting dependencies on ray.get() (#6412) 
* Remove ref counting dependencies on Get()

* comment

* don't send IDs when disabled

* pass through internal config

* fix

* allow reinit

* remove flag
 2019-12-10 18:11:34 -08:00
mehrdadn
2f19b01898 Factor out stream protocols for Windows (#6408) 2019-12-10 15:23:30 -08:00
Chaokun Yang
6272907a57 [Streaming] Streaming data transfer and python integration (#6185) 2019-12-10 20:33:24 +08:00
Rong Rong
c1d4ab8bb4 Move top level RayletClient to ray::raylet::RayletClient (#6404) 2019-12-09 21:08:59 -08:00
Eric Liang
a6bc2b1842
Misc direct call fixes from unit tests (#6394) 2019-12-08 19:34:02 -08:00
Kai Yang
eb912b68b1 [Java] Fix instanceof RayPyActor (#6377) 2019-12-07 16:28:29 +08:00
Kai Yang
7e9fddf3ed [Java] Add java exception check in JNI (#6378) 2019-12-07 16:25:17 +08:00
Kai Yang
2003d2d952 explicit delete local reference in task_execution_callback for garbage collection (#6379) 2019-12-06 18:53:24 +08:00
Zhijun Fu
b88b8202cc fix java build failure (#6062) 2019-12-06 14:38:43 +08:00
Ion
1c638a11a7 Refactor helper methods for new scheduler integration (#6354) 2019-12-05 18:49:25 -08:00
Edward Oakes
f63b64310a
Bump version to 0.8.0.dev7 (#6303) 2019-12-05 18:33:54 -08:00
Eric Liang
6223d2ed0b
[direct call] Assign resource ids for direct call tasks (#6364) 2019-12-05 10:16:04 -08:00
micafan
668ce47360 [GCS]Add abstract interface of actor to GCS Client (#6269) 2019-12-05 13:38:29 +08:00
Zhijun Fu
7611e484ec properly handle a forwarded task that gets forwarded back (#6271) 2019-12-05 13:37:52 +08:00
Zhijun Fu
fa98694dd0 Fix raylet crash during cluster shutdown (#6272) 2019-12-05 11:08:58 +08:00
Edward Oakes
f65d65f5de
Add WorkerID check to AssignTask (#6355) 2019-12-04 12:38:29 -08:00
Eric Liang
1a3b83abf8
[direct call] Fix hang when caller id changes for actor task submission (#6338) 2019-12-04 12:01:35 -08:00
Stephanie Wang
a82fb5585d
[direct task] Remove timeout for resolving futures that were deserialized (#6337) 
* Reply GetObjectStatus once the task completes

* Remove timeout-based future resolution

* fix

* Update core_worker.h
 2019-12-03 12:04:59 -08:00
Stephanie Wang
d5720779b3 Set the actor ID as the assigned task ID for direct actor workers (#6335) 
* Fix

* rename
 2019-12-03 10:54:26 -08:00
Edward Oakes
d2c66ba795
Don't add assigned tasks to SWAP queue (#6325) 2019-12-02 16:39:02 -08:00
Ion
2a3adf2d70 New scheduler integration (#6321) 2019-12-02 14:42:16 -08:00
Stephanie Wang
69dd5c9319
[direct task] Fix bug that starts duplicate connections from the worker to the local raylet (#6307) 
* Fix bug and add unit test

* rename
 2019-12-02 10:25:05 -08:00
Stephanie Wang
da41180dc0
[direct task] Retry tasks on failure and turn on RAY_FORCE_DIRECT for test_multinode_failures.py (#6306) 
* multinode failures direct

* Add number of retries allowed for tasks

* Retry tasks

* Add failing test for object reconstruction

* Handle return status and debug

* update

* Retry task unit test

* update

* update

* todo

* Fix max_retries decorator, fix test

* Fix test that flaked

* lint

* comments
 2019-12-02 10:20:57 -08:00
mehrdadn
75cc994e0a Update various build options relating to Windows (#6315) 
* Update .bazelrc for Windows compatibility

* Block inclusion of (legacy) WinSock.h to avoid errors

* Suppress warnings for Windows code

* Include boost::asio in includes so that it is passed as -isystem to avoid warnings

* Link with -lpthread only on non-Windows

* Undefine BOOST_FALLTHROUGH, which is unnecessary and causes macro redefinition warnings

* Define RAY_STATIC and ARROW_STATIC to compile for Windows

* Add WinSock import library for Arrow
 2019-12-01 15:05:50 -08:00
mehrdadn
10d49a3f6f Use Boost's socket_holder instead of manually managing the socket (#6314) 
* Use Boost's socket_holder instead of manually managing sockets.

Socket types are not ints on Windows, and we need to use wrapper for proper lifetime management regardless.
 2019-12-01 13:27:52 -08:00
fangfengbin
7275556365 Reconstruct local dead actors immediately instead of waiting for initial_reconstruction_timeout_ms (#6243) 2019-11-30 18:03:48 +08:00
mehrdadn
e28e464158 Convert io_service_ from reference to smart pointer (#6285) 2019-11-29 16:09:46 -08:00
mehrdadn
b8cfdba752 Bazelify hiredis (#6203) 2019-11-29 15:32:45 -08:00
Eric Liang
b7b655c851
Also use NotifyDirectCallTaskBlock/Unblocked for plasma store accesses (#6249) 
* wip

* fix it

* lint

* wip

* fix

* unblock

* flaky

* use fetch only flag

* Revert "use fetch only flag"

This reverts commit 56e938a0ee2024f5c99c9ab2d55fd35558fb15e1.

* restore error resolution

* use worker task id

* proto comments

* fix if
 2019-11-27 22:46:15 -08:00
Stephanie Wang
31a0b11e16 Revert SubmitTask over grpc, use RayletConnection instead (#6305) 
* Revert SubmitTask over grpc

* comment
 2019-11-27 19:28:12 -06:00
Stephanie Wang
2797c11b69
[direct task] For serialized object IDs, check with owner before declaring object unreconstructable (#6286) 
* Track borrowed vs owned objects

* Serialize owner address with object ID

* serialize owner task id

* Deserialize object IDs

* Pass direct task ID instead of plasma ID

* it works

* Fix ref count test

* Add unit test

* update warning

* we own ray.put objects

* missing file

* doc

* Fix unit test

* comments

* Fix py2

* lint

* update
 2019-11-27 15:31:44 -08:00
Edward Oakes
8622559e0c
Use one queue per resource shape in direct task transport (#6277) 2019-11-26 20:56:05 -06:00
Eric Liang
30b2fc1d81
Fix actor creation hang due to race in SWAP queue (#6280) 2019-11-26 15:21:03 -08:00
Stephanie Wang
f6a0408173
Track pending tasks with TaskManager (#6259) 
* TaskStateManager to track and complete pending tasks

* Convert actor transport to use task state manager

* Refactor direct actor transport to use TaskStateManager

* rename

* Unit test

* doc

* IsTaskPending

* Fix?

* Shared ptr

* HUH?

* Update src/ray/core_worker/task_manager.cc

Co-Authored-By: Zhijun Fu <37800433+zhijunfu@users.noreply.github.com>

* Revert "HUH?"

This reverts commit f80f0ba204ff4da5e0b03191fa0d5a4d9f552434.

* Fix memory issue

* oops
 2019-11-25 16:37:26 -08:00
mehrdadn
ed5154d7fe Modify RayLogLevel to avoid conflicts with DEBUG macro and ERROR macros that are defined externally (#6204) 
* Prevent name collision of ERROR macro from Windows with RayLogLevel::ERROR
 2019-11-25 17:02:26 -07:00
Eric Liang
64a3a7239e
Set RAY_FORCE_DIRECT=1 for run_rllib_tests, test_basic (#6171) 2019-11-25 14:12:11 -08:00