Barak Michener
8e76796fd0
ci: Redo format.sh --all script & backfill lint fixes ( #9956 )
2020-08-07 16:49:49 -07:00
Barak Michener
1d01c668f0
rpc: Core Worker client pool ( #9934 )
2020-08-07 16:34:29 -07:00
Tao Wang
8bea875673
[TEST]Check if port is free before start up redis ( #9974 )
...
* [TEST]Check if port is free before start up redis
* per comment
2020-08-07 10:15:12 -07:00
SangBin Cho
44826878ff
[Core] Remove Legacy Raylet Code ( #9936 )
...
* Remove a flag and some methods in node manager including HandleDisconnectedActor, ResubmitTask, and HandleTaskReconstruction
* Make actor creator always required + remove raylet transport
* Remove actor reporter + remove FinishAssignedActorCreationTask
* Remove actor tasks.
* Remove finishactortask and switched it to finishactorcreation task
* Remove reconstruction policy.
* Remove lineage cache.
* Formatting.
* Remove actor frontier code.
* Removed build error.
* Revert "Remove reconstruction policy."
This reverts commit 9d25c9bced4da5fbcac5d484d51013345f16513b.
* Recover HandleReconstruction to mark expired objects as failed.
2020-08-06 16:37:50 -07:00
SangBin Cho
ec2f1a225e
[Stats] Metrics Export User Interface Part 1 ( #9913 )
...
* Metrics export port expose done.
* Support exposing metrics port + metrics agent service discovery through ray.nodes()
* Formatting.
* Added a doc.
* Linting.
* Change the location of metrics agent port.
* Addressed code review.
* Addressed code review.
2020-08-06 16:16:29 -07:00
Eric Liang
7d4f204aa8
[Placement Group] Allow scheduling a task on any bundle (-1, default) ( #9885 )
...
* wip
* wip
* fix tests
* wip
* wip
* wip
* wip
* wip
* add test
* update
* update
* remov debug
* comments
2020-08-06 00:05:21 -07:00
Tao Wang
1760586628
[GCS]Use an asynchronous PING to avoid blocking other operations ( #9871 )
...
* Use separate redis client to avoid its sync command blocking other operations
* use redis_failure_detector_client_
* use async command to ping redis
* format log
2020-08-05 19:10:53 -07:00
SangBin Cho
68899e2f8e
[GCS Actor Management] Fix race condition for DEPENDENCIES_UNREADY states. ( #9883 )
...
* Fix issues.
* Address code review.
* Addressed code review 2.
* Fix formatting.
* Addressed code review 3/
* Addressed code review.
2020-08-05 12:22:12 -07:00
SangBin Cho
685182923c
[Core] Fix detached actor local mode when gcs actor management is on. ( #9839 )
...
* Fix local mode detached actor.
* Revert changes.
2020-08-05 09:04:24 -07:00
kisuke95
ddc1e483fb
Fix actor table Delete bug ( #9499 )
2020-08-05 18:05:51 +08:00
kisuke95
80d2544f6b
Fix vector<bool> for loop ( #9907 )
2020-08-05 17:49:37 +08:00
fangfengbin
193d11ab8b
Optimize placement group log ( #9891 )
2020-08-05 14:41:32 +08:00
chaokunyang
3323ad9d59
[HOTFIX] Fix master build with missing placement group argument ( #9868 )
...
* fix common task submit default placement group
* fix java_function
2020-08-04 11:19:15 -07:00
Barak Michener
c16e1b9524
src/ray/protobuf: Break proto rules into a proper BUILD file ( #9792 )
2020-08-04 11:12:45 -07:00
Kai Yang
27cd323ce1
[Core] Multi-tenancy: Job isolation & implement per job config (except for env variables) ( #9500 )
2020-08-04 15:51:29 +08:00
kisuke95
28b1f7710c
[Core] Error info pubsub (Remove ray.errors API) ( #9665 )
2020-08-04 14:04:29 +08:00
fangfengbin
8c3fc1db76
Optimize actor creation log ( #9781 )
2020-08-04 10:29:30 +08:00
Zhijun Fu
4f2e4f31dd
async grpc calls should always return void ( #9533 )
2020-08-03 12:44:02 -07:00
Stephanie Wang
37a9c5783c
[core] Report resource load by shape ( #9806 )
...
* Report and aggregate resource load by shape
* python test
* python test
* x
* update
2020-07-31 16:57:30 -07:00
Eric Liang
b73080c85f
Allow tasks to be used with placement groups ( #9738 )
2020-07-31 10:51:37 -07:00
fangfengbin
3900643948
Add actor states definitions & transition diagram doc ( #9754 )
2020-07-31 15:35:25 +08:00
Kai Yang
02fd950252
[Java] Local and distributed ref counting in Java ( #9371 )
2020-07-31 11:49:31 +08:00
Eric Liang
73df3f7bd2
Clean up formatting of placement group resources ( #9740 )
2020-07-30 15:52:32 -07:00
SangBin Cho
e6d1e3afe2
Use pass by reference for const auto in for loop. ( #9811 )
2020-07-30 12:34:24 -05:00
Kai Yang
9be5a2f0fc
Fix GCS related tests ( #9783 )
2020-07-30 11:46:36 +08:00
SangBin Cho
826f14c824
[Stats] Fix harvestor threads + Fix flaky stats shutdown. ( #9745 )
2020-07-29 18:57:59 -05:00
mehrdadn
07022f3f11
Fix src/ray/core_worker/common.h deleted constructor ( #9785 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-29 15:49:02 -07:00
Alex Wu
72297dc46f
[Core] Socket creation race condition bug fixes ( #9764 )
...
* fix issues
* hot fixes
* test
* test
* Always info log
2020-07-29 13:17:46 -07:00
SangBin Cho
d1b37ca7e4
[GCS Actor Management] Fix flaky test_dead_actors. ( #9715 )
...
* Fix.
* Add logs.
* Add an unit test.
2020-07-29 10:54:18 -07:00
Tao Wang
2babad9906
[GCS]Use a separate thread in node failure detector to handle heartbeat ( #9416 )
...
* use a sole thread to handle heartbeat
* separate signal thread
* use work to avoid exiting when task is underway
* protect shared data structure to avoid deadlock
* add comments
* decrease io service num
* minor changes
* fix test
* per stephanie's comments
* use single io service instead of 1-size io service pool
* typo
2020-07-29 09:58:58 -07:00
Lingxuan Zuo
156067b423
[Stats] enable core worker stats ( #9355 )
2020-07-29 17:28:33 +08:00
fangfengbin
a484947742
Fix leased worker leak bug if lease worker requests that are still waiting to be scheduled when GCS restarts ( #9719 )
2020-07-29 14:16:03 +08:00
Alex Wu
21af0ceb0c
Register function race ( #9346 )
2020-07-28 13:51:34 -07:00
SangBin Cho
7e3ba289dc
[Stats] Basic Metrics Infrastructure (Metrics Agent + Prometheus Exporter) ( #9607 )
2020-07-28 10:28:01 -07:00
Alex Wu
feb3751824
[New scheduler] First unit test for task manager ( #9696 )
...
* .
* .
* refactor WorkerInterface
* .
* Basic unit test structure complete?
* .
* bad git >:-(
* small clean up
* CR
* .
* .
* One more fixture
* One more fixture
* .
* .
* bazel-format
* .
2020-07-28 09:44:58 -07:00
fangfengbin
bd18e975c0
fix windows compile bug ( #9741 )
...
Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-07-28 21:30:31 +08:00
Lingxuan Zuo
1049c9e53b
[Stats] fix stats shutdown crash if opencensus exporter not initialized ( #9727 )
2020-07-28 11:21:10 +08:00
Alisa
51e12ee97c
Python api of placement group ( #9243 )
2020-07-27 14:57:05 -07:00
Ian Rodney
d35605079e
[core] Removes Error when Internal Config is not set ( #9700 )
2020-07-27 11:47:54 -07:00
fangfengbin
2790818c53
[GCS]GCS client support multi-thread subscribe&resubscribe&unsubscribe ( #9718 )
2020-07-27 13:58:39 +08:00
ZhuSenlin
a269ae9bc4
[GCS] Fix actor task hang when its owner exits before local dependencies resolved ( #8045 )
2020-07-27 10:56:52 +08:00
Tao Wang
c6a7b3ac68
[Tests]lock vector to avoid potential flaky test ( #9656 )
2020-07-26 11:57:28 +08:00
Siyuan (Ryans) Zhuang
54a0d8b69e
[Core] Try remove all windows compat shims ( #9671 )
...
* try remove compat for arrow
* remove unistd.h
* remove socket compat
* delete arrow windows patch
2020-07-25 12:00:36 -07:00
SangBin Cho
d49b19c24c
[Stats] Improve Stats::Init & Add it to GCS server ( #9563 )
2020-07-25 10:42:08 -07:00
fangfengbin
28d5f9696d
Add placement group java api ( #9611 )
...
* add part code
* add part code
* add part code
* fix code style
* fix review comment
* fix review comment
* add part code
* add part code
* add part code
* add part code
* fix review comment
* fix review comment
* fix code style
* fix review comment
* fix lint error
* fix lint error
Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-07-25 15:39:05 +08:00
mehrdadn
6beed5a2df
Fix ERROR logging not being printed to standard error ( #9633 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-24 11:15:00 -07:00
fangfengbin
d4324a4a8f
fix java coreworker crash ( #9674 )
2020-07-24 15:36:49 +08:00
Stephanie Wang
f2705e2c73
[core] Enable object reconstruction for retryable actor tasks ( #9557 )
...
* Test actor plasma reconstruction
* Allow resubmission of actor tasks
* doc
* Test for actor constructor
* Kill PID before removing node
* Kill pid before node
2020-07-23 21:15:12 -07:00
Alex Wu
239196fffc
[Core] WorkerInterface refactor ( #9655 )
...
* .
* .
* refactor WorkerInterface
* .
* Basic unit test structure complete?
* .
* .
* .
* .
* Fixed tests
* Fixed tests
* .
2020-07-23 21:13:29 -07:00
Siyuan (Ryans) Zhuang
e4beffa49d
Remove unused windows shims ( #9583 )
2020-07-23 17:18:39 -07:00
