Commit graph

1382 commits

Author SHA1 Message Date
kisuke95
ddc1e483fb
Fix actor table Delete bug (#9499) 2020-08-05 18:05:51 +08:00
kisuke95
80d2544f6b
Fix vector<bool> for loop (#9907) 2020-08-05 17:49:37 +08:00
fangfengbin
193d11ab8b
Optimize placement group log (#9891) 2020-08-05 14:41:32 +08:00
chaokunyang
3323ad9d59
[HOTFIX] Fix master build with missing placement group argument (#9868)
* fix common task submit default placement group

* fix java_function
2020-08-04 11:19:15 -07:00
Barak Michener
c16e1b9524
src/ray/protobuf: Break proto rules into a proper BUILD file (#9792) 2020-08-04 11:12:45 -07:00
Kai Yang
27cd323ce1
[Core] Multi-tenancy: Job isolation & implement per job config (except for env variables) (#9500) 2020-08-04 15:51:29 +08:00
kisuke95
28b1f7710c
[Core] Error info pubsub (Remove ray.errors API) (#9665) 2020-08-04 14:04:29 +08:00
fangfengbin
8c3fc1db76
Optimize actor creation log (#9781) 2020-08-04 10:29:30 +08:00
Zhijun Fu
4f2e4f31dd
async grpc calls should always return void (#9533) 2020-08-03 12:44:02 -07:00
Stephanie Wang
37a9c5783c
[core] Report resource load by shape (#9806)
* Report and aggregate resource load by shape

* python test

* python test

* x

* update
2020-07-31 16:57:30 -07:00
Eric Liang
b73080c85f
Allow tasks to be used with placement groups (#9738) 2020-07-31 10:51:37 -07:00
fangfengbin
3900643948
Add actor states definitions & transition diagram doc (#9754) 2020-07-31 15:35:25 +08:00
Kai Yang
02fd950252
[Java] Local and distributed ref counting in Java (#9371) 2020-07-31 11:49:31 +08:00
Eric Liang
73df3f7bd2
Clean up formatting of placement group resources (#9740) 2020-07-30 15:52:32 -07:00
SangBin Cho
e6d1e3afe2
Use pass by reference for const auto in for loop. (#9811) 2020-07-30 12:34:24 -05:00
Kai Yang
9be5a2f0fc
Fix GCS related tests (#9783) 2020-07-30 11:46:36 +08:00
SangBin Cho
826f14c824
[Stats] Fix harvestor threads + Fix flaky stats shutdown. (#9745) 2020-07-29 18:57:59 -05:00
mehrdadn
07022f3f11
Fix src/ray/core_worker/common.h deleted constructor (#9785)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-29 15:49:02 -07:00
Alex Wu
72297dc46f
[Core] Socket creation race condition bug fixes (#9764)
* fix issues

* hot fixes

* test

* test

* Always info log
2020-07-29 13:17:46 -07:00
SangBin Cho
d1b37ca7e4
[GCS Actor Management] Fix flaky test_dead_actors. (#9715)
* Fix.

* Add logs.

* Add an unit test.
2020-07-29 10:54:18 -07:00
Tao Wang
2babad9906
[GCS]Use a separate thread in node failure detector to handle heartbeat (#9416)
* use a sole thread to handle heartbeat

* separate signal thread

* use work to avoid exiting when task is underway

* protect shared data structure to avoid deadlock

* add comments

* decrease io service num

* minor changes

* fix test

* per stephanie's comments

* use single io service instead of 1-size io service pool

* typo
2020-07-29 09:58:58 -07:00
Lingxuan Zuo
156067b423
[Stats] enable core worker stats (#9355) 2020-07-29 17:28:33 +08:00
fangfengbin
a484947742
Fix leased worker leak bug if lease worker requests that are still waiting to be scheduled when GCS restarts (#9719) 2020-07-29 14:16:03 +08:00
Alex Wu
21af0ceb0c
Register function race (#9346) 2020-07-28 13:51:34 -07:00
SangBin Cho
7e3ba289dc
[Stats] Basic Metrics Infrastructure (Metrics Agent + Prometheus Exporter) (#9607) 2020-07-28 10:28:01 -07:00
Alex Wu
feb3751824
[New scheduler] First unit test for task manager (#9696)
* .

* .

* refactor WorkerInterface

* .

* Basic unit test structure complete?

* .

* bad git >:-(

* small clean up

* CR

* .

* .

* One more fixture

* One more fixture

* .

* .

* bazel-format

* .
2020-07-28 09:44:58 -07:00
fangfengbin
bd18e975c0
fix windows compile bug (#9741)
Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-07-28 21:30:31 +08:00
Lingxuan Zuo
1049c9e53b
[Stats] fix stats shutdown crash if opencensus exporter not initialized (#9727) 2020-07-28 11:21:10 +08:00
Alisa
51e12ee97c
Python api of placement group (#9243) 2020-07-27 14:57:05 -07:00
Ian Rodney
d35605079e
[core] Removes Error when Internal Config is not set (#9700) 2020-07-27 11:47:54 -07:00
fangfengbin
2790818c53
[GCS]GCS client support multi-thread subscribe&resubscribe&unsubscribe (#9718) 2020-07-27 13:58:39 +08:00
ZhuSenlin
a269ae9bc4
[GCS] Fix actor task hang when its owner exits before local dependencies resolved (#8045) 2020-07-27 10:56:52 +08:00
Tao Wang
c6a7b3ac68
[Tests]lock vector to avoid potential flaky test (#9656) 2020-07-26 11:57:28 +08:00
Siyuan (Ryans) Zhuang
54a0d8b69e
[Core] Try remove all windows compat shims (#9671)
* try remove compat for arrow

* remove unistd.h

* remove socket compat

* delete arrow windows patch
2020-07-25 12:00:36 -07:00
SangBin Cho
d49b19c24c
[Stats] Improve Stats::Init & Add it to GCS server (#9563) 2020-07-25 10:42:08 -07:00
fangfengbin
28d5f9696d
Add placement group java api (#9611)
* add part code

* add part code

* add part code

* fix code style

* fix review comment

* fix review comment

* add part code

* add part code

* add part code

* add part code

* fix review comment

* fix review comment

* fix code style

* fix review comment

* fix lint error

* fix lint error

Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-07-25 15:39:05 +08:00
mehrdadn
6beed5a2df
Fix ERROR logging not being printed to standard error (#9633)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-24 11:15:00 -07:00
fangfengbin
d4324a4a8f
fix java coreworker crash (#9674) 2020-07-24 15:36:49 +08:00
Stephanie Wang
f2705e2c73
[core] Enable object reconstruction for retryable actor tasks (#9557)
* Test actor plasma reconstruction

* Allow resubmission of actor tasks

* doc

* Test for actor constructor

* Kill PID before removing node

* Kill pid before node
2020-07-23 21:15:12 -07:00
Alex Wu
239196fffc
[Core] WorkerInterface refactor (#9655)
* .

* .

* refactor WorkerInterface

* .

* Basic unit test structure complete?

* .

* .

* .

* .

* Fixed tests

* Fixed tests

* .
2020-07-23 21:13:29 -07:00
Siyuan (Ryans) Zhuang
e4beffa49d
Remove unused windows shims (#9583) 2020-07-23 17:18:39 -07:00
Siyuan (Ryans) Zhuang
4594f7db2e
[Core] Remove unnecessary windows syscall in plasma store (#9602) 2020-07-23 17:14:25 -07:00
fangfengbin
5ab17e0dd8
Fix TestObjectTableResubscribe testcase bug (#9650) 2020-07-23 20:28:31 +08:00
Lingxuan Zuo
64b0f173c8
fix window jni unhappy compiler (#9635) 2020-07-23 19:43:53 +08:00
Siyuan (Ryans) Zhuang
993ff5fd81
[Core] Fix concurrency issues in plasma store runner (#9642) 2020-07-23 00:12:10 -07:00
SangBin Cho
ca391ed052
[Core] GCS Actor management on by default. (#8845)
* GCS Actor management on by default.

* Fix travis config.

* Change condition.

* Remove unnecessary CI.
2020-07-22 14:31:41 -07:00
Kai Yang
bfa0605282
[Java] Avoid data copy from C++ to Java for ByteBuffer type (#9033) 2020-07-22 16:25:32 +08:00
mehrdadn
b14728d999
Shellcheck quoting (#9596)
* Fix SC2006: Use $(...) notation instead of legacy backticked `...`.

* Fix SC2016: Expressions don't expand in single quotes, use double quotes for that.

* Fix SC2046: Quote this to prevent word splitting.

* Fix SC2053: Quote the right-hand side of == in [[ ]] to prevent glob matching.

* Fix SC2068: Double quote array expansions to avoid re-splitting elements.

* Fix SC2086: Double quote to prevent globbing and word splitting.

* Fix SC2102: Ranges can only match single chars (mentioned due to duplicates).

* Fix SC2140: Word is of the form "A"B"C" (B indicated). Did you mean "ABC" or "A\"B\"C"?

* Fix SC2145: Argument mixes string and array. Use * or separate argument.

* Fix SC2209: warning: Use var=$(command) to assign output (or quote to assign string).

Co-authored-by: Mehrdad <noreply@github.com>
2020-07-21 21:56:41 -05:00
ZhuSenlin
382b314241
[GCS] fix the fault tolerance about gcs node manager (#9380) 2020-07-22 10:55:51 +08:00
Lingxuan Zuo
cd42450fc1
[Metrics] Java metric API (#9377) 2020-07-22 10:35:08 +08:00