Stephanie Wang
0389735d7a
[core] Pass owner address from the workers to the raylet ( #9299 )
...
* Add intended worker ID to GetObjectStatus, tests
* Remove TaskID owner_id
* lint
* Add owner address to task args
* Make TaskArg a virtual class, remove multi args
* Set owner address for task args
* merge
* Fix tests
* Add ObjectRefs to task dependency manager, pass from task spec args
* tmp
* tmp
* Fix
* Add ownership info for task arguments
* Convert WaitForDirectActorCallArgs
* lint
* build
* update
* build
* java
* Move code
* build
* Revert "Fix Google log directory again (#9063 )"
This reverts commit 275da2e400
2020-07-09 14:35:54 -07:00
mehrdadn
4687b807c4
Combine different severities into the same log files ( #9230 )
...
* Combine different severities into the same log files
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-09 14:14:28 -07:00
Siyuan (Ryans) Zhuang
7e1326c0f6
[Core] Enhance common client connection ( #9367 )
...
* enhance client connection
* add write buffer async
* read message
* add test
2020-07-09 08:59:37 -07:00
fangfengbin
b97b474ae9
Fix GcsActorManagerTest multi-thread bug ( #9358 )
2020-07-09 20:03:46 +08:00
fangfengbin
62fe403a57
Fix the bug that PollOwnerForActorOutOfScope's reply may be missing ( #9330 )
2020-07-09 17:33:52 +08:00
Lingxuan Zuo
0562236533
[Stats] Make aggeration window and report interval configurable. ( #9235 )
2020-07-09 12:44:38 +08:00
SangBin Cho
9f8ff2e3b1
[Core] GCS Actor Management Race Condition ( #9215 )
...
* GCS Actor management on by default.
* Fix travis config.
* Change condition.
* Finish the initial race condition fix.
* Lint.
* Refine the codebase.
* Finish the initial version
* Improve logic.
* Remove unnecessary log messages.
* Address code review.
* Add tests
* Revert the second race condition that doesn't happen anymore, handle some edge cases. add tests.
* Address the second race condition found.
* Addressed code review.
* Addressed code review.
* Run a new unit test only when gcs actor management is on.
2020-07-08 10:56:52 -07:00
SangBin Cho
c5aa1eb042
[GCS Actor Management] Fix a dashboard doesn't show the correct actor state. ( #9275 )
...
* Fix a bug.
* Address code review.
* Remove an unnecessary comment.
2020-07-08 10:55:47 -07:00
Ian Rodney
9172f8c3a6
[core] Store Internal Config in GCS ( #8921 )
2020-07-08 11:22:08 -05:00
ZhuSenlin
42f8f16c04
[GCS] Fix detached actor with empty name ( #9283 )
2020-07-08 12:21:24 +08:00
yncxcw
4ba4110dec
[Core] Make worker_register_timeout_seconds configurable ( #9221 )
2020-07-07 18:27:15 -05:00
SangBin Cho
8f19f1eafb
[Core] Actor handle refactoring ( #8895 )
...
* Marking needed changes.
* Resolve basic dependencies.
* In progress.
* linting.
* In progress 2.
* Linting.
* Refactor done. Cleanup needed.
* Linting.
* Recover kill actor in core worker because it is used inside raylet
* Cleanup.
* Use unique pointer instead. Unit tests are broken now.
* Fix the upstream change.
* Addressed code review 1.
* Lint.
* Addressed code review 2.
* Fix weird github history.
* Lint.
* Linting using clang 7.0.
* Use a better check message.
* Revert cpp stuff.
* Fix weird linting errors.
* Manuall fix all lint issues.
* Update a newline.
* Refactor some interface.
* Addressed all code review.
* Addressed code review
2020-07-07 11:11:41 -07:00
Ian Rodney
a1e14380ce
[core] Switch Async Callback to C++ [WIP] ( #9228 )
...
Co-authored-by: simon-mo <simon.mo@hey.com>
2020-07-07 09:47:25 -07:00
fangfengbin
8391f66086
Fix gcs actor manager destroy actor crash bug ( #9329 )
2020-07-07 21:12:30 +08:00
Stephanie Wang
b42d6a1ddc
[core] Refactor task arguments and attach owner address ( #9152 )
...
* Add intended worker ID to GetObjectStatus, tests
* Remove TaskID owner_id
* lint
* Add owner address to task args
* Make TaskArg a virtual class, remove multi args
* Set owner address for task args
* merge
* Fix tests
* Fix
* build
* update
* build
* java
* Move code
* build
* Revert "Fix Google log directory again (#9063 )"
This reverts commit 275da2e400#9063 )""
This reverts commit 4a326fcb148ca09a35bc7de11d89df10edbb56e7.
* lint
2020-07-06 21:25:14 -07:00
ChenZhilei
6f3d993681
GCS server use worker table to handle RegisterWorker instead of redis accessor ( #9168 )
2020-07-06 10:37:25 +08:00
Tao Wang
f7ac495a68
[Core] Use map instead of list to represent resources in heartbeat message ( #9294 )
2020-07-05 10:59:25 +08:00
Lingxuan Zuo
7a2d7964d8
Bool config string stream must be in true or 1 ( #9292 )
2020-07-03 20:37:19 +08:00
ZhuSenlin
726c4cd6d4
fix actor accessor initialize inside redis gcs client ( #9284 )
...
Co-authored-by: senlin.zsl <senlin.zsl@antfin.com>
2020-07-03 15:48:32 +08:00
mehrdadn
29acf272b7
Build with Visual C++ ( #9190 )
...
Co-authored-by: Mehrdad <noreply@github.com>
Co-authored-by: Simon Mo <xmo@berkeley.edu>
2020-07-02 09:34:24 -07:00
Lingxuan Zuo
8a1cc7f8f9
Reset bool config by checking type first ( #9247 )
2020-07-02 18:32:41 +08:00
fangfengbin
8fcfcc4100
GCS server error handling for actor creation ( #8899 )
2020-07-02 16:27:32 +08:00
fangfengbin
a7a7bef622
Fix Java test crash due to incorrect destruction order ( #9236 )
2020-07-02 15:44:08 +08:00
Tao Wang
d35776e92d
[Core] Do not heartbeat resources if no update ( #9182 )
2020-07-02 13:56:49 +08:00
Lingxuan Zuo
1491508859
[Metrics] Stats supports metric exporters ( #8941 )
2020-07-01 10:54:55 -05:00
ChenZhilei
c11855728a
Remove raylet monitor after use GCS service ( #9179 )
2020-07-01 20:01:52 +08:00
chaokunyang
ab8df7afa3
Refine GetNamedActorHandle acync rpc wait ( #9078 )
2020-06-29 10:52:38 +08:00
Alex Wu
f8ecf6de71
Small fix ( #9149 )
...
Fixes some basic test cases
2020-06-28 13:43:03 -07:00
fangfengbin
1dc018d0bb
Add gcs service discovery to automatically detect GCS server restart ( #8735 )
2020-06-28 16:50:06 +08:00
Zhilei Chen
d78128a69c
Refactor GcsJobInfoHandle to GcsJobManager and filename ( #9112 )
2020-06-28 16:10:13 +08:00
mehrdadn
aac40af948
Fix some nonstandard code ( #9172 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-28 00:22:52 -07:00
Siyuan (Ryans) Zhuang
d7549d6184
Replace arrow::Status with ray::Status in Plasma ( #9154 )
...
* add object store status
* replace arrow status with ray status
* cleanup
* remove plasma common.cc
2020-06-26 14:06:34 -07:00
Stephanie Wang
490cddc250
[core] Refactor distributed ref counting to remove owner task ID ( #9049 )
...
* Add intended worker ID to GetObjectStatus, tests
* Remove TaskID owner_id
* lint
* Update message
* lint
* Fix build
2020-06-25 17:55:03 -07:00
mehrdadn
ab3413c124
Make process termination more robust ( #9081 )
2020-06-25 15:30:53 -05:00
Eric Liang
4522038259
[iter] Add .transform() function for arbitrary generator transforms ( #8978 )
2020-06-25 11:04:14 -07:00
Siyuan (Ryans) Zhuang
80bcbe20c7
[Core] Remove object notification IPC between Plasma and Raylet (initial step) ( #8939 )
...
* initial refactoring
redirect notifications to eventloop
implement direct notifications
* protect vector with mutex
2020-06-24 13:54:40 -07:00
Siyuan (Ryans) Zhuang
613abdf1b6
Remove arrow macros in plasma store ( #9115 )
2020-06-23 23:34:44 -07:00
Siyuan (Ryans) Zhuang
acb7270bd7
Adopt upstream plasma changes ( #9061 )
...
* adopt upstream plasma changes
2020-06-23 14:19:57 -07:00
Simon Mo
b6d425526d
Move actor task submission to io service ( #9093 )
2020-06-23 10:07:33 -07:00
Siyuan (Ryans) Zhuang
306ca75737
Fix ray arrow logs ( #9097 )
...
* convert arrow logs to ray logs
* remove extra plasma tests and modules
2020-06-23 10:02:30 -07:00
Zhilei Chen
8f2564f1a6
fix a bug that move a const variable ( #9080 )
2020-06-23 11:54:18 +08:00
Siyuan (Ryans) Zhuang
7a110b9401
[Core] Remove digests in plasma (4x performance improvement) ( #8980 )
...
* remove digest in plasma
* totally remove list
2020-06-22 14:24:32 -07:00
mehrdadn
275da2e400
Fix Google log directory again ( #9063 )
2020-06-22 14:56:28 -05:00
mehrdadn
1a40d24174
Handle loop_ NULL case ( #9067 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-22 11:05:29 -07:00
SangBin Cho
e254dd3115
Do not add reference count when it is local mode. ( #8979 )
2020-06-21 16:01:06 -05:00
mehrdadn
981f67bfb0
Fix more Windows issues ( #9011 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-19 18:51:45 -07:00
mehrdadn
f8d49d69c1
Fix and merge asio client read/write operations ( #9026 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-19 18:49:55 -07:00
Gabriele Oliaro
311c55132c
redefined SchedulingClass to avoid including the FunctionDescriptor ( #9022 )
...
* redefined SchedulingClass to avoid including the FunctionDescriptor
* updated TestSchedulingKeys test in DirectTaskTransportTest
2020-06-19 13:12:48 -07:00
mehrdadn
f43cad6371
Fix Google log directory ( #9025 )
2020-06-19 11:00:02 -05:00
Zhilei Chen
d8a9247448
Remove gcs_service_disabled ci jobs and code ( #8854 )
2020-06-19 11:32:27 +08:00
