Commit graph

2363 commits

Author SHA1 Message Date
Yi Cheng
995d3cb487
Update id_specification.md (#18035) 2021-08-24 10:49:56 -07:00
Alex Wu
6e3dd7b3cf
Revert "[Core]make thread of client manager in gcs server configurable (#17978)" (#18041)
This reverts commit f0edbf0d30.
2021-08-24 07:57:59 -07:00
Qing Wang
7c1f14ddd8
Do not connect in constructor to avoid potential risk. (#17916)
* Do not connect in ctor.

* Fix lint.

Co-authored-by: Qing Wang <jovany.wq@antgroup.com>
2021-08-24 16:41:30 +08:00
wanxing
abb46de4dc
[object store refactor 5/n] Add eviction policy tests (#17984)
* add eviction policy tests

* fix object_lifecycle_manager_test build

* make IsObjectExists private
2021-08-24 00:50:28 -07:00
Tao Wang
f0edbf0d30
[Core]make thread of client manager in gcs server configurable (#17978) 2021-08-24 11:27:35 +08:00
chenk008
b9978dd02b
[Core] revert: revert Unified worker starter (#18008) 2021-08-23 13:34:32 -07:00
Clark Zinzow
5ca28b1cc8
[Core] Update Bazel (to 3.4.1), gRPC, boringssl, and absl as a precursor to gRPC streaming PR. (#17903)
* Update Bazel (to 3.4.1), gRPC, boringssl, absl.

* Always reinstall Bazel if needing to upgrade to a new Bazel version.

* Add patch for properly detecting Windows Python headers when building gRPC.

* Add minimum Bazel version check.

* Update docs with new Bazel version.
2021-08-21 11:33:11 -07:00
Edward Oakes
b969aa3c80
[dashboard] Don't start dashboard agent when missing dependencies (#17966) 2021-08-21 01:04:21 -07:00
Lixin Wei
05502da271
Add dispatch proxy to event tracker (#17983) 2021-08-20 15:32:10 -07:00
SangBin Cho
cd42d30d7b
[Core] Removing GCS object directory from raylet (#17962) 2021-08-20 12:57:16 -07:00
Stephanie Wang
b8fe776638
[core] Fix inlined nested ids (#17834)
* test

* Use ObjectRef instead of ObjectID in nested refs

* java

* doc

* java

* build

* build

* x

* lint

* simplify

* fix
2021-08-20 08:58:29 -07:00
Eric Liang
236b772465
Revert "[GCS] GCS Based Actor Scheduler (#16580)" (#17941)
This reverts commit a9b4545502.
2021-08-19 21:46:52 -07:00
Eric Liang
661ac4e37b
Remove last traces of ref-counting flag (#17932) 2021-08-19 21:08:13 -07:00
Chen Shen
a16a25852a
[Core] fix event race condition (#17947) 2021-08-19 14:20:34 -07:00
Eric Liang
a9073d16f4
Revert "[Core] Unified worker initiators (#17401)" (#17935)
This reverts commit c3764ffd7d.
2021-08-18 18:06:24 -07:00
Chen Shen
89d83228f6
[Core][Plasma-store] add stats-collector that eagerly collect stats 2021-08-18 13:47:50 -07:00
Chong-Li
a9b4545502
[GCS] GCS Based Actor Scheduler (#16580) 2021-08-18 13:44:59 -07:00
Eric Liang
5536c5fff6
Add namespace argument to Ray client get actor call (#17878) 2021-08-17 16:41:18 -07:00
Chen Shen
880797d5c2
[Core][Test] Add ubsan support for C++ tests (#17812)
* support ubsan

* update
2021-08-17 10:22:03 -07:00
chenk008
c3764ffd7d
[Core] Unified worker initiators (#17401)
* use setup_worker as starter

* use setup_worker as starter

* add java test

* fix

* fix

* lint

* sleep in ci

* sleep in ci

* fix ut

* fix

* fix

* fix

* fix

* fix

* fix

* change test size

* test

* fix

* fix

* fix ut

* restore sgd test

* change test size

* fix merge confict

* restore cpp worker flag

* fix

* fix

* add worker-languange in setup_runtime_env.py

* lint

* fix java command

Co-authored-by: root <chenk008>
2021-08-17 19:37:26 +08:00
Guyang Song
8227e24424
[event] event framework integration in raylet, gcs server and core worker (#17671) 2021-08-17 11:21:23 +08:00
Chen Shen
a9757a86b3
[Core] Fix nested ref count bug: add NestedIds to reference_counter once a task returns (#17802)
* add nested reference

* fix bug
2021-08-16 19:02:26 -07:00
Yi Cheng
03a82d733a
Revert "Revert "Export useful metrics"" (#17755)
* Revert "Revert "[Observability] Export useful metrics (#17578)" (#17752)"

This reverts commit 02e79f3fe5.

* Update metric.h

* up

* up

* Update server_call.h

* Update test_metrics_agent.py

* up

* fix comment
2021-08-16 17:05:56 -07:00
Ian Rodney
2f200e5c2b
[Client] Pass ray.init() args to the remote server (#17776) 2021-08-16 12:34:01 -07:00
Alex Wu
1209a87ead
[core] Remove push based resource report code path (#17825) 2021-08-16 12:03:38 -07:00
Chen Shen
b349c6bc4f
[object store refactor 4/n] object lifecycle manager (#17344)
* lifecycle

* address comments
2021-08-16 09:58:35 -07:00
qicosmos
a2a1c46c83
[C++ Worker]Fix for mac (#17633)
* linkopts shared

* replace gflags with absl flags

* fix

* add test option

* fix

* add cpp worker to mac ci

* fix

* support empty redis password;mod arc argv

* add encoding

* test

* ignore example test on mac

* support mac

* fix

* fix and update doc

* fix

* fix run.sh

* fix init

* fix typo

* fix run.sh

* fix lint

Co-authored-by: 久龙 <guyang.sgy@antfin.com>
2021-08-13 12:22:37 +08:00
SangBin Cho
21635b32e5
[Core] Fix the segfault (#17772) 2021-08-12 18:17:50 -07:00
Yi Cheng
e32d33f39c
Fix ray.init hanging due to failure. (#17732)
* up

* change to 30s

* up

* up

* format
2021-08-12 16:56:10 -07:00
wanxing
e4c8125c86
Make some function private (#17729)
* ReceiveObjectChunk

* more
2021-08-12 15:27:37 -07:00
Lixin Wei
d287fc941b
[Core] Add Running Count to instrumented_io_context (#17664) 2021-08-12 13:56:40 -07:00
Chen Shen
9565fa549e
[Core][RFC] limit the total number of inlined bytes in task request rpc
Co-authored-by: Clark Zinzow <clarkzinzow@gmail.com>
2021-08-12 13:55:54 -07:00
Eric Liang
ce171f10a1
Remove legacy plasma unlimited and pull manager pinning flag (#17753) 2021-08-11 20:19:12 -07:00
Qing Wang
6d6a1ea43e
Support reading system configs from native in Java. (#17703)
* Support reading system configs from native in Java.

* Fix lint

* Lint cpp

* Fix Java cases.

* Address comments.

* Address comments.
2021-08-12 10:06:01 +08:00
Yi Cheng
02e79f3fe5
Revert "[Observability] Export useful metrics (#17578)" (#17752)
This reverts commit bd4db53df2.
2021-08-11 12:21:50 -07:00
SongGuyang
4176e43ef2
Remove binary printing from RAY_CHECK log (#17728) 2021-08-11 18:32:12 +08:00
Yi Cheng
bd4db53df2
[Observability] Export useful metrics (#17578)
* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* checkpoint

* up

* up

* up

* up

* fix

* up

* up

* up

* up

* up

* up

* up

* up

* up

* up

* add comments

* up

* up

* up

* up

* add tests
2021-08-10 17:14:42 -07:00
SongGuyang
63c15d7ced
[core] make 'PopWorker' to be an async function (#17202)
* make 'PopWorker' to be an async function

* pop worker async works

* fix

* address comments

* bugfix

* fix cluster_task_manager_test

* fix

* bugfix of detached actor

* address comments

* fix

* address comments

* fix aioredis

* Revert "fix aioredis"

This reverts commit 041b983eac95b105ab0e853e84c4cf2647008431.

* bug fix

* fix

* fix test_step_resources test

* format

* add unit test

* fix

* add test case PopWorkerStatus

* address commit

* fix lint

* address comments

* add python test

* address comments

* make an independent function

* Update test_basic_3.py

Co-authored-by: Hao Chen <chenh1024@gmail.com>
2021-08-10 17:03:17 -07:00
SangBin Cho
6160c06c69
[Core] Fix a bug where get_actor crashes gcs if the actor is already killed. (#17670)
* Fix a bug where get_actor crashes gcs if the actor is already killed.

* Test the restart code path.

* Add an additional test

* Add a comment

* addressed code review.
2021-08-10 09:58:09 -07:00
Yi Cheng
473740b739
[gcs] Fix actor killing hang due to race condition (#17634)
* Revert "Revert "[gcs] Fix actor killing race condition (#17456)" (#17599)"

This reverts commit 381ffdb6d0.

* update

* format

* up
2021-08-09 21:11:26 -07:00
qicosmos
05da724521
[C++ Worker] Replace Ray::xxx with ray::xxx and update namespaces (#17388) 2021-08-10 11:17:59 +08:00
wanxing
8312628c30
Remove unused Spill function (#17607) 2021-08-09 10:10:03 -07:00
Tao Wang
5990b60f8b
[Core]Cache named actor in local in case of getting them from GCS frequently. (#17339)
* [Core]Cach named actor in local in case of getting them from GCS frequently

* lint

* fix nullptr

* typo

* add namespace to cache

* lint

* lock, reference and others

* lint

* fix comments and add test

* lint

* lint

* optimize test

* add necessary fields in pub for caching

* add removing test

* fix test
2021-08-09 14:01:57 +08:00
Hao Chen
0858f0e4f2
Change core worker C++ namespace to ray::core (#17610) 2021-08-08 23:34:25 +08:00
SangBin Cho
654718902f
Fix (#17660) 2021-08-07 18:07:27 -07:00
Qing Wang
4cc34588db
[Core] Support ConcurrentGroup part1 (#16795)
* Core change and Java change.

* Fix void call.

* Address comments and fix cases.

* Fix asyncio
2021-08-07 22:41:33 +08:00
SangBin Cho
4616e8a03c
Fix wrong invariant pubsub (#17620)
* ip

* loose check failure

* Fix the bug properly.

* Fix comments.
2021-08-06 14:14:54 -07:00
liuyang-my
12bd904594
[Serve] Define BackendConfig protobuf and adapt it in Java (#17201) 2021-08-06 09:50:45 -07:00
Zhi Lin
82123123c4
[object store] Java API for Assign the object owner in Ray.put() (#17237)
Co-authored-by: Qing Wang <kingchin1218@126.com>
Co-authored-by: Kai Yang <kfstorm@outlook.com>
2021-08-06 15:26:59 +08:00
Stephanie Wang
a06d71477f
[core] Do not spill back tasks blocked on args to blocked nodes (#17550)
Co-authored-by: Eric Liang <ekhliang@gmail.com>
2021-08-05 20:43:32 -07:00