mehrdadn
2554a1a997
Bazel fixes ( #9519 )
2020-07-19 12:53:08 -07:00
Lingxuan Zuo
ce3f542739
[Metric] new cython interface for python worker metric ( #9469 )
2020-07-19 10:43:21 +08:00
Alex Wu
a78c5d5ef2
[New scheduler] Queueing refactor ( #9491 )
...
* .
* test_args passes
* .
* test_basic.py::test_many_fractional_resources causes ray to hang
* test_basic.py::test_many_fractional_resources causes ray to hang
* .
* .
* useful
* test_many_fractional_resources fails instead of hanging now :)
* Passes test_fractional_resources
* .
* .
* Some cleanup
* git is hard
* cleanup
* .
* .
* .
* .
* .
* .
* .
* cleanup
* address reviews
* address reviews
* more refactor
* :)
* travis pls
* .
* travis pls
* .
2020-07-17 11:08:03 -07:00
Alisa
f080aa6ce3
Add placement group manager and some code in core_worker ( #9120 )
...
Co-authored-by: Lingxuan Zuo <skyzlxuan@gmail.com>
2020-07-17 20:49:51 +08:00
mehrdadn
37942ea1e7
Windows cleanup ( #9508 )
...
* Remove unneeded code for Windows
* Get rid of usleep()
* Make platform_shims includes non-transitive
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-17 02:08:15 -07:00
Lingxuan Zuo
3a74164289
[Stats] Fix metric exporter test ( #9376 )
2020-07-17 14:38:24 +08:00
SangBin Cho
f6eb47fc1f
[Stats] metrics agent exporter ( #9361 )
2020-07-14 11:49:16 -05:00
Siyuan (Ryans) Zhuang
d57ff5e2af
Remove legacy C++ code ( #9459 )
2020-07-14 00:57:42 -07:00
Siyuan (Ryans) Zhuang
4da97a7c99
[Core] Build raylet client as an independent component ( #9434 )
2020-07-13 16:00:32 -07:00
mehrdadn
3d65682e62
Bazel selects compiler flags based on compiler ( #9313 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-13 15:31:46 -07:00
mehrdadn
5c853eaa6a
Fix copy to workspace ( #9400 )
2020-07-11 14:27:56 +02:00
Alisa
f0a72ad985
[Core] Add placement group scheduler and some api in resource scheduler ( #9039 )
...
* Add placement group scheduler and some api of resource scheduler.
Merge fix cv hang in multithread variables race (#8984 ).
* change the bundle id and delete unit count in bundle
change vector<bundle_spec> to vector<shared_ptr<bundle_spec>>
Add placement group scheduler and some api of resource scheduler.
Merge fix cv hang in multithread variables race (#8984 ).
change the bundle id and delete unit count in bundle
remove CheckIfSchedulable()
add comments and fix the bug in resource
* fix placement group schedule
* add placement group scheduler and change some api in resource scheduler
* fix by the comments
* fix conflict
* fix lint
* fix lint
* fix bug in merge
* fix lint
Co-authored-by: Lingxuan Zuo <skyzlxuan@gmail.com>
2020-07-09 15:37:18 -07:00
mehrdadn
b6c11f3dd7
Bazel move more shell to native rules ( #9314 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-09 09:02:35 -07:00
Siyuan (Ryans) Zhuang
7e1326c0f6
[Core] Enhance common client connection ( #9367 )
...
* enhance client connection
* add write buffer async
* read message
* add test
2020-07-09 08:59:37 -07:00
SangBin Cho
8f19f1eafb
[Core] Actor handle refactoring ( #8895 )
...
* Marking needed changes.
* Resolve basic dependencies.
* In progress.
* linting.
* In progress 2.
* Linting.
* Refactor done. Cleanup needed.
* Linting.
* Recover kill actor in core worker because it is used inside raylet
* Cleanup.
* Use unique pointer instead. Unit tests are broken now.
* Fix the upstream change.
* Addressed code review 1.
* Lint.
* Addressed code review 2.
* Fix weird github history.
* Lint.
* Linting using clang 7.0.
* Use a better check message.
* Revert cpp stuff.
* Fix weird linting errors.
* Manuall fix all lint issues.
* Update a newline.
* Refactor some interface.
* Addressed all code review.
* Addressed code review
2020-07-07 11:11:41 -07:00
Stephanie Wang
b42d6a1ddc
[core] Refactor task arguments and attach owner address ( #9152 )
...
* Add intended worker ID to GetObjectStatus, tests
* Remove TaskID owner_id
* lint
* Add owner address to task args
* Make TaskArg a virtual class, remove multi args
* Set owner address for task args
* merge
* Fix tests
* Fix
* build
* update
* build
* java
* Move code
* build
* Revert "Fix Google log directory again (#9063 )"
This reverts commit 275da2e400#9063 )""
This reverts commit 4a326fcb148ca09a35bc7de11d89df10edbb56e7.
* lint
2020-07-06 21:25:14 -07:00
mehrdadn
7135cb2aec
Fix .exe file extensions ( #9197 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-02 15:29:34 -05:00
mehrdadn
29acf272b7
Build with Visual C++ ( #9190 )
...
Co-authored-by: Mehrdad <noreply@github.com>
Co-authored-by: Simon Mo <xmo@berkeley.edu>
2020-07-02 09:34:24 -07:00
Lingxuan Zuo
1491508859
[Metrics] Stats supports metric exporters ( #8941 )
2020-07-01 10:54:55 -05:00
ChenZhilei
c11855728a
Remove raylet monitor after use GCS service ( #9179 )
2020-07-01 20:01:52 +08:00
mehrdadn
79c4c67ed9
Split hiredis from Redis and use it directly ( #9176 )
...
* Split hiredis from Redis and use it directly
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-29 18:16:32 -07:00
Siyuan (Ryans) Zhuang
d7549d6184
Replace arrow::Status with ray::Status in Plasma ( #9154 )
...
* add object store status
* replace arrow status with ray status
* cleanup
* remove plasma common.cc
2020-06-26 14:06:34 -07:00
Siyuan (Ryans) Zhuang
80bcbe20c7
[Core] Remove object notification IPC between Plasma and Raylet (initial step) ( #8939 )
...
* initial refactoring
redirect notifications to eventloop
implement direct notifications
* protect vector with mutex
2020-06-24 13:54:40 -07:00
fangfengbin
c295284370
Optimize gcs server resubscribe ( #8896 )
2020-06-17 20:05:50 +08:00
mehrdadn
101c215125
Get more tests running on Windows ( #6537 )
...
* Get rid of system() calls
* Work around '/usr/share/mini' showing up on GitHub Actions (probably due to psutil truncation)
https://github.com/ray-project/ray/runs/722480047?check_suite_focus=true
* Don't check for socket max path length on Windows
* Don't check for socket existence on Windows
* Fix race condition in Windows fate-sharing
* Work around missing .exe extension for Redis tests
* Add more tests to GitHub Actions
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-12 21:32:10 -07:00
Siyuan (Ryans) Zhuang
ed77c8b16c
[Core] Use global variable to eliminate force thread termination in plasma ( #8912 )
...
* use global variable to eliminate force thread termination
2020-06-12 14:20:53 -07:00
Siyuan (Ryans) Zhuang
4b31b383f3
[Core] Run Plasma Store as a Raylet thread (with a feature flag) ( #8897 )
...
* integrate plasma store as a thread (C++)
* integrate plasma store as a thread (Python)
* fix config issues
* remove plasma component fail tests
* without forcefully kill the plasma store thread
2020-06-11 22:54:08 -07:00
mehrdadn
4bda333412
Fix Windows build issue introduced in #8852 ( #8869 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-09 19:21:41 -07:00
Siyuan (Ryans) Zhuang
3d473600a8
[Core] Use Ray ObjectID in Plasma ( #8852 )
...
* Use Ray ObjectIDs instead
* remove unused code
2020-06-09 10:10:49 -07:00
Siyuan (Ryans) Zhuang
c1e6813cea
[core] Move plasma store under object_manager ( #8832 )
...
* move plasma under object directory
* update include paths
* cleanup
* disable lint of third-party libraries
* lint
2020-06-08 18:21:41 -07:00
Siyuan (Ryans) Zhuang
a0247ffe55
Build plasma store as a library ( #8817 )
...
* build plasma store as a library
* remove unused headers
* windows support
2020-06-06 22:11:37 -07:00
Stephanie Wang
b160b83d3e
[core] Queue subscription/unsubscription commands in the GCS ( #8756 )
...
* Only remove callback index if in map
* test
* Queue subscription commands
* lint
* Check status
* update
* update
* update
* Disable GCS restart tests
* lint
2020-06-05 19:49:19 -07:00
mehrdadn
d78757623d
bazel build --compilation_mode=debug ( #6457 )
2020-06-05 14:36:10 +02:00
Siyuan (Ryans) Zhuang
ea05ebe89e
Ship plasma store with Ray ( #7901 )
2020-06-03 17:44:34 -07:00
Lingxuan Zuo
bd4fbcd7fc
Global state accessor jni ( #8637 )
2020-05-27 17:43:47 +08:00
Tao Wang
a1298686d7
[TEST]Use manager class to start/stop components instead of spreading duplicated codes everywhere ( #8500 )
2020-05-27 16:51:51 +08:00
fangfengbin
765d470c40
Add gcs object manager ( #8298 )
2020-05-25 17:21:35 +08:00
fangfengbin
9347a5d10c
Add global state accessor of jobs ( #8401 )
2020-05-18 20:32:05 +08:00
Tao Wang
acffdb2349
[TEST]use cc_test to run core_worker_test, enforce/reuse RedisServiceManagerForTest ( #8443 )
2020-05-17 18:43:00 +08:00
mehrdadn
ac1ed293e3
Patch redis-py bug for Windows ( #8386 )
2020-05-12 10:41:45 -05:00
mehrdadn
4bdef78e2e
Various CI fixes and cleanup ( #8289 )
2020-05-05 10:47:49 -07:00
mehrdadn
254b1ec370
Set up testing and wheels for Windows on GitHub Actions ( #8131 )
...
* Move some Java tests into ci.sh
* Move C++ worker tests into ci.sh
* Define run()
* Prepare to move Python tests into ci.sh
* Fix issues in install-dependencies.sh
* Reload environment for GitHub Actions
* Move wheels to ci.sh and fix related issues
* Don't bypass failures in install-ray.sh anymore
* Make CI a little quieter
* Move linting into ci.sh
* Add vitals test right after build
* Fix os.uname() unavailability on Windows
Co-authored-by: Mehrdad <noreply@github.com>
2020-04-29 21:19:02 -07:00
fangfengbin
deffc340ea
[GCS]Add in-memory gcs table storage ( #8184 )
2020-04-28 17:19:46 +08:00
fangfengbin
5bff707d20
[GCS]Add in-memory store client ( #8144 )
2020-04-26 19:09:26 +08:00
fangfengbin
713e375d50
[GCS]GCS adapts to job table pub sub ( #8145 )
2020-04-24 16:33:25 +08:00
mehrdadn
ba00c29b67
Factor out Travis 'install' sections for use with GitHub Actions ( #7988 )
2020-04-15 08:10:22 -07:00
fangfengbin
efbaf155b2
[GCS]Add publish and subscribe function of gcs table ( #7909 )
2020-04-15 04:24:52 -07:00
fangfengbin
c17404918c
[GCS]Add gcs table storage interface ( #7949 )
2020-04-15 10:48:12 +08:00
ZhuSenlin
4a81793ba5
GCS-Based actor management implementation ( #6763 )
...
* add gcs actor manager
* fix test_metrics.py
* fix TestTaskInfo
* fix comment
* fix comment
* fix comment
* fix comment
* fix comment
* fix comment
* fix compile error
* fix merge error
Co-authored-by: senlin.zsl <senlin.zsl@antfin.com>
2020-04-13 09:48:48 -07:00
micafan
c222d64ca1
[GCS] Add MessagePublisher to GCS ( #7771 )
2020-04-13 19:32:28 +08:00
