mehrdadn
5331c30e35
Improve Clang-IWYU to automatically make #include fixes ( #9858 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-08-10 12:49:58 -07:00
Barak Michener
1d01c668f0
rpc: Core Worker client pool ( #9934 )
2020-08-07 16:34:29 -07:00
SangBin Cho
44826878ff
[Core] Remove Legacy Raylet Code ( #9936 )
...
* Remove a flag and some methods in node manager including HandleDisconnectedActor, ResubmitTask, and HandleTaskReconstruction
* Make actor creator always required + remove raylet transport
* Remove actor reporter + remove FinishAssignedActorCreationTask
* Remove actor tasks.
* Remove finishactortask and switched it to finishactorcreation task
* Remove reconstruction policy.
* Remove lineage cache.
* Formatting.
* Remove actor frontier code.
* Removed build error.
* Revert "Remove reconstruction policy."
This reverts commit 9d25c9bced4da5fbcac5d484d51013345f16513b.
* Recover HandleReconstruction to mark expired objects as failed.
2020-08-06 16:37:50 -07:00
Barak Michener
c16e1b9524
src/ray/protobuf: Break proto rules into a proper BUILD file ( #9792 )
2020-08-04 11:12:45 -07:00
chaokunyang
5aba53e9b2
[dist] Fix travis deploy for java dist ( #9768 )
2020-07-30 10:59:11 +08:00
Lingxuan Zuo
156067b423
[Stats] enable core worker stats ( #9355 )
2020-07-29 17:28:33 +08:00
mehrdadn
fb5280f21b
Fix some Windows CI issues ( #9708 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-28 18:10:23 -07:00
SangBin Cho
423dc96cc4
Revert "[dist] swap mac/linux wheel build order ( #9746 )" and "Fix package and upload ray jar ( #9742 )" ( #9758 )
...
* Revert "[dist] swap mac/linux wheel build order (#9746 )"
This reverts commit a9340565ff#9742 )"
This reverts commit c290c308fe
2020-07-28 15:34:29 -07:00
Alex Wu
feb3751824
[New scheduler] First unit test for task manager ( #9696 )
...
* .
* .
* refactor WorkerInterface
* .
* Basic unit test structure complete?
* .
* bad git >:-(
* small clean up
* CR
* .
* .
* One more fixture
* One more fixture
* .
* .
* bazel-format
* .
2020-07-28 09:44:58 -07:00
chaokunyang
c290c308fe
Fix package and upload ray jar ( #9742 )
2020-07-28 11:53:25 +08:00
Simon Mo
7740136b93
Revert "Package and upload ray cross-platform jar ( #9540 )" ( #9730 )
...
This reverts commit 881032593d
2020-07-27 10:40:21 -07:00
chaokunyang
881032593d
Package and upload ray cross-platform jar ( #9540 )
2020-07-27 17:20:20 +08:00
SangBin Cho
d49b19c24c
[Stats] Improve Stats::Init & Add it to GCS server ( #9563 )
2020-07-25 10:42:08 -07:00
Lingxuan Zuo
cd42450fc1
[Metrics] Java metric API ( #9377 )
2020-07-22 10:35:08 +08:00
Siyuan (Ryans) Zhuang
4accc16995
[Core] Replace the Plasma eventloop with boost::asio ( #9431 )
2020-07-20 02:52:51 -07:00
mehrdadn
2554a1a997
Bazel fixes ( #9519 )
2020-07-19 12:53:08 -07:00
Lingxuan Zuo
ce3f542739
[Metric] new cython interface for python worker metric ( #9469 )
2020-07-19 10:43:21 +08:00
Alex Wu
a78c5d5ef2
[New scheduler] Queueing refactor ( #9491 )
...
* .
* test_args passes
* .
* test_basic.py::test_many_fractional_resources causes ray to hang
* test_basic.py::test_many_fractional_resources causes ray to hang
* .
* .
* useful
* test_many_fractional_resources fails instead of hanging now :)
* Passes test_fractional_resources
* .
* .
* Some cleanup
* git is hard
* cleanup
* .
* .
* .
* .
* .
* .
* .
* cleanup
* address reviews
* address reviews
* more refactor
* :)
* travis pls
* .
* travis pls
* .
2020-07-17 11:08:03 -07:00
Alisa
f080aa6ce3
Add placement group manager and some code in core_worker ( #9120 )
...
Co-authored-by: Lingxuan Zuo <skyzlxuan@gmail.com>
2020-07-17 20:49:51 +08:00
mehrdadn
37942ea1e7
Windows cleanup ( #9508 )
...
* Remove unneeded code for Windows
* Get rid of usleep()
* Make platform_shims includes non-transitive
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-17 02:08:15 -07:00
Lingxuan Zuo
3a74164289
[Stats] Fix metric exporter test ( #9376 )
2020-07-17 14:38:24 +08:00
SangBin Cho
f6eb47fc1f
[Stats] metrics agent exporter ( #9361 )
2020-07-14 11:49:16 -05:00
Siyuan (Ryans) Zhuang
d57ff5e2af
Remove legacy C++ code ( #9459 )
2020-07-14 00:57:42 -07:00
Siyuan (Ryans) Zhuang
4da97a7c99
[Core] Build raylet client as an independent component ( #9434 )
2020-07-13 16:00:32 -07:00
mehrdadn
3d65682e62
Bazel selects compiler flags based on compiler ( #9313 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-13 15:31:46 -07:00
mehrdadn
5c853eaa6a
Fix copy to workspace ( #9400 )
2020-07-11 14:27:56 +02:00
Alisa
f0a72ad985
[Core] Add placement group scheduler and some api in resource scheduler ( #9039 )
...
* Add placement group scheduler and some api of resource scheduler.
Merge fix cv hang in multithread variables race (#8984 ).
* change the bundle id and delete unit count in bundle
change vector<bundle_spec> to vector<shared_ptr<bundle_spec>>
Add placement group scheduler and some api of resource scheduler.
Merge fix cv hang in multithread variables race (#8984 ).
change the bundle id and delete unit count in bundle
remove CheckIfSchedulable()
add comments and fix the bug in resource
* fix placement group schedule
* add placement group scheduler and change some api in resource scheduler
* fix by the comments
* fix conflict
* fix lint
* fix lint
* fix bug in merge
* fix lint
Co-authored-by: Lingxuan Zuo <skyzlxuan@gmail.com>
2020-07-09 15:37:18 -07:00
mehrdadn
b6c11f3dd7
Bazel move more shell to native rules ( #9314 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-09 09:02:35 -07:00
Siyuan (Ryans) Zhuang
7e1326c0f6
[Core] Enhance common client connection ( #9367 )
...
* enhance client connection
* add write buffer async
* read message
* add test
2020-07-09 08:59:37 -07:00
SangBin Cho
8f19f1eafb
[Core] Actor handle refactoring ( #8895 )
...
* Marking needed changes.
* Resolve basic dependencies.
* In progress.
* linting.
* In progress 2.
* Linting.
* Refactor done. Cleanup needed.
* Linting.
* Recover kill actor in core worker because it is used inside raylet
* Cleanup.
* Use unique pointer instead. Unit tests are broken now.
* Fix the upstream change.
* Addressed code review 1.
* Lint.
* Addressed code review 2.
* Fix weird github history.
* Lint.
* Linting using clang 7.0.
* Use a better check message.
* Revert cpp stuff.
* Fix weird linting errors.
* Manuall fix all lint issues.
* Update a newline.
* Refactor some interface.
* Addressed all code review.
* Addressed code review
2020-07-07 11:11:41 -07:00
Stephanie Wang
b42d6a1ddc
[core] Refactor task arguments and attach owner address ( #9152 )
...
* Add intended worker ID to GetObjectStatus, tests
* Remove TaskID owner_id
* lint
* Add owner address to task args
* Make TaskArg a virtual class, remove multi args
* Set owner address for task args
* merge
* Fix tests
* Fix
* build
* update
* build
* java
* Move code
* build
* Revert "Fix Google log directory again (#9063 )"
This reverts commit 275da2e400#9063 )""
This reverts commit 4a326fcb148ca09a35bc7de11d89df10edbb56e7.
* lint
2020-07-06 21:25:14 -07:00
mehrdadn
7135cb2aec
Fix .exe file extensions ( #9197 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-02 15:29:34 -05:00
mehrdadn
29acf272b7
Build with Visual C++ ( #9190 )
...
Co-authored-by: Mehrdad <noreply@github.com>
Co-authored-by: Simon Mo <xmo@berkeley.edu>
2020-07-02 09:34:24 -07:00
Lingxuan Zuo
1491508859
[Metrics] Stats supports metric exporters ( #8941 )
2020-07-01 10:54:55 -05:00
ChenZhilei
c11855728a
Remove raylet monitor after use GCS service ( #9179 )
2020-07-01 20:01:52 +08:00
mehrdadn
79c4c67ed9
Split hiredis from Redis and use it directly ( #9176 )
...
* Split hiredis from Redis and use it directly
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-29 18:16:32 -07:00
Siyuan (Ryans) Zhuang
d7549d6184
Replace arrow::Status with ray::Status in Plasma ( #9154 )
...
* add object store status
* replace arrow status with ray status
* cleanup
* remove plasma common.cc
2020-06-26 14:06:34 -07:00
Siyuan (Ryans) Zhuang
80bcbe20c7
[Core] Remove object notification IPC between Plasma and Raylet (initial step) ( #8939 )
...
* initial refactoring
redirect notifications to eventloop
implement direct notifications
* protect vector with mutex
2020-06-24 13:54:40 -07:00
fangfengbin
c295284370
Optimize gcs server resubscribe ( #8896 )
2020-06-17 20:05:50 +08:00
mehrdadn
101c215125
Get more tests running on Windows ( #6537 )
...
* Get rid of system() calls
* Work around '/usr/share/mini' showing up on GitHub Actions (probably due to psutil truncation)
https://github.com/ray-project/ray/runs/722480047?check_suite_focus=true
* Don't check for socket max path length on Windows
* Don't check for socket existence on Windows
* Fix race condition in Windows fate-sharing
* Work around missing .exe extension for Redis tests
* Add more tests to GitHub Actions
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-12 21:32:10 -07:00
Siyuan (Ryans) Zhuang
ed77c8b16c
[Core] Use global variable to eliminate force thread termination in plasma ( #8912 )
...
* use global variable to eliminate force thread termination
2020-06-12 14:20:53 -07:00
Siyuan (Ryans) Zhuang
4b31b383f3
[Core] Run Plasma Store as a Raylet thread (with a feature flag) ( #8897 )
...
* integrate plasma store as a thread (C++)
* integrate plasma store as a thread (Python)
* fix config issues
* remove plasma component fail tests
* without forcefully kill the plasma store thread
2020-06-11 22:54:08 -07:00
mehrdadn
4bda333412
Fix Windows build issue introduced in #8852 ( #8869 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-09 19:21:41 -07:00
Siyuan (Ryans) Zhuang
3d473600a8
[Core] Use Ray ObjectID in Plasma ( #8852 )
...
* Use Ray ObjectIDs instead
* remove unused code
2020-06-09 10:10:49 -07:00
Siyuan (Ryans) Zhuang
c1e6813cea
[core] Move plasma store under object_manager ( #8832 )
...
* move plasma under object directory
* update include paths
* cleanup
* disable lint of third-party libraries
* lint
2020-06-08 18:21:41 -07:00
Siyuan (Ryans) Zhuang
a0247ffe55
Build plasma store as a library ( #8817 )
...
* build plasma store as a library
* remove unused headers
* windows support
2020-06-06 22:11:37 -07:00
Stephanie Wang
b160b83d3e
[core] Queue subscription/unsubscription commands in the GCS ( #8756 )
...
* Only remove callback index if in map
* test
* Queue subscription commands
* lint
* Check status
* update
* update
* update
* Disable GCS restart tests
* lint
2020-06-05 19:49:19 -07:00
mehrdadn
d78757623d
bazel build --compilation_mode=debug ( #6457 )
2020-06-05 14:36:10 +02:00
Siyuan (Ryans) Zhuang
ea05ebe89e
Ship plasma store with Ray ( #7901 )
2020-06-03 17:44:34 -07:00
Lingxuan Zuo
bd4fbcd7fc
Global state accessor jni ( #8637 )
2020-05-27 17:43:47 +08:00
