Commit graph

191 commits

Author SHA1 Message Date
Siyuan (Ryans) Zhuang
b8dda0e3d0
[Serialization] Fix buffer alignment issues (#11888)
* fix buffer alignment issues

* remove unused fields

* aligned memory allocation

* windows compat

* license. fix compiler warnings

* fix compilation error

* reinterpret_cast
2020-11-10 23:44:16 -08:00
Eric Liang
ee2da0cf45
[Core] PushManager for reliable broadcast (#11869) 2020-11-09 18:01:47 -08:00
Barak Michener
27c810a97e
Basic protos for ray client (#11762) 2020-11-05 16:23:54 -08:00
Stephanie Wang
0ba777af99
[Object spilling] Add policy to automatically spill objects on OutOfMemory (#11673) 2020-11-02 12:42:02 -08:00
Stephanie Wang
427b5af0ae
[Object spilling] Refactor raylet to add a local object manager class (#11647)
* Fix pytest...

* Release objects that have been spilled

* GCS object table interface refactor

* Add spilled URL to object location info

* refactor to include spilled URL in notifications

* improve tests

* Add spilled URL to object directory results

* Remove force restore call

* Merge spilled URL and location

* fix

* tmp

* refactor

* unit test skeleton

* unit testing

* unit test fixes

* cleanup

* cleanup

* update

* Separate pinning from waiting for object free, fixes pytest

* Update src/ray/raylet/local_object_manager.h

Co-authored-by: Eric Liang <ekhliang@gmail.com>

Co-authored-by: Tyler Westenbroek <westenbroekt@berkeley.edu>
Co-authored-by: Eric Liang <ekhliang@gmail.com>
2020-10-28 10:38:42 -04:00
Scott Graham
c4ae94d60b
[autoscaler] Azure deployment fixes (#11613)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-10-27 15:27:18 -07:00
Max Fitton
caf3b04b27
[Dashboard] Turn on new dashboard by default pt 2 (#11510) 2020-10-23 15:52:14 -05:00
Ian Rodney
acbd12eabf
[Docker] Set Docker as the Default (#11416) 2020-10-19 10:53:30 -07:00
SangBin Cho
b1481c6acf
Revert "[PlacementGroup]Add node manager test framework (#11174)" (#11398)
This reverts commit 241e765d3a.
2020-10-14 11:09:20 -07:00
fangfengbin
241e765d3a
[PlacementGroup]Add node manager test framework (#11174)
* add part code

* add part code

* add part code

* add part code

* add part code

* add part code

* fix ut bug

* fix ut bug

Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-10-13 19:27:11 -07:00
fangfengbin
180c259702
[GCS]Remove unused api(ServiceBasedActorInfoAccessor::AsyncRegister/ServiceBasedActorInfoAccessor::AsyncUpdate) (#11099)
* remove unused gcs api

* fix ut bug

Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-10-02 00:54:28 -07:00
Eric Liang
6a227ae501
[autoscaler] Split autoscaler interface public private (#10898) 2020-09-18 18:16:23 -07:00
Basasuya
5e030db8a5
[EVENT] add log reporter (#10419) 2020-09-16 11:54:05 +08:00
chaokunyang
ccf27a9ad2
[Streaming] Fix streaming ci (#10665) 2020-09-09 16:53:43 +08:00
SangBin Cho
b7040f1310
Revert "[Streaming] fix streaming ci (#9675)" (#10656)
This reverts commit 3645a05644.
2020-09-08 19:07:21 -07:00
chaokunyang
3645a05644
[Streaming] fix streaming ci (#9675) 2020-09-08 22:20:58 +08:00
Eric Liang
519354a39a
[api] Initial API deprecations for Ray 1.0 (#10325) 2020-08-28 15:03:50 -07:00
fyrestone
05c103af94
[Dashboard] Start the new dashboard (#10131)
* Use new dashboard if environment var RAY_USE_NEW_DASHBOARD exists; new dashboard startup

* Make fake client/build/static directory for dashboard

* Add test_dashboard.py for new dashboard

* Travis CI enable new dashboard test

* Update new dashboard

* Agent manager service

* Add agent manager

* Register agent to agent manager

* Add a new line to the end of agent_manager.cc

* Fix merge; Fix lint

* Update dashboard/agent.py

Co-authored-by: SangBin Cho <rkooo567@gmail.com>

* Update dashboard/head.py

Co-authored-by: SangBin Cho <rkooo567@gmail.com>

* Fix bug

* Add tests for dashboard

* Fix

* Remove const from Process::Kill() & Fix bugs

* Revert error check of execute_after

* Raise exception from DashboardAgent.run

* Add more tests.

* Fix compile on Linux

* Use dict comprehension instead of dict(generator)

* Fix lint

* Fix windows compile

* Fix lint

* Test Windows CI

* Revert "Test Windows CI"

This reverts commit 945e01051ec95cff5fcc1c0bc37045b46e7ad9a6.

* Fix ParseWindowsCommandLine bug

* Update src/ray/util/util.cc

Co-authored-by: Robert Nishihara <robertnishihara@gmail.com>

Co-authored-by: 刘宝 <po.lb@antfin.com>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Robert Nishihara <robertnishihara@gmail.com>
2020-08-24 13:24:23 -07:00
Simon Mo
bedc2c24c8
Export Metrics in OpenCensus Protobuf Format (#10080) 2020-08-18 11:32:42 -07:00
Robert Nishihara
36e626e95d
Revert "[Dashboard] Start the new dashboard (#9860)" (#10116)
This reverts commit 739933e5b8.
2020-08-14 14:06:57 -07:00
fyrestone
739933e5b8
[Dashboard] Start the new dashboard (#9860) 2020-08-13 11:01:46 +08:00
Zhuohan Li
a6fed4820e
[Core] Preliminary implementation of ownership-based object directory (#9735) 2020-08-11 15:04:13 -07:00
Basasuya
0400a88bf1
[EVENT] Basic Function and Definition (#9657) 2020-08-11 17:36:07 +08:00
mehrdadn
5331c30e35
Improve Clang-IWYU to automatically make #include fixes (#9858)
Co-authored-by: Mehrdad <noreply@github.com>
2020-08-10 12:49:58 -07:00
Barak Michener
1d01c668f0
rpc: Core Worker client pool (#9934) 2020-08-07 16:34:29 -07:00
SangBin Cho
44826878ff
[Core] Remove Legacy Raylet Code (#9936)
* Remove a flag and some methods in node manager including HandleDisconnectedActor, ResubmitTask, and HandleTaskReconstruction

* Make actor creator always required + remove raylet transport

* Remove actor reporter + remove FinishAssignedActorCreationTask

* Remove actor tasks.

* Remove finishactortask and switched it to finishactorcreation task

* Remove reconstruction policy.

* Remove lineage cache.

* Formatting.

* Remove actor frontier code.

* Removed build error.

* Revert "Remove reconstruction policy."

This reverts commit 9d25c9bced4da5fbcac5d484d51013345f16513b.

* Recover HandleReconstruction to mark expired objects as failed.
2020-08-06 16:37:50 -07:00
Barak Michener
c16e1b9524
src/ray/protobuf: Break proto rules into a proper BUILD file (#9792) 2020-08-04 11:12:45 -07:00
chaokunyang
5aba53e9b2
[dist] Fix travis deploy for java dist (#9768) 2020-07-30 10:59:11 +08:00
Lingxuan Zuo
156067b423
[Stats] enable core worker stats (#9355) 2020-07-29 17:28:33 +08:00
mehrdadn
fb5280f21b
Fix some Windows CI issues (#9708)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-28 18:10:23 -07:00
SangBin Cho
423dc96cc4
Revert "[dist] swap mac/linux wheel build order (#9746)" and "Fix package and upload ray jar (#9742)" (#9758)
* Revert "[dist] swap mac/linux wheel build order (#9746)"

This reverts commit a9340565ff.

* Revert "Fix package and upload ray jar (#9742)"

This reverts commit c290c308fe.
2020-07-28 15:34:29 -07:00
Alex Wu
feb3751824
[New scheduler] First unit test for task manager (#9696)
* .

* .

* refactor WorkerInterface

* .

* Basic unit test structure complete?

* .

* bad git >:-(

* small clean up

* CR

* .

* .

* One more fixture

* One more fixture

* .

* .

* bazel-format

* .
2020-07-28 09:44:58 -07:00
chaokunyang
c290c308fe
Fix package and upload ray jar (#9742) 2020-07-28 11:53:25 +08:00
Simon Mo
7740136b93
Revert "Package and upload ray cross-platform jar (#9540)" (#9730)
This reverts commit 881032593d.
2020-07-27 10:40:21 -07:00
chaokunyang
881032593d
Package and upload ray cross-platform jar (#9540) 2020-07-27 17:20:20 +08:00
SangBin Cho
d49b19c24c
[Stats] Improve Stats::Init & Add it to GCS server (#9563) 2020-07-25 10:42:08 -07:00
Lingxuan Zuo
cd42450fc1
[Metrics] Java metric API (#9377) 2020-07-22 10:35:08 +08:00
Siyuan (Ryans) Zhuang
4accc16995
[Core] Replace the Plasma eventloop with boost::asio (#9431) 2020-07-20 02:52:51 -07:00
mehrdadn
2554a1a997
Bazel fixes (#9519) 2020-07-19 12:53:08 -07:00
Lingxuan Zuo
ce3f542739
[Metric] new cython interface for python worker metric (#9469) 2020-07-19 10:43:21 +08:00
Alex Wu
a78c5d5ef2
[New scheduler] Queueing refactor (#9491)
* .

* test_args passes

* .

* test_basic.py::test_many_fractional_resources causes ray to hang

* test_basic.py::test_many_fractional_resources causes ray to hang

* .

* .

* useful

* test_many_fractional_resources fails instead of hanging now :)

* Passes test_fractional_resources

* .

* .

* Some cleanup

* git is hard

* cleanup

* .

* .

* .

* .

* .

* .

* .

* cleanup

* address reviews

* address reviews

* more refactor

* :)

* travis pls

* .

* travis pls

* .
2020-07-17 11:08:03 -07:00
Alisa
f080aa6ce3
Add placement group manager and some code in core_worker (#9120)
Co-authored-by: Lingxuan Zuo <skyzlxuan@gmail.com>
2020-07-17 20:49:51 +08:00
mehrdadn
37942ea1e7
Windows cleanup (#9508)
* Remove unneeded code for Windows

* Get rid of usleep()

* Make platform_shims includes non-transitive

Co-authored-by: Mehrdad <noreply@github.com>
2020-07-17 02:08:15 -07:00
Lingxuan Zuo
3a74164289
[Stats] Fix metric exporter test (#9376) 2020-07-17 14:38:24 +08:00
SangBin Cho
f6eb47fc1f
[Stats] metrics agent exporter (#9361) 2020-07-14 11:49:16 -05:00
Siyuan (Ryans) Zhuang
d57ff5e2af
Remove legacy C++ code (#9459) 2020-07-14 00:57:42 -07:00
Siyuan (Ryans) Zhuang
4da97a7c99
[Core] Build raylet client as an independent component (#9434) 2020-07-13 16:00:32 -07:00
mehrdadn
3d65682e62
Bazel selects compiler flags based on compiler (#9313)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-13 15:31:46 -07:00
mehrdadn
5c853eaa6a
Fix copy to workspace (#9400) 2020-07-11 14:27:56 +02:00
Alisa
f0a72ad985
[Core] Add placement group scheduler and some api in resource scheduler (#9039)
* Add placement group scheduler and some api of resource scheduler.
Merge fix cv hang in multithread variables race (#8984).

* change the bundle id and delete unit count in bundle

change vector<bundle_spec> to vector<shared_ptr<bundle_spec>>

Add placement group scheduler and some api of resource scheduler.
Merge fix cv hang in multithread variables race (#8984).

change the bundle id and delete unit count in bundle

remove CheckIfSchedulable()

add comments and fix the bug in resource

* fix placement group schedule

* add placement group scheduler and change some api in resource scheduler

* fix by the comments

* fix conflict

* fix lint

* fix lint

* fix bug in merge

* fix lint

Co-authored-by: Lingxuan Zuo <skyzlxuan@gmail.com>
2020-07-09 15:37:18 -07:00