Commit graph

152 commits

Author SHA1 Message Date
Qing Wang
ad90693ca8
[Java] Enhancement single process mode (#6795)
* enhancement

* Add ut

* Update java/runtime/src/main/java/org/ray/runtime/task/LocalModeTaskSubmitter.java

Co-Authored-By: Kai Yang <kfstorm@outlook.com>

* Update java/test/src/main/java/org/ray/api/test/RunModeTest.java

Co-Authored-By: Kai Yang <kfstorm@outlook.com>

* Address comments

* Use ExecutorSerivce to replace raw thread

Co-authored-by: Kai Yang <kfstorm@outlook.com>
2020-01-15 21:38:53 +08:00
Kai Yang
ddd4c42fe5 [Java] Add killActor API in Java (#6728)
* Add killActor API in Java

* fix javadoc

* update test case

* Address comments
2020-01-14 17:12:00 +08:00
chaokunyang
4097d076d4 Package ray java jars into wheels (#6600) 2020-01-10 11:41:00 +08:00
Sven
60d4d5e1aa Remove future imports (#6724)
* Remove all __future__ imports from RLlib.

* Remove (object) again from tf_run_builder.py::TFRunBuilder.

* Fix 2xLINT warnings.

* Fix broken appo_policy import (must be appo_tf_policy)

* Remove future imports from all other ray files (not just RLlib).

* Remove future imports from all other ray files (not just RLlib).

* Remove future import blocks that contain `unicode_literals` as well.
Revert appo_tf_policy.py to appo_policy.py (belongs to another PR).

* Add two empty lines before Schedule class.

* Put back __future__ imports into determine_tests_to_run.py. Fails otherwise on a py2/print related error.
2020-01-09 00:15:48 -08:00
Zhijun Fu
d2bba596ab Fix actor reconstruction with direct call (#6570) 2019-12-26 10:59:50 +08:00
Kai Yang
af9f76359a Fix crash caused by unit test testWaitAndCrash (#6549) 2019-12-20 11:06:53 +08:00
Kai Yang
3bb680a719 [Java] Add getAsyncContext and setAsyncContext API (#6439)
* Add getAsyncContext and setAsyncContext API

* address comment

* fix bug

* Add test case
2019-12-19 18:08:58 +08:00
Kai Yang
7e6defca72 fix ClassLoaderTest.java (#6502) 2019-12-18 14:48:25 +08:00
Kai Yang
b7d5c8f220 [Java] Fix multiple FunctionManagers creating multiple ClassLoader s (#6434) 2019-12-16 14:04:44 +08:00
Kai Yang
c2499c802f disable actor checkpointing and reconstruction test in direct call mode (#6490) 2019-12-15 17:54:31 +08:00
Kai Yang
cd250ba0bc [Java] ID length fix (#6454) 2019-12-15 16:01:05 +08:00
Hao Chen
5cc3e1341a
[Java] Cache result in RayObjectImpl (#6414) 2019-12-11 11:26:01 +08:00
Kai Yang
eb912b68b1 [Java] Fix instanceof RayPyActor (#6377) 2019-12-07 16:28:29 +08:00
Zhijun Fu
b88b8202cc fix java build failure (#6062) 2019-12-06 14:38:43 +08:00
Kai Yang
d51583dbd6 Add test listener to show the test progress of java UT (#6341) 2019-12-03 16:34:07 +08:00
Kai Yang
c580955840 [Java] Fix some potential bugs about Ray.shutdown() (#5693) 2019-09-24 10:44:17 +08:00
Kai Yang
ed761900f6 [Java] Support direct actor call in Java worker (#5504) 2019-09-09 14:29:20 +08:00
Kai Yang
d8f5804690 Support metadata for passing by value task arguments (#5527) 2019-09-08 11:07:48 +08:00
Kai Yang
732336fc4f [Java] Support multiple workers in Java worker process (#5505) 2019-09-07 22:52:05 +08:00
Hao Chen
fab5ae64c4 [Java] Automatically clean up temp files. (#5507)
* Remove unused code

* Auto clean up temp files

* refine

* fix

* print warning

* fix
2019-08-24 17:20:27 +08:00
Hao Chen
239c177fe8
[Java] Support calling functions returning void (#5494) 2019-08-23 21:10:15 +08:00
Kai Yang
7812dd5636 [Java] Fix getCurrentActorId in multi-threading scenario. (#5506) 2019-08-23 17:56:10 +08:00
Philipp Moritz
599cc2be60
Revert raylet to worker GRPC communication back to asio (#5450) 2019-08-17 19:11:32 -07:00
Kai Yang
b1aae0e398 [Java worker] Migrate task execution and submission on top of core worker (#5370) 2019-08-16 13:52:13 +08:00
Qing Wang
d372f24e3c
[ID Refactor] Refactor ActorID, TaskID and ObjectID (#5286)
* Refactor ActorID, TaskID on the Java side.

Left a TODO comment

WIP for ObjectID

ADD test

Fix

Add java part

Fix Java test

Fix

Refine test.

Enable test in CI

* Extra a helper function.

* Resolve TODOs

* Fix Python CI

* Fix Java lint

* Update .travis.yml

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Address some comments.

Address some comments.

Add id_specification.rst

Reanme id_specification.rst to id_specification.md

typo

Address zhijun's comments.

Fix test

Address comments.

Fix lint

Address comments

* Fix test

* Address comments.

* Fix build error

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Address comments

* Update src/ray/common/id.h

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/common/id.h

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/common/id.h

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update src/ray/design_docs/id_specification.md

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Address comments.

* Address comments.

* Address comments.

* Update C++ part to make sure task id is generated determantic

* WIP

* Fix core worker

* Fix Java part

* Fix comments.

* Add Python side

* Fix python

* Address comments

* Fix linting

* Fix

* Fix C++ linting

* Add JobId() method to TaskID

* Fix linting

* Update src/ray/common/id.h

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update java/api/src/main/java/org/ray/api/id/TaskId.java

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update java/api/src/main/java/org/ray/api/id/TaskId.java

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update java/api/src/main/java/org/ray/api/id/ActorId.java

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Address comments

* Add DriverTaskId embeding job id

* Fix tests

* Add python dor_fake_driver_id

* Address comments and fix linting

* Fix CI
2019-08-07 11:04:51 +08:00
Kai Yang
384cbfb211 Fix duplicated timeout logic in AbstractRayRuntime.get() (#5338) 2019-08-06 13:36:49 +08:00
Qing Wang
a3d4f9f16d
Fix the issue when passing multiple options in one string (#5241)
* Fix

* Fix linting

* Fix linting

* Address

* Fix test
2019-07-23 12:28:54 +08:00
Kai Yang
806524384b [Java worker] Refactor object store and worker context on top of core worker (#5079) 2019-07-16 20:58:02 +08:00
Hao Chen
0ec3a16bbd
Fix Java MultithreadingTest (#5182) 2019-07-12 19:00:13 +08:00
Qing Wang
f2293243cc
[ID Refactor] Shorten the length of JobID to 4 bytes (#5110)
* WIP

* Fix

* Add jobid test

* Fix

* Add python part

* Fix

* Fix tes

* Remove TODOs

* Fix C++ tests

* Lint

* Fix

* Fix exporting functions in multiple ray.init

* Fix java test

* Fix lint

* Fix linting

* Address comments.

* FIx

* Address and fix linting

* Refine and fix

* Fix

* address

* Address comments.

* Fix linting

* Fix

* Address

* Address comments.

* Address

* Address

* Fix

* Fix

* Fix

* Fix lint

* Fix

* Fix linting

* Address comments.

* Fix linting

* Address comments.

* Fix linting

* address comments.

* Fix
2019-07-11 14:25:16 +08:00
Hao Chen
88365d4112
Fix Java MultithreadingTest (#5170) 2019-07-11 13:40:40 +08:00
Qing Wang
62e4b591e3
[ID Refactor] Rename DriverID to JobID (#5004)
* WIP

WIP

WIP

Rename Driver -> Job

Fix complition

Fix

Rename in Java

In py

WIP

Fix

WIP

Fix

Fix test

Fix

Fix C++ linting

Fix

* Update java/runtime/src/main/java/org/ray/runtime/config/RayConfig.java

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Update src/ray/core_worker/core_worker.cc

Co-Authored-By: Stephanie Wang <swang@cs.berkeley.edu>

* Address comments

* Fix

* Fix CI

* Fix cpp linting

* Fix py lint

* FIx

* Address comments and fix

* Address comments

* Address

* Fix import_threading
2019-06-28 00:44:51 +08:00
Qing Wang
e33d0eac68
Add dynamic worker options for worker command. (#4970)
* Add fields for fbs

* WIP

* Fix complition errors

* Add java part

* FIx

* Fix

* Fix

* Fix lint

* Refine API

* address comments and add test

* Fix

* Address comment.

* Address comments.

* Fix linting

* Refine

* Fix lint

* WIP: address comment.

* Fix java

* Fix py

* Refin

* Fix

* Fix

* Fix linting

* Fix lint

* Address comments

* WIP

* Fix

* Fix

* minor refine

* Fix lint

* Fix raylet test.

* Fix lint

* Update src/ray/raylet/worker_pool.h

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update java/runtime/src/main/java/org/ray/runtime/AbstractRayRuntime.java

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Address comments.

* Address comments.

* Fix test.

* Update src/ray/raylet/worker_pool.h

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Address comments.

* Address comments.

* Fix

* Fix lint

* Fix lint

* Fix

* Address comments.

* Fix linting
2019-06-23 18:08:33 +08:00
Qing Wang
7bda5edc16 Fix Java CI failure (#4995) 2019-06-19 11:36:21 +08:00
Hao Chen
2bf92e02e2
[gRPC] Use gRPC for inter-node-manager communication (#4968) 2019-06-17 19:00:50 +08:00
Qing Wang
e6baffba56
[Java] Add inner class Builder to build call options. (#4956)
* Add Builder class

* format

* Refactor by IDE

* Remove uncessary dependency
2019-06-10 23:52:08 +08:00
Qing Wang
671c0f769e
[Java] Fix serializing issues of RaySerializer (#4887)
* Fix

* Address comment.
2019-06-08 22:56:00 +08:00
Yuhong Guo
1a39fee9c6
Refactor ID Serial 1: Separate ObjectID and TaskID from UniqueID (#4776)
* Enable BaseId.

* Change TaskID and make python test pass

* Remove unnecessary functions and fix test failure and change TaskID to
16 bytes.

* Java code change draft

* Refine

* Lint

* Update java/api/src/main/java/org/ray/api/id/TaskId.java

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update java/api/src/main/java/org/ray/api/id/BaseId.java

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update java/api/src/main/java/org/ray/api/id/BaseId.java

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update java/api/src/main/java/org/ray/api/id/ObjectId.java

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Address comment

* Lint

* Fix SINGLE_PROCESS

* Fix comments

* Refine code

* Refine test

* Resolve conflict
2019-05-22 14:46:30 +08:00
Qing Wang
081708bdef [Java] Dynamic resource API in Java (#4824) 2019-05-21 17:13:48 +08:00
Hao Chen
643f62dc43
[Java][Bazel] Refine auto-generated pom files (#4780) 2019-05-16 11:19:31 +08:00
Ruifang Chen
c578be23a5 [Bazel] Modifying WORKSPACE file, so that you can make the project used as a thirdparty project (#4711)
* Modifying WORKSPACE file, so that you can make the project used as a thirdparty

* Modifying WORKSPACE file, so that you can make the project used as a thirdparty

* add some files

* modify some repositories

* modify the name of 'ray_deps_build_all'
2019-04-28 22:02:49 -07:00
Qing Wang
4dd628a837 [Java] Do not expose GcsClient to normal users (#4675) 2019-04-24 10:50:35 +08:00
Hao Chen
d52b080081
[Java] Avoid unnecessary memory copy and addd a benchmark (#4611) 2019-04-14 00:17:04 +08:00
Romil Bhardwaj
0f42f87ebc Updating zero capacity resource semantics (#4555) 2019-04-12 16:53:57 -07:00
Wang Qing
5cfbfe5df6 [Java] Implement GcsClient (#4601) 2019-04-12 22:44:47 +08:00
Wang Qing
fe07a5b4b1 Add delete_creating_tasks option for internal.free() (#4588)
* add delete creating task objects.

* format code style

* Fix lint

* add tests add address comments.

* Refine test

* Refine java test

* Fix CI

* Refine

* Fix lint

* Fix CI
2019-04-12 13:38:31 +08:00
bibabolynn
ab55a1f93a [Java] Clean up outdated dependencies (#4489) 2019-03-28 14:33:45 +08:00
Robert Nishihara
a22bf1e511 Minor aesthetic changes to python file. (#4492) 2019-03-28 10:59:29 +08:00
bibabolynn
7a9d1546d4 [java] Fix getWorker and add create multi actors test (#4472) 2019-03-26 20:26:13 +08:00
Ruifang Chen
59d74d5e92 [Java] Build Java code with Bazel (#4284) 2019-03-22 14:30:05 +08:00