Commit graph

262 commits

Author SHA1 Message Date
chaokunyang
cf3875bd8c
[Java] add exitActor API for java (#10496) 2020-09-04 10:11:42 +08:00
chaokunyang
5e4db6ad24
[Java] add default kill option (#10473) 2020-09-04 10:08:52 +08:00
Clark Zinzow
0c0b0d0a73
[Core] Added support for submission-time task names. (#10449)
* Added support for submission-time task names.

* Suggestions from code review: add missing consts

Co-authored-by: SangBin Cho <rkooo567@gmail.com>

* Add num_returns arg to actor method options docstring example.

* Add process name line and proctitle assertion to submission-time task name section of advanced docs.

* Add submission-time task name --> proctitle test for Python worker.

* Added Python actor options tests for num_returns and name.

* Added Java test for submission-time task names.

* Add dashboard image to task name docs section.

* Move to fstrings.

Co-authored-by: SangBin Cho <rkooo567@gmail.com>
2020-09-03 11:45:24 -07:00
chaokunyang
ea95e6f7cc
[Java] lint java code (#10494) 2020-09-03 10:39:14 +08:00
fyrestone
b04222dbd9
[xlang] Cross language serialization for ActorHandle (#10335) 2020-09-02 10:11:53 +08:00
Lixin Wei
6bde6b493e
[Streaming]Format java code using IDEA (#10440) 2020-08-31 19:45:00 +08:00
Stephanie Wang
f75dfd60a3
[api] API deprecations and cleanups for 1.0 (internal_config and Checkpointable actor) (#10333)
* remove

* internal config updates, remove Checkpointable

* Lower object timeout default

* remove json

* Fix flaky test

* Fix unit test
2020-08-27 10:19:53 -07:00
fyrestone
08adbb371f
Cross language exception (#10023) 2020-08-26 10:46:05 +08:00
fangfengbin
36c6c4b298
[Placement group] Check if placement group bundle index is valid (#10194)
* add part code

* rebase master

* add java testcase

* fix review comments

* fix lint error

* rebase master

* fix lint error

Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-08-21 11:04:56 -07:00
Kai Yang
3bc17fa62a
[Core] Multi-tenancy: Pass env variables from job config to worker processes (#10022) 2020-08-10 14:31:37 -07:00
Kai Yang
27cd323ce1
[Core] Multi-tenancy: Job isolation & implement per job config (except for env variables) (#9500) 2020-08-04 15:51:29 +08:00
chaokunyang
64d6446cf3
change version from 0.1-SNAPSHOT to 0.9.0-SNAPSHOT (#9778) 2020-08-01 22:38:22 +08:00
Kai Yang
006e034cdb
fix lint for ReferenceCountingTest.java (#9837) 2020-07-31 17:00:00 +08:00
bermaker
88e8714bcb
Fix ray java worker metric test indentation (#9834) 2020-07-31 14:39:41 +08:00
Kai Yang
02fd950252
[Java] Local and distributed ref counting in Java (#9371) 2020-07-31 11:49:31 +08:00
chaokunyang
6464bf55c6
[dist] Mvn deploy (#9777) 2020-07-30 11:48:31 +08:00
Hao Chen
260bc52254
Java doc: "Ray Core Walkthrough" page (#8595) 2020-07-30 11:13:38 +08:00
Kai Yang
2cafc7cebe
[Java] Fix MetricTest.java due to incomplete changes from #9703 (#9770) 2020-07-29 12:18:17 +08:00
Kai Yang
bdc005a4d4
[Java] Use test groups to filter tests of different run modes (#9703) 2020-07-29 11:18:45 +08:00
bermaker
6e23aff723
[Metrics]Ray java worker metric registry (#9636)
* ray worker metrics gauge init

* ray java metric mapping

* add jni source files for gauge and tagkey

* mapping all metric classes to stats object

* check non-null for tags and name

* lint

* add symbol for native metric JNI

* extern c for symbol

* add tests for all metrics

* Update Metric.java

use metricNativePointer instead.

* unify metric native stuff to one class

* fix jni file

* add comments for metric transform function in jni utils

* move metric function to native metric file

* remove unused disconnect jni

* Add a metric registry for java metircs

* Restore install-bazel.sh

* Add some comments for metric registry

* Fix thread safe problem of metrics

* Fix metric tests and remove sleep code from tests

* Fix comments of metrics

Co-authored-by: lingxuan.zlx <skyzlxuan@gmail.com>
2020-07-28 21:29:33 +08:00
fangfengbin
28d5f9696d
Add placement group java api (#9611)
* add part code

* add part code

* add part code

* fix code style

* fix review comment

* fix review comment

* add part code

* add part code

* add part code

* add part code

* fix review comment

* fix review comment

* fix code style

* fix review comment

* fix lint error

* fix lint error

Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-07-25 15:39:05 +08:00
fangfengbin
01d6edae9c
fix named actor single process mode bug (#9652) 2020-07-23 20:28:50 +08:00
Kai Yang
bfa0605282
[Java] Avoid data copy from C++ to Java for ByteBuffer type (#9033) 2020-07-22 16:25:32 +08:00
Lingxuan Zuo
cd42450fc1
[Metrics] Java metric API (#9377) 2020-07-22 10:35:08 +08:00
chaokunyang
9318e76b81
[Java] Named java actor (#9037) 2020-07-16 11:31:18 +08:00
chaokunyang
b5fafe1fce
[Java] fix java test (#9079) 2020-06-22 16:56:47 +08:00
chaokunyang
cb6f337372
[Java] Refine python function (#8943) 2020-06-16 16:22:49 +08:00
chaokunyang
dfa4768fc6
[Java] Refactor java api (#8858) 2020-06-12 10:49:01 +08:00
chaokunyang
700d81fa20
[Java] Remove java api sub package from test module (#8853) 2020-06-11 14:59:45 +08:00
chaokunyang
31a4d07bc4
[Java] Rename java ObjectRef/ActorHandle (#8799) 2020-06-09 11:40:43 +08:00
Hao Chen
dc3e98890f
Improve Java logging (#8640) 2020-06-02 16:15:20 +08:00
Edward Oakes
a76434ccde
Add ability to specify worker and driver ports (#8071) 2020-05-20 15:31:13 -05:00
Edward Oakes
16f48078d9
Remove use of ObjectID transport flag (#7699) 2020-05-17 11:29:49 -05:00
Max Fitton
00325eb2b2
Rename max_reconstructions to max_restarts and use -1 for infinite (#8274)
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
2020-05-14 10:30:29 -05:00
Kai Yang
1d5bceddf0
fix java UT about multi-threading (#8014) 2020-04-27 15:11:22 +08:00
Qing Wang
98bfcd53bc
[Java] Rename group id and package name. (#7864)
* Initial

* Change streaming's

* Fix

* Fix

* Fix org_ray

* Fix cpp file name

* Fix streaming

* Fix

* Fix

* Fix testlistening

* Fix missing sth in python

* Fix

* Fix

* Fix SPI

* Fix

* Fix complation

* Fix

* Fix CI

* Fix checkstyle

Fix checkstyle

* Fix streaming tests

* Fix streaming CI

* Fix streaming checkstyle.

* Fix build

* Fix bazel dep

* Fix

* Fix ray checkstyle

* Fix streaming checkstyle

* Fix bazel checkstyle
2020-04-12 17:59:34 +08:00
fyrestone
fc6259a656
Cross language serialization for primitive types (#7711)
* Cross language serialization for Java and Python

* Use strict types when Python serializing

* Handle recursive objects in Python; Pin msgpack >= 0.6.0, < 1.0.0

* Disable gc for optimizing msgpack loads

* Fix merge bug

* Java call Python use returnType; Fix ClassLoaderTest

* Fix RayMethodsTest

* Fix checkstyle

* Fix lint

* prepare_args raises exception if try to transfer a non-deserializable object to another language

* Fix CrossLanguageInvocationTest.java, Python msgpack treat float as double

* Minor fixes

* Fix compile error on linux

* Fix lint in java/BUILD.bazel

* Fix test_failure

* Fix lint

* Class<?> to Class<T>; Refine metadata bytes.

* Rename FST to Fst; sort java dependencies

* Change Class<?>[] to Optional<Class<?>>; sort requirements in setup.py

* Improve CrossLanguageInvocationTest

* Refactor MessagePackSerializer.java

* Refactor MessagePackSerializer.java; Refine CrossLanguageInvocationTest.java

* Remove unnecessary dependencies for Java; Add getReturnType() for RayFunction in Java

* Fix bug

* Remove custom cross language type support

* Replace Serializer.Meta with MutableBoolean

* Remove @SuppressWarnings support from checkstyle.xml; Add null test in CrossLanguageInvocationTest.java

* Refine MessagePackSerializer.pack

* Ray.get support RayObject as input

* Improve comments and error info

* Remove classLoader argument from serializer

* Separate msgpack from pickle5 in Python

* Pair<byte[], MutableBoolean> to Pair<byte[], Boolean>

* Remove public static <T> T get(RayObject<T> object), use RayObject.get() instead

* Refine test

* small fixes

Co-authored-by: 刘宝 <po.lb@antfin.com>
Co-authored-by: Hao Chen <chenh1024@gmail.com>
2020-04-08 21:10:57 +08:00
Kai Yang
48b48cc8c2
Support multiple core workers in one process (#7623) 2020-04-07 11:01:47 +08:00
Robert Nishihara
ee8c9ff732
Remove six and cloudpickle from setup.py. (#7694) 2020-03-23 11:42:05 -07:00
fyrestone
a1ae935839
Java call Python use structured function descriptors (#7634) 2020-03-20 17:29:45 +08:00
Kai Yang
6b888b0247
[Java] Make both RayActor and RayPyActor inheriting from BaseActor (#7462) 2020-03-17 21:45:56 +08:00
fyrestone
7697ea2be2
Java call Python actor method use actor.call (#7614) 2020-03-17 14:52:43 +08:00
Kai Yang
630e48967d
[Java] Allow passing internal config from raylet to Java worker (#7532) 2020-03-15 12:03:38 +08:00
Qing Wang
d6365c2586
[Java] Enable stress test. (#7596) 2020-03-13 21:02:13 +08:00
Kai Yang
d6e8f47065
Add a flag to disable reconstruction for a killed actor (#7346) 2020-03-13 19:10:21 +08:00
Qing Wang
f4656d8cc3
[Java] Enable direct call by default. (#7408)
* WIP

* Address comments.

* Linting

* Fix

* Fix

* Fix test

* Fix

* Fix single process ci

* Fix ut

* Update java/test/src/main/java/org/ray/api/test/PlasmaFreeTest.java

* Address comments

* Fix linting

* Minor update comments.

* Fix streaming CI
2020-03-13 12:25:30 +08:00
Kai Yang
932a749fa9
Fix the java_worker_options parameter (#7537)
* fix Java CI

* Minor fix

* move json.loads out of build_java_worker_command

* lint

* fix cross language test
2020-03-12 10:44:23 +08:00
Stephanie Wang
fdb528514b
[core] Ref counting for actor handles (#7434)
* tmp

* Move Exit handler into CoreWorker, exit once owner's ref count goes to 0

* fix build

* Remove __ray_terminate__ and add test case for distributed ref counting

* lint

* Remove unused

* Fixes for detached actor, duplicate actor handles

* Remove unused

* Remove creation return ID

* Remove ObjectIDs from python, set references in CoreWorker

* Fix crash

* Fix memory crash

* Fix tests

* fix

* fixes

* fix tests

* fix java build

* fix build

* fix

* check status

* check status
2020-03-10 17:45:07 -07:00
fangfengbin
fa785a2ad2
ServiceBasedGcsClient support detect gcs server availability and retry (#7292) 2020-03-10 21:01:07 +08:00
Qing Wang
7a33a6ea3c
[Java] Enable skipped direct call cases (#7363)
* Comment out

* Refine

* Revert
2020-03-06 16:22:08 +08:00