hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-06 02:21:39 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
13823 commits 227 branches 67 tags 234 MiB
Commit graph

344 commits

Author SHA1 Message Date
Qing Wang
84f7062329
[Java] Cleanup temp file of libcore_worker.so (#20748) 
Why are these changes needed?
Replace the existing temp file to avoid the issue that the previous worker dies and leaves the temp file there, resulting in the next coming workers are not able to write a new temp file since there is an existing one.
 2021-11-29 16:05:06 +08:00
Guyang Song
53630ee03b
Revert "Revert "[runtime env] redefine runtime env to protobuf"" and fix windows compiling (#20692) 
- Fix windows compiling and revert https://github.com/ray-project/ray/pull/20641
- Seems the pr https://github.com/ray-project/ray/pull/20670 can solve the windows compiling issue.
 2021-11-24 09:01:01 -08:00
Alex Wu
9388d28233
Revert "[runtime env] redefine runtime env to protobuf" (#20641) 
Reverts #19511

Breaks windows compilation
 2021-11-22 13:11:30 -08:00
Guyang Song
ad56b9b432
[runtime env] redefine runtime env to protobuf (#19511) 2021-11-20 16:54:42 +08:00
Larry
454db6902c
[Java] Add timeout parameter for Ray.get() API (#20282) 
Why are these changes needed?

Add timeout(ms) param for Java ray.get. The API changes have been updated to doc ([Ray Core Walkthrough]->[Fetching Results]).

eg:
ObjectRef<Integer> objRef = Ray.put(1);
objRef.get(1000) 
Ray.get(Ray.task(MyRayApp::slowFunction).remote(), 3000)

Related issue number
#20247
 2021-11-17 11:02:17 +08:00
Yi Cheng
a4e187c0e7
[gcs] Update function table to use internal kv (#20152) 
## Why are these changes needed?
This is a part of redis removal. This PR remove redis kv in function table. 
rpush related code is not updated in this PR.

## Related issue number
 2021-11-15 23:34:41 -08:00
Qing Wang
1172195571
[Java] Remove global named actor and global pg (#20135) 
This PR removes global named actor and global PGs.

I believe these APIs are not used widely in OSS.
CPP part is not included in this PR.
@kfstorm @clay4444 @raulchen Please take a look if this change is reasonable.


IMPORTANT NOTE: This is a Java API change and will lead backward incompatibility in Java global named actor and global PG usage.

CPP part is not included in this PR.
INCLUDES:

 Remove setGlobalName() and getGlobalActor() APIs.
 Remove getGlobalPlacementGroup() and setGlobalPG
 Add getActor(name, namespace) API
 Add getPlacementGroup(name, namespace) API
 Update doc pages.
 2021-11-15 16:28:53 +08:00
Qing Wang
7500f7d88a
Remove deprecated Java PG APIs. (#20219) 
These APIs were deprecated at least 7+ months and 4+ versions, it's the time and very necessary to remove them.
 2021-11-12 09:29:48 +08:00
Yi Cheng
e54d3117a4
[gcs] Update all redis kv usage in python except function table (#20014) 
## Why are these changes needed?
This is part of redis removal project. In this PR all direct usage of redis got removed except function table.
Function table will be migrated in the next PR

## Related issue number
#19443
 2021-11-10 20:24:53 -08:00
Stephanie Wang
ffcc5935d7
[core] Evict lineage to bound memory usage (#19946) 
* bound lineage

* Bound lineage in bytes

* test

* Lineage evicted error

* Lineage evicted

* lint

* test

* test

* comment

* doc

* x

* x

* x

* x
 2021-11-08 21:53:40 -08:00
Qing Wang
6d8a7291ab
Add getNamespace API for Java worker (#20057) 
[Java API] Add getNamespace API for Java worker.
 2021-11-08 15:51:14 +08:00
Qing Wang
4373aa1e3b
Support generating a UUID string as the anonymous namespace for Java worker. (#19986) 
Why are these changes needed?
For Java worker, we generate a UUID string as the namespace if a job is not specified a namespace by user.

Related issue number
#16474
 2021-11-04 11:40:17 +08:00
Jiajun Yao
5de4a38948
[CI] Run Java CI on Mac (#19757) 
Why are these changes needed?
Enable Java tests on Mac CI to avoid more breakages.

Related issue number
Closes #19700
 2021-11-03 23:40:05 +08:00
Qing Wang
da6894848d
Support Java namespace APIs (#19468) 
## Why are these changes needed?

## Related issue number
#16474
 2021-11-02 11:05:40 +08:00
Tao Wang
7a2e9e00e8
[Tiny]Remove duplicated assignment (#19866) 2021-11-01 11:44:01 +08:00
Jiajun Yao
e4542be0d1
[Java] Run java on mac with public ip (#19701) 2021-10-25 11:38:33 -07:00
Jiajun Yao
805ce453dd
[Java] Remove auto-generated pom.xml files. (#19475) 2021-10-19 17:35:37 +08:00
Gagandeep Singh
d226cbf21a
Added StartupToken to idenitfy a process at startup (#19014) 
* Added StartupToken to idenitfy a process at startup

* Applied linting formats

* Addressed reviews

* Fixing worker_pool_test

* Fixed worker_pool_test

* Applied linting formatting

* Added documentation for StartupToken

* Fixed linting

* Reordered initialisation of WorkerPool members

* Fixed Python docs

* Fixing bugs in cluster_mode_test

* Fixing Java tests

* Create and set shim process after verifying startup_token

* shim_process.GetId() -> worker_shim_pid

* Improvements in startup token and modifying java files

* update io_ray_runtime_RayNativeRuntime.h

* Fixed java tests by adding startup-token to conf

* Applied linting

* Increased arg count for startup_token

* Attempt to fix streaming tests

* Type correction

* applied linting

* Corrected index of startup token arg

* Modified, mock_worker.cc to accept startup tokens

* Applied linting

* Applied linting changes from CI

* Removed override from worker.h

* Applied linting from scripts/format.sh

* Addressed reviews and applied scripts/format.sh

* Applied linting script from ci/travis

* Removed unrequired methods from public scope

* Applied linting
 2021-10-15 15:13:13 -07:00
Qing Wang
2cc164e616
[Java] Fix incompleted core worker dynamic library. (#19342) 
* Fix incompleted core worker dynamic library.

* Fix lint.
 2021-10-14 14:42:05 +08:00
hazeone
c2f0035fd2
[Java]Support getGpuIds API (#19031) 
Add java getGpuIds() API which is the same as get_gpu_ids in python. We can get deviceId if we've allocated a GPU to a worker.
 2021-10-13 23:40:26 +08:00
Guyang Song
ab55b808c5
[runtime env] move worker env to runtime env in Java (#19060) 2021-10-11 17:25:09 +08:00
Stephanie Wang
940f84cedb
[core] Remove unused plasma promotion path (#19122) 
* remove unused

* lint

* lint

* lint
 2021-10-07 10:55:50 -07:00
Qing Wang
90d2456ec7
[Java] Support userloggers. (#18846) 
Co-authored-by: Kai Yang <kfstorm@outlook.com>
 2021-09-26 16:53:06 +08:00
Stephanie Wang
284dee493e
[core][usability] Disambiguate ObjectLostErrors for better understandability (#18292) 
* Define error types, throw error for ObjectReleased

* x

* Disambiguate OBJECT_UNRECONSTRUCTABLE and OBJECT_LOST

* OwnerDiedError

* fix test

* x

* ObjectReconstructionFailed

* ObjectReconstructionFailed

* x

* x

* print owner addr

* str

* doc

* rename

* x
 2021-09-13 16:16:17 -07:00
Guyang Song
3bc5f0501f
fix WaitPlacementGroupReady API (#18464) 2021-09-13 14:07:40 +08:00
Qing Wang
371f03fa48
Remove dynamic resource from client side. (#18514) 2021-09-11 10:39:59 -07:00
Qing Wang
d87441cda7
[Java] ConcurrencyGroup in Java local mode. (#18241) 
* WIP

* Fix

* Fix test

* Refine

* Fix lint,

* WIP2

* WIP2

* Refine

* Put a default concurrency group.

* Fix submitting task with concurrency group name.

* Remove unnecessary changes.

* Update java/runtime/src/main/java/io/ray/runtime/task/LocalModeTaskSubmitter.java

Co-authored-by: Kai Yang <kfstorm@outlook.com>

Co-authored-by: Kai Yang <kfstorm@outlook.com>
 2021-09-07 20:43:31 +08:00
Zhi Lin
d3786ac131
Bump Java version to 2.0.0-SNAPSHOT (#15394) 
* bump java version to 2.0.0-SNAPSHOT

* update
 2021-08-30 12:25:30 +08:00
Lingxuan Zuo
f2a3085ce2
[Metric]Java metric api enhancement (#17811) 
* Java metric api enhancement:
make tagkey transparent for upper level users

* add java metric tags test

* mark Deprecated
 2021-08-16 22:38:27 +08:00
Qing Wang
9d5c68ff55
[Java] Better log message when failed to invoke task. (#17737) 2021-08-13 17:31:58 +08:00
Kai Yang
ab53c5fc93
[Java] Update rolling logging configuration (#17741) 2021-08-12 10:15:27 +08:00
Qing Wang
6d6a1ea43e
Support reading system configs from native in Java. (#17703) 
* Support reading system configs from native in Java.

* Fix lint

* Lint cpp

* Fix Java cases.

* Address comments.

* Address comments.
 2021-08-12 10:06:01 +08:00
Qing Wang
4cc34588db
[Core] Support ConcurrentGroup part1 (#16795) 
* Core change and Java change.

* Fix void call.

* Address comments and fix cases.

* Fix asyncio
 2021-08-07 22:41:33 +08:00
Kai Yang
9b3c0ad35b
Fix “argument type mismatch” when an exception occurs in chained tasks (#17636) 2021-08-07 17:47:43 +08:00
Zhi Lin
82123123c4
[object store] Java API for Assign the object owner in Ray.put() (#17237) 
Co-authored-by: Qing Wang <kingchin1218@126.com>
Co-authored-by: Kai Yang <kfstorm@outlook.com>
 2021-08-06 15:26:59 +08:00
Qing Wang
b8baac3cb0
[Java] Filter error log for intentional system exit. (#17289) 2021-07-30 13:17:38 +08:00
Tao Wang
90187433b1
[Java] Remove redis dependency(jedis) in java lang layer (#17029) 2021-07-13 17:34:10 +08:00
Tao Wang
5b7e76770d
[Java] Use gcs client instead of redis client to get session dir (#16773) 
* Use gcs client instead of redis client to get session dir

* fix compile and add comments

* fix compile

* lint

* fix

* lint

* lint

* Update src/ray/gcs/gcs_client/global_state_accessor.h

Co-authored-by: Qing Wang <kingchin1218@126.com>

* Update java/runtime/src/main/java/io/ray/runtime/RayNativeRuntime.java

Co-authored-by: Qing Wang <kingchin1218@126.com>

* per comment

Co-authored-by: Qing Wang <kingchin1218@126.com>
 2021-07-13 14:01:22 +08:00
Qing Wang
4bde71ca86
[Java][Core] Support get current actor handle. (#14900) 2021-07-12 15:27:54 -07:00
Tao Wang
eed0ffc6ff
[Core]Align storage of session_dir in java/python so it can be accessed u… (#16958) 
* Align storage of session_dir in java/python so they can be accessed using internal kv manager

* align cpp
 2021-07-12 17:42:13 +08:00
Kai Yang
e925051ce4
[Core] Get node to connect for driver in global state accessor (#16810) 2021-07-08 11:21:12 +08:00
Qing Wang
b9a3ab73b6
Fix Java driver log error (#16832) 
* Fix

* Silence initial logs.
 2021-07-03 11:53:42 +08:00
Qing Wang
89b07572da
[Java] Upgrade log4j (#16657) 2021-06-24 21:01:27 -07:00
Qing Wang
3272997b0d
[Java] Upgrade some deps to fix CVEs (#16650) 2021-06-24 10:56:20 -07:00
Tao Wang
2523072a3d
[large scale]Use gcs client instead of redis client to increase job id (#16190) 
Co-authored-by: Alex Wu <itswu.alex@gmail.com>
 2021-06-17 15:01:32 +08:00
Qing Wang
9a58e6a2e0
Remove unintended introduced words in comment. (#16367) 2021-06-11 13:35:42 +08:00
Qing Wang
d6d27e9d34
[Java] Enable concurrent calls in local mode. (#14896) 
* Enable concurrent calls in local mode.

* Fix submitting actor tasks before actor creation task executed.

Co-authored-by: Qing Wang <jovany.wq@antgroup.com>
 2021-06-10 23:21:11 +08:00
mwtian
dce13d3a81
Explicitly set protobuf dependency version to allow building ray with bazel 4.0.0 (#15756) 
Java protobuf dependency version is made to be consistent as well.
 2021-05-13 10:34:09 -07:00
Kai Yang
b0ea947fa3
[Java] Support getCurrentActorId in local mode (#14890) 2021-03-31 21:39:39 +08:00
Kai Yang
6278df8604
[Java] refine generation of jvm options (#14931) 2021-03-31 21:04:52 +08:00