Commit graph

15 commits

Author SHA1 Message Date
Kai Yang
ed761900f6 [Java] Support direct actor call in Java worker (#5504) 2019-09-09 14:29:20 +08:00
Kai Yang
806524384b [Java worker] Refactor object store and worker context on top of core worker (#5079) 2019-07-16 20:58:02 +08:00
Hao Chen
8a30b93e42
Define common data structures with protobuf. (#5121) 2019-07-08 22:41:37 +08:00
Hao Chen
0131353d42 [gRPC] Migrate gcs data structures to protobuf (#5024) 2019-06-25 14:31:19 -07:00
Qing Wang
671c0f769e
[Java] Fix serializing issues of RaySerializer (#4887)
* Fix

* Address comment.
2019-06-08 22:56:00 +08:00
Hao Chen
ffd596d5bb
Fix pom file generation (#4800) 2019-05-17 10:56:39 +08:00
Hao Chen
643f62dc43
[Java][Bazel] Refine auto-generated pom files (#4780) 2019-05-16 11:19:31 +08:00
bibabolynn
ab55a1f93a [Java] Clean up outdated dependencies (#4489) 2019-03-28 14:33:45 +08:00
Hao Chen
f8d12b0418
[Java] Package native dependencies into jar (#4367) 2019-03-15 12:38:40 +08:00
Wang Qing
e1c68a0881 Enable including Java worker for ray start command (#3838) 2019-02-04 16:23:43 +08:00
Wang Qing
3cf59855af [Java] Replace junit with testNG (#3768) 2019-01-14 17:49:17 +08:00
Wang Qing
8e8e123777 [Java] Simplify Java worker configuration (#2938)
## What do these changes do?
Previously, Java worker configuration is complicated, because it requires setting environment variables as well as command-line arguments.

This PR aims to simplify Java worker's configuration. 
1) Configuration management is now migrated to [lightbend config](https://github.com/lightbend/config), thus doesn't require setting environment variables.
2) Many unused config items are removed.
3) Provide a simple `example.conf` file, so users can get started quickly.
4) All possible options and their default values are declared and documented in `ray.default.conf` file.

This PR also simplifies and refines the following code:
1) The process of `Ray.init()`.
2) `RunManager`.
3) `WorkerContext`. 

### How to use this configuration?
1. Copy `example.conf` into your classpath and rename it to `ray.conf`.
2. Modify/add your configuration items. The all items are declared in `ray.default.conf`.
3. You can also set the items in java system prosperities.

Note: configuration is read in this priority:
System properties > `ray.conf` > `ray.default.conf`

## Related issue number
N/A
2018-09-26 20:14:22 +08:00
Wang Qing
0e552fbb22 [Java] Update maven version to 0.1-SNAPSHOT
Update the version in maven from 0.1 to 0.1-SNAPSHOT, because SNAPSHOT is the conventional version name in dev process. Non-snapshot versions are only used for release.
2018-09-26 18:08:46 +08:00
Hao Chen
971df5ea8a [java] put function meta in task spec and load functions with function meta (#2881)
This PR adds a `function_desc` field into task spec. a function descriptor is a list of strings that can uniquely describe a function.
- For a Python function, it should be: [module_name, class_name, function_name]
- For a Java function, it should be: [class_name, method_name, type_descriptor]

There're a couple of purposes to add this field:

In this PR:
- Java worker needs to know function's class name to load it. Previously, since task spec didn't have such a field to hold this info, we did a hack by appending the class name to the argument list. With this change, we fixed that hack and significantly simplified function management in Java.

Will be done in subsequent PRs:
- Support cross-language invocation (#2576): currently Python worker manages functions by saving them in GCS and pass function id in task spec. However, if we want to call a Python function from Java, we cannot save it in GCS and get the function id. But instead, we can pass the function descriptor (module name, class name, function name) in task spec and use it to load the function.
- Support deployment: one major problem of Python worker's current function management mechanism is #2327. In prod env, we should have a mechanism to deploy code and dependencies to the cluster. And when code is already deployed, we don't need to save functions to GCS any more and can use `function_desc` to manage functions.
2018-09-25 23:05:05 -07:00
Hao Chen
8414e413a2 [java] refine and simplify java worker code structure (#2838) 2018-09-10 10:48:17 -07:00
Renamed from java/common/pom.xml (Browse further)