hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-06 10:31:39 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
4354 commits 227 branches 67 tags 234 MiB
Commit graph

4354 commits

This branch
This branch
All branches
Author SHA1 Message Date
Eric Liang
e7bc5c612d
Add testing strategy to PR template (#7505) 2020-03-08 15:16:49 -07:00
Sven Mika
f08687f550
[RLlib] rllib train crashes when using torch PPO/PG/A2C. (#7508) 
* Fix.

* Rollback.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.

* TEST.
 2020-03-08 13:03:18 -07:00
Sven Mika
bc637a2546
[Tune Jenkins tests] Add dm_tree to docker. (#7500) 
* Fix.

* Rollback.

* Add dm_tree to docker examples and tune_test containers.
 2020-03-07 23:16:00 -08:00
Eric Liang
a644060daa
[rllib] First pass at pipeline implementation of DQN (#7433) 
* wip iters

* add test

* speed up

* update docs

* document it

* support serial sampling

* add test

* spacing

* annotate it

* update

* rename to pipeline

* comment

* iter2 wip

* update

* update

* context test

* update

* fix

* fix

* a3c pipeline

* doc

* update

* move timer

* comment

* add piepline test

* fix

* clean up

* document

* iter s

* wip dqn

* wip

* wip

* metrics

* metrics rename

* metrics ctx

* wip

* constants

* add todo

* suppport .union

* wip

* support union

* remove prints

* add todo

* remove auto timer

* fix up

* fix pipeline test

* typing

* fix breakage

* remove bad assert

* wip

* fix multiagent example

* fixapply

* update a3c

* remove a2c pl

* 0 workers

* wip

* wip

* share metrics

* wip

* wip

* doc

* fix weight sync and global var updates

* mode

* fix

* fix

* doc

* fix
 2020-03-07 14:47:58 -08:00
Landcold7
beb9b02dbd
Add numba test (#7298) (#7487) 2020-03-07 11:12:25 -08:00
Richard Liaw
115468de2c
[tune] Repeated evals (#7366) 
* easyrepeat

* done

* suggest

* doc

* ok

* commit

* Apply suggestions from code review

Co-Authored-By: Ujval Misra <misraujval@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Ujval Misra <misraujval@gmail.com>

* Apply suggestions from code review

* ok

* docs

Co-authored-by: Ujval Misra <misraujval@gmail.com>
 2020-03-07 11:08:23 -08:00
mehrdadn
a8bda9b551
Fix incorrect handling of command-lines (#7439) 2020-03-06 15:51:49 -08:00
Sven Mika
876a1ba5bd
[RLlib] Issue 7421: can't convert cuda tensor to numpy in torch ppo. (#7445) 2020-03-06 12:45:30 -08:00
Sven Mika
510c850651
[RLlib] SAC add discrete action support. (#7320) 
* Exploration API (+EpsilonGreedy sub-class).

* Exploration API (+EpsilonGreedy sub-class).

* Cleanup/LINT.

* Add `deterministic` to generic Trainer config (NOTE: this is still ignored by most Agents).

* Add `error` option to deprecation_warning().

* WIP.

* Bug fix: Get exploration-info for tf framework.
Bug fix: Properly deprecate some DQN config keys.

* WIP.

* LINT.

* WIP.

* Split PerWorkerEpsilonGreedy out of EpsilonGreedy.
Docstrings.

* Fix bug in sampler.py in case Policy has self.exploration = None

* Update rllib/agents/dqn/dqn.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* WIP.

* Update rllib/agents/trainer.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* WIP.

* Change requests.

* LINT

* In tune/utils/util.py::deep_update() Only keep deep_updat'ing if both original and value are dicts. If value is not a dict, set

* Completely obsolete syn_replay_optimizer.py's parameters schedule_max_timesteps AND beta_annealing_fraction (replaced with prioritized_replay_beta_annealing_timesteps).

* Update rllib/evaluation/worker_set.py

Co-Authored-By: Eric Liang <ekhliang@gmail.com>

* Review fixes.

* Fix default value for DQN's exploration spec.

* LINT

* Fix recursion bug (wrong parent c'tor).

* Do not pass timestep to get_exploration_info.

* Update tf_policy.py

* Fix some remaining issues with test cases and remove more deprecated DQN/APEX exploration configs.

* Bug fix tf-action-dist

* DDPG incompatibility bug fix with new DQN exploration handling (which is imported by DDPG).

* Switch off exploration when getting action probs from off-policy-estimator's policy.

* LINT

* Fix test_checkpoint_restore.py.

* Deprecate all SAC exploration (unused) configs.

* Properly use `model.last_output()` everywhere. Instead of `model._last_output`.

* WIP.

* Take out set_epsilon from multi-agent-env test (not needed, decays anyway).

* WIP.

* Trigger re-test (flaky checkpoint-restore test).

* WIP.

* WIP.

* Add test case for deterministic action sampling in PPO.

* bug fix.

* Added deterministic test cases for different Agents.

* Fix problem with TupleActions in dynamic-tf-policy.

* Separate supported_spaces tests so they can be run separately for easier debugging.

* LINT.

* Fix autoregressive_action_dist.py test case.

* Re-test.

* Fix.

* Remove duplicate py_test rule from bazel.

* LINT.

* WIP.

* WIP.

* SAC fix.

* SAC fix.

* WIP.

* WIP.

* WIP.

* FIX 2 examples tests.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Fix.

* LINT.

* Renamed test file.

* WIP.

* Add unittest.main.

* Make action_dist_class mandatory.

* fix

* FIX.

* WIP.

* WIP.

* Fix.

* Fix.

* Fix explorations test case (contextlib cannot find its own nullcontext??).

* Force torch to be installed for QMIX.

* LINT.

* Fix determine_tests_to_run.py.

* Fix determine_tests_to_run.py.

* WIP

* Add Random exploration component to tests (fixed issue with "static-graph randomness" via py_function).

* Add Random exploration component to tests (fixed issue with "static-graph randomness" via py_function).

* Rename some stuff.

* Rename some stuff.

* WIP.

* update.

* WIP.

* Gumbel Softmax Dist.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP

* WIP.

* WIP.

* Hypertune.

* Hypertune.

* Hypertune.

* Lock-in.

* Cleanup.

* LINT.

* Fix.

* Update rllib/policy/eager_tf_policy.py

Co-Authored-By: Kristian Hartikainen <kristian.hartikainen@gmail.com>

* Update rllib/agents/sac/sac_policy.py

Co-Authored-By: Kristian Hartikainen <kristian.hartikainen@gmail.com>

* Update rllib/agents/sac/sac_policy.py

Co-Authored-By: Kristian Hartikainen <kristian.hartikainen@gmail.com>

* Update rllib/models/tf/tf_action_dist.py

Co-Authored-By: Kristian Hartikainen <kristian.hartikainen@gmail.com>

* Update rllib/models/tf/tf_action_dist.py

Co-Authored-By: Kristian Hartikainen <kristian.hartikainen@gmail.com>

* Fix items from review comments.

* Add dm_tree to RLlib dependencies.

* Add dm_tree to RLlib dependencies.

* Fix DQN test cases ((Torch)Categorical).

* Fix wrong pip install.

Co-authored-by: Eric Liang <ekhliang@gmail.com>
Co-authored-by: Kristian Hartikainen <kristian.hartikainen@gmail.com>
 2020-03-06 10:37:12 -08:00
Qing Wang
7a33a6ea3c
[Java] Enable skipped direct call cases (#7363) 
* Comment out

* Refine

* Revert
 2020-03-06 16:22:08 +08:00
Stephanie Wang
7c174d0ffe
Make the ref counting test more stressful (#7473) 2020-03-05 20:51:24 -08:00
Edward Oakes
e29f2ef788
[operator] Small bugfixes (#7459) 2020-03-05 10:57:56 -08:00
Eric Liang
1989eed3bf
[RLlib] Issue 7136: rollout not working for ES and ARS. (#7444) 
* Fix.

* Fix issue #7136.

* ARS fix.
 2020-03-04 23:57:44 -08:00
Eric Liang
476b5c6196
[Parallel Iterators] Allow for operator chaining after repartition (#7268) 
* bug fix repartition

* change add_transform from private to inner

* formatting

* addressing comments

* formatting
 2020-03-04 14:42:52 -08:00
Richard Liaw
c7f0b303f3
Mention that calling some_function.remote() is non-blocking (#7417) 
* Mention that calling some_function.remote() is non-blocking.

* Apply suggestions from code review

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
 2020-03-04 13:35:46 -08:00
Richard Liaw
beddaf65b4
Small correction in documentation (#7453) 
* corrected import statement in docs

* Update doc/source/tune-usage.rst

Co-Authored-By: Richard Liaw <rliaw@berkeley.edu>

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
 2020-03-04 13:28:28 -08:00
Philipp Moritz
0d7ef46c83
Bazel improvements (#7427) 
* Make wget quiet

* Make sphinx-build quiet

* Remove -q from pip install in CI script as config already takes care of it

* Add documentation on custom dependencies

* formatting

* python
 2020-03-04 13:13:21 -08:00
Eric Liang
596b39e36a
[rllib] Make timestep a required arg for exploration classes (#7380) 2020-03-04 13:00:37 -08:00
Eric Liang
fddeb6809c
[RLlib] Issue 7401: In eval mode (if evaluation_episodes > 0), agent hangs if Env does not terminate. (#7448) 
* Fix.

* Rollback.

* Fix issue 7421.

* Fix.
 2020-03-04 12:58:34 -08:00
Eric Liang
c38224d8e5
[RLlib] Issue 7438 evaluation not working in pytorch. (#7443) 2020-03-04 12:53:04 -08:00
Philipp Moritz
de0c99876e
Fix fate_share not being passed to Redis shards (#7432) 2020-03-04 11:29:45 -08:00
Edward Oakes
0abcca258f
Add entries to in-memory store on Put() (#7085) 2020-03-04 10:17:27 -08:00
Eric Liang
aa4861c2a0
Checkpoint Adam momenta for DDPG (#7449) 2020-03-04 10:03:41 -08:00
Hao Chen
fe7820fec9
[Java] New Java actor API (#7414) 2020-03-04 22:39:23 +08:00
Sven Mika
4198db5038
Torch multicat support (7419) 2020-03-04 00:41:40 -08:00
Philipp Moritz
fb1c1e2d27
Revert "Keep cloudpickle up-to-date with the upstream (#7406)" (#7437) 
This reverts commit f6883bf725.
 2020-03-03 18:36:15 -08:00
Sven Mika
7faf0d8f89
[RLlib] Make rollout always use evaluation_config. (#7396) 2020-03-03 17:20:35 -08:00
Maksim Smolin
3a134c7224
[RaySGD] Rename PyTorch API endpoints to start with Torch (#7425) 
* Start renaming pytorch to torch

* Rename PyTorchTrainer to TorchTrainer

* Rename PyTorch runners to Torch runners

* Finish renaming API

* Rename to torch in tests

* Finish renaming docs + tests

* Run format + fix DeprecationWarning

* fix

* move tests up

* rename

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
 2020-03-03 16:44:42 -08:00
Siyuan (Ryans) Zhuang
f6883bf725
Keep cloudpickle up-to-date with the upstream (#7406) 2020-03-03 13:52:54 -08:00
Edward Oakes
b0bf5450c2
Fix flaky multiprocessing tests (#7413) 2020-03-03 15:07:59 -06:00
ijrsvt
fb76092d75
Re-route asyncio plasma code path through raylet instead of direct plasma connection (#7234) 2020-03-03 15:43:46 -05:00
Philipp Moritz
c2c6d96490
Fix install documentation on readthedocs (#7423) 2020-03-03 11:03:18 -08:00
Edward Oakes
04ec599441
Use ray.kill() in multiprocessing.Pool (#7409) 2020-03-03 12:49:13 -06:00
Allen
b74eb5fce6
Capture output for commands run by the autoscaler (#7381) 2020-03-03 10:19:21 -08:00
mehrdadn
4d42664b2a
Use prctl(PR_SET_PDEATHSIG) on Linux instead of reaper (#7150) 2020-03-03 11:45:42 -06:00
fangfengbin
f5b1062ed9
Fix TwoNodeTest.TestActorTaskCrossNodes testcase when enable gcs service (#7416) 2020-03-03 19:37:38 +08:00
ijrsvt
584645cc7d
Fix Experimental Async API (#7391) 2020-03-02 22:24:20 -06:00
Edward Oakes
580b017b43
Fix flaky global GC tests (#7407) 2020-03-02 21:03:01 -06:00
Edward Oakes
9e9f1962c7
Enable test_actor_pool in CI (#7405) 2020-03-02 20:24:36 -06:00
Edward Oakes
2b6f00724a
Enable test_joblib in CI (#7404) 2020-03-02 20:03:27 -06:00
Edward Oakes
d69fe54f6d
Temporarily skip testEndToEndReporting (#7402) 2020-03-02 18:27:34 -06:00
Eric Liang
0f88444686
[rllib] Support multi-agent training in pipeline impls, add easy flag to enable (#7338) 2020-03-02 15:16:37 -08:00
Sven Mika
d8eeb96413
Fix issue with torch PPO not handling action spaces of shape=(>1,). (#7398) 2020-03-02 10:53:19 -08:00
Qing Wang
2771af1036
Fix the bug of unregistered workers in worker pool (#7343) 
* Fix

* Fix

* Fix complie

* Fix lint

* Fix linting

* Fix testDeleteObject

* Fix linting

* Update src/ray/raylet/worker_pool.cc

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update src/ray/raylet/worker_pool.cc

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update src/ray/raylet/worker_pool.h

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Update src/ray/raylet/worker_pool.cc

Co-Authored-By: Hao Chen <chenh1024@gmail.com>

* Address comments.

* FIx linting

Co-authored-by: Hao Chen <chenh1024@gmail.com>
 2020-03-02 16:30:39 +08:00
Siyuan (Ryans) Zhuang
0792b5cb93
Fix the numpy ndarray subclass serialization bug (#7392) 2020-03-01 23:05:59 -08:00
Richard Liaw
48cdca843f
[raysgd] Custom training operator (#7211) 2020-03-01 21:22:48 -08:00
Sven Mika
2d97650b1e
[RLlib] Add Exploration API documentation. (#7373) 
* Add Exploration API documentation.

* Add Exploration API documentation.

* Add Exploration API documentation.

* Update exporation docs.
 2020-03-01 16:55:41 -08:00
mehrdadn
44aded5272
Bazel mirrors (#7385) 
* Switch to mirrors.bazel.build where possible

* Switch from .zip to .tar.gz for smaller downloads (it's also the default download on UNIX)

* Use direct GitHub URLs in Bazel files for clarity

* Don't pass patches to local_repository

* Remove github_repository()

* Switch to GitHub actions/checkout@v2 which is faster

* Use faster extraction method for LLVm on Windows

* Move LLVM_VERSION_WINDOWS to the shell script since it's not a CI-specific value

* Change GITHUB_TOKEN to GITHUB

* Don't show timestamps for GitHub Actions

* Factor out some options from GitHub Actions

* Tell Bazel to stay on the same volume in GitHun Actions

* Display progress output when downloading toolchains

Co-authored-by: GitHub Web Flow <noreply@github.com>
 2020-03-01 14:04:06 -08:00
Sven Mika
83e06cd30a
[RLlib] DDPG refactor and Exploration API action noise classes. (#7314) 
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Fix

* WIP.

* Add TD3 quick Pendulum regresison.

* Cleanup.

* Fix.

* LINT.

* Fix.

* Sort quick_learning test cases, add TD3.

* Sort quick_learning test cases, add TD3.

* Revert test_checkpoint_restore.py (debugging) changes.

* Fix old soft_q settings in documentation and test configs.

* More doc fixes.

* Fix test case.

* Fix test case.

* Lower test load.

* WIP.
 2020-03-01 11:53:35 -08:00
Eric Liang
3c6b94f3f5
[rllib] Enable performance metrics reporting for RLlib pipelines, add A3C (#7299) 2020-02-28 16:44:17 -08:00