Tanay Wakhare
15aa08a3d1
[RLLib] WindowStat bug fix ( #9213 )
...
* WindowStat error catching, which processes NaNs properly instead of erroring. This ought to resolve issue #7910 .
https://github.com/ray-project/ray/issues/7910
2020-07-12 23:01:32 +02:00
Sven Mika
fcdf410ae1
[RLlib] Tf2.x native. ( #8752 )
2020-07-11 22:06:35 +02:00
Hao Chen
d49dadf891
Change Python's ObjectID to ObjectRef ( #9353 )
2020-07-10 17:49:04 +08:00
Sven Mika
01125b8fcf
[RLlib] DQN rainbow eager-mode (keras style NoisyLayer) (preparation for native tf2.x support). ( #9304 )
2020-07-09 10:44:10 +02:00
Sven Mika
4da0e542d5
[RLlib] DDPG and SAC eager support (preparation for tf2.x) ( #9204 )
2020-07-08 16:12:20 +02:00
Sven Mika
f43d934817
[RLlib] Type annotations for policy. ( #9248 )
2020-07-05 13:09:51 +02:00
Eric Liang
4b62a888cc
[rllib] Remove deprecated policy optimizer package. ( #9262 )
2020-07-02 14:39:40 -07:00
Sven Mika
5b2a97597b
[RLlib] Retire try_import_tree (should be installed along with other requirements). ( #9211 )
...
- Retire try_import_tree.
- Stabilize test_supported_multi_agent.py.
2020-07-02 13:06:34 +02:00
Sven Mika
43043ee4d5
[RLlib] Tf2x preparation; part 2 (upgrading try_import_tf()). ( #9136 )
...
* WIP.
* Fixes.
* LINT.
* WIP.
* WIP.
* Fixes.
* Fixes.
* Fixes.
* Fixes.
* WIP.
* Fixes.
* Test
* Fix.
* Fixes and LINT.
* Fixes and LINT.
* LINT.
2020-06-30 10:13:20 +02:00
Sven Mika
5c6d5d4ab1
This PR fixes the currently broken lstm_use_prev_action_reward flag for default lstm models (model.use_lstm=True). ( #8970 )
2020-06-27 20:50:01 +02:00
Sven Mika
4fd8977eaf
[RLlib] Minor cleanup in preparation to tf2.x support. ( #9130 )
...
* WIP.
* Fixes.
* LINT.
* Fixes.
* Fixes and LINT.
* WIP.
2020-06-25 19:01:32 +02:00
Tanay Wakhare
f77c638d6d
Pytorch AttentionNet ( #9088 )
2020-06-23 20:42:30 +02:00
Eric Liang
1e0e1a45e6
[rllib] Add type annotations for evaluation/, env/ packages ( #9003 )
2020-06-19 13:09:05 -07:00
Joseph Suarez
c6ee3cdff4
Refactor #8792 to integrate latest master ( #8956 )
2020-06-17 10:55:52 +02:00
Sven Mika
7008902cff
[RLlib] Minor rllib.utils cleanup. ( #8932 )
2020-06-16 08:52:20 +02:00
Sven Mika
4ed796a7d6
[RLlib] Add testing Policy.compute_single_action() for all agents. ( #8903 )
2020-06-13 17:51:50 +02:00
Eric Liang
34bae27ac7
[rllib] Flexible multi-agent replay modes and replay_sequence_length ( #8893 )
2020-06-12 20:17:27 -07:00
Kristian Holsheimer
ea965d7c52
[RLlib] use Mapping instead of dict in summarize() to accommodate non-dict grads/params (e.g. haiku's frozendict) ( #8793 )
2020-06-11 00:37:15 -07:00
Eric Liang
831b2fe51d
[rllib] Set framework to tf by default and remove import checks; "Auto" option ( #8748 )
...
* tf by default
* Update rllib/agents/trainer.py
Co-authored-by: Sven Mika <sven@anyscale.io>
* remove it
* fix
* remove
* fix
* lint
Co-authored-by: Sven Mika <sven@anyscale.io>
2020-06-08 23:04:50 -07:00
mehrdadn
f93bb008bb
Change os.uname()[1] and socket.gethostname() to the portable and faster platform.node_ip() ( #8839 )
...
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-08 21:29:46 -07:00
Eric Liang
be26a7b1b0
[rllib] Support for complex / variable-length observation spaces ( #8393 )
2020-06-06 12:22:19 +02:00
Sven Mika
25c0974543
[RLlib] Issue 8412 (Adam vars not stored in ModelV2). ( #8480 )
2020-06-05 21:07:02 +02:00
Sven Mika
c74dc58f8b
[RLlib] Fix use_lstm flag for ModelV2 (w/o ModelV1 wrapping) and add it for PyTorch. ( #8734 )
2020-06-05 15:40:30 +02:00
Eric Liang
1e4a1360fd
[rllib] Add type annotations to Trainer class ( #8642 )
...
* type trainer
* type it
* fxi
2020-06-03 12:47:35 -07:00
Sven Mika
b37a162076
[RLlib] Make envs specifiable in configs by their class path. ( #8750 )
2020-06-03 08:14:29 +02:00
Sven Mika
d8a081a185
[RLlib] Unity3D integration (n Unity3D clients vs learning server). ( #8590 )
2020-05-30 22:48:34 +02:00
Sven Mika
2746fc0476
[RLlib] Auto-framework, retire use_pytorch in favor of framework=... ( #8520 )
2020-05-27 16:19:13 +02:00
konichuvak
13c2e13120
fixing polynomial schedule horizon ( #7795 )
2020-05-27 10:59:28 +02:00
Sven Mika
6d196197bc
[RLlib] utils/spaces ... ( #8608 )
2020-05-27 10:21:30 +02:00
Sven Mika
baa053496a
[RLlib] Benchmark and regression test yaml cleanup and restructuring. ( #8414 )
2020-05-26 11:10:27 +02:00
Eric Liang
9a83908c46
[rllib] Deprecate policy optimizers ( #8345 )
2020-05-21 10:16:18 -07:00
Sven Mika
d76578700d
[RLlib] Policy.compute_single_action() broken for nested actions (Issue 8411). ( #8514 )
2020-05-20 22:29:08 +02:00
Sven Mika
c9435cad43
WIP. ( #8456 )
...
Fix multi-GPU histogram metrics for > 0D tensors.
2020-05-15 21:43:27 +02:00
Sven Mika
754290daad
[RLlib] Add light-weight Trainer.compute_action() tests for all Algos. ( #8356 )
2020-05-08 16:31:31 +02:00
Sven Mika
d7eaacb5fe
[RLlib] Issue 8319 DDPG (MA or num_envs_per_worker > 1) broken. ( #8324 )
2020-05-08 08:26:32 +02:00
Sven Mika
5f278c6411
[RLlib] Examples folder restructuring (models) part 1 ( #8353 )
2020-05-08 08:20:18 +02:00
Sven Mika
6c2b9a4cfa
[RLlib] Remove tf.py_function from all Schedule classes (not differentiable and causes other bugs in MA setups). ( #8304 )
...
Remove tf.py_function from all Schedule classes (not differentiable and causes other bugs in MA setups). (#8304 )
2020-05-04 23:53:38 +02:00
Sven Mika
a00144f746
[RLlib] Fix issue 8135 (DDPG inf actions when using [-inf,inf] action space). ( #8302 )
2020-05-04 22:27:30 +02:00
Sven Mika
b95e28faea
[RLlib] APEX_DDPG (PyTorch) test case and docs. ( #8288 )
...
APEX_DDPG (PyTorch) test case and docs.
2020-05-04 09:36:27 +02:00
Sven Mika
166bb5d690
[RLlib] IMPALA PyTorch ( #8287 )
...
This PR adds an IMPALA PyTorch implementation.
- adds compilation tests for LSTM and w/o LSTM.
- adds learning test for CartPole.
2020-05-03 13:44:25 +02:00
Sven Mika
76e1a4df9e
Fix TD3 torch via GaussianNoise torch bug. ( #8276 )
2020-05-02 08:12:21 +02:00
Sven Mika
bf25aee392
[RLlib] Deprecate all Model(v1) usage. ( #8146 )
...
Deprecate all Model(v1) usage.
2020-04-29 12:12:59 +02:00
Sven Mika
1775e89f26
[RLlib] Remove TupleActions and support arbitrarily nested action spaces. ( #8143 )
...
Deprecate TupleActions and support arbitrarily nested action spaces.
Closes issue #8143 .
2020-04-28 14:59:16 +02:00
Sven Mika
7ec2223c84
[RLlib] DDPG PyTorch actor-model was missing sigmoid layer ( #8188 )
...
Fix DDPG PyTorch (missing sigmoid layer (to squash action outputs) after deterministic action outputs).
2020-04-26 23:08:13 +02:00
Eric Liang
2298f6fb40
[rllib] Port DQN/Ape-X to training workflow api ( #8077 )
2020-04-23 12:39:19 -07:00
Sven Mika
499ad5fbe4
[RLlib] PyTorch version of APPO. ( #8120 )
...
- Translate all vtrace functionality to torch and added torch to the framework_iterator-loop in all existing vtrace test cases.
- Add learning test cases for APPO torch (both w/ and w/o v-trace).
- Add quick compilation tests for APPO (tf and torch, v-trace and no v-trace).
2020-04-23 09:11:12 +02:00
Sven Mika
e9ee5c4e5f
[RLlib] Nested action space PR (minimally invasive; torch only + test). ( #8101 )
...
- Add TorchMultiActionDistribution class.
- Add framework-agnostic test cases for TorchMultiActionDistribution.
2020-04-23 09:09:22 +02:00
Sven Mika
3812bfedda
[RLlib] PyTorch version of ES (Evolution Strategies). ( #8104 )
...
PyTorch version of Evolution Strategies (ES) Algo.
2020-04-20 21:47:28 +02:00
Eric Liang
0c80efa2a3
[rllib] Disable explicit free, which is no longer needed and causes memory leaks
2020-04-16 16:06:58 -07:00
Sven Mika
d0fab84e4d
[RLlib] DDPG PyTorch version. ( #7953 )
...
The DDPG/TD3 algorithms currently do not have a PyTorch implementation. This PR adds PyTorch support for DDPG/TD3 to RLlib.
This PR:
- Depends on the re-factor PR for DDPG (Functional Algorithm API).
- Adds learning regression tests for the PyTorch version of DDPG and a DDPG (torch)
- Updates the documentation to reflect that DDPG and TD3 now support PyTorch.
* Learning Pendulum-v0 on torch version (same config as tf). Wall time a little slower (~20% than tf).
* Fix GPU target model problem.
2020-04-16 10:20:01 +02:00
