Jun Gong
acf2bf9b2f
[RLlib] Get rid of all these deprecation warnings. ( #27085 )
2022-07-27 10:48:54 -07:00
Kai Fricke
8fda425eca
[tune/rllib] Hotfix ml_utils deprecation import error ( #27095 )
...
The changes conflicted with a recently merged PR that refactored the package structure (#27005 ).
Signed-off-by: Kai Fricke <kai@anyscale.com>
2022-07-27 16:11:58 +01:00
Kai Fricke
a5ea99cf95
[rfc] [tune/rllib] Fetch _progress_metrics from trainable for verbose=2 display ( #26967 )
...
RLLibs trainables produce a large number of metrics which makethe log output with verbose=2 illegible. This PR introduces a private `_progress_metrics` property for trainables. If set, the trial progress callback will only print these metrics per default, unless overridden e.g. with a custom `TrialProgressCallback`.
2022-07-27 16:04:23 +01:00
Amog Kamsetty
862d10c162
[AIR] Remove ML code from ray.util ( #27005 )
...
Removes all ML related code from `ray.util`
Removes:
- `ray.util.xgboost`
- `ray.util.lightgbm`
- `ray.util.horovod`
- `ray.util.ray_lightning`
Moves `ray.util.ml_utils` to other locations
Closes #23900
Signed-off-by: Amog Kamsetty <amogkamsetty@yahoo.com>
Signed-off-by: Kai Fricke <kai@anyscale.com>
Co-authored-by: Kai Fricke <kai@anyscale.com>
2022-07-27 14:24:19 +01:00
xwjiang2010
fcf897ee72
[air] update rllib example to use Tuner API. ( #26987 )
...
update rllib example to use Tuner API.
Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com>
2022-07-27 12:12:59 +01:00
Jun Gong
c7ae787cc8
[RLlib] Beef up worker failure test. ( #26953 )
2022-07-27 00:10:45 -07:00
Jun Gong
a22457b548
[RLlib] Small bug fix ( #27003 )
2022-07-27 00:02:18 -07:00
Jun Gong
54df8bfe42
[RLlib] Try to checkpoint a durable policy name ( #27016 )
2022-07-27 00:01:14 -07:00
kourosh hakhamaneshi
8ddcf89096
[RLlib] Implemented ViewRequirementConnector ( #26998 )
2022-07-26 21:52:14 -07:00
Rohan Potdar
deccf33912
[RLlib]: Add Off-Policy Estimation docs ( #26809 )
...
Co-authored-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
2022-07-26 13:57:56 -07:00
Jun Gong
ca5e0dcaf4
[RLLib] Record framework and algorithm used by an RLlib run. ( #26956 )
...
Automatically record framework and algorithm used by RLlib jobs.
For better planning.
2022-07-25 16:16:36 -07:00
kourosh hakhamaneshi
5030a4c1d3
[RLlib] Simplify agent collector ( #26803 )
2022-07-25 13:17:17 -07:00
Artur Niederfahrenhorst
b1594260ba
[RLlib] Small SlateQ example fix. ( #26948 )
2022-07-25 15:12:42 +02:00
Avnish Narayan
41c9ef709a
[RLlib] Using PG when not doing microbatching kills A2C performance. ( #26844 )
2022-07-25 15:11:26 +02:00
Artur Niederfahrenhorst
e9a8f7d9ae
[RLlib] Unify gnorm mixin for tf and torch policies. ( #26102 )
2022-07-24 15:31:09 +02:00
Ishant Mrinal
b32c784c7f
[RLLib] RE3 exploration algorithm TF2 framework support ( #25221 )
2022-07-23 18:05:01 -07:00
Rohan Potdar
a53bbe49bf
[RLlib]: Raise deprecation warning in MARWIL OPE methods. ( #26893 )
2022-07-23 13:55:40 +02:00
Rohan Potdar
97bcf38ec0
[RLlib] Fix torch None conversion in torch_utils.py::convert_to_torch_tensor. ( #26863 )
2022-07-23 13:54:57 +02:00
Rohan Potdar
69f6b843da
[RLlib] Test output length in DatasetReader with default IOContext. ( #26852 )
2022-07-23 13:53:59 +02:00
Rohan Potdar
2f22262d39
[RLlib]: Fix SampleBatch.split_by_episode to use dones if episode id is not available ( #26492 )
2022-07-22 16:46:05 -07:00
Steven Morad
259429bdc3
Bump gym dep to 0.24 ( #26190 )
...
Co-authored-by: Steven Morad <smorad@anyscale.com>
Co-authored-by: Avnish <avnishnarayan@gmail.com>
Co-authored-by: Avnish Narayan <38871737+avnishn@users.noreply.github.com>
2022-07-22 12:37:16 -07:00
Avnish Narayan
2a0ef663c9
[rllib] Use compress observations where replay buffers and image obs are used in tuned examples ( #26735 )
2022-07-22 10:10:51 -07:00
Fabian Witter
dc2ad6c8b4
[RLlib] Fix ModelCatalog for nested complex inputs ( #25620 )
2022-07-22 00:45:25 -07:00
Jun Gong
6c1acd1a2f
[RLlib] Quick state buffer connector fix ( #26836 )
2022-07-21 20:43:59 -07:00
Jun Gong
0bc560bd54
[RLlib] Make sure we step() after adding init_obs. ( #26827 )
2022-07-21 20:43:46 -07:00
Rohan Potdar
2b13ac85f9
[RLLib]: Make IOContext optional for DatasetReader ( #26694 )
2022-07-21 13:05:00 -07:00
Olaf Lipinski
8271406a04
[RLLib] Fix MultiDiscrete not being one-hotted correctly ( #26558 )
...
Co-authored-by: Jun Gong <jungong@anyscale.com>
2022-07-20 15:25:53 -07:00
kourosh hakhamaneshi
aec79afda1
[RLlib] Fixes CRR flakeyness ( #26770 )
2022-07-20 12:08:57 -07:00
Avnish Narayan
9063cc9d5e
[RLlib] Fix memory leak in APEX_DQN ( #26691 )
2022-07-19 16:16:24 -07:00
Avnish Narayan
af41f21be0
[RLlib] Make queue placement ops blocking ( #26581 )
...
Signed-off-by: avnish avnish@anyscale.com
This change should fix issues with IMPALA and potentially APEX that stem from the various learner threads
Signed-off-by: avnish <avnish@anyscale.com>
2022-07-19 20:07:36 +01:00
Jun Gong
6b6d3017ba
[RLlib] more connector polishes and fixes. ( #26645 )
2022-07-19 08:50:28 -07:00
Rohan Potdar
4fded80813
[RLlib]: Fix FQE Policy call ( #26671 )
2022-07-19 00:58:31 -07:00
Riatre
591cd22be7
Revert "Revert "Bump pytest from 5.4.3 to 7.0.1"" ( #26525 )
...
* Revert "Revert "Bump pytest from 5.4.3 to 7.0.1""
This reverts commit ab10890e90
2022-07-18 21:21:19 -07:00
Artur Niederfahrenhorst
0ce3bc5e48
[RLlib] Add/reorder Args of Prioritized/MixIn MultiAgentReplayBuffer. ( #26428 )
2022-07-18 18:04:03 +02:00
Rohan Potdar
38c9e1d52a
[RLlib]: Fix OPE trainables ( #26279 )
...
Co-authored-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
2022-07-17 14:25:53 -07:00
kourosh hakhamaneshi
569fe01096
[RLlib] improved unittests for dataset_reader and fixed bugs ( #26458 )
2022-07-17 13:38:15 -07:00
Ishant Mrinal
57244aeee3
[RLlib] Make DQN update_target use only trainable variables. ( #25226 )
2022-07-15 09:17:06 +02:00
mgerstgrasser
f0e9d1a9bb
[RLlib] In env check, step only expected agents. ( #26425 )
2022-07-15 09:16:09 +02:00
Sven Mika
4aea24c8a8
[RLlib] restart_failed_sub_environments now works for MA cases and crashes during reset(); +more tests and logging; add eval worker sub-env fault tolerance test. ( #26276 )
2022-07-15 08:55:14 +02:00
Avnish Narayan
a322ac463c
[RLlib] Make JSONReader default, users will have to use the DatasetReader for any speedups. ( #26541 )
2022-07-14 17:19:38 +02:00
Jun Gong
104407a6e5
[RLlib] Fix all the erroneous on_trainer_init warning. ( #26433 )
2022-07-13 18:56:01 +02:00
Jun Gong
b383d987d1
[RLlib] Fix a bunch of issues related to connectors. ( #26510 )
2022-07-13 18:55:20 +02:00
Avnish Narayan
5df66b917d
[Lint Check] Remove broken link ( #26505 )
...
The paper is not available anymore.
2022-07-13 10:30:20 +01:00
Christy Bergman
7c925fe99f
[RLlib; docs] Re-organize algorithms so TOC matches README. ( #26339 )
2022-07-13 10:46:36 +02:00
Ram Rachum
14800e5ac7
Fix exception cause in preprocessors.py ( #26322 )
2022-07-12 20:15:04 -07:00
Rohan Potdar
09ce4711fd
[RLlib]: Move OPE to evaluation config ( #25911 )
2022-07-12 11:04:34 -07:00
kourosh hakhamaneshi
be6e4c644f
[RLlib] Feature importance evaluation for offline RL ( #26412 )
2022-07-11 18:12:50 -07:00
Jun Gong
0c469e490e
[RLlib] Checkpoint and restore connectors. ( #26253 )
2022-07-09 01:06:24 -07:00
Jun Gong
d234348bd2
[RLlib] Minor simplification of code. ( #26312 )
2022-07-08 13:21:54 -07:00
Avnish Narayan
1243ed62bf
[RLlib] Make Dataset reader default reader and enable CRR to use dataset ( #26304 )
...
Co-authored-by: avnish <avnish@avnishs-MBP.local.meter>
2022-07-08 12:43:35 -07:00
