Eric Liang
5d7afe8092
[rllib] Try moving RLlib to top level dir ( #5324 )
2019-08-05 23:25:49 -07:00
Philipp Moritz
bbe3e5b4ed
[rllib] Give error if sample_async is used with pytorch for A3C ( #5000 )
...
* give error if sample_async is used with pytorch
* update
* Update a3c.py
2019-06-25 22:06:35 -07:00
Eric Liang
7501ee51db
[rllib] Rename PolicyEvaluator => RolloutWorker ( #4820 )
2019-06-03 06:49:24 +08:00
Eric Liang
02583a8598
[rllib] Rename PolicyGraph => Policy, move from evaluation/ to policy/ ( #4819 )
...
This implements some of the renames proposed in #4813
We leave behind backwards-compatibility aliases for *PolicyGraph and SampleBatch.
2019-05-20 16:46:05 -07:00
Eric Liang
6cb5b90bd6
[rllib] [RFC] Dynamic definition of loss functions and modularization support ( #4795 )
...
* dynamic graph
* wip
* clean up
* fix
* document trainer
* wip
* initialize the graph using a fake batch
* clean up dynamic init
* wip
* spelling
* use builder for ppo pol graph
* add ppo graph
* fix naming
* order
* docs
* set class name correctly
* add torch builder
* add custom model support in builder
* cleanup
* remove underscores
* fix py2 compat
* Update dynamic_tf_policy_graph.py
* Update tracking_dict.py
* wip
* rename
* debug level
* rename policy_graph -> policy in new classes
* fix test
* rename ppo tf policy
* port appo too
* forgot grads
* default policy optimizer
* make default config optional
* add config to optimizer
* use lr by default in optimizer
* update
* comments
* remove optimizer
* fix tuple actions support in dynamic tf graph
2019-05-18 00:23:11 -07:00
Eric Liang
6e7680bf21
[rllib] Clean up concepts documentation and policy optimizer creation ( #4592 )
2019-04-12 21:03:26 -07:00
Eric Liang
37208216ae
[rllib] Rename Agent to Trainer ( #4556 )
2019-04-07 00:36:18 -07:00
Eric Liang
2ffe67c5c3
[rllib] Minor cleanups to TFPolicyGraph: add init args, constants for loss inputs ( #4478 )
2019-03-29 12:44:23 -07:00
Eric Liang
4b8b703561
[rllib] Some API cleanups and documentation improvements ( #4409 )
2019-03-21 21:34:22 -07:00
Eric Liang
27cd6ea401
[rllib] Flip sign of A2C, IMPALA entropy coefficient; raise DeprecationWarning if negative ( #4374 )
2019-03-17 18:07:37 -07:00
Eric Liang
8b5827b9da
[rllib] Better document which methods are abstract and which ones are overrides ( #3480 )
2018-12-08 16:28:58 -08:00
Eric Liang
65c27c70cf
[rllib] Clean up agent resource configurations ( #3296 )
...
Closes #3284
2018-11-13 18:00:03 -08:00
Robert Nishihara
e49839c73f
Fix linting. ( #3155 )
2018-10-28 20:43:29 -07:00
Jones Wong
d6bf890648
Solve hang caused by ray.get in collect_metrics ( #3096 )
2018-10-28 11:52:18 -07:00
Eric Liang
221d1663c1
[rllib] switch to python logger ( #3098 )
...
* logg
* set rllib logger
* comment
* info
* rlib
* comment
* add format
* fix lint
* add file info
* update
* add ts
* lint
* better docs
* fix value error
* soft log level
2018-10-21 23:43:57 -07:00
Eric Liang
a9e454f6fd
[rllib] Include config dicts in the sphinx docs ( #3064 )
2018-10-16 15:55:11 -07:00
Eric Liang
65dcafdc3f
[rllib] Refactor save() / restore() code of agents and avoid O(n_workers) save size ( #2982 )
2018-09-30 01:15:13 -07:00
Eric Liang
25ffe57a5c
[rllib] Auto-synchronize filters for all agents ( #2791 )
...
This makes sure we always update the local filter, and adds an option to synchronize the remote filters as well. In APEX_DDPG we previously didn't do either. The first is needed for checkpoint correctness, the second might help performance.
2018-09-03 20:01:53 -07:00
Eric Liang
aa014af85b
[rllib] Fix atari reward calculations, add LR annealing, explained var stat for A2C / impala ( #2700 )
...
Changes needed to reproduce Atari plots in IMPALA / A2C: https://github.com/ray-project/rl-experiments
2018-08-23 17:49:10 -07:00
Eric Liang
fbe6c59f72
[rllib] Misc fixes, A2C ( #2679 )
...
A bunch of minor rllib fixes:
pull in latest baselines atari wrapper changes (and use deepmind wrapper by default)
move reward clipping to policy evaluator
add a2c variant of a3c
reduce vision network fc layer size to 256 units
switch to 84x84 images
doc tweaks
print timesteps in tune status
2018-08-20 15:28:03 -07:00
Richard Liaw
bb44456f6f
[rllib, tune] TrainingResult -> Dict, Removes C408 from flake8 ( #2565 )
2018-08-07 12:17:44 -07:00
Eric Liang
d9a36c4e39
[rllib] Document auto-concat in a3c ( #2533 )
...
* docs
* update hyperparm docs
2018-08-01 15:11:30 -07:00
Eric Liang
38d00986a5
[rllib] Cleanups: deep merge configs properly; enforce min iter time on APEX ( #2500 )
...
The dict merge prevents crashes when tune is trying to get resource requests for agents and you override a config subkey. The min iter time prevents iterations from getting too small, incurring high overhead. This is easy to run into on Ape-X since throughput can get very high.
2018-07-30 13:25:35 -07:00
Eric Liang
d01dc9e22d
[rllib] format with yapf ( #2427 )
...
* initial yapf
* manual fix yapf bugs
2018-07-19 15:30:36 -07:00
Eric Liang
b316afeb43
[rllib] Add debug info back to PPO and fix optimizer compatibility ( #2366 )
2018-07-12 19:22:46 +02:00
Eric Liang
d24f19fd1e
[rllib] Fix stats collection and some docs bugs since the refactoring ( #2361 )
...
* fix
* fix pbt example
* fix
* fix
* single thread by default
* vec
* fix
* fix
2018-07-07 13:29:20 -07:00
Richard Liaw
e32aed8717
[rllib] more user-friendly Optimizer signature + compute_apply ( #2335 )
...
* Move signature of optimizers
* fix
* expose compute_apply for policy_graphs
* dictionaries and such
* test for multiagent
2018-07-07 12:08:49 -07:00
Eric Liang
8aa56c12e6
[rllib] Document "v2" APIs ( #2316 )
...
* re
* wip
* wip
* a3c working
* torch support
* pg works
* lint
* rm v2
* consumer id
* clean up pg
* clean up more
* fix python 2.7
* tf session management
* docs
* dqn wip
* fix compile
* dqn
* apex runs
* up
* impotrs
* ddpg
* quotes
* fix tests
* fix last r
* fix tests
* lint
* pass checkpoint restore
* kwar
* nits
* policy graph
* fix yapf
* com
* class
* pyt
* vectorization
* update
* test cpe
* unit test
* fix ddpg2
* changes
* wip
* args
* faster test
* common
* fix
* add alg option
* batch mode and policy serving
* multi serving test
* todo
* wip
* serving test
* doc async env
* num envs
* comments
* thread
* remove init hook
* update
* fix ppo
* comments1
* fix
* updates
* add jenkins tests
* fix
* fix pytorch
* fix
* fixes
* fix a3c policy
* fix squeeze
* fix trunc on apex
* fix squeezing for real
* update
* remove horizon test for now
* multiagent wip
* update
* fix race condition
* fix ma
* t
* doc
* st
* wip
* example
* wip
* working
* cartpole
* wip
* batch wip
* fix bug
* make other_batches None default
* working
* debug
* nit
* warn
* comments
* fix ppo
* fix obs filter
* update
* wip
* tf
* update
* fix
* cleanup
* cleanup
* spacing
* model
* fix
* dqn
* fix ddpg
* doc
* keep names
* update
* fix
* com
* docs
* clarify model outputs
* Update torch_policy_graph.py
* fix obs filter
* pass thru worker index
* fix
* rename
* vlad torch comments
* fix log action
* debug name
* fix lstm
* remove unused ddpg net
* remove conv net
* revert lstm
* wip
* wip
* cast
* wip
* works
* fix a3c
* works
* lstm util test
* doc
* clean up
* update
* fix lstm check
* move to end
* fix sphinx
* fix cmd
* remove bad doc
* envs
* vec
* doc prep
* models
* rl
* alg
* up
* clarify
* copy
* async sa
* fix
* comments
* fix a3c conf
* tune lstm
* fix reshape
* fix
* back to 16
* tuned a3c update
* update
* tuned
* optional
* merge
* wip
* fix up
* move pg class
* rename env
* wip
* update
* tip
* alg
* readme
* fix catalog
* readme
* doc
* context
* remove prep
* comma
* add env
* link to paper
* paper
* update
* rnn
* update
* wip
* clean up ev creation
* fix
* fix
* fix
* fix lint
* up
* no comma
* ma
* Update run_multi_node_tests.sh
* fix
* sphinx is stupid
* sphinx is stupid
* clarify torch graph
* no horizon
* fix config
* sb
* Update test_optimizers.py
2018-07-01 00:05:08 -07:00
