Sven Mika
d15609ba2a
[RLlib] PyTorch version of ARS (Augmented Random Search). ( #8106 )
...
This PR implements a PyTorch version of RLlib's ARS algorithm using RLlib's functional algo builder API. It also adds a regression test for ARS (torch) on CartPole.
2020-04-21 09:47:52 +02:00
Sven Mika
3812bfedda
[RLlib] PyTorch version of ES (Evolution Strategies). ( #8104 )
...
PyTorch version of Evolution Strategies (ES) Algo.
2020-04-20 21:47:28 +02:00
Bill Chambers
77655749fb
[RayServe] RayServe Introduction and Overview ( #8038 )
2020-04-20 12:05:59 -05:00
Sven Mika
165a86f1ab
[RLlib] SAC MuJoCo instability issues (tf and torch versions). ( #8063 )
...
SAC (both torch and tf versions) are showing issues (crashes) due to numeric instabilities in the SquashedGaussian distribution (sampling + logp after extreme NN outputs).
This PR fixes these. Stable MuJoCo learning (HalfCheetah) has been confirmed on both tf and torch versions. A Distribution stability test (using extreme NN outputs) has been added for SquashedGaussian (can be used for any other type of distribution as well).
2020-04-19 10:20:23 +02:00
Sumanth Ratna
bdb03a0544
[tune] Update dragonfly installation instructions ( #8086 )
...
Closes #8084
2020-04-18 20:25:38 -07:00
Richard Liaw
857e4dba2f
[sgd] HuggingFace GLUE Fine-tuning Example ( #7792 )
...
* Init fp16
* fp16 and schedulers
* scheduler linking and fp16
* to fp16
* loss scaling and documentation
* more documentation
* add tests, refactor config
* moredocs
* more docs
* fix logo, add test mode, add fp16 flag
* fix tests
* fix scheduler
* fix apex
* improve safety
* fix tests
* fix tests
* remove pin memory default
* rm
* fix
* Update doc/examples/doc_code/raysgd_torch_signatures.py
* fix
* migrate changes from other PR
* ok thanks
* pass
* signatures
* lint'
* Update python/ray/experimental/sgd/pytorch/utils.py
* Apply suggestions from code review
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* should address most comments
* comments
* fix this ci
* first_pass
* add overrides
* override
* fixing up operators
* format
* sgd
* constants
* rm
* revert
* save
* failures
* fixes
* trainer
* run test
* operator
* code
* op
* ok done
* operator
* sgd test fixes
* ok
* trainer
* format
* Apply suggestions from code review
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update doc/source/raysgd/raysgd_pytorch.rst
* docstring
* dcgan
* doc
* commits
* nit
* testing
* revert
* Start renaming pytorch to torch
* Rename PyTorchTrainer to TorchTrainer
* Rename PyTorch runners to Torch runners
* Finish renaming API
* Rename to torch in tests
* Finish renaming docs + tests
* Run format + fix DeprecationWarning
* fix
* move tests up
* benchmarks
* rename
* remove some args
* better metrics output
* fix up the benchmark
* benchmark-yaml
* horovod-benchmark
* benchmarks
* Remove benchmark code for cleanups
* benchmark-code
* nits
* benchmark yamls
* benchmark yaml
* ok
* ok
* ok
* benchmark
* nit
* finish_bench
* makedatacreator
* relax
* metrics
* autosetsampler
* profile
* movements
* OK
* smoothen
* fix
* nitdocs
* loss
* envflag
* comments
* nit
* format
* visible
* images
* move_images
* fix
* rernder
* rrender
* rest
* multgpu
* fix
* nit
* finish
* extrra
* setup
* experimental
* as_trainable
* fix
* ok
* format
* create_torch_pbt
* setup_pbt
* ok
* format
* ok
* format
* docs
* ok
* Draft head-is-worker
* Fix missing concurrency between local and remote workers
* Fix tqdm to work with head-is-worker
* Cleanup
* Implement state_dict and load_state_dict
* Reserve resources on the head node for the local worker
* Update the development cluster setup
* Add spot block reservation to the development yaml
* ok
* Draft the fault tolerance fix
* Small fixes to local-remote concurrency
* Cleanup + fix typo
* fixes
* worker_counts
* some formatting and asha
* fix
* okme
* fixactorkill
* unify
* Revert the cluster mounts
* Cut the handler-reporter API
* Fix most tests
* Rm tqdm_handler.py
* Re-add tune test
* Automatically force-shutdown on actor errors on shutdown
* Formatting
* fix_tune_test
* Add timeout error verification
* Rename tqdm to use_tqdm
* fixtests
* ok
* remove_redundant
* deprecated
* deactivated
* ok_try_this
* lint
* nice
* done
* retries
* fixes
* kill
* retry
* init_transformer
* init
* deployit
* improve_example
* trans
* rename
* formats
* format-to-py37
* time_to_test
* more_changes
* ok
* update_args_and_script
* fp16_epoch
* huggingface
* training stats
* distributed
* Apply suggestions from code review
* transformer
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Co-authored-by: Maksim Smolin <maximsmol@gmail.com>
2020-04-17 15:17:30 -07:00
Maksim Smolin
d6f4e5b3e1
[SGD] Imagenet example (basic) ( #8020 )
...
* Checkpoint the image-models example
* Update cluster definition
* Fix copyright info
* Use original args
* Checkpoint fixes
* Add README
* Add some missing features
* Format
* Get rid of the unused Namespace class
* Address comments
* Link the imagenet example in docs
* Cleanup
* Fix lint
2020-04-17 13:33:55 -07:00
roireshef
dbcad35022
[RLlib] Added DefaultCallbacks which replaces old callbacks dict interface ( #6972 )
2020-04-16 16:06:42 -07:00
Richard Liaw
2cb3355495
[docs] Move css to right location ( #8053 )
2020-04-16 13:46:50 -07:00
Richard Liaw
d5f517b2f5
[docs] Hotfix for missing css files. ( #8051 )
2020-04-16 11:44:55 -07:00
Richard Liaw
4d8bf5635d
[hotfix] Lint formatting for new Tune optimizer ZOOpt ( #8040 )
...
* formatting
* removedill
* lint
2020-04-16 09:24:30 -07:00
Sven Mika
d0fab84e4d
[RLlib] DDPG PyTorch version. ( #7953 )
...
The DDPG/TD3 algorithms currently do not have a PyTorch implementation. This PR adds PyTorch support for DDPG/TD3 to RLlib.
This PR:
- Depends on the re-factor PR for DDPG (Functional Algorithm API).
- Adds learning regression tests for the PyTorch version of DDPG and a DDPG (torch)
- Updates the documentation to reflect that DDPG and TD3 now support PyTorch.
* Learning Pendulum-v0 on torch version (same config as tf). Wall time a little slower (~20% than tf).
* Fix GPU target model problem.
2020-04-16 10:20:01 +02:00
Servon
5c274fe631
[Tune] Add ZOOpt search algorithm ( #7960 )
...
* add zoopt
* add zoopt search algo
* add zoopt
* fix zoopt
* add zoopt requirements
* fix zoopt
* remove generated guides
* Apply suggestions from code review
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-04-15 21:13:29 -07:00
Simon Mo
7455610d5a
Serve Doc: Quickstart ( #7940 )
2020-04-15 12:25:37 -07:00
Robert Nishihara
d985d7537e
Replace all instances of ray.readthedocs.io with ray.io ( #7994 )
2020-04-13 16:17:05 -07:00
Richard Liaw
e97adba6ac
[autoscaler] Improve argument handling for submit ( #7986 )
...
* docs
* Apply suggestions from code review
Co-Authored-By: Kristian Hartikainen <kristian.hartikainen@gmail.com>
* ok
Co-authored-by: Kristian Hartikainen <kristian.hartikainen@gmail.com>
2020-04-13 15:53:42 -07:00
Richard Liaw
e68d601ec7
[docs] Add link master <-> latest via sphinx version warnings ( #8010 )
2020-04-13 15:21:08 -07:00
Richard Liaw
dd63178e91
[sgd] Semantic Segmentation Example ( #7825 )
...
* better_example
* test
* improve some usability things
* submit
* fix
* making a segmentation example
* segmentation_example
* segmentation
* device
* flake
* Update python/ray/util/sgd/torch/training_operator.py
* uti
* finished_example
* block
* format
* locationg
* fix
* ok
* revert
* segmentation
* lint_and_test
* address_comments
2020-04-10 20:35:45 -07:00
Sven Mika
d2b5c171cb
[RLlib] Add pytorch sigils to toc and add links to algo overview table. ( #7950 )
...
* Add torch sigils to toc-tree for DQN/APEX.
* WIP.
2020-04-09 10:40:18 -07:00
acxz
8f94f9c372
[arch linux] add package installation instructions ( #7898 )
2020-04-08 11:13:42 -07:00
Richard Liaw
a67edc4051
[tune] Improve user guides and API docs ( #7716 )
...
* create guide gallery for Tune
* mods
* ok
* fix
* fix_up_gallery
* ok
* Apply suggestions from code review
Co-Authored-By: Sven Mika <sven@anyscale.io>
* Apply suggestions from code review
Co-Authored-By: Sven Mika <sven@anyscale.io>
Co-authored-by: Sven Mika <sven@anyscale.io>
2020-04-06 12:16:35 -07:00
Sven Mika
22ccc43670
[RLlib] DQN torch version. ( #7597 )
...
* Fix.
* Rollback.
* WIP.
* WIP.
* WIP.
* WIP.
* WIP.
* WIP.
* WIP.
* WIP.
* Fix.
* Fix.
* Fix.
* Fix.
* Fix.
* WIP.
* WIP.
* Fix.
* Test case fixes.
* Test case fixes and LINT.
* Test case fixes and LINT.
* Rollback.
* WIP.
* WIP.
* Test case fixes.
* Fix.
* Fix.
* Fix.
* Add regression test for DQN w/ param noise.
* Fixes and LINT.
* Fixes and LINT.
* Fixes and LINT.
* Fixes and LINT.
* Fixes and LINT.
* Comment
* Regression test case.
* WIP.
* WIP.
* LINT.
* LINT.
* WIP.
* Fix.
* Fix.
* Fix.
* LINT.
* Fix (SAC does currently not support eager).
* Fix.
* WIP.
* LINT.
* Update rllib/evaluation/sampler.py
Co-Authored-By: Eric Liang <ekhliang@gmail.com>
* Update rllib/evaluation/sampler.py
Co-Authored-By: Eric Liang <ekhliang@gmail.com>
* Update rllib/utils/exploration/exploration.py
Co-Authored-By: Eric Liang <ekhliang@gmail.com>
* Update rllib/utils/exploration/exploration.py
Co-Authored-By: Eric Liang <ekhliang@gmail.com>
* WIP.
* WIP.
* Fix.
* LINT.
* LINT.
* Fix and LINT.
* WIP.
* WIP.
* WIP.
* WIP.
* Fix.
* LINT.
* Fix.
* Fix and LINT.
* Update rllib/utils/exploration/exploration.py
* Update rllib/policy/dynamic_tf_policy.py
Co-Authored-By: Eric Liang <ekhliang@gmail.com>
* Update rllib/policy/dynamic_tf_policy.py
Co-Authored-By: Eric Liang <ekhliang@gmail.com>
* Update rllib/policy/dynamic_tf_policy.py
Co-Authored-By: Eric Liang <ekhliang@gmail.com>
* Fixes.
* WIP.
* LINT.
* Fixes and LINT.
* LINT and fixes.
* LINT.
* Move action_dist back into torch extra_action_out_fn and LINT.
* Working SimpleQ learning cartpole on both torch AND tf.
* Working Rainbow learning cartpole on tf.
* Working Rainbow learning cartpole on tf.
* WIP.
* LINT.
* LINT.
* Update docs and add torch to APEX test.
* LINT.
* Fix.
* LINT.
* Fix.
* Fix.
* Fix and docstrings.
* Fix broken RLlib tests in master.
* Split BAZEL learning tests into cartpole and pendulum (reached the 60min barrier).
* Fix error_outputs option in BAZEL for RLlib regression tests.
* Fix.
* Tune param-noise tests.
* LINT.
* Fix.
* Fix.
* test
* test
* test
* Fix.
* Fix.
* WIP.
* WIP.
* WIP.
* WIP.
* LINT.
* WIP.
Co-authored-by: Eric Liang <ekhliang@gmail.com>
2020-04-06 11:56:16 -07:00
Richard Liaw
f63b4c1110
[sgd] make ddp optional ( #7875 )
...
* loosen
* devices
* tryitout
* fix
* fix
* fix
* easy
* test
* fix
* fix
* better visibility
* fix
2020-04-06 11:41:36 -07:00
Robert Nishihara
38fad274aa
Add links to 3.8 nightly wheels to docs. ( #7902 )
2020-04-04 21:59:45 -07:00
Markus Cozowicz
b853df7a3b
[autoscaler] Switch to ARM for Azure deployment ( #7717 )
...
* switch to ARM templates for config and VMs
* switch to ARM templates for config and VMs
* auto-formatting
* addressed Scotts comment
* added missing imports
* fixed gpu templates
fixed wheel reference
* added missing reference
* cleanup wording and yamls
* Update doc/source/autoscaling.rst
Co-Authored-By: Scott Graham <5720537+gramhagen@users.noreply.github.com>
Co-authored-by: Ubuntu <marcozo@marcozodev2.zqvgrdyupqrudayw1il1agipig.jx.internal.cloudapp.net>
Co-authored-by: Scott Graham <5720537+gramhagen@users.noreply.github.com>
2020-04-03 15:51:56 -07:00
Simon Mo
a3181816b3
Add result from 0.8.4 release ( #7876 )
2020-04-02 15:31:03 -07:00
Richard Liaw
314250d072
[docs] Make Ray slack more prominent ( #7870 )
2020-04-02 11:14:02 -07:00
Richard Liaw
24bf6ad607
[raysgd] Improve raysgd examples ( #7818 )
...
* better_example
* test
* improve some usability things
* submit
* fix
* flake
* Update python/ray/util/sgd/torch/training_operator.py
* trythis
* fix
* fix
* smoke
* fail
* fix
* fix
2020-04-01 08:58:39 -07:00
Robert Nishihara
b011c604d7
Remove ray.tasks() from API. ( #7807 )
2020-04-01 10:10:40 -05:00
Richard Liaw
18327254b6
[docs] Fix readthedocs rendering ( #7810 )
2020-03-30 11:40:08 -07:00
Richard Liaw
86cff17e7e
[tune/raysgd] Tune API for TorchTrainer + Fix State Restoration ( #7547 )
2020-03-30 12:58:49 -05:00
Eric Liang
5cebee68d6
[rllib] Add scaling guide to documentation, improve bandit docs ( #7780 )
...
* update
* reword
* update
* ms
* multi node sgd
* reorder
* improve bandit docs
* contrib
* update
* ref
* improve refs
* fix build
* add pillow dep
* add pil
* update pil
* pillow
* remove false
2020-03-27 22:05:43 -07:00
Richard Liaw
875309fc48
Revert wide docs ( #7782 )
2020-03-27 17:46:08 -07:00
Richard Liaw
e10dc91821
Fix doc build ( #7781 )
2020-03-27 17:39:38 -07:00
Simon Mo
838c1e854f
Add results from 0.8.3 release ( #7745 )
2020-03-27 11:14:15 -07:00
Saurabh Gupta
6ddf84b019
Contextual Bandit algorithms (WIP) ( #7642 )
2020-03-26 13:41:16 -07:00
hubcity
3d0a8662b3
#7246 - Fixing broken links ( #7247 )
...
* #7246 - Fixing broken links
* Apply suggestions from code review
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-03-25 21:46:13 -07:00
Richard Liaw
3fa2e4a346
[docs] Fix import breaking docs build ( #7715 )
...
* psutil missing
* ok
2020-03-23 13:21:39 -07:00
Richard Liaw
e311013afd
[tune] Reformat Sections of API Reference ( #7706 )
...
* moveit
* moveit
* docstrings to ref
* Update tune-usage.rst
Co-authored-by: Sven Mika <sven@anyscale.io>
2020-03-23 12:23:21 -07:00
Sven Mika
1138f2ebed
[RLlib] Issue 7046 cannot restore keras model from h5 file. ( #7482 )
2020-03-23 12:19:30 -07:00
Robert Nishihara
1a0c9228d0
Remove pytest from setup.py and other minor changes. ( #7700 )
2020-03-23 08:46:56 -07:00
SangBin Cho
79767fe425
Fix wording in dashboard documentation. ( #7703 )
2020-03-22 22:16:40 -07:00
Robert Nishihara
8b4c2b7e88
Remove unnecessary handling of setproctitle and psutil. ( #7702 )
2020-03-22 22:06:42 -07:00
Robert Nishihara
4d722bf003
Remove dependence on funcsigs. ( #7701 )
2020-03-22 21:37:24 -07:00
Richard Liaw
81d311031b
[tune] Update API Reference Page ( #7671 )
...
* widerdocs
* init
* docs
* fix
* moveit
* mix
* better_docs
* remove
* Apply suggestions from code review
Co-Authored-By: Sven Mika <sven@anyscale.io>
Co-authored-by: Sven Mika <sven@anyscale.io>
2020-03-22 16:42:20 -07:00
Eric Liang
288933ec6b
[rllib] Fix shared metrics context in parallel iterators ( #7666 )
...
* debug
* build
* update
* wip
* wpi
* update
* recurisve sync
* comment
* stream
* fix
* Update .travis.yml
2020-03-22 14:15:01 -07:00
SangBin Cho
1b90196bef
[doc] Dashboard documentation ( #7304 )
...
* Completed the first half of dashboard documentation.
* Dashboard document initial versions.
* Formatting.
* Fixed tune note is not visible.
* Half of comments from code reivew are handled.
* Fixed based on code review.
* Improved memory usage page.
* Addressed code review.
* Fixed image not found issue.
* Add gitkeep again.
* Refactored document.
* Addressed Robert's feedback.
* Addressed code reviews.
* Addressed last comments.
* Update doc/source/ray-dashboard.rst
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-03-20 22:00:33 -07:00
Edward Oakes
31845f17a5
[docs] Add documentation for reference counting and 'ray memory' ( #7661 )
2020-03-20 15:47:00 -05:00
Eric Liang
9392cdbf74
[rllib] Add high-performance external application connector ( #7641 )
2020-03-20 12:43:57 -07:00
Eric Liang
745b9d643d
First pass at ray memory command for memory debugging ( #7589 )
2020-03-17 20:45:07 -07:00
