* Added basic functionality and tests
* Feature parity with old tune search space config
* Convert Optuna search spaces
* Introduced quantized values
* Updated Optuna resolving
* Added HyperOpt search space conversion
* Convert search spaces to AxSearch
* Convert search spaces to BayesOpt
* Added basic functionality and tests
* Feature parity with old tune search space config
* Convert Optuna search spaces
* Introduced quantized values
* Updated Optuna resolving
* Added HyperOpt search space conversion
* Convert search spaces to AxSearch
* Convert search spaces to BayesOpt
* Re-factored samplers into domain classes
* Re-added base classes
* Re-factored into list comprehensions
* Added `from_config` classmethod for config conversion
* Applied suggestions from code review
* Removed truncated normal distribution
* Set search properties in tune.run
* Added test for tune.run search properties
* Move sampler initializers to base classes
* Add tune API sampling test, fixed includes, fixed resampling bug
* Add to API docs
* Fix docs
* Update metric and mode only when set. Set default metric and mode to experiment analysis object.
* Fix experiment analysis tests
* Raise error when delimiter is used in the config keys
* Added randint/qrandint to API docs, added additional check in tune.run
* Fix tests
* Fix linting error
* Applied suggestions from code review. Re-aded tune.function for the time being
* Fix sampling tests
* Fix experiment analysis tests
* Fix tests and linting error
* Removed unnecessary default_config attribute from OptunaSearch
* Revert to set AxSearch default metric
* fix-min-max
* fix
* nits
* Added function check, enhanced loguniform error message
* fix-print
* fix
* fix
* Raise if unresolved values are in config and search space is already set
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
* create guide gallery for Tune
* mods
* ok
* fix
* fix_up_gallery
* ok
* Apply suggestions from code review
Co-Authored-By: Sven Mika <sven@anyscale.io>
* Apply suggestions from code review
Co-Authored-By: Sven Mika <sven@anyscale.io>
Co-authored-by: Sven Mika <sven@anyscale.io>
* Add base for Soft Actor-Critic
* Pick changes from old SAC branch
* Update sac.py
* First implementation of sac model
* Remove unnecessary SAC imports
* Prune unnecessary noise and exploration code
* Implement SAC model and use that in SAC policy
* runs but doesn't learn
* clear state
* fix batch size
* Add missing alpha grads and vars
* -200 by 2k timesteps
* doc
* lazy squash
* one file
* ignore tfp
* revert done
* Implement Actor checkpointing
* docs
* fix
* fix
* fix
* move restore-from-checkpoint to HandleActorStateTransition
* Revert "move restore-from-checkpoint to HandleActorStateTransition"
This reverts commit 9aa4447c1e3e321f42a1d895d72f17098b72de12.
* resubmit waiting tasks when actor frontier restored
* add doc about num_actor_checkpoints_to_keep=1
* add num_actor_checkpoints_to_keep to Cython
* add checkpoint_expired api
* check if actor class is abstract
* change checkpoint_ids to long string
* implement java
* Refactor to delay actor creation publish until checkpoint is resumed
* debug, lint
* Erase from checkpoints to restore if task fails
* fix lint
* update comments
* avoid duplicated actor notification log
* fix unintended change
* add actor_id to checkpoint_expired
* small java updates
* make checkpoint info per actor
* lint
* Remove logging
* Remove old actor checkpointing Python code, move new checkpointing code to FunctionActionManager
* Replace old actor checkpointing tests
* Fix test and lint
* address comments
* consolidate kill_actor
* Remove __ray_checkpoint__
* fix non-ascii char
* Loosen test checks
* fix java
* fix sphinx-build
* Factor out starting Ray processes.
* Detect flags through environment variables.
* Return ProcessInfo from start_ray_process.
* Print valgrind errors at exit.
* Test valgrind in travis.
* Some valgrind fixes.
* Undo raylet monitor change.
* Only test plasma store in valgrind.
This PR introduces the following changes:
* Ray Tune -> Tune
* [breaking] Creation of `schedulers/`, moving PBT, HyperBand into a submodule
* [breaking] Search Algorithms now must take in experiment configurations via `add_configurations` rather through initialization
* Support `"run": (function | class | str)` with automatic registering of trainable
* Documentation Changes
to support TF version < 1.5
to support rmsprop optimizer in Impala
Before TF1.5, tf.reduce_sum() and tf.reduce_max() has an argument keep_dims which has been renamed as keepdims in later versions.
In the original paper of Impala, they use rmsprop algorithm to optimize the model. We'd better also support it so that users can reproduce their experiments. Without any tuning, say that using the same hyper-parameters as AdamOptimizer, it reaches "episode_reward_mean": 19.083333333333332 in Pong after consume 3,610,350 samples.
