Eric Liang
52888e4c6f
[tune] Improve the tune Python API and variant generation ( #1154 )
...
* new variant gen
* wip
* Sat Oct 21 18:21:34 PDT 2017
* update
* comment
* fix
* update
* update readme
* fix
* Update README.rst
* Update README.rst
* fix repeat
* update
* note on restore
2017-11-06 23:41:17 -08:00
Richard Liaw
6222ec3bd7
[tune] hyperband ( #1156 )
...
* trial scheduler interface
* remove
* wip median stopping
* remove
* median stopping rule
* update
* docs
* update
* Revrt
* update
* hyperband untested
* small changes before moving on
* added endpoints
* good changes
* init tests
* smore tests
* unfinished tests
* testing
* testing code
* morbugs
* fixes
* end
* tests and typo
* nit
* try this
* tests
* testing
* lint
* lint
* lint
* comments and docs
* almost screwed up
* lint
2017-11-06 22:30:25 -08:00
Eric Liang
d06beacd84
[tune] Implement median stopping rule ( #1170 )
...
* trial scheduler interface
* remove
* wip median stopping
* remove
* median stopping rule
* update
* docs
* update
* Revrt
* update
* comments
* fix tesT
2017-11-03 11:25:02 -07:00
Philipp Moritz
fdf069bd1d
update version to 0.2.2 ( #1178 )
2017-11-01 20:41:24 -07:00
Robert Nishihara
3317d38278
Replace hostnames with numerical IP addresses in redis address. ( #1177 )
...
* Replace hostnames with numerical IP addresses in redis address.
* Also do conversion for node_ip_address. Add test.
* Simplifications.
2017-11-01 17:13:22 -07:00
Eric Liang
202e7bf19a
fix ( #1174 )
2017-11-01 13:45:39 -07:00
Richard Liaw
dc66a2d7d5
[rllib] A3C Refactoring ( #1166 )
...
* fixing policy
* Compute Action is singular, fixed weird issue with arrays
* remove vestige
* extraneous ipdb
* Can Drop in Pytorch Model
* lint
* naming
* finish comments
2017-10-29 11:12:17 -07:00
Eric Liang
4cace0976d
[rllib] Fix DQN inefficiency, and cleanup for different modes of parallelism ( #1151 )
...
* initial checkin
* flake
* dqn
* docs
* add tuned pong
* remove
* upd
* add both
* better gamma
* update
* Last nit
2017-10-29 10:52:30 -07:00
Richard Liaw
304c3cade4
[tune] 10 second timeout for stopping ( #1169 )
...
* 10 second timeout for stopping
* prints for travis
* lint
* try better returning mechanism
* lint
2017-10-29 00:49:29 -07:00
Robert Nishihara
6852e8839e
Expose custom serializers through the API. ( #1147 )
...
* Expose custom serializers through the API.
* minor renaming
* Add test.
* Remove comment.
* Clean up assertions.
2017-10-29 00:08:55 -07:00
Eric Liang
3b157ab933
[tune] Allow resources to not all be assigned to the driver ( #1150 )
...
* dgpu
* update
* update
* update
* also support cmdline
* limit
* Update README.rst
* documentation
* typo
* small coverage for driver_gpu_limit
* lint
* fix lint
2017-10-28 22:16:05 -07:00
Robert Nishihara
f59867850e
Upgrade to cloudpickle 0.4.1. ( #1164 )
...
* Upgrade to cloudpickle 0.4.1.
* Catch more general exceptions thrown by cloudpickle.
2017-10-28 01:35:35 -07:00
Eric Liang
2b6c7af8ad
[tune] Trial scheduler interface ( #1160 )
...
* trial scheduler interface
* remove
* update
2017-10-27 13:29:15 -07:00
Richard Liaw
797f4fcbf3
Fixing Lint after flake upgrade ( #1162 )
...
* Fixing Lint after flake upgrade
* more lint fixes
2017-10-26 21:02:07 -05:00
Eric Liang
cd9dc398ff
[rllib] Support discrete observation spaces such as FrozenLake-v0 ( #1140 )
...
* add
* remove transform_shape
* fix test
* fix
2017-10-23 23:16:52 -07:00
Richard Liaw
0c9817fa76
[tune] Tune Pausing ( #1136 )
...
* fix yaml bug
* add ext agent
* gpus
* update
* tuning
* docs
* Sun Oct 15 21:09:25 PDT 2017
* lint
* update
* Sun Oct 15 22:39:55 PDT 2017
* Sun Oct 15 22:40:17 PDT 2017
* Sun Oct 15 22:43:06 PDT 2017
* Sun Oct 15 22:46:06 PDT 2017
* Sun Oct 15 22:46:21 PDT 2017
* Sun Oct 15 22:48:11 PDT 2017
* Sun Oct 15 22:48:44 PDT 2017
* Sun Oct 15 22:49:23 PDT 2017
* Sun Oct 15 22:50:21 PDT 2017
* Sun Oct 15 22:53:00 PDT 2017
* Sun Oct 15 22:53:34 PDT 2017
* Sun Oct 15 22:54:33 PDT 2017
* Sun Oct 15 22:54:50 PDT 2017
* Sun Oct 15 22:55:20 PDT 2017
* Sun Oct 15 22:56:56 PDT 2017
* Sun Oct 15 22:59:03 PDT 2017
* fix
* Update tune_mnist_ray.py
* remove script trial
* fix
* reorder
* fix ex
* py2 support
* upd
* comments
* comments
* cleanup readme
* fix trial
* annotate
* Update rllib.rst
* init pausing
* Docs, Lint
* fix danglings and restore endpoint moved to trialrunner
* renaming
* nit
* start always starts from checkpoint
* smalls
* nits
* lint
* last change
2017-10-22 23:04:15 -07:00
Eric Liang
81ca27dc08
[rllib] [minor] Rename agent_id to experiment_tag ( #1143 )
...
* tagstr
* doc
* rename
* fix test
2017-10-22 18:44:18 -07:00
Robert Nishihara
97c6369b49
Update arrow to include custom serializer for pytorch and register default serialization handlers. ( #1152 )
...
* Update arrow to include custom serializer for pytorch.
* Call pyarrow function for registering default custom serialization handlers.
* Change class ID used in serialization context for object IDs.
2017-10-21 21:24:10 -07:00
Stephanie Wang
af47737bd5
Prototype distributed actor handles ( #1137 )
...
* Add actor handle ID to the task spec
* Local scheduler dispatches actor tasks according to a task counter per handle
* Fix python test
* Allow passing actor handles into tasks. Not completely working yet. Also this is very messy.
* Fixes, should be roughly working now.
* Refactor actor handle wrapper
* Fix __init__ tests
* Terminate actor when the original handle goes out of scope
* TODO and a couple test cases
* Make tests for unsupported cases
* Fix Python mode tests
* Linting.
* Cache actor definitions that occur before ray.init() is called.
* Fix export actor class
* Deterministically compute actor handle ID
* Fix __getattribute__
* Fix string encoding for python3
* doc
* Add comment and assertion.
2017-10-19 23:49:59 -07:00
Philipp Moritz
2f45ac9e95
Make travis runs less verbose. ( #1145 )
...
* make travis runs less verbose
* update
* more -q flags
2017-10-19 22:25:56 -07:00
Robert Nishihara
8ab56b5906
Always redirect redis stdout/stderr. ( #1142 )
2017-10-19 17:09:09 -07:00
Eric Liang
782125ef3f
warn if agent failed ( #1141 )
2017-10-19 11:39:25 -07:00
Eric Liang
5a50e0e1d7
[rllib] Add the ability to run arbitrary Python scripts with ray.tune ( #1132 )
...
* fix yaml bug
* add ext agent
* gpus
* update
* tuning
* docs
* Sun Oct 15 21:09:25 PDT 2017
* lint
* update
* Sun Oct 15 22:39:55 PDT 2017
* Sun Oct 15 22:40:17 PDT 2017
* Sun Oct 15 22:43:06 PDT 2017
* Sun Oct 15 22:46:06 PDT 2017
* Sun Oct 15 22:46:21 PDT 2017
* Sun Oct 15 22:48:11 PDT 2017
* Sun Oct 15 22:48:44 PDT 2017
* Sun Oct 15 22:49:23 PDT 2017
* Sun Oct 15 22:50:21 PDT 2017
* Sun Oct 15 22:53:00 PDT 2017
* Sun Oct 15 22:53:34 PDT 2017
* Sun Oct 15 22:54:33 PDT 2017
* Sun Oct 15 22:54:50 PDT 2017
* Sun Oct 15 22:55:20 PDT 2017
* Sun Oct 15 22:56:56 PDT 2017
* Sun Oct 15 22:59:03 PDT 2017
* fix
* Update tune_mnist_ray.py
* remove script trial
* fix
* reorder
* fix ex
* py2 support
* upd
* comments
* comments
* cleanup readme
* fix trial
* annotate
* Update rllib.rst
2017-10-18 11:49:28 -07:00
Robert Nishihara
f3e3c7ec71
Add is_actor_checkpoint_method to TaskSpec. ( #1117 )
...
* Add is_actor_checkpoint_method to TaskSpec.
* Fix linting.
* Fix rebase error.
* Fix errors from rebase.
2017-10-15 16:52:10 -07:00
Eric Liang
802941994d
[rllib] Use RLlib preprocessors in DQN (fixes PongDeterministic-v4) ( #1124 )
...
* fix pong
* rename
* update
2017-10-14 20:16:36 -07:00
Stephanie Wang
15486a14a0
Refactor actor task queues ( #1118 )
...
* Refactor add_task_to_actor_queue into queue_actor_task and insert_actor_task_queue
* Refactor actor task queue to share the waiting task queue
* Fix
2017-10-13 20:52:11 -07:00
Eric Liang
79ea205b3e
[rllib] Initial work on integrating hyperparameter search tool ( #1107 )
...
* clean up train
* update
* update train script
* add tuned examples
* add agent catalog
* add tune lib
* update
* fix
* testS
* remove
* train docs
* comments
* todo
* fix resource parsing
* fix cr test
* add test
* try to fix travis test
2017-10-13 16:18:16 -07:00
Stephanie Wang
3764f2f2e1
Actor checkpointing with object lineage reconstruction ( #1004 )
...
* Worker reports error in previous task, actor task counter is incremented after task is successful
* Refactor actor task execution
- Return new task counter in GetTaskRequest
- Update worker state for actor tasks inside of the actor method
executor
* Manually invoked checkpoint method
* Scheduling for actor checkpoint methods
* Fix python bugs in checkpointing
* Return task success from worker to local scheduler instead of actor counter
* Kill local schedulers halfway through actor execution instead of waiting for all tasks to execute once
* Remove redundant actor tasks during dispatch, reconstruct missing dependencies for actor tasks
* Make executor for temporary actor methods
* doc
* Set default argument for whether the previous task was a success
* Refactor actor method call
* Simplify checkpoint task submission
* lint
* fix philipp's comments
* Add missing line
* Make actor reconstruction tests run faster
* Unimportant whitespace.
* Unimportant whitespace.
* Update checkpoint method signature
* Documentation and handle exceptions during checkpoint save/resume
* Rename get_task message field to actor_checkpoint_failed
* Fix bug.
* Remove debugging check, redirect test output
2017-10-12 09:53:32 -07:00
Richard Liaw
379b0604b4
Fixing MeanSTDFilter ( #1101 )
...
* Fixing MeanSTDFilter
* documentation
2017-10-11 18:16:08 -07:00
Philipp Moritz
0684258d2e
Update arrow to include pandas serialization ( #1102 )
...
* update arrow to include pandas serialization
* update
2017-10-10 22:16:35 -07:00
Eric Liang
b1660c4edf
[rllib] Refactor to support passing custom env_creator function ( #1096 )
...
* refactor to use env creator
* doc
* lint
2017-10-10 12:49:42 -07:00
Robert Nishihara
8f1a73f041
Allow Ray to be built without UI by setting INCLUDE_UI=0. ( #1094 )
...
* Allow building Ray without UI by setting INCLUDE_UI=0.
* Fix bash.
* Fix linting.
2017-10-09 23:32:38 -07:00
Robert Nishihara
a52a1e893f
Automatically set CUDA_VISIBLE_DEVICES when worker gets task. ( #1044 )
...
* Automatically set CUDA_VISIBLE_DEVICES when worker gets task.
* Add test.
2017-10-06 18:38:08 -07:00
Robert Nishihara
4669c59fa8
Release GPU resources as soon as an actor exits. ( #1088 )
...
* Release GPU resources as soon as an actor exits.
* Add a test.
* Store local_scheduler_id and driver_id in the worker object instead of the actor object.
2017-10-06 17:58:19 -07:00
Stephanie Wang
aebe9f9374
Fix actor garbage collection by breaking cyclic references ( #1064 )
...
* Fix bug in wait_for_pid_to_exit, add test for actor deletion.
* Fix actor garbage collection by breaking cyclic references
* Add test for calling actor method immediately after actor creation.
* Fix bug, must dispatch tasks when workers are killed.
* Fix python test
* Fix cyclic reference problem by creating ActorMethod objects on the fly.
* Try simply increasing the time allowed for many_drivers_test.py.
2017-10-05 00:55:33 -07:00
Robert Nishihara
971becc905
[rllib] Use NoFilter instead of MeanStdFilter for PPO. ( #1082 )
...
* Make NoFilter the default observation filter for PPO.
* Make reward filter NoFilter for PPO.
2017-10-04 21:31:17 -07:00
Eric Liang
6ecc899cf2
[rllib] Fix DQN checkpoint/restore and enable test in jenkins ( #1063 )
...
* fix dqn restore and add test
* Update .gitignore
* Update test_checkpoint_restore.py
* add checkpoint restore
2017-10-03 23:17:54 -07:00
Richard Liaw
cb6dea94bc
[rllib] Fix Preprocessor for ATARI ( #1066 )
...
* Removing squeeze, fix atari preprocessing
* nit comment
* comments
* jenkins
* Lint
2017-10-03 18:45:02 -07:00
Philipp Moritz
57bd1d6ff5
Specialize Serialization for OrderedDict ( #1035 )
...
Specialize Serialization for OrderedDict and defaultdict
2017-10-02 17:33:10 -07:00
Philipp Moritz
b94d85fb5d
[rllib] Fix logging to Athena ( #1058 )
...
* Fix logging to Athena
* fixes
2017-10-02 17:16:52 -07:00
Richard Liaw
54aaeb25ab
fixing ( #1048 )
2017-10-01 22:34:55 -07:00
Robert Nishihara
2892955119
Pass string into json.loads, not bytes object. ( #991 )
2017-10-01 14:56:39 -07:00
Robert Nishihara
6d393828db
Bump version number to 0.2.1. ( #1026 )
2017-10-01 12:33:13 -07:00
Robert Nishihara
ad61af7333
Workaround for passing empty list to ray.wait. ( #1043 )
...
* Workaround for passing empty list to ray.wait.
* Add test for passing empty list to wait.
2017-10-01 11:45:02 -07:00
Augusto Yao
0d90a17426
Pass cleanup argument to start_monitor. ( #1040 )
2017-09-30 15:35:25 -07:00
Wapaul1
97b3355adc
Register Class Only Creates Entry in Redis Once ( #1038 )
...
Don't export the same custom class definition multiple times.
2017-09-30 15:30:27 -07:00
Richard Liaw
16e82b43d1
[rllib] Changes for preprocessors ( #1033 )
...
* Changes for preprocessors
* removed comments
* Changes + push for lint
* linted
* adding dependency for travis
* linting won't pass
* reordering
* needed for testing
* added comments
* pip it
* pip dependencies
2017-09-30 13:11:20 -07:00
Alexey Tumanov
2d0f439b7b
hugepage + plasma directory support plumbing + documentation ( #1030 )
...
* hugepage + plasma directory support plumbing + documentation
* Indentation fix.
* huge_pages_enabled --> huge_pages
* One more change
2017-09-30 09:56:52 -07:00
Robert Nishihara
b991dc8900
Add flag for ignoring the UI, don't start UI in jenkins tests. ( #1021 )
2017-09-29 15:22:51 -07:00
Eric Liang
9f3a4fce50
[rllib] Parallelize sample collection and gradient computation in DQN ( #746 )
...
* wip
* works with cartpole
* lint
* fix pg
* comment
* action dist rename
* preprocessor
* fix test
* typo
* fix the action[0] nonsense
* revert
* satisfy the lint
* wip
* wip
* works with cartpole
* lint
* fix pg
* comment
* action dist rename
* preprocessor
* fix test
* typo
* fix the action[0] nonsense
* revert
* satisfy the lint
* Minor indentation changes.
* fix merge
* add humanoid
* initial dqn refactor
* remove tfutil
* fix calls
* fix tf errors 1
* closer
* runs now
* lint
* tensorboard graph
* fix linting
* more 4 space
* fix
* fix linT
* more lint
* oops
* es parity
* remove example.py
* fix training bug
* add cartpole demo
* try fixing cartpole
* allow model options, configure cartpole
* debug
* simplify
* no dueling
* avoid out of file handles
* Test dqn in jenkins.
* Minor formatting.
* lint
* fix py3
* fix issue
* remove chekcpoint
* revert
* Fixit
* sanity check configs
* update cuda
* fix
* parallel gradient computation
* update
* upd
* bug
* upd
* always record training stats
* fix
* comments
* revert assert
* add gpu mask
* fofset
* a tie
* Merge
* fix
* fix
* fix examples
* A3C -> DQN
* fix dqn test
* remove submodule
* fix linting
2017-09-29 00:06:51 -07:00
