Commit graph

1171 commits

Author SHA1 Message Date
Eric Liang
cd9dc398ff [rllib] Support discrete observation spaces such as FrozenLake-v0 (#1140)
* add

* remove transform_shape

* fix test

* fix
2017-10-23 23:16:52 -07:00
Richard Liaw
0c9817fa76 [tune] Tune Pausing (#1136)
* fix yaml bug

* add ext agent

* gpus

* update

* tuning

* docs

* Sun Oct 15 21:09:25 PDT 2017

* lint

* update

* Sun Oct 15 22:39:55 PDT 2017

* Sun Oct 15 22:40:17 PDT 2017

* Sun Oct 15 22:43:06 PDT 2017

* Sun Oct 15 22:46:06 PDT 2017

* Sun Oct 15 22:46:21 PDT 2017

* Sun Oct 15 22:48:11 PDT 2017

* Sun Oct 15 22:48:44 PDT 2017

* Sun Oct 15 22:49:23 PDT 2017

* Sun Oct 15 22:50:21 PDT 2017

* Sun Oct 15 22:53:00 PDT 2017

* Sun Oct 15 22:53:34 PDT 2017

* Sun Oct 15 22:54:33 PDT 2017

* Sun Oct 15 22:54:50 PDT 2017

* Sun Oct 15 22:55:20 PDT 2017

* Sun Oct 15 22:56:56 PDT 2017

* Sun Oct 15 22:59:03 PDT 2017

* fix

* Update tune_mnist_ray.py

* remove script trial

* fix

* reorder

* fix ex

* py2 support

* upd

* comments

* comments

* cleanup readme

* fix trial

* annotate

* Update rllib.rst

* init pausing

* Docs, Lint

* fix danglings and restore endpoint moved to trialrunner

* renaming

* nit

* start always starts from checkpoint

* smalls

* nits

* lint

* last change
2017-10-22 23:04:15 -07:00
Eric Liang
81ca27dc08 [rllib] [minor] Rename agent_id to experiment_tag (#1143)
* tagstr

* doc

* rename

* fix test
2017-10-22 18:44:18 -07:00
Robert Nishihara
97c6369b49 Update arrow to include custom serializer for pytorch and register default serialization handlers. (#1152)
* Update arrow to include custom serializer for pytorch.

* Call pyarrow function for registering default custom serialization handlers.

* Change class ID used in serialization context for object IDs.
2017-10-21 21:24:10 -07:00
Philipp Moritz
684e62e784 upgrade arrow to include numpy bool fix (#1148) 2017-10-20 17:25:15 -07:00
Peter Schafhalter
ad4cbd4016 Updated outstanding_callbacks to unordered_map (#1108)
* Updated outstanding_callbacks to unordered_map

* Fix bug in destroy_outstanding_callbacks and comments
2017-10-20 10:06:22 -07:00
Stephanie Wang
af47737bd5 Prototype distributed actor handles (#1137)
* Add actor handle ID to the task spec

* Local scheduler dispatches actor tasks according to a task counter per handle

* Fix python test

* Allow passing actor handles into tasks. Not completely working yet. Also this is very messy.

* Fixes, should be roughly working now.

* Refactor actor handle wrapper

* Fix __init__ tests

* Terminate actor when the original handle goes out of scope

* TODO and a couple test cases

* Make tests for unsupported cases

* Fix Python mode tests

* Linting.

* Cache actor definitions that occur before ray.init() is called.

* Fix export actor class

* Deterministically compute actor handle ID

* Fix __getattribute__

* Fix string encoding for python3

* doc

* Add comment and assertion.
2017-10-19 23:49:59 -07:00
Philipp Moritz
2f45ac9e95 Make travis runs less verbose. (#1145)
* make travis runs less verbose

* update

* more -q flags
2017-10-19 22:25:56 -07:00
Robert Nishihara
8ab56b5906 Always redirect redis stdout/stderr. (#1142) 2017-10-19 17:09:09 -07:00
Robert Nishihara
1cdc2fb011 Clean up event loop and callbacks when processes exit. (#1125)
* Clean up event loop and callbacks when processes exit.

* Fix bug.
2017-10-19 17:07:03 -07:00
Eric Liang
782125ef3f warn if agent failed (#1141) 2017-10-19 11:39:25 -07:00
Eric Liang
5a50e0e1d7 [rllib] Add the ability to run arbitrary Python scripts with ray.tune (#1132)
* fix yaml bug

* add ext agent

* gpus

* update

* tuning

* docs

* Sun Oct 15 21:09:25 PDT 2017

* lint

* update

* Sun Oct 15 22:39:55 PDT 2017

* Sun Oct 15 22:40:17 PDT 2017

* Sun Oct 15 22:43:06 PDT 2017

* Sun Oct 15 22:46:06 PDT 2017

* Sun Oct 15 22:46:21 PDT 2017

* Sun Oct 15 22:48:11 PDT 2017

* Sun Oct 15 22:48:44 PDT 2017

* Sun Oct 15 22:49:23 PDT 2017

* Sun Oct 15 22:50:21 PDT 2017

* Sun Oct 15 22:53:00 PDT 2017

* Sun Oct 15 22:53:34 PDT 2017

* Sun Oct 15 22:54:33 PDT 2017

* Sun Oct 15 22:54:50 PDT 2017

* Sun Oct 15 22:55:20 PDT 2017

* Sun Oct 15 22:56:56 PDT 2017

* Sun Oct 15 22:59:03 PDT 2017

* fix

* Update tune_mnist_ray.py

* remove script trial

* fix

* reorder

* fix ex

* py2 support

* upd

* comments

* comments

* cleanup readme

* fix trial

* annotate

* Update rllib.rst
2017-10-18 11:49:28 -07:00
Philipp Moritz
4157bcb80b Improve deserialization performance by rebasing on latest arrow (#1129)
* improve serialization performance by rebasing on latest arrow

* update

* revert worker.py
2017-10-17 14:56:11 -07:00
Robert Nishihara
004ffe7e21 Edits to serialization blog post. (#1131)
* Fix typo.

* Move API section further up, and add specific commit for experiments.

* Improve wording.

* Modify figures.
2017-10-16 00:03:22 -07:00
Philipp Moritz
e5a57a7ce4 Blog post on Serialization and Apache Arrow Integration (#1130)
* Add initial blog post on serialization libraries.

* Shrink PNGs.

* Small rewording.

* Link to pyarrow serialization documentation.

* More rewordings and authors.

* Undo accidental change.

* Small fixes.

* Add code and other minor fixes.

* Add note.
2017-10-15 22:13:36 -07:00
Robert Nishihara
f3e3c7ec71 Add is_actor_checkpoint_method to TaskSpec. (#1117)
* Add is_actor_checkpoint_method to TaskSpec.

* Fix linting.

* Fix rebase error.

* Fix errors from rebase.
2017-10-15 16:52:10 -07:00
Eric Liang
802941994d [rllib] Use RLlib preprocessors in DQN (fixes PongDeterministic-v4) (#1124)
* fix pong

* rename

* update
2017-10-14 20:16:36 -07:00
Robert Nishihara
d6062ef8f6 Compile with -rdynamic for better debugging symbols. (#1123)
* Compile with -rdynamic.

* Only use -rdynamic on Linux.

* Add comment.
2017-10-13 21:39:11 -07:00
Stephanie Wang
15486a14a0 Refactor actor task queues (#1118)
* Refactor add_task_to_actor_queue into queue_actor_task and insert_actor_task_queue

* Refactor actor task queue to share the waiting task queue

* Fix
2017-10-13 20:52:11 -07:00
Eric Liang
79ea205b3e [rllib] Initial work on integrating hyperparameter search tool (#1107)
* clean up train

* update

* update train script

* add tuned examples

* add agent catalog

* add tune lib

* update

* fix

* testS

* remove

* train docs

* comments

* todo

* fix resource parsing

* fix cr test

* add test

* try to fix travis test
2017-10-13 16:18:16 -07:00
Robert Nishihara
486cb64e3f Compile with -Werror and -Wall (#1116)
* Compile global scheduler with -Werror -Wall.

* Compile plasma manager with -Werror -Wall.

* Compile local scheduler with -Werror -Wall.

* Compile common code with -Werror -Wall.

* Signed/unsigned comparisons.

* More signed/unsigned fixes.

* More signed/unsigned fixes and added extern keyword.

* Fix linting.

* Don't check strict-aliasing because Python.h doesn't pass.
2017-10-12 21:00:23 -07:00
Stephanie Wang
3764f2f2e1 Actor checkpointing with object lineage reconstruction (#1004)
* Worker reports error in previous task, actor task counter is incremented after task is successful

* Refactor actor task execution

- Return new task counter in GetTaskRequest
- Update worker state for actor tasks inside of the actor method
  executor

* Manually invoked checkpoint method

* Scheduling for actor checkpoint methods

* Fix python bugs in checkpointing

* Return task success from worker to local scheduler instead of actor counter

* Kill local schedulers halfway through actor execution instead of waiting for all tasks to execute once

* Remove redundant actor tasks during dispatch, reconstruct missing dependencies for actor tasks

* Make executor for temporary actor methods

* doc

* Set default argument for whether the previous task was a success

* Refactor actor method call

* Simplify checkpoint task submission

* lint

* fix philipp's comments

* Add missing line

* Make actor reconstruction tests run faster

* Unimportant whitespace.

* Unimportant whitespace.

* Update checkpoint method signature

* Documentation and handle exceptions during checkpoint save/resume

* Rename get_task message field to actor_checkpoint_failed

* Fix bug.

* Remove debugging check, redirect test output
2017-10-12 09:53:32 -07:00
Robert Nishihara
b585001881 When a task is passed to the global scheduler, if it is not received,… (#1106)
* When a task is passed to the global scheduler, if it is not received, then try again.
* Call give_task_to_global_scheduler directly (same with local).
2017-10-12 00:04:38 -07:00
Philipp Moritz
6109cc0782 Add instructions to clean the source tree (#1114)
* add instructions to clean the source tree

* update
2017-10-11 23:13:07 -07:00
Robert Nishihara
7a954f4b5f Use monotonic clock for some python tests. (#1112) 2017-10-11 19:58:59 -07:00
Robert Nishihara
9f1e385335 Return errno from handle_sigpipe. (#1051) 2017-10-11 18:36:28 -07:00
Richard Liaw
379b0604b4 Fixing MeanSTDFilter (#1101)
* Fixing MeanSTDFilter

* documentation
2017-10-11 18:16:08 -07:00
Peter Schafhalter
46f6c163dc Converted ClientConnection to C++ standard library (#1099) 2017-10-11 11:12:15 -07:00
Stephanie Wang
1e0ab3d386 Switch to monotonic clock (#1100) 2017-10-10 22:35:21 -07:00
Philipp Moritz
0684258d2e Update arrow to include pandas serialization (#1102)
* update arrow to include pandas serialization

* update
2017-10-10 22:16:35 -07:00
Eric Liang
b1660c4edf [rllib] Refactor to support passing custom env_creator function (#1096)
* refactor to use env creator

* doc

* lint
2017-10-10 12:49:42 -07:00
Robert Nishihara
1837824881 Don't install boost when building on Linux. (#1095) 2017-10-09 23:39:03 -07:00
Robert Nishihara
8f1a73f041 Allow Ray to be built without UI by setting INCLUDE_UI=0. (#1094)
* Allow building Ray without UI by setting INCLUDE_UI=0.

* Fix bash.

* Fix linting.
2017-10-09 23:32:38 -07:00
Robert Nishihara
3944e9a450 Upload wheels to S3 from Travis. (#1097)
* Upload wheels to S3 from Travis.

* Make wheels publicly readable and also only deploy from wheel builds.
2017-10-09 23:10:58 -07:00
Eric Liang
90013eda2d [rllib] Fix docs to reference new code locations (#1092)
* fix rllib docs

* Update example-a3c.rst
2017-10-09 22:58:58 -07:00
Robert Nishihara
a52a1e893f Automatically set CUDA_VISIBLE_DEVICES when worker gets task. (#1044)
* Automatically set CUDA_VISIBLE_DEVICES when worker gets task.

* Add test.
2017-10-06 18:38:08 -07:00
Robert Nishihara
4669c59fa8 Release GPU resources as soon as an actor exits. (#1088)
* Release GPU resources as soon as an actor exits.

* Add a test.

* Store local_scheduler_id and driver_id in the worker object instead of the actor object.
2017-10-06 17:58:19 -07:00
Stephanie Wang
aebe9f9374 Fix actor garbage collection by breaking cyclic references (#1064)
* Fix bug in wait_for_pid_to_exit, add test for actor deletion.

* Fix actor garbage collection by breaking cyclic references

* Add test for calling actor method immediately after actor creation.

* Fix bug, must dispatch tasks when workers are killed.

* Fix python test

* Fix cyclic reference problem by creating ActorMethod objects on the fly.

* Try simply increasing the time allowed for many_drivers_test.py.
2017-10-05 00:55:33 -07:00
Robert Nishihara
971becc905 [rllib] Use NoFilter instead of MeanStdFilter for PPO. (#1082)
* Make NoFilter the default observation filter for PPO.

* Make reward filter NoFilter for PPO.
2017-10-04 21:31:17 -07:00
shane
a72084c568 adding -x flag for better debugging during builds (#1079) 2017-10-04 13:56:14 -07:00
Eric Liang
6ecc899cf2 [rllib] Fix DQN checkpoint/restore and enable test in jenkins (#1063)
* fix dqn restore and add test

* Update .gitignore

* Update test_checkpoint_restore.py

* add checkpoint restore
2017-10-03 23:17:54 -07:00
Mitar
a0d3fb1de1 Fix Arrow's repository URL. (#1072)
Thanks!
2017-10-03 21:40:21 -07:00
Richard Liaw
cb6dea94bc [rllib] Fix Preprocessor for ATARI (#1066)
* Removing squeeze, fix atari preprocessing

* nit comment

* comments

* jenkins

* Lint
2017-10-03 18:45:02 -07:00
Robert Nishihara
0dcf36c91e Switch Arrow commit. (#1068) 2017-10-03 13:56:53 -07:00
Robert Nishihara
c2f5b4c2ed Add a 'Getting Involved' section to Readme. (#1067) 2017-10-03 10:23:47 -07:00
Philipp Moritz
57bd1d6ff5 Specialize Serialization for OrderedDict (#1035)
Specialize Serialization for OrderedDict and defaultdict
2017-10-02 17:33:10 -07:00
Philipp Moritz
b94d85fb5d [rllib] Fix logging to Athena (#1058)
* Fix logging to Athena

* fixes
2017-10-02 17:16:52 -07:00
Robert Nishihara
1488975d1b Add timing statement to loop that calls redis_get_cached_db_client be… (#1045)
* Add timing statement to loop that calls redis_get_cached_db_client because it has been slow in the past.

* Fix linting.

* Refactoring to make manager vectors into std::vector.

* Fix linting.

* Fixes.
2017-10-02 10:46:21 -07:00
Robert Nishihara
a31d138f21 Don't log when a worker can't be started. (#1056) 2017-10-02 10:32:46 -07:00
Richard Liaw
54aaeb25ab fixing (#1048) 2017-10-01 22:34:55 -07:00