Commit graph

7507 commits

Author SHA1 Message Date
Eric Liang
e2f2a7e57a [rllib] Pick preprocessor based on obs shape (#855)
* update

* auto choose
2017-08-22 16:46:55 -07:00
Eric Liang
c81821b856 [rllib] Make Pong-v0 + EvolutionStrategies work by sharing preprocessors with PPO (#848)
* fix by sharing preprocessors

* revert param changeg

* Update evolution_strategies.py

* Update catalog.py
2017-08-21 18:51:49 -07:00
Robert Nishihara
be4beb19c1 Changes to build to fix creation of wheels. (#840)
* Pass DPYTHON_EXECUTABLE into cmake for arrow and for ray.

* Add cython to setup.py install_requires.

* Revert custom code for finding python in cmake.

* Correctly find arrow on CentOS.

* In cmake, don't find PythonLibs, just find PYTHON_INCLUDE_DIRS.

* Fix typo.

* Do not use boost shared libraries when building arrow.

* Add six to the setup.py install_requires because it is needed by pyarrow.

* Don't link numbuf against boost_system and boost_filesystem.

* Compile boost when we are on Linux.

* Make numbuf find the correct boost libraries.

* Only use find_package Boost on Linux, suppress output when building boost.

* Changes to wheel building scripts, install cython in mac script.

* Compile flatbuffers ourselves on Linux and pass it in when compiling Arrow.

* Clean up build_flatbuffers.sh and build_boost.sh scripts a little.

* Install cython when building linux wheel.
2017-08-21 17:49:35 -07:00
Robert Nishihara
cf41964816 Fix some bugs causing Travis test failures. (#839)
* Fix bug in which worker has no actor_class attribute.

* Remove case where we check if processes are defunct.
2017-08-16 01:18:32 -07:00
Robert Nishihara
ca53e9ae7b Fix bugs in task timeline visualization. (#836)
* Fix bugs in task timeline visualization.

* Some cleanups.

* Remove print statements.
2017-08-13 23:39:37 -07:00
Robert Nishihara
a75ccc8032 Fix socket error exception in Python 2. (#833) 2017-08-13 23:14:26 -07:00
alanamarzoev
bfe473fa8c Embedded task trace with object dependencies. (#818)
* Embedded timeline

* Yeah

* Fixed arrows not showing up.

* Fixed arrows not showing up, and added check boxes for the kinds of dependencies that should be included in the trace.

* first

* Fixes

* Fixed typo in comments, added more comments. fixed linting.

* Added more comments.

* Formatting.

* fixes

* Fixed state.py linting.

* Fixed ui.py linting errors.

* Fixed linting errors.

* Renamed task dependencies and included instructions for viewing arrows.

* Fixed according to PR comments.

* Fixed bug.

* Undid changes to metadata blocks.

* Fixes according to comments.

* Fixed linting.

* Fixed linting.

* NOQA keyword added to link line.
2017-08-09 23:00:14 -07:00
Alexey Tumanov
fc885bd918 Adding basic support for a user-interpretable resource label (#761)
* adding support for the user-interpretable label(UIR)

* more plumbing for num_uirs further upstream; set to infty when specified on cmd line

* pass default num_uirs for actors; update GlobalStateAPI

* support num_uirs in ray.init()

* local scheduler resource accounting: support num_uirs; prep for vectorized resource accounting

* global scheduler test updated

* Fix bug introduced by rebase.

* Rename UIR -> CustomResource and add test.

* Small changes and use constexpr instead of macros.

* Linting and some renaming.

* Reorder some code.

* Remove cpus_in_use and fix bug.

* Add another test and make a small change.

* Rephrase documentation about feature stability.
2017-08-08 02:53:59 -07:00
Robert Nishihara
03f2325780 Package pyarrow along with ray. (#822)
* Rough pass at installing pyarrow along with Ray.

* Remove hardcoded path and try to find correct path automatically.

* Add print.

* Fix linting.

* Copy pyarrow files to a location that we manually add to python path in order to avoid interfering with pre-existing pyarrow installations.

* Move call to build.sh back into build_ext in setup.py.

* Ignore some linting errors.

* Fix problem in which pyarrow files to copy were listed before they were built.

* Fix tests by importing ray before pyarrow.
2017-08-07 21:17:28 -07:00
Philipp Moritz
862e56000b [rllib] Unify RLLib examples and add jenkins test for policy gradients (#815)
* add jenkins test

* correct handling of the number of iterations

* convert policy gradient and evolution strategies script

* convert DQN

* fix A3C

* fix

* fix

* fixes

* remove redundant A3C example
2017-08-07 19:05:48 -07:00
Robert Nishihara
dbe3d9351c Prototype actor checkpointing. (#814)
* Initial testing of checkpointing functions.

* Save checkpoints in Redis.

* Pipe checkpoint_interval through remote decorator.

* Add a test.

* Small cleanups.

* Submit dummy tasks when reconstructing tasks before the most recent tasks so that we don't end up reconstructing the arguments for those tasks.

* Remove old checkpoints to save space.

* Fix linting.
2017-08-07 17:52:39 -07:00
Philipp Moritz
0225581078 [rllib] Improve performance for small rollouts (#812)
* batch small rollouts together

* implement minimum number of samples for each task

* add total time

* fix linting

* style

* fix

* factor out parameters and document stuff

* add rollout batchsize

* address comments

* linting

* small fix
2017-08-05 22:13:30 -07:00
alanamarzoev
64eaaaebf0 Show timeline button. (#809) 2017-08-03 20:11:50 -07:00
Richard Liaw
c30fdb4ab0 [rllib] Code for Supporting Shared Models (#775)
* Code for Supporting Shared Models

* Running (with vnet modification) - needs to be tested for performance

* Small fix for jenkins

* Linting

* linting

* Summaries

* Small refactoring + generalized to more domains

* Addressing changes

* Addressing changes

* Update envs.py

* Addressing changes

* convnet

* final touches

* Merge - new model

* final linting

* Changing iterations back

* Policy option removed, fixed small things

* Nits

* nit

* Linting

* Linting
2017-08-03 19:29:01 -07:00
Philipp Moritz
df65e87fc7 [rllib] Tune ppo more on control tasks (#777)
* tune ppo on control tasks

* introduce free log_std

* fix

* flag for writing logs

* fixes

* fixes
2017-08-03 16:34:06 -07:00
alanamarzoev
99badc7ae4 UI functions in separate file. (#801)
* UI file.

* Fixed linting.

* Change UI instructions slightly.
2017-08-02 19:32:18 -07:00
Robert Nishihara
cb84972f6b Recreate actors when local schedulers die. (#804)
* Reconstruct actor state when local schedulers fail.

* Simplify construction of arguments to pass into default_worker.py from local scheduler.

* Remove deprecated ray.actor.

* Simplify actor reconstruction method.

* Fix linting.

* Small fixes.
2017-08-02 18:02:52 -07:00
Robert Nishihara
fcd07b10b5 Don't call colorama.init(). (#799) 2017-08-01 18:58:48 -07:00
Robert Nishihara
8c8258de20 Move worker methods into Worker class and expose more TaskSpec fields to Python. (#796)
* Move worker methods inside worker class. Move some helper methods from actor.py into utils.py and state.py.

* Add more methods exposing task spec fields to Python.

* Fix linting.

* Fix error.

* Remove unused code in default worker.
2017-08-01 17:16:57 -07:00
Robert Nishihara
52a27be364 Better logging in tests. (#790) 2017-07-31 22:30:46 -07:00
Philipp Moritz
c3b39b4d86 Pull Plasma from Apache Arrow and remove Plasma store from Ray. (#692)
* Rebase Ray on top of Plasma in Apache Arrow

* add thirdparty building scripts

* use rebased arrow

* fix

* fix build

* fix python visibility

* comment out C tests for now

* fix multithreading

* fix

* reduce logging

* fix plasma manager multithreading

* make sure old and new object IDs can coexist peacefully

* more rebasing

* update

* fixes

* fix

* install pyarrow

* install cython

* fix

* install newer cmake

* fix

* rebase on top of latest arrow

* getting runtest.py run locally (needed to comment out a test for that to work)

* work on plasma tests

* more fixes

* fix local scheduler tests

* fix global scheduler test

* more fixes

* fix python 3 bytes vs string

* fix manager tests valgrind

* fix documentation building

* fix linting

* fix c++ linting

* fix linting

* add tests back in

* Install without sudo.

* Set PKG_CONFIG_PATH in build.sh so that Ray can find plasma.

* Install pkg-config

* Link -lpthread, note that find_package(Threads) doesn't seem to work reliably.

* Comment in testGPUIDs in runtest.py.

* Set PKG_CONFIG_PATH when building pyarrow.

* Pull apache/arrow and not pcmoritz/arrow.

* Fix installation in docker image.

* adapt to changes of the plasma api

* Fix installation of pyarrow module.

* Fix linting.

* Use correct python executable to build pyarrow.
2017-07-31 21:04:15 -07:00
alanamarzoev
dfcd399dbb Cluster heat map. (#792) 2017-07-31 20:49:31 -07:00
alanamarzoev
a2852f2329 Allow multiple web UIs to be open at the same time. (#784)
* Allow multiple web UIs to be open at the same time.

* Changed formatting.
2017-07-31 18:13:24 -07:00
Robert Nishihara
37dafa4d14 Simplify put test and move it to failure tests. (#788) 2017-07-31 17:57:48 -07:00
Robert Nishihara
c394a65ffc Wait longer when getting redis shards to initialize global state API. (#786) 2017-07-31 17:56:11 -07:00
Eric Liang
b6a18cb39b [rllib] Also refactor DQN to use shared RLlib models (#730)
* wip

* works with cartpole

* lint

* fix pg

* comment

* action dist rename

* preprocessor

* fix test

* typo

* fix the action[0] nonsense

* revert

* satisfy the lint

* wip

* works with cartpole

* lint

* fix pg

* comment

* action dist rename

* preprocessor

* fix test

* typo

* fix the action[0] nonsense

* revert

* satisfy the lint

* Minor indentation changes.

* fix merge

* add humanoid

* initial dqn refactor

* remove tfutil

* fix calls

* fix tf errors 1

* closer

* runs now

* lint

* tensorboard graph

* fix linting

* more 4 space

* fix

* fix linT

* more lint

* oops

* es parity

* remove example.py

* fix training bug

* add cartpole demo

* try fixing cartpole

* allow model options, configure cartpole

* debug

* simplify

* no dueling

* avoid out of file handles

* Test dqn in jenkins.

* Minor formatting.

* fix issue

* fix another

* Fix problem in which we log to a directory that hasn't been created.
2017-07-26 12:29:00 -07:00
alanamarzoev
0f0acb8ac1 CPU Time Series. (#765)
Add time series of CPU utilization to web UI.
2017-07-26 00:15:50 -07:00
Robert Nishihara
ff996330e8 If a worker dies unexpectedly, then let it exit. (#762) 2017-07-21 06:36:25 +00:00
Robert Nishihara
13000b7503 Start processes using the same version of Python that was used to start Ray. (#760)
* Make local scheduler start workers using the same version of Python that was used to start the local scheduler.

* Use current version of python to start new processes instead of hardcoded python executable.

* Fix linting.
2017-07-21 00:05:10 +00:00
alanamarzoev
c31c20ca9c Code toggling instructions. (#757) 2017-07-20 10:51:33 -07:00
alanamarzoev
853b2913b7 Task duration distribution plot. (#743)
* Task duration distribution plot.

* Fixed bug.

* Changed axis labels.

* Modify task start point.

* Modified task_profiles func to decode in ascii.

* Nvm

* Changed to double quotes and added comments.

* fixed linting

* Fixed linting.

* Fixed bug.
2017-07-19 23:15:17 -07:00
Philipp Moritz
d356dd3ec4 [rllib] Expose algorithm parameters and tune policy gradient parameters for humanoid (#753)
* parameters for humanoid

* fix
2017-07-19 16:45:05 -07:00
Philipp Moritz
ade6d80820 [rllib] use ray.wait to speed up parallel simulations for policy gradients (#754)
* use ray.wait to speed up parallel simulations for policy gradients

* linting
2017-07-19 16:09:15 -07:00
alanamarzoev
2b3190ad13 Chrome trace timeline with sliders. (#731)
* Trace timeline with sliders.

* Trace.

* Switched ujson to json.

* Fixed tests.

* linting fixes

* Fixed bug.

* Cleaned up code.

* Fixes according to comments.

* removed checkpoints.

* Undid accidental delete.

* Fixed linting error.

* Added documentation to notebook.

* Undid accidental deletes.

* Add comments and small formatting fixes.

* Small fix.
2017-07-17 19:59:49 -07:00
Eric Liang
420013774c [rllib] Pull out shared models for evolution strategies and policy gradient. (#719)
* wip

* works with cartpole

* lint

* fix pg

* comment

* action dist rename

* preprocessor

* fix test

* typo

* fix the action[0] nonsense

* revert

* satisfy the lint

* wip

* works with cartpole

* lint

* fix pg

* comment

* action dist rename

* preprocessor

* fix test

* typo

* fix the action[0] nonsense

* revert

* satisfy the lint

* Minor indentation changes.

* fix merge

* add humanoid

* fix linting

* more 4 space

* fix

* fix linT

* oops

* es parity
2017-07-17 08:58:54 +00:00
Eric Liang
86a7909149 make es worker count independent (#740) 2017-07-16 16:23:56 -07:00
Robert Nishihara
80e8426b5e Test example applications and rllib in jenkins tests. (#707)
* Test example applications in Jenkins.

* Fix default upload_dir argument for Algorithm class.

* Fix evolution strategies.

* Comment out policy gradient example which doesn't seem to work.

* Set --env-name for evolution strategies.
2017-07-16 18:51:33 +00:00
Robert Nishihara
e0867c8845 Switch Python indentation from 2 spaces to 4 spaces. (#726)
* 4 space indentation for actor.py.

* 4 space indentation for worker.py.

* 4 space indentation for more files.

* 4 space indentation for some test files.

* Check indentation in Travis.

* 4 space indentation for some rl files.

* Fix failure test.

* Fix multi_node_test.

* 4 space indentation for more files.

* 4 space indentation for remaining files.

* Fixes.
2017-07-13 21:53:57 +00:00
Philipp Moritz
c24c07613c [rllib] unify writing performance metrics and make it queryable (#708)
* write config to s3

* add train file

* write performance to S3

* writing needs to be fixed, replacing result.json at the moment

* update

* add experiment_id

* more logging and example queries

* update

* add info

* fill in other algorithms

* fix linting

* convert readme to rst

* fixes

* simplejson -> json

* make files executable

* edit README.rst

* unify storing logs in S3 and on local filesystem

* use 'info' entry in TrainingResult for algorithm specific info

* don't install smart_open with ray

* fixes

* linting fixes
2017-07-11 01:36:14 +02:00
alanamarzoev
8464d77c76 Change event logs to store one Redis ZSET per worker. (#705)
* Changing to zset

* Fixed bug.

* Fixed another bug.

* Modified task_profiles.

* Removed extra file.

* Modified task_profiles test.

* WIP

* WIP

* Undid changes

* Updated

* WIP

* Made changes according to comments.

* Removed unneeded print.

* Removed ujson usage.

* failing test

* tests passing

* Fixed linting errors and modified style.

* Fixed bug.

* Fixed linting

* Fixed according to comments.

* Redis crashing?

* Fixed linting

* Fixed linting
2017-07-09 01:42:29 +02:00
Eric Liang
cd12ea7e09 [rllib] Pull out the GPU-parallel optimizer from policy gradients to common (#711)
* refactor

* docs

* cleanup

* clean up more

* Update parallel.py

* add imports from future
2017-07-07 22:20:02 +00:00
Robert Nishihara
5b3d0c00f2 Create /tmp/ray directory in services.py. (#715) 2017-07-07 18:41:56 +00:00
Eric Liang
f012e597c2 [rllib] Basic port of baselines/deepq to rllib (#709)
* rllib v0

* fix imports

* lint

* comments

* update docs

* a3c wip

* a3c wip

* report stats

* update doc

* add common logdir attr

* name is too long

* fix small bug

* propagate exception on error

* fetch metrics

* initial port

* fix lint

* add right license

* port to common alg format

* fix lint

* rename dqn

* add imports from future

* fix lint
2017-07-07 18:37:00 +00:00
Eric Liang
66734847bb [rllib] Standardize writing output logs and other files to /tmp/ray (#706)
* rllib v0

* fix imports

* lint

* comments

* update docs

* a3c wip

* a3c wip

* report stats

* update doc

* add common logdir attr

* name is too long

* fix small bug

* propagate exception on error

* fetch metrics

* fix small nits
2017-07-03 16:01:47 +00:00
alanamarzoev
2b11a7bca2 Add task ID and object ID search boxes to web UI. (#704)
* Task search box.

* Cleaned up.

* Small reformatting.

* Add object table search box.
2017-07-01 17:48:23 -04:00
alanamarzoev
716469160e Enable dumping profiling information to timeline format viewable by chrome tracing. (#703)
* Chrome tracing timeline.

* Modified decode statement.

* Some cleanups and add test.

* Remove example.

* Fix test.
2017-06-30 12:14:11 -04:00
Eric Liang
2d81edfcdc [rllib] Move a3c implementation from examples/ to python/ray/rllib/ (#698)
* rllib v0

* fix imports

* lint

* comments

* update docs

* a3c wip

* a3c wip

* report stats

* update doc

* name is too long

* fix small bug

* propagate exception on error

* fetch metrics

* fix lint
2017-06-29 15:49:56 +00:00
Robert Nishihara
efce49cfbc Bump version to 0.1.2 in preparation for uploading wheels to PyPI. (#700) 2017-06-27 04:35:42 +00:00
Eric Liang
a674ec958c [rllib] Move policy gradient and evolution strategies algorithms from examples/ to ray/rllib/ (#694)
* rllib v0

* fix imports

* lint

* comments

* update docs
2017-06-25 22:13:03 +00:00
Robert Nishihara
8bc9c275fa Increase the number of log file names and handle errors better in log monitor. (#693) 2017-06-25 05:20:50 +00:00