Commit graph

507 commits

Author SHA1 Message Date
Eric Liang
87e107edd8 [tune] Sync logs from workers and improve tensorboard reporting (#1567) 2018-02-26 11:35:51 -08:00
Devin Petersohn
1fa59f1887 [DataFrame] Adding insert, set_axis, set_index, reset_index and tests (#1603) 2018-02-26 08:58:15 -08:00
Richard Liaw
c2ad800cbf
[rllib] Registry fix for DQN Replay Evaluators (#1593) 2018-02-25 22:30:11 -08:00
Robert Nishihara
00ef2f6285 Only pass in --allow-root if the user is root. (#1594) 2018-02-25 13:40:51 -08:00
Devin Petersohn
529397b35e [DataFrames] Updating Index implementation, performance improvements (#1598) 2018-02-25 13:39:28 -08:00
Richard Liaw
31fefa20b7
[tune] HyperBand Fixes (#1586) 2018-02-25 13:26:58 -08:00
Robert Nishihara
5859a2d249 Replace python setup.py install with pip install -e . (#1460) 2018-02-22 11:15:03 -08:00
Robert Nishihara
f4b1881fec Update arrow to use updated pandas serializer. (#1582) 2018-02-22 11:10:52 -08:00
Robert Nishihara
330159d8bd Allow setting redis shard ports through ray start (also object store memory). (#1581)
* Allow passing in --object-store-memory to ray start.

* Allow setting ports for the redis shards.

* Reorder arguments and infer number of shards from ports.

* Move code block into only the head node case.

* Add test.
2018-02-22 11:05:37 -08:00
Robert Nishihara
a3b44309dd Pass --allow-root into jupyter notebook. (#1571) 2018-02-21 22:42:19 -08:00
Robert Nishihara
6a8661a014 Reduce verbosity of log monitor. (#1569) 2018-02-21 22:40:16 -08:00
Richard Liaw
e62ad7007d
[autoscaler] Improve UX for Autoscaler (#1558) 2018-02-21 22:19:04 -08:00
Devin Petersohn
de6fa02c85 [DataFrame] Fix transpose with nan values and add functionality needed for Index (#1545) 2018-02-21 08:46:37 -08:00
Eric Liang
1b596f7d3b
[rllib] Rollout script needs to pipe in config and update states (#1566)
* Mon Feb 19 15:20:09 PST 2018

* fix it actually
2018-02-20 12:04:41 -08:00
Helen Che
fd03fb967f [DataFrame] Implement iteritems, items, itertuples, and iterrows. (#1543)
* items

* Can't pickle generator so return list

* Add itterows method

* Finish flak8

* Add itertuples

* Some changes

* Add iter tests to mixed types test

* Appease flake8
2018-02-20 10:07:36 -08:00
Richard Liaw
1cd2703cac
[autoscaler] Docker Support (#1505) 2018-02-20 00:24:01 -08:00
Richard Liaw
0f766ae24b
[rllib] Fix testGetFilters in A3C (#1557) 2018-02-19 22:44:14 -08:00
Richard Liaw
73be235701
Quick Fix for Killing Ray Notebooks (#1563) 2018-02-19 16:10:37 -08:00
Eric Liang
ab37d0cd19 Allow ray.init with localhost address (#1556) 2018-02-18 16:36:58 -08:00
Eric Liang
4a6cfee887 [rllib] add tuned example for pendulum (#1552) 2018-02-18 00:46:42 -08:00
Robert Nishihara
61d8a17de0 [rllib] Change NotImplemented -> NotImplementedError. (#1535) 2018-02-16 17:08:25 -08:00
Hari Subbaraj
8d1a0b0d04 [DataFrame] Dataframe functions (max, min, notnull, notna) (#1500)
* Finished max, min, notna, notnull

* flake8 satisfied

* fixed pytest fixture error

* flake8 sufficed

* post-code review

* added methods to new mixed types test
2018-02-16 14:00:59 -08:00
Simon Mo
a24cc28773 [DataFrame] Add Parquet Support in Build Process (#1531)
* Add shell script for building parquet

* Use parquet ci script; remove anaconda

* Remove gcc flag, use default

* add boost_root

* Fix $TP_DIR reference issue

* fix the PR

* check out specific parquet-cpp commit
2018-02-16 07:18:42 -08:00
Alexey Tumanov
844a6afcdd Implement simple random spillback policy. (#1493)
* spillback policy implementation: global + local scheduler

* modernize global scheduler policy state; factor out random number engine and generator

* Minimal version.

* Fix test.

* Make load balancing test less strenuous.
2018-02-13 00:09:35 -08:00
William Paul
f2b6a7b58d Polished TensorFlowVariables code and documentation (#566) 2018-02-12 15:38:58 -08:00
Eric Liang
ca0f08d100 [tune] Recover experiments from last checkpoint (#1532) 2018-02-12 14:01:19 -08:00
Eric Liang
7e998db656
[rllib] Reduce concat memory usage, allow object store memory to be specified in init (#1529)
* c

* stop agents

* comment

* Sat Feb 10 02:33:30 PST 2018

* Sat Feb 10 02:33:39 PST 2018

* Update sample_batch.py

* Sun Feb 11 14:38:55 PST 2018

* add ppo config warn
2018-02-11 19:14:51 -08:00
Eric Liang
b6a06b81ed
[tune] Allow perturbations of categorical variables (#1507)
* categorical perturb

* Sat Feb  3 00:28:13 PST 2018

* explicitly clean up nested actors

* Sat Feb 10 02:21:57 PST 2018

* Sat Feb 10 02:22:29 PST 2018
2018-02-11 18:20:27 -08:00
eugenevinitsky
639df85fda updated multiagent docs (#1523)
* updated multiagent docs

* Update rllib.rst

* Update multiagent_mountaincar_env.py

* Update multiagent_pendulum_env.py
2018-02-11 16:35:03 -08:00
alvkao58
81a4be8f65 [rllib] Added vanilla policy gradient (#1497) 2018-02-10 13:54:51 -08:00
Helen Che
62680011ee [DataFrame] Add implementation for get method (#1496)
* Add implementation for get method
Add tests for get method
Add implementation/tests for get_dtype_counts method
Add implementation/tests for get_ftype_counts method

* Add test fixtures

* Change method tests to fixtures

* Flake8
2018-02-08 22:12:03 -08:00
Eric Liang
41007722f9 [tune] improve s3 log sync (#1511) 2018-02-08 00:42:25 -08:00
Robert Nishihara
89db7841d2 Update arrow version. (#1512) 2018-02-07 23:05:16 -08:00
Devin Petersohn
fa37564511 [DataFrame] Implementation for head, idxmax, idxmin, pop, tail, and Ray Index (#1520)
* Adding head implementation

* Adding idxmax, idxmin, pop, tail

* Adding index skeleton

* Addressing reviewer comments

* Fixing tests to reflect Series constructor changes
2018-02-07 15:43:45 -08:00
Stephanie Wang
ff8e7f8259
Actor checkpointing for distributed actor handles (#1498)
* Expose calls to get and set the actor frontier

* Remove fields used for old checkpointing prototype, change actor_checkpoint_failed -> succeeded

* Prototype for actor checkpointing

* Filter out duplicate tasks on the local scheduler

* Clean up some of the Python checkpointing code

* More cleanups

* Documentation

* cleanup and fix unit test

* Allow remote checkpoint calls through actor handle

* Check whether object is local before reconstructing

* Enable checkpointing for distributed actor handles, refactor tests

* Fix local scheduler tests

* lint

* Address comments

* lint

* Skip tests that fail on new GCS

* style

* Don't put same object twice when setting the actor frontier

* Address Philipp's comments, cleaner fbs naming
2018-02-07 11:19:32 -08:00
Eric Liang
0a9dbc84b5 Tue Feb 6 20:57:42 PST 2018 (#1521)
The test failure was unrelated
2018-02-06 23:11:31 -08:00
Eric Liang
4ec51a4660 [rllib] Occasional Thread Error from RLlib (#1441)
* fix

* Revert "fix"

This reverts commit 808f7d7688a837e5ce4cc4209ca28390bc29f1d8.

* Driver's should ignore imports from other drivers.
2018-02-06 20:30:11 -08:00
Robert Nishihara
12095611c6 Update wheel in example.yaml. (#1515) 2018-02-05 15:06:02 -08:00
Robert Nishihara
df7252b2fe Bump version to 0.3.1. (#1397) 2018-02-04 13:25:25 -08:00
Robert Nishihara
fcb59cba81 Add missing availability_zone argument to development example. (#1509) 2018-02-03 15:04:22 -08:00
the-sea
d0dd33e13c not share registered objects between _Regitry objects (#1508) 2018-02-03 15:03:52 -08:00
Simon Mo
0a79442954 [DataFrame] MVP (1/4) (#1495)
* Implement __{getitem, delitem, copy, deepcopy}__

* Implement all(), any()

* Revert "Implement all(), any()"

This reverts commit 784052414f063662cdb30943297dc9ddfd3ca300.

* Address Comments + Fix axis indexing

* Update syntax for test_axes

* Implement bfill, bool, count

* Implement round

* Resolve bfill inplace issue

* Deimplement bfill; wait for the distributed version

* Fix format

* Copy df for __delitem__
2018-02-03 09:26:18 -08:00
Eric Liang
b948405532
[tune] clean up population based training prototype (#1478)
* patch up pbt

* Sat Jan 27 01:00:03 PST 2018

* Sat Jan 27 01:04:14 PST 2018

* Sat Jan 27 01:04:21 PST 2018

* Sat Jan 27 01:15:15 PST 2018

* Sat Jan 27 01:15:42 PST 2018

* Sat Jan 27 01:16:14 PST 2018

* Sat Jan 27 01:38:42 PST 2018

* Sat Jan 27 01:39:21 PST 2018

* add pbt

* Sat Jan 27 01:41:19 PST 2018

* Sat Jan 27 01:44:21 PST 2018

* Sat Jan 27 01:45:46 PST 2018

* Sat Jan 27 16:54:42 PST 2018

* Sat Jan 27 16:57:53 PST 2018

* clean up test

* Sat Jan 27 18:01:15 PST 2018

* Sat Jan 27 18:02:54 PST 2018

* Sat Jan 27 18:11:18 PST 2018

* Sat Jan 27 18:11:55 PST 2018

* Sat Jan 27 18:14:09 PST 2018

* review

* try out a ppo example

* some tweaks to ppo example

* add postprocess hook

* Sun Jan 28 15:00:40 PST 2018

* clean up custom explore fn

* Sun Jan 28 15:10:21 PST 2018

* Sun Jan 28 15:14:53 PST 2018

* Sun Jan 28 15:17:04 PST 2018

* Sun Jan 28 15:33:13 PST 2018

* Sun Jan 28 15:56:40 PST 2018

* Sun Jan 28 15:57:36 PST 2018

* Sun Jan 28 16:00:35 PST 2018

* Sun Jan 28 16:02:58 PST 2018

* Sun Jan 28 16:29:50 PST 2018

* Sun Jan 28 16:30:36 PST 2018

* Sun Jan 28 16:31:44 PST 2018

* improve tune doc

* concepts

* update humanoid

* Fri Feb  2 18:03:33 PST 2018

* fix example

* show error file
2018-02-02 23:03:12 -08:00
the-sea
a936468f99 [tune] using None as the parameter default value instead of mutable dict (#1501)
* do not use dict as default parameter

* Update trial.py
2018-02-02 21:47:51 -08:00
eugenevinitsky
369773d3e8 [rllib] minor bug fix to shared model, model wasnt actually shared due to new scope (#1503) 2018-02-02 20:37:00 -08:00
Robert Nishihara
ed77a4c415 Make ray.get_gpu_ids() respect existing CUDA_VISIBLE_DEVICES. (#1499)
* Make ray.get_gpu_ids() respect existing CUDA_VISIBLE_DEVICES.

* Comment out failing GPUID check.

* Add import.

* Fix test.

* Remove test.

* Factor out environment variable setting/getting into utils.
2018-02-01 21:29:14 -08:00
Richard Liaw
a5b00a545e
[tune] Fix Tune ParallelCoordinateViz Notebook (#1494) 2018-02-01 00:13:57 -08:00
Philipp Moritz
7550b628bf fix indentation for ES (#1484) 2018-01-31 17:22:38 -08:00
Philipp Moritz
a3f8fa426b Start integrating new GCS APIs (#1379)
* Start integrating new GCS calls

* fixes

* tests

* cleanup

* cleanup and valgrind fix

* update tests

* fix valgrind

* fix more valgrind

* fixes

* add separate tests for GCS

* fix linting

* update tests

* cleanup

* fix python linting

* more fixes

* fix linting

* add plasma manager callback

* add some documentation

* fix linting

* fix linting

* fixes

* update

* fix linting

* fix

* add spillback count

* fixes

* linting

* fixes

* fix linting

* fix

* fix

* fix
2018-01-31 11:01:12 -08:00
Eric Liang
35b1d6189b
[tune] save error msg, cleanup after object checkpoints 2018-01-29 18:48:45 -08:00