Commit graph

449 commits

Author SHA1 Message Date
Robert Nishihara
d78de0d41f Provide experimental API for changing number of return values and res… (#1735)
* Provide experimental API for changing number of return values and resource requirements at task submission time.

* Remove code duplication and add tests.
2018-03-19 15:32:23 -07:00
Richard Liaw
23954e7ce2
[tune] Tune Documentation and expose better API (#1681) 2018-03-19 12:55:10 -07:00
Philipp Moritz
7b493aa4a1 Register credis with redis (#1730) 2018-03-18 14:02:19 -07:00
Christian Barra
070e27ea7a Add external module as a node scaler. (#1703)
* WIP: add external module as a node scaler.

* Fix style.

* Add tests, fix style issues.

* Fix typos.

* Fix test error.

* Fix node provider path.

* Add function to spli pkg from class.

* Add doc.

* Correct documentation.

* Debugging....

* Debugging....

* Add __init__.py to tests.

* add more output for debugging

* Add more test, fix error with import.

* Add a small detail to the documentation.

* Update autoscaler.py
2018-03-17 16:59:13 -07:00
Eric Liang
e3685fca5e
[rllib] remove redundant docs (#1728)
* wip

* more work

* fix apex

* docs

* apex doc

* pool comment

* clean up

* make wrap stack pluggable

* Mon Mar 12 21:45:50 PDT 2018

* clean up comment

* table

* Mon Mar 12 22:51:57 PDT 2018

* Mon Mar 12 22:53:05 PDT 2018

* Mon Mar 12 22:55:03 PDT 2018

* Mon Mar 12 22:56:18 PDT 2018

* Mon Mar 12 22:59:54 PDT 2018

* Update apex_optimizer.py

* Update index.rst

* Update README.rst

* Update README.rst

* comments

* Wed Mar 14 19:01:02 PDT 2018

* Fri Mar 16 15:44:27 PDT 2018
2018-03-17 14:45:04 -07:00
Richard Liaw
9b361115c3
[tune] Added Async HyperBand example (#1709) 2018-03-16 13:25:29 -07:00
Robert Nishihara
75dc106303 Install flex and bison in autoscaler development example. (#1725) 2018-03-16 11:55:59 -07:00
Robert Nishihara
96913be939 Treat actor creation like a regular task. (#1668)
* Treat actor creation like a regular task.

* Small cleanups.

* Change semantics of actor resource handling.

* Bug fix.

* Minor linting

* Bug fix

* Fix jenkins test.

* Fix actor tests

* Some cleanups

* Bug fix

* Fix bug.

* Remove cached actor tasks when a driver is removed.

* Add more info to taskspec in global state API.

* Fix cyclic import bug in tune.

* Fix

* Fix linting.

* Fix linting.

* Don't schedule any tasks (especially actor creaiton tasks) on local schedulers with 0 CPUs.

* Bug fix.

* Add test for 0 CPU case

* Fix linting

* Address comments.

* Fix typos and add comment.

* Add assertion and fix test.
2018-03-16 11:18:07 -07:00
Rohan Singh
1f027344f1 [Dataframes] Implemented .describe() (#1696)
* added describe methods

* mean updates and added truediv func

* updates

* updated truediv test

* porting stocks to ubuntu

* hacky solution for describe, mean, median, quantile by transposing df

* removed data file

* removed faulty truediv implementation

* flake8 and documentation updates

* updated mean, median, var, std to handle mixed values

* added describe methods

* mean updates and added truediv func

* updates

* updated truediv test

* porting stocks to ubuntu

* hacky solution for describe, mean, median, quantile by transposing df

* removed data file

* removed faulty truediv implementation

* flake8 and documentation updates

* fixed quantile to drop object typed columns

* syntax improvements"

* fixed flatten issue

* fixing flatten issue

* minor updates

* added describe methods

* mean updates and added truediv func

* updates

* updated truediv test

* porting stocks to ubuntu

* hacky solution for describe, mean, median, quantile by transposing df

* removed data file

* removed faulty truediv implementation

* flake8 and documentation updates

* updated mean, median, var, std to handle mixed values

* added describe methods

* mean updates and added truediv func

* updates

* updated truediv test

* porting stocks to ubuntu

* hacky solution for describe, mean, median, quantile by transposing df

* removed data file

* removed faulty truediv implementation

* flake8 and documentation updates

* fixed quantile to drop object typed columns

* syntax improvements"

* fixed flatten issue

* fixing flatten issue

* improved describe syntax
2018-03-15 21:16:59 -07:00
Richard Liaw
459fd5e152
[tune][minor] Move helper function (#1722) 2018-03-15 18:41:02 -07:00
Eric Liang
882a649f0c
[rllib] [docs] Cleanup RLlib API and make docs consistent with upcoming blog post (#1708)
* wip

* more work

* fix apex

* docs

* apex doc

* pool comment

* clean up

* make wrap stack pluggable

* Mon Mar 12 21:45:50 PDT 2018

* clean up comment

* table

* Mon Mar 12 22:51:57 PDT 2018

* Mon Mar 12 22:53:05 PDT 2018

* Mon Mar 12 22:55:03 PDT 2018

* Mon Mar 12 22:56:18 PDT 2018

* Mon Mar 12 22:59:54 PDT 2018

* Update apex_optimizer.py

* Update index.rst

* Update README.rst

* Update README.rst

* comments

* Wed Mar 14 19:01:02 PDT 2018
2018-03-15 15:57:31 -07:00
Robert Nishihara
adffc7bfea Pin Cython to 0.27.3 to fix travis builds. (#1718) 2018-03-13 22:18:08 -07:00
Devin Petersohn
8c1066cdba
[DataFrame] Implemented cummax, cummin, cumsum, cumprod (#1705)
* cummax, cummin, cumsum, cumprod

* added remote function

* Fix lint

* Fixing tests and linting

* Fix lint
2018-03-13 10:06:34 -07:00
Jae Min Kim
737120952e [Dataframes] Reorganization (#1676)
* moved helper functions for dataframes into df_utils

* Updating base on review comments

* fixed bug with from_pandas

* Updating formatting

* Fix lint
2018-03-12 19:13:33 -07:00
Peter Veerman
6455ec934b [DataFrame] Implements DataFrame.rename, DataFrame.rename_axis, and Index.set_names (#1573)
* Index update

* Fixed transpose bug with nan values

* Fix lint

* Add rename tests

* Implement DataFrame.rename, DataFrame.rename_axis, and Index.set_names

* Temp

* Fixing rename for new index implementation

Fix rebase merges

* Fix rename and rename_axis to work with new index.

Re-add pytest fixture

Clean up rebase artifacts

Remove index.py file

* Addressing minor points

* Addressing comments
2018-03-12 19:05:32 -07:00
Eric Liang
076936a7f5
[rllib] Switch DQN to using deepmind wrappers (#1655)
* deepmind wrap

* use 80x80

* respect custom prep

* fix replay size

* fix chekc

* batch idx

* Wed Mar  7 11:00:39 PST 2018

* random starts and reward clipping

* Fri Mar  9 17:27:17 PST 2018

* Fri Mar  9 17:36:15 PST 2018

* Sat Mar 10 19:47:10 PST 2018

* Sat Mar 10 19:47:37 PST 2018

* Sat Mar 10 20:05:12 PST 2018

* Sat Mar 10 20:54:21 PST 2018

* Sat Mar 10 21:03:52 PST 2018
2018-03-11 21:14:38 -07:00
Robert Nishihara
cae108d019 Replace special single quote with regular single quote. (#1693) 2018-03-10 20:36:01 -08:00
Richard Liaw
40799fee37
[autoscaler] Fix Defaults (#1661) 2018-03-09 16:59:21 -08:00
Peter Veerman
2b747ba46c [DataFrame] Implement .fillna(), .ffill(), .bfill(), .eval(), and .drop() (#1544)
* Implement ray.DataFrame.drop w/ tests

* Implement ray.DataFrame.eval w/ tests

Fix flake8 issues

* Fix flake8 issues in dataframe.py

* Implement fillna

* Implement fillna

* Implement ffill and bfill

* Define helper functions outside of method invocation

* Implement ray.DataFrame.eval w/ tests

* Index update

* Fixed transpose bug with nan values

* Fix lint

* Implement fillna

* Use ray index to check if labels exist in df

* Fix ValueError catching

* Remove duplicate test methods

* Add documentation for .fillna(), .ffill(), .bfill(), .eval(), and .drop()

Fix flake8 errors

* Remove notebook files

* Change fillna, eval, drop to use new index type

* Fix documentation for fillna, eval and drop

temp

Temp

temp

temp

temp

* Update drop to work with new type of ray index

* Fix flake8 errors

* Refactor fillna fix for index
2018-03-09 07:37:27 -08:00
Eric Liang
75e825177f [rllib] Move Ape-X metrics behind a debug flag and remove some of them (#1656) 2018-03-08 00:48:49 -08:00
Robert Nishihara
b0510ee461 Give error when actor is created before ray.init. (#1666) 2018-03-07 10:36:49 -08:00
Philipp Moritz
a9acfab3a6 Start chain replicated GCS with Ray (#1538) 2018-03-07 10:18:58 -08:00
Rohan Singh
0abebb0975 [Dataframes] Implement .__len__(), .__contains__(), .first_valid_index(), and .last_valid_index() (#1664)
* added len, contains, first_valid_index, last_valid_index

* fixed contains test cases

* test files updated for PR
2018-03-06 23:56:11 -08:00
Devin Petersohn
4af42d5bb6 [DataFrame] Adding error checking for pandas version (#1662)
* Adding error checking for pandas version

* Addressing comments
2018-03-06 09:57:49 -08:00
butchcom
936bebef99 [rllib] Upgrade to OpenAI Gym 0.10.3 (#1601) 2018-03-06 00:31:02 -08:00
Richard Liaw
162d063f0d
[autoscaler/tune] Optional YAML Fields + Fix Pretty Printing for Tune (#1541) 2018-03-04 23:35:58 -08:00
Richard Liaw
78716094b5
[tune] Async Hyperband (#1595) 2018-03-04 14:05:56 -08:00
Eric Liang
ecb811c26e
[rllib] Ape-X implementation and DQN refactor to handle replay in policy optimizer (#1604)
* minimal apex checkin

* cleanup dqn options

* actor utils

* Sun Feb 25 17:39:54 PST 2018

* update

* compression refactor

* fix

* add test

* fix models

* Sun Feb 25 21:46:27 PST 2018

* Wed Feb 28 10:26:34 PST 2018

* Wed Feb 28 10:28:09 PST 2018

* Wed Feb 28 10:42:59 PST 2018

* refactor

* Wed Feb 28 11:17:19 PST 2018

* Wed Feb 28 11:42:08 PST 2018

* Wed Feb 28 11:42:13 PST 2018

* Wed Feb 28 11:59:02 PST 2018

* Wed Feb 28 11:59:58 PST 2018

* Wed Feb 28 12:00:08 PST 2018

* Wed Feb 28 12:02:19 PST 2018

* Wed Feb 28 13:44:31 PST 2018

* Wed Feb 28 17:01:20 PST 2018

* Sat Mar  3 14:55:59 PST 2018

* make optimizer construction explicit

* Sat Mar  3 18:23:08 PST 2018

* Sat Mar  3 18:24:28 PST 2018

* Sat Mar  3 18:49:28 PST 2018

* Sat Mar  3 18:50:42 PST 2018

* Sat Mar  3 18:56:10 PST 2018
2018-03-04 12:25:25 -08:00
Eric Liang
9b33f3a7b7
[autoscaler] Bad error message when dict field omitted (#1632)
* Wed Feb 28 23:22:55 PST 2018

* Wed Feb 28 23:24:07 PST 2018
2018-03-03 20:25:58 -08:00
Eric Liang
75293a0ba0
[rllib] Basic regression tests on CartPole (#1608)
* Sun Feb 25 21:36:22 PST 2018

* Sun Feb 25 21:42:09 PST 2018

* Sun Feb 25 21:44:30 PST 2018

* fix lint

* Wed Feb 28 12:41:49 PST 2018
2018-03-03 16:27:56 -08:00
Eric Liang
80d7def9dc
[autoscaler] [tune] More doc fixes (#1560)
* Fri Feb 16 13:53:50 PST 2018

* Sat Feb 17 15:32:08 PST 2018

* Sat Feb 17 15:44:59 PST 2018

* fix

* Sun Feb 18 14:46:24 PST 2018

* Sun Feb 18 14:46:37 PST 2018

* Sun Feb 18 14:55:52 PST 2018

* Sun Feb 18 15:14:32 PST 2018

* Wed Feb 21 17:34:17 PST 2018

* Sun Feb 25 17:51:17 PST 2018

* Sun Feb 25 22:18:40 PST 2018

* Wed Feb 28 13:19:05 PST 2018

* Wed Feb 28 13:22:13 PST 2018

* Wed Feb 28 13:33:29 PST 2018

* Wed Feb 28 13:35:33 PST 2018

* add ex

* Fri Mar  2 12:50:17 PST 2018

* Fri Mar  2 12:54:31 PST 2018
2018-03-03 13:01:49 -08:00
Richard Liaw
96d7938fc4 [tune] Hyperband Max Iter Fix (#1620)
* nits

* cumul r

* docs

* min
2018-03-03 13:00:55 -08:00
Kunal Gosar
6685d4c446 fix tail and finish repr and str (#1628) 2018-03-02 15:26:54 -08:00
Robert Nishihara
1222d09224 Fix dataframe test linting and test. (#1629) 2018-02-28 15:21:49 -08:00
Devin Petersohn
e7df293946 [DataFrames] Updating Error messages to encourage contribution. (#1623) 2018-02-27 21:44:33 -08:00
Kunal Gosar
4a15c2c65c [Dataframes] Call ray.init() on ray.dataframe import (#1626)
* ray.init on dataframe import

* wrapping ray.init in a try/except

* removing ray.init calls from test code

* resolving flake8
2018-02-27 16:11:23 -08:00
Kunal Gosar
34664dbf76 [DataFrame] Pass lengths to _default_index instead of df (#1621)
* Pass lengths to remote function over DataFrame

* Increasing performance by moving length to remote
2018-02-27 02:38:26 -08:00
Simon Mo
4ab16d7fb3 [DataFrame] Implement loc, iloc (#1612)
* Add parquet-cpp to gitignore

* Add read_csv and read_parquet

* Gitignore pytest_cache

* Fix flake8

* Add io to __init__

* Changing Index. Currently running tests, but so far untested.

* Removing issue of reassigning DF in from_pandas

* Fixing lint

* Fix bug

* Fix bug

* Fix bug

* Better performance

* Fixing index issue with sum

* Address comments

* Update io with index

* Updating performance and implementation. Adding tests

* Fixing off-by-1

* Fix lint

* Address Comments

* Make pop compatible with new to_pandas

* Format Code

* Cleanup some index issue

* Bug fix: assigned reset_index back

* Implement loc and iloc

* Revert whitespace

* Format code

* Address comments
2018-02-27 01:57:52 -08:00
Richard Liaw
b79597dc00
[rllib] PPO Thread Limit (#1568) 2018-02-26 22:22:05 -08:00
Kunal Gosar
f43328f332 moved _default_index to remote fn (#1617) 2018-02-26 21:12:04 -08:00
Kunal Gosar
48bd7b147d [DataFrame] Added Implementations for equals, query, and some other operations (#1610)
* Implemented Dataframe __abs__ and __iter__

* implemented __neg__

* implemented query

* Implemented equals

* Implemented __eq__ and __ne__ operators

* Added method level comments

* resolved flake8 comments

* resolving devin's comments
2018-02-26 18:31:00 -08:00
Simon Mo
d78a22f94c [DataFrame] Implement IO for ray_df (#1599)
* Add parquet-cpp to gitignore

* Add read_csv and read_parquet

* Gitignore pytest_cache

* Fix flake8

* Add io to __init__

* Changing Index. Currently running tests, but so far untested.

* Removing issue of reassigning DF in from_pandas

* Fixing lint

* Fix bug

* Fix bug

* Fix bug

* Better performance

* Fixing index issue with sum

* Address comments

* Update io with index

* Updating performance and implementation. Adding tests

* Fixing off-by-1

* Fix lint

* Address Comments

* Make pop compatible with new to_pandas

* Format Code

* Cleanup some index issue

* Bug fix: assigned reset_index back

* Remove unused debug line
2018-02-26 18:26:38 -08:00
Eric Liang
87e107edd8 [tune] Sync logs from workers and improve tensorboard reporting (#1567) 2018-02-26 11:35:51 -08:00
Devin Petersohn
1fa59f1887 [DataFrame] Adding insert, set_axis, set_index, reset_index and tests (#1603) 2018-02-26 08:58:15 -08:00
Richard Liaw
c2ad800cbf
[rllib] Registry fix for DQN Replay Evaluators (#1593) 2018-02-25 22:30:11 -08:00
Robert Nishihara
00ef2f6285 Only pass in --allow-root if the user is root. (#1594) 2018-02-25 13:40:51 -08:00
Devin Petersohn
529397b35e [DataFrames] Updating Index implementation, performance improvements (#1598) 2018-02-25 13:39:28 -08:00
Richard Liaw
31fefa20b7
[tune] HyperBand Fixes (#1586) 2018-02-25 13:26:58 -08:00
Robert Nishihara
5859a2d249 Replace python setup.py install with pip install -e . (#1460) 2018-02-22 11:15:03 -08:00
Robert Nishihara
f4b1881fec Update arrow to use updated pandas serializer. (#1582) 2018-02-22 11:10:52 -08:00