hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-08 19:41:38 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
1592 commits 227 branches 67 tags 234 MiB
Commit graph

537 commits

Author SHA1 Message Date
Devin Petersohn
806b2c844e Fix getattr compat (#1871) 2018-04-10 21:28:59 -07:00
Patrick Yang
521b549e4a [DataFrame] Encapsulate index and lengths into separate class (#1849) 
* baseline impl for index_df.py

* added skeleton for index_df.py

* initial impl index_df

* separate out partition and non-partition impls

* add len function

* drop returns index_df slice of dropped indices

* housecleaning

* Integrate index overhaul

* Rename index df to index metadata

* Fix flake8 issues

* Addressing issues

* fix import issue

* Added metadata passing to constructor
 2018-04-10 14:30:20 -07:00
Peter Schafhalter
405b05d58a [DataFrame] Implemented __getattr__ (#1753) 
* __getattr__ accesses columns

* Added test
 2018-04-10 10:19:33 -07:00
adgirish
efeaacbedc Adding support for concat (#1739) 
adding tests

fixing flake8

adding init

flake 8 on test

fixing tests, imports, and flake8

handling for index

adding tests for row, index

added more robust error handling for axis

fixing test failures

cleaning up error sfor 2.7

updating travis

resolving import

fixing flake8

moved import order

Fixing to refactor and delaying implementing ray-pd inner concat

resolving ray-pd concat and from_pandas mutation

Revert "resolving ray-pd concat and from_pandas mutation"

This reverts commit 5db43e4e89e328286532f3ef98a4526575c5d08d.
 2018-04-09 21:36:24 -07:00
Philipp Moritz
834e594709 [XRay] Register object store and raylet with the GCS (#1860) 2018-04-09 18:56:33 -07:00
Robert Nishihara
7c9e291b4b In the UI, display task breakdowns by default. (#1857) 2018-04-09 13:24:38 -07:00
Jones Wong
6b85d15bd9 supplement missed optimizer argument clip_rewards in default DQN configuration (#1853) 2018-04-08 17:58:05 -07:00
Robert Nishihara
256389dc59 Use new task spec for computing IDs in raylet code path. (#1830) 
* Use new task spec for computing IDs in raylet code path.

* Fix linting.

* Fixes

* Fix test.
 2018-04-08 13:31:55 -07:00
Eric Liang
79d37ce240
[rllib] Switch to use lz4 instead of snappy (#1847) 
* lz4

* comment

* updates
 2018-04-07 14:43:45 -07:00
Eric Liang
e6c00b2b5e
[tune] Add util function to broadcast objects (#1845) 
* add util

* Fri Apr  6 15:09:20 PDT 2018

* doc

* Fri Apr  6 15:21:42 PDT 2018

* Fri Apr  6 15:28:07 PDT 2018

* Fri Apr  6 15:28:26 PDT 2018

* Update tune-config.rst

* Update tune-config.rst
 2018-04-07 11:37:14 -07:00
Richard Liaw
bc8f62c947
[tune] Fix Median Stopping Rule Verbosity (#1833) 2018-04-06 22:58:13 -07:00
Stephanie Wang
bf194db4bc [xray] Basic actor support (#1835) 2018-04-06 00:17:14 -07:00
Stephanie Wang
cbf3181fd2 [xray] Monitor for Raylet processes (#1831) 
* Add raylet monitor script to timeout Raylet heartbeats

* Unit test for removing a different client from the client table

* Set node manager heartbeat according to global config

* Doc and fixes

* Add regression test for client table disconnect, refactor client table

* Fix linting.
 2018-04-05 20:45:38 -07:00
Devin Petersohn
0d9a7a3c19 [DataFrame] Update architecture to be more flexible and performant (#1821) 2018-04-05 15:14:33 -07:00
Robert Nishihara
5bde5e75e7 Implement unsafe method for flushing entire object table and task table. (#1824) 
* Implement unsafe method for flushing entire object table and task table.

* Add test.

* Fix test.
 2018-04-04 18:29:24 -07:00
Richard Liaw
888e70f1be
[tune] HyperOpt Support (v2) (#1763) 2018-04-04 11:08:26 -07:00
Robert Nishihara
fbfbb1c079 [xray] Integrate worker.py with raylet. (#1810) 
* Integrate worker with raylet.

* Begin allowing worker to attach to cluster.

* Fix linting and documentation.

* Fix linting.

* Comment tests back in.

* Fix type of worker command.

* Remove xray python files and tests.

* Fix from rebase.

* Add test.

* Copy over raylet executable.

* Small cleanup.
 2018-04-03 02:38:56 -07:00
Robert Nishihara
0fc989c6c1 Don't use 127.0.0.1 for local ip address. (#1596) 
* Don't use 127.0.0.1 for ip address.

* Update test
 2018-04-02 00:34:20 -07:00
Robert Nishihara
d3e974a9a4 Increase ulimit -n in autoscaler examples. (#1769) 2018-04-02 00:32:56 -07:00
Robert Nishihara
23b8793f0e Update documentation and autoscaler to find 0.4.0. (#1789) 2018-04-02 00:28:47 -07:00
Philipp Moritz
40c9b9cd60 Fix the setuptools_scm issue (#1784) 2018-03-31 10:33:40 -07:00
Eric Liang
faaa123046 [rllib] Set num_cpu=None for workers in the default settings (#1793) 2018-03-29 16:33:40 -07:00
Eric Liang
4116c64698
[tune] Remove rllib dep again, and add a test (#1792) 
* tune should not depend on rllib

* fix dep test

* Tue Mar 27 16:55:41 PDT 2018

* f401
 2018-03-29 15:36:49 -07:00
Robert Nishihara
8d52fe931b Add experimental feature for flushing event logs and logfiles. (#1659) 
* Add experimental feature for flushing event logs and logfiles.

* Add documentation.
 2018-03-27 11:57:52 -07:00
Robert Nishihara
f69cbd35d4 Bump version to 0.4.0. (#1745) 2018-03-26 22:37:16 -07:00
Robert Nishihara
de3cfa223d Fix monitor.py bottleneck by removing excess Redis queries. (#1786) 
* Fix monitor.py bottleneck by removing excess Redis queries.

* Remove unnecessary default value.
 2018-03-26 22:30:38 -07:00
Robert Nishihara
1ab0d0ea69 Acquire worker lock when importing actor. (#1783) 2018-03-26 18:31:26 -07:00
Eric Liang
7c4afa4b04 [tune] Fix linting error (#1777) 2018-03-25 23:44:14 -07:00
Yan Facai (颜发才)
6b1e592d5c [tune] Added pbt with keras on cifar10 dataset example (#1729) 
* [tune] Added pbt with keras on cifar10 dataset example

* ENH: add gpu resources

* CLN: requires 4 GPUs resource

* CLN: use single quotes

* CLN: don't save model by default
 2018-03-25 15:57:23 -07:00
Eric Liang
72595cca0d [tune] Change tune resource request syntax to be less confusing (#1764) 
* update

* update examples

* Wed Mar 21 15:19:56 PDT 2018

* Wed Mar 21 15:21:32 PDT 2018

* Update train_a3c.py

* Update train.py

* fix resources accounting
 2018-03-23 06:25:01 -07:00
Robert Nishihara
10dabce4d7 Remove from X import Y convention in RLlib ES. (#1774) 2018-03-23 05:54:31 -07:00
Christian Barra
13b3df9321 Check if the provider is external before getting the config. (#1743) 2018-03-22 22:59:29 -07:00
Robert Nishihara
c6ad71fc9d Fix bug when connecting another driver in local case. (#1760) 
* Allow connecting another driver when using ip address 127.0.0.1.

* Add test.
 2018-03-21 11:49:53 -07:00
Eric Liang
b41bdcefa0
[rllib] Update RLlib to work with new actor scheduling behavior (#1754) 
* Mon Mar 19 21:23:01 PDT 2018

* Mon Mar 19 21:23:07 PDT 2018

* Mon Mar 19 21:30:49 PDT 2018

* Mon Mar 19 21:32:05 PDT 2018

* Mon Mar 19 21:35:43 PDT 2018

* fix cpu limits

* Mon Mar 19 22:25:07 PDT 2018
 2018-03-20 19:29:52 -07:00
Robert Nishihara
4bccabd910 Redirect output of all processes by default. (#1752) 
* Redirect output of all processes by default.

* Add separate flag for redirecting worker output.

* Fix tests.
 2018-03-20 18:14:54 -07:00
Robert Nishihara
2922e1c388 Add API for getting total cluster resources. (#1736) 
* Add API for getting total cluster resources.

* Add test.
 2018-03-20 15:57:00 -07:00
Robert Nishihara
4658d0a180 Print error when actor takes too long to start, and refactor error me… (#1747) 
* Print error when actor takes too long to start, and refactor error message pushing.

* Print warning every ten seconds.

* Fix linting and tests.

* Fix tests.
 2018-03-19 20:24:35 -07:00
Robert Nishihara
f88a2544bf Speed up actor creation task submission by generating IDs with uuid. (#1744) 
* Speed up actor creation task submission by generating IDs deterministically.

* Revert "Speed up actor creation task submission by generating IDs deterministically."

This reverts commit 175d9587302664916ce9db4071185485da8da041.

* Don't generate actor IDs deterministically yet.

* Factor out ID generation method.
 2018-03-19 19:32:46 -07:00
Robert Nishihara
d78de0d41f Provide experimental API for changing number of return values and res… (#1735) 
* Provide experimental API for changing number of return values and resource requirements at task submission time.

* Remove code duplication and add tests.
 2018-03-19 15:32:23 -07:00
Richard Liaw
23954e7ce2
[tune] Tune Documentation and expose better API (#1681) 2018-03-19 12:55:10 -07:00
Philipp Moritz
7b493aa4a1 Register credis with redis (#1730) 2018-03-18 14:02:19 -07:00
Christian Barra
070e27ea7a Add external module as a node scaler. (#1703) 
* WIP: add external module as a node scaler.

* Fix style.

* Add tests, fix style issues.

* Fix typos.

* Fix test error.

* Fix node provider path.

* Add function to spli pkg from class.

* Add doc.

* Correct documentation.

* Debugging....

* Debugging....

* Add __init__.py to tests.

* add more output for debugging

* Add more test, fix error with import.

* Add a small detail to the documentation.

* Update autoscaler.py
 2018-03-17 16:59:13 -07:00
Eric Liang
e3685fca5e
[rllib] remove redundant docs (#1728) 
* wip

* more work

* fix apex

* docs

* apex doc

* pool comment

* clean up

* make wrap stack pluggable

* Mon Mar 12 21:45:50 PDT 2018

* clean up comment

* table

* Mon Mar 12 22:51:57 PDT 2018

* Mon Mar 12 22:53:05 PDT 2018

* Mon Mar 12 22:55:03 PDT 2018

* Mon Mar 12 22:56:18 PDT 2018

* Mon Mar 12 22:59:54 PDT 2018

* Update apex_optimizer.py

* Update index.rst

* Update README.rst

* Update README.rst

* comments

* Wed Mar 14 19:01:02 PDT 2018

* Fri Mar 16 15:44:27 PDT 2018
 2018-03-17 14:45:04 -07:00
Richard Liaw
9b361115c3
[tune] Added Async HyperBand example (#1709) 2018-03-16 13:25:29 -07:00
Robert Nishihara
75dc106303 Install flex and bison in autoscaler development example. (#1725) 2018-03-16 11:55:59 -07:00
Robert Nishihara
96913be939 Treat actor creation like a regular task. (#1668) 
* Treat actor creation like a regular task.

* Small cleanups.

* Change semantics of actor resource handling.

* Bug fix.

* Minor linting

* Bug fix

* Fix jenkins test.

* Fix actor tests

* Some cleanups

* Bug fix

* Fix bug.

* Remove cached actor tasks when a driver is removed.

* Add more info to taskspec in global state API.

* Fix cyclic import bug in tune.

* Fix

* Fix linting.

* Fix linting.

* Don't schedule any tasks (especially actor creaiton tasks) on local schedulers with 0 CPUs.

* Bug fix.

* Add test for 0 CPU case

* Fix linting

* Address comments.

* Fix typos and add comment.

* Add assertion and fix test.
 2018-03-16 11:18:07 -07:00
Rohan Singh
1f027344f1 [Dataframes] Implemented .describe() (#1696) 
* added describe methods

* mean updates and added truediv func

* updates

* updated truediv test

* porting stocks to ubuntu

* hacky solution for describe, mean, median, quantile by transposing df

* removed data file

* removed faulty truediv implementation

* flake8 and documentation updates

* updated mean, median, var, std to handle mixed values

* added describe methods

* mean updates and added truediv func

* updates

* updated truediv test

* porting stocks to ubuntu

* hacky solution for describe, mean, median, quantile by transposing df

* removed data file

* removed faulty truediv implementation

* flake8 and documentation updates

* fixed quantile to drop object typed columns

* syntax improvements"

* fixed flatten issue

* fixing flatten issue

* minor updates

* added describe methods

* mean updates and added truediv func

* updates

* updated truediv test

* porting stocks to ubuntu

* hacky solution for describe, mean, median, quantile by transposing df

* removed data file

* removed faulty truediv implementation

* flake8 and documentation updates

* updated mean, median, var, std to handle mixed values

* added describe methods

* mean updates and added truediv func

* updates

* updated truediv test

* porting stocks to ubuntu

* hacky solution for describe, mean, median, quantile by transposing df

* removed data file

* removed faulty truediv implementation

* flake8 and documentation updates

* fixed quantile to drop object typed columns

* syntax improvements"

* fixed flatten issue

* fixing flatten issue

* improved describe syntax
 2018-03-15 21:16:59 -07:00
Richard Liaw
459fd5e152
[tune][minor] Move helper function (#1722) 2018-03-15 18:41:02 -07:00
Eric Liang
882a649f0c
[rllib] [docs] Cleanup RLlib API and make docs consistent with upcoming blog post (#1708) 
* wip

* more work

* fix apex

* docs

* apex doc

* pool comment

* clean up

* make wrap stack pluggable

* Mon Mar 12 21:45:50 PDT 2018

* clean up comment

* table

* Mon Mar 12 22:51:57 PDT 2018

* Mon Mar 12 22:53:05 PDT 2018

* Mon Mar 12 22:55:03 PDT 2018

* Mon Mar 12 22:56:18 PDT 2018

* Mon Mar 12 22:59:54 PDT 2018

* Update apex_optimizer.py

* Update index.rst

* Update README.rst

* Update README.rst

* comments

* Wed Mar 14 19:01:02 PDT 2018
 2018-03-15 15:57:31 -07:00
Robert Nishihara
adffc7bfea Pin Cython to 0.27.3 to fix travis builds. (#1718) 2018-03-13 22:18:08 -07:00