No description
Find a file
Si-Yuan c2c501bbe6 Experimental asyncio support (#2015)
* Init commit for async plasma client

* Create an eventloop model for ray/plasma

* Implement a poll-like selector base on `ray.wait`. Huge improvements.

* Allow choosing workers & selectors

* remove original design

* initial implementation of epoll-like selector for plasma

* Add a param for `worker` used in `PlasmaSelectorEventLoop`

* Allow accepting a `Future` which returns object_id

* Do not need `io.py` anymore

* Create a basic testing model

* fix: `ray.wait` returns tuple of lists

* fix a few bugs

* improving performance & bug fixing

* add test

* several improvements & fixing

* fix relative import

* [async] change code format, remove old files

* [async] Create context wrapper for the eventloop

* [async] fix: context should return a value

* [async] Implement futures grouping

* [async] Fix bugs & replace old functions

* [async] Fix bugs found in tests

* [async] Implement `PlasmaEpoll`

* [async] Make test faster, add tests for epoll

* [async] Fix code format

* [async] Add comments for main code.

* [async] Fix import path.

* [async] Fix test.

* [async] Compatibility.

* [async] less verbose to not annoy the CI.

* [async] Add test for new API

* [async] Allow showing debug info in some of the test.

* [async] Fix test.

* [async] Proper shutdown.

* [async] Lint~

* [async] Move files to experimental and create API

* [async] Use async/await syntax

* [async] Fix names & styles

* [async] comments

* [async] bug fixing & use pytest

* [async] bug fixing & change tests

* [async] use logger

* [async] add tests

* [async] lint

* [async] type checking

* [async] add more tests

* [async] fix bugs on waiting a future while timeout. Add more docs.

* [async] Formal docs.

* [async] Add typing info since these codes are compatible with py3.5+.

* [async] Documents.

* [async] Lint.

* [async] Fix deprecated call.

* [async] Fix deprecated call.

* [async] Implement a more reasonable way for dealing with pending inputs.

* [async] Fix docs

* [async] Lint

* [async] Fix bug: Type for time

* [async] Set our eventloop as the default eventloop so that we can get it through `asyncio.get_event_loop()`.

* [async] Update test & docs.

* [async] Lint.

* [async] Temporarily print more debug info.

* [async] Use `Poll` as a default option.

* [async] Limit resources.

* new async implementation for Ray

* implement linked list

* bug fix

* update

* support seamless async operations

* update

* update API

* fix tests

* lint

* bug fix

* refactor names

* improve doc

* properly shutdown async_api

* doc

* Change the table on the index page.

* Adjust table size.

* Only keeps `as_future`.

* change how we init connection

* init connection in `ray.worker.connect`

* doc

* fix

* Move initialization code into the module.

* Fix docs & code

* Update pyarrow version.

* lint

* Restore index.rst

* Add known issues.

* Apply suggestions from code review

Co-Authored-By: suquark <suquark@gmail.com>

* rename

* Update async_api.rst

* Update async_api.py

* Update async_api.rst

* Update async_api.py

* Update worker.py

* Update async_api.rst

* fix tests

* lint

* lint

* replace the magic number
2018-12-06 17:39:05 -08:00
.github Add docs for contributors. (#1191) 2017-11-10 00:40:19 -08:00
.travis [tune] Component notification on node failure + Tests (#3414) 2018-12-04 14:47:31 -08:00
cmake/Modules Experimental asyncio support (#2015) 2018-12-06 17:39:05 -08:00
dev Document the release process. (#2760) 2018-08-29 00:06:33 -07:00
doc Experimental asyncio support (#2015) 2018-12-06 17:39:05 -08:00
docker [tune] Node Fault Tolerance (#3238) 2018-11-21 12:38:16 -08:00
examples [rllib] Propagate model options correctly in ARS / ES, to action dist of PPO (#2974) 2018-10-01 12:49:39 -07:00
java Add stress test for Java worker (#3424) 2018-12-01 16:11:09 -08:00
python Experimental asyncio support (#2015) 2018-12-06 17:39:05 -08:00
site Update Gemfile Jekyll version (#3140) 2018-10-25 21:43:08 -07:00
src/ray Fix bug in ray.wait (#3445) 2018-12-01 19:40:33 -08:00
test Fix failure of test_free_objects_multi_node (#3481) 2018-12-06 15:55:49 -05:00
thirdparty/scripts Ship Modin with Ray. (#3109) 2018-11-29 20:05:24 +01:00
.clang-format Remove legacy Ray code. (#3121) 2018-10-26 13:36:58 -07:00
.gitignore Ship Modin with Ray. (#3109) 2018-11-29 20:05:24 +01:00
.style.yapf YAPF, take 3 (#2098) 2018-05-19 16:07:28 -07:00
.travis.yml Experimental asyncio support (#2015) 2018-12-06 17:39:05 -08:00
build-docker.sh adding -x flag for better debugging during builds (#1079) 2017-10-04 13:56:14 -07:00
build.sh enable incremental builds (#3292) 2018-11-12 21:49:09 -05:00
CMakeLists.txt UI changes, fix the task timeline and add the object transfer timeline to UI. (#3397) 2018-11-25 10:16:49 -08:00
CONTRIBUTING.rst Replace special single quote with regular single quote. (#1693) 2018-03-10 20:36:01 -08:00
LICENSE [rllib] add augmented random search (#2714) 2018-08-24 22:20:02 -07:00
pylintrc adding pylint (#233) 2016-07-08 12:39:11 -07:00
README.rst [docs] Switch docs to use rllib train instead of train.py 2018-12-04 17:36:06 -08:00
scripts Improve yapf speed and document its usage (#2160) 2018-06-05 20:22:11 -07:00
setup_thirdparty.sh update ray cmake build process (#2853) 2018-09-12 11:19:33 -07:00

.. raw:: html

    <a href=http://ray.readthedocs.io/en/latest/index.html><img align="right" width="30%" src="https://github.com/devin-petersohn/ray/raw/docs/update_readme/doc/source/images/ray_logo.png"></a>

.. image:: https://travis-ci.com/ray-project/ray.svg?branch=master
    :target: https://travis-ci.com/ray-project/ray

.. image:: https://readthedocs.org/projects/ray/badge/?version=latest
    :target: http://ray.readthedocs.io/en/latest/?badge=latest

.. image:: https://img.shields.io/badge/pypi-0.6.0-blue.svg
    :target: https://pypi.org/project/ray/

|

**Ray is a flexible, high-performance distributed execution framework.**


Ray is easy to install: ``pip install ray``

Example Use
-----------

+------------------------------------------------+----------------------------------------------------+
| **Basic Python**                               | **Distributed with Ray**                           |
+------------------------------------------------+----------------------------------------------------+
|.. code-block:: python                          |.. code-block:: python                              |
|                                                |                                                    |
|  # Execute f serially.                         |  # Execute f in parallel.                          |
|                                                |                                                    |
|                                                |  @ray.remote                                       |
|  def f():                                      |  def f():                                          |
|      time.sleep(1)                             |      time.sleep(1)                                 |
|      return 1                                  |      return 1                                      |
|                                                |                                                    |
|                                                |                                                    |
|                                                |  ray.init()                                        |
|  results = [f() for i in range(4)]             |  results = ray.get([f.remote() for i in range(4)]) |
+------------------------------------------------+----------------------------------------------------+


Ray comes with libraries that accelerate deep learning and reinforcement learning development:

- `Tune`_: Hyperparameter Optimization Framework
- `RLlib`_: Scalable Reinforcement Learning
- `Distributed Training <http://ray.readthedocs.io/en/latest/distributed_sgd.html>`__

.. _`Tune`: http://ray.readthedocs.io/en/latest/tune.html
.. _`RLlib`: http://ray.readthedocs.io/en/latest/rllib.html

Installation
------------

Ray can be installed on Linux and Mac with ``pip install ray``.

To build Ray from source or to install the nightly versions, see the `installation documentation`_.

.. _`installation documentation`: http://ray.readthedocs.io/en/latest/installation.html

More Information
----------------

- `Documentation`_
- `Tutorial`_
- `Blog`_
- `Ray paper`_
- `Ray HotOS paper`_

.. _`Documentation`: http://ray.readthedocs.io/en/latest/index.html
.. _`Tutorial`: https://github.com/ray-project/tutorial
.. _`Blog`: https://ray-project.github.io/
.. _`Ray paper`: https://arxiv.org/abs/1712.05889
.. _`Ray HotOS paper`: https://arxiv.org/abs/1703.03924

Getting Involved
----------------

- Ask questions on our mailing list `ray-dev@googlegroups.com`_.
- Please report bugs by submitting a `GitHub issue`_.
- Submit contributions using `pull requests`_.

.. _`ray-dev@googlegroups.com`: https://groups.google.com/forum/#!forum/ray-dev
.. _`GitHub issue`: https://github.com/ray-project/ray/issues
.. _`pull requests`: https://github.com/ray-project/ray/pulls