* add pages about examples on training language models with fairseq and ray autoscaler
* better format
* update ray_train.sh
* Move EFS to the autoscaler file
* nits
* add comments to the code & use a new way to implement checkpoint hook
* small bug fix
* polish the doc
* fix formatting
* yaml
* update docs
* fix the bugs and add preprocess.sh
* fix lint
* Reduce batch size & fix lint
* shorttitle
* Limit Redis max memory to 10GB/shard by default.
* Update stress tests.
* Reorganize
* Update
* Add minimum cap size for object store and redis.
* Small test update.
- Surfaces local cluster usage
- Increases visability of these instructions
- Removes some docker docs (that are really out of scope for Ray
documentation IMO)
Closes#3517.
* Init commit for async plasma client
* Create an eventloop model for ray/plasma
* Implement a poll-like selector base on `ray.wait`. Huge improvements.
* Allow choosing workers & selectors
* remove original design
* initial implementation of epoll-like selector for plasma
* Add a param for `worker` used in `PlasmaSelectorEventLoop`
* Allow accepting a `Future` which returns object_id
* Do not need `io.py` anymore
* Create a basic testing model
* fix: `ray.wait` returns tuple of lists
* fix a few bugs
* improving performance & bug fixing
* add test
* several improvements & fixing
* fix relative import
* [async] change code format, remove old files
* [async] Create context wrapper for the eventloop
* [async] fix: context should return a value
* [async] Implement futures grouping
* [async] Fix bugs & replace old functions
* [async] Fix bugs found in tests
* [async] Implement `PlasmaEpoll`
* [async] Make test faster, add tests for epoll
* [async] Fix code format
* [async] Add comments for main code.
* [async] Fix import path.
* [async] Fix test.
* [async] Compatibility.
* [async] less verbose to not annoy the CI.
* [async] Add test for new API
* [async] Allow showing debug info in some of the test.
* [async] Fix test.
* [async] Proper shutdown.
* [async] Lint~
* [async] Move files to experimental and create API
* [async] Use async/await syntax
* [async] Fix names & styles
* [async] comments
* [async] bug fixing & use pytest
* [async] bug fixing & change tests
* [async] use logger
* [async] add tests
* [async] lint
* [async] type checking
* [async] add more tests
* [async] fix bugs on waiting a future while timeout. Add more docs.
* [async] Formal docs.
* [async] Add typing info since these codes are compatible with py3.5+.
* [async] Documents.
* [async] Lint.
* [async] Fix deprecated call.
* [async] Fix deprecated call.
* [async] Implement a more reasonable way for dealing with pending inputs.
* [async] Fix docs
* [async] Lint
* [async] Fix bug: Type for time
* [async] Set our eventloop as the default eventloop so that we can get it through `asyncio.get_event_loop()`.
* [async] Update test & docs.
* [async] Lint.
* [async] Temporarily print more debug info.
* [async] Use `Poll` as a default option.
* [async] Limit resources.
* new async implementation for Ray
* implement linked list
* bug fix
* update
* support seamless async operations
* update
* update API
* fix tests
* lint
* bug fix
* refactor names
* improve doc
* properly shutdown async_api
* doc
* Change the table on the index page.
* Adjust table size.
* Only keeps `as_future`.
* change how we init connection
* init connection in `ray.worker.connect`
* doc
* fix
* Move initialization code into the module.
* Fix docs & code
* Update pyarrow version.
* lint
* Restore index.rst
* Add known issues.
* Apply suggestions from code review
Co-Authored-By: suquark <suquark@gmail.com>
* rename
* Update async_api.rst
* Update async_api.py
* Update async_api.rst
* Update async_api.py
* Update worker.py
* Update async_api.rst
* fix tests
* lint
* lint
* replace the magic number
This PR introduces the following changes:
* Ray Tune -> Tune
* [breaking] Creation of `schedulers/`, moving PBT, HyperBand into a submodule
* [breaking] Search Algorithms now must take in experiment configurations via `add_configurations` rather through initialization
* Support `"run": (function | class | str)` with automatic registering of trainable
* Documentation Changes
* Ray documentation - created new section 'Profiling for Ray Users', opposed to current Profiling section for Ray developers. Completed three sections 'A Basic Profiling Example', 'Timing Performance Using Python's Timestamps', and 'Profiling Using An External Profiler (Line_Profiler).' Left to-do two sections on CProfile and Ray Timeline Visualization.'
* Ray documentation - Fixed rst codeblock linebreaks in 'User Profiling'
* Ray documentation - For User Profiling, added section on cProfile
* Ray documentation - For User Profiling, completed Ray Timeline Visualization section, including graphical images
* Ray documentation - made User Profiling timeline image larger, minor wording edits
* Ray documentation - minor wording edits to User Profiling
* Ray documentation - User Profiling- fixed broken link
* Minor wording changes requested by Philipp Moritz addressed. Still need to address (1) compressing the image files, (2) correcting ex 3 to not be remote, and (3) using cProfile on an actor
* Ray documentation - For user-profiling.rst, revised example 3 to show a semi-parallelized example. Compressed timeline example image to be under 50 KB, removed view timeline GUI image. Updated timeline example image to reflect revised example 3. cProfile actor example left
* Ray documentation - in user-profiling.rst, added a new example including actors in the cProfile section
* Ray documentation - For user-profiling.rst, added section header for the Ray actor cProfile example
* Update user-profiling.rst
* Update user-profiling.rst
* 4 space indentation
* Update user-profiling.rst
* Update user-profiling.rst
* Update user-profiling.rst
* corrections
