Commit graph

14133 commits

Author SHA1 Message Date
Clark Zinzow
bfc38de009
[Datasets] [Docs] Improve .limit() and .take() docstrings (#27367)
Improve docstrings for .limit() and .take(), making the distinction more clear.

Signed-off-by: Clark Zinzow <clarkzinzow@gmail.com>
2022-08-05 12:17:24 -07:00
Stephanie Wang
4d448e0b3e
[docs] Add codeowners for subdirectories (#27569)
Signed-off-by: Stephanie Wang swang@cs.berkeley.edu

CODEOWNERS only respects the last matching entry for a file. This PR hopefully adds the top-level docs group to all subdirs.
2022-08-05 11:37:15 -07:00
Richard Liaw
4629a3a649
[air/docs] Update Trainer documentation (#27481)
Co-authored-by: xwjiang2010 <xwjiang2010@gmail.com>
Co-authored-by: Kai Fricke <kai@anyscale.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Eric Liang <ekhliang@gmail.com>
2022-08-05 11:21:19 -07:00
Cade Daniel
f94a2fe166
[docs][Ray Clusters] New Ray Clusters getting started page. (#27391)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2022-08-05 10:21:56 -07:00
zcin
22db41c21a
[Serve][doc] Modify and Combine Tensorflow, Pytorch, Sklearn Tutorials (#26817) 2022-08-05 11:55:31 -05:00
zcin
04c7ccacf1
[Serve][Doc] Moves Serve REST API and Serve CLI API into separate subpages (#26914) 2022-08-05 11:51:53 -05:00
zcin
8a9d994dd0
[serve] Integrate and Document Bring-Your-Own Gradio Applications (#26403)
Integration between Ray Serve and Gradio. Users of Gradio can wrap their Gradio app in a Serve deployment by using `GradioIngress`, and scale it up through more replicas or more CPU/GPU resources.
2022-08-05 11:31:00 -05:00
zcin
b5927caaae
[serve] Update version if import_path or runtime_env in config is changed (#27498)
Previous PR that adds in lightweight config updates: https://github.com/ray-project/ray/pull/27000. It only tracks the config options for `deployments` (bumps version if certain deployment options are changed, but otherwise keeps versions the same). However we should bump the versions of all deployments if `import_path` or `runtime_env` is changed.
2022-08-05 11:30:22 -05:00
Jialing He
ccf411604e
Revert "Revert "[Job Submission][refactor 1/N] Add AgentInfo to GCSNodeInfo (…" (#27308) 2022-08-05 16:32:48 +08:00
Jiajun Yao
b11d3061d8
[Doc] Core getting started page revamp (#27303)
- Add a calculating pi example to getting started page.
- Move installing ray c++ to the installation page.

Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
2022-08-04 23:36:16 -07:00
Jiajun Yao
d7dcb1f938
Replace boost::filesystem with std::filesystem (#27522)
This redos #27319

Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
2022-08-04 21:33:51 -07:00
Archit Kulkarni
1714d0266b
[Doc] [Serve] Refresh code for "monitoring" for 2.0 (#27400) 2022-08-04 20:10:12 -07:00
Dmitri Gekhtman
b1d838446c
[autoscaler] Fix Prometheus metric autoscaler hang bug (#27532)
Failed node launch can lead to an extra unexpected error in the node launcher due to the definition of a mock prometheus metric method.
This failure leads to a permanently hanging autoscaler with "launching nodes" never cleared out and the autoscaler unable to proceed to launch nodes.

This PR fixes the method signature leading to the unexpected failure.
2022-08-04 19:48:31 -07:00
Avnish Narayan
6a31b61580
[RLlib] CQL change hparams and data reading strategy (#27451) 2022-08-04 18:55:32 -07:00
Bill Chambers
73bc572405
[AIR/docs] Adding Source Libraries (#27518)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2022-08-04 15:56:40 -07:00
Alex Wu
eb9c5d8fa7
[autoscaler][aws] Bump max keys per account (#27506)
Signed-off-by: Alex Wu <alex@anyscale.io>

This is a minor QoL improvement to bump the hardcoded limit for number of aws keys per account. The limit is arbitrary and has been bumped before. AFAICT the fundamental aws limit is a 5000 key per region limit which we are not close to.
2022-08-04 15:12:55 -07:00
SangBin Cho
5298ee83b2
[Test] Revert (partially) Fix windows buildkite (#26615) (#27495)
Root cause:
https://www.shell-tips.com/bash/source-dot-command/#gsc.tab=0
Using . will execute the command in the "current shell" in a bash script. It looks like removing . command from ci.sh init means that we will lose the set -eo command used within ci.sh init applied to next test running commands because set -eo is called within a child process, not the current shell (so the future command won't have the set -eo configured).
2022-08-04 13:55:48 -07:00
Philipp Moritz
ef260702a2
[docs] Better defaults for installing Ray (#27500) 2022-08-04 11:20:08 -07:00
Bill Chambers
19dc19a2c5
Fix Ray Air Docs Install (#27501) 2022-08-04 10:47:10 -07:00
shrekris-anyscale
11abc89746
[Serve] [Docs] Use dashboard agent port in REST API documentation (#27450) 2022-08-04 10:24:57 -07:00
Philipp Moritz
64fc1155b7
[docs] K8s docs intro polish and KubeRay architecture diagram (#27488)
* Save work

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

* Update

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

* consistency

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

* update

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

* fixes

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

* simplify

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

* update

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

* fix

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

* update

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

* wording

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

* update

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>
2022-08-04 10:07:15 -07:00
Richard Liaw
b2cd34cc5c
[air] Remove checkpoint user guide and update key concepts and docstring (#27455) 2022-08-04 08:55:26 -07:00
xwjiang2010
8d5c07b781
[air/train/docs] Add trainer user guide and update trainer docs (#27389)
This PR adds a user guide to AIR for using Ray Train. It provides a high level overview of the trainers and removes redundant sections.

The main file to review is here: doc/source/ray-air/trainer.rst.

Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com>
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
Signed-off-by: Kai Fricke <kai@anyscale.com>

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
Co-authored-by: Kai Fricke <kai@anyscale.com>
2022-08-04 13:59:50 +01:00
SangBin Cho
afd6597056
Revert "Replace boost::filesystem with std::filesystem (#27338)" (#27483)
This reverts commit c50faa126c.
2022-08-04 02:18:59 -07:00
Tao Wang
d4a1cebaa3
[C++ worker]Support ActorHandle type parameter (#27364)
Now c++ worker doesn't support `ActorHandle` type parameter.
When we pass an `ActorHandle` object to a task, it will incur this error:
![image](https://user-images.githubusercontent.com/5276001/182349872-a616ff55-6a2b-454d-9831-18877b56c228.png)
The reason is that caller just deserializes the actor handle but doesn't register it to core worker, so if we call tasks of the actor, it will not be found in local.
2022-08-04 16:39:52 +08:00
Kai Fricke
b6765bb4f3
[air/tune/train] Update/fix API annotations (#27428)
This bumps annotations to beta or demotes to DeveloperAPI

Signed-off-by: Kai Fricke <kai@anyscale.com>
2022-08-04 09:05:04 +01:00
Eric Liang
cd9cabcadf
[docs] Change data tagline to "Distributed Data Preprocessing" (#27434) 2022-08-03 16:57:07 -07:00
Avnish Narayan
55209692ee
[RLlib] Deflake MARWIL and BC and remove memory leak from torch MARWIL policy (#27406) 2022-08-03 16:53:12 -07:00
Eric Liang
67a306f92f
[docs] Update colors and styling of ray diagrams (#27474) 2022-08-03 16:49:25 -07:00
Eric Liang
340f0960d6
[docs] Improve the AIR introductory page (#27347) 2022-08-03 16:04:04 -07:00
Ricky Xu
8498a56fe2
[Core][fix] Increasing timeout on non-windows for test_metrics (#27379)
The test was timing out.

A normal pass was ~17secs.
2022-08-03 15:22:00 -07:00
Alan Guo
2cf9ecf48e
Make it so pydantic is required before we launch dashboard api server (#27345)
* Make it so pydantic is required before we launch dashboard api server

Signed-off-by: Alan Guo <aguo@anyscale.com>
2022-08-03 14:24:51 -07:00
Balaji Veeramani
fd381927c1
[AIR] Add optional mode parameter and make size parameter optional (#27295)
1. If a user reads a folder with grayscale and color images, ImageFolderDatasource errors.
2. There's no way to retain image shapes.

Co-authored-by: Clark Zinzow <clarkzinzow@gmail.com>
2022-08-03 13:20:46 -07:00
Eric Liang
c7056309c4
[docs] Revamp README and Ray intro doc page (#27405)
This PR revamps and aligns the README and Ray intro doc page:

New "What is Ray" diagram that introduces AIR vs Ray core (diagram TBD finalized, this is the working placeholder)
Update the description of Ray
Link out to the user guides for key libraries and key concepts
Remove old / broken links, as well as the inline library descriptions from the README
2022-08-03 13:19:00 -07:00
Archit Kulkarni
9f0d8e364d
[Doc] Update Serve architecture doc for 2.0 (#26861)
- Move autoscaling architecture from autoscaling page to architecture page
- Update architecture page
    - Remove "Router" actor
    - Update description of ServeHandle
    - Update defaults about HTTPproxy (default one on each node -> default just one per cluster, on the head node)
- Add note about fault tolerance in different failure scenarios
- Assorted typos/usage nits

Co-authored-by: shrekris-anyscale <92341594+shrekris-anyscale@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
2022-08-03 14:30:33 -05:00
clarng
a2eaa7a5a4
[docs][Core] rename 'more topics' to 'advanced topics' (#27385)
Ray 2.0 doc update : Rename 'more topics' to 'advanced topics'. Also cleaned up misc topics to have a consistent name
2022-08-03 12:14:43 -07:00
Cade Daniel
99ad0667a5
[docs][Ray Clusters] Migrate Community Supported Cluster Launcher to new structure. (#27376)
This PR migrates the old Community Supported Cluster Launcher docs to the new Ray Clusters doc structure.

Signed-off-by: Cade Daniel <cade@anyscale.com>
2022-08-03 11:07:10 -07:00
zcin
286343601a
[Serve] Enable lightweight config update (#27000) 2022-08-03 11:49:41 -05:00
xwjiang2010
ff2b728e9a
[air] add tuner user guide (#26837)
Co-authored-by: Kai Fricke <kai@anyscale.com>
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2022-08-03 09:43:42 -07:00
Archit Kulkarni
a12c04a2fe
[Serve] [Doc] Update key concepts for 2.0, remove deprecated APIs (#26965)
Removes deprecated APIs:
- serve.start()
- get_handle()

Rewrites the ServeHandle doc snippet to use the recommended workflow for ServeHandles (only access them from other deployments, pass Deployments in as input args to `.bind()`, which get resolved to ServeHandles at runtime)

Co-authored-by: shrekris-anyscale <92341594+shrekris-anyscale@users.noreply.github.com>
2022-08-03 11:27:23 -05:00
Jimmy Yao
1c1cca2736
[release/ray-lightning] adjust the release test of ray lightning master
First of all, sorry i messed up with the previous pr when sync with the master (#27374). This PR is the duplicate of previous pr until we update the changes (change: adding the version check for the ray_lightning for the compatibility). Also, apology for the massive review requests on the previous PR.
2022-08-03 16:01:32 +01:00
Kai Fricke
20119c7022
[tune] Fix test_actor_reuse.py::ActorReuseMultiTest test (#27427)
Increase time to allow for scheduling latency

Signed-off-by: Kai Fricke <kai@anyscale.com>
2022-08-03 13:54:11 +01:00
Kai Fricke
46ed3557ba
[tune] Fix test_resource_exhausted_info test (#27426)
#27213 broke this test

Signed-off-by: Kai Fricke <kai@anyscale.com>
2022-08-03 13:53:46 +01:00
Simon Mo
4e07019b88
[Serve] Fix Graph Repeated Invocation (#27417) 2022-08-03 01:40:19 -07:00
shrekris-anyscale
adc7c4dc87
[Serve] Make serve.run() and deployment.bind() beta APIs (#27401) 2022-08-02 23:11:23 -07:00
Simon Mo
8ac6d02502
[Serve][Nightly] Environment for Nightly K8s Tests (#27126) 2022-08-02 23:05:47 -07:00
Jiajun Yao
8b7e4ac701
[Doc] Test ray core doc code (#27334)
- Currently not all code under ray-core/doc_code is covered by CI.
- tf_example.py and torch_example.py are not used anywhere.

Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
2022-08-02 20:51:47 -07:00
Simon Mo
6084eb6a9f
Revert "Revert "[Serve] ServeHandle detects ActorError and drop replicas from target group (#26685)" (#27283)" (#27348) 2022-08-02 20:04:03 -07:00
Rohan Potdar
5b6a58ed28
[RLlib] Add OPE Learning Tests (#27154) 2022-08-02 17:51:38 -07:00
Richard Liaw
6dc3dbdd37
[air] Update to beta (#27393)
Update API references to beta. Needed as we are going to beta in 2.0.

I left out RL/Scikit-Learn/HuggingFace.
2022-08-02 17:10:41 -07:00