1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-21 18:36:40 -04:00
Commit graph

13210 commits

Author SHA1 Message Date
SangBin Cho
d59d6ad653
[RFC][Usability] Improve general Ray stacktrace including adding Actor repr ()
* 1. Added a label to the stack trace. 2. Remove ray code from user stacktrace. Improve stacktrace message.

* Add a test to the build

* Fix the issue

* Addressed code review.

* Addressed code review and debugging

* fix

* Try fixing tests.

* Fixed the issue.

* Fixed a bug for real. Tests need to be re-written

* Try one test.

* Formatting

* Addressed code review.

* Addressed the last code review.
2021-08-05 17:12:24 -07:00
SangBin Cho
99b26b476d
Fix flaky windows reconstruction test () 2021-08-05 17:10:54 -07:00
Amog Kamsetty
e4cf26ea6e
[SGD] v2 Prototype sgd.report() implementation ()
* finish session

* finish

* formatting

* tests

* wip

* remove pdb

* remove import

* add tests

* raise from None

* Address comments

* Exception

* remove from None

* fix test

* address comments

* Update python/ray/util/sgd/v2/constants.py

Co-authored-by: matthewdeng <matthew.j.deng@gmail.com>

* add tests for session

Co-authored-by: matthewdeng <matthew.j.deng@gmail.com>
2021-08-05 16:03:21 -07:00
SangBin Cho
381ffdb6d0
Revert "[gcs] Fix actor killing race condition ()" ()
This reverts commit 521457b51b.
2021-08-05 15:54:03 -07:00
Edward Oakes
839ceba6db
[serve] Replace "backend" with "deployment" in metrics & logging () 2021-08-05 17:37:21 -05:00
architkulkarni
05b0da94b7
[core] [runtime env] [test] Add runtime env release test () 2021-08-05 16:06:20 -05:00
Simon Mo
c2ebb67c1d
[Doc] Fold TOC index by default ()
* [Doc] Fold TOC index by default

* pin
2021-08-05 14:02:05 -07:00
architkulkarni
e84ae6caa5
[Core] [runtime env] Avoid spurious worker startup () 2021-08-05 15:46:23 -05:00
SangBin Cho
667851f0ad
Prototype done. () 2021-08-05 13:32:44 -07:00
Eric Liang
8ff3fce4ba
Add a warning if the number of queued tasks to an actor exceeds 5k () 2021-08-05 12:03:48 -07:00
Kai Fricke
86fcf076d6
[release] use job timeout with ray/anyscale connect () 2021-08-05 17:53:14 +01:00
Amog Kamsetty
be238e159d
[Tune] Update docs for with_parameters ()
* with_parameters_doc

* update docstring

* address comments
2021-08-05 08:48:34 -07:00
Sven Mika
3b447265d8
[RLlib] Fix Trainer.add_policy for num_workers>0 (self play example scripts). () 2021-08-05 11:41:18 -04:00
Sven Mika
0eb0e0ff58
[RLlib] Add support for multi-GPU to DDPG. () 2021-08-05 11:39:51 -04:00
Sven Mika
3013d9b341
[RLlib] Fix "Cannot convert a symbolic Tensor (default_policy/strided_slice_3:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported" () 2021-08-05 11:39:15 -04:00
architkulkarni
3ae5229b44
[core] Skip adding "script directory" to workers' sys.path when in interactive shell () 2021-08-05 10:05:19 -05:00
SongGuyang
79bec61e12
[event] support WithField option in RAY_EVENT api () 2021-08-05 20:45:55 +08:00
qicosmos
f1f7d4a085
[C++ Worker]Add some APIs for task call part one () 2021-08-05 17:25:36 +08:00
Amog Kamsetty
d3155bc1a8
increase timeout () 2021-08-05 10:20:46 +01:00
matthewdeng
46c1db1aa7
[release] increase golden notebook test timeout () 2021-08-05 10:00:38 +01:00
architkulkarni
8c1317067d
move variable updates from middle of loop to end () 2021-08-05 09:53:01 +01:00
SongGuyang
93094e567b
[C++ API] C++ API doc update () 2021-08-05 16:24:07 +08:00
Siyuan (Ryans) Zhuang
ffe5b45cc1
[workflow] Enable test () 2021-08-04 21:18:50 -07:00
matthewdeng
1eca6ac154
[SGD] v2 alpha: Tensorflow Backend ()
* [SGD] Implement Tensorflow Backend

* addres comments

* address comments

* format
2021-08-04 16:49:50 -07:00
Eric Liang
6db63990af
Don't capture child tasks in placement groups by default () 2021-08-04 16:09:45 -07:00
Chen Shen
53a0c74413
[nightly-test] fix non_streaming_shuffle_1tb_5000_partitions 2021-08-04 16:06:53 -07:00
Eric Liang
d4f9d3620e
Move ray.data out of experimental () 2021-08-04 13:31:10 -07:00
architkulkarni
63708468df
[runtime env] [Doc] Runtime env doc and messaging improvements () 2021-08-04 12:28:42 -07:00
Siyuan (Ryans) Zhuang
e3c09b0af1
[Workflow] Fix nested virtual actor ()
* fix nested actor

* fix nested actor serialization

* one more example

* update exception message
2021-08-04 10:46:45 -07:00
SangBin Cho
3d13781e67
[Test ]Unflake raylet signal test () 2021-08-04 10:38:59 -07:00
Yi Cheng
521457b51b
[gcs] Fix actor killing race condition () 2021-08-04 10:37:56 -07:00
Lixin Wei
a2b0d2f99f
[Core] Add Back Pressure to GCS's gRPC Server () 2021-08-04 10:36:39 -07:00
Kai Fricke
5d56a8aac5
[RLlib] Fix ignoring "sample_collector" config key () 2021-08-04 10:27:35 -04:00
Navneet Nandan
208d997414
Added support for the imagePullSecrets in helm chart () 2021-08-04 09:45:39 -04:00
SongGuyang
3e42f54910
Support copyright format for c++ files () 2021-08-04 17:19:38 +08:00
Sven Mika
b539560429
[RLlib] Remove SampleBatch._get_slice_indices() deprecated logging spam (moderate). () 2021-08-04 04:21:32 -04:00
Chen Shen
8393b9add8
[Test] increase timeout for autoscaling shuffle () 2021-08-03 23:16:00 -07:00
Eric Liang
cb48f3a712
Be more conservative in warning about too many workers () 2021-08-03 22:30:18 -07:00
Chris K. W
a33cbec12a
[client][docs] update docs for new client support in init ()
* start

* check formatting

* undo changes from base branch

* Client builder API docs

* indent

* 8

* minor fixes

* absolute path to runtime env docs

* fix runtime_env link

* Update worker.init docs

* drop clientbuilder docs, link to 1.4.1 docs instead. Specify local:// behavior when address passed

* add debug info for ray.init("local")

* local:// attaches a driver directly

* update ray.init return wording

* remote init.connect() from example

* drop local:// docs, add section on when to use ray client

* link to 1.4.1 docs in code example instead of mentioning clientbuilder

* fix backticks, doc mentions of ray.util.connect

* remove ray.util.connect mentions from examples and comments

* update tune example

* wording

* localhost:<port> also works if you're on the head node

* add quotes

* drop mentions of ray client from ray.init docstring

* local->remote

* fix section ref

* update ray start output

* fix section link

* try to fix doc again

* fix link wording

* drop local:// from docs and special handling from code

* update ray start message

* lint

* doc lint

* remove local:// codepath

* remove 'internal_config'

* Update doc/source/cluster/ray-client.rst

Co-authored-by: Ameer Haj Ali <ameerh@berkeley.edu>

* doc suggestion

* Update doc/source/cluster/ray-client.rst

Co-authored-by: Ameer Haj Ali <ameerh@berkeley.edu>
2021-08-04 05:31:44 +03:00
James Mishra
6240d22060
Validate Redis addresses before making the client () 2021-08-03 16:56:53 -07:00
Siyuan (Ryans) Zhuang
bef519b373
[Workflow] Simplify storage and bug fix ()
* simplify storage

* bug fix

* use a key-value like naming

* update workflow API

* fix s3

* add test
2021-08-03 16:38:54 -07:00
Sven Mika
5107d16ae5
[RLlib] Add @Deprecated decorator to simplify/unify deprecation of classes, methods, functions. () 2021-08-03 18:30:02 -04:00
Alex Wu
fb3be7a7dd
Fix dataset inference nightly test ()
Co-authored-by: Alex Wu <alex@anyscale.com>
2021-08-03 14:17:11 -07:00
Ian Rodney
f3acae6eb6
[Autoscaler] Sync Files before Starting Docker () 2021-08-03 13:25:08 -07:00
Sven Mika
a708cca4bc
[RLlib, Testing] Add RLlib tests to nightly/weekly release test automation. () 2021-08-03 13:44:00 -04:00
Alex Wu
63e335caf2
Update build_pipeline.py () 2021-08-03 10:40:29 -07:00
Alex Wu
8efa6be913
[Dataset] Fix reading parquet from gcs ()
* .

* .

* comments

Co-authored-by: Alex Wu <alex@anyscale.com>
2021-08-03 10:10:42 -07:00
Kai Fricke
3a90804713
[Testing] Add RLlib release tests () 2021-08-03 12:34:27 -04:00
Sven Mika
924f11cd45
[RLlib] Torch algos use now-framework-agnostic MultiGPUTrainOneStep execution op (~33% speedup for PPO-torch + GPU). () 2021-08-03 11:35:49 -04:00
Sasha Sobol
5dbbaf7261
[autoscaler] Enforce per-node-type max workers ()
* Enforce per-node-type max workers

* type annonation

Co-authored-by: Ameer Haj Ali <ameerh@berkeley.edu>

* cleanup. comments. type annotations

* additional type annotation

Co-authored-by: Ameer Haj Ali <ameerh@berkeley.edu>

* additional cleanup. comments. type annotations

* _get_nodes_needed_for_request_resources to use FrozenSet

* comments

* whitespace

* [Placement Group] Fix resource index assignment between with bundle index and without bundle index pg ()

* [serve] Add Ray API stability annotations ()

* Support streaming output of runtime env setup to logger/driver ()

* [SGD] v2 prototype: ``WorkerGroup`` implementation ()

* wip

* formatting

* increase timeouts

* address comments

* comments

* fix

* address comments

* Update python/ray/util/sgd/v2/worker_group.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Update python/ray/util/sgd/v2/worker_group.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* address comments

* formatting

* fix

* avoid race condition

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* [RLlib] Discussion 3001: Fix comment on internal state shape (must be [B x S=state dim]). ()

* [autoscaler] GCP TPU VM autoscaler ()

* [Rllib] set self._allow_unknown_config ()

Co-authored-by: Sven Mika <sven@anyscale.io>

* [RLlib] Discussion 2294: Custom vector env example and fix. ()

* [docs] Link broken in Tune's page () ()

* [Serve] Fix response_model for class based view routes as well ()

* [serve] Fix single deployment nightly test ()

* [RLlib] SAC tuple observation space fix ()

* Support schema on read for csv/json ()

* [RLlib] New and changed version of parametric actions cartpole example + small suggested update in policy_client.py ()

* [gcs] Fix GCS related issues: ByteSizeLong and redis connection ()

* [runtime_env] Gracefully fail tasks when an environment fails to be set up ()

* [docs] update docs with pip requirements ()

* removed nodes_to_keep. cleanup

* formatting

* +comment

* treat max_workers=0 as 0 workers (as opposed to unlimited)

* fix wrong comment

* warning for inconsistent config

* terminate nodes with no matching node type right away

* quotes

* special handling for head node when enforcing max_workers per type. tests. cleanup

* cleanup comments and prints

* comments

* cleanup. removed special handling of head node.

* adding an eplicit non-None check in schedule_node_termination

* raise the exception

Co-authored-by: Ameer Haj Ali <ameerh@berkeley.edu>

Co-authored-by: Ameer Haj Ali <ameerh@berkeley.edu>
Co-authored-by: DK.Pino <loushang.ls@antfin.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
Co-authored-by: Sven Mika <sven@anyscale.io>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
Co-authored-by: Rohan138 <66227218+Rohan138@users.noreply.github.com>
Co-authored-by: amavilla <takashi.tameshige.jj@hitachi.com>
Co-authored-by: Jiao <sophchess@gmail.com>
Co-authored-by: Julius Frost <33183774+juliusfrost@users.noreply.github.com>
Co-authored-by: Eric Liang <ekhliang@gmail.com>
Co-authored-by: kk-55 <63732956+kk-55@users.noreply.github.com>
Co-authored-by: Yi Cheng <74173148+iycheng@users.noreply.github.com>
Co-authored-by: matthewdeng <matt@anyscale.com>
2021-08-03 11:31:32 -04:00