Commit graph

8518 commits

Author SHA1 Message Date
Alex Wu
9f8f108e3f
[deflek] Split test failure into test failure 4 (#16264)
* .

* .

Co-authored-by: Alex Wu <alex@anyscale.com>
2021-06-07 11:54:55 -07:00
Lixin Wei
75196cf7f4
[scheduler] Clean up TaskRequest (#16288) 2021-06-07 11:38:34 -07:00
Edward Oakes
418dd1e8b9
fix serve start namespace issue and add test (#16291) 2021-06-07 11:30:31 -07:00
Siyuan (Ryans) Zhuang
480e5e822e
Inital workflow API implementation (#16174) 2021-06-07 10:00:15 -07:00
SangBin Cho
f867c27eda
[Object spilling] Fix race condition that deletes files at the wrong timing. (#16153)
* Error fix.

* remove debug code

* Add unit test

* Fix a test failure
2021-06-07 09:56:55 -07:00
SangBin Cho
3572d0837e
[Test] Dask on ray sort nightly (#16213)
* Make dask on ray sort works

* lint

* revert unrelated change
2021-06-06 15:58:48 -07:00
SangBin Cho
03c33cf443
add a streaming shuffl etest (#16258) 2021-06-06 15:58:14 -07:00
Eric Liang
1d8cb2d19e
Add event stats documentation, fix misc race condition (#16236)
* update

* stats

* udpate

* fix
2021-06-06 12:44:30 -07:00
architkulkarni
b88163f010
[Core] [runtime env] Fix injection of ray[default] (#16275) 2021-06-05 17:32:50 -05:00
architkulkarni
b3a0b97737
Revert "[Core] [runtime env] Inject ray[default] into pip dependencies (#16268)" (#16274)
This reverts commit e5fad4bc2d.
2021-06-05 21:26:19 +03:00
Eric Liang
ca861ee47f
update (#16270) 2021-06-05 11:16:01 -07:00
Dmitri Gekhtman
7d1e7a0d4f
[autoscaler] Fix local node provider (#16202)
* Don't override resources for local node provider.

* Wip

* Local node provider prep logic

* ../python/ray/autoscaler/local/defaults.yaml

* wip

* Fix example-full

* defaults comment

* wip

* head type max workers

* sync-state

* No docker

* Fix

* external head ip option

* wip

* move external_ip out of tags

* Update examples

* Update comment

* Skip local defaults

* Config test

* Test external ip

* Change ray start commands to what they were before

* missing yamls

* Fix test

* Remove scary Docker

* Fixes

* Extra test

* address comments

* fixes pre-single-node-type-attempy

* rewrite comment a bit

* One type

* fix

* get rid of pdb

* no placeholders

* fix

* worker nodes and head node optional during launch

* fix

* fix again

* config comment fixes

* mock -> aws, not local

* Update python/ray/autoscaler/_private/local/config.py

Co-authored-by: Ian Rodney <ian.rodney@gmail.com>

* second pop fixed

* Explanatory comments for config logic

* deprecation comments

* Update python/ray/autoscaler/_private/local/config.py

Co-authored-by: Ameer Haj Ali <ameerh@berkeley.edu>

* update test

* fix

* More descriptive name for local provider check

* Remove external-ip from example minimal and add a more detailed doc string.

* Make clearer the equivalence between a ray restart and non-empty ray-start commands

* extra comment

* Update python/ray/autoscaler/_private/local/node_provider.py

* Update python/ray/autoscaler/_private/commands.py

* Update python/ray/autoscaler/_private/commands.py

* Update python/ray/autoscaler/_private/util.py

* lint

* Update python/ray/autoscaler/_private/local/node_provider.py

Co-authored-by: Ameer Haj Ali <ameerh@berkeley.edu>

Co-authored-by: Ian Rodney <ian.rodney@gmail.com>
Co-authored-by: Ameer Haj Ali <ameerh@berkeley.edu>
2021-06-05 19:29:19 +03:00
Dmitri Gekhtman
e58ba66681
[gcp][doc][minor] project_id is required (#16266) 2021-06-05 01:00:11 -07:00
Chris K. W
2e11ac678f
[autoscaler] Additional Autoscaler Metrics (#16198) 2021-06-04 23:19:17 -07:00
architkulkarni
e5fad4bc2d
[Core] [runtime env] Inject ray[default] into pip dependencies (#16268)
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
2021-06-05 00:22:33 -05:00
architkulkarni
6c99267972
[Core] [runtime_env] add get_release_wheel_url() (#16267) 2021-06-04 22:00:17 -05:00
Stephanie Wang
dd73e8d31b
[core] Add object store debug information (#16232)
* debug

* todo

* periodic dump

* Build and debug

* x

* debug

* more debug
2021-06-04 19:42:00 -07:00
yncxcw
e13509075d
[Core] Make the the exit type explict for workers being killed TryKillingIdleWorkers (#16211) 2021-06-04 18:23:36 -07:00
Lixin Wei
59a2879216
[New Scheduler] Remove Useless Fields in Cluster Resource Data (#16254)
* non-tests done

* test modifed
2021-06-04 18:00:13 -07:00
Clark Zinzow
227f252c39
[Release] Release 1.4.0 stress tests, scalability envelope, and microbenchmark release logs (#16228) 2021-06-04 16:36:41 -07:00
Dmitri Gekhtman
a60ee3a8b2
[autoscaler][kubernetes][minor] latest images everywhere (#16205)
* latest images everywhere

* add back some documentation on the images

* Doc update
2021-06-04 16:01:39 -07:00
Eric Liang
527d51b83a
Allow configuring internal config with RAY_{name} env vars. 2021-06-04 15:37:31 -07:00
Eric Liang
472fe46a75
skip on win (#16256) 2021-06-04 15:36:55 -07:00
Lixin Wei
cf58cd76c7
[Logging] Disable Core Dumps in Fatal Logging (#16189) 2021-06-04 11:44:08 -07:00
Ian Rodney
ba14b1c538
skip test_delay_in_rewriting_environment (#16255) 2021-06-04 09:50:02 -07:00
Gerges Dib
f8cf4a1985
[RLlib] Fixed import tensorflow when module not available (#16171) 2021-06-04 10:07:59 +02:00
Simon Mo
d6b3050632
[Buildkite] Wheels and Docker fixup (#16241) 2021-06-04 00:48:12 -07:00
Ian Rodney
799af7d7c0
[client] Better Error Messages (#16163) 2021-06-04 00:32:21 -07:00
SongGuyang
331ea6b72d
[C++ worker] fix doc (#16239) 2021-06-04 13:41:47 +08:00
Yi Cheng
dea178caac
[core] Convert the log from exception to warning for setup function (#16225) 2021-06-03 23:53:29 -05:00
architkulkarni
6be5ec8f39
[Core] [runtime env] Fix test_get_master_wheel_url (#16234) 2021-06-03 23:09:43 -05:00
architkulkarni
2d26c6ea5d
[Core] suppress unpickle error for function broadcasting (#16223) 2021-06-03 22:59:24 -05:00
architkulkarni
0dcd996475
[core] [runtime_env] Remove opentelemetry from pip dependencies in test (#16233) 2021-06-03 22:18:47 -05:00
Simon Mo
ab93a3a64a
[CI] Buildkite upload wheels to S3 and docker (#16138) 2021-06-03 20:10:31 -07:00
architkulkarni
4efbd9680e
[Core] [runtime env] use ray instead of ray[all] when adding ray dependency (#16229) 2021-06-03 21:29:49 -05:00
Eric Liang
608991999c
Fix release resources race that leads to extra worker launches (#16184) 2021-06-03 18:35:45 -07:00
Eric Liang
a9db4e62cb
Unlimited plasma allocations by falling back to a filesystem allocator (off by default) (#16097) 2021-06-03 18:35:09 -07:00
Simon Mo
b376fd3458
[Serve] Create default namespace when ray is not initialized (#16227) 2021-06-03 17:14:52 -07:00
Sven Mika
e2be41b407
[RLlib] MARWIL + BC: Various fixes and enhancements. (#16218) 2021-06-03 22:29:00 +02:00
Ian Rodney
acfb484f5c
[Tracing] Delay Opentelemetry Import to Avoid Pickling (#16177) 2021-06-03 12:07:12 -07:00
Ian Rodney
22bd7cebeb
[Client][Proxy] Prevent Logstream from Timing Out when Delays in DataClient (#16180) 2021-06-03 11:59:52 -07:00
Alex Wu
fa292a4edf
[Doc] Document memory and object store memory as autoscaler resources (#16210) 2021-06-03 10:10:03 -07:00
Ian Rodney
206802b96f
[client] Fix ClientBuilder for Local Clusters (#16204)
* Fix client builder

* Make tests actually run in CI (required marking a few Windows tests as flaky)
2021-06-03 08:27:37 -07:00
mwtian
51da90aa09
Rollforward #f14f197 (#16201) 2021-06-03 07:17:58 -07:00
Eric Liang
43c97c2afb
Disable timeline events collection in Ray by default (#15989) 2021-06-02 18:04:29 -07:00
architkulkarni
46acac03ae
[Core] [runtime env] Fix bug where None is added to pip dependencies when built from source (#16173) 2021-06-02 18:12:59 -05:00
Alex Wu
e1da31f149
[dashboard] Include ray session name in dashboard snapshot (#16199)
* .

* .

* .

Co-authored-by: Alex Wu <alex@anyscale.com>
2021-06-02 15:07:06 -07:00
Ian Rodney
513196d71b
[Client] Make dir() work for ClientActorHandle (#16157) 2021-06-02 14:51:17 -07:00
Ian Rodney
ca3737c6aa
[Client][Test] Make "test_client_references.py" medium (#16193) 2021-06-02 13:13:31 -07:00
Alex Wu
9942505b63
Revert "[Client] Make Client{ObjectRef,ActorRef} subclasses of their server-side counterparts (#16110)" (#16196)
This reverts commit f14f197d42.
2021-06-02 10:31:01 -07:00