Commit graph

4235 commits

Author SHA1 Message Date
Stefan Schneider
49ba51979e
Functions for restoring from last or best checkpoint (#14735)
Adds a helper function to retrieve the latest checkpoint after selecting the best trial according to a metric.
2021-04-06 12:19:09 +02:00
Edward Oakes
43932f5330
[serve] Health checking in the controller instead of using max_restarts (#15047) 2021-04-05 21:22:40 -05:00
Clark Zinzow
f17e59093d
[dask-on-ray] Fix Dask-on-Ray scheduler break caused by changing internal Dask API. (#15100) 2021-04-05 17:58:44 -07:00
Yi Cheng
afc92130fa
Unpack runtime env to runtime_resource (#15111) 2021-04-05 17:35:31 -07:00
Dmitri Gekhtman
ee291ae762
[kubernetes][test][minor] Fix K8s test by an adding an underscore. (#15122) 2021-04-05 15:18:20 -07:00
Ian Rodney
29566c3527
[Code Cleanup] Rename to Allowlist + Denylist (#15117)
Co-authored-by: Simon Mo <simon.mo@hey.com>
2021-04-05 13:58:18 -07:00
Ian Rodney
c1bffc2784
[Core] Pick Up GCS Port From Env Variable (#15097) 2021-04-05 13:57:55 -07:00
Richard Liaw
6269140ef8
[minor] improve warning message for Ray (#15005) 2021-04-05 12:39:42 -07:00
Yi Cheng
5806e726f4
[core] Internal kv support in gcs (#14656)
* server side ready

* client size

* py

* fix

* up

* format

* add files

* add pyx

* up

* up

* up

* add keys

* format

* update

* format

* add unittests

* add files

* up

* up

* fix

* up

* fix thread issue

* format

* fix

* Fix

* format

* fix

* more

* fix conflict

* fix

* fix order

* format

* compiling fix

* lint

* fix

* fix some

* some fix

* fix comment

* fix name

* format

* fix compatible issue

* fix name

* fix lint

* disconnect safe

* up

* format

* fix

Co-authored-by: Yi Cheng <singye888@gmail.com>
2021-04-05 10:26:46 -07:00
Richard Liaw
e72f6b0377
Fix ray[full] -> ray[cluster] #15112
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
2021-04-05 09:55:00 -07:00
valiantljk-tt
9b85b0b82b
[runtime_env] Continue hashing if data is none (#14906) 2021-04-05 11:24:09 -05:00
architkulkarni
0d1b30620b
[Serve] Un-revert #15000 Set controller and HTTP proxy num_cpus=0 by default (#15092) 2021-04-05 09:58:55 -05:00
Dmitri Gekhtman
47a3bd7b67
[kubernetes][autoscaler] Rename autoscaler K8s module (#15101) 2021-04-04 15:30:00 -07:00
Dmitri Gekhtman
e0ed549006
return False, don't raise (#15107) 2021-04-03 14:04:50 -07:00
Yi Cheng
1920db0a64
Deflak dask related task on win32 (#15102)
* disable dask test on win32

* disable dask test on win32

* disable dask test on win32

* disable dask test on win32
2021-04-03 12:26:29 -07:00
Richard Liaw
e9f365ad2e
[tune] support skopt/bayesopt search restore (#15075) 2021-04-03 12:24:30 -07:00
Alex Wu
aea28c53ce
. (#15093)
Co-authored-by: Alex <alex@anyscale.com>
2021-04-02 16:39:38 -07:00
Yi Cheng
4caf7a511d
Deflaky test failure in win32 (#15090)
* deflake win32 failure

* skip dask on w32
2021-04-02 14:56:24 -07:00
architkulkarni
d8f8583e80
Revert "[Serve] Set controller and HTTP proxy num_cpus=0 by default (#15000)" (#15091)
This reverts commit 39aa01fc2c.
2021-04-02 13:01:57 -07:00
Yi Cheng
ecb94b3fe9
Add test case to check job conf compatible issue (#15082) 2021-04-02 12:03:21 -07:00
Dmitri Gekhtman
42565d5bbe
[autoscaler] Fix update/terminate race condition (#15019)
Co-authored-by: AmeerHajAli <ameerh@berkeley.edu>
2021-04-02 11:57:02 -07:00
Edward Oakes
96cc7897f7
[serve] Use longest prefix matching for path routing (#15041) 2021-04-02 12:01:47 -05:00
architkulkarni
39aa01fc2c
[Serve] Set controller and HTTP proxy num_cpus=0 by default (#15000) 2021-04-02 12:01:22 -05:00
Dmitri Gekhtman
6f81ec1998
[kubernetes][test] Operator test tweaks. (#15074) 2021-04-02 09:20:52 -07:00
Kai Fricke
8de66fce3d
[tune] Improve BOHB/ConfigSpace dependency check (#15064) 2021-04-02 10:19:49 +02:00
Alex Wu
f52c855704
[core] Fix placement group GPU assignment bug (#15049) 2021-04-01 17:46:09 -07:00
Yi Cheng
d4c20c970b
[core] Fix UTIL worker issue (#14925)
* Fix

* format

* more

* format

* fix

* fix

* fix comment

* fix test failure
2021-04-01 17:36:45 -07:00
Simon Mo
c9dac9328e
[Serve] Fix serializing nested fields in Pydantic (#15069) 2021-04-01 17:20:34 -07:00
Alex Wu
4fba05ae4d
[core] Hybrid scheduling policy. (#14790) 2021-04-01 16:59:59 -07:00
Edward Oakes
346994745a
[serve] Get handle in starlette endpoint constructor instead of lazily (#15066) 2021-04-01 16:07:28 -05:00
Ian Rodney
22c1aeb240
[Tests] Skip autoscaler tests on Windows (#15033) 2021-04-01 10:16:42 -07:00
SangBin Cho
005cff0092
Revert "Revert "[Core] Implement long polling-based pubsub to reduce … (#14909) 2021-04-01 09:03:15 -07:00
Kai Fricke
d33b0e4bc3
[tune] Reconcile placement groups every N seconds to avoid bottlenecks when running many short trials (#15011)
Closes a release blocking issue
2021-04-01 17:04:44 +02:00
Hao Chen
3e1a0439b7
Fix concurrent actor starting too many threads. (#14927) 2021-04-01 19:58:18 +08:00
Ameer Haj Ali
e02bd990d8
Move monitor.py to autoscaler/_private directory (#15050)
Co-authored-by: Ameer Haj Ali <ameerhajali@Ameers-MacBook-Pro.local>
2021-04-01 07:41:47 +03:00
SangBin Cho
463e9b2ef9
Try fixing it. (#15046) 2021-03-31 16:31:47 -07:00
Stephanie Wang
a86a7a6a98
[core] Cap total memory used by executing tasks' arguments (#15027)
* Task dependency map

* Pinned args threshold

* Unit test and fix

* no leaks

* update

* update

* remove assertion
2021-03-31 15:38:40 -07:00
Edward Oakes
126b9a6c14
[serve] Add basic upscaling test using cluster_utils (#15044) 2021-03-31 17:18:02 -05:00
Alex Wu
70f45af541
Deflake test_failure (#15026) 2021-03-31 14:59:38 -07:00
Edward Oakes
4061b72f2e
[serve] Add serve.get_deployment() API (#14953) 2021-03-31 14:57:39 -05:00
Simon Mo
57256b456a
[serve] Make sure test_imported_backend is ran (#15043) 2021-03-31 14:32:45 -05:00
Yi Cheng
4480132229
[core] Integration runtime_env with ray client (#14881)
* server side ready

* client size

* py

* fix

* up

* format

* add files

* add pyx

* up

* up

* up

* add keys

* format

* update

* format

* add unittests

* add files

* up

* up

* fix

* up

* fix thread issue

* format

* fix

* update proto

* Fix

* format

* fix

* more

* fix conflict

* fix

* fix order

* format

* add

* up

* compiling fix

* lint

* fix

* format

* fix some

* some fix

* fix comment

* test cases

* add test

* comments

* fix name

* format

* fix

* revert gcs-kv

* fix comments

* fix failure

* fix test

* format

* fix timeout

* fix

* fix

* fix

* format

* format

* fix flaky test

Co-authored-by: Yi Cheng <singye888@gmail.com>
2021-03-31 11:39:34 -07:00
Clark Zinzow
91cf272c2e
[Core] Exit autoscaler with a non-zero exit code upon handling SIGINT/SIGTERM (#14518) 2021-03-31 10:08:02 -07:00
Ian Rodney
32e50b8c67
[Docker] Run docker stop in parallel (#14901)
* first pass at parallel docker stop

* real impl

* use env var variable

* lint fix
2021-03-31 08:41:52 -07:00
Edward Oakes
107effb370
[serve] Add tests for reconnecting to cluster with ray client (#15029) 2021-03-31 10:08:12 -05:00
Edward Oakes
12f5e5ab62
[serve] Small cleanup in HTTP proxy (#15028) 2021-03-31 09:18:11 -05:00
Ian Rodney
73fb5d6022
[Autoscaler][Docker] Make disable_shm_size_detection more usable (#14913) 2021-03-30 18:10:09 -07:00
Siyuan (Ryans) Zhuang
3aa39142db
[Core] Remove code paths that run plasma store as a process (#14924)
* enable plasma store as thread by default

remove unused code path that runs plasma store as a process
2021-03-30 16:19:03 -07:00
Clark Zinzow
ccb0cdaa35
Revert "skip on windows (#14988)" (#15017)
This reverts commit fe39c88a57.
2021-03-30 11:47:39 -05:00
Edward Oakes
c5e7ed5671
Revert "Add support for Python 3.9 (#12613)" (#15003)
This reverts commit 208cde8d9b.
2021-03-30 08:38:54 -05:00