Commit graph

3593 commits

Author SHA1 Message Date
Alex Wu
0031723ace
[New scheduler] Object spilling (#12857) 2020-12-15 11:05:38 -08:00
architkulkarni
ba12fb1451
Fix for RLIMIT patch (#12882)
Implement new soft limit introduced by https://github.com/ray-project/ray/pull/12853.
2020-12-15 10:38:46 -08:00
Max Fitton
e077bc4206
[Release] Bump master to 1.2.0 for 1.1.0 release (#12856) 2020-12-15 09:40:26 -08:00
Simon Mo
b291dd4486
[Metrics] Call GetMeasureDoubleByName to prevent override (#12860) 2020-12-15 09:39:39 -08:00
Gekho457
5a142d5bd6
Use nightly images in all kubernetes examples. (#12868) 2020-12-14 20:49:41 -08:00
Simon Mo
b56db5a22f
[Serve] Wait for actor name to be cleaned up (#12215) 2020-12-14 15:09:43 -08:00
architkulkarni
231518e86f
[Serve] Support basic Starlette response types (#12811) 2020-12-14 17:03:56 -06:00
Eric Liang
1eb4ac12b1
Clip RLIMIT_NOFILE increase to avoid redis failing to start on Big Sur 2020-12-14 14:05:19 -08:00
SangBin Cho
69b0bc2132
[Logging] Use file handle temporalily (#12839) 2020-12-14 11:42:44 -08:00
Gekho457
11ce1dc743
Ray cluster CRD and example CR + multi-ray-cluster operator (#12098) 2020-12-14 10:26:01 -06:00
Tao Wang
35f7d84dbe
Revert heartbeat interval to keep ci stable (#12836)
* Revert heartbeat interval to keep ci stable

* fix missing one
2020-12-14 16:58:40 +08:00
Eric Squires
22c1968d62
Runing -> Running (#12826) 2020-12-13 22:23:48 -08:00
Ameer Haj Ali
aaa11941f6
[autoscaler] Fix flaky autoscaler test (#12829) 2020-12-13 17:09:30 -08:00
DK.Pino
153b24746c
[Placement Group] Refactor pg resource constrain in node manager (#12538)
* first version by pointer

* second version reference

* clean up

* add cpp ut

* lint

* extract LocalPlacementGroupManagerInterface

* lint

* fix commemt

* add idempotency test

* lint

* fix pg ut

* fix pg ut

* python lint

* fix pg ut timeout

* python lint

* fix comment

* lint

* lint
2020-12-12 23:32:15 -08:00
Eric Liang
bdc6624da8
Revert "[PlacementGroup]Add PlacementGroup wait python api (#12601)" (#12825)
This reverts commit 401d342602.
2020-12-12 12:13:48 -08:00
Richard Liaw
2f2bd884a3
[tune] upgrade gpytorch, bump default pytorch to 1.7.0 (#12776)
* upgrade gpytorch

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>

* pin

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>

* version-torch

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>

* fix-build

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
2020-12-12 10:35:33 -08:00
Richard Liaw
7e09f1d934
remove-xgboost-build (#12822)
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
2020-12-12 10:34:56 -08:00
Kai Fricke
5f04ade6ef
[tune] add more stoppers and stopper documentation (#12750)
* Add new stoppers & docs

* Add tests for maximum iteration stopper and trial plateau stopper

* Update python/ray/tune/stopper.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Update doc/source/tune/api_docs/stoppers.rst

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Update doc/source/tune/api_docs/stoppers.rst

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Apply suggestions from code review

* Apply suggestions from code review

* Update python/ray/tune/stopper.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-12-12 01:47:19 -08:00
Kai Fricke
905652cdd6
[tune] migrate xgboost callback api (#12745)
* Migrate to new-style xgboost callbacks

* Fix flaky progress reporter test

* Fix import error

* Take last value (not first)
2020-12-12 01:42:20 -08:00
Kai Fricke
42c70be073
[tune] Hyperopt: Directly accept category variables instead of indices (#12715)
* [tune] Hyperopt: Directly accept category variables instead of indices

* Fix interrupt test

* Update python/ray/tune/suggest/hyperopt.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Apply suggestions from code review

* Update python/ray/tune/suggest/hyperopt.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* lint

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-12-12 01:40:53 -08:00
Hao Zhang
0b1fbc5e83
[PR 1/6] Collective in Ray (#12637)
Co-authored-by: YLJALDC <dal177@ucsd.edu>
2020-12-12 01:26:36 -08:00
Alex Wu
aa64cd4534
[New scheduler] Fix test_global_state (#12586) 2020-12-11 21:47:01 -08:00
Edward Oakes
03d869d51c
Hold GIL while submitting (actor) tasks (#12803) 2020-12-11 21:47:16 -06:00
Edward Oakes
aec5c9879e
Add tests for atexit handler behavior (#12808) 2020-12-11 21:47:05 -06:00
Edward Oakes
6262ee1f76
Clarify docs for atexit behavior when using ray.kill (#12807) 2020-12-11 21:45:39 -06:00
Eric Liang
1ce745cf44
Add automatic local GC and plasma debug logs every 10 minutes by default (#12804) 2020-12-11 17:09:58 -08:00
Simon Mo
3d8c1cbae6
[Serve] Fix Serve Release Tests (#12777) 2020-12-11 11:53:47 -08:00
fangfengbin
9ded69fdaa
[Hotfix] Fix python client lint error (#12783) 2020-12-11 10:15:53 -08:00
Simon Mo
68d7fa2137
Fix exit_actor in asyncio mode (#12693) 2020-12-11 09:35:17 -08:00
Edward Oakes
699ded5328
[serve] Initial commit for CLI (#12770) 2020-12-11 10:31:29 -06:00
Tao Wang
295b6e5ce4
Split heartbeat message (#12535)
* first

* xxx

* Split heartbeat message

* only report resource usage when changed

* Fix GetAllResourceUsage

* Fix report resource usage

* Increase default heartbeat interval

* regularize heartbeat interval in test case
2020-12-11 21:19:57 +08:00
Stephanie Wang
86b0741026
[new scheduler] Allocate resources for spilled back task to a local view of the remote node (#12711)
* Force report heartbeats if remote resources may be dirty

* lint

* typo

* typo

* unit test

* debug

* Revert "lint"

This reverts commit 6dc7e982ffee98185665eb7c3c8fde0d91938919.

* Revert "Force report heartbeats if remote resources may be dirty"

This reverts commit cbfa9405197df62f874107d55b46715ceae2abd2.

* Local view of resources

* debug travis

* debug

* debug

* debug

* weaken test

* cleanups

* lint

* Revert "debug travis"

This reverts commit 11ff5f4f84e64e9fbd4eecda5b3c7fd07a7130a4.

* revert

* const view, remove unused
2020-12-10 22:43:29 -05:00
Barak Michener
b7f246c451
[ray_client] Include multiple facets of the Ray API (#12736) 2020-12-10 19:09:34 -08:00
Edward Oakes
62d6b0a558
Fix max_task_retries for named actors (#12762) 2020-12-10 18:24:55 -06:00
Edward Oakes
c7b6ec88ef
[serve] Make serve __del__ log DEBUG level (#12766) 2020-12-10 18:14:55 -06:00
Edward Oakes
3c44c0d3e4
[serve] Long polling for routes in http server (#12724) 2020-12-10 18:02:02 -06:00
Eric Squires
9f70293700
Remove debug extras from setup.py (#12751) 2020-12-10 16:23:11 -06:00
architkulkarni
3fd3cb96ed
[Utils] Add Queue async and batch methods (#12578) 2020-12-10 10:49:18 -06:00
Ian Rodney
38ba238606
[serve] Create FutureResults from ControllerAPI (#12577) 2020-12-10 10:44:08 -06:00
Kai Yang
e3b5deb741
[Multi-tenancy] Delete flag enable_multi_tenancy and remove old code path (#10573) 2020-12-10 19:01:40 +08:00
Ameer Haj Ali
2f8e308444
[autoscaler] LoadMetrics missed logger.debug (#12714) 2020-12-09 17:19:36 -08:00
Richard Liaw
974570b4fb
oops (#12728)
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
2020-12-09 13:38:10 -08:00
Edward Oakes
c9873cdbc3
[Serve] Remove unused assign_request wrapper (#12721) 2020-12-09 12:22:43 -08:00
Ian Rodney
19542c5eb0
[docker] Default to ray-ml image (#12703) 2020-12-09 11:49:16 -08:00
Alex Wu
bd7e26b768
[Autoscaler] Temporarily suppress "Removed stale ip mappings" message. (#12689) 2020-12-08 21:55:10 -08:00
Barak Michener
dc4b5c7aa3
[ray_client] Passing actors to actors (#12585)
* start building tests around passing handles to handles

Change-Id: Ie8c3de5c8ce789c3ec8d29f0702df80ba598279f

* clean up the switch statements by moving to a method, implement state tranfer, extend test

Change-Id: Ie7b6493db3a6c203d3a0b262b8fbacb90e5cdbc5

* passing

Change-Id: Id88dc0a41da1c9d5ba68f754c5b57141aae47beb

* flush out tests

Change-Id: If77c0f586e9e99449d494be4e85f854e4a7a4952

* formatting

Change-Id: I497c07cee70b52453b221ed4393f04f6f560061e

* fix python3.6 and other attributes

Change-Id: I5a2c5231e8a021184d9dfc3e346df7f71fc93257

* address documentation

Change-Id: I049d841ed1f85b7350c17c05da4a4d81d5cb03df

* formatting

Change-Id: I6a2b32a2466ffc9f03fc91ac17901b9c1a49505c

* use the pickled handle as the id bytes for actors

Change-Id: I9ddcb41d614de65d42d6f0382fe0faa7ad2c2ade

* pydoc

Change-Id: I9b32a0f383d5ff5ac052e61929b7ae3e42a89fc5

* format

Change-Id: Iac0010bb990a4025a98139ab88700030b2e9e7f5

* todos

Change-Id: I7b550800cf7499403e8a17b77484bc46f20f0afc

* tests

Change-Id: If8ebf6a335baeb113c1332acc930c41a6b4f5384

* fix lint

Change-Id: I019f41e0ec341d39bbbbd39aa43d9fb5f8b57cf0

* nits

Change-Id: I2e6813d8db34f4ce008326faa095d414c10eee95

* add some tricky, python3.6-troublesome type checking

Change-Id: Ib887fc943a6e7084002bc13dfbe113b69b4d9317
2020-12-08 21:54:55 -08:00
Ameer Haj Ali
a4dbb271bd
[hotfix][autoscaler] Request resources refactor2 (#12661)
* prepare for head node

* move command runner interface outside _private

* remove space

* Eric

* flake

* min_workers in multi node type

* fixing edge cases

* eric not idle

* fix target_workers to consider min_workers of node types

* idle timeout

* minor

* minor fix

* test

* lint

* eric v2

* eric 3

* min_workers constraint before bin packing

* Update resource_demand_scheduler.py

* Revert "Update resource_demand_scheduler.py"

This reverts commit 818a63a2c86d8437b3ef21c5035d701c1d1127b5.

* reducing diff

* make get_nodes_to_launch return a dict

* merge

* weird merge fix

* auto fill instance types for AWS

* Alex/Eric

* Update doc/source/cluster/autoscaling.rst

* merge autofill and input from user

* logger.exception

* make the yaml use the default autofill

* docs Eric

* remove test_autoscaler_yaml from windows tests

* lets try changing the test a bit

* return test

* lets see

* edward

* Limit max launch concurrency

* commenting frac TODO

* move to resource demand scheduler

* use STATUS UP TO DATE

* Eric

* make logger of gc freed refs debug instead of info

* add cluster name to docker mount prefix directory

* grrR

* fix tests

* moving docker directory to sdk

* move the import to prevent circular dependency

* smallf fix

* ian

* fix max launch concurrency bug to assume failing nodes as pending and consider only load_metric's connected nodes as running

* small fix

* request_resources -> min workers

* test fixes

* add race condition tests

* Eric

* fixes

* semi final

* semi final

* lint

* lint

Co-authored-by: Ameer Haj Ali <ameerhajali@ameers-mbp.lan>
Co-authored-by: Alex Wu <alex@anyscale.io>
Co-authored-by: Alex Wu <itswu.alex@gmail.com>
Co-authored-by: Eric Liang <ekhliang@gmail.com>
Co-authored-by: Ameer Haj Ali <ameerhajali@Ameers-MacBook-Pro.local>
2020-12-08 18:41:30 -08:00
Philipp Moritz
343b479ae2
[TEST] Fix Ray windows build for debugger (#12671)
* Fix Ray windows build for debugger

* update
2020-12-08 18:12:48 -08:00
Stephanie Wang
50f28811ac
[new scheduler] Always spill back to a feasible node if the local node is not feasible (#12557)
* fix

lint

* feasible nodes

* Enable test, cleanup

* Revert "fix"

This reverts commit aef81d04c0b4560b758f846e1afdafbdb5552efe.

* unit test

* doc
2020-12-08 13:46:58 -05:00
Kai Fricke
df10b84113
[Release] release tests yamls for Tune & GPU (#12496) 2020-12-08 10:15:07 -08:00