hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-06 18:41:40 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
9460 commits 227 branches 67 tags 234 MiB
Commit graph

5056 commits

Author SHA1 Message Date
Yi Cheng
29352e7fa3
[workflow] Fix some usability issues (#17284) 2021-07-23 11:39:49 -07:00
Eric Liang
df7fe8dd6d
[data] Cleanup Block type by dropping Generic[T] (#17276) 
* wip

* update

* update

* quotes
 2021-07-23 09:23:06 -07:00
Dmitri Gekhtman
e701ded54f
[autoscaler] Tweaks to support remote (K8s) operators (#17194) 
* node provider hooks

* disable node updaters

* pending means not completed

* draft wip

* add flag to autoscaler initialization

* Explain

* terminate unhealthy nodes

* fix, add event summarizer message

* Revert node provider

* remove hooks from autoscaler.py

* avert indent apocalypse

* wip

* copy-node-termination-logic

* Added a test

* Finish tests

* test cleanup

* Move disable node updaters to config yaml

* fix

* Drop arg
 2021-07-23 11:30:18 -04:00
Edward Oakes
811eb4b092
[debugger] Enable attaching to breakpoints on remote nodes (off by default) (#17275) 2021-07-23 09:37:40 -05:00
Siyuan (Ryans) Zhuang
57b2328e7b
[workflow] Virtual actor writer - Part I (#17256) 
* update readonly virtual actor

use signature module

refactoring workflow

new execution interface

advance progress of a workflow

update storage

last_step_of_workflow

prevent setting dynamic output of "output.json" in workflow directory

use alternative exception

* fix

* fix comments

* better step names

* add TODO

* fix comments

* log errors when retry

* fix storage test
 2021-07-22 22:53:04 -07:00
Clark Zinzow
1ab4f0def7
[Datasets] Port read_binary_files to Datasource API. (#17225) 2021-07-22 19:03:10 -07:00
Yi Cheng
5f4d9085d2
[workflow] workflow ci enable (#17255) 
* Enable workflow tests

* update

* Fix one bug
 2021-07-22 17:59:24 -07:00
Simon Mo
b9b79cd5f4
[Runtime Env] Support per task/actor uri override job_config (#17252) 2021-07-22 16:37:43 -07:00
Simon Mo
aaf8afb78d
[Runtime Env] Add a test for working_dir inheritance (#17245) 2021-07-22 10:48:25 -07:00
Yi Cheng
760b11263a
[workflow] Workflow manager API (#17226) 2021-07-22 09:30:52 -07:00
Richard Liaw
a78a2263e5
[RLlib] Fix reverted RockPaperScissors Pettingzoo example (#16896) 2021-07-22 10:55:07 -04:00
xwjiang2010
f3a31a3b94
[tune] Add test for flatten_dict. (#17241) 
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
 2021-07-21 22:01:01 -07:00
Alexis DUBURCQ
362f7b7c56
[RLlib] Do not deepcopy input dict for efficiency and consistency with similar methods. (#15709) 
Co-authored-by: Alexis Duburcq <alexis.duburcq@wandercraft.eu>
 2021-07-21 20:09:41 -07:00
Chen Shen
70ab8aa1d4
Revert "[core] Do not spill back tasks blocked on args to blocked nodes (#16488)" (#17247) 
This reverts commit dad8db46e1.
 2021-07-21 19:41:35 -07:00
Vince Jankovics
05c9dfbbda
[RLlib] CV2 to Skimage dependency change (#16841) 2021-07-21 22:24:18 -04:00
Clark Zinzow
05a7102104
[Datasets] Port read_parquet to Datasource API. (#17230) 
* Port read_parquet to Datasource API.

* Update to new block representation.

* Remove unused _parse_paths.

* Support column selection.

* Formatting.

* Add column selection test.
 2021-07-21 17:39:39 -07:00
Simon Mo
7b44dd8ecb
Revert "[core] remove opencensus/prometheus_exporter dependencies" (#17251) 
This reverts commit 64874e1877.
 2021-07-21 16:57:47 -07:00
Yi Cheng
5accfa662c
[workflow] Test for better coverage (#17233) 
* update

* workflow.init

* update

* update

* update tests

* check

* up

* update

* update

* check

* merge

* fix tests

* update

* add tests

* up

* format

* add space

* Update test_storage.py

Co-authored-by: Siyuan <suquark@gmail.com>
 2021-07-21 14:48:36 -07:00
Antoni Baum
2e37826458
[tune] Function API support for ResourceChangingScheduler (#17150) 
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
 2021-07-21 14:14:12 -07:00
Siyuan (Ryans) Zhuang
a550eb3e97
[Workflow] Update workflow storage interface (#17222) 
* workflow.init

* update tests

* add tests
 2021-07-21 11:51:01 -07:00
chenk008
afd59be8ca
[Core] Add worker resource limit (#17179) 
* add resource restricted

* fix test

* lint

* lint
 2021-07-21 22:00:34 +08:00
Kai Fricke
64874e1877
[core] remove opencensus/prometheus_exporter dependencies (#17182) 2021-07-21 12:57:31 +01:00
Simon Mo
250f0c24e0
[Runtime Env] Refactor local dev mode for linking ray packages (#17227) 2021-07-21 00:48:58 -07:00
Kai Fricke
e881c6cff8
[core] remove aiohttp dependencies (#17181) 2021-07-21 07:18:19 +01:00
Stephanie Wang
dad8db46e1
[core] Do not spill back tasks blocked on args to blocked nodes (#16488) 2021-07-20 17:13:02 -07:00
Eric Liang
877076160e
[data] Enable zero-copy access to underlying Arrow tables (#17192) 2021-07-20 16:38:21 -07:00
Eric Liang
d6e91a5b46
Update PublicAPI annotations #17224 2021-07-20 16:37:53 -07:00
Clark Zinzow
09f32b68d3
[Datasets] Slice off S3 protocol from S3 URIs. (#17219) 
* Ensure that S3 protocols are sliced off of S3 URIs.

* Use urllib to parse and trim URI to path.
 2021-07-20 15:23:35 -07:00
Clark Zinzow
08a50bf3b7
[Datasets] Allow for Parquet metadata file to be missing. (#17217) 
* Allow for Parquet metadata file to be missing.

* Remove for-else.
 2021-07-20 15:20:26 -07:00
Ian Rodney
e6bf0a8ea6
[autoscaler][docstring] Add Docstring for StandardAutoscaler ctor (#17213) 2021-07-20 12:19:54 -07:00
Patrick Ames
efed07023f
[autoscaler] Custom AWS network interface error condition tests and missing security group bug fix. (#17207) 2021-07-20 11:17:27 -07:00
Jialing He
492076806d
[object store] Assign the object owner in ray.put() (#16833) 2021-07-20 11:06:00 -07:00
Amog Kamsetty
4ece5247d6
[Datasets] to_torch no DataLoader (#17211) 2021-07-20 11:05:17 -07:00
Siyuan (Ryans) Zhuang
859cba7655
[Workflow] Remove namespace in workflow 2021-07-20 11:04:46 -07:00
Yi Cheng
8253064163
[workflow] workflow error handling (#17175) 2021-07-20 11:03:53 -07:00
Simon Mo
908aa2c7f3
Fix runtime env and dispatch queue take 2 (#17163) 2021-07-20 10:24:08 -07:00
SangBin Cho
d6b6356173
[Core] Properly call shutdown instead of deleting a reference (#17096) 
* Properly call shutdown instead of deleting a reference

* Add unit tests

* Add test ray shutdown

* Formatting

* format2

* Revert main logic to see if windows issue still fail

* Skip tests for windows.

* formatting

* Try fixing flakiness

* Remove node removed code path

Co-authored-by: Eric Liang <ekhliang@gmail.com>
 2021-07-20 08:22:33 -07:00
Antoni Baum
5e9b680e39
[docs] Add LightGBM-Ray docs, update XGBoost-Ray docs (#17188) 2021-07-20 16:06:47 +01:00
Siyuan (Ryans) Zhuang
8efc04a8a6
[Core] Actor namespace (#17178) 
* set actor namespace in Python on creation

* get actor with namespace in Python

* update message
 2021-07-19 21:51:04 -07:00
matthewdeng
fef74aa94f
[sgd] add placement group support (#17037) 
* [sgd] add placement group support

* add logic for removing placement group upon shutdown

* set placement group; add tests

* address comments - add timeout and improve error handling

* remove unused import

* mock SGD_PLACEMENT_GROUP_TIMEOUT_S
 2021-07-19 21:50:37 -07:00
Siyuan (Ryans) Zhuang
9ca6bda3a1
[Workflow] Fix recovery storage mismatch issue (#17166) 
* fix recovery path issue and add test

* add TODOs
 2021-07-19 21:49:12 -07:00
dependabot[bot]
2de7b8f084
[tune](deps): Bump tune-sklearn in /python/requirements/tune (#17173) 
Bumps [tune-sklearn](https://github.com/ray-project/tune-sklearn) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/ray-project/tune-sklearn/releases)
- [Commits](https://github.com/ray-project/tune-sklearn/compare/v0.3.0...v0.4.0)

---
updated-dependencies:
- dependency-name: tune-sklearn
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-07-19 19:42:54 -07:00
Amog Kamsetty
c9522e9a6f
Remove requests from Core Dependencies (#17066) 
* remove requests

* update

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>

* update

* lint

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
 2021-07-19 19:38:29 -07:00
Eric Liang
fabba96fad
Re-merge large function def, skipping test failing on Windows (#17191) 2021-07-19 18:03:26 -07:00
Amog Kamsetty
777921b2e7
[dependencies] vendor colorama (#17183) 2021-07-19 16:29:29 -07:00
Eric Liang
d59da075a6
Re-merge TMPDIR support, but only for Linux. OSX requires RAY_TMPDIR (#17190) 2021-07-19 15:45:03 -07:00
Patrick Ames
34789b3e56
[autoscaler] Add support for custom EC2 instance network interfaces (#14080) 
* [autoscaler] Add support for custom EC2 instance network interfaces.

* [autoscaler] Add unit tests for custom EC2 network interfaces and support for AWS node provider stubs.
 2021-07-19 17:21:21 -04:00
Amog Kamsetty
cb74053ee5
Retry remove gpustat dependency (#17115) 
* remove gpustat

* move psutil imports
 2021-07-19 11:14:10 -07:00
Siyuan (Ryans) Zhuang
1fbfbfc55b
[Serializatioin] Bump pickle5 version (#17124) 2021-07-19 10:40:38 -07:00
Siyuan (Ryans) Zhuang
9b110f9344
[Workflow] Update API (#17165) 
* actor_id same as workflow_id

* @workflow.actor -> @workflow.virtual_actor

* readonly decorator

* run/run_async for virtual actor

* get_or_create for virtual actor

* update doc

* run/run_async for steps

* update tests

* update comments
 2021-07-19 10:19:46 -07:00