Commit graph

2735 commits

Author SHA1 Message Date
Robert Nishihara
a8efb214de
Fix a few flaky tests (#9709)
Fix test_custom_resources, Remove test_pandas_parquet_serialization, Better error message for test_output.py, Potentially fix test_dynres::test_dynamic_res_creation_scheduler_consistency
2020-07-25 17:11:38 -07:00
SangBin Cho
d49b19c24c
[Stats] Improve Stats::Init & Add it to GCS server (#9563) 2020-07-25 10:42:08 -07:00
krfricke
9f3570828a
[tune] move jenkins tests to travis (#9609)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
Co-authored-by: Kai Fricke <kai@anyscale.com>
2020-07-24 21:22:54 -07:00
Maksim Smolin
d364afbd31
[Autoscaler] CLI Logger docs (#9690)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-07-24 16:59:25 -07:00
mehrdadn
56d2cf6479
Shellcheck rewrites (#9597)
* Fix SC2001: See if you can use ${variable//search/replace} instead.

* Fix SC2010: Don't use ls | grep. Use a glob or a for loop with a condition to allow non-alphanumeric filenames.

* Fix SC2012: Use find instead of ls to better handle non-alphanumeric filenames.

* Fix SC2015: Note that A && B || C is not if-then-else. C may run when A is true.

* Fix SC2028: echo may not expand escape sequences. Use printf.

* Fix SC2034: variable appears unused. Verify use (or export if used externally).

* Fix SC2035: Use ./*glob* or -- *glob* so names with dashes won't become options.

* Fix SC2071: > is for string comparisons. Use -gt instead.

* Fix SC2154: variable is referenced but not assigned

* Fix SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

* Fix SC2188: This redirection doesn't have a command. Move to its command (or use 'true' as no-op).

* Fix SC2236: Use -n instead of ! -z.

* Fix SC2242: Can only exit with status 0-255. Other data should be written to stdout/stderr.

* Fix SC2086: Double quote to prevent globbing and word splitting.

Co-authored-by: Mehrdad <noreply@github.com>
2020-07-24 17:24:19 -05:00
Simon Mo
a078a21437
[Serve] Allow multiple HTTP servers. (#9523) 2020-07-24 12:41:20 -07:00
Stephanie Wang
f2705e2c73
[core] Enable object reconstruction for retryable actor tasks (#9557)
* Test actor plasma reconstruction

* Allow resubmission of actor tasks

* doc

* Test for actor constructor

* Kill PID before removing node

* Kill pid before node
2020-07-23 21:15:12 -07:00
Robert Nishihara
06c3518aa1
Drop support for Python 3.5. (#9622)
* Drop support for Python 3.5.

* Update setup.py
2020-07-23 19:26:06 -07:00
krfricke
ea4797bf38
[RaySGD] revised existing transformer example to work with transformers>=3.0 (#9661)
Co-authored-by: Kai Fricke <kai@anyscale.com>
2020-07-23 10:58:09 -07:00
Clark Zinzow
9f969260e8
[core] Fix Ray service startup when logging redirection is disabled. (#9547) 2020-07-23 11:26:24 -05:00
Maksim Smolin
908c0c630a
[Autoscaler] Command Line Interface improvements (#9322)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-07-22 12:21:44 -07:00
Max Fitton
456e012029
Revert "Dashboard next-version API support in backend (#9345)" (#9639)
This reverts commit fca1fb18f3.
2020-07-22 13:30:12 -05:00
mehrdadn
6346c70792
Fix bug in Bazel version check (#9626)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-21 20:58:46 -07:00
mehrdadn
b14728d999
Shellcheck quoting (#9596)
* Fix SC2006: Use $(...) notation instead of legacy backticked `...`.

* Fix SC2016: Expressions don't expand in single quotes, use double quotes for that.

* Fix SC2046: Quote this to prevent word splitting.

* Fix SC2053: Quote the right-hand side of == in [[ ]] to prevent glob matching.

* Fix SC2068: Double quote array expansions to avoid re-splitting elements.

* Fix SC2086: Double quote to prevent globbing and word splitting.

* Fix SC2102: Ranges can only match single chars (mentioned due to duplicates).

* Fix SC2140: Word is of the form "A"B"C" (B indicated). Did you mean "ABC" or "A\"B\"C"?

* Fix SC2145: Argument mixes string and array. Use * or separate argument.

* Fix SC2209: warning: Use var=$(command) to assign output (or quote to assign string).

Co-authored-by: Mehrdad <noreply@github.com>
2020-07-21 21:56:41 -05:00
ZhuSenlin
382b314241
[GCS] fix the fault tolerance about gcs node manager (#9380) 2020-07-22 10:55:51 +08:00
Philipp Moritz
a5f4659d9f
Support ray task type checking (#9574) 2020-07-21 19:05:42 -07:00
Simon Mo
d8fd74d528
[Serve] Document Metric Infrastructure (#9389) 2020-07-21 14:52:18 -07:00
mehrdadn
a2f31195c6
Fix Lint in setup.py (#9618)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-21 14:24:24 -07:00
mehrdadn
c5cde65bc6
Add bazel to the PATH in setup.py (#9590)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-21 13:35:29 -07:00
Arne Sachtler
4a36f72ce1
Use UTF-8 for encoding of python code for collision hashing (#9586)
Co-authored-by: Arne Sachtler <arne.sachtler@dlr.de>
Co-authored-by: simon-mo <simon.mo@hey.com>
2020-07-21 13:16:54 -07:00
Lingxuan Zuo
9c4cf0f961
fix tag key typo (#9606) 2020-07-21 19:50:54 +08:00
Clark Zinzow
9b1772253f
Ensure unique log file names across same-node raylets. (#9561) 2020-07-20 16:03:11 -05:00
Ian Rodney
b8fc259796
[serve] Rename to Controller (#9566) 2020-07-20 12:50:29 -07:00
mehrdadn
3d0a3c47a8
Fix Windows CI (#9588)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-20 12:47:27 -07:00
mehrdadn
02cbd4fb7e
Skip uneeded steps on CI (#9582)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-20 12:41:57 -07:00
Max Fitton
fca1fb18f3
Dashboard next-version API support in backend (#9345) 2020-07-20 12:54:11 -05:00
Max Fitton
051973ad23
Add dashboard dependencies to default ray installation (#9447) 2020-07-20 12:53:08 -05:00
Simon Mo
d0398bf7e1
[Serve] Serialize Query object directly (#9490) 2020-07-20 10:10:07 -07:00
mehrdadn
bc842a7888
Fix setup.py bug (#9581)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-20 15:04:35 +02:00
mehrdadn
2554a1a997
Bazel fixes (#9519) 2020-07-19 12:53:08 -07:00
Lingxuan Zuo
ce3f542739
[Metric] new cython interface for python worker metric (#9469) 2020-07-19 10:43:21 +08:00
krfricke
ad0219b80d
[tune] fix pbt checkpoint_freq (#9517)
* Only delete old checkpoint if it is not the same as the new one

* Return early if old checkpoint value coincides with new checkpoint value

Co-authored-by: Kai Fricke <kai@anyscale.com>
2020-07-18 00:58:16 -07:00
jerrylee.io
4c18463fb7
[cli] ray memory: added redis_password (#9492) 2020-07-18 00:08:41 -07:00
Richard Liaw
ed476be4ad
quickfix (#9552) 2020-07-17 20:54:03 -07:00
Tom
cf719dd470
[Tune] Copy default_columns in new ProgressReporter instances (#9537) 2020-07-17 15:44:38 -07:00
Siyuan (Ryans) Zhuang
cba9545266
Add plasma store benchmark for small objects (#9549) 2020-07-17 14:20:59 -07:00
krfricke
87630cf024
[tune] Unflattened lookup for ProgressReporter (#9525)
Co-authored-by: Kai Fricke <kai@anyscale.com>
2020-07-17 13:52:54 -07:00
fangfengbin
5881417ec4
Fix bug that test_multi_node.py::test_multi_driver_logging hangs when GCS actor management is turned on (#9539)
Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-07-17 11:53:29 -07:00
Simon Mo
e78945db96
[Serve] Add internal instruction for running benchmarks (#9531) 2020-07-17 11:17:37 -07:00
Gabriele Oliaro
026c009086
Pipelining task submission to workers (#9363)
* first step of pipelining

* pipelining tests & default configs
- added pipelining unit tests in direct_task_transport_test.cc
- added an entry in ray_config_def.h, ray_config.pxi, and ray_config.pxd to configure the parameter controlling the maximum number of tasks that can be in fligh to each worker
- consolidated worker_to_lease_client_ and worker_to_lease_client_ hash maps in direct_task_transport.h into a single one called worker_to_lease_entry_

* post-review revisions

* linting, following naming/style convention

* linting
2020-07-17 10:45:13 -07:00
Stephanie Wang
b351d13940
[core] Add flag to enable object reconstruction during ray start (#9488)
* Add flag

* doc

* Fix tests
2020-07-17 10:13:14 -07:00
Simon Mo
22a5c3d3ee
Hotfix Lint for Serve (#9535) 2020-07-17 00:40:33 -07:00
mehrdadn
aa33aba1fa
Fix pip and Bazel interaction messing up CI (#9506)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-16 16:28:37 -07:00
Simon Mo
60a838f0f0
[Serve] Use pickle instead of clouldpickle (#9479) 2020-07-16 15:44:10 -07:00
Simon Mo
761b584814
[Serve] Improve buffering for simple cases (#9485) 2020-07-16 15:34:08 -07:00
Stephanie Wang
baf4be245d
Fix flaky test_actor_failures::test_actor_restart (#9509)
* Fix flaky test

* os exit
2020-07-16 10:48:33 -07:00
Ameer Haj Ali
1e46d4e29f
[Autoscaler] Making bootstrap config part of the node provider interface (#9443)
* supporting custom bootstrap config for external node providers

* bootstrap config

* renamed config to cluster_config

* lint

* remove 2 args from importer

* complete move of bootstrap to node_provider

* renamed provider_cls

* move imports outside functions

* lint

* Update python/ray/autoscaler/node_provider.py

Co-authored-by: Eric Liang <ekhliang@gmail.com>

* final fixes

* keeping lines to reduce diff

* lint

* lamba config

* filling in -> adding for lint

Co-authored-by: Ameer Haj Ali <ameerhajali@Ameers-MacBook-Pro.local>
Co-authored-by: Eric Liang <ekhliang@gmail.com>
2020-07-16 09:54:20 -07:00
SangBin Cho
41ad5de1c4
Fix broken test_raylet_info_endpoint (#9511) 2020-07-16 11:51:06 -05:00
mehrdadn
ac39e23145
Get rid of build shell scripts and move them to Python (#6082) 2020-07-16 11:26:47 -05:00
SangBin Cho
2f674728a6
[GCS Actor Management] Gcs actor management broken detached actor (#9473) 2020-07-16 15:41:18 +08:00