Commit graph

142 commits

Author SHA1 Message Date
Yi Cheng
d6b0b9a209
Revert "Revert "[grpc] Upgrade grpc to 1.45.2"" (#24201)
* Revert "Revert "[grpc] Upgrade grpc to 1.45.2 (#24064)" (#24145)"

This reverts commit f1a1f97992.
2022-04-26 10:49:54 -07:00
Yi Cheng
f1a1f97992
Revert "[grpc] Upgrade grpc to 1.45.2 (#24064)" (#24145)
This reverts commit 3c0a3f4cc1.
2022-04-23 23:47:11 -07:00
Yi Cheng
3c0a3f4cc1
[grpc] Upgrade grpc to 1.45.2 (#24064)
Upgrade grpc to the newest version to use grpc internal implementation of retry.
2022-04-22 19:15:15 -07:00
mwtian
3ccc2aa17a
Revert "[Core] Update grpc to 1.44.0 (#22384)" (#22958)
This reverts commit 5ebc32d7c2.
2022-03-09 11:40:35 -08:00
Akash Patel
5ebc32d7c2
[Core] Update grpc to 1.44.0 (#22384)
Updates grpc to 1.44.0 to remove local patch needed for grpc to build.

EDIT: there have been changes to how python is found (mostly removing python2 support) and as such the local python-patch we have for grpc needs to be modified.

This time contributing it to upstream (grpc/grpc#28895) so that it'll get added in a newer version!

For anyone that comes across this:
Here is the error itself for why we need the grpc-python.patch file: https://buildkite.com/ray-project/ray-builders-pr/builds/24659#d293616f-225d-41f9-8de2-03780f12b13f/2386-2416
2022-03-07 04:53:48 -08:00
Akash Patel
ae6068277b
update grpc to 1.43 (#21866)
add patch for newer setuptools, can be removed once grpc 1.44 is release

Why are these changes needed?
With grpc updated to 1.43, one of the patches is not needed.

Patch needed when building locally for newer setuptools version. See grpc/grpc#28392 for more details.
Also needed as a prereq to #21221
2022-02-15 00:20:56 -08:00
mwtian
dd3a4aa63e
[Core] upgrade grpc and boringssl (#20919)
Upgrade these dependencies to their respective latest stable versions. The latest version of GRPC C++ API has marked the callback API stable.
2021-12-07 14:35:46 -08:00
Akash Patel
7e10f6a876
add missing <limits> header for prometheus_cpp (#19108) 2021-10-19 13:33:31 -07:00
gjoliver
635010d460
Update build rules and patches for darwin_arm64 platform. (#19037)
* Update build rules and patches for darwin_arm64 platform.

Changes include:

Update nelhage/rules_boost package from current version (08/5/2020) to 5/27/2021 version.
Remove rules_boost-undefine-boost_fallthrough.patch, since BOOST_FALLTHROUGH seems to be defined now.
Minor changes to rules_boost-windows-linkopts.patch to use default condition to add -lpthread flag for all platforms.
Add darwin_arm64 config to BUILD files for lib civetweb pulled in via prometheu dependency.

* upgrade boost to 1.74.0 from 1.71.0 to match the udpated build file for windows.

* Fix ray_cpp_pkg

* Use boost/bind/bind.hpp

boost/bind.hpp and global namespace placeholders are deprecated.

* lint

* Use absl::bind_front when possible. Otherwise, NOLINT

* lint

* lint

* lint

* lint

* more lint

* final lint

* trigger build
2021-10-09 18:48:35 -07:00
Clark Zinzow
5ca28b1cc8
[Core] Update Bazel (to 3.4.1), gRPC, boringssl, and absl as a precursor to gRPC streaming PR. (#17903)
* Update Bazel (to 3.4.1), gRPC, boringssl, absl.

* Always reinstall Bazel if needing to upgrade to a new Bazel version.

* Add patch for properly detecting Windows Python headers when building gRPC.

* Add minimum Bazel version check.

* Update docs with new Bazel version.
2021-08-21 11:33:11 -07:00
qicosmos
298d2afc35
[Ray Log] remove glog dependency (#16077) 2021-07-12 17:06:52 +08:00
Siyuan (Ryans) Zhuang
46cf433f0e
[Core] Remove Arrow dependencies (#13157)
* remove arrow ubsan

* remove arrow build depend

* remove arrow buffer
2021-01-04 11:19:09 -08:00
Barak Michener
de6df51bd2
[redis, docs]: Bump redis and docs/Pillow dependencies (#11371) 2020-11-11 18:15:27 -08:00
Lingxuan Zuo
149ec5f6bf
[Log] dump stacktrace from glog lib (#11360)
* dump stacktrace from glog lib

* fix windows compile

* add comments for getcallstack
2020-10-14 10:52:12 -07:00
Lingxuan Zuo
27e1f513e3
[Log] make glog flush and RAY_LOG thread-safe (#11002)
* make glog flush and RAY_LOG thread-safe

* dump error log to console

* mapping all levels to destination

* hack glog for exporting message to stdout if no base name given

* patch lint

* use stdout logger by default

* add raylet std/err pytest checker

* add worker logs file check

* fix asan check

* loop in glog enums

* fix python lint

* lint for autoindent

* fix indent lint

* make raylet.err is not empty
2020-09-28 22:15:15 +08:00
SangBin Cho
826f14c824
[Stats] Fix harvestor threads + Fix flaky stats shutdown. (#9745) 2020-07-29 18:57:59 -05:00
Siyuan (Ryans) Zhuang
54a0d8b69e
[Core] Try remove all windows compat shims (#9671)
* try remove compat for arrow

* remove unistd.h

* remove socket compat

* delete arrow windows patch
2020-07-25 12:00:36 -07:00
SangBin Cho
d49b19c24c
[Stats] Improve Stats::Init & Add it to GCS server (#9563) 2020-07-25 10:42:08 -07:00
mehrdadn
dd2cc6eb48
Update hiredis and remove Windows patches (#9289)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-09 18:45:44 -07:00
Lingxuan Zuo
0562236533
[Stats] Make aggeration window and report interval configurable. (#9235) 2020-07-09 12:44:38 +08:00
mehrdadn
29acf272b7
Build with Visual C++ (#9190)
Co-authored-by: Mehrdad <noreply@github.com>
Co-authored-by: Simon Mo <xmo@berkeley.edu>
2020-07-02 09:34:24 -07:00
mehrdadn
79c4c67ed9
Split hiredis from Redis and use it directly (#9176)
* Split hiredis from Redis and use it directly

Co-authored-by: Mehrdad <noreply@github.com>
2020-06-29 18:16:32 -07:00
mehrdadn
11bda7272b
[windows] Work around msgpack issue on PowerPC64LE (#9140)
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-25 20:36:22 -07:00
Amit Sadaphule
f0b7de7cfe
Fix '//:redis_client' build on RHEL 7.6 ppc64le (#9035)
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-19 16:28:37 -07:00
mehrdadn
07637e5b5b
Upgrade Bazel and add required patches (#8847) 2020-06-12 14:59:22 -05:00
Siyuan (Ryans) Zhuang
ea05ebe89e
Ship plasma store with Ray (#7901) 2020-06-03 17:44:34 -07:00
mehrdadn
ebf060d484
Make more tests run on Windows (#8446)
* Remove worker Wait() call due to SIGCHLD being ignored

* Port _pid_alive to Windows

* Show PID as well as TID in glog

* Update TensorFlow version for Python 3.8 on Windows

* Handle missing Pillow on Windows

* Work around dm-tree PermissionError on Windows

* Fix some lint errors on Windows with Python 3.8

* Simplify torch requirements

* Quiet git clean

* Handle finalizer issues

* Exit with the signal number

* Get rid of wget

* Fix some Windows compatibility issues with tests

Co-authored-by: Mehrdad <noreply@github.com>
2020-05-20 12:25:04 -07:00
mehrdadn
4bdef78e2e
Various CI fixes and cleanup (#8289) 2020-05-05 10:47:49 -07:00
mehrdadn
f86e623095
Fix & improve GitHub Actions CI builds (#7784) 2020-03-30 16:29:54 -07:00
mehrdadn
b4030cdbbe
File HANDLE/descriptor translation layer for Windows (#7657)
* Use TCP sockets on Windows with custom HANDLE <-> FD translation layer

* Get Plasma working on Windows

Co-authored-by: Mehrdad <noreply@github.com>
2020-03-23 21:08:25 -07:00
mehrdadn
fc76586518
Redis on Windows (#7509)
* Switch hiredis on Windows to that of the Windows port of Redis

* Use boost::asio::ip::tcp::socket::native_handle_type

* Use normal hiredis instead of Windows-specific one

* Finish up using normal hiredis

Co-authored-by: Mehrdad <noreply@github.com>
2020-03-09 18:49:54 -07:00
mehrdadn
5fb5be0ba5
Some bug fixes for Windows (#7374)
* Fix MAP_SHARED check in sys/mman.h

* Fix missing :platform_shims dependency for ray_util

* dlmalloc patch for Arrow
2020-02-28 10:22:32 -08:00
mehrdadn
57b33f1bed
Upgrade Boost (#6899) 2020-02-25 14:33:12 -08:00
mehrdadn
83c4e947c7
Make Cython rules more consistent for Bazel (#6840) 2020-02-10 10:45:54 -08:00
mehrdadn
ad4ac9aa70
Add clang-iwyu (#7081)
* Add iwyu

Co-authored-by: GitHub Web Flow <noreply@github.com>
2020-02-07 16:19:46 -08:00
mehrdadn
bde575b8dd Revert "Use Boost.Process instead of pid_t (#6510)" (#6909)
This reverts commit fb8e3615d5.
2020-01-26 10:26:44 -06:00
mehrdadn
fb8e3615d5 Use Boost.Process instead of pid_t (#6510)
* Use Boost.Process instead of pid_t

This will let us handle child processes (mostly) uniformly across platforms.
TODO: There is no SIGTERM on Windows; achieving something equivalent is fairly involved.
2020-01-15 20:05:02 -08:00
mehrdadn
1e817f48df Fix Arrow and Windows linking issues in Bazel (#6546)
* Fix Windows system library dependencies in Boost rules

* Fix missing utf8cpp source and header files in arrow target
2019-12-20 11:53:20 -08:00
mehrdadn
ab5a9f0946 Patch hiredis for Windows (#6446) 2019-12-17 15:32:47 -08:00
mehrdadn
9d6e03aba0 Fix use of select() instead of poll() (#6477)
* Fix Arrow poll() patch

- Negative timeout for poll() was not translated to infinite timeout for select()
- Only use select() on Windows, as other systems limit the range of the file descriptors

* Apply poll() -> select() patch to Redis's ae.c as well
2019-12-17 02:33:37 -08:00
mehrdadn
9948a3779d Simplify patches and make them more robust (#6478)
* Get rid of 'index' lines in patches, which are unnecessary and likely wrong anyway (esp. when there are multiple patches)

* Simplify patches to remove unnecessary context and make them more robust
2019-12-16 19:28:06 -08:00
Chaokun Yang
6272907a57 [Streaming] Streaming data transfer and python integration (#6185) 2019-12-10 20:33:24 +08:00
mehrdadn
17103376c5 Patch arrow for Windows (#6363) 2019-12-05 16:09:21 -08:00
mehrdadn
75cc994e0a Update various build options relating to Windows (#6315)
* Update .bazelrc for Windows compatibility

* Block inclusion of (legacy) WinSock.h to avoid errors

* Suppress warnings for Windows code

* Include boost::asio in includes so that it is passed as -isystem to avoid warnings

* Link with -lpthread only on non-Windows

* Undefine BOOST_FALLTHROUGH, which is unnecessary and causes macro redefinition warnings

* Define RAY_STATIC and ARROW_STATIC to compile for Windows

* Add WinSock import library for Arrow
2019-12-01 15:05:50 -08:00
mehrdadn
b8cfdba752 Bazelify hiredis (#6203) 2019-11-29 15:32:45 -08:00
mehrdadn
cafdaa346f Update glog (#6287) 2019-11-26 14:54:36 -08:00
mehrdadn
5340e5280a Patch prometheus-cpp internally (#6281) 2019-11-26 14:49:24 -08:00
mehrdadn
82d17888e0 Patch grpc for Windows (#6282) 2019-11-26 14:45:21 -08:00
mehrdadn
94d37eee28 Update Boost via our own rule instead of managing our own fork (#6238) 2019-11-22 16:10:47 -08:00
mehrdadn
ba86c75c21 Patch Cython in grpc to use our COPTS (#6223) 2019-11-21 15:32:48 -08:00