Commit graph

83 commits

Author SHA1 Message Date
Akash Patel
ae6068277b
update grpc to 1.43 (#21866)
add patch for newer setuptools, can be removed once grpc 1.44 is release

Why are these changes needed?
With grpc updated to 1.43, one of the patches is not needed.

Patch needed when building locally for newer setuptools version. See grpc/grpc#28392 for more details.
Also needed as a prereq to #21221
2022-02-15 00:20:56 -08:00
Lingxuan Zuo
43ea467896
Ray support internal native deps reused (#21641)
To make other system or internal project reuse ray deps bazel function, we need change this local accessing style to global accessing with ray-project namespace.

Co-authored-by: 林濯 <lingxuzn.zlx@antgroup.com>
2022-01-21 13:56:40 +08:00
Akash Patel
cbcd03b779
Upgrade cython to 0.29.26 for py310 (#21244) 2021-12-26 20:26:08 -08:00
Qing Wang
e653d47533
[Java] Shade some widely used dependencies in bazel_jar_jar rule. (#21237)
These dependencies are widely used:
- com.google.common
- com.google.protobuf
- com.google.thirdparty

So that we need to shade them to avoid being conflict with jars introduced by user.

In this PR, we introduce a `bazel_jar_jar` rule for doing these and also shade them in maven pom files.
2021-12-23 16:54:31 +08:00
mwtian
dd3a4aa63e
[Core] upgrade grpc and boringssl (#20919)
Upgrade these dependencies to their respective latest stable versions. The latest version of GRPC C++ API has marked the callback API stable.
2021-12-07 14:35:46 -08:00
Mark
379732a181
Bump abseil-cpp LTS 20211102 for clang-13 build (#20565)
Abseil LTS 20210324.2 will fail the compilation with clang-13. After this version bump, ray can be successfully built with clang-13.
2021-11-19 14:09:40 -08:00
gjoliver
635010d460
Update build rules and patches for darwin_arm64 platform. (#19037)
* Update build rules and patches for darwin_arm64 platform.

Changes include:

Update nelhage/rules_boost package from current version (08/5/2020) to 5/27/2021 version.
Remove rules_boost-undefine-boost_fallthrough.patch, since BOOST_FALLTHROUGH seems to be defined now.
Minor changes to rules_boost-windows-linkopts.patch to use default condition to add -lpthread flag for all platforms.
Add darwin_arm64 config to BUILD files for lib civetweb pulled in via prometheu dependency.

* upgrade boost to 1.74.0 from 1.71.0 to match the udpated build file for windows.

* Fix ray_cpp_pkg

* Use boost/bind/bind.hpp

boost/bind.hpp and global namespace placeholders are deprecated.

* lint

* Use absl::bind_front when possible. Otherwise, NOLINT

* lint

* lint

* lint

* lint

* more lint

* final lint

* trigger build
2021-10-09 18:48:35 -07:00
mwtian
26fd10c9e8
[CI] Add clang-tidy to lint (#18124)
* clang-tidy

* fix

* fix script

* test clang compiler

* fix clang-tidy rules

* Fix windows and other issues.

* Fix

* Improve information when running check-git-clang-tidy-output.sh on different OS
2021-09-09 00:41:53 -07:00
Yi Cheng
7126d01c91
[core] upgrade gtest (#18288)
* up

* up

* format

* up

* flaky fix

* format

* up

* up

* format

* add debug

* up

* up

* up

* up

* up

* format

* fix

* format

* up

* up

* format
2021-09-08 11:15:34 -07:00
Clark Zinzow
5ca28b1cc8
[Core] Update Bazel (to 3.4.1), gRPC, boringssl, and absl as a precursor to gRPC streaming PR. (#17903)
* Update Bazel (to 3.4.1), gRPC, boringssl, absl.

* Always reinstall Bazel if needing to upgrade to a new Bazel version.

* Add patch for properly detecting Windows Python headers when building gRPC.

* Add minimum Bazel version check.

* Update docs with new Bazel version.
2021-08-21 11:33:11 -07:00
qicosmos
298d2afc35
[Ray Log] remove glog dependency (#16077) 2021-07-12 17:06:52 +08:00
Kai Yang
7c21be5450
[Object spilling] Clean up spilled objects on disk when Raylet starts (#16669) 2021-07-05 12:01:25 +08:00
architkulkarni
76d602363b
update URL for boost 1.71.0 in bazel setup (#15991) 2021-06-17 12:34:35 -07:00
mwtian
dce13d3a81
Explicitly set protobuf dependency version to allow building ray with bazel 4.0.0 (#15756)
Java protobuf dependency version is made to be consistent as well.
2021-05-13 10:34:09 -07:00
Jirka Fajfr
153dcd3734
Upgrading Redis to 6.0.10 in order to be functional on Apple silicon (M1) (#14299)
Co-authored-by: Jirka Fajfr <jirka.fajfr@enverus.com>
2021-03-10 15:17:01 -08:00
Siyuan (Ryans) Zhuang
46cf433f0e
[Core] Remove Arrow dependencies (#13157)
* remove arrow ubsan

* remove arrow build depend

* remove arrow buffer
2021-01-04 11:19:09 -08:00
chaokunyang
d1dd3410c8
[Java] Format ray java code (#13056) 2020-12-29 10:36:16 +08:00
Barak Michener
de6df51bd2
[redis, docs]: Bump redis and docs/Pillow dependencies (#11371) 2020-11-11 18:15:27 -08:00
Lingxuan Zuo
149ec5f6bf
[Log] dump stacktrace from glog lib (#11360)
* dump stacktrace from glog lib

* fix windows compile

* add comments for getcallstack
2020-10-14 10:52:12 -07:00
Lingxuan Zuo
27e1f513e3
[Log] make glog flush and RAY_LOG thread-safe (#11002)
* make glog flush and RAY_LOG thread-safe

* dump error log to console

* mapping all levels to destination

* hack glog for exporting message to stdout if no base name given

* patch lint

* use stdout logger by default

* add raylet std/err pytest checker

* add worker logs file check

* fix asan check

* loop in glog enums

* fix python lint

* lint for autoindent

* fix indent lint

* make raylet.err is not empty
2020-09-28 22:15:15 +08:00
Basasuya
5e030db8a5
[EVENT] add log reporter (#10419) 2020-09-16 11:54:05 +08:00
Simon Mo
bedc2c24c8
Export Metrics in OpenCensus Protobuf Format (#10080) 2020-08-18 11:32:42 -07:00
qicosmos
83e4501cf6
Update com_github_nelhage_rules_boost version in ray_deps_setup.bzl (#9975)
support boost.dll
2020-08-10 10:34:48 +08:00
Siyuan (Ryans) Zhuang
54a0d8b69e
[Core] Try remove all windows compat shims (#9671)
* try remove compat for arrow

* remove unistd.h

* remove socket compat

* delete arrow windows patch
2020-07-25 12:00:36 -07:00
SangBin Cho
d49b19c24c
[Stats] Improve Stats::Init & Add it to GCS server (#9563) 2020-07-25 10:42:08 -07:00
mehrdadn
dd2cc6eb48
Update hiredis and remove Windows patches (#9289)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-09 18:45:44 -07:00
mehrdadn
b6c11f3dd7
Bazel move more shell to native rules (#9314)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-09 09:02:35 -07:00
Lingxuan Zuo
0562236533
[Stats] Make aggeration window and report interval configurable. (#9235) 2020-07-09 12:44:38 +08:00
mehrdadn
7135cb2aec
Fix .exe file extensions (#9197)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-02 15:29:34 -05:00
mehrdadn
29acf272b7
Build with Visual C++ (#9190)
Co-authored-by: Mehrdad <noreply@github.com>
Co-authored-by: Simon Mo <xmo@berkeley.edu>
2020-07-02 09:34:24 -07:00
mehrdadn
79c4c67ed9
Split hiredis from Redis and use it directly (#9176)
* Split hiredis from Redis and use it directly

Co-authored-by: Mehrdad <noreply@github.com>
2020-06-29 18:16:32 -07:00
mehrdadn
e060d99c6e
Bazel: strip_prefix=True should strip leading 'v' (#9175)
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-29 11:27:56 -07:00
mehrdadn
07637e5b5b
Upgrade Bazel and add required patches (#8847) 2020-06-12 14:59:22 -05:00
Siyuan (Ryans) Zhuang
ea05ebe89e
Ship plasma store with Ray (#7901) 2020-06-03 17:44:34 -07:00
Simon Mo
a8e4ddbd57
Update Redis to 5.0.9 (#8722) 2020-06-02 14:10:07 -07:00
mehrdadn
ebf060d484
Make more tests run on Windows (#8446)
* Remove worker Wait() call due to SIGCHLD being ignored

* Port _pid_alive to Windows

* Show PID as well as TID in glog

* Update TensorFlow version for Python 3.8 on Windows

* Handle missing Pillow on Windows

* Work around dm-tree PermissionError on Windows

* Fix some lint errors on Windows with Python 3.8

* Simplify torch requirements

* Quiet git clean

* Handle finalizer issues

* Exit with the signal number

* Get rid of wget

* Fix some Windows compatibility issues with tests

Co-authored-by: Mehrdad <noreply@github.com>
2020-05-20 12:25:04 -07:00
mehrdadn
4bdef78e2e
Various CI fixes and cleanup (#8289) 2020-05-05 10:47:49 -07:00
acxz
11c2b500b8
[bazel] Remove deprecated and now non-existent flag (#7886) 2020-04-03 22:36:01 -07:00
mehrdadn
65054a2c7c
Python 3.8 compatibility (#7754) 2020-04-01 10:03:23 -07:00
mehrdadn
f86e623095
Fix & improve GitHub Actions CI builds (#7784) 2020-03-30 16:29:54 -07:00
SongGuyang
c195dc8f88
Basic C++ worker implementation (#6125) 2020-03-27 23:01:08 +08:00
mehrdadn
b4030cdbbe
File HANDLE/descriptor translation layer for Windows (#7657)
* Use TCP sockets on Windows with custom HANDLE <-> FD translation layer

* Get Plasma working on Windows

Co-authored-by: Mehrdad <noreply@github.com>
2020-03-23 21:08:25 -07:00
Stephanie Wang
53549314c5
[core] Option to fallback to LRU on OutOfMemory (#7410)
* Add a test for LRU fallback

* Update error message

* Upgrade arrow to master

* Integrate with arrow

* Revert "Bazel mirrors (#7385)"

This reverts commit 44aded5272.

* Don't LRU evict

* Revert "Revert "Bazel mirrors (#7385)""

This reverts commit b6359fea78d1bd3925452ca88ac71e0c9e5c7dd3.

* Add lru_evict flag

* fix internal config

* Fix

* upgrade arrow

* debug

* Set free period in config for lru_evict, override max retries to fix
test

* Fix test?

* fix test

* Revert "debug"

This reverts commit 98f01c63a267f38218f5047b1866e4c1c8280017.

* fix exception str

* Fix ref count test

* Shorten travis test?
2020-03-14 11:28:43 -07:00
mehrdadn
fc76586518
Redis on Windows (#7509)
* Switch hiredis on Windows to that of the Windows port of Redis

* Use boost::asio::ip::tcp::socket::native_handle_type

* Use normal hiredis instead of Windows-specific one

* Finish up using normal hiredis

Co-authored-by: Mehrdad <noreply@github.com>
2020-03-09 18:49:54 -07:00
Stephanie Wang
95bb0c5357
Upgrade plasma to latest version, use synchronous Seal (#7470)
* Upgrade arrow to master

* fix build

* todo

* lint

* Fix hanging test
2020-03-09 10:30:44 -07:00
mehrdadn
44aded5272
Bazel mirrors (#7385)
* Switch to mirrors.bazel.build where possible

* Switch from .zip to .tar.gz for smaller downloads (it's also the default download on UNIX)

* Use direct GitHub URLs in Bazel files for clarity

* Don't pass patches to local_repository

* Remove github_repository()

* Switch to GitHub actions/checkout@v2 which is faster

* Use faster extraction method for LLVm on Windows

* Move LLVM_VERSION_WINDOWS to the shell script since it's not a CI-specific value

* Change GITHUB_TOKEN to GITHUB

* Don't show timestamps for GitHub Actions

* Factor out some options from GitHub Actions

* Tell Bazel to stay on the same volume in GitHun Actions

* Display progress output when downloading toolchains

Co-authored-by: GitHub Web Flow <noreply@github.com>
2020-03-01 14:04:06 -08:00
mehrdadn
5fb5be0ba5
Some bug fixes for Windows (#7374)
* Fix MAP_SHARED check in sys/mman.h

* Fix missing :platform_shims dependency for ray_util

* dlmalloc patch for Arrow
2020-02-28 10:22:32 -08:00
mehrdadn
0efaa9b310
Use Redis for Windows (#7364) 2020-02-28 10:18:56 -08:00
mehrdadn
57b33f1bed
Upgrade Boost (#6899) 2020-02-25 14:33:12 -08:00
mehrdadn
b97b8c2be1
Fix github_repository(path=...) accidentally requiring other fields that shouldn't be necessary (#7275)
Co-authored-by: GitHub Web Flow <noreply@github.com>
2020-02-24 11:39:08 -08:00