Commit graph

46 commits

Author SHA1 Message Date
qicosmos
298d2afc35
[Ray Log] remove glog dependency (#16077) 2021-07-12 17:06:52 +08:00
Siyuan (Ryans) Zhuang
46cf433f0e
[Core] Remove Arrow dependencies (#13157)
* remove arrow ubsan

* remove arrow build depend

* remove arrow buffer
2021-01-04 11:19:09 -08:00
Barak Michener
de6df51bd2
[redis, docs]: Bump redis and docs/Pillow dependencies (#11371) 2020-11-11 18:15:27 -08:00
Lingxuan Zuo
149ec5f6bf
[Log] dump stacktrace from glog lib (#11360)
* dump stacktrace from glog lib

* fix windows compile

* add comments for getcallstack
2020-10-14 10:52:12 -07:00
Lingxuan Zuo
27e1f513e3
[Log] make glog flush and RAY_LOG thread-safe (#11002)
* make glog flush and RAY_LOG thread-safe

* dump error log to console

* mapping all levels to destination

* hack glog for exporting message to stdout if no base name given

* patch lint

* use stdout logger by default

* add raylet std/err pytest checker

* add worker logs file check

* fix asan check

* loop in glog enums

* fix python lint

* lint for autoindent

* fix indent lint

* make raylet.err is not empty
2020-09-28 22:15:15 +08:00
SangBin Cho
826f14c824
[Stats] Fix harvestor threads + Fix flaky stats shutdown. (#9745) 2020-07-29 18:57:59 -05:00
Siyuan (Ryans) Zhuang
54a0d8b69e
[Core] Try remove all windows compat shims (#9671)
* try remove compat for arrow

* remove unistd.h

* remove socket compat

* delete arrow windows patch
2020-07-25 12:00:36 -07:00
SangBin Cho
d49b19c24c
[Stats] Improve Stats::Init & Add it to GCS server (#9563) 2020-07-25 10:42:08 -07:00
mehrdadn
dd2cc6eb48
Update hiredis and remove Windows patches (#9289)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-09 18:45:44 -07:00
Lingxuan Zuo
0562236533
[Stats] Make aggeration window and report interval configurable. (#9235) 2020-07-09 12:44:38 +08:00
mehrdadn
29acf272b7
Build with Visual C++ (#9190)
Co-authored-by: Mehrdad <noreply@github.com>
Co-authored-by: Simon Mo <xmo@berkeley.edu>
2020-07-02 09:34:24 -07:00
mehrdadn
79c4c67ed9
Split hiredis from Redis and use it directly (#9176)
* Split hiredis from Redis and use it directly

Co-authored-by: Mehrdad <noreply@github.com>
2020-06-29 18:16:32 -07:00
mehrdadn
11bda7272b
[windows] Work around msgpack issue on PowerPC64LE (#9140)
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-25 20:36:22 -07:00
Amit Sadaphule
f0b7de7cfe
Fix '//:redis_client' build on RHEL 7.6 ppc64le (#9035)
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-19 16:28:37 -07:00
mehrdadn
07637e5b5b
Upgrade Bazel and add required patches (#8847) 2020-06-12 14:59:22 -05:00
Siyuan (Ryans) Zhuang
ea05ebe89e
Ship plasma store with Ray (#7901) 2020-06-03 17:44:34 -07:00
mehrdadn
ebf060d484
Make more tests run on Windows (#8446)
* Remove worker Wait() call due to SIGCHLD being ignored

* Port _pid_alive to Windows

* Show PID as well as TID in glog

* Update TensorFlow version for Python 3.8 on Windows

* Handle missing Pillow on Windows

* Work around dm-tree PermissionError on Windows

* Fix some lint errors on Windows with Python 3.8

* Simplify torch requirements

* Quiet git clean

* Handle finalizer issues

* Exit with the signal number

* Get rid of wget

* Fix some Windows compatibility issues with tests

Co-authored-by: Mehrdad <noreply@github.com>
2020-05-20 12:25:04 -07:00
mehrdadn
4bdef78e2e
Various CI fixes and cleanup (#8289) 2020-05-05 10:47:49 -07:00
mehrdadn
f86e623095
Fix & improve GitHub Actions CI builds (#7784) 2020-03-30 16:29:54 -07:00
mehrdadn
b4030cdbbe
File HANDLE/descriptor translation layer for Windows (#7657)
* Use TCP sockets on Windows with custom HANDLE <-> FD translation layer

* Get Plasma working on Windows

Co-authored-by: Mehrdad <noreply@github.com>
2020-03-23 21:08:25 -07:00
mehrdadn
fc76586518
Redis on Windows (#7509)
* Switch hiredis on Windows to that of the Windows port of Redis

* Use boost::asio::ip::tcp::socket::native_handle_type

* Use normal hiredis instead of Windows-specific one

* Finish up using normal hiredis

Co-authored-by: Mehrdad <noreply@github.com>
2020-03-09 18:49:54 -07:00
mehrdadn
5fb5be0ba5
Some bug fixes for Windows (#7374)
* Fix MAP_SHARED check in sys/mman.h

* Fix missing :platform_shims dependency for ray_util

* dlmalloc patch for Arrow
2020-02-28 10:22:32 -08:00
mehrdadn
57b33f1bed
Upgrade Boost (#6899) 2020-02-25 14:33:12 -08:00
mehrdadn
83c4e947c7
Make Cython rules more consistent for Bazel (#6840) 2020-02-10 10:45:54 -08:00
mehrdadn
bde575b8dd Revert "Use Boost.Process instead of pid_t (#6510)" (#6909)
This reverts commit fb8e3615d5.
2020-01-26 10:26:44 -06:00
mehrdadn
fb8e3615d5 Use Boost.Process instead of pid_t (#6510)
* Use Boost.Process instead of pid_t

This will let us handle child processes (mostly) uniformly across platforms.
TODO: There is no SIGTERM on Windows; achieving something equivalent is fairly involved.
2020-01-15 20:05:02 -08:00
mehrdadn
1e817f48df Fix Arrow and Windows linking issues in Bazel (#6546)
* Fix Windows system library dependencies in Boost rules

* Fix missing utf8cpp source and header files in arrow target
2019-12-20 11:53:20 -08:00
mehrdadn
ab5a9f0946 Patch hiredis for Windows (#6446) 2019-12-17 15:32:47 -08:00
mehrdadn
9d6e03aba0 Fix use of select() instead of poll() (#6477)
* Fix Arrow poll() patch

- Negative timeout for poll() was not translated to infinite timeout for select()
- Only use select() on Windows, as other systems limit the range of the file descriptors

* Apply poll() -> select() patch to Redis's ae.c as well
2019-12-17 02:33:37 -08:00
mehrdadn
9948a3779d Simplify patches and make them more robust (#6478)
* Get rid of 'index' lines in patches, which are unnecessary and likely wrong anyway (esp. when there are multiple patches)

* Simplify patches to remove unnecessary context and make them more robust
2019-12-16 19:28:06 -08:00
Chaokun Yang
6272907a57 [Streaming] Streaming data transfer and python integration (#6185) 2019-12-10 20:33:24 +08:00
mehrdadn
17103376c5 Patch arrow for Windows (#6363) 2019-12-05 16:09:21 -08:00
mehrdadn
75cc994e0a Update various build options relating to Windows (#6315)
* Update .bazelrc for Windows compatibility

* Block inclusion of (legacy) WinSock.h to avoid errors

* Suppress warnings for Windows code

* Include boost::asio in includes so that it is passed as -isystem to avoid warnings

* Link with -lpthread only on non-Windows

* Undefine BOOST_FALLTHROUGH, which is unnecessary and causes macro redefinition warnings

* Define RAY_STATIC and ARROW_STATIC to compile for Windows

* Add WinSock import library for Arrow
2019-12-01 15:05:50 -08:00
mehrdadn
b8cfdba752 Bazelify hiredis (#6203) 2019-11-29 15:32:45 -08:00
mehrdadn
cafdaa346f Update glog (#6287) 2019-11-26 14:54:36 -08:00
mehrdadn
5340e5280a Patch prometheus-cpp internally (#6281) 2019-11-26 14:49:24 -08:00
mehrdadn
82d17888e0 Patch grpc for Windows (#6282) 2019-11-26 14:45:21 -08:00
mehrdadn
94d37eee28 Update Boost via our own rule instead of managing our own fork (#6238) 2019-11-22 16:10:47 -08:00
mehrdadn
ba86c75c21 Patch Cython in grpc to use our COPTS (#6223) 2019-11-21 15:32:48 -08:00
mehrdadn
7237ec4124 Windows compatibility (#57)
* Add Python and Redis submodules, and remove old third-party modules

* Update VS projects (WARNING: references files that do not exist yet)

* Update code & add shims for APIs except AF_UNIX/{send,recv}msg()

* Minor style changes.
2016-11-22 17:04:24 -08:00
mehrdadn
c6ec36865d Update Windows patches for GRPC (#444) 2016-10-06 11:50:40 -07:00
mehrdadn
c5be9b96de Visual Studio - Project cleanup, warning suppression, and removal of zlib dependency (#272)
* windows_intermodule_singleton warning

* Fix some warnings from Visual Studio

* More Visual Studio project changes

* Replace zlib Visual Studio project with stub; it's not needed

* Un-enable link-time code generation
2016-07-16 13:02:51 -07:00
mehrdadn
69ee25eb51 Fix GRPC project patch, again (#255) 2016-07-10 22:31:12 -07:00
mehrdadn
30023b456a Fix GRPC project patch (#254) 2016-07-10 22:05:50 -07:00
mehrdadn
fc711abda0 Remove parquet for Windows -- we don't need it yet (#240) 2016-07-08 20:06:21 -07:00
mehrdadn
eff6924b7c Update third-party patches (#203) 2016-07-05 12:17:09 -07:00