Commit graph

104 commits

Author SHA1 Message Date
mehrdadn
ab5a9f0946 Patch hiredis for Windows (#6446) 2019-12-17 15:32:47 -08:00
mehrdadn
9d6e03aba0 Fix use of select() instead of poll() (#6477)
* Fix Arrow poll() patch

- Negative timeout for poll() was not translated to infinite timeout for select()
- Only use select() on Windows, as other systems limit the range of the file descriptors

* Apply poll() -> select() patch to Redis's ae.c as well
2019-12-17 02:33:37 -08:00
mehrdadn
9948a3779d Simplify patches and make them more robust (#6478)
* Get rid of 'index' lines in patches, which are unnecessary and likely wrong anyway (esp. when there are multiple patches)

* Simplify patches to remove unnecessary context and make them more robust
2019-12-16 19:28:06 -08:00
Chaokun Yang
6272907a57 [Streaming] Streaming data transfer and python integration (#6185) 2019-12-10 20:33:24 +08:00
mehrdadn
17103376c5 Patch arrow for Windows (#6363) 2019-12-05 16:09:21 -08:00
mehrdadn
75cc994e0a Update various build options relating to Windows (#6315)
* Update .bazelrc for Windows compatibility

* Block inclusion of (legacy) WinSock.h to avoid errors

* Suppress warnings for Windows code

* Include boost::asio in includes so that it is passed as -isystem to avoid warnings

* Link with -lpthread only on non-Windows

* Undefine BOOST_FALLTHROUGH, which is unnecessary and causes macro redefinition warnings

* Define RAY_STATIC and ARROW_STATIC to compile for Windows

* Add WinSock import library for Arrow
2019-12-01 15:05:50 -08:00
mehrdadn
b8cfdba752 Bazelify hiredis (#6203) 2019-11-29 15:32:45 -08:00
mehrdadn
cafdaa346f Update glog (#6287) 2019-11-26 14:54:36 -08:00
mehrdadn
5340e5280a Patch prometheus-cpp internally (#6281) 2019-11-26 14:49:24 -08:00
mehrdadn
82d17888e0 Patch grpc for Windows (#6282) 2019-11-26 14:45:21 -08:00
mehrdadn
94d37eee28 Update Boost via our own rule instead of managing our own fork (#6238) 2019-11-22 16:10:47 -08:00
mehrdadn
ba86c75c21 Patch Cython in grpc to use our COPTS (#6223) 2019-11-21 15:32:48 -08:00
Philipp Moritz
010270b3dc
Cleanup left over shell scripts in build process (#6017) 2019-10-26 15:46:46 -07:00
Devin Petersohn
c33d6662ce Remove Modin from Ray wheels. (#5647)
There are several reasons for this:

* We no longer support python2
* There should be only 1 way of installing Modin
  * Issue management on these wheels
* I have never heard of anyone using this feature
* It is rarely kept up to date
* Modin depends on specific versions of Ray because of past API changes
2019-09-05 23:46:27 -07:00
Simon Mo
1f8ae17f60 Silence some installation process for build from source (#5396) 2019-08-07 16:54:48 -07:00
Philipp Moritz
a6a02fccd0 Do not compile redis twice (#5074) 2019-07-01 15:42:54 -07:00
Devin Petersohn
3a72430e87 Bump Modin version to 0.5 (#4752)
* Bump Modin version

* Update Travis pandas version
2019-05-07 01:12:52 -07:00
Philipp Moritz
95254b3d71 Remove the old web UI (#4301) 2019-03-07 23:15:11 -08:00
William Ma
4be3d0c5d3 Update shipped modin to 0.3.1 (#4058) 2019-02-15 15:49:38 -08:00
Devin Petersohn
a1bcd2a4f5 Update Modin to 0.3.0 (#3936) 2019-02-02 23:06:16 -08:00
Robert Nishihara
2887dac427 Use Redis version 5.0.3. (#3886) 2019-01-29 19:19:05 -08:00
Philipp Moritz
e578a38116 Fix TensorFlow and PyTorch compatibility (#3574)
* remove tensorflow workaround
* update docker
* add boost threads
* add date_time, too
* change link order
* cosmetics
2018-12-22 13:25:48 -08:00
Devin Petersohn
3833ba4e4b Bump modin version to 0.2.5 (#3553) 2018-12-17 14:36:47 -05:00
Yuhong Guo
2a4685a08b Add a script to collect built thirdparty libs to avoid download and building again. (#3521) 2018-12-13 23:56:40 -08:00
Devin Petersohn
4d2010a852 Ship Modin with Ray. (#3109) 2018-11-29 20:05:24 +01:00
Yuhong Guo
93ded5a3d5 Update arrow using Plasma with glog (#2913)
* Update Arrow to Plasma with glog and update the building process

* Remove ParquetExternalProject.cmake

* Fix Mac building error in CI

* Use find_package(BISON) instead of hard code

* Revert BISON binary to hard code.

* Remove build_parquet.sh

* Update setup.sh
2018-09-20 13:37:44 -07:00
Hanwei Jin
fbf214e408 update ray cmake build process (#2853)
* use cmake to build ray project, no need to appply build.sh before cmake, fix some abuse of cmake, improve the build performance

* support boost external project, avoid using the system or build.sh boost

* keep compatible with build.sh, remove boost and arrow build from it.

* bugfix: parquet bison version control, plasma_java lib install problem

* bugfix: cmake, do not compile plasma java client if no need

* bugfix: component failures test timeout machenism has problem for plasma manager failed case

* bugfix: arrow use lib64 in centos, travis check-git-clang-format-output.sh does not support other branches except master

* revert some fix

* set arrow python executable, fix format error in component_failures_test.py

* make clean arrow python build directory

* update cmake code style, back to support cmake minimum version 3.4
2018-09-12 11:19:33 -07:00
Philipp Moritz
869ee8e25d Integrate plasma store list facility (#2752) 2018-09-01 16:53:51 -07:00
Philipp Moritz
b4c47a5861 Upgrade arrow to include more detailed flushing message (#2706) 2018-08-24 11:44:04 -07:00
Yuhong Guo
344a83f327 Fix build failure of Arrow and Parquet when the folder is empty. (#2720) 2018-08-23 09:44:26 -07:00
Hao Chen
dd924a388b silence progress log from 'git clone' and 'pip install' (#2667) 2018-08-15 22:54:35 -07:00
Hao Chen
3c75e71afc reduce noisy log messages from wget (#2656) 2018-08-15 09:10:28 -07:00
Philipp Moritz
f13e3e22f2 Upgrade arrow to include tensorflow op fix (#2607) 2018-08-14 21:47:01 -07:00
Yuhong Guo
4bd98eed45 Support building Java and Python version at the same time. (#2640)
* Support building Java and Python version at the same time.

* Remove duplicated definition.

* Refine the building process of local_scheduler

* Refine

* Add comment for languages

* Modify instruction and add python,jave building to CI.

* change according to comment
2018-08-14 11:33:51 -07:00
Philipp Moritz
5e59cc6a20 Update arrow to include plasma memory footprint reduction (#2545) 2018-08-02 14:37:37 -07:00
Philipp Moritz
d9a019b8e5 Upgrade arrow to include pytorch fix (#2522)
This fixes https://github.com/ray-project/ray/issues/2520
2018-07-30 20:20:18 -07:00
Philipp Moritz
ecc100cb3b Upgrade arrow to include pytorch fix (#2496) 2018-07-28 01:28:44 -07:00
Yuhong Guo
b35ce5dbf1 Update Arrow Package with breaking changes (#2440)
* Merge the breaking change of Arrow Package.

* Fix typo

* Fix lint.

* put forward declarations into header

* fix

* add protocol.h

* fix linting
2018-07-25 14:28:33 -07:00
Philipp Moritz
4c82ac72df Upgrade arrow to include the plasma TensorFlow op (#2412) 2018-07-18 12:33:02 -07:00
Yuhong Guo
e3badb9b09 Fix that parquet and arrow will build every time. (#2405)
* Fix the bug that parquet and arrow will build every time.

* Update build_arrow.sh

* Update build_arrow.sh
2018-07-16 22:56:14 -07:00
Yuhong Guo
bbea73155a Fix parquet missing error and improve arrow commit id changing (#2319)
* Fix parquet missing error and improve arrow commit id changing

* Remove build cache for arrow.

* Update build_parquet.sh

* Update build_ui.sh

* Update build_arrow.sh
2018-07-14 16:08:13 -07:00
Philipp Moritz
4dadc60968 Update arrow to include uninitialized memory fixes (#2371) 2018-07-11 07:52:02 -05:00
Philipp Moritz
e45bec1e51 Fix import error when tensorflow-gpu and tensorflow are both installed (#2364) 2018-07-06 22:29:54 -07:00
Philipp Moritz
fbde8cad74 Update apache arrow to include TensorFlow fix (#2345) 2018-07-06 13:18:56 -07:00
Philipp Moritz
4f1d14e176 improve error message when flex or bison is not installed (#2355) 2018-07-05 23:46:55 -07:00
Philipp Moritz
c0c7146dec fix Travis linux build (#2308) 2018-06-26 20:19:07 -10:00
Yuhong Guo
aa42331844 Fix build failure while using make -j1. Issue 2257 (#2279)
* Fix build failure while using make -j1

* Fix java test failure
2018-06-21 15:18:00 -07:00
Zongheng Yang
8190ff1fd0 Experimental: enable automatic GCS flushing with configurable policy. (#2266)
* build_credis.sh: use an up-to-date credis commit.

* build_credis.sh: leveldb is updated, so update build cmds for it

* WIP: make monitor.py issue flush; switch gcs client to use credis

* Experimental: enable automatic GCS flushing with configurable policy.

* Fix linux compilation error

* Fix leveldb build

* Use optimized build for credis

* Address comments

* Attempt to fix tests
2018-06-20 14:40:57 -07:00
songqing
78a48fa1e0 Fix build error when building Ray for Java later than Python (#2241) 2018-06-12 21:11:30 -07:00
songqing
4dd4698564 unify build dir for Python and Java (#2171)
* unify build dir for Python and Java

* enable executables auto installed when just running 'make'

* fix plasma_store copy error

* fix cmake error about copying executables

* lint fix

* recover python/setup.py

* enable to copy optional file automatically

* a small fix of path

* lint fix

* lint fix

* lint fix

* Add comment.
2018-06-01 16:28:27 -07:00