Commit graph

138 commits

Author SHA1 Message Date
Devin Petersohn
3a72430e87 Bump Modin version to 0.5 (#4752)
* Bump Modin version

* Update Travis pandas version
2019-05-07 01:12:52 -07:00
Philipp Moritz
95254b3d71 Remove the old web UI (#4301) 2019-03-07 23:15:11 -08:00
William Ma
4be3d0c5d3 Update shipped modin to 0.3.1 (#4058) 2019-02-15 15:49:38 -08:00
Devin Petersohn
a1bcd2a4f5 Update Modin to 0.3.0 (#3936) 2019-02-02 23:06:16 -08:00
Robert Nishihara
2887dac427 Use Redis version 5.0.3. (#3886) 2019-01-29 19:19:05 -08:00
Philipp Moritz
e578a38116 Fix TensorFlow and PyTorch compatibility (#3574)
* remove tensorflow workaround
* update docker
* add boost threads
* add date_time, too
* change link order
* cosmetics
2018-12-22 13:25:48 -08:00
Devin Petersohn
3833ba4e4b Bump modin version to 0.2.5 (#3553) 2018-12-17 14:36:47 -05:00
Yuhong Guo
2a4685a08b Add a script to collect built thirdparty libs to avoid download and building again. (#3521) 2018-12-13 23:56:40 -08:00
Devin Petersohn
4d2010a852 Ship Modin with Ray. (#3109) 2018-11-29 20:05:24 +01:00
Yuhong Guo
93ded5a3d5 Update arrow using Plasma with glog (#2913)
* Update Arrow to Plasma with glog and update the building process

* Remove ParquetExternalProject.cmake

* Fix Mac building error in CI

* Use find_package(BISON) instead of hard code

* Revert BISON binary to hard code.

* Remove build_parquet.sh

* Update setup.sh
2018-09-20 13:37:44 -07:00
Hanwei Jin
fbf214e408 update ray cmake build process (#2853)
* use cmake to build ray project, no need to appply build.sh before cmake, fix some abuse of cmake, improve the build performance

* support boost external project, avoid using the system or build.sh boost

* keep compatible with build.sh, remove boost and arrow build from it.

* bugfix: parquet bison version control, plasma_java lib install problem

* bugfix: cmake, do not compile plasma java client if no need

* bugfix: component failures test timeout machenism has problem for plasma manager failed case

* bugfix: arrow use lib64 in centos, travis check-git-clang-format-output.sh does not support other branches except master

* revert some fix

* set arrow python executable, fix format error in component_failures_test.py

* make clean arrow python build directory

* update cmake code style, back to support cmake minimum version 3.4
2018-09-12 11:19:33 -07:00
Philipp Moritz
869ee8e25d Integrate plasma store list facility (#2752) 2018-09-01 16:53:51 -07:00
Philipp Moritz
b4c47a5861 Upgrade arrow to include more detailed flushing message (#2706) 2018-08-24 11:44:04 -07:00
Yuhong Guo
344a83f327 Fix build failure of Arrow and Parquet when the folder is empty. (#2720) 2018-08-23 09:44:26 -07:00
Hao Chen
dd924a388b silence progress log from 'git clone' and 'pip install' (#2667) 2018-08-15 22:54:35 -07:00
Hao Chen
3c75e71afc reduce noisy log messages from wget (#2656) 2018-08-15 09:10:28 -07:00
Philipp Moritz
f13e3e22f2 Upgrade arrow to include tensorflow op fix (#2607) 2018-08-14 21:47:01 -07:00
Yuhong Guo
4bd98eed45 Support building Java and Python version at the same time. (#2640)
* Support building Java and Python version at the same time.

* Remove duplicated definition.

* Refine the building process of local_scheduler

* Refine

* Add comment for languages

* Modify instruction and add python,jave building to CI.

* change according to comment
2018-08-14 11:33:51 -07:00
Philipp Moritz
5e59cc6a20 Update arrow to include plasma memory footprint reduction (#2545) 2018-08-02 14:37:37 -07:00
Philipp Moritz
d9a019b8e5 Upgrade arrow to include pytorch fix (#2522)
This fixes https://github.com/ray-project/ray/issues/2520
2018-07-30 20:20:18 -07:00
Philipp Moritz
ecc100cb3b Upgrade arrow to include pytorch fix (#2496) 2018-07-28 01:28:44 -07:00
Yuhong Guo
b35ce5dbf1 Update Arrow Package with breaking changes (#2440)
* Merge the breaking change of Arrow Package.

* Fix typo

* Fix lint.

* put forward declarations into header

* fix

* add protocol.h

* fix linting
2018-07-25 14:28:33 -07:00
Philipp Moritz
4c82ac72df Upgrade arrow to include the plasma TensorFlow op (#2412) 2018-07-18 12:33:02 -07:00
Yuhong Guo
e3badb9b09 Fix that parquet and arrow will build every time. (#2405)
* Fix the bug that parquet and arrow will build every time.

* Update build_arrow.sh

* Update build_arrow.sh
2018-07-16 22:56:14 -07:00
Yuhong Guo
bbea73155a Fix parquet missing error and improve arrow commit id changing (#2319)
* Fix parquet missing error and improve arrow commit id changing

* Remove build cache for arrow.

* Update build_parquet.sh

* Update build_ui.sh

* Update build_arrow.sh
2018-07-14 16:08:13 -07:00
Philipp Moritz
4dadc60968 Update arrow to include uninitialized memory fixes (#2371) 2018-07-11 07:52:02 -05:00
Philipp Moritz
e45bec1e51 Fix import error when tensorflow-gpu and tensorflow are both installed (#2364) 2018-07-06 22:29:54 -07:00
Philipp Moritz
fbde8cad74 Update apache arrow to include TensorFlow fix (#2345) 2018-07-06 13:18:56 -07:00
Philipp Moritz
4f1d14e176 improve error message when flex or bison is not installed (#2355) 2018-07-05 23:46:55 -07:00
Philipp Moritz
c0c7146dec fix Travis linux build (#2308) 2018-06-26 20:19:07 -10:00
Yuhong Guo
aa42331844 Fix build failure while using make -j1. Issue 2257 (#2279)
* Fix build failure while using make -j1

* Fix java test failure
2018-06-21 15:18:00 -07:00
Zongheng Yang
8190ff1fd0 Experimental: enable automatic GCS flushing with configurable policy. (#2266)
* build_credis.sh: use an up-to-date credis commit.

* build_credis.sh: leveldb is updated, so update build cmds for it

* WIP: make monitor.py issue flush; switch gcs client to use credis

* Experimental: enable automatic GCS flushing with configurable policy.

* Fix linux compilation error

* Fix leveldb build

* Use optimized build for credis

* Address comments

* Attempt to fix tests
2018-06-20 14:40:57 -07:00
songqing
78a48fa1e0 Fix build error when building Ray for Java later than Python (#2241) 2018-06-12 21:11:30 -07:00
songqing
4dd4698564 unify build dir for Python and Java (#2171)
* unify build dir for Python and Java

* enable executables auto installed when just running 'make'

* fix plasma_store copy error

* fix cmake error about copying executables

* lint fix

* recover python/setup.py

* enable to copy optional file automatically

* a small fix of path

* lint fix

* lint fix

* lint fix

* Add comment.
2018-06-01 16:28:27 -07:00
Yujie Liu
a8d3c057c1 [JavaWorker] Enable java worker support (#2094)
* Enable java worker support
--------------------------
This commit includes a tailored version of the Java worker implementation from Ant Financial.
The changes for build system, python module, src module and arrow are in other commits, this commit consists of the following modules:
 - java/api: Ray API definition
 - java/common: utilities
 - java/hook: binary rewrite of the Java byte-code for remote execution
 - java/runtime-common: common implementation of the runtime in worker
 - java/runtime-dev: a pure-java mock implementation of the runtime for fast development
 - java/runtime-native: a native implementation of the runtime
 - java/test: various tests

Contributors for this work:
 Guyang Song, Peng Cao, Senlin Zhu,Xiaoying Chu, Yiming Yu, Yujie Liu, Zhenyu Guo

* change the format of java help document from markdown to RST

* update the vesion of Arrow for java worker

* adapt the new version of plasma java client from arrow which use byte[] instead of custom type

* add java worker test to ci

* add the example module for better usage guide
2018-05-26 14:38:50 -07:00
Zongheng Yang
fa97acbc89 Integrate credis with Ray & route task table entries into credis. (#1841) 2018-05-24 23:35:25 -07:00
Philipp Moritz
225608ec66 Update arrow to latest master (#2100) 2018-05-24 00:26:13 -07:00
Philipp Moritz
af88fdefcf Incorporate C++ Buffer management and Seal global threadpool fix from arrow (#1950) 2018-04-25 22:53:44 -07:00
Robert Nishihara
24c944e499 Update arrow to efficiently serialize more types of numpy arrays. (#1889) 2018-04-13 09:41:51 -07:00
Robert Nishihara
d0fffec2d0 Update arrow and parquet-cpp. (#1875)
* Update arrow.

* Fix bug.

* Cherry-pick commit for fixing parquet segfault.

* Update arrow and revert auto-releasing buffer commit.

* Remove parquet cherry-pick.
2018-04-12 16:17:12 -07:00
alonamid
202f9683ea check if arrow build dir exists (#1863) 2018-04-10 14:52:51 -07:00
Robert Nishihara
d8adfbd440 Fail build if pyarrow hasn't successfully been built. (#1721) 2018-03-16 12:02:36 -07:00
Philipp Moritz
5ef0892236 Compile boost from source to fix macOS wheels (#1688) 2018-03-08 23:22:23 -08:00
Philipp Moritz
a9acfab3a6 Start chain replicated GCS with Ray (#1538) 2018-03-07 10:18:58 -08:00
Zhenyu Guo
f1e5789c26 restructure how to organize 3rd party libs (#1630)
* restructure how to organize 3rd party libs

* Minor whitespace changes.

* Fix compilation on Linux.

* Pass around Python executable so that the correct version of Python is used.
2018-03-01 14:29:56 -08:00
mehrdadn
7237ec4124 Windows compatibility (#57)
* Add Python and Redis submodules, and remove old third-party modules

* Update VS projects (WARNING: references files that do not exist yet)

* Update code & add shims for APIs except AF_UNIX/{send,recv}msg()

* Minor style changes.
2016-11-22 17:04:24 -08:00
Robert Nishihara
f83a98d71a Changes to make tests pass on Travis. (#3)
* Remove hiredis submodule.

* Squashed 'src/common/thirdparty/hiredis/' content from commit acd1966

git-subtree-dir: src/common/thirdparty/hiredis
git-subtree-split: acd1966bf7f5e1be74b426272635c672def78779

* Make Plasma tests pass.

* Make Photon tests pass.

* Compile and test with Travis.

* Deactive fetch test so that the tests pass.
2016-10-25 22:39:21 -07:00
Robert Nishihara
7c1b2f702f Remove git submodules and C++ files. 2016-10-25 12:51:30 -07:00
mehrdadn
c6ec36865d Update Windows patches for GRPC (#444) 2016-10-06 11:50:40 -07:00
Robert Nishihara
de6ec47f9e Add a recursion depth for serialization to prevent infinite loops. (#440) 2016-09-19 17:17:42 -07:00