Commit graph

37 commits

Author SHA1 Message Date
Ian Rodney
c5209e2dab
[Docker] default to /home/ray (#13738) 2021-01-27 13:46:07 -08:00
Ian Rodney
b4bcb9b60a
[Docker] Use Cuda 11 (#13691) 2021-01-27 13:45:30 -08:00
Ian Rodney
ca6c2b2442
[docker] Use cuDNN7, not 8 (#12375) 2020-11-25 12:06:53 -08:00
Gekho457
2b293832e7
[Docker][minor] Update path in base-deps Dockerfile (#12391)
* Update path in base-deps Dockerfile

* development Dockerfile
2020-11-25 10:30:10 -08:00
Lee moon soo
9920933e31
[docker] Support non-root container (#11407) 2020-11-12 08:41:50 -08:00
Ian Rodney
f37e967ada
[Docs] READmes for DockerHub (#11350) 2020-10-16 15:46:13 -07:00
Ian Rodney
6426fb3fff
[CI] Fix-Up Docker Build (Use Python) (#11139) 2020-10-12 14:22:51 -07:00
Lee moon soo
65065cc19d
[docker] Make python version configurable on docker image (#10783) 2020-09-15 16:07:29 -07:00
Ian Rodney
dee2ab55eb
[docker] Use sh syntax and pull ray-deps (#10517) 2020-09-03 09:30:03 -07:00
Ian Rodney
4324dd5929
[docker] Refactor "autoscaler" image into "-autoscaler" tag and "ray-ml" image. (#10351) 2020-09-02 13:03:35 -07:00
Ian Rodney
2526c06b5e
[WIP] [docker] Cleanup Docker Base-Deps (#9988)
* cleanup-base deps

* only build base-deps a bit

* remove parens

* formatting

* add ray-deps

* gpu enabled

* always include wheel

* fix script

* log new variables

* run tests for docker

* try to include env variables

* source files

* remove bash when sourcing

* add new lines

* use wget

* dual build autoscaler

* switch to gnupg

* add gcc cmake

* remove blist

* clarify build-docker-images
2020-08-26 19:36:11 -07:00
Simon Mo
c218f2eff6
[docker] Build docker in Travis PR & Fix image build failing (#9787)
Co-authored-by: Ian Rodney <ian.rodney@gmail.com>
2020-08-03 16:37:15 -07:00
Ian Rodney
ebcfef012f
[docker] Uses Latest Conda & Py 3.7 (#9732) 2020-07-27 16:18:12 -07:00
Ian Rodney
78c34ae35e
Include open-ssh-client for transparency (#9693) 2020-07-27 15:31:35 -07:00
Ian Rodney
bad7e4c821
[docker] run Ubuntu 20.04 as base image (#9556) 2020-07-20 11:16:57 -07:00
Ian Rodney
dcf989292e
[docker] Update base image (#9281) 2020-07-05 17:31:59 -07:00
Raman Narasimhan
21d5b49c56
[Docker] Slim down autoscaler image (#8683)
* Modifed dockerfiles for base-deps and autoscaler

* Reverted base-deps to use xenial instead of bionic, reverted addition of gnupg2

* Added --no-cache-dir to pip install

Co-authored-by: jenkins-x-bot <jenkins-x@googlegroups.com>
2020-06-01 13:56:08 -07:00
Simon Mo
59867dad75
Move Jenkins test to Github action (#7342) 2020-04-09 10:27:19 -07:00
Richard Liaw
bd8aceb896 [ci] Change Jenkins to py3 (#5022)
* conda3

* integration

* add nevergrad, remotedata

* pytest 0.3.1

* otherdockers

* setup

* tune
2019-06-24 21:50:37 -07:00
Philipp Moritz
b0f6ddf6d1 Remove CMake files (#4493) 2019-04-02 22:17:33 -07:00
Philipp Moritz
b3bf608608 Update arrow to reduce plasma IPCs. (#3497) 2018-12-14 23:49:37 -05:00
Robert Nishihara
e467f546b5 Upgrade version of anaconda. (#2730) 2018-08-23 19:14:39 -07:00
Robert Nishihara
4379e9cea0 Pin cython version in docker base dependencies file. (#1898) 2018-04-13 20:33:20 -07:00
James Lamb
6dbf4f6318 Remove vim from base-deps container and reduce number of build layers (#1667) 2018-03-07 10:16:08 -08:00
Simon Mo
a24cc28773 [DataFrame] Add Parquet Support in Build Process (#1531)
* Add shell script for building parquet

* Use parquet ci script; remove anaconda

* Remove gcc flag, use default

* add boost_root

* Fix $TP_DIR reference issue

* fix the PR

* check out specific parquet-cpp commit
2018-02-16 07:18:42 -08:00
Robert Nishihara
ab5d4a6010 Bring cloudpickle inside the repository. (#1445)
* Bring cloudpickle version 0.5.2 inside the repo.

* Use internal copy of cloudpickle everywhere.

* Fix linting.

* Import ordering.

* Change __init__.py.

* Set pickler in serialization context.

* Don't check ray location.
2018-01-25 11:36:37 -08:00
Robert Nishihara
2ae5a8484f Upgrade cloudpickle to 0.5.2. (#1244) 2017-11-22 20:23:04 -08:00
Robert Nishihara
3a37d1cf7d Pin cloudpickle to 0.4.1. (#1200) 2017-11-08 21:14:09 -08:00
Robert Nishihara
1837824881 Don't install boost when building on Linux. (#1095) 2017-10-09 23:39:03 -07:00
Mitar
0e6e38115f Be more precise which boost dependencies are needed. (#823) 2017-08-07 20:02:48 -07:00
Philipp Moritz
c3b39b4d86 Pull Plasma from Apache Arrow and remove Plasma store from Ray. (#692)
* Rebase Ray on top of Plasma in Apache Arrow

* add thirdparty building scripts

* use rebased arrow

* fix

* fix build

* fix python visibility

* comment out C tests for now

* fix multithreading

* fix

* reduce logging

* fix plasma manager multithreading

* make sure old and new object IDs can coexist peacefully

* more rebasing

* update

* fixes

* fix

* install pyarrow

* install cython

* fix

* install newer cmake

* fix

* rebase on top of latest arrow

* getting runtest.py run locally (needed to comment out a test for that to work)

* work on plasma tests

* more fixes

* fix local scheduler tests

* fix global scheduler test

* more fixes

* fix python 3 bytes vs string

* fix manager tests valgrind

* fix documentation building

* fix linting

* fix c++ linting

* fix linting

* add tests back in

* Install without sudo.

* Set PKG_CONFIG_PATH in build.sh so that Ray can find plasma.

* Install pkg-config

* Link -lpthread, note that find_package(Threads) doesn't seem to work reliably.

* Comment in testGPUIDs in runtest.py.

* Set PKG_CONFIG_PATH when building pyarrow.

* Pull apache/arrow and not pcmoritz/arrow.

* Fix installation in docker image.

* adapt to changes of the plasma api

* Fix installation of pyarrow module.

* Fix linting.

* Use correct python executable to build pyarrow.
2017-07-31 21:04:15 -07:00
Robert Nishihara
53dffe0bf2 Use flatbuffers for some messages from Redis. (#341)
* Compile the Ray redis module with C++.

* Redo parsing of object table notifications with flatbuffers.

* Update redis module python tests.

* Redo parsing of task table notifications with flatbuffers.

* Fix linting.

* Redo parsing of db client notifications with flatbuffers.

* Redo publishing of local scheduler heartbeats with flatbuffers.

* Fix linting.

* Remove usage of fixed-width formatting of scheduling state in channel name.

* Reply with flatbuffer object to task table queries, also simplify redis string to flatbuffer string conversion.

* Fix linting and tests.

* fix

* cleanup

* simplify logic in ReplyWithTask
2017-03-10 18:35:25 -08:00
Johann Schleier-Smith
ad4b03bf7f Docker Updates (#308)
* new path for python build

* add flag

* build tar using git archive

* no exit from start_ray.sh

* update Docker instructions

* update build docker script

* add git revision

* fix typo

* bug fixes and clarifications

* mend

* add objectmanager ports to docker instructions

* rewording

* Small updates to documentation.
2017-02-28 18:57:51 -08:00
Johann Schleier-Smith
4f6100b67f fix docker build bug (#207) 2017-01-18 23:23:34 -08:00
Philipp Moritz
a708e36225 Switch build system to use CMake completely. (#200)
* switch to CMake completely

...

* cleanup

* Run C tests, update installation instructions.
2017-01-17 16:56:40 -08:00
Robert Nishihara
973716d310 Use cloudpickle 0.2.2. (#189) 2017-01-08 17:30:06 -08:00
Johann Schleier-Smith
8bb87a4f6b updated Docker files (#171)
* updated Docker files

* single Docker RUN for apt-get installs and cleanup

* stylistic cleanup
2016-12-31 17:21:33 -08:00