Commit graph

14122 commits

Author SHA1 Message Date
Robert Nishihara
d7f313a026 Remove type information from remote decorator. 2016-08-29 22:05:59 -07:00
Philipp Moritz
93e6c9947b update numbuf (#392)
* update numbuf

* Augment serialization tests.
2016-08-25 20:05:48 -07:00
Robert Nishihara
74406e92e4 Merge pull request #4 from pcmoritz/key_tuples
Tuple keys in dicts can be serialized
2016-08-25 16:16:43 -07:00
Philipp Moritz
5c409c3916 make variable names more uniform 2016-08-25 16:04:31 -07:00
Wapaul1
420bcc0477 Remote function returning non-serializable type no longer shuts worker down (#384)
* Moved put_objects in main_loop to inside of try block

* Added test for failed serialization

* Fixed naming

* Minor
2016-08-25 15:26:22 -07:00
William Paul
f468871b5d Tuples as keys in dicts can be serialized 2016-08-24 23:31:09 +00:00
Robert Nishihara
43452b9ab3 Try grpc version 1.0 (#386) 2016-08-23 19:29:18 -07:00
Robert Nishihara
a0490846a2 Merge pull request #7 from pcmoritz/manager
Object store manager
2016-08-22 16:18:11 -07:00
Philipp Moritz
e1627319b3 clean up tests 2016-08-22 15:35:47 -07:00
Philipp Moritz
98b80ea446 Merge pull request #11 from pcmoritz/python
Test Python 3 in Travis.
2016-08-18 23:51:54 -07:00
Robert Nishihara
87447256c4 Test Python 3 in Travis. 2016-08-18 23:26:08 -07:00
Robert Nishihara
b1db890cd8 Export GRPC environment variable in Python instead of in setup-env.sh (#382)
* Export GRPC logging environment variable in Python.

* Don't wipe PYTHONPATH in setup-env.sh
2016-08-18 15:39:19 -07:00
Robert Nishihara
3d6d1e8fe5 Test plasma manager. 2016-08-18 10:04:00 -07:00
Philipp Moritz
23327a18e0 add plasma manager 2016-08-17 13:01:56 -07:00
Philipp Moritz
32fadd54f5 Merge pull request #6 from pcmoritz/make
Remove directories as well with make clean.
2016-08-17 11:21:59 -07:00
Philipp Moritz
36af79427d Merge pull request #9 from pcmoritz/test
Basic Python unit tests.
2016-08-16 17:53:51 -07:00
Robert Nishihara
b6b17f3ac3 Retry if plasma client fails to connect to plasma store. 2016-08-16 17:52:18 -07:00
Robert Nishihara
1b66ac54ba Basic Python unit tests. 2016-08-16 17:17:11 -07:00
Robert Nishihara
6511ea8cc5 Merge pull request #8 from pcmoritz/fix
fix error handling
2016-08-16 16:03:14 -07:00
Philipp Moritz
6db8d1c15a fix error handling 2016-08-16 15:58:14 -07:00
Robert Nishihara
1b22b54cfc Merge pull request #4 from pcmoritz/python
Python library for plasma
2016-08-16 15:43:02 -07:00
Philipp Moritz
535f4403e4 add python client 2016-08-16 15:38:45 -07:00
Robert Nishihara
29f0489b6e Remove directories as well with make clean. 2016-08-16 15:06:29 -07:00
Robert Nishihara
88a14870df Merge pull request #5 from pcmoritz/cleanup
Clean up and add client
2016-08-16 15:04:30 -07:00
Philipp Moritz
7a18347b5c cleanup 2016-08-16 15:00:01 -07:00
Robert Nishihara
314bc9e980 Test blocking behavior of select. (#379) 2016-08-16 14:54:54 -07:00
Robert Nishihara
e06311d415 Automatically add relevant directories to Python paths of workers (#380)
* Make ray.init set python paths of workers.

* Decouple starting cluster from copying user source code

* also add current directory to path

* Add comments about deallocation.

* Add test for new code path.
2016-08-16 14:53:55 -07:00
Philipp Moritz
c01e99284a Merge pull request #1 from pcmoritz/readme
Add readme
2016-08-15 18:48:36 -07:00
Wapaul1
7246013008 Implement select to enable waiting for a specific number of remote objects to be ready. (#369) 2016-08-15 16:51:59 -07:00
Robert Nishihara
b29fc0c481 Use random string for worker c++ logfile. (#378) 2016-08-15 15:55:34 -07:00
Robert Nishihara
e94ed2fc97 Throw fatal error when attempting to send message on full queue. (#377) 2016-08-15 11:51:23 -07:00
Robert Nishihara
87bb7a8f67 [WIP] Large changes to make the tests pass. (#376)
* Revert "Make tests more informative (#372)"

This reverts commit fd353250c8.

* fix bugs, in particular deactivate worker service on driver and remove condition variables

* changes to minimize the changes in this PR

* switch from faulty mutex synchronization to using atomics

* Increase the default size of the message queues, to accommodate exporting large numbers of remote functions. This is a temporary fix, but not a long term solution.

* Reorganize the scheduler export code to queue up exports. This does not solve the underlying problem yet, but sets up a solution.

* Start a separate thread on driver to print error messages by constantly querying the scheduler. This is a temporary solution because the solution based on starting a worker service for the driver which the scheduler can push error messages to is buggy.

* Fix segfault in taskcapsule destructor.

* Move tests for catching errors into a separate test file.

* Revert "roll back grpc (#368)"

This reverts commit c01ef95d04.
2016-08-15 11:02:54 -07:00
Philipp Moritz
504044c033 Merge pull request #2 from pcmoritz/travis
Minimal travis file
2016-08-14 01:51:53 -07:00
Robert Nishihara
e5ecd497f3 minimal travis file 2016-08-14 01:49:58 -07:00
Robert Nishihara
ff13386659 Merge pull request #3 from pcmoritz/initial
Initial plasma commit
2016-08-14 01:47:38 -07:00
Philipp Moritz
97087b0791 initial version of object store 2016-08-14 01:39:44 -07:00
Robert Nishihara
81bfd8c201 add readme 2016-08-13 15:24:01 -07:00
Philipp Moritz
68056dafb1 Initial commit 2016-08-13 15:20:19 -07:00
Robert Nishihara
fd353250c8 Make tests more informative (#372)
* Make tests more informative

* Change grpc status checks to warning instead of fatal
2016-08-11 12:40:55 -07:00
Philipp Moritz
b0ecff69ad Check for GRPC status codes (#366)
* check for GRPC status codes

* more checkss
2016-08-11 11:10:00 -07:00
Wapaul1
362ffa1f3c Changing hard coded ports for objstore and workers to choose unused ports (#365)
* let grpc choose unused worker and object store ports

* Add objstore addresses to scheduler info to bring back test
2016-08-10 19:08:38 -07:00
Robert Nishihara
fbc49410ec properly shutdown worker service thread (#367) 2016-08-10 16:53:24 -07:00
Philipp Moritz
c01ef95d04 roll back grpc (#368) 2016-08-10 16:45:40 -07:00
Johann Schleier-Smith
d8dd9de81b remove verbose flag in tar (#363) 2016-08-09 13:43:25 -07:00
Philipp Moritz
9903fc0335 Improve error message for serialization of custom objects (#359)
* Improve error message for serialization of custom objects

* gitignore numbuf build directory
2016-08-09 13:38:21 -07:00
Robert Nishihara
ac36cd6dcd remove protobuf version from pip install and add instructions for adding Ray to path (#362) 2016-08-09 12:09:06 -07:00
Johann Schleier-Smith
149c13913a ignoring build output and example datasets (#360) 2016-08-09 11:30:33 -07:00
Johann Schleier-Smith
4d5f85e77c Fix Travis failure detection on Mac + testing enhancements (#358)
* properly handle failure in continuous integration

* add stack overflow reference
2016-08-08 18:02:52 -07:00
Philipp Moritz
7b548bddd3 gitignore build directory 2016-08-08 17:58:44 -07:00
Philipp Moritz
b6b2b03950 throw proper error if numpy array that contains object is serialized 2016-08-08 16:29:59 -07:00