Commit graph

82 commits

Author SHA1 Message Date
Wapaul1
362ffa1f3c Changing hard coded ports for objstore and workers to choose unused ports (#365)
* let grpc choose unused worker and object store ports

* Add objstore addresses to scheduler info to bring back test
2016-08-10 19:08:38 -07:00
Robert Nishihara
fbc49410ec properly shutdown worker service thread (#367) 2016-08-10 16:53:24 -07:00
Robert Nishihara
13df8302e6 enable running example apps in cluster mode (#357) 2016-08-08 16:01:13 -07:00
Robert Nishihara
a1e4268d37 Catch errors in importing reusable variables and remote functions (#354)
* catch errors in importing reusable variables and remote functions

* updates
2016-08-07 13:53:33 -07:00
Philipp Moritz
8bf877ac1e Serialize and Deserialize unicode (#349) 2016-08-04 21:06:31 -07:00
Robert Nishihara
ac363bf451 Let worker get worker address and object store address from scheduler (#350) 2016-08-04 17:47:08 -07:00
Johann Schleier-Smith
3ee0fd8f34 Update cluster guide (#347)
* clarify cluster setup instructions

* update multinode documentation, update cluster script, fix minor bug in worker.py

* clarify cluster documentation and fix update_user_code
2016-08-04 09:14:20 -07:00
Robert Nishihara
de200ff912 better error messages when composing remote functions (#339)
Better error messages when composing remote functions
2016-08-03 16:47:13 -07:00
Robert Nishihara
07baf44f26 only display failed task information for new tasks, remove SHELL_MODE (#344) 2016-08-02 17:59:42 -07:00
Robert Nishihara
c27e6c076c Make sure no Python modules mutually import each other. (#334) 2016-08-01 17:55:38 -07:00
Wapaul1
96a70e1316 Changed equality to None in worker.py to use is keyword (#335) 2016-08-01 16:56:54 -07:00
Wapaul1
97b923a750 Changed how ray treats deserialization of custom classes (#333) 2016-08-01 15:38:05 -07:00
Robert Nishihara
98a508d6ca Terminology change Object Reference -> Object ID (#330) 2016-07-31 19:58:03 -07:00
Robert Nishihara
2040372084 unify starting local cluster with attaching to existing cluster (#327) 2016-07-31 19:26:35 -07:00
Robert Nishihara
0e5b858324 change remote function invocation from func() to func.remote() (#328) 2016-07-31 15:25:19 -07:00
Robert Nishihara
352e5e1dd2 use Ray specific logger so logging does not interfere with other python modules that use the logging module (#321) 2016-07-29 12:40:45 -07:00
mehrdadn
4c323d4ecf libraylib -> lib (#311) 2016-07-27 23:23:13 -07:00
Philipp Moritz
dd301e4baa add version string (#304) 2016-07-27 11:29:22 -07:00
Robert Nishihara
975bfe89ad check that we are connected before calling ray commands (#303) 2016-07-27 11:20:37 -07:00
Philipp Moritz
b5215f1e6a make it possible to use directory as user source directory that doesn't contain worker.py (#297) 2016-07-26 18:39:06 -07:00
Robert Nishihara
aa2f618ab7 add directory containing script to python path of workers (#296) 2016-07-26 16:18:39 -07:00
Robert Nishihara
3bae6f136b export remote functions and reusable variables that were defined before connect was called (#292) 2016-07-26 11:40:09 -07:00
Robert Nishihara
8e9f98c5ff Properly import remote functions and reusable variables on workers that register late (#290) 2016-07-25 16:17:17 -07:00
Philipp Moritz
4a0f35b042 new arrow serialization code (serialize python objects recursively) (#284) 2016-07-25 13:41:47 -07:00
mehrdadn
5ff00e0e81 Fix pickling (#289) 2016-07-24 12:53:55 -07:00
Robert Nishihara
03f1830cd0 implement key value store for sharing reusable variables 2016-07-22 18:43:33 -07:00
Robert Nishihara
3bf46ce5ac make remote functions pickleable (#281) 2016-07-19 16:05:45 -07:00
mehrdadn
f2c43bec87 Function serialization (#261) 2016-07-17 22:05:07 -07:00
mehrdadn
06ceecf6f5 Kill child processes when parent dies, and die on fatal errors (#265) 2016-07-14 16:10:26 -07:00
Robert Nishihara
a5d8138f88 mock modules for readthedocs (#264) 2016-07-14 13:57:03 -07:00
Robert Nishihara
80526f7777 add documentation and refactor cluster.py (#238) 2016-07-12 23:54:18 -07:00
Wapaul1
292abaa41c Moved Imagenet loading library to example applications; Changed code to return filenames as well as arrays (#257) 2016-07-11 18:06:58 -07:00
Robert Nishihara
8952ff8cf9 allow cluster script to update worker code on nodes (#243) 2016-07-11 17:58:16 -07:00
Robert Nishihara
191909dd93 adding pylint (#233) 2016-07-08 12:39:11 -07:00
Robert Nishihara
82ab6d68f1 remove recursive option from register_module (#232) 2016-07-08 11:13:03 -07:00
Robert Nishihara
875b20e397 only run cleanup if we've started ray in local mode and actually started the processes (#228) 2016-07-08 00:14:26 -07:00
Robert Nishihara
8e6b7929d6 make services.cleanup happen automatically (#224) 2016-07-07 14:05:25 -07:00
mehrdadn
199b4efd50 int and long should be treated similarly (#220) 2016-07-06 17:31:58 -07:00
Robert Nishihara
e1e09d3a4b catch type checking exceptions 2016-07-06 14:20:57 -07:00
Robert Nishihara
49decce6f6 print python backtrace in error message (#209) 2016-07-06 00:25:02 -07:00
Robert Nishihara
8297522580 don't destruct worker object before all of the object references go out of scope (#212) 2016-07-05 19:13:57 -07:00
Robert Nishihara
0947024ad9 fix bug for functions with no return values and with one return value (#211) 2016-07-05 15:57:05 -07:00
mehrdadn
be26532182 Better code for getting log directory (#198)
* get_log_file_path instead of LOG_DIRECTORY

* Tabs -> spaces

Whoops

* Log file extension
2016-07-03 12:58:49 -07:00
Philipp Moritz
2e2bc2c740 Get Sphinx infrastructure in place 2016-07-01 18:21:02 -07:00
Robert Nishihara
0ffe657e27 enable restarting workers in singlenode case, plus cleanups to cluster.py (#190) 2016-07-01 14:10:51 -07:00
Robert Nishihara
f5316d50fc Merge pull request #193 from amplab/visualizegraph
Visualize the computation graph
2016-07-01 13:21:01 -07:00
Philipp Moritz
9ad2c033d4 Visualize computation graph 2016-07-01 12:13:23 -07:00
Mehrdad
18f033fefb Use proper temp directory 2016-06-30 16:29:24 -07:00
Mehrdad
5802ea58b7 Look in PATH for binaries also 2016-06-29 23:25:19 -07:00
mehrdadn
5328890001 Kill workers (#148) 2016-06-29 22:40:04 -07:00