Commit graph

766 commits

Author SHA1 Message Date
Philipp Moritz
ff8018db75 First pass at local scheduler (#2)
* draft of local scheduler

* API

* update APIs

* fix

* update

* Rename halo -> photon.

* Add build directory.

* Update common submodule.

* More renaming.

* Fix python ctypes.

* Compile in travis.

* Process generic messages and not just tasks.

* Move free outside of switch.

* Formatting and address comments.

* Remove event loop from local scheduler state.

* Use accept_client from common.

* Use bind_ipc_sock from common.

* Fix tests.

* Update common submodule.

* Fix formatting.
2016-09-27 19:11:09 -07:00
Robert Nishihara
084220b0e7 Allow reading/writing generic message types, not just tasks. (#24)
* Allow reading/writing generic message types, not just tasks.

* Allow messages of length 0 to be read/written, and handle closed sockets.

* Address comments.

* Simplify accept_client.

* Allow ports to be reused in bind_ipc_sock.
2016-09-27 18:51:35 -07:00
Philipp Moritz
21d6ceba01 Merge pull request #4 from ray-project/trav
Add basic travis file and lint check.
2016-09-27 10:56:35 -07:00
Robert Nishihara
2776d6a37c Fix permissions. 2016-09-27 00:19:21 -07:00
Robert Nishihara
57214b3a11 Fixes. 2016-09-27 00:15:54 -07:00
Robert Nishihara
184242a85c Add basic travis file and lint check. 2016-09-27 00:13:28 -07:00
Richard Liaw
8925580fb6 nit for filter (#443) 2016-09-26 20:58:19 -07:00
Richard Liaw
d22321da7a Changes to run TRPO (#442)
Filter/new updates

final keras changes

some changes
2016-09-26 20:20:45 -07:00
Robert Nishihara
631de92170 Build redis before libcommon. (#20) 2016-09-26 00:12:11 -07:00
Robert Nishihara
9a6991116f Small fix in test. (#441) 2016-09-25 23:08:27 -07:00
Robert Nishihara
9f26c2d793 Merge pull request #3 from ray-project/rename
Update README.md
2016-09-25 22:51:00 -07:00
Philipp Moritz
675ea40171 Update README.md 2016-09-25 22:49:55 -07:00
Philipp Moritz
db8c0acc71 Refactor state database (#22)
* make db_connect return the connection

* rename db_conn -> db_handle

* more renaming

* clang-format

* free the db_handle
2016-09-25 21:52:06 -07:00
Philipp Moritz
1e08629013 fix submit_task for task queue (#21) 2016-09-25 16:51:24 -07:00
Philipp Moritz
7907992609 [WIP] Event loop refactoring (#19)
* task queue tests and extensions

* event loop refactor

* fix formating
2016-09-23 22:53:58 -07:00
Philipp Moritz
e1b8711a01 Redis logging (#17)
* Redis logging

* Rearrange logging interfaces

* Fix test case

* Changes to logging interface and test case for logging

* Fixes

* Fix memory leaks

* Add interface method to destroy logger

* is_local -> is_direct

* Merge fix
2016-09-23 17:10:15 -07:00
Robert Nishihara
d41566a499 Implement delete and contains for plasma client and store. (#28)
* Implement delete and contains for plasma client and store.

* Fix style, free object, add tests.
2016-09-23 15:07:50 -07:00
Philipp Moritz
7a079547b0 task queue tests and extensions (#18)
* task queue tests and extensions

* clean up test
2016-09-22 23:15:45 -07:00
Stephanie Wang
313241e303 Asynchronous Redis IPC (#14)
* Asynchronous Redis IPC

* make valgrind happy

* cleanup
2016-09-20 22:40:35 -07:00
Philipp Moritz
6c6f2d0473 Implement object table API (#16) 2016-09-20 17:02:56 -07:00
Robert Nishihara
de6ec47f9e Add a recursion depth for serialization to prevent infinite loops. (#440) 2016-09-19 17:17:42 -07:00
Robert Nishihara
7055c6f793 Limit recursion depth for serializing objects to avoid infinite loops. (#17) 2016-09-19 14:54:56 -07:00
Philipp Moritz
d11161bb01 make static libraries (#13) 2016-09-18 20:47:53 -07:00
Philipp Moritz
37f035dbd0 implementing reading and writing tasks (#11) 2016-09-18 18:06:42 -07:00
Philipp Moritz
c238ae4aa0 do not re-download and rebuild redis if it already exists (#10) 2016-09-18 13:57:27 -07:00
Philipp Moritz
b18f214d55 Make it possible to read and write data that is not null-terminated (#9)
* Make it possible to read and write data that is not null-terminated

* formating
2016-09-18 13:35:43 -07:00
Stephanie Wang
ff11ee21ef Convert to streaming sockets (#7)
* Convert to streaming sockets

* Formatting
2016-09-17 15:15:18 -07:00
Robert Nishihara
91f16a3df0 Migrate repositories to ray-project. (#438)
* Migrate repositories to ray-project.

* Update numbuf to the migrated version.
2016-09-17 00:52:05 -07:00
Robert Nishihara
4fc42c3e1e Fix in documentation link. (#439) 2016-09-17 00:45:22 -07:00
Philipp Moritz
77d5365cc0 Merge pull request #1 from ray-project/name
Update README.md
2016-09-17 00:36:52 -07:00
Robert Nishihara
9e4203c18c Update README.md 2016-09-17 00:30:31 -07:00
Robert Nishihara
0527736490 Migrate repository to ray-project. (#15) 2016-09-17 00:14:35 -07:00
Philipp Moritz
0b7d81cae6 API for creating task specifications (#5)
* API for creating task specifications

* fixes

* add more checks and improve comments
2016-09-17 00:03:10 -07:00
Robert Nishihara
52d953bde6 Document serialization in the object store. (#436) 2016-09-16 23:49:59 -07:00
Robert Nishihara
228525eec7 Add documentation for reusable variables. (#437) 2016-09-16 23:10:34 -07:00
Robert Nishihara
4863a5155c Cleanup setting and getting of tensorflow weights. (#385)
* Cleanup setting and getting of tensorflow weights.

* Add documentation for using TensorFlow.

* Group get_weights and set_weights in a function.

* Update readme.
2016-09-16 23:05:14 -07:00
Philipp Moritz
4521e5f183 Initial commit 2016-09-16 18:26:57 -07:00
Stephanie Wang
73f4b96253 Sockets (#3)
* Socket methods to be used by an event loop

* Git ignore build files

* File renames

* Some fixes

* Fixes

* Fixes

* Memory leakage fix
2016-09-15 16:28:52 -07:00
Robert Nishihara
1aa89a4ae6 Update numbuf to properly handle Python floats. (#435) 2016-09-15 15:44:11 -07:00
Philipp Moritz
0198a0d299 make plasma robust wrt to number of open file descriptors (#25) 2016-09-15 15:39:33 -07:00
Robert Nishihara
d469366c75 Serialize Python floats properly as doubles. (#14) 2016-09-15 15:11:51 -07:00
Robert Nishihara
292656013a Suppress exceptions in the error logging thread when program exits. (#432) 2016-09-15 13:48:23 -07:00
Robert Nishihara
a5c026c033 Remove unused serialize/deserialize code in raylib. (#434) 2016-09-15 13:46:38 -07:00
Robert Nishihara
13560bdb6b Fix offset in get_malloc_mapinfo. (#24)
* Fix offset in get_malloc_mapinfo.

* Don't add offset inside mmap_record

* make clang-format happy
2016-09-14 17:45:18 -07:00
Robert Nishihara
9ecf72484b This solves a problem where exports were sent to workers out of order. (#430)
* Revert queueing of exports in scheduler.

* Export each category of exports to workers in the order that they were received.

* Small fixes.
2016-09-14 17:18:02 -07:00
Wapaul1
d5815673a5 Changed ray.select() to ray.wait() and its functionality (#426)
* Re-implemented select, changed name to wait

* Changed tests for select to tests for wait

* Updated the hyperopt example to match wait

* Small fixes and improve example readme.

* Make tests pass.
2016-09-14 17:14:11 -07:00
Robert Nishihara
8c6d3a88a9 Properly decrement Python reference count in PyObjectID_hash. (#429) 2016-09-14 17:09:11 -07:00
Robert Nishihara
28c19a38c9 Catch incorrect arguments to PlasmaClient constructor. (#18) 2016-09-14 14:21:24 -07:00
Robert Nishihara
72361c9b44 Add metadata handling (#23)
* Automatic whitespace fixes.

* Add metadata handling.

* Make create take a buffer instead of a string for the metadata.

* Small fixes.
2016-09-14 14:20:34 -07:00
Philipp Moritz
7d629d4e48 Adding object table (#1)
* code for maintaining the object table

* Makefile fix

* Clone git submodules.

* directory -> object_table

* Fix Makefile and remove unnecessary files.

* Fix formatting.

* make code more generic
2016-09-13 18:54:26 -07:00