* First pass at a policy to solve deadlock
* Address Robert's comments
* stress test
* unit test
* Fix test cases
* Fix test for python3
* add more logging
* White space.
* Remove import counter and export counter.
* Provide isolation between drivers for remote functions.
* Add test for driver function isolation.
* Hash source code into function ID to reduce likelihood of collisions.
* Fix failure test example.
* Replace assertTrue with assertIn to improve failure messages in tests.
* Fix failure test.
* Implement actor field for tasks
* Implement actor management in local scheduler.
* initial python frontend for actors
* import actors on worker
* IPython code completion and tests
* prepare creating actors through local schedulers
* add actor id to PyTask
* submit actor calls to local scheduler
* starting to integrate
* simple fix
* Fixes from rebasing.
* more work on python actors
* Improve local scheduler actor handlers.
* Pass actor ID to local scheduler when connecting a client.
* first working version of actors
* fixing actors
* fix creating two copies of the same actor
* fix actors
* remove sleep
* get rid of export synchronization
* update
* insert actor methods into the queue in the right order
* remove print statements
* make it compile again after rebase
* Minor updates.
* fix python actor ids
* Pass actor_id to start_worker.
* add test
* Minor changes.
* Update actor tests.
* Temporary plan for import counter.
* Temporarily fix import counters.
* Fix some tests.
* Fixes.
* Make actor creation non-blocking.
* Fix test?
* Fix actors on Python 2.
* fix rare case.
* Fix python 2 test.
* More tests.
* Small fixes.
* Linting.
* Revert tensorflow version to 0.12.0 temporarily.
* Small fix.
* Enhance inheritance test.
* attribute-based heterogeneity-awareness in global scheduler and photon
* minor post-rebase fix
* photon: enforce dynamic capacity constraint on task dispatch
* globalsched: cap the number of times we try to schedule a task in round robin
* propagating ability to specify resource capacity to ray.init
* adding resources to remote function export and fetch/register
* globalsched: remove unused functions; update cached photon resource capacity (until next photon heartbeat)
* Add some integration tests.
* globalsched: cleanup + factor out constraint checking
* lots of style
* task_spec_required_resource: global refactor
* clang format
* clang format + comment update in photon
* clang format photon comment
* valgrind
* reduce verbosity for Travis
* Add test for scheduler load balancing.
* addressing comments
* refactoring global scheduler algorithm
* Minor cleanups.
* Linting.
* Fix array_test.py and linting.
* valgrind fix for photon tests
* Attempt to fix stress tests.
* fix hashmap free
* fix hashmap free comment
* memset photon resource vectors to 0 in case they get used before the first heartbeat
* More whitespace changes.
* Undo whitespace error I introduced.
* Refactor local scheduler to remove worker indices.
* Change scheduling state enum to int in all function signatures.
* Bug fix, don't use pointers into a resizable array.
* Remove total_num_workers.
* Fix tests.
* First pass at reconstruction in the worker
Modify reconstruction stress testing to start Plasma service before rest of Ray cluster
TODO about reconstructing ray.puts
Fix ray.put error for double creates
Distinguish between empty entry and no entry in object table
Fix test case
Fix Python test
Fix tests
* Only call reconstruct on objects we have not yet received
* Address review comments
* Fix reconstruction for Python3
* remove unused code
* Address Robert's comments, stress tests are crashing
* Test and update the task's scheduling state to suppress duplicate
reconstruction requests.
* Split result table into two lookups, one for task ID and the other as a
test-and-set for the task state
* Fix object table tests
* Fix redis module result_table_lookup test case
* Multinode reconstruction tests
* Fix python3 test case
* rename
* Use new start_redis
* Remove unused code
* lint
* indent
* Address Robert's comments
* Use start_redis from ray.services in state table tests
* Remove unnecessary memset
* Provide functionality for local scheduler to start new workers.
* Pass full command for starting new worker in to local scheduler.
* Separate out configuration state of local scheduler.
* Optimizations:
- Track mapping of missing object to dependent tasks to avoid iterating over task queue
- Perform all fetch requests for missing objects using the same timer
* Fix bug and add regression test
* Record task dependencies and active fetch requests in the same hash table
* fix typo
* Fix memory leak and add test cases for scheduling when dependencies are evicted
* Fix python3 test case
* Minor details.
* Split local scheduler task queue into waiting and dispatch queue
* Fix memory leak
* Add a new task scheduling status for when a task has been queued locally
* Fix global scheduler test case and add task status doc
* Documentation
* Address Philipp's comments
* Move tasks back to the waiting queue if their dependencies become unavailable
* Update existing task table entries instead of overwriting
* Object notification test for Photon, and turn on valgrind for Photon C tests
* Test object notification handler in the plasma manager
* Fix hanging test case
* Object reconstruction in Photon and C test cases for Photon
* Fix hanging test case on mac
* Remove unnecessary event from photon tests
* make photon_disconnect not leak file descriptors
* fix some of the memory errors
* Fix valgrind
* lint
* Address Robert's comments and add test case for object reconstruction suppression
* Remove OWNER
* Consolidate wait implementations.
* Consolidate fetch implementations.
* Share callback between wait and fetch to address issue in which only one callback can be run for a given subscribe channel.
* Reactivate manager tests.
* Remove more code.
* Add some documentation.
* Fetch missing dependencies from local scheduler.
* Factor out global scheduler policy state.
* Use object_table_subscribe instead of object_table_lookup.
* Fix bug in which timer was being created twice for a single fetch request.
* Free old manager vector.
* Use sizeof(field) instead of sizeof(type) and other fixes.
* Fix formatting.
* Bug fix.
* Zero-initialize structs. There are many more instances of these that I haven't changed yet.
* Bug fix.
* Revert from atexit to signaling to fix valgrind tests.
* Address Philipp's comments.
* Initial scheduler commit
* global scheduler
* add global scheduler
* Implement global scheduler skeleton.
* Formatting.
* Allow local scheduler to be started without a connection to redis so that we can test it without a global scheduler.
* Fail if there are no local schedulers when the global scheduler receives a task.
* Initialize uninitialized value and formatting fix.
* Generalize local scheduler table to db client table.
* Remove code duplication in local scheduler and add flag for whether a task came from the global scheduler or not.
* Queue task specs in the local scheduler instead of tasks.
* Simple global scheduler tests, including valgrind.
* Factor out functions for starting processes.
* Fixes.
* Set logging levels in Makefile using -DRAY_COMMON_LOG_LEVEL=level
* Lower level of some LOG_ERROR messages, log the name of the table operation on failure
* Address rest of Robert's comments
* Fix spurious log message
* Merge task table and task log
* Fix test in db tests
* Address Robert's comments and some better error checking
* Add a LOG_FATAL that exits the program
* Put infrastructure in place to compute task IDs and object IDs.
* Fix version number for common library.
* Compute task IDs and object IDs deterministically.
* Address Stephanie's comments.
* Update task documentation.
* Fix formatting.
* Add more tests and checks.
* Fix formatting.
* Enable DCHECKs and change CHECKs to DCHECKs.
* Update worker code and services code to use plasma and the local scheduler.
* Cleanups.
* Fix bug in which threads were started before the worker mode was set. This caused remote functions to be defined on workers before the worker knew it was in WORKER_MODE.
* Fix bug in install-dependencies.sh.
* Lengthen timeout in failure_test.py.
* Cleanups.
* Cleanup services.start_ray_local.
* Clean up random name generation.
* Cleanups.
* Ion and Philipp's table retries
* Refactor the retry struct:
- Rename it from retry_struct to retry_info
- Retry information contains the failure callback, not the retry callback
- All functions take in retry information as an arg instead of its expanded fields
* Rename cb -> callback
* Remove prints
* Fix compiler warnings
* Change some CHECKs to greatest ASSERTs
* Key outstanding callbacks hash table with timer ID instead of callback data pointer
* Use the new retry API for table commands
* Memory cleanup in plasma unit tests
* fix Robert's comments
* add valgrind for common
