ray/test/jenkins_tests/multi_node_tests/test_0.py
Robert Nishihara 0ac125e9b2 Clean up when a driver disconnects. (#462)
* Clean up state when drivers exit.

* Remove unnecessary field in ActorMapEntry struct.

* Have monitor release GPU resources in Redis when driver exits.

* Enable multiple drivers in multi-node tests and test driver cleanup.

* Make redis GPU allocation a redis transaction and small cleanups.

* Fix multi-node test.

* Small cleanups.

* Make global scheduler take node_ip_address so it appears in the right place in the client table.

* Cleanups.

* Fix linting and cleanups in local scheduler.

* Fix removed_driver_test.

* Fix bug related to vector -> list.

* Fix linting.

* Cleanup.

* Fix multi node tests.

* Fix jenkins tests.

* Add another multi node test with many drivers.

* Fix linting.

* Make the actor creation notification a flatbuffer message.

* Revert "Make the actor creation notification a flatbuffer message."

This reverts commit af99099c8084dbf9177fb4e34c0c9b1a12c78f39.

* Add comment explaining flatbuffer problems.
2017-04-24 18:10:21 -07:00

34 lines
932 B
Python

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
import time
import ray
@ray.remote
def f():
time.sleep(0.1)
return ray.services.get_node_ip_address()
if __name__ == "__main__":
driver_index = int(os.environ["RAY_DRIVER_INDEX"])
redis_address = os.environ["RAY_REDIS_ADDRESS"]
print("Driver {} started at {}.".format(driver_index, time.time()))
ray.init(redis_address=redis_address)
# Check that tasks are scheduled on all nodes.
num_attempts = 30
for i in range(num_attempts):
ip_addresses = ray.get([f.remote() for i in range(1000)])
distinct_addresses = set(ip_addresses)
counts = [ip_addresses.count(address) for address in distinct_addresses]
print("Counts are {}".format(counts))
if len(counts) == 5:
break
assert len(counts) == 5
print("Driver {} finished at {}.".format(driver_index, time.time()))