ray/doc/scheduler.md

26 lines
1.3 KiB
Markdown
Raw Normal View History

2016-05-17 15:08:49 -07:00
# Scheduler
2016-06-10 14:12:15 -07:00
The scheduling strategies currently implemented in Ray are fairly basic and
2016-05-17 15:08:49 -07:00
all use a central scheduler.
* The naive scheduler assigns tasks to workers just taking into account
dependencies between tasks (no other information like data locality). It is
supposed to be an example for how to write a scheduler. We do not recommend
its use and it only works well for single node setups. Tasks are assigned in the
following way: For each idle worker, we iterate over the tasks in the task
queue. The first task that has all its requirements satisfied will be scheduled
on the worker.
* The locality aware scheduler is more suited for multi node setups, but still
inappropriate for very large clusters. This is because the computational
overhead for each scheduling decision is O(mn) where m is the number of idle
workers and n is the number of tasks in the task queue. For each idle worker,
all tasks in the task queue are considered and the one that requires the
smallest number of objects to be shipped will be executed.
We expect to implement more refined scheduling strategies in the future,
including more computationally efficient location aware scheduling,
scheduling that takes into account sizes of the shipped objects, and strategies
that do not require a central scheduler (which is a bottleneck for large
clusters).