Why are these changes needed?
Consumers (e.g. Train) may expect generated batches to be of the same size. Prior to this change, the default behavior would be for each batch to be one block, which may be of different sizes.
Changes
Set default batch_size to 256. This was chosen to be a sensible default for training workloads, which is intentionally different from the existing default batch_size value for Dataset.map_batches.
Update docs for Dataset.iter_batches, Dataset.map_batches, and DatasetPipeline.iter_batches to be consistent.
Updated tests and examples to explicitly pass in batch_size=None as these tests were intentionally testing block iteration, and there are other tests that test explicit batch sizes.
ray.init() will currently start a new Ray instance even if one is already existing, which is very confusing if you are a new user trying to go from local development to a cluster. This PR changes it so that, when no address is specified, we first try to find an existing Ray cluster that was created through `ray start`. If none is found, we will start a new one.
This makes two changes to the ray.init() resolution order:
1. When `ray start` is called, the started cluster address was already written to a file called `/tmp/ray/ray_current_cluster`. For ray.init() and ray.init(address="auto"), we will first check this local file for an existing cluster address. The file is deleted on `ray stop`. If the file is empty, autodetect any running cluster (legacy behavior) if address="auto", or we will start a new local Ray instance if address=None.
2. When ray.init(address="local") is called, we will create a new local Ray instance, even if one is already existing. This behavior seems to be necessary mainly for `ray.client` use cases.
This also surfaces the logs about which Ray instance we are connecting to. Previously these were hidden because we didn't set up the log until after connecting to Ray. So now Ray will log one of the following messages during ray.init:
```
(Connecting to existing Ray cluster at address: <IP>...)
...connection...
(Started a local Ray cluster.| Connected to Ray Cluster.)( View the dashboard at <URL>)
```
Note that this changes the dashboard URL to be printed with `ray.init()` instead of when the dashboard is first started.
Co-authored-by: Eric Liang <ekhliang@gmail.com>
This PR tries to automatically cast tensor columns to our TensorArray extension type when building Pandas blocks, logging a warning and falling back to the opaque object-typed column if the cast fails. This should allow users to remain mostly tensor extension agnostic.
TensorArray now eagerly validates the underlying tensor data, raising an error if e.g. the underlying ndarrays have heterogeneous shapes; previously, TensorArray wouldn't validate this on construction and would instead let failures happen downstream. This means that our internal TensorArray use needs to follow a try-except pattern, falling back to a plain NumPy object column.
This PR adds .iter_torch_batches() and .iter_tf_batches() convenience APIs, which takes care of ML framework tensor conversion, the narrow tensor waste for the .iter_batches() call ("numpy" format), and unifies batch formats around two options: a single tensor for simple/pure-tensor/single-column datasets, and a dictionary of tensors for multi-column datasets.
We don't have a way to specify resource requirements with the Tuner() API. This PR introduces tune.with_resources() to attach a resource request to class and function trainables. In class trainables, it will override potential existing default resource requests.
Signed-off-by: Kai Fricke <kai@anyscale.com>
Having the indicator about who's running the stage and who created a blocklist will enable the eager memory releasing.
This is an alternative with better abstraction to https://github.com/ray-project/ray/pull/26196.
Note: this doesn't work for Dataset.split() yet, will do in a followup PR.
The current split_at_index might generate empty blocks and also trigger unnecessary split task. The empty blocks happens when there are duplicate split indices, or the split index falls at the block boundaries. The unnecessary split tasks are triggered when the split index falls at the block boundaries.
This PR fix that by checking if the split index is duplicated or falls at the boundaries of blocks. in that case, we could safely ignore those indices.
This PR adds a Pandas-native implementation of groupby and sorting for Pandas blocks. Before this PR, we were converting to Arrow, doing groupbys + aggregations and sorting in Arrow land, and then converting back to Pandas; this to-from-Arrow conversion was happening both on the map side and the reduce side, which was very inefficient for Pandas blocks (many extra table copies). By adding Pandas-native groupby + sorting, we should see a decrease in memory utilization and faster performance when using the AIR preprocessors.
As followup of #26669 (comment), we want to add AWS CLI command information into S# credential error message, so users have a better idea to further debug the read issue.
The previously observed Python grpc warning / logspam seems to have been fixed for grpcio >= 1.48. And users would like to upgrade beyond grpcio 1.43 for better M1 support. However, grpcio 1.48 has not been released yet, so there is still a risk this change needs to be reverted if any problem is discovered later with Ray nightly + grpcio 1.48.
This sets the CUDA Stream on the correct device (and not the default one) when calling train.torch.prepare_data_loader(auto_transfer=True).
Signed-off-by: Matthew Deng <matt@anyscale.com>
Previously, using an env_hook with Ray Client would only execute the env_hook on the server side (a Ray cluster machine). An env_hook defined on the client side would never be executed. But the main problem is with the server-side env_hook.
Consider the simple example where the env_hook rewrites the `working_dir` or `py_modules` with a local directory.
Currently, when using Ray Client, the `working_dir` and `py_modules` are uploaded to the GCS before `ray.init()` is called on the server. This is a fundamental constraint because the server-side driver script needs to be able to import modules from the `working_dir` or `py_modules`. After the upload, these fields are overwritten with the URIs for the uploaded packages.
After this happens, on the server side Ray expects the `working_dir` and `py_modules` fields to only contain GCS URIs. So overwriting `working_dir` to be a local directory after this occurs doesn't make sense (and Ray will rightfully throw a RuntimeEnv validation error here.)
If a cluster is set up with such an env hook, it will only work when `ray.init()` is called by the user on a cluster machine; i.e. it will only work in non-Ray Client cases. If a user ever wants to use Ray Client with this cluster, it will be broken with no way to disable the env hook. To remedy this, this PR disables the execution of the env_hook when using Ray Client.
We can consider adding support in the future for env_hooks to be executed on the client side when using Ray Client.
This picks up https://github.com/ray-project/ray/pull/24088
The `get_node_table` already has resources of nodes, so we don't need to invoke `get_node_resource_info` for every node again. This change will reduce lots of rpc calls and make the api more efficient.
This PR prevents the log monitor for keeping files open for long periods of time. In settings in which the autoscaler and head node are not tightly coupled, leaving files open implies that the inode for a file never changes, but depending on how fs synchronization between the autoscaler and head node containers works, the inode could change. Thus, we should keep try reopening files.
This is done via setting max open files to 1, so that it's easy to revert this behavior.
Co-authored-by: Alex <alex@anyscale.com>
Why are these changes needed?
Together with ray-project/kuberay#391, this should address #26064
Have the autoscaler try to make the Ray log directory before setting up logging.
ray-project/kuberay#391 should be good enough for this, but this PR makes things safer in case the KubeRay user overrides log mounts or something like that.
I am surprised by the fact that `GetTimeoutError` is not a subclass of `TimeoutError`, which is counter-intuitive and may discourage users from trying the timeout feature in `ray.get`, because you have to "guess" the correct error type. For most people, I believe the first error type in their mind would be `TimeoutError`.
This PR fixes this.
✨ Add extra types for datasets
Why are these changes needed?
By using Protocols for the definition of callable classes, the type parameter in datasets can be preserved for the methods that work on rows.
This allows getting typing information, editor support, in transformed datasets. It also enables editor support, including autocompletion, inside of the parameters inside of lambdas passed to things like ds.map() and ds.filter().
For example, after several transformations, ds.filter() still gets autocompletion for the x parameter in the lambda, the editor knows it's an int.
https://user-images.githubusercontent.com/1326112/179423609-6d77da23-5f5e-47ce-a17f-6eb0d06d82d0.png
I see there was a TODO comment to do this trick with the protocol, so Clark already had this idea. 💡 The good news is that it's not necessary to wait until only Python 3.8 is supported by using typing_extensions.
