mirror of
https://github.com/vale981/ray
synced 2025-03-06 10:31:39 -05:00
4444150c29
This PR is a general overhaul of the "Creating Datasets" feature guide, providing complete coverage of all (public) dataset creation APIs and highlighting features and quirks of the individual APIs, data modalities, storage backends, etc. In order to keep the page from getting too long and keeping it easy to navigate, tabbed views are used heavily.
16 lines
567 B
ReStructuredText
16 lines
567 B
ReStructuredText
.. _datasets_custom_datasource:
|
|
|
|
------------------------
|
|
Using Custom Datasources
|
|
------------------------
|
|
|
|
Datasets can read and write in parallel to `custom datasources <package-ref.html#custom-datasource-api>`__ defined in Python.
|
|
Once you have implemented `YourCustomDataSource`, you can use it like any other source in Ray Data:
|
|
|
|
.. code-block:: python
|
|
|
|
# Read from a custom datasource.
|
|
ds = ray.data.read_datasource(YourCustomDatasource(), **read_args)
|
|
|
|
# Write to a custom datasource.
|
|
ds.write_datasource(YourCustomDatasource(), **write_args)
|