mirror of
https://github.com/vale981/ray
synced 2025-03-11 21:56:39 -04:00
8f9e0d7f6b
The [original PR](https://github.com/ray-project/ray/pull/21864) was [reverted](https://github.com/ray-project/ray/pull/22117) because it caused `torch` (more specifically, `torch>=1.8.1`) to be required to use `ray.train`. ``` | File "ray_sgd_training.py", line 18, in <module> | from ray import train | File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/train/__init__.py", line 2, in <module> | from ray.train.callbacks import TrainingCallback | File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/train/callbacks/__init__.py", line 8, in <module> | from ray.train.callbacks.profile import TorchTensorboardProfilerCallback | File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/train/callbacks/profile.py", line 6, in <module> | from torch.profiler import profile | ModuleNotFoundError: No module named 'torch.profiler' ``` A [minimal installation test suite](https://github.com/ray-project/ray/pull/22300) was added to detect this. Further, in this PR we make the following changes: 1. Move `TorchWorkerProfiler` to `ray.train.torch` so all torch imports are centralized. 2. Add import validation logic to `TorchWorkerProfiler.__init__` so an exception will only be raised if the user tries to initialize a `TorchWorkerProfiler` without having a valid version of `torch` installed: ``` >>> import ray >>> import ray.train >>> import ray.train.torch >>> from ray.train.torch import TorchWorkerProfiler >>> twp = TorchWorkerProfiler() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/matt/workspace/ray/python/ray/train/torch.py", line 365, in __init__ "Torch Profiler requires torch>=1.8.1. " ImportError: Torch Profiler requires torch>=1.8.1. Run `pip install 'torch>=1.8.1'` to use TorchWorkerProfiler. ```
200 lines
No EOL
3.6 KiB
ReStructuredText
200 lines
No EOL
3.6 KiB
ReStructuredText
|
|
.. _train-api:
|
|
|
|
Ray Train API
|
|
=============
|
|
|
|
.. _train-api-trainer:
|
|
|
|
Trainer
|
|
-------
|
|
|
|
.. autoclass:: ray.train.Trainer
|
|
:members:
|
|
|
|
.. _train-api-iterator:
|
|
|
|
TrainingIterator
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: ray.train.TrainingIterator
|
|
:members:
|
|
|
|
.. _train-api-backend-config:
|
|
|
|
Backend Configurations
|
|
----------------------
|
|
|
|
.. _train-api-torch-config:
|
|
|
|
TorchConfig
|
|
~~~~~~~~~~~
|
|
|
|
.. autoclass:: ray.train.torch.TorchConfig
|
|
|
|
.. _train-api-tensorflow-config:
|
|
|
|
TensorflowConfig
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: ray.train.tensorflow.TensorflowConfig
|
|
|
|
.. _train-api-horovod-config:
|
|
|
|
HorovodConfig
|
|
~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: ray.train.horovod.HorovodConfig
|
|
|
|
|
|
Callbacks
|
|
---------
|
|
|
|
.. _train-api-callback:
|
|
|
|
TrainingCallback
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: ray.train.TrainingCallback
|
|
:members:
|
|
|
|
.. _train-api-print-callback:
|
|
|
|
PrintCallback
|
|
~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: ray.train.callbacks.PrintCallback
|
|
|
|
.. _train-api-json-logger-callback:
|
|
|
|
JsonLoggerCallback
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: ray.train.callbacks.JsonLoggerCallback
|
|
|
|
.. _train-api-tbx-logger-callback:
|
|
|
|
TBXLoggerCallback
|
|
~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: ray.train.callbacks.TBXLoggerCallback
|
|
|
|
.. _train-api-mlflow-logger-callback:
|
|
|
|
MLflowLoggerCallback
|
|
~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: ray.train.callbacks.MLflowLoggerCallback
|
|
|
|
|
|
.. _train-api-torch-tensorboard-profiler-callback:
|
|
|
|
TorchTensorboardProfilerCallback
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: ray.train.callbacks.TorchTensorboardProfilerCallback
|
|
|
|
ResultsPreprocessors
|
|
~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. _train-api-results-preprocessor:
|
|
|
|
ResultsPreprocessor
|
|
+++++++++++++++++++
|
|
|
|
.. autoclass:: ray.train.callbacks.results_preprocessors.ResultsPreprocessor
|
|
:members:
|
|
|
|
SequentialResultsPreprocessor
|
|
+++++++++++++++++++++++++++++++
|
|
|
|
.. autoclass:: ray.train.callbacks.results_preprocessors.SequentialResultsPreprocessor
|
|
|
|
IndexedResultsPreprocessor
|
|
+++++++++++++++++++++++++++++++
|
|
|
|
.. autoclass:: ray.train.callbacks.results_preprocessors.IndexedResultsPreprocessor
|
|
|
|
ExcludedKeysResultsPreprocessor
|
|
+++++++++++++++++++++++++++++++
|
|
|
|
.. autoclass:: ray.train.callbacks.results_preprocessors.ExcludedKeysResultsPreprocessor
|
|
|
|
Checkpointing
|
|
-------------
|
|
|
|
.. _train-api-checkpoint-strategy:
|
|
|
|
CheckpointStrategy
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: ray.train.CheckpointStrategy
|
|
|
|
Training Function Utilities
|
|
---------------------------
|
|
|
|
train.report
|
|
~~~~~~~~~~~~
|
|
|
|
.. autofunction:: ray.train.report
|
|
|
|
train.load_checkpoint
|
|
~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autofunction:: ray.train.load_checkpoint
|
|
|
|
train.save_checkpoint
|
|
~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autofunction:: ray.train.save_checkpoint
|
|
|
|
train.world_rank
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
.. autofunction:: ray.train.world_rank
|
|
|
|
train.local_rank
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
.. autofunction:: ray.train.local_rank
|
|
|
|
train.world_size
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
.. autofunction:: ray.train.world_size
|
|
|
|
.. _train-api-torch-utils:
|
|
|
|
PyTorch Training Function Utilities
|
|
-----------------------------------
|
|
|
|
train.torch.prepare_model
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autofunction:: ray.train.torch.prepare_model
|
|
|
|
train.torch.prepare_data_loader
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autofunction:: ray.train.torch.prepare_data_loader
|
|
|
|
train.torch.get_device
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autofunction:: ray.train.torch.get_device
|
|
|
|
.. _train-api-torch-worker-profiler:
|
|
|
|
train.torch.TorchWorkerProfiler
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: ray.train.torch.TorchWorkerProfiler
|
|
:members:
|
|
|
|
TensorFlow Training Function Utilities
|
|
--------------------------------------
|
|
|
|
train.tensorflow.prepare_dataset_shard
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autofunction:: ray.train.tensorflow.prepare_dataset_shard |