hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-04 17:41:43 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

[AIR] Split train dataset in HuggingFaceTrainer (#28170)

Browse source 
https://github.com/ray-project/ray/pull/25428 inadvertently turned off train dataset splitting for the `HuggingFaceTrainer`, which meant it wasn't actually running in a data parallel fashion. This PR fixes that.

Signed-off-by: Antoni Baum antoni.baum@protonmail.com
This commit is contained in:
Antoni Baum 2022-08-30 21:35:44 +02:00 • committed by GitHub
parent f74f155af4
commit d7f712d202
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 5 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
python/ray/train/huggingface/huggingface_trainer.py
View file

@ -266,7 +266,11 @@ class HuggingFaceTrainer(TorchTrainer):
_checkpoint_manager_cls = _DataParallelSyncingCheckpointManager
_dataset_config = {
"train": DatasetConfig(fit=True, split=False, required=True),
# training dataset should be split by us
"train": DatasetConfig(fit=True, split=True, required=True),
# do not split eval dataset, as HF has a system to parallelize
# evaluation across workers, and it requires each worker
# to have the full eval dataset
"evaluation": DatasetConfig(split=False),
}