Commit graph

4 commits

Author SHA1 Message Date
Balaji Veeramani
7f1bacc7dc
[CI] Format Python code with Black (#21975)
See #21316 and #21311 for the motivation behind these changes.
2022-01-29 18:41:57 -08:00
Antoni Baum
7ce22b72ed
[datasets] Expand to_torch's functionality (#21117)
Expands the `to_torch` method for Datasets with:
* An ability to choose to output a list/dict of feature tensors instead of just one (through setting `feature_columns` to be a list of lists or a dict of lists)
* An ability to choose whether the label should be unsqueezed or not
* An ability to pass `None` as the label (for prediction).

Furthermore, this changes how the `feature_column_dtypes` argument works. Previously, it took a list of dtypes for each feature. However, as the tensor was concatenated in the end, only one dtype mattered (the biggest one). Now, this argument expects a single dtype which will be applied to the features tensor (or a list/dict if `feature_columns` is a list of list/dict of lists).

Unit tests for all cases are included.

Co-authored-by: matthewdeng <matthew.j.deng@gmail.com>
2022-01-03 09:03:50 -08:00
Clark Zinzow
ca731d7c86
[Datasets] Fix API breakage in Datasets nightly test. 2021-10-07 15:07:19 -07:00
Chen Shen
89f988e9cc
add dataset shuffle data loader (#17917) 2021-08-20 11:26:01 -07:00