Skip to content

Commit e3e1fa2

Browse files
authored
fix exausting dataset hf (#116)
1 parent 155f83c commit e3e1fa2

File tree

1 file changed

+1
-4
lines changed

1 file changed

+1
-4
lines changed

src/zeroband/data.py

+1-4
Original file line numberDiff line numberDiff line change
@@ -218,10 +218,7 @@ def _load_datasets(
218218
datasets.append(_ds)
219219
# logger.debug(f"Loaded dataset: {ds_arg}")
220220

221-
ds = interleave_datasets(
222-
datasets=datasets,
223-
probabilities=probabilities,
224-
)
221+
ds = interleave_datasets(datasets=datasets, probabilities=probabilities, stopping_strategy="all_exhausted")
225222
logger.info(f"Loaded datasets ({split})")
226223
return ds
227224

0 commit comments

Comments
 (0)