You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/modules/ROOT/pages/lm-eval-tutorial.adoc
+5-5Lines changed: 5 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -182,18 +182,18 @@ A list of paired name and value arguments for the model type. Each model type or
182
182
Specify the task using the Unitxt recipe format:
183
183
184
184
* `card`: Use the `name` to specify a Unitxt card or `custom` for a custom card
185
-
** `name`: Specify a Unitxt card from the link:https://www.unitxt.ai/en/latest/catalog/catalog.cards.__dir__.html[Unitxt catalog]. Use the card's ID as the value.
185
+
** `name`: Specify a Unitxt card from the link:++https://www.unitxt.ai/en/latest/catalog/catalog.cards.__dir__.html++[Unitxt catalog]. Use the card's ID as the value.
186
186
For example: The ID of link:https://www.unitxt.ai/en/latest/catalog/catalog.cards.wnli.html[Wnli card] is `cards.wnli`.
187
187
** `custom`: Define a custom card and use it. The value is a JSON string for a custom Unitxt card which contains the custom dataset.
188
188
Use the documentation link:https://www.unitxt.ai/en/latest/docs/adding_dataset.html#adding-to-the-catalog[here] to compose a custom card, store it as a JSON file, and use the JSON content as the value here.
189
189
If the dataset used by the custom card needs an API key from an environment variable or a persistent volume, you have to
190
190
set up corresponding resources under the `pod` field. Check the `pod` field below.
191
-
* `template`: Specify a Unitxt template from the link:https://www.unitxt.ai/en/latest/catalog/catalog.templates.__dir__.html[Unitxt catalog]. Use the template's ID as the value.
192
-
* `task` (optional): Specify a Unitxt task from the link:https://www.unitxt.ai/en/latest/catalog/catalog.cards.__dir__.html[Unitxt catalog]. Use the task's ID as the value.
191
+
* `template`: Specify a Unitxt template from the link:++https://www.unitxt.ai/en/latest/catalog/catalog.templates.__dir__.html++[Unitxt catalog]. Use the template's ID as the value.
192
+
* `task` (optional): Specify a Unitxt task from the link:++https://www.unitxt.ai/en/latest/catalog/catalog.cards.__dir__.html++[Unitxt catalog]. Use the task's ID as the value.
193
193
A Unitxt card has a pre-defined task. Only specify a value for this if you want to run different task.
194
-
* `metrics` (optional): Specify a list of Unitx metrics from the link:https://www.unitxt.ai/en/latest/catalog/catalog.metrics.__dir__.html[Unitxt catalog]. Use the metric's ID as the value.
194
+
* `metrics` (optional): Specify a list of Unitx metrics from the link:++https://www.unitxt.ai/en/latest/catalog/catalog.metrics.__dir__.html++[Unitxt catalog]. Use the metric's ID as the value.
195
195
A Unitxt task has a set of pre-defined metrics. Only specify a set of metrics if you need different metrics.
196
-
* `format` (optional): Specify a Unitxt format from the link:https://www.unitxt.ai/en/latest/catalog/catalog.formats.__dir__.html[Unitxt catalog]. Use the format's ID as the value.
196
+
* `format` (optional): Specify a Unitxt format from the link:++https://www.unitxt.ai/en/latest/catalog/catalog.formats.__dir__.html++[Unitxt catalog]. Use the format's ID as the value.
197
197
* `loaderLimit` (optional): Specifies the maximum number of instances per stream to be returned from the loader (used to reduce loading time in large datasets).
198
198
* `numDemos` (optional): Number of fewshot to be used.
199
199
* `demosPoolSize` (optional): Size of the fewshot pool.
0 commit comments