Skip to content

Commit 56af7ab

Browse files
authored
fix bleurt docs (#708)
1 parent c07bbe7 commit 56af7ab

2 files changed

Lines changed: 13 additions & 7 deletions

File tree

metrics/bleurt/README.md

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -42,9 +42,15 @@ This metric takes as input lists of predicted sentences and reference sentences:
4242
```
4343

4444
### Inputs
45+
46+
For the `load` function:
47+
48+
- **config_name** (`str`): BLEURT checkpoint. Will default to `"bleurt-base-128"` if not specified. Other models that can be chosen are: `"bleurt-tiny-128"`, `"bleurt-tiny-512"`, `"bleurt-base-128"`, `"bleurt-base-512"`, `"bleurt-large-128"`, `"bleurt-large-512"`, `"BLEURT-20-D3"`, `"BLEURT-20-D6"`, `"BLEURT-20-D12"` and `"BLEURT-20"`.
49+
50+
For the `compute` function:
51+
4552
- **predictions** (`list` of `str`s): List of generated sentences to score.
4653
- **references** (`list` of `str`s): List of references to compare to.
47-
- **checkpoint** (`str`): BLEURT checkpoint. Will default to `BLEURT-tiny` if not specified. Other models that can be chosen are: `"bleurt-tiny-128"`, `"bleurt-tiny-512"`, `"bleurt-base-128"`, `"bleurt-base-512"`, `"bleurt-large-128"`, `"bleurt-large-512"`, `"BLEURT-20-D3"`, `"BLEURT-20-D6"`, `"BLEURT-20-D12"` and `"BLEURT-20"`.
4854

4955
### Output Values
5056
- **scores** : a `list` of scores, one per prediction.
@@ -65,7 +71,7 @@ BLEURT is used to compare models across different asks (e.g. (Table to text gene
6571

6672
### Examples
6773

68-
Example with the default model:
74+
Example with the default model (`"bleurt-base-128"`):
6975
```python
7076
>>> predictions = ["hello there", "general kenobi"]
7177
>>> references = ["hello there", "general kenobi"]
@@ -75,14 +81,14 @@ Example with the default model:
7581
{'scores': [1.0295498371124268, 1.0445425510406494]}
7682
```
7783

78-
Example with the `"bleurt-base-128"` model checkpoint:
84+
Example with the full `"BLEURT-20"` model checkpoint:
7985
```python
8086
>>> predictions = ["hello there", "general kenobi"]
8187
>>> references = ["hello there", "general kenobi"]
82-
>>> bleurt = load("bleurt", module_type="metric", checkpoint="bleurt-base-128")
88+
>>> bleurt = load("bleurt", module_type="metric", config_name="BLEURT-20")
8389
>>> results = bleurt.compute(predictions=predictions, references=references)
8490
>>> print(results)
85-
{'scores': [1.0295498371124268, 1.0445425510406494]}
91+
{'scores': [1.015415906906128, 0.9985226988792419]}
8692
```
8793

8894
## Limitations and Bias

metrics/bleurt/bleurt.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -100,8 +100,8 @@ def _download_and_prepare(self, dl_manager):
100100
# check that config name specifies a valid BLEURT model
101101
if self.config_name == "default":
102102
logger.warning(
103-
"Using default BLEURT-Base checkpoint for sequence maximum length 128. "
104-
"You can use a bigger model for better results with e.g.: evaluate.load('bleurt', 'bleurt-large-512')."
103+
"Using default checkpoint 'bleurt-base-128' for sequence maximum length 128. "
104+
"You can use a bigger model for better results with e.g.: evaluate.load('bleurt', config_name='bleurt-large-512')."
105105
)
106106
self.config_name = "bleurt-base-128"
107107

0 commit comments

Comments
 (0)