Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Commit 88f1903

Browse files
authored
Update README.md
1 parent d01b050 commit 88f1903

File tree

1 file changed

+1
-15
lines changed

1 file changed

+1
-15
lines changed

README.md

Lines changed: 1 addition & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -92,21 +92,6 @@ curl http://localhost:3928/v1/chat/completions \
9292
}'
9393
```
9494

95-
***OPTIONAL***: You can constrain the sampling using GBNF grammars by providing path to a grammar file
96-
```bash title="Nitro Inference With Grammar"
97-
curl http://localhost:3928/v1/chat/completions \
98-
-H "Content-Type: application/json" \
99-
-d '{
100-
"messages": [
101-
{
102-
"role": "user",
103-
"content": "Who won the world series in 2020?"
104-
},
105-
],
106-
"grammar_file": "/path/to/grammarfile"
107-
}'
108-
```
109-
11095
Table of parameters
11196

11297
| Parameter | Type | Description |
@@ -128,6 +113,7 @@ Table of parameters
128113
|`grp_attn_n`|Integer|Group attention factor in self-extend|
129114
|`grp_attn_w`|Integer|Group attention width in self-extend|
130115
|`mlock`|Boolean|Prevent system swapping of the model to disk in macOS|
116+
|`grammar_file`| String |You can constrain the sampling using GBNF grammars by providing path to a grammar file|
131117

132118
***OPTIONAL***: You can run Nitro on a different port like 5000 instead of 3928 by running it manually in terminal
133119
```zsh

0 commit comments

Comments
 (0)