Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Commit 79acc35

Browse files
tikikuntikikun
authored
Merge pull request #177 from janhq/feat_example_nitro_openai_sdk
feat(example): Add openai integration with nitro
2 parents 7f89c16 + 3f432e6 commit 79acc35

File tree

3 files changed

+441
-1
lines changed

3 files changed

+441
-1
lines changed

‎docs/docs/examples/openai-node.md‎

Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
---
2+
title: Nitro with openai-node
3+
---
4+
5+
You can migrate from OAI API or Azure OpenAI to Nitro using your existing NodeJS code quickly
6+
> The ONLY thing you need to do is to override `baseURL` in `openai` init with `Nitro` URL
7+
- NodeJS OpenAI SDK: https://www.npmjs.com/package/openai
8+
9+
## Chat Completion
10+
<table>
11+
<tr>
12+
<td> Engine </td> <td> Typescript Code </td>
13+
</tr>
14+
<tr>
15+
<td> Nitro </td>
16+
<td>
17+
18+
```typescript
19+
import OpenAI from 'openai';
20+
21+
const openai = new OpenAI({
22+
apiKey: '', // defaults to process.env["OPENAI_API_KEY"]
23+
baseURL: "http://localhost:3928/v1/" // https://api.openai.com/v1
24+
});
25+
26+
async function chatCompletion() {
27+
const stream = await openai.beta.chat.completions.stream({
28+
model: 'gpt-3.5-turbo',
29+
messages: [{ role: 'user', content: 'Say this is a test' }],
30+
stream: true,
31+
});
32+
33+
stream.on('content', (delta, snapshot) => {
34+
process.stdout.write(delta);
35+
});
36+
37+
for await (const chunk of stream) {
38+
process.stdout.write(chunk.choices[0]?.delta?.content || '');
39+
}
40+
41+
const chatCompletion = await stream.finalChatCompletion();
42+
console.log(chatCompletion); // {id: "…", choices: […], …}
43+
}
44+
chatCompletion()
45+
```
46+
</td>
47+
</tr>
48+
<tr>
49+
<td> OAI </td>
50+
<td>
51+
52+
```typescript
53+
import OpenAI from 'openai';
54+
55+
const openai = new OpenAI({
56+
apiKey: '', // defaults to process.env["OPENAI_API_KEY"]
57+
});
58+
59+
async function chatCompletion() {
60+
const stream = await openai.beta.chat.completions.stream({
61+
model: 'gpt-3.5-turbo',
62+
messages: [{ role: 'user', content: 'Say this is a test' }],
63+
stream: true,
64+
});
65+
66+
stream.on('content', (delta, snapshot) => {
67+
process.stdout.write(delta);
68+
});
69+
70+
for await (const chunk of stream) {
71+
process.stdout.write(chunk.choices[0]?.delta?.content || '');
72+
}
73+
74+
const chatCompletion = await stream.finalChatCompletion();
75+
console.log(chatCompletion); // {id: "…", choices: […], …}
76+
}
77+
chatCompletion()
78+
```
79+
80+
</td>
81+
</tr>
82+
<tr>
83+
<td> Azure OAI </td>
84+
<td>
85+
86+
```typescript
87+
import OpenAI from 'openai';
88+
// The name of your Azure OpenAI Resource.
89+
// https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource
90+
const resource = '<your resource name>';
91+
92+
// Corresponds to your Model deployment within your OpenAI resource, e.g. my-gpt35-16k-deployment
93+
// Navigate to the Azure OpenAI Studio to deploy a model.
94+
const model = '<your model>';
95+
96+
// https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#rest-api-versioning
97+
const apiVersion = '2023-06-01-preview';
98+
99+
const apiKey = process.env['AZURE_OPENAI_API_KEY'];
100+
if (!apiKey) {
101+
throw new Error('The AZURE_OPENAI_API_KEY environment variable is missing or empty.');
102+
}
103+
104+
const openai = new OpenAI({
105+
apiKey,
106+
baseURL: `https://${resource}.openai.azure.com/openai/deployments/${model}`,
107+
defaultQuery: { 'api-version': apiVersion },
108+
defaultHeaders: { 'api-key': apiKey },
109+
});
110+
111+
async function chatCompletion() {
112+
const stream = await openai.beta.chat.completions.stream({
113+
model: 'gpt-3.5-turbo',
114+
messages: [{ role: 'user', content: 'Say this is a test' }],
115+
stream: true,
116+
});
117+
118+
stream.on('content', (delta, snapshot) => {
119+
process.stdout.write(delta);
120+
});
121+
122+
for await (const chunk of stream) {
123+
process.stdout.write(chunk.choices[0]?.delta?.content || '');
124+
}
125+
126+
const chatCompletion = await stream.finalChatCompletion();
127+
console.log(chatCompletion); // {id: "…", choices: […], …}
128+
}
129+
chatCompletion()
130+
```
131+
132+
</td>
133+
</tr>
134+
</table>
135+
136+
## Embedding
137+
<table>
138+
<tr>
139+
<td> Engine </td> <td> Embedding </td>
140+
</tr>
141+
<tr>
142+
<td> Nitro </td>
143+
<td>
144+
145+
```typescript
146+
import OpenAI from 'openai';
147+
148+
const openai = new OpenAI({
149+
apiKey: '', // defaults to process.env["OPENAI_API_KEY"]
150+
baseURL: "http://localhost:3928/v1/" // https://api.openai.com/v1
151+
});
152+
153+
async function embedding() {
154+
const embedding = await openai.embeddings.create({input: 'Hello How are you?', model: 'text-embedding-ada-002'});
155+
console.log(embedding); // {object: "list", data: […], …}
156+
}
157+
158+
chatCompletion();
159+
```
160+
</td>
161+
</tr>
162+
<tr>
163+
<td> OAI </td>
164+
<td>
165+
166+
```typescript
167+
import OpenAI from 'openai';
168+
169+
const openai = new OpenAI({
170+
apiKey: '', // defaults to process.env["OPENAI_API_KEY"]
171+
});
172+
173+
async function embedding() {
174+
const embedding = await openai.embeddings.create({input: 'Hello How are you?', model: 'text-embedding-ada-002'});
175+
console.log(embedding); // {object: "list", data: […], …}
176+
}
177+
178+
chatCompletion();
179+
```
180+
181+
</td>
182+
</tr>
183+
<tr>
184+
<td> Azure OAI </td>
185+
<td>
186+
187+
```typescript
188+
import OpenAI from 'openai';
189+
// The name of your Azure OpenAI Resource.
190+
// https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource
191+
const resource = '<your resource name>';
192+
193+
// Corresponds to your Model deployment within your OpenAI resource, e.g. my-gpt35-16k-deployment
194+
// Navigate to the Azure OpenAI Studio to deploy a model.
195+
const model = '<your model>';
196+
197+
// https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#rest-api-versioning
198+
const apiVersion = '2023-06-01-preview';
199+
200+
const apiKey = process.env['AZURE_OPENAI_API_KEY'];
201+
if (!apiKey) {
202+
throw new Error('The AZURE_OPENAI_API_KEY environment variable is missing or empty.');
203+
}
204+
205+
const openai = new OpenAI({
206+
apiKey,
207+
baseURL: `https://${resource}.openai.azure.com/openai/deployments/${model}`,
208+
defaultQuery: { 'api-version': apiVersion },
209+
defaultHeaders: { 'api-key': apiKey },
210+
});
211+
212+
async function embedding() {
213+
const embedding = await openai.embeddings.create({input: 'Hello How are you?', model: 'text-embedding-ada-002'});
214+
console.log(embedding); // {object: "list", data: […], …}
215+
}
216+
217+
chatCompletion();
218+
```
219+
220+
</td>
221+
</tr>
222+
</table>
223+
224+
## Audio
225+
Coming soon
226+
227+
## How to reproduce
228+
1. Step 1: Dependencies installation
229+
```
230+
npm install --save openai typescript
231+
# or
232+
yarn add openai
233+
```
234+
2. Step 2: Fill `tsconfig.json`
235+
```json
236+
{
237+
"compilerOptions": {
238+
"moduleResolution": "node",
239+
"sourceMap": true,
240+
"outDir": "dist",
241+
"target": "es2020",
242+
"lib": ["es2020"],
243+
"module": "commonjs",
244+
},
245+
"lib": ["es2015"]
246+
}
247+
```
248+
3. Step 3: Fill `index.ts` file with code
249+
3. Step 4: Build with `npx tsc`
250+
4. Step 5: Run the code with `node dist/index.js`
251+
5. Step 6: Enjoy!

0 commit comments

Comments
 (0)