From b45c3bd5dee759c16c353347b65a896456da2e4e Mon Sep 17 00:00:00 2001 From: Michael Malave Date: Thu, 24 Jul 2025 12:48:34 -0700 Subject: [PATCH 1/3] initial commit to include Heroku as an openai compatible provider --- .../02-providers-and-models.mdx | 4 +- .../45-heroku.mdx | 116 ++++++++++++++++++ .../02-openai-compatible-providers/index.mdx | 1 + 3 files changed, 119 insertions(+), 2 deletions(-) create mode 100644 content/providers/02-openai-compatible-providers/45-heroku.mdx diff --git a/content/docs/02-foundations/02-providers-and-models.mdx b/content/docs/02-foundations/02-providers-and-models.mdx index e7139cdb9c87..5add14b7cc9f 100644 --- a/content/docs/02-foundations/02-providers-and-models.mdx +++ b/content/docs/02-foundations/02-providers-and-models.mdx @@ -9,7 +9,7 @@ Companies such as OpenAI and Anthropic (providers) offer access to a range of la Each provider typically has its own unique method for interfacing with their models, complicating the process of switching providers and increasing the risk of vendor lock-in. -To solve these challenges, AI SDK Core offers a standardized approach to interacting with LLMs through a [language model specification](https://github.com/vercel/ai/tree/main/packages/provider/src/language-model/v2) that abstracts differences between providers. This unified interface allows you to switch between providers with ease while using the same API for all providers. +To solve these challenges, AI SDK Core offers a standardized approach to interacting with LLMs through a [language model specification](https://github.com/vercel/ai/tree/main/packages/provider/src/language-model/v1) that abstracts differences between providers. This unified interface allows you to switch between providers with ease while using the same API for all providers. Here is an overview of the AI SDK Provider Architecture: @@ -53,6 +53,7 @@ You can also use the [OpenAI Compatible provider](/providers/openai-compatible-p - [LM Studio](/providers/openai-compatible-providers/lmstudio) - [Baseten](/providers/openai-compatible-providers/baseten) +- [Heroku](/providers/openai-compatible-providers/heroku) Our [language model specification](https://github.com/vercel/ai/tree/main/packages/provider/src/language-model/v1) is published as an open-source package, which you can use to create [custom providers](/providers/community-providers/custom-providers). @@ -76,7 +77,6 @@ The open-source community has created the following providers: - [Dify Provider](/providers/community-providers/dify) (`dify-ai-provider`) - [Sarvam Provider](/providers/community-providers/sarvam) (`sarvam-ai-provider`) - [Claude Code Provider](/providers/community-providers/claude-code) (`ai-sdk-provider-claude-code`) -- [Built-in AI Provider](/providers/community-providers/built-in-ai) (`built-in-ai`) ## Self-Hosted Models diff --git a/content/providers/02-openai-compatible-providers/45-heroku.mdx b/content/providers/02-openai-compatible-providers/45-heroku.mdx new file mode 100644 index 000000000000..0501004f5df5 --- /dev/null +++ b/content/providers/02-openai-compatible-providers/45-heroku.mdx @@ -0,0 +1,116 @@ +--- +title: Heroku +description: Use a Heroku OpenAI compatible API with the AI SDK. +--- + +# Heroku Provider + +[Heroku](https://heroku.com/) is a cloud platform that allows you to deploy and run applications, including AI models with OpenAI API compatibility. +You can deploy models that are OpenAI API compatible and use them with the AI SDK. + +## Setup + +The Heroku provider is available via the `@ai-sdk/openai-compatible` module as it is compatible with the OpenAI API. +You can install it with + + + + + + + + + + + + + +### Heroku Setup + +1. Create a test app in Heroku: +```bash +heroku create +``` + +2. Inference using claude-3-5-haiku: +```bash +heroku ai:models:create -a $APP_NAME claude-3-5-haiku +``` + +3. Export Variables: +```bash +export INFERENCE_KEY=$(heroku config:get INFERENCE_KEY -a $APP_NAME) +export INFERENCE_MODEL_ID=$(heroku config:get INFERENCE_MODEL_ID -a $APP_NAME) +export INFERENCE_URL=$(heroku config:get INFERENCE_URL -a $APP_NAME) +``` + +## Provider Instance + +To use Heroku, you can create a custom provider instance with the `createOpenAICompatible` function from `@ai-sdk/openai-compatible`: + +```ts +import { createOpenAICompatible } from '@ai-sdk/openai-compatible'; + +const heroku = createOpenAICompatible({ + name: 'heroku', + baseURL: process.env.INFERENCE_URL + '/v1', + apiKey: process.env.INFERENCE_KEY, +}); +``` + +Be sure to have your `INFERENCE_KEY`, `INFERENCE_MODEL_ID`, and `INFERENCE_URL` set in your environment variables. + +## Language Models + +You can create Heroku models using a provider instance. +The first argument is the served model name, e.g. `claude-3-5-haiku`. + +```ts +const model = heroku('claude-3-5-haiku'); +``` + +### Example + +You can use Heroku language models to generate text with the `generateText` function: + +```ts +import { createOpenAICompatible } from '@ai-sdk/openai-compatible'; +import { generateText } from 'ai'; + +const heroku = createOpenAICompatible({ + name: 'heroku', + baseURL: process.env.INFERENCE_URL + '/v1', + apiKey: process.env.INFERENCE_KEY, +}); + +const { text } = await generateText({ + model: heroku('claude-3-5-haiku'), + prompt: 'Tell me about yourself in one sentence', +}); + +console.log(text); +``` + +Heroku language models are also able to generate text in a streaming fashion with the `streamText` function: + +```ts +import { createOpenAICompatible } from '@ai-sdk/openai-compatible'; +import { streamText } from 'ai'; + +const heroku = createOpenAICompatible({ + name: 'heroku', + baseURL: process.env.INFERENCE_URL + '/v1', + apiKey: process.env.INFERENCE_KEY, +}); + +const result = streamText({ + model: heroku('claude-3-5-haiku'), + prompt: 'Tell me about yourself in one sentence', +}); + +for await (const message of result.textStream) { + console.log(message); +} +``` + +Heroku language models can also be used in the `generateObject`, and `streamObject` functions. \ No newline at end of file diff --git a/content/providers/02-openai-compatible-providers/index.mdx b/content/providers/02-openai-compatible-providers/index.mdx index 1269421b7abd..937051ed365b 100644 --- a/content/providers/02-openai-compatible-providers/index.mdx +++ b/content/providers/02-openai-compatible-providers/index.mdx @@ -14,6 +14,7 @@ We provide detailed documentation for the following OpenAI compatible providers: - [LM Studio](/providers/openai-compatible-providers/lmstudio) - [NIM](/providers/openai-compatible-providers/nim) - [Baseten](/providers/openai-compatible-providers/baseten) +- [Heroku](/providers/openai-compatible-providers/heroku) The general setup and provider instance creation is the same for all of these providers. From ce667f1b974b00206832ed0625a9ad64c2974521 Mon Sep 17 00:00:00 2001 From: Michael Malave Date: Thu, 24 Jul 2025 14:10:52 -0700 Subject: [PATCH 2/3] fixed docs with pnpm prettier fix --- .../providers/02-openai-compatible-providers/45-heroku.mdx | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/content/providers/02-openai-compatible-providers/45-heroku.mdx b/content/providers/02-openai-compatible-providers/45-heroku.mdx index 0501004f5df5..2ad7f60641ed 100644 --- a/content/providers/02-openai-compatible-providers/45-heroku.mdx +++ b/content/providers/02-openai-compatible-providers/45-heroku.mdx @@ -28,16 +28,19 @@ You can install it with ### Heroku Setup 1. Create a test app in Heroku: + ```bash heroku create ``` 2. Inference using claude-3-5-haiku: + ```bash heroku ai:models:create -a $APP_NAME claude-3-5-haiku ``` 3. Export Variables: + ```bash export INFERENCE_KEY=$(heroku config:get INFERENCE_KEY -a $APP_NAME) export INFERENCE_MODEL_ID=$(heroku config:get INFERENCE_MODEL_ID -a $APP_NAME) @@ -113,4 +116,4 @@ for await (const message of result.textStream) { } ``` -Heroku language models can also be used in the `generateObject`, and `streamObject` functions. \ No newline at end of file +Heroku language models can also be used in the `generateObject`, and `streamObject` functions. From f8054e778fd8adc93283bceef00476070b113387 Mon Sep 17 00:00:00 2001 From: Timothy Lowrimore Date: Fri, 8 Aug 2025 14:21:30 -0600 Subject: [PATCH 3/3] revert change to api version in docs --- content/docs/02-foundations/02-providers-and-models.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/02-foundations/02-providers-and-models.mdx b/content/docs/02-foundations/02-providers-and-models.mdx index 6213a5bbc33f..29788199a978 100644 --- a/content/docs/02-foundations/02-providers-and-models.mdx +++ b/content/docs/02-foundations/02-providers-and-models.mdx @@ -9,7 +9,7 @@ Companies such as OpenAI and Anthropic (providers) offer access to a range of la Each provider typically has its own unique method for interfacing with their models, complicating the process of switching providers and increasing the risk of vendor lock-in. -To solve these challenges, AI SDK Core offers a standardized approach to interacting with LLMs through a [language model specification](https://github.com/vercel/ai/tree/main/packages/provider/src/language-model/v1) that abstracts differences between providers. This unified interface allows you to switch between providers with ease while using the same API for all providers. +To solve these challenges, AI SDK Core offers a standardized approach to interacting with LLMs through a [language model specification](https://github.com/vercel/ai/tree/main/packages/provider/src/language-model/v2) that abstracts differences between providers. This unified interface allows you to switch between providers with ease while using the same API for all providers. Here is an overview of the AI SDK Provider Architecture: