Skip to content

[Weave] OpenAI rRaltime audio integration#2282

Open
dbrian57 wants to merge 1 commit intomainfrom
weave/realtime-audio
Open

[Weave] OpenAI rRaltime audio integration#2282
dbrian57 wants to merge 1 commit intomainfrom
weave/realtime-audio

Conversation

@dbrian57
Copy link
Contributor

@dbrian57 dbrian57 commented Mar 13, 2026

Description

Resolves DOCS-1186. Documents how to integrate with with OpenAI's Realtime API and capture audio from speech-to-speech interactions.

@github-actions
Copy link
Contributor

📚 Mintlify Preview Links

🔗 View Full Preview

✨ Added (1 total)

📄 Pages (1)

File Preview
weave/guides/integrations/openai-realtime-audio.mdx Openai Realtime Audio

📝 Changed (1 total)

⚙️ Other (1)
File
docs.json

🤖 Generated automatically when Mintlify deployment succeeds
📍 Deployment: 68dba7f at 2026-03-13 20:44:37 UTC

@github-actions
Copy link
Contributor

🔗 Link Checker Results

All links are valid!

No broken links were detected.

Checked against: https://wb-21fd5541-weave-realtime-audio.mintlify.app

Copy link

@zbirenbaum zbirenbaum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving since repo transfers are sometimes difficult but I think the long term state should be a bit simpler

pip install weave openai-agents websockets pyaudio numpy
```

1. Create a file titled `weave_voice_assistant.py` and add the following code to it.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be helpful to link to the full repo. The page has a ton of content and requires a lot of copy paste.
git clone <repo>
uv sync
PROJECT_ID={entity}/{project} uv run <example>
is all we really need
Its good for now I guess but lets make a ticket to update after we transfer ownership of that repo

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before we link to the repo, we might need to clean it up.

For example, this step seems a little clunky, and is more useful to someone who is developing weave than to someone who is simply consuming it.

Image

It's also not obvious to me that the symlink method still works. I was running into some issues with SSL and dev-mtsaas the other day, and the fix might have involved getting rid of this and just installing the weave package.

1. Start your Python environment and install the following libraries:

```bash
pip install weave openai-agents websockets pyaudio numpy

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally have a tab that shows uv instructions and python instructions with uv the default

The highlighted lines indicate Weave's integration in the application. The rest of the code creates the basic voice assistant app.

<Accordion title="weave_voice_assistant.py">
```python lines {8,11,14,51-55}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there something wrong with the ```? The preview in github looks crazy

Image

pip install weave openai-agents websockets pyaudio numpy
```

1. Create a file titled `weave_voice_assistant.py` and add the following code to it.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before we link to the repo, we might need to clean it up.

For example, this step seems a little clunky, and is more useful to someone who is developing weave than to someone who is simply consuming it.

Image

It's also not obvious to me that the symlink method still works. I was running into some issues with SSL and dev-mtsaas the other day, and the fix might have involved getting rid of this and just installing the weave package.

python weave_voice_assistant.py
```

Once running, press **T** on your keyboard to mute or unmute the mic. The assistant uses server-side voice activity detection to handle turn-taking and interruptions.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm on the fence about whether the push-to-talk feature should be present.

Why add it?

  • It's useful for testing without headphones
  • It's useful for tech demos

Why not?

  • It adds complexity to the code.
  • If the user has headphones and is not building a demo, it's not needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants