|
| 1 | +# LiveKit Bridge |
| 2 | + |
| 3 | +A simplified, high-level C++ wrapper around the [LiveKit C++ SDK](../README.md). The bridge abstracts away room lifecycle management, track creation, publishing, and subscription boilerplate so that external codebases can interface with LiveKit in just a few lines. It is intended that this library will be used to bridge the LiveKit C++ SDK into other SDKs such as, but not limited to, Foxglove, ROS, and Rerun. |
| 4 | + |
| 5 | +It is intended that this library closely matches the style of the core LiveKit C++ SDK. |
| 6 | + |
| 7 | +# Prerequisites |
| 8 | +Since this is an extention of the LiveKit C++ SDK, go through the LiveKit C++ SDK installation instructions first: |
| 9 | +*__[LiveKit C++ SDK](../README.md)__* |
| 10 | + |
| 11 | +## Usage Overview |
| 12 | + |
| 13 | +```cpp |
| 14 | +#include "livekit_bridge/livekit_bridge.h" |
| 15 | +#include "livekit/audio_frame.h" |
| 16 | +#include "livekit/video_frame.h" |
| 17 | +#include "livekit/track.h" |
| 18 | + |
| 19 | +// 1. Connect |
| 20 | +livekit_bridge::LiveKitBridge bridge; |
| 21 | +livekit::RoomOptions options; |
| 22 | +options.auto_subscribe = true; // automatically subscribe to all remote tracks |
| 23 | +options.dynacast = false; |
| 24 | +bridge.connect("wss://my-server.livekit.cloud", token, options); |
| 25 | + |
| 26 | +// 2. Create outgoing tracks (RAII-managed) |
| 27 | +auto mic = bridge.createAudioTrack("mic", 48000, 2, |
| 28 | + livekit::TrackSource::SOURCE_MICROPHONE); // name, sample_rate, channels, source |
| 29 | +auto cam = bridge.createVideoTrack("cam", 1280, 720, |
| 30 | + livekit::TrackSource::SOURCE_CAMERA); // name, width, height, source |
| 31 | + |
| 32 | +// 3. Push frames to remote participants |
| 33 | +mic->pushFrame(pcm_data, samples_per_channel); |
| 34 | +cam->pushFrame(rgba_data, timestamp_us); |
| 35 | + |
| 36 | +// 4. Receive frames from a remote participant |
| 37 | +bridge.setOnAudioFrameCallback("remote-peer", livekit::TrackSource::SOURCE_MICROPHONE, |
| 38 | + [](const livekit::AudioFrame& frame) { |
| 39 | + // Called on a background reader thread |
| 40 | + }); |
| 41 | + |
| 42 | +bridge.setOnVideoFrameCallback("remote-peer", livekit::TrackSource::SOURCE_CAMERA, |
| 43 | + [](const livekit::VideoFrame& frame, int64_t timestamp_us) { |
| 44 | + // Called on a background reader thread |
| 45 | + }); |
| 46 | + |
| 47 | +// 5. Cleanup is automatic (RAII), or explicit: |
| 48 | +mic.reset(); // unpublishes the audio track |
| 49 | +cam.reset(); // unpublishes the video track |
| 50 | +bridge.disconnect(); |
| 51 | +``` |
| 52 | +
|
| 53 | +## Building |
| 54 | +
|
| 55 | +The bridge is a component of the `client-sdk-cpp` build. See the "⚙️ BUILD" section of the [LiveKit C++ SDK README](../README.md) for instructions on how to build the bridge. |
| 56 | +
|
| 57 | +This produces `liblivekit_bridge` (shared library) and optional `robot_stub`, `human_stub`, `robot`, and `human` executables. |
| 58 | +
|
| 59 | +### Using the bridge in your own CMake project |
| 60 | +TODO(sderosa): add instructions on how to use the bridge in your own CMake project. |
| 61 | +
|
| 62 | +## Architecture |
| 63 | +
|
| 64 | +### Data Flow Overview |
| 65 | +
|
| 66 | +``` |
| 67 | +Your Application |
| 68 | + | | |
| 69 | + | pushFrame() -----> BridgeAudioTrack | (sending to remote participants) |
| 70 | + | pushFrame() -----> BridgeVideoTrack | |
| 71 | + | | |
| 72 | + | callback() <------ Reader Thread | (receiving from remote participants) |
| 73 | + | | |
| 74 | + +------- LiveKitBridge -----------------+ |
| 75 | + | |
| 76 | + LiveKit Room |
| 77 | + | |
| 78 | + LiveKit Server |
| 79 | +``` |
| 80 | +
|
| 81 | +### Core Components |
| 82 | +
|
| 83 | +**`LiveKitBridge`** -- The main entry point. Owns the full room lifecycle: SDK initialization, room connection, track publishing, and frame callback management. |
| 84 | +
|
| 85 | +**`BridgeAudioTrack` / `BridgeVideoTrack`** -- RAII handles for published local tracks. Created via `createAudioTrack()` / `createVideoTrack()`. When the `shared_ptr` is dropped, the track is automatically unpublished and all underlying SDK resources are freed. Call `pushFrame()` to send audio/video data to remote participants. |
| 86 | +
|
| 87 | +**`BridgeRoomDelegate`** -- Internal (not part of the public API; lives in `src/`). Listens for `onTrackSubscribed` / `onTrackUnsubscribed` events from the LiveKit SDK and wires up reader threads automatically. |
| 88 | +
|
| 89 | +### What is a Reader? |
| 90 | +
|
| 91 | +A **reader** is a background thread that receives decoded media frames from a remote participant. |
| 92 | +
|
| 93 | +When a remote participant publishes an audio or video track and the bridge subscribes to it (auto-subscribe is enabled by default), the bridge creates an `AudioStream` or `VideoStream` from that track and spins up a dedicated thread. This thread loops on `stream->read()`, which blocks until a new frame arrives. Each received frame is forwarded to the user's registered callback. |
| 94 | +
|
| 95 | +In short: |
| 96 | +
|
| 97 | +- **Sending** (you -> remote): `BridgeAudioTrack::pushFrame()` / `BridgeVideoTrack::pushFrame()` |
| 98 | +- **Receiving** (remote -> you): reader threads invoke your registered callbacks |
| 99 | +
|
| 100 | +Reader threads are managed entirely by the bridge. They are created when a matching remote track is subscribed, and torn down (stream closed, thread joined) when the track is unsubscribed, the callback is unregistered, or `disconnect()` is called. |
| 101 | +
|
| 102 | +### Callback Registration Timing |
| 103 | +
|
| 104 | +Callbacks are keyed by `(participant_identity, track_source)`. You can register them **before** the remote participant has joined the room. The bridge stores the callback and automatically wires it up when the matching track is subscribed. |
| 105 | +
|
| 106 | +> **Note:** Only one callback may be set per `(participant_identity, track_source)` pair. Calling `setOnAudioFrameCallback` or `setOnVideoFrameCallback` again with the same identity and source will silently replace the previous callback. If you need to fan-out a single stream to multiple consumers, do so inside your callback. |
| 107 | +
|
| 108 | +This means the typical pattern is: |
| 109 | +
|
| 110 | +```cpp |
| 111 | +// Register first, connect second -- or register after connect but before |
| 112 | +// the remote participant joins. |
| 113 | +bridge.setOnAudioFrameCallback("robot-1", livekit::TrackSource::SOURCE_MICROPHONE, my_callback); |
| 114 | +livekit::RoomOptions options; |
| 115 | +options.auto_subscribe = true; |
| 116 | +bridge.connect(url, token, options); |
| 117 | +// When robot-1 joins and publishes a mic track, my_callback starts firing. |
| 118 | +``` |
| 119 | + |
| 120 | +### Thread Safety |
| 121 | + |
| 122 | +- `LiveKitBridge` uses a mutex to protect the callback map and active reader state. |
| 123 | +- Frame callbacks fire on background reader threads. If your callback accesses shared application state, you are responsible for synchronization. |
| 124 | +- `disconnect()` closes all streams and joins all reader threads before returning -- it is safe to destroy the bridge immediately after. |
| 125 | + |
| 126 | +## API Reference |
| 127 | + |
| 128 | +### `LiveKitBridge` |
| 129 | + |
| 130 | +| Method | Description | |
| 131 | +|---|---| |
| 132 | +| `connect(url, token, options)` | Connect to a LiveKit room. Initializes the SDK, creates a Room, and connects with auto-subscribe enabled. | |
| 133 | +| `disconnect()` | Disconnect and release all resources. Joins all reader threads. Safe to call multiple times. | |
| 134 | +| `isConnected()` | Returns whether the bridge is currently connected. | |
| 135 | +| `createAudioTrack(name, sample_rate, num_channels, source)` | Create and publish a local audio track with the given `TrackSource` (e.g. `SOURCE_MICROPHONE`, `SOURCE_SCREENSHARE_AUDIO`). Returns an RAII `shared_ptr<BridgeAudioTrack>`. | |
| 136 | +| `createVideoTrack(name, width, height, source)` | Create and publish a local video track with the given `TrackSource` (e.g. `SOURCE_CAMERA`, `SOURCE_SCREENSHARE`). Returns an RAII `shared_ptr<BridgeVideoTrack>`. | |
| 137 | +| `setOnAudioFrameCallback(identity, source, callback)` | Register a callback for audio frames from a specific remote participant + track source. | |
| 138 | +| `setOnVideoFrameCallback(identity, source, callback)` | Register a callback for video frames from a specific remote participant + track source. | |
| 139 | +| `clearOnAudioFrameCallback(identity, source)` | Clear the audio callback for a specific remote participant + track source. Stops and joins the reader thread if active. | |
| 140 | +| `clearOnVideoFrameCallback(identity, source)` | Clear the video callback for a specific remote participant + track source. Stops and joins the reader thread if active. | |
| 141 | + |
| 142 | +### `BridgeAudioTrack` |
| 143 | + |
| 144 | +| Method | Description | |
| 145 | +|---|---| |
| 146 | +| `pushFrame(data, samples_per_channel, timeout_ms)` | Push interleaved int16 PCM samples. Accepts `std::vector<int16_t>` or raw pointer. | |
| 147 | +| `mute()` / `unmute()` | Mute/unmute the track (stops/resumes sending audio). | |
| 148 | +| `release()` | Explicitly unpublish and free resources. Called automatically by the destructor. | |
| 149 | +| `name()` / `sampleRate()` / `numChannels()` | Accessors for track configuration. | |
| 150 | + |
| 151 | +### `BridgeVideoTrack` |
| 152 | + |
| 153 | +| Method | Description | |
| 154 | +|---|---| |
| 155 | +| `pushFrame(data, timestamp_us)` | Push RGBA pixel data. Accepts `std::vector<uint8_t>` or raw pointer + size. | |
| 156 | +| `mute()` / `unmute()` | Mute/unmute the track (stops/resumes sending video). | |
| 157 | +| `release()` | Explicitly unpublish and free resources. Called automatically by the destructor. | |
| 158 | +| `name()` / `width()` / `height()` | Accessors for track configuration. | |
| 159 | + |
| 160 | +## Examples |
| 161 | +- examples/robot.cpp: publishes video and audio from a webcam and microphone. This requires a webcam and microphone to be available. |
| 162 | +- examples/human.cpp: receives and renders video to the screen, receives and plays audio through the speaker. |
| 163 | + |
| 164 | +### Running the examples: |
| 165 | +Note: the following workflow works for both `human` and `robot`. |
| 166 | + |
| 167 | +1. create a `robo_room` |
| 168 | +``` |
| 169 | +lk token create \ |
| 170 | + --join --room robo_room --identity test_user \ |
| 171 | + --valid-for 24h |
| 172 | +``` |
| 173 | + |
| 174 | +2. generate tokens for the robot and human |
| 175 | +``` |
| 176 | +lk token create --api-key <key> --api-secret <secret> \ |
| 177 | + --join --room robo_room --identity robot --valid-for 24h |
| 178 | +
|
| 179 | +lk token create --api-key <key> --api-secret <secret> \ |
| 180 | + --join --room robo_room --identity human --valid-for 24h |
| 181 | +``` |
| 182 | + |
| 183 | +save these tokens as you will need them to run the examples. |
| 184 | + |
| 185 | +3. kick off the robot: |
| 186 | +``` |
| 187 | +export LIVEKIT_URL="wss://your-server.livekit.cloud" |
| 188 | +export LIVEKIT_TOKEN=<token> |
| 189 | +./build-release/bin/robot_stub |
| 190 | +``` |
| 191 | + |
| 192 | +4. kick off the human (in a new terminal): |
| 193 | +``` |
| 194 | +export LIVEKIT_URL="wss://your-server.livekit.cloud" |
| 195 | +export LIVEKIT_TOKEN=<token> |
| 196 | +./build-release/bin/human |
| 197 | +``` |
| 198 | + |
| 199 | +The human will print periodic summaries like: |
| 200 | + |
| 201 | +``` |
| 202 | +[human] Audio frame #1: 480 samples/ch, 48000 Hz, 1 ch, duration=0.010s |
| 203 | +[human] Video frame #1: 640x480, 1228800 bytes, ts=0 us |
| 204 | +[human] Status: 500 audio frames, 150 video frames received so far. |
| 205 | +``` |
| 206 | + |
| 207 | +## Testing |
| 208 | + |
| 209 | +The bridge includes a unit test suite built with [Google Test](https://github.com/google/googletest). Tests cover |
| 210 | +1. `CallbackKey` hashing/equality, |
| 211 | +2. `BridgeAudioTrack`/`BridgeVideoTrack` state management, and |
| 212 | +3. `LiveKitBridge` pre-connection behaviour (callback registration, error handling). |
| 213 | + |
| 214 | +### Building and running tests |
| 215 | + |
| 216 | +Bridge tests are automatically included when you build with the `debug-tests` or `release-tests` command: |
| 217 | + |
| 218 | +```bash |
| 219 | +./build.sh debug-tests |
| 220 | +``` |
| 221 | + |
| 222 | +Then run them directly: |
| 223 | + |
| 224 | +```bash |
| 225 | +./build-debug/bin/livekit_bridge_tests |
| 226 | +``` |
| 227 | + |
| 228 | +### Standalone bridge tests only |
| 229 | + |
| 230 | +If you want to build bridge tests independently (without the parent SDK tests), set `LIVEKIT_BRIDGE_BUILD_TESTS=ON`: |
| 231 | + |
| 232 | +```bash |
| 233 | +cmake --preset macos-debug -DLIVEKIT_BRIDGE_BUILD_TESTS=ON |
| 234 | +cmake --build build-debug --target livekit_bridge_tests |
| 235 | +``` |
| 236 | + |
| 237 | +## Limitations |
| 238 | + |
| 239 | +The bridge is designed for simplicity and currently only supports limited audio and video features. It does not expose: |
| 240 | + |
| 241 | +- We dont support all events defined in the RoomDelegate interface. |
| 242 | +- E2EE configuration |
| 243 | +- RPC / data channels / data tracks |
| 244 | +- Simulcast tuning |
| 245 | +- Video format selection (RGBA is the default; no format option yet) |
| 246 | +- Custom `RoomOptions` or `TrackPublishOptions` |
| 247 | +- **One callback per (participant, source):** Only a single callback can be registered for each `(participant_identity, track_source)` pair. Re-registering with the same key silently replaces the previous callback. To fan-out a stream to multiple consumers, dispatch from within your single callback. |
| 248 | + |
| 249 | +For advanced use cases, use the full `client-sdk-cpp` API directly, or expand the bridge to support your use case. |
0 commit comments