Skip to content

Commit c51db08

Browse files
authored
feat: push or pull audio frames with a headless audio device module (#201)
* feat: implement HeadlessAudioDeviceModule for playout-only audio handling * docs: add Headless Audio Device Module documentation * refactor: simplify HeadlessADMIntegrationTest * refactor: improve synchronization in HeadlessADMIntegrationTest * refactor: separate AudioDeviceModule into base class * feat: enhance HeadlessAudioDeviceModule with recording capabilities * test: add tests for recording device functionality in HeadlessADM * docs: add recording capabilities to HeadlessAudioDeviceModule documentation * test: enhance HeadlessADMIntegrationTest with CLOSED state handling * docs: update HeadlessAudioDeviceModule description * test: HeadlessADMIntegrationTest to use a single instance of HeadlessAudioDeviceModule
1 parent a5cdb18 commit c51db08

22 files changed

+2495
-844
lines changed

.github/actions/prepare-linux/action.yml

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -31,20 +31,14 @@ runs:
3131
- name: Install required packages
3232
run: |
3333
sudo apt update
34-
sudo apt install -y binutils cmake git locales lsb-release ninja-build pkg-config python3 python3-setuptools rsync unzip wget xz-utils
34+
sudo apt install -y binutils cmake git locales lsb-release ninja-build pipewire pipewire-pulse pkg-config python3 python3-setuptools rsync unzip wget xz-utils
3535
3636
# Chromium Clang to be used with the clang toolchain file
3737
#curl -s https://raw.githubusercontent.com/chromium/chromium/main/tools/clang/scripts/update.py | python3 - --output-dir=/opt/clang
3838
# Use a more stable version of Clang
3939
sudo mkdir -p /opt/clang
4040
wget https://commondatastorage.googleapis.com/chromium-browser-clang/Linux_x64/clang-llvmorg-20-init-9764-gb81d8e90-72.tar.xz
4141
sudo tar -xvf clang-llvmorg-20-init-9764-gb81d8e90-72.tar.xz -C /opt/clang
42-
43-
# Required for testing
44-
#pulseaudio --start
45-
sudo apt install -y pipewire pipewire-pulse gstreamer1.0-pipewire libspa-0.2-bluetooth libspa-0.2-jack pipewire-audio-client-libraries
46-
systemctl --user daemon-reload
47-
systemctl --user --now enable pipewire{,-pulse}.{socket,service}
4842
shell: bash
4943

5044
- name: Install required packages for x86-64

docs/_sidebar.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
- [Audio Device Selection](guide/audio_devices.md)
1010
- [Audio Processing](guide/audio_processing.md)
1111
- [Custom Audio Source](guide/custom_audio_source.md)
12+
- [Headless Audio](guide/headless_audio_device_module.md)
1213
- [Bitrate and Framerate Constraints](guide/constraints.md)
1314
- [Camera Capture](guide/camera_capture.md)
1415
- [Desktop Capture](guide/desktop_capture.md)
Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
# Headless Audio Device Module
2+
3+
The `HeadlessAudioDeviceModule` is a convenience implementation of the `AudioDeviceModule` that uses WebRTC's dummy audio layer. It avoids touching real OS audio devices while still enabling the WebRTC audio pipeline to pull and render audio frames (headless playout) and to simulate capture (recording path).
4+
5+
This is ideal for:
6+
- Server-side or CI environments without audio hardware
7+
- Automated tests where deterministic, no-op audio IO is desired
8+
- Receive-only applications that should render audio via the WebRTC pipeline without producing audible output
9+
- Applications that implement custom audio ingestion but do not want to interact with system devices
10+
11+
## Key characteristics
12+
- Uses dummy audio drivers; no real system devices are opened
13+
- Exposes at least one dummy playout and recording device to allow initialization
14+
- Supports playout and recording initialization and start/stop lifecycle
15+
- Intended primarily for headless scenarios where you want the WebRTC audio pipelines to run without touching physical devices
16+
17+
---
18+
19+
## Playout path
20+
21+
Create the module and pass it to the PeerConnectionFactory. This ensures your peer connection stack uses a headless (dummy) audio backend.
22+
23+
```java
24+
import dev.onvoid.webrtc.PeerConnectionFactory;
25+
import dev.onvoid.webrtc.media.audio.HeadlessAudioDeviceModule;
26+
27+
// Create the headless ADM
28+
HeadlessAudioDeviceModule audioModule = new HeadlessAudioDeviceModule();
29+
30+
// Initialize and start playout
31+
audioModule.initPlayout();
32+
audioModule.startPlayout();
33+
34+
// Create a factory that uses the headless ADM
35+
PeerConnectionFactory factory = new PeerConnectionFactory(audioModule);
36+
37+
// ... use the factory to build peer connections ...
38+
39+
// Cleanup
40+
try {
41+
audioModule.stopPlayout();
42+
}
43+
catch (Throwable e) {
44+
// Ignore errors during stopPlayout()
45+
}
46+
finally {
47+
audioModule.dispose();
48+
factory.dispose();
49+
}
50+
```
51+
52+
Notes:
53+
- Calling startPlayout without a prior initPlayout will throw an error. Always call initPlayout first.
54+
- If you only need the audio pipeline to be ready when remote audio arrives, you may delay playout initialization until after creating your RTCPeerConnection.
55+
56+
---
57+
58+
## Recording path (capture)
59+
60+
The headless module also implements a recording path that simulates a microphone. When started, it periodically pulls 10 ms of PCM from the registered AudioTransport (your Java audio source) and feeds it into WebRTC’s capture pipeline. This is particularly useful in tests or server-side senders.
61+
62+
Typical steps:
63+
64+
```java
65+
HeadlessAudioDeviceModule adm = new HeadlessAudioDeviceModule();
66+
67+
// Initialize and start the recording pipeline (capture)
68+
adm.initRecording();
69+
adm.startRecording();
70+
71+
PeerConnectionFactory factory = new PeerConnectionFactory(adm);
72+
73+
// Use a custom or built-in AudioSource to provide audio frames
74+
CustomAudioSource source = new CustomAudioSource();
75+
AudioTrack senderTrack = factory.createAudioTrack("audio0", source);
76+
peerConnection.addTrack(senderTrack, Collections.singletonList("stream0"));
77+
78+
// Push PCM frames into the CustomAudioSource (10 ms chunks work well)
79+
byte[] pcm = new byte[480 /* frames */ * 2 /* ch */ * 2 /* bytes */];
80+
source.pushAudio(pcm, 16, 48000, 2, 480);
81+
82+
// ... later, stop
83+
adm.stopRecording();
84+
adm.dispose();
85+
factory.dispose();
86+
```
87+
88+
Details:
89+
- Initialization order matters: call `initRecording()` before `startRecording()`.
90+
- The module exposes one virtual recording device; selection calls succeed with index 0.
91+
- Stereo can be enabled/disabled via the standard ADM methods; by default 1 channel is used.
92+
- If no AudioTransport is registered (no source), silence is injected to keep timings consistent.
93+
94+
---
95+
96+
## When to use HeadlessAudioDeviceModule vs. dummy audio layer on AudioDeviceModule
97+
98+
- Prefer `HeadlessAudioDeviceModule` when you need to receive remote audio frames in a headless environment and consume them via `AudioTrack.addSink(AudioSink)`, or when you need to send audio from a custom source without touching physical devices. The headless module drives both playout and recording pipelines while no real system audio device is opened.
99+
- Using a standard `AudioDeviceModule` with `AudioLayer.kDummyAudio` disables actual audio I/O; the audio pipeline is not started for playout and sinks will typically not receive audio frame callbacks. Use this only when you intentionally do not want any audio delivery (e.g., video‑only or fully custom audio).
100+
101+
Related guides:
102+
- [Audio Device Selection](audio_devices.md)
103+
- [Custom Audio Source](custom_audio_source.md)
104+
105+
---
106+
107+
## Limitations and notes
108+
- No real audio is played or captured; playout frames are pulled from the render pipeline and discarded, and capture frames are pulled from your source (or zeroed) and delivered into WebRTC.
109+
- Always follow the lifecycles: `initPlayout()` before `startPlayout()`, and `initRecording()` before `startRecording()`. Stop before dispose.
110+
- The library handles native loading internally; instantiate and use the module as shown above.

docs/guide/overview.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ This section provides detailed guides for various features of the webrtc-java li
88
- [Audio Device Selection](guide/audio_devices.md) - Selecting and configuring audio devices
99
- [Audio Processing](guide/audio_processing.md) - Voice processing components
1010
- [Custom Audio Source](guide/custom_audio_source.md) - Using custom audio sources with WebRTC
11+
- [Headless Audio](guide/headless_audio_device_module.md) - Playout pull without touching real OS audio devices
1112
- [Bitrate and Framerate Constraints](guide/constraints.md) - Controlling media quality
1213
- [Camera Capture](guide/camera_capture.md) - Capturing video from cameras
1314
- [Desktop Capture](guide/desktop_capture.md) - Capturing and sharing screens and windows

webrtc-jni/src/main/cpp/include/JNI_AudioDeviceModule.h

Lines changed: 1 addition & 217 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)