Skip to content

Minimal set of changes for allowing faster than realtime recording (non live)#1192

Merged
milos-lk merged 11 commits into
mainfrom
replay-async-export-gst-no-testfeeder
Apr 20, 2026
Merged

Minimal set of changes for allowing faster than realtime recording (non live)#1192
milos-lk merged 11 commits into
mainfrom
replay-async-export-gst-no-testfeeder

Conversation

@milos-lk
Copy link
Copy Markdown
Contributor

This branch generalizes the gstreamer pipeline to support non-live sources (replay/export) in addition to the existing live capture flow. Historically the pipeline assumed every source produced data in real time, which baked several implicit behaviors into the build: leaky queues to shed overruns, is-live=true appsrc properties, an asynchronous file sink, and a single New() constructor that owned source creation. Those assumptions break down the moment a source pushes buffers faster than wall-clock, so the changes here rework the pipeline to be driven by an explicit Live flag on PipelineConfig instead.
The centerpiece is the new Live bool field, set to true by default in both pipeline config constructors. The pre-existing IsReplay boolean is replaced with an IsReplay() method derived from !Live, which keeps replay-specific integration points (IPC calls into the service, storage access) working without conflating them with generic pipeline concerns like queue leakiness or backpressure.

Downstream of that flag, audio and video bins now build blocking queues when Live is false so backpressure actually throttles the source, the appsrc elements toggle is-live to match the pipeline mode and add block=true in non-live mode, and the file sink sets async=false to disable preroll for non-live runs — there's no real-time clock to sync against, so making the sink wait for PAUSED→PLAYING preroll-complete serves no purpose and just stalls the state transition . The previously named buildLeakyVideoQueue is renamed to buildVideoQueue to reflect that leakiness is now a property of the mode, not the call site.

On the output side, updateOutputs rejects stream outputs (RTMP, WebSocket) for non-live pipelines upfront, since those destinations cannot ingest faster than 1x playback speed and would otherwise fail deep inside the pipeline.

The controller is refactored to decouple source construction from pipeline construction. The existing New() keeps its behavior, but the shared setup is extracted into newController() and a new NewWithSource() constructor accepts a pre-built source which is an interface, implementors might choose to build for different kind of sources.

Original PR draft with test sources feeding data as fast as possible: #1182

@milos-lk milos-lk requested a review from a team as a code owner April 17, 2026 12:42
Comment thread pkg/pipeline/controller.go Outdated
// the source is constructed externally (e.g. TestSource for non-live pipeline
// testing, or ReplaySource for offline export). The source must have already
// populated the PipelineConfig with track information before calling this.
func NewWithSource(ctx context.Context, conf *config.PipelineConfig, src source.Source) (*Controller, error) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can New be implemented, using NewWithSource? There is quite a bit of code duplication currently.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes - the only reason they got duplicated is because source creation needed controller callbacks for New - but if we apply inversion control that's no longer an issue and these could be compacted. Updated.

@milos-lk milos-lk merged commit f374c79 into main Apr 20, 2026
33 of 36 checks passed
@milos-lk milos-lk deleted the replay-async-export-gst-no-testfeeder branch April 20, 2026 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants