A2v sound encoder inference#39

Merged

lfengad merged 27 commits into

mainfrom

a2v-sound-encoder-inference

Jun 13, 2026

Commits on Jun 12, 2026

Add design spec for audio_image2video (A2V) inference

lfengad
and
claude
committed
Add implementation plan for audio_image2video (A2V) inference

lfengad
and
claude
committed
Plan: resample via scipy not torchaudio (absent from inference container)

lfengad
and
claude
committed
Register Cosmos3-Nano-SoundEncoder checkpoint

lfengad
and
claude
committed
Add audio_image2video model mode

lfengad
and
claude
committed
Add sound_path input + audio_image2video arg validation

lfengad
and
claude
committed
Fix sound_path override type and test fixtures

lfengad
and
claude
committed
Add load_conditioning_audio helper

lfengad
and
claude
committed
Support sound conditioning (ts2v) in inject_sound_into_batch

lfengad
and
claude
committed
Load and condition on real input audio in get_sample_data

lfengad
and
claude
committed
Add audio_image2video (a2v) example input and conditioning audio

lfengad
and
claude
committed
Document audio_image2video mode and sound-encoder checkpoint

lfengad
and
claude
committed
Load full AVAE (encoder+decoder) for sound-conditioned A2V inference

lfengad
and
claude
committed
Use Cosmos3-Nano (not the experimental repo) for sound-conditioned A2V

lfengad
and
claude
committed
Remove A2V implementation plan doc

lfengad
and
claude
committed
Remove A2V example input (a2v.json + conditioning audio asset)

lfengad
and
claude
committed
Drop dead a2v.json links from inference docs, keep audio_image2video documented

lfengad
and
claude
committed
Clarify AVAE comments: sound_tokenizer is decoder-only until updated

lfengad
and
claude
committed
Simplify AVAE materialize comments

lfengad
and
claude
committed
Source AVAE sound_tokenizer from the loaded checkpoint's bundled sound_tokenizer/

lfengad
and
claude
committed
Add sound_tokenizer 'from_checkpoint' control (inference-only)

lfengad
and
claude
committed
Make AVAE sound_tokenizer registered-first, bundled fallback

lfengad
and
claude
committed
Simplify AVAE source check to avae_path presence

lfengad
and
claude
committed
Simplify comments in A2V sound inference

lfengad
and
claude
committed
Revert docs/inference.md changes on this branch

lfengad
and
claude
committed
Merge branch 'main' into a2v-sound-encoder-inference
lfengad
authored

Commits on Jun 13, 2026

Merge branch 'main' into a2v-sound-encoder-inference
lfengad
authored

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

A2v sound encoder inference#39

A2v sound encoder inference#39
lfengad merged 27 commits into
mainfrom
a2v-sound-encoder-inference

Commits on Jun 12, 2026

Add design spec for audio_image2video (A2V) inference

Add implementation plan for audio_image2video (A2V) inference

Plan: resample via scipy not torchaudio (absent from inference container)

Register Cosmos3-Nano-SoundEncoder checkpoint

Add audio_image2video model mode

Add sound_path input + audio_image2video arg validation

Fix sound_path override type and test fixtures

Add load_conditioning_audio helper

Support sound conditioning (ts2v) in inject_sound_into_batch

Load and condition on real input audio in get_sample_data

Add audio_image2video (a2v) example input and conditioning audio

Document audio_image2video mode and sound-encoder checkpoint

Load full AVAE (encoder+decoder) for sound-conditioned A2V inference

Use Cosmos3-Nano (not the experimental repo) for sound-conditioned A2V

Remove A2V implementation plan doc

Remove A2V example input (a2v.json + conditioning audio asset)

Drop dead a2v.json links from inference docs, keep audio_image2video documented

Clarify AVAE comments: sound_tokenizer is decoder-only until updated

Simplify AVAE materialize comments

Source AVAE sound_tokenizer from the loaded checkpoint's bundled sound_tokenizer/

Add sound_tokenizer 'from_checkpoint' control (inference-only)

Make AVAE sound_tokenizer registered-first, bundled fallback

Simplify AVAE source check to avae_path presence

Simplify comments in A2V sound inference

Revert docs/inference.md changes on this branch

Merge branch 'main' into a2v-sound-encoder-inference

Commits on Jun 13, 2026

Merge branch 'main' into a2v-sound-encoder-inference