Audio or video, but not both?

[gdhgdhgdh]

Hello - thanks for this, it looks really exciting 👍

I can get this working by changing the value of map from 0 to 1 - this means the output stream contains EITHER video OR audio, but not both :(

#!/bin/bash

ffmpeg -thread_queue_size 32 \
        -f v4l2 -video_size 640x360 -framerate 30 -i /dev/video0 \
        -f alsa -i plughw:0,1 -map 0 \
        -r 30 \
        -c:v libx264 -b:v 1m -preset fast -pix_fmt yuv420p -tune zerolatency -x264-params keyint=30:min-keyint=30\
        -c:a aac -ac 2 -b:a 96k \
        -f ssegment -segment_list_flags live \
            -segment_list_type csv -segment_time 1 -segment_list pipe:1 \
            work/out%03d.ts \
    | node ./ahead-server.js

$ ~/ahead-server # bash test.sh 
ffmpeg version 4.2.4-1ubuntu0.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[video4linux2,v4l2 @ 0x56439a7abc40] The V4L2 driver changed the video from 640x360 to 1704x720
[video4linux2,v4l2 @ 0x56439a7abc40] The driver does not permit changing the time per frame
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 5280615.706658, bitrate: 588902 kb/s
    Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 1704x720, 588902 kb/s, 30 fps, 30 tbr, 1000k tbn, 1000k tbc
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, alsa, from 'plughw:0,1':
  Duration: N/A, start: 1606934775.396563, bitrate: 1536 kb/s
    Stream #1:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x56439a7cb680] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x56439a7cb680] profile High, level 3.2
[stream_segment,ssegment @ 0x56439a7c9fc0] Opening 'pipe:1' for writing
[stream_segment,ssegment @ 0x56439a7c9fc0] Opening 'work/out000.ts' for writing
Output #0, stream_segment,ssegment, to 'work/out%03d.ts':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: h264 (libx264), yuv420p, 1704x720, q=-1--1, 30 fps, 90k tbn, 30 tbc
    Metadata:
      encoder         : Lavc58.54.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
chunk 00016 fps=0.0 q=25.0 size=N/A time=00:00:00.50 bitrate=N/A speed=0.973x    
[stream_segment,ssegment @ 0x56439a7c9fc0] Opening 'work/out001.ts' for writing
chunk 00146 fps= 30 q=26.0 size=N/A time=00:00:01.50 bitrate=N/A speed=0.985x    
[stream_segment,ssegment @ 0x56439a7c9fc0] Opening 'work/out002.ts' for writing
[alsa @ 0x56439a7af7c0] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
chunk 002
frame=   74 fps= 30 q=25.0 Lsize=N/A time=00:00:02.43 bitrate=N/A speed=0.988x    
video:1444kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

VLC plays the video, with no audio

When I change the -f alsa -i plughw:0,1 -map 0 \ to -f alsa -i plughw:0,1 -map 1 \ ... VLC plays the audio, without video :/

root@Ubuntu-2004-focal-64-minimal ~/ahead-server # bash test.sh 
ffmpeg version 4.2.4-1ubuntu0.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[video4linux2,v4l2 @ 0x55a688ca8c40] The V4L2 driver changed the video from 640x360 to 1704x720
[video4linux2,v4l2 @ 0x55a688ca8c40] The driver does not permit changing the time per frame
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 5280680.640070, bitrate: 588902 kb/s
    Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 1704x720, 588902 kb/s, 30 fps, 30 tbr, 1000k tbn, 1000k tbc
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, alsa, from 'plughw:0,1':
  Duration: N/A, start: 1606934840.332530, bitrate: 1536 kb/s
    Stream #1:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
Codec AVOption preset (Configuration preset) specified for output file #0 (work/out%03d.ts) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
Codec AVOption tune (Tune the encoding to a specific scenario) specified for output file #0 (work/out%03d.ts) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
Codec AVOption x264-params (Override the x264 configuration using a :-separated list of key=value parameters) specified for output file #0 (work/out%03d.ts) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
Stream mapping:
  Stream #1:0 -> #0:0 (pcm_s16le (native) -> aac (native))
Press [q] to stop, [?] for help
[stream_segment,ssegment @ 0x55a688cc6fc0] Opening 'pipe:1' for writing
[stream_segment,ssegment @ 0x55a688cc6fc0] Opening 'work/out000.ts' for writing
[mpegts @ 0x55a688d04580] frame size not set
Output #0, stream_segment,ssegment, to 'work/out%03d.ts':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 96 kb/s
    Metadata:
      encoder         : Lavc58.54.100 aac
chunk 000time=00:00:00.94 bitrate=N/A speed=0.94x     
[stream_segment,ssegment @ 0x55a688cc6fc0] Opening 'work/out001.ts' for writing
[mpegts @ 0x55a688d04580] frame size not set
chunk 001time=00:00:01.97 bitrate=N/A speed=0.979x    

[MichalCz]

Can you check one of the work/out*.ts files if these have both video and audio?

I think the apple format allowed those to be muxed, but if not - let's get ffmpeg to split those up and save to separate files and then just rewrite the parser inside a little.

I'm not actively working on this project, but if you like to test this and get involved I can reviwe it.