To run the benchmark :
- Install FFmpeg 7.1.
- Install Powershell 7.
- To run the Spatial and Temporal Information experiment, also install siti-tools.
To generate the plots :
- Install the uv package manager. It will download the right version of Python automatically.
- Install the required Python packages using
uv sync.
A description of each script parameters is available inside the script.
This script calculates the spatial and temporal information of videos according to the ITU-T P.910 (07/2022) recommendation using the reference software.
siTi.ps1 -tiles "tile1.y4m", "tile2.y4m" -resultsFile sitiTiles.csvThis script evaluates the bitrate, visual quality and the segment's encoding time of videos with different encoding parameters. First, generate the tasks to be performed.
bitrateVmafSegmentsEncodingTimeTasks.ps1 -tiles "tile1.y4m", "tile2.y4m" -codecs "h264_nvenc", "hevc_nvenc" -presets p1, p2 -qps 18, 20 -heights 0, 320 -outputFile tasks.csvThen, start the benchmark.
bitrateVmafSegmentsEncodingTime.ps1 -inputFile tasks.csv -segmentTime 2 -segmentGOP 60 -logsDirectory ".\logs" -outputFile bitrateVmafSegmentsEncodingTime.csvIf the benchmark stops for any reason, you may restart it again by using the incomplete output file (here bitrateVmafSegmentsEncodingTime.csv) as the input file.
The script will automatically resume the execution of the remaining tasks.
This script evaluates the time needed to segment and encode the videos with different encoding parameters. First, generate the tasks to be performed.
totalEncodingTimeTasks.ps1 -tiles "tile1.y4m", "tile2.y4m" -codecs "h264_nvenc", "hevc_nvenc" -presets p1, p2 -qps 18, 20 -heights 0, 320 -repetitions 5 -outputFile tasks.csvThen, start the benchmark.
totalEncodingTime.ps1 -inputFile tasks.csv -segmentTime 2 -segmentGOP 60 -outputFile totalEncodingTime.csvIf the benchmark stops for any reason, you may restart it again by using the incomplete output file (here totalEncodingTime.csv) as the input file.
The script will automatically resume the execution of the remaining tasks.
This script plots the spatial and temporal information of videos. Example usage :
uv run -m src.plots.plotSiTi siTi.csv siTi.pdfThis script plots the BD-rate performance for all tiles, codecs and presets. Example usage :
uv run -m src.plots.plotBdRate bitrateVmafSegmentsEncodingTime.csv h264_nvenc p1 bdRate.pdf --heightLabels 0='8K' 320='4K' This script plots the encoding speed in frames per second of different codecs, presets and resolutions. Example usage :
uv run -m src.plots.plotEncodingSpeed totalEncodingTime.csv encodingSpeed.pdf --heightLabels 0='8K' 320='4K'