Adding rocAL+rocJPEG decode performance harness

[essamROCm]

Motivation

This PR adds a rocAL-focused performance harness for validating and measuring rocJPEG-backed image decode behavior, especially for multi-GPU sharded decode workloads.

The main goals are:

• Improve rocJPEG usage inside rocAL by allowing the four-thread decode path to use up to four dedicated rocJPEG decoder instances, each handling a sub-batch, instead of funneling the full batch through a single shared decoder instance.
• Add a repeatable way to compare rocAL + rocJPEG decode performance with the dedicated rocJPEG OpenMP split path enabled and disabled.
• Add C++ and Python benchmark coverage for the same decode scenario so changes can be checked from both rocAL API surfaces.
• Add reporting scripts that summarize per-GPU/per-shard decode timing, decoded image counts, and speedup/reduction metrics.
• Add a standalone helper launcher for running jpegdecodeperf across GPU shards, making it easier to compare rocAL decode results against rocJPEG sample-level performance.
• Update the existing dataloader_multithread test app so it can drive the new rocJPEG split-path benchmarking with configurable CPU thread count and effective batch sizing.

Technical Details

This PR adds the main rocAL decode enhancement being measured by the new harness: rocJPEG decode work can now be split across multiple dedicated rocJPEG decoder instances instead of sending the whole batch through one shared rocJPEG decoder.

The rocJPEG dedicated OpenMP split path is enabled by default. With the default behavior, rocAL creates up to four rocJPEG decoder instances, bounded by the configured CPU thread count and batch size. The input batch is divided into per-decoder sub-batches, and OpenMP dispatches those sub-batches across the dedicated decoder workers. This allows the benchmark configuration of four CPU threads to use four rocJPEG decoder instances, reducing contention around one decoder and improving decode throughput for sharded/multi-GPU image loading workloads.

To compare against the previous behavior, set ROCAL_ROCJPEG_DEDICATED_OMP_SPLIT=0. In that mode, rocAL keeps the previous single-decoder rocJPEG path, so the benchmark scripts can compare the old and new behavior directly.

This PR adds tests/cpp_api/rocjpeg_decode_perf/ as a manual performance harness for the rocJPEG split-decoder change. These scripts are not regular CTest unit tests; they are intended for explicit developer/reviewer runs on systems with a suitable dataset and GPU configuration.

The harness is needed because the change is performance-sensitive. A correctness-only test would not show whether splitting rocJPEG work across multiple decoder instances improves decode time or whether ON/OFF behavior remains comparable.

The harness provides:

• A C++ rocAL benchmark path using dataloader_multithread.
• A Python rocAL benchmark path using fn.readers.file and fn.decoders.image.
• ON/OFF comparison using ROCAL_ROCJPEG_DEDICATED_OMP_SPLIT=0 and =1.
• TurboJPEG comparison runs.
• Per-GPU/per-shard decoded image counts and decode timing summaries.
• A standalone sharded jpegdecodeperf launcher for rocJPEG sample-level comparison.
• Reporting scripts to produce PR-friendly summaries of speedup and decode-time reduction.

This PR adds a new manual benchmark/support folder:

tests/cpp_api/rocjpeg_decode_perf/

New files:

tests/cpp_api/rocjpeg_decode_perf/README.md
tests/cpp_api/rocjpeg_decode_perf/perf_sharded_launcher.cpp
tests/cpp_api/rocjpeg_decode_perf/reporting_perf_sharded_results.sh
tests/cpp_api/rocjpeg_decode_perf/reporting_test_results.sh
tests/cpp_api/rocjpeg_decode_perf/rocal_decode_call_bench.py
tests/cpp_api/rocjpeg_decode_perf/run_tests_twice_solution_on_off.sh

The new README.md documents the benchmark purpose, required environment variables, common workflow, log locations, and example commands for PR reviewers or developers running the tests manually.

run_tests_twice_solution_on_off.sh is the main rocAL comparison driver. It runs six benchmark cases:

C++ rocAL + rocJPEG, split solution OFF
C++ rocAL + rocJPEG, split solution ON
C++ rocAL + TurboJPEG
Python rocAL + rocJPEG, split solution OFF
Python rocAL + rocJPEG, split solution ON
Python rocAL + TurboJPEG

The script uses:

ROCAL_ROCJPEG_DEDICATED_OMP_SPLIT=0
ROCAL_ROCJPEG_DEDICATED_OMP_SPLIT=1

to toggle the dedicated split path for rocJPEG runs. It writes logs under configurable LOG_DIR, defaulting to:

/tmp/rocjpeg_decode_perf

The script requires only the machine-specific inputs to be exported:

DATASET
ROCAL_CPP_BIN

and supports optional:

DATASET_LABEL
GPU_COUNT
ROCM_PATH
LOG_DIR
WORKSPACE
ROCAL_PY_BENCH
ROCJPEG_DECODER_CREATE_LOG

reporting_test_results.sh parses the logs produced by run_tests_twice_solution_on_off.sh. It summarizes:

• decoded image counts for C++ and Python runs
• per-GPU/per-shard decode times
• average decode time for each mode
• rocJPEG split-path decode-time reduction
• rocJPEG split-path speedup

rocal_decode_call_bench.py is a Python rocAL decode benchmark. It builds a simple readers.file + decoders.image pipeline and supports:

• CPU or GPU decode mode
• configurable batch size
• configurable CPU thread count
• configurable device id
• configurable number of GPUs
• configurable number of shards
• multi-process shard execution for multi-GPU tests
• rocAL internal timing extraction through pipe.timing_info()

perf_sharded_launcher.cpp is a standalone helper launcher for rocJPEG jpegdecodeperf. It:

• recursively scans a dataset for .jpg and .jpeg files
• creates per-GPU shard directories using symlinks
• launches one jpegdecodeperf process per GPU
• writes per-GPU logs as jpegdecodeperf_gpu<N>.log
• supports configurable batch size, thread count, output format, shard work directory, and log directory

reporting_perf_sharded_results.sh parses the logs produced by perf_sharded_launcher.cpp. It reports:

• per-GPU decoded image counts
• per-GPU decode time
• total decoded image count
• average decode time across GPU shards
• wall/max decode time across GPU shards

Changes in:

rocAL/include/loaders/image/image_read_and_decode.h
rocAL/source/loaders/image/image_read_and_decode.cpp

include:

• Add a vector of rocJPEG decoder instances for split execution.
• Add per-decoder sub-batch size tracking.
• Add _use_rocjpeg_dedicated_omp_split, controlled by ROCAL_ROCJPEG_DEDICATED_OMP_SPLIT.
• Initialize multiple rocJPEG decoder instances when the split path is enabled.
• Split the rocJPEG batch across up to four decoder workers, bounded by batch size and configured CPU thread count.
• Run rocJPEG decode-info/decode-batch work across decoder shards using OpenMP.
• Preserve the existing single-decoder rocJPEG path when the split path is disabled.

This PR also updates:

tests/cpp_api/dataloader_multithread/dataloader_multithread.cpp

to support this benchmark path by:

• accepting an additional cpu_thread_count argument
• passing cpu_thread_count into rocalCreate
• detecting ROCAL_ROCJPEG_DEDICATED_OMP_SPLIT
• increasing the effective rocAL batch size for rocJPEG split-path tests
• avoiding unnecessary output copies when display is disabled
• resizing image name buffers and display buffers according to effective batch size
• updating usage text to document rocJPEG mode and CPU thread count

Test Plan

Lightweight validation was run after applying the changes:

1. Verified shell syntax for all new scripts:

for f in tests/cpp_api/rocjpeg_decode_perf/*.sh; do
  bash -n "$f" || exit 1
done

2. Verified the helper C++ launcher compiles independently:

g++ -std=c++17 -Wall -Wextra -pedantic \
  tests/cpp_api/rocjpeg_decode_perf/perf_sharded_launcher.cpp \
  -o /tmp/perf_sharded_launcher_check

3. Verified the Python benchmark script compiles:

python3 -m py_compile tests/cpp_api/rocjpeg_decode_perf/rocal_decode_call_bench.py

Manual benchmark workflow documented in the README:

export WORKSPACE=/path/to/rocAL/tests/cpp_api/rocjpeg_decode_perf
cd "$WORKSPACE"

export DATASET=/path/to/image_dataset
export DATASET_LABEL=my_dataset
export ROCM_PATH=/opt/rocm
export LOG_DIR=/tmp/rocjpeg_decode_perf
export ROCAL_CPP_BIN=/path/to/dataloader_multithread

./run_tests_twice_solution_on_off.sh 1
./reporting_test_results.sh 1

Standalone jpegdecodeperf workflow documented in the README:

g++ -std=c++17 -O2 -Wall perf_sharded_launcher.cpp -o perf_sharded_launcher

./perf_sharded_launcher \
  "$DATASET" \
  1 \
  /path/to/jpegdecodeperf \
  32 \
  4 \
  rgb \
  "$LOG_DIR/shards" \
  "$LOG_DIR"

./reporting_perf_sharded_results.sh 1

Test Result

The following local checks passed:

• Shell syntax validation passed for:
  • reporting_perf_sharded_results.sh
  • reporting_test_results.sh
  • run_tests_twice_solution_on_off.sh
• perf_sharded_launcher.cpp compiled successfully with g++ -std=c++17 -Wall -Wextra -pedantic.
• rocal_decode_call_bench.py passed Python bytecode compilation with python3 -m py_compile.

The benchmark harness was added specifically to report the effect of this change by comparing:

ROCAL_ROCJPEG_DEDICATED_OMP_SPLIT=0

against:

ROCAL_ROCJPEG_DEDICATED_OMP_SPLIT=1

for both C++ and Python rocAL decode paths.

No full hardware benchmark results are included in this PR note because the added harness is intended to support manual performance validation on systems with the target ROCm/rocAL/rocJPEG installation, dataset, and GPU configuration.

[Copilot]

Pull request overview

This PR introduces a manual performance harness for measuring rocAL image decode throughput (rocJPEG vs TurboJPEG) in multi-GPU sharded workloads, and updates rocAL’s rocJPEG path to optionally split decode work across multiple dedicated rocJPEG decoder instances.

Changes:

• Add a new manual benchmark folder (tests/cpp_api/rocjpeg_decode_perf/) with C++/Python runners and reporting scripts for repeatable on/off comparisons.
• Enhance rocAL’s rocJPEG decode implementation to optionally shard a batch across up to 4 dedicated rocJPEG decoder instances using OpenMP.
• Update dataloader_multithread to support configurable CPU thread count and an “effective batch size” for rocJPEG split-path benchmarking.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
tests/cpp_api/rocjpeg_decode_perf/run_tests_twice_solution_on_off.sh	Driver script to run C++/Python benchmarks with rocJPEG split on/off plus TurboJPEG baselines.
tests/cpp_api/rocjpeg_decode_perf/rocal_decode_call_bench.py	Python rocAL decode benchmark with optional multi-process shard execution and timing extraction.
tests/cpp_api/rocjpeg_decode_perf/reporting_test_results.sh	Parses rocAL C++/Python logs and summarizes per-shard decode times and computed speedups.
tests/cpp_api/rocjpeg_decode_perf/reporting_perf_sharded_results.sh	Parses sharded jpegdecodeperf logs and summarizes per-GPU decode results.
tests/cpp_api/rocjpeg_decode_perf/README.md	Documents benchmark purpose, required env, workflows, and log/report generation.
tests/cpp_api/rocjpeg_decode_perf/perf_sharded_launcher.cpp	C++ helper to shard a dataset via symlinks and launch jpegdecodeperf per GPU with logs.
tests/cpp_api/dataloader_multithread/dataloader_multithread.cpp	Adds CPU thread-count arg and adjusts effective batch sizing/output handling for rocJPEG split benchmarking.
rocAL/source/loaders/image/image_read_and_decode.cpp	Implements optional rocJPEG dedicated OpenMP split path with multiple decoder instances and per-shard decode.
rocAL/include/loaders/image/image_read_and_decode.h	Adds state for multiple rocJPEG decoders, sub-batch sizes, and split toggle flag.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[rrawther]

what if the _num_threads < 4 set by user?

[essamROCm]

This is already bounded by _num_threads. The decoder count is computed as min(batch_size, max(1, min(4, _num_threads))), so if the user configures fewer than 4 CPU threads, rocAL creates only that many rocJPEG decoder instances. For example, _num_threads=2 creates 2 decoder instances; _num_threads=1 creates 1. The hard cap of 4 only applies when the user provides 4 or more CPU threads.

[rrawther]

this will make the sub-batch_size and odd number. Please check is this is OK for hardware decoder

[essamROCm]

Odd sub-batch sizes should be OK. The existing single-decoder rocJPEG path already passes the user-provided rocAL batch size directly to rocJPEG, so odd batch sizes were already possible before this new split. In the new split path, each rocJPEG decoder owns its own streams and is initialized with its own sub-batch size. The split logic only divides the same total batch across decoder instances; it does not introduce a new odd-size requirement that did not already exist.

[rrawther]

suggest having a helper function for this since this is duplicated in multiple files. Eventually we need to get rid of this. So it is OK to check for just 0/1 for simplicity. No need for OFF and False etc. Make the function case insensitive like below
static bool env_flag_disabled(const char* name) {
const char* val = std::getenv(name);
if (!val || val[0] == '\0') return false;
std::string s(val);
std::transform(s.begin(), s.end(), s.begin(),
[](unsigned char c) { return std::tolower(c); });
return s == "0" || s == "no";
}

[essamROCm]

Addressed, in commit: eb7750c. I added a small case-insensitive env_flag_disabled() helper and replaced the duplicated env parsing in both the rocAL loader path and the dataloader benchmark. The helper keeps the split path enabled by default and disables it only for 0 or no, matching the benchmark scripts' 0/1 usage.

[rrawther]

There is a duplicate hipSetDevice on line #370. Why this is required here

[essamROCm]

This is intentional. The outer hipSetDevice sets the device for the load routine thread, but the split path runs rocJPEG work inside OpenMP worker threads. HIP current device is thread-local, so each worker needs to set the device before invoking rocJPEG/HIP-backed work. I added a comment to clarify this at line number 395 in commit code: eb7750c.

[rrawther]

for line #366 and line #317 (if (!A && !B)... else if (A || B) else ..) it is very confusing. Not sure what all these cases are and why we need

[essamROCm]

Addressed in commit: eb7750c. I replaced the if (!A && !B) ... else if (A || B) condition with a named is_rocjpeg_decoder boolean and a plain if/else. This keeps the same behavior but makes the two decode paths clearer: non-rocJPEG decoders use the existing per-image path, and rocJPEG/rocJPEG cropped use the batched rocJPEG path.

[rrawther]

suggest putting all the decode_info calls for a shard within while loop. I think the current logic is substituting failed image with other image from batch. THis has to be done at the end of decoding by padding or something. Otherwise we are sending the same frame to decode twice to rocJpeg

[essamROCm]

Addressed in this commit: c0526b7.
I refactored the rocJPEG split-path decode-info logic so the original candidate and fallback candidates are handled in a single while loop per shard item. This keeps the same fallback/substitution behavior but avoids having one decode_info call outside the retry loop and another inside it.

[essamROCm]

Latest Code Change Test Summary

Test Configuration

AMD-SMI 26.2.2+671d39a71e
amdgpu version: 6.18.2
ROCm version: 7.2.2
VBIOS version: 00102431
Platform: Linux
AMD Instinct MI300X - SPX/NPS1

Dataset: ImageNet 5 Classes
Total Images : 19,500
GPU Count : 8

---

Decoded Image Count

Test Configuration	Images Decoded
C++ rocAL + rocJPEG OFF	19,504
C++ rocAL + rocJPEG ON	19,504
C++ rocAL + TurboJPEG	19,504
PY rocAL + rocJPEG OFF	19,500
PY rocAL + rocJPEG ON	19,500
PY rocAL + TurboJPEG	19,500

---

C++ rocAL Sample Decode-Time Results

Mode	Average decode time
rocAL + rocJPEG OFF	1.504794 seconds
rocAL + rocJPEG ON	0.651028 seconds
rocAL + TurboJPEG	1.986759 seconds

Python rocAL Benchmark Decode-Time Results

Mode	Average decode time
rocAL + rocJPEG OFF	1.265821 seconds
rocAL + rocJPEG ON	0.616735 seconds
rocAL + TurboJPEG	2.052351 seconds

rocAL Patch Solution Enhancement

Path	Decode-time reduction	Speedup
C++ rocAL + rocJPEG	56.74%	2.31x
Python rocAL + rocJPEG	51.28%	2.05x

[rrawther]

Throw inside an openmp pragma can produce unwanted results. Probably you need to catch the return code and throw exception after all threads are done

[essamROCm]

Addressed in this commit: c0526b7.
I removed the THROW calls from inside the OpenMP worker loop. Worker failures are now recorded through a shared error flag/message protected by an OpenMP critical section, and the main thread throws once after the parallel region completes.

[rrawther]

I don't understand what is all these tests and scripts used for. Let's discuss

[rrawther]

please address review comments

[essamROCm]

The Added Test Script Files

These files are not intended to be regular correctness/unit tests. They are a manual performance harness for validating the rocJPEG split-decoder change in rocAL.

The rocAL code change affects how rocJPEG decode work is scheduled internally: instead of using one rocJPEG decoder instance for the full batch, the split path can use multiple dedicated rocJPEG decoder instances and divide the batch across them. To validate that type of change, we need more than a normal pass/fail test; we need a repeatable way to compare decode timing with the split path ON and OFF across C++ and Python rocAL entry points.

The scripts are organized as follows:

• run_tests_twice_solution_on_off.sh: Main driver. Runs rocAL C++ and Python decode benchmarks with ROCAL_ROCJPEG_DEDICATED_OMP_SPLIT=0 and =1, plus TurboJPEG comparison runs.
• reporting_test_results.sh: Parses those logs and summarizes decoded image counts, per-shard/per-GPU decode times, average decode time, and ON-vs-OFF speedup/reduction.
• rocal_decode_call_bench.py: Python rocAL decode benchmark equivalent to the C++ sample path, used to validate the Python API path.
• perf_sharded_launcher.cpp: Helper to shard a dataset and launch rocJPEG jpegdecodeperf once per GPU. This is for comparing against the rocJPEG sample-level benchmark.
• reporting_perf_sharded_results.sh: Summarizes the jpegdecodeperf per-GPU logs.

This folder gives developers a reproducible workflow for answering:

1. Does the old single-decoder path still work?
2. Does the new split-decoder path work?
3. How many images were decoded in each mode?
4. What is the per-GPU/per-shard decode time?
5. What speedup or reduction does the split path provide?
6. Does behavior look consistent through both C++ rocAL and Python rocAL APIs?

This is why the folder is under tests/cpp_api: it depends on the existing dataloader_multithread C++ sample/test and is intended for manual performance validation, not automatic CTest execution.