RAISR

A Python & a C/Vulkan implementation of RAISR

INPUT FORMAT: Only JPEG input is supported. PNG output is supported.

How To Use

Prerequisites

You can install most of the following packages using pip.

• OpenCV-Python
• NumPy
• SciPy
• Python Imaging Library (PIL)
• Matplotlib
• scikit-image

Training

Put your training images in the train directory. The training images are the high resolution (HR) ones. Run the following command to start training.

python train.py

In the training stage, the program virtually downscales the high resolution images. The program then trains the model using the downscaled version images and the original HR images. The learned filters filter.p will be saved in the root directory of the project. The result Q, V matrix (q.p and v.p) will also be saved for further retraining. To train an improved model with your previous Q, V, use the following command.

python train.py -q q.p -v v.p

Testing

Put your testing images in the test directory. Basically, you can use some low resolution (LR) images as your testing images. By running the following command, the program takes filter.p generated by training as your default filters.

python test.py

The result (HR version of the testing images) will be saved in the results directory. To use an alternative filter file, take using the pretrained filters/filter_BSDS500 for example, use the following command.

python test.py -f filters/filter_BSDS500

Testing (C with Vulkan acceleration)

This repository also includes a C implementation with optional Vulkan GPU acceleration in the vulkan directory.

To build and run the C version:

cd vulkan
make
./raisr input_image.jpg output_image.jpg

The C implementation automatically uses Vulkan if available, otherwise falls back to multi-threaded CPU. Thread count is auto-detected at runtime.

Options:

Option	Description	Default
-f, --format <fmt>	Output format: jpeg or png	jpeg
-j, --jpeg <quality>	JPEG quality 1–100	82
-c, --cpu	Force CPU mode (disable Vulkan)	Auto-detect

./raisr input.jpg output.jpg --cpu          # force CPU
./raisr input.jpg output.png --format png   # lossless PNG output
./raisr input.jpg output.jpg --jpeg 95      # higher JPEG quality

Converting Filters for the C Implementation

After training, convert the filters to binary format:

import pickle, struct, numpy as np

with open("filters/filter_BSDS500", "rb") as f:
    filters = pickle.load(f, encoding="latin1")

h = np.array(filters).flatten().astype(np.float64)
with open("filter.bin", "wb") as f:
    f.write(struct.pack("I", len(h)))
    f.write(h.tobytes())

The original repo uses .p pickle files (float64) — this script converts them to the filter.bin binary format required by the C implementation.

Required files (place in vulkan/ directory):

• filter.bin — converted filter coefficients
• raisr.spv — compiled Vulkan compute shader. Recompile with:

  glslc -fshader-stage=compute raisr.glsl -o raisr.spv

Visualization

Visualing the learned filters

python train.py -p

Visualing the process of RAISR image upscaling

python test.py -p

For more details, use the help command argument -h.

Testing Results

Comparing between original image, bilinear interpolation and RAISR:

Origin	Bilinear Interpolation	RAISR

Other results using images taken from BSDS500 database and ArTe-Lab 1D Medium Barcode Dataset:

Origin	RAISR

Troubleshooting

Issue	Solution
filter.bin not found	Run the conversion script above
Filter size mismatch	Regenerate filter.bin with the script above
raisr.spv not found	Run make or use --cpu
Vulkan not available	Install GPU drivers or use --cpu
Crash on large images	GPU VRAM may be insufficient — use --cpu

CPU Performance Note: Thread count is auto-detected via sysconf(_SC_NPROCESSORS_ONLN) at runtime.

Contribute

We actively welcome pull requests. Learn how to contribute.

References

• Y. Romano, J. Isidoro and P. Milanfar, "RAISR: Rapid and Accurate Image Super Resolution" in IEEE Transactions on Computational Imaging, vol. 3, no. 1, pp. 110-125, March 2017.
• P. Arbelaez, M. Maire, C. Fowlkes and J. Malik, "Contour Detection and Hierarchical Image Segmentation", IEEE TPAMI, Vol. 33, No. 5, pp. 898-916, May 2011.
• Alessandro Zamberletti, Ignazio Gallo and Simone Albertini, "Robust Angle Invariant 1D Barcode Detection", Proceedings of the 2nd Asian Conference on Pattern Recognition (ACPR), Okinawa, Japan, 2013

License

MIT. Copyright (c) 2017 James Chen