This simulation package interfaces NVIDIA OptiX with Geant4 to accelerate optical photon transport for physics experiments. It supports detector geometries defined in the GDML format and is based on the work by Simon Blyth, whose original Opticks framework can be found here.

Prerequisites

Before building or running this package, ensure that your system meets both the hardware and software requirements listed below.

• A CUDA-capable NVIDIA GPU

• CUDA 12+

• NVIDIA OptiX 7+

• Geant4 11+

• CMake 3.18+

• Python 3.8+

Optionally, if you plan to develop or run the simulation in a containerized environment, ensure that your system has the following tools installed:

• Docker Engine
• NVIDIA container toolkit (installation guide)

Build

git clone https://github.com/BNLNPPS/eic-opticks.git
cmake -S eic-opticks -B build
cmake --build build

Docker

Build latest eic-opticks image by hand:

docker build -t ghcr.io/bnlnpps/eic-opticks:latest https://github.com/BNLNPPS/eic-opticks.git

Build and run for development:

docker build -t ghcr.io/bnlnpps/eic-opticks:develop --target=develop .

Example commands for interactive and non-interactive tests:

docker run --rm -it -v $HOME/.Xauthority:/root/.Xauthority -e DISPLAY=$DISPLAY --net=host ghcr.io/bnlnpps/eic-opticks:develop

docker run --rm -it -v $HOME:/esi -v $HOME/eic-opticks:/src/eic-opticks -e DISPLAY=$DISPLAY -e HOME=/esi --net=host ghcr.io/bnlnpps/eic-opticks:develop

docker run ghcr.io/bnlnpps/eic-opticks bash -c 'simg4ox -g tests/geom/sphere_leak.gdml -m tests/run.mac -c sphere_leak'

Singularity

singularity run --nv -B eic-opticks-prefix/:/opt/eic-opticks -B eic-opticks:/src/eic-opticks docker://ghcr.io/bnlnpps/eic-opticks:develop

Running a test job at NERSC (Perlmutter)

To submit a test run of eic-opticks on Perlmutter, use the following example. Make sure to update any placeholder values as needed.

sbatch scripts/submit.sh

#!/bin/bash

#SBATCH -N 1                    # number of nodes
#SBATCH -C gpu                  # constraint: use GPU partition
#SBATCH -G 1                    # request 1 GPU
#SBATCH -q regular              # queue
#SBATCH -J eic-opticks          # job name
#SBATCH --mail-user=<USER_EMAIL>
#SBATCH --mail-type=ALL
#SBATCH -A m4402                # allocation account
#SBATCH -t 00:05:00             # time limit (hh:mm:ss)

# Path to your image on Perlmutter
IMAGE="docker:bnlnpps/eic-opticks:develop"
CMD='cd /src/eic-opticks && simg4ox -g $OPTICKS_HOME/tests/geom/sphere_leak.gdml -m $OPTICKS_HOME/tests/run.mac -c sphere_leak'

# Launch the container using Shifter
srun -n 1 -c 8 --cpu_bind=cores -G 1 --gpu-bind=single:1 shifter --image=$IMAGE /bin/bash -l -c "$CMD"

Optical Surface Models in Geant4

In Geant4, optical surface properties such as finish, model, and type are defined using enums in the G4OpticalSurface and G4SurfaceProperty header files:

• G4OpticalSurface.hh
• G4SurfaceProperty.hh

These enums allow users to configure how optical photons interact with surfaces, controlling behaviors like reflection, transmission, and absorption based on physical models and surface qualities. The string values corresponding to these enums (e.g. "ground", "glisur", "dielectric_dielectric") can also be used directly in GDML files when defining <opticalsurface> elements for geometry. Geant4 will parse these attributes and apply the corresponding surface behavior.

For a physics-motivated explanation of how Geant4 handles optical photon boundary interactions, refer to the Geant4 Application Developer Guide — Boundary Process.

<gdml>
  ...
  <solids>
    <opticalsurface finish="ground" model="glisur" name="medium_surface" type="dielectric_dielectric" value="1">
      <property name="EFFICIENCY" ref="EFFICIENCY_DEF"/>
      <property name="REFLECTIVITY" ref="REFLECTIVITY_DEF"/>
    </opticalsurface>
  </solids>
  ...
</gdml>

User/developer defined inputs

Defining primary particles

There are certain user defined inputs that the user/developer has to define. In the src/simg4oxmt example that imports src/g4appmt.h we provide a working example with a simple geometry. The User/developer has to change the following details: Number of primary particles to simulate in a macro file and the number of G4 threads. For example:

/run/numberOfThreads {threads}
/run/verbose 1
/process/optical/cerenkov/setStackPhotons {flag}
/run/initialize
/run/beamOn 500

Here setStackPhotons defines whether G4 will propagate optical photons or not. In production Opticks (GPU) takes care of the optical photon propagation. Additionally the user has to define the starting position, momentum etc of the primary particles define in the GeneratePrimaries function in ``src/g4appmt.h```. The hits of the optical photons are returned in the EndOfRunAction function. If more photons are simulated than can fit in the GPU RAM the execution of a GPU call should be moved to EndOfEventAction together with retriving the hits.

Loading in geometry into EIC-Opticks

EIC-Opticks can import geometries with GDML format automatically. There are about 10 primitives supported now, eg. G4Box. G4Trd or G4Trap are not supported yet, we are working on them. src/simg4oxmt takes GDML files through arguments, eg. src/simg4oxmt -g mygdml.gdml.

The GDML must define all optical properties of surfaces of materials including:

• Efficiency (used by EIC-Opticks to specify detection efficiency and assign sensitive surfaces)
• Refractive index
• Group velocity
• Reflectivity
• Etc.

Performance studies

In order to quantify the speed-up achieved by EIC-Opticks compared to G4 we provide a python code that runs the same G4 simulation with and without tracking optical photons in G4. The difference of the runs will yield the time required to simulate photons. Meanwhile the same photons are simulated on GPU with EIC-Opticks and the simulation time is saved.

mkdir -p /tmp/out/dev
mkdir -p /tmp/out/rel

docker build -t eic-opticks:perf-dev --target=develop
docker run --rm -t -v /tmp/out:/tmp/out eic-opticks:perf-dev run-performance -o /tmp/out/dev

docker build -t eic-opticks:perf-rel --target=release
docker run --rm -t -v /tmp/out:/tmp/out eic-opticks:perf-rel run-performance -o /tmp/out/rel