Skip to content

Collect CORSIKA limits plots#2219

Open
GernotMaier wants to merge 19 commits into
mainfrom
collect-corsika-limits-plots
Open

Collect CORSIKA limits plots#2219
GernotMaier wants to merge 19 commits into
mainfrom
collect-corsika-limits-plots

Conversation

@GernotMaier
Copy link
Copy Markdown
Contributor

Update the file collection feature for simtools-run-application:

  • use of globular patterns
  • use of several collection steps

e.g., we can have now:

collection:
  - output_path: tmp_data/corsika_lookups/v1.0.0/electron/
    files:
    - corsika_limits_north.ecsv
    - corsika_limits_north.meta.yml
  - output_path: tmp_data/corsika_lookups/v1.0.0/electron/plots/
    files:
    - core_distance_vs_energy_triggered*.png
    - angular_distance_vs_energy_triggered*.png
    - energy_trigger*.png

Update and improve the file naming such that the intended patterns actually work (and to not collect all files).

@GernotMaier GernotMaier self-assigned this May 29, 2026
@GernotMaier GernotMaier changed the base branch from main to single-telescope-layouts May 29, 2026 13:21
@ctao-sonarqube
Copy link
Copy Markdown

@GernotMaier GernotMaier marked this pull request as ready for review May 29, 2026 14:13
@GernotMaier GernotMaier requested a review from Copilot May 29, 2026 14:14
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends the simtools-run-application collection feature to support glob patterns and multiple output-path collection entries, and reshapes CORSIKA-limits plot filenames so glob patterns can target meaningful subsets. Also moves the CORSIKA limits output table layout into a schema file and drops event_data_file from the output columns.

Changes:

  • _copy_collection_files accepts a list of {output_path, files} entries and _find_collection_files handles glob patterns recursively, with collision detection on the destination basename.
  • Plot filenames for triggered histograms get a _triggered override, and optionally include array_name, z<zenith>, az<azimuth>, nsb<level> from the limits dict.
  • CORSIKA limits result columns / descriptions / file-info mappings are now read from a new corsika_limits_table.schema.yml; event_data_file column removed; production subdirectory naming simplified to prefer the parent directory name.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/simtools/runners/simtools_runner.py Glob/list collection logic with collision detection.
src/simtools/visualization/plot_simtel_event_histograms.py Filename overrides for triggered histograms and metadata-based filename suffix.
src/simtools/production_configuration/derive_corsika_limits.py Schema-driven RESULT_COLUMNS/FILE_INFO_COLUMNS; simplified subdir naming.
src/simtools/applications/production_derive_corsika_limits.py Doc table updated (drops event_data_file).
src/simtools/schemas/corsika_limits_table.schema.yml New schema defining CORSIKA limits output table.
src/simtools/schemas/model_parameter_and_data_schema.metaschema.yml Allows optional file_info_key on TableColumn.
tests/unit_tests/runners/test_simtools_runner.py Tests for glob/list collection, collisions, and skips.
tests/unit_tests/visualization/test_plot_simtel_event_histograms.py Tests for new filename helpers and triggered overrides.
tests/unit_tests/production_configuration/test_derive_corsika_limits.py Tests for schema-driven mappings and removed event_data_file.
tests/integration_tests/config/plot_simulated_event_distributions.yml Updated expected triggered 2D filename.
docs/changes/2203.feature.md, docs/changes/2219.feature.md Changelog fragments.

Comment on lines +121 to +122
if isinstance(collection_config, dict):
collection_config = [collection_config]
Base automatically changed from single-telescope-layouts to main May 30, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants