Skip to content

feat(gguf_scanning): scan gguf metadata, augmenting AI BOM#26

Open
afogel wants to merge 2 commits intoGenAI-Security-Project:v0.2from
afogel:scan_gguf_metadata
Open

feat(gguf_scanning): scan gguf metadata, augmenting AI BOM#26
afogel wants to merge 2 commits intoGenAI-Security-Project:v0.2from
afogel:scan_gguf_metadata

Conversation

@afogel
Copy link

@afogel afogel commented Jan 15, 2026

Summary

This PR adds comprehensive GGUF metadata extraction and chat template consistency reporting to the AIBOM generator. It closes a key execution‑configuration blind spot (tokenizer + chat template) and brings reporting in line with the current registry‑driven scoring model. The result is a more complete, compliance‑oriented AIBOM that captures both model weights and execution‑time configuration integrity. Closes #25

Background / Why

GGUF is now a primary distribution format for quantized LLMs. It bundles execution‑time configuration such as the tokenizer and chat template alongside weights. This execution layer is a real attack surface: a poisoned chat template can influence outputs without altering weights, bypassing traditional scanning. This PR surfaces that data and adds a consistency signal that flags mismatches across quantizations.

What’s Included

  • GGUF metadata extraction
    • Parses GGUF metadata (header + key/value metadata), not full file contents.
    • Captures architecture, quantization, and template metadata embedded in GGUF files.
  • Chat template consistency
    • Computes and reports whether chat templates remain consistent across GGUF quantizations.
    • Provides a human‑readable output so the signal is clear to auditors and reviewers.
  • Scoring/report alignment
    • “How AIBOM Completeness is Scored” now reflects dynamic max points from the registry.
  • New tests
    • Added comprehensive coverage for GGUF parsing and chat template consistency logic.

Testing Coverage (broad cases)

  • GGUF metadata extraction for multiple quantized models
  • Chat template hash detection and consistency across quantizations
  • Template mismatch detection and reporting in the model card
  • Template consistency confirmation when all quantizations match
  • Scoring/report output aligned with registry max points

QA / Validation

Deployed and testable at:
https://huggingface.co/spaces/ariel-pillar/OWASP-AIBOM-Generator

Suggested QA models:

  • ariel-pillar/phi-4_function_calling
    • Expected: detects a GGUF chat template mismatch across quantizations.
  • unsloth/Nemotron-3-Nano-30B-A3B-GGUF
    • Expected: no deviations; chat templates are consistent across quantizations.

Breaking Changes

  • None expected. Existing AIBOM output is extended with additional GGUF metadata and chat template fields.

@afogel afogel changed the base branch from main to v0.2 February 2, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enhancement: Add GGUF metadata scanning + chat template compliance signal to AI BOM generation

1 participant