Skip to content

WIP: [SAP] Add volume history tracking#319

Open
hemna wants to merge 9 commits intostable/2025.1-m3from
feature/volume-history-tracking
Open

WIP: [SAP] Add volume history tracking#319
hemna wants to merge 9 commits intostable/2025.1-m3from
feature/volume-history-tracking

Conversation

@hemna
Copy link
Copy Markdown

@hemna hemna commented Mar 11, 2026

Summary

This PR adds a volume history tracking feature that records every mutation to a volume's DB row in a new volume_history table. Each history record captures JSON deltas of old/new values along with contextual metadata.

Status: Work In Progress

Changes

  • New VolumeHistory model in models.py with FK to volumes.id
  • Alembic migration 633b14d87cec to create the volume_history table
  • DB API hooks in volume_create(), volume_update(), volume_destroy(), volume_attached(), volume_detached()
  • Query function volume_history_get_all_by_volume() to retrieve history
  • Config option volume_history_enabled to disable tracking in high-throughput environments
  • Unit tests for all history tracking functionality
  • SAP documentation in sap-doc/volume-history.md

Configuration

Volume history tracking is enabled by default but can be disabled:

[DEFAULT]
volume_history_enabled = False

Tracked Actions

Action Description
create Volume creation - all initial field values
update Volume field update - only changed fields
destroy Volume deletion - status transition
attach Volume attachment - status changes
detach Volume detachment - status changes

Performance Overhead Analysis

volume_update() — Heaviest overhead (most frequent operation)

Added work Cost
Extra SELECT query (query.filter_by(id=volume_id).first()) 1 DB round-trip
Python loop comparing old vs new values Negligible
json.dumps() of changes dict Negligible
session.add() for history INSERT 1 INSERT (batched in same transaction)

Each volume_update adds 1 extra SELECT by PK + 1 extra INSERT, both within the same DB transaction that was already open. The SELECT is a fast indexed lookup on the volumes.id primary key.

volume_destroy() — Moderate overhead

Added work Cost
query.first() to get current status 1 DB round-trip (consumes existing query, so we re-create it)
Re-creating the query object 0 extra round-trips (Python object construction only)
session.add() for history INSERT 1 INSERT (batched)

Adds 1 extra SELECT + 1 extra INSERT.

volume_create() — Minimal overhead

Added work Cost
Dict comprehension to build changes Negligible (pure Python, no DB)
session.add() for history INSERT 1 INSERT (batched in same transaction)

Just 1 extra INSERT, no extra SELECT since we already have the values dict.

volume_attached() / volume_detached() — Minimal overhead

Same as create — just 1 extra INSERT each, no extra SELECT.

Mitigation

Set volume_history_enabled = False in cinder.conf to completely disable history tracking and eliminate all overhead.

Testing

All tests pass in Docker (Python 3.10):

  • 8 new volume history unit tests (including config disable test)
  • 67 existing volume tests (including updated test_volume_destroy_deletes_dependent_data)
  • ORM relationship tests
  • Purge tests

TODO

  • Review and feedback
  • Consider REST API endpoint (future iteration)

hemna added 7 commits March 11, 2026 10:38
Add a new VolumeHistory model to track changes to volumes over time.
Each record captures a JSON delta of changed fields (old/new values)
along with contextual metadata (user_id, project_id, request_id).

Change-Id: Ib676eb81507847393183fe05f9870266b79d5b4b
Creates the volume_history table with columns for tracking volume
changes over time including volume_id, project_id, user_id, request_id,
action type, and a JSON changes column for the delta.

Change-Id: Ifa0e979f0c6746a13f5a64c6dd15565f949ebef3
- Add import json for serializing history changes
- Add VolumeHistory to VOLUME_DEPENDENT_MODELS for cascade soft-delete
- Add _record_volume_history() helper function
- Instrument volume_create() to record initial values
- Instrument volume_update() to record delta of changed fields
- Instrument volume_destroy() to record status->deleted transition
- Instrument volume_attached() to record status/attach_status changes
- Instrument volume_detached() to record status/attach_status changes
- Add volume_history_get_all_by_volume() query function
- Add pass-through in db/api.py

Change-Id: I60fcd9c887860bd78b1124b2c79a4e2de891c0ec
Verify the migration creates the volume_history table with all
expected columns and the volume_id index.

Change-Id: I13ca352590632d836b870377c46e64a4855507b3
Add DBAPIVolumeHistoryTestCase with tests for:
- Volume create records history
- Volume update records history with changed fields
- Volume update with no changes doesn't record history
- Volume destroy records history
- History records are ordered by created_at
- Request ID is captured in history
- Getting history for non-existent volume returns empty list

Change-Id: I8d2e773eeafdaee483667f9dcb029b2eddef3b03
Change-Id: Ifdec8eccf9e7ac4a1f438d166096a9d5f1ef37bd
Add a config option to disable volume history tracking for high-throughput
environments where the additional DB overhead may be a concern.

- Add volume_history_enabled option to cinder/common/config.py (default: True)
- Update _record_volume_history() to check config before recording
- Add test for disabled config behavior
- Update SAP documentation with configuration details

Change-Id: I066aa79e4e2538831bbe152007c84ec23bdc9866
@hemna hemna force-pushed the feature/volume-history-tracking branch from 3b610af to f75b318 Compare March 11, 2026 17:25
Change-Id: I118f4ea376857bcd5bd740e62f06106c253a4fcc
Base automatically changed from update-from-antelope to stable/2025.1-m3 March 11, 2026 19:52
The volume_update hook was not recording changes for operations
that go through Volume.save() (extend, rename, etc.) because
SQLAlchemy's ORM reflects the UPDATE back onto the model object
within the same session.

Fix by copying the relevant old values into a dictionary BEFORE
the query.filter_by().update() call, so we compare against the
true old values rather than the post-update values.

Also adds datetime serialization handling for proper JSON storage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant