chore(xtest): Audit and local improvements together by dmihalcik-virtru · Pull Request #390 · opentdf/tests

dmihalcik-virtru · 2026-01-27T22:05:20Z

feat(xtest): add tmux-based local test environment with modular scripts
feat(xtest): add audit log processing framework and test integration

gemini-code-assist · 2026-01-27T22:06:38Z

Summary of Changes

Hello @dmihalcik-virtru, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly overhauls the local testing infrastructure for the xtest suite, introducing a new, highly structured approach to managing test environments and debugging. The changes aim to provide developers with a more reliable, consistent, and user-friendly experience when running and troubleshooting integration tests. By modularizing shell scripts and integrating a dedicated audit log analysis framework, the PR enhances the overall maintainability and diagnostic capabilities of the test harness.

Highlights

New Local Test Environment: Introduced a comprehensive tmux-based local test environment for xtest, enabling streamlined management of all services (Docker, Platform, KAS instances) through modular shell scripts. This includes dedicated scripts for starting, stopping, status checks, and log viewing.
Modular Shell Script Library: Developed a new modular shell script library (xtest/scripts/lib/) with 55 unit tests, providing reusable utilities for logging, platform detection, path resolution, health checks, service waiting, tmux management, KAS-specific configurations, and YAML manipulation. This library ensures consistent and robust scripting across the test suite.
KAS Audit Log Framework: Added a KAS audit log collection and assertion framework (xtest/audit_logs.py and xtest/fixtures/audit.py). This framework allows tests to capture KAS service logs in the background, mark timestamps, and assert on log contents, significantly enhancing debugging capabilities for KAS-related issues.
Updated KAS Naming Conventions: Refactored KAS instance naming in fixtures from generic 'value1', 'value2', 'attr', 'ns' to more descriptive 'alpha', 'beta', 'gamma', 'delta' to align with CI workflow naming and improve clarity.
EC-Wrapped TDF Support: Updated the manifest.schema.json to include 'ec-wrapped' as a valid key access object type, reflecting enhanced support for Elliptic Curve-based TDF wrapping algorithms.
Test Infrastructure Improvements: Refactored the skip_if_unsupported logic in tdfs.py for better clarity and removed the pytest-xdist dependency from pyproject.toml. The tmp_dir fixture in conftest.py was also simplified.
Updated LTS Versions: Adjusted the Long Term Support (LTS) versions for Go, Java, JavaScript SDKs, and the Platform in resolve-version.py.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Ignored Files

Ignored by pattern: .github/workflows/** (1)
- .github/workflows/xtest.yml

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a new audit logging framework for tests and refactors the local test environment to use a modular, tmux-based shell script system. The changes are extensive and well-structured. The new audit log framework is a great addition for improving test diagnostics, and the shell script refactoring improves maintainability. I've found a critical issue in a path helper function within the new script library and some minor path issues in the updated documentation. Overall, this is a significant improvement to the test infrastructure.

gemini-code-assist · 2026-01-27T22:10:11Z

xtest/scripts/lib/core/paths.sh

+get_platform_dir() {
+  local xtest_dir
+  xtest_dir="$(get_xtest_dir)"
+  echo "$(cd "$xtest_dir/../../platform" && pwd)"
+}


The get_platform_dir function appears to construct an incorrect path. It uses cd "$xtest_dir/../../platform", which resolves to a directory one level above the repository root. Assuming the platform directory is a sibling of the xtest directory within the repository, the correct relative path should be ../platform from the xtest directory.

Suggested change

get_platform_dir() {

local xtest_dir

xtest_dir="$(get_xtest_dir)"

echo "$(cd "$xtest_dir/../../platform" && pwd)"

}

get_platform_dir() {

local xtest_dir

xtest_dir="$(get_xtest_dir)"

echo "$(cd "$xtest_dir/../platform" && pwd)"

}

gemini-code-assist · 2026-01-27T22:10:11Z

AGENTS.md


-## Build, Test, and Development Commands
+### Structure
+- **Test Directory**: `tests/xtest/` - pytest-based integration tests


This documentation contains a few incorrect paths that could confuse users:

On line 8, the test directory should be xtest/, not tests/xtest/.

Several example commands (e.g., on lines 347, 361, 362, 714) use paths like platform/opentdf-dev.yaml. Since test commands are typically run from the xtest/ directory, the correct relative path to the platform directory would be ../platform/.

- Implement lmgmt Python CLI tool for managing local test environment - Add service abstraction (Platform, KAS, Docker, Provisioner) - Implement health checks and wait utilities - Add structured log aggregation with filtering capabilities - Include comprehensive pytest integration tests - Update CI workflows for audit log collection

- Add comprehensive audit log test coverage for TDF roundtrip, ABAC, and policy tests - Implement clock skew estimation for accurate timestamp comparison - Increase audit log assertion timeout and report late-arriving logs - Add integration tests for audit log collection in CI pipeline - Make audit assertions required by default with override option

- Fix subprocess deadlock in otdfctl by properly handling stdout/stderr - Capture stderr output for better debugging when CLI calls fail - Add env command to lmgmt for outputting pytest environment variables

- Document lmgmt Python CLI as recommended tool for environment management - Add quick reference commands for service management - Update troubleshooting workflow to use lmgmt - Include audit log assertions requirements and disabling procedures

- Add .shfmt configuration for shell script formatting - Update .gitignore for generated files - Update xtest dependencies and manifest schema

Export the manifest schema file path in the lmgmt env command to support manifest validation tests. Tests can now validate TDF manifests against the schema without requiring manual SCHEMA_FILE configuration. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

- Add setup_golden_keys() to extract keys from extra-keys.json - Platform service now automatically adds golden-r1 key on startup - Add append_to_list() YAML helper with deduplication by kid - Update AGENTS.md with golden key configuration documentation This enables test_legacy.py golden file tests to work automatically when the environment is started with `lmgmt up`. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Convert kas_registry_create_if_not_present to use upsert pattern: - Try to create KAS entry - On 'already_exists' / 'unique field violation', list and verify existing - Verify public keys match expectations (when entry has keys populated) - Allow entries without keys (added later via kas_registry_create_public_key_only) Fixes concurrent pytest-xdist workers attempting simultaneous KAS registration. Applies same recovery pattern already used by kas_registry_create_public_key_only and kas_registry_import_key methods. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

…lue expectations Replace polling-based audit log assertion loops with Condition-based wakeup so assertion threads are notified immediately when new log data arrives instead of sleeping 100ms between checks. Also fix two tests that expected separate attribute_value audit events when the platform embeds values in the attribute_definition event. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Apply the same upsert pattern used in kas_registry_create_public_key_only and kas_registry_import_key: on already_exists/unique field violation, fetch and return the existing key instead of asserting. This fixes concurrent xdist workers racing to create the same managed key. Also relax attribute_value count assertion to >= since concurrent workers may create additional attribute_definition events. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Queries Go git tags, npm registry, Maven Central, and GitHub Releases to list available CLI versions with install methods. Supports --stable, --latest N, --table, and --releases filters. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add configure.py to install SDK CLIs from registries (Go via go run, JS via npm, Java via GitHub Releases) instead of always building from source. CI now skips checkout+make for released versions, reserving source builds for head/branch/PR testing only. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Upgrade cryptography from 46.0.3 to 46.0.5+ to address CVE-2024-9287 (Subgroup Attack Due to Missing Subgroup Validation for SECT Curves). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

This reverts commit ce2d343.

Unify ad-hoc SDK management scripts (configure.py, list-versions.py, resolve-version.py, shell scripts) into a proper uv-managed Typer CLI following the lmgmt pattern. Existing scripts become thin backward- compatible wrappers so CI workflows continue working without changes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace actions/setup-python with astral-sh/setup-uv across all workflows. Update xtest.yml and setup-cli-tool action to call otdf-sdk-mgr directly instead of the backward-compatible wrapper scripts (configure.py, resolve-version.py, post-checkout-java.sh). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Root CLAUDE.md was bloated with duplicated lmgmt command references. Move operational content (restarts, tmux, golden keys, troubleshooting) into lmgmt/CLAUDE.md and add env command + tmux docs to lmgmt/README.md. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The resolve command was URL-encoding the release field (e.g., sdk/v0.4.0 -> sdk%2Fv0.4.0), which caused npm to reject the version string when installing JS SDK artifacts. Also handle the sdk/ infix prefix in install_js_release so it correctly extracts the semver for npm. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add fixtures and tests for additional key types: - ec:secp384r1 (key_e384) - ec:secp521r1 (key_e521) - rsa:4096 (key_r4096) New fixtures in keys.py: - Three managed key fixtures with session-unique key_id suffixes - attribute_allof_with_extended_mechanisms fixture that creates an ALL_OF attribute with all three mechanisms and maps them to the test client for encryption New test in test_abac.py: - test_key_mapping_extended_mechanisms verifies encryption and decryption with all three extended mechanisms, validating that the manifest contains the correct key IDs and KAS URLs

… matching The previous implementation walked up the directory tree looking for a directory named exactly "tests". This failed in CI where the checkout path is "otdftests". Now we look for the presence of "xtest/sdk" subdirectory structure instead, which works regardless of the repository root directory name. Fixes: https://github.com/opentdf/tests/actions/runs/21961400726 Error was: FileNotFoundError: /home/runner/work/tests/tests/xtest/sdk/js/cli.sh Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

The Java SDK v0.12.0 and other recent releases don't publish cmdline.jar artifacts to GitHub Releases, causing CI failures when trying to install them from artifacts. Changes: 1. Modified resolve logic to mark Java "latest" as head (source build) when no CLI artifacts are available in recent releases 2. Modified Java installer to check artifact availability before download and exit gracefully if missing (caller can fall back to source) 3. Modified CI action to catch installation failures and continue (allowing source build fallback) This ensures Java SDKs are always built from source when artifacts aren't available, preventing CI failures. Fixes opentdf/tests#<issue-number> Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

- Rename package from lmgmt to otdf-local - Rename Python module from lmgmt to otdf_local (PEP 8) - Update CLI command to otdf-local - Update environment variable prefix: LMGMT_* → OTDF_LOCAL_* - Update all imports and documentation - Preserve git history with git mv BREAKING CHANGE: Command changed from 'lmgmt' to 'otdf-local' Environment variables now use OTDF_LOCAL_ prefix instead of LMGMT_ Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

sonarqubecloud · 2026-02-12T22:28:32Z

Quality Gate failed

Failed conditions
2 Security Hotspots
C Reliability Rating on New Code (required ≥ A)
E Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

gemini-code-assist bot reviewed Jan 27, 2026

View reviewed changes

dmihalcik-virtru force-pushed the chore/the-claudiest-day-tmux branch from 7dfa6b5 to 41fc3fe Compare January 29, 2026 22:05

dmihalcik-virtru force-pushed the chore/the-claudiest-day-tmux branch 3 times, most recently from 8d79d5c to 4374e9c Compare February 11, 2026 15:21

dmihalcik-virtru and others added 23 commits February 12, 2026 10:21

fix(xtest): Prevent subprocess deadlock and improve error handling

2c4338e

- Fix subprocess deadlock in otdfctl by properly handling stdout/stderr - Capture stderr output for better debugging when CLI calls fail - Add env command to lmgmt for outputting pytest environment variables

chore: Config and dependency updates

5c780e0

- Add .shfmt configuration for shell script formatting - Update .gitignore for generated files - Update xtest dependencies and manifest schema

chore(xtest): fix lmgmt for platforms <=0.9

c6f54a8

chore(xtest): default RSA key mapping

dc8f9f5

chore(xtest): xdist worksteal

278814f

chore(lmgmt): better error around lmgmt provision

114bf0b

chore(xtest): More careful fixture creation

5f71639

style: format and fix E501 line length issues

1dcac78

Update test_audit_logs_integration.py

90b797c

chore(xtest): Adds 'cancel' audit event tests

a49fcb0

ruff format

b1dd736

style: move import_module to top-level imports in configure.py

10df6ea

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Update list-versions.py

a57e409

dmihalcik-virtru force-pushed the chore/the-claudiest-day-tmux branch from e7eccfb to a57e409 Compare February 12, 2026 15:28

dmihalcik-virtru and others added 11 commits February 12, 2026 10:31

fix(xtest): upgrade cryptography to patch subgroup attack vulnerability

c1cd82e

Upgrade cryptography from 46.0.3 to 46.0.5+ to address CVE-2024-9287 (Subgroup Attack Due to Missing Subgroup Validation for SECT Curves). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fix(xtest): uv sync for sdk/apps

ce2d343

Revert "fix(xtest): uv sync for sdk/apps"

9f3f0c4

This reverts commit ce2d343.

chore(xtest): fix lint and pyright issues

e5cd528

cleanups

a623634

docs: move otdf-sdk-mgr content out of root CLAUDE.md into README

5f0e30d

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Update uv.lock

2065bf9

dmihalcik-virtru force-pushed the chore/the-claudiest-day-tmux branch from 57f8707 to b9ee3de Compare February 12, 2026 21:44

dmihalcik-virtru and others added 4 commits February 12, 2026 17:27

dmihalcik-virtru force-pushed the chore/the-claudiest-day-tmux branch from bd3b1a7 to 18d9cea Compare February 12, 2026 22:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(xtest): Audit and local improvements together#390

chore(xtest): Audit and local improvements together#390
dmihalcik-virtru wants to merge 38 commits intoopentdf:mainfrom
dmihalcik-virtru:chore/the-claudiest-day-tmux

dmihalcik-virtru commented Jan 27, 2026

Uh oh!

gemini-code-assist bot commented Jan 27, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Jan 27, 2026

Uh oh!

gemini-code-assist bot Jan 27, 2026

Uh oh!

sonarqubecloud bot commented Feb 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

dmihalcik-virtru commented Jan 27, 2026

Uh oh!

gemini-code-assist bot commented Jan 27, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Feb 12, 2026

Quality Gate failed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant