feat: [#339] AI training resources - config examples and questionnaire#345
Merged
josecelano merged 15 commits intomainfrom Feb 13, 2026
Merged
feat: [#339] AI training resources - config examples and questionnaire#345josecelano merged 15 commits intomainfrom
josecelano merged 15 commits intomainfrom
Conversation
dc494af to
06165ca
Compare
Core examples (01-05): - 01-minimal-lxd: Basic LXD setup with SQLite - 02-full-stack-lxd: Complete LXD with MySQL, HTTPS, monitoring, backups - 03-minimal-hetzner: Basic Hetzner cloud deployment - 04-full-stack-hetzner: Production Hetzner with all features - 05-mysql-development: MySQL-focused development setup Extended examples (06-15): - 06-production-https: Production with HTTPS and monitoring - 07-udp-only: UDP-only minimal tracker - 08-http-only-https: HTTP API-focused with HTTPS - 09-monitoring-stack: Full monitoring without HTTPS complexity - 10-multi-domain: Multi-domain architecture - 11-private-tracker: Private tracker with user whitelisting - 12-high-availability: HA production setup with backups - 13-backup-focused: Aggressive backup strategy - 14-lightweight-production: Cost-effective cloud deployment - 15-sqlite-monitoring: SQLite with full monitoring All configurations include: - Descriptive environment.description fields - Realistic SSH credential paths - Valid schema-compliant structure - Tested with validate command
Created docs/ai-training/README.md with: - Overview of questionnaire and example configurations - Questionnaire workflow with Mermaid diagram - Complete example configuration tables (core + extended) - Scenario-to-example mapping guide - Detailed configuration customization guide - Common decision points (Provider, Database, HTTPS, Monitoring) - AI agent best practices - Troubleshooting section - Related documentation references
Add comprehensive integration test that validates all 15 AI training example configurations. The test suite includes: - Configuration validation using the validate command - File count verification (ensures exactly 15 examples) - Naming convention enforcement (NN-descriptive-name.json pattern) - Description field presence validation This automated test ensures: - All examples remain valid as the schema evolves - CI catches configuration issues before they reach users - Examples follow consistent naming patterns - Every example has a descriptive explanation Dependencies: - Added regex = "1.0" to dev-dependencies for pattern matching
All 6 implementation phases have been successfully completed: Phase 1: Description field added to schema ✓ Phase 2: Questionnaire created ✓ Phase 3: Core 6 examples created ✓ Phase 4: Extended 9 examples created ✓ Phase 5: README documentation created ✓ Phase 6: Integration test created ✓ All acceptance criteria met: - All linters passing - All 15 examples validated - Integration test passes (4 test functions) - Backward compatibility maintained - Comprehensive documentation included
Task 11.3 (AI agent configuration examples and questionnaire) has been fully implemented with all deliverables completed: - Questionnaire template: 494 lines with decision tree - Example configurations: 15 validated JSON scenarios - README documentation: 469 lines with comprehensive guidance - Integration test suite: 4 test functions validating all examples This completes the work for improving AI agent experience with configuration creation.
Implement Option B: Generate rendered deployment artifacts for all AI
training examples to complete the dataset with input/output pairs.
Added:
- Script: scripts/generate-ai-training-outputs.sh
* Iterates over all 15 example configurations
* Replaces generic SSH paths with fixture paths dynamically
* Calls render command with placeholder IP (203.0.113.1)
* Generates artifacts to docs/ai-training/outputs/
- Outputs: docs/ai-training/outputs/ (4.1 MB, 15 subdirectories)
* Complete rendered templates for all examples
* Each output includes: Ansible playbooks, Docker Compose,
configuration files (tracker.toml, prometheus.yml, etc.)
* Demonstrates deployer transformation: config → artifacts
- Documentation: docs/ai-training/outputs/README.md
* Explains purpose and structure of outputs
* Documents placeholder values used
* Provides regeneration instructions
Updated:
- docs/ai-training/README.md: Added outputs section explaining
complete input/output pairs for AI training
Benefits:
- AI agents can learn full input/output transformation patterns
- Users can see concrete examples of what gets deployed
- Diff analysis reveals how config options affect rendered templates
- Complete dataset for few-shot learning and pattern recognition
Script features:
- Validates fixture SSH keys exist
- Uses jq for JSON manipulation
- Detailed progress reporting
- Error handling with failure summary
- --clean flag for regeneration
- --help for usage documentation
All linters passing (markdown, yaml, toml, cspell, clippy, rustfmt, shellcheck)
- examples → environment-configs (user input configurations) - outputs → rendered-templates (generated deployment artifacts) - Updated all path references throughout codebase This improves directory naming to clearly distinguish between: - User-created configuration files (environment-configs) - System-generated template outputs (rendered-templates) Files updated: - scripts/generate-ai-training-outputs.sh - tests/validate_ai_training_examples.rs - docs/ai-training/README.md - docs/ai-training/rendered-templates/README.md - docs/issues/339-provide-config-examples-and-questionnaire-for-ai-agents.md - docs/roadmap.md
- Group inputs and outputs under docs/ai-training/dataset/ - environment-configs/ (user configuration inputs) - rendered-templates/ (generated deployment outputs) This structure makes it explicit that these directories form a complete AI training dataset with input/output pairs. Files updated: - scripts/generate-ai-training-outputs.sh - tests/validate_ai_training_examples.rs (added empty assertions, removed verbose output) - docs/ai-training/README.md - docs/ai-training/dataset/rendered-templates/README.md - docs/issues/339-provide-config-examples-and-questionnaire-for-ai-agents.md - docs/roadmap.md
- Add it_should_render_all_ai_training_example_configurations() test - Validates that all 15 example configs can be rendered into deployment artifacts - Uses tempfile for automatic cleanup of test outputs - Converts fixture paths to absolute paths (required by render command) - Verifies ansible and docker-compose directories are created - Add empty assertions to fail when no examples found - Remove verbose print statements for cleaner test output - Fix clippy warnings (if_not_else, map_clone) This ensures the AI training dataset outputs can actually be generated from the input configurations, not just that configs are valid.
Update remaining path references that were missed in the previous dataset/ reorganization commit: - docs/ai-training/README.md: Update all table paths and examples - docs/ai-training/dataset/rendered-templates/README.md: Update input path reference - docs/issues/339-*.md: Update validation paths - docs/roadmap.md: Update component paths - scripts/generate-ai-training-outputs.sh: Update directory paths in comments All paths now correctly reference dataset/environment-configs and dataset/rendered-templates.
…d helper functions - Replace direct Command usage with ProcessRunner for consistency with E2E tests - Extract helper functions: get_file_name, setup_test_examples, path_to_str, report_failures_and_panic, has_valid_description - Decompose render_configuration into focused functions: prepare_test_config, replace_ssh_credentials_with_fixtures, execute_render_command, verify_render_output_directories - Reduce code duplication and improve readability - Better error messages with contextual information - Improve variable naming (obj -> credentials_map) - Add test support module allow directives for shared utilities Code reduced from 345 to 299 lines (~13% reduction) while improving maintainability.
Document the problem of stale rendered templates in AI training dataset and explore potential solutions including timestamp mocking for deterministic testing. Deferred for future implementation when needed.
d4127ca to
0bd9547
Compare
Member
Author
|
ACK 0bd9547 |
josecelano
added a commit
that referenced
this pull request
Feb 13, 2026
The PR #345 has been merged to main. The issue tracking file is no longer needed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR implements Issue #339, providing structured resources to help AI agents guide users through environment configuration creation.
Implementation Summary
All 6 phases completed:
Phase 1: Schema Extension ✅
descriptionfield to environment configuration schemaschemas/environment-config.json)src/application/command_handlers/create/config/environment_config.rs)Phase 2: Questionnaire Template ✅
docs/ai-training/questionnaire.md(494 lines)Phase 3-4: Example Configurations ✅
cargo run -- validate --env-filePhase 5: Documentation ✅
docs/ai-training/README.md(469 lines)Phase 6: Integration Test ✅
tests/validate_ai_training_examples.rsExample Scenarios Coverage
Benefits
For AI Agents:
For Users:
For Training/RAG:
Testing
cargo run --bin linter allcargo run -- validate --env-filecargo test --test validate_ai_training_examples./scripts/pre-commit.shFiles Changed
schemas/environment-config.json- Added description fieldsrc/application/command_handlers/create/config/environment_config.rs- EnvironmentSection with descriptiondocs/ai-training/questionnaire.md- Decision tree questionnaire (494 lines)docs/ai-training/examples/*.json- 15 validated example configurationsdocs/ai-training/README.md- Comprehensive documentation (469 lines)tests/validate_ai_training_examples.rs- Integration test suiteCargo.toml- Added regex dev-dependencydocs/issues/339-provide-config-examples-and-questionnaire-for-ai-agents.md- Updated progressdocs/roadmap.md- Marked task 11.3 as completedAcceptance Criteria
All criteria met:
Closes #339