Skip to content

Commit ea5b89b

Browse files
josecelanojosecelano
committed
feat: [#339] add AI training outputs with complete input/output dataset
Implement Option B: Generate rendered deployment artifacts for all AI training examples to complete the dataset with input/output pairs. Added: - Script: scripts/generate-ai-training-outputs.sh * Iterates over all 15 example configurations * Replaces generic SSH paths with fixture paths dynamically * Calls render command with placeholder IP (203.0.113.1) * Generates artifacts to docs/ai-training/outputs/ - Outputs: docs/ai-training/outputs/ (4.1 MB, 15 subdirectories) * Complete rendered templates for all examples * Each output includes: Ansible playbooks, Docker Compose, configuration files (tracker.toml, prometheus.yml, etc.) * Demonstrates deployer transformation: config → artifacts - Documentation: docs/ai-training/outputs/README.md * Explains purpose and structure of outputs * Documents placeholder values used * Provides regeneration instructions Updated: - docs/ai-training/README.md: Added outputs section explaining complete input/output pairs for AI training Benefits: - AI agents can learn full input/output transformation patterns - Users can see concrete examples of what gets deployed - Diff analysis reveals how config options affect rendered templates - Complete dataset for few-shot learning and pattern recognition Script features: - Validates fixture SSH keys exist - Uses jq for JSON manipulation - Detailed progress reporting - Error handling with failure summary - --clean flag for regeneration - --help for usage documentation All linters passing (markdown, yaml, toml, cspell, clippy, rustfmt, shellcheck)
1 parent 06165ca commit ea5b89b

498 files changed

Lines changed: 76336 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/ai-training/README.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ This directory contains resources to help AI agents guide users through creating
66

77
- **[questionnaire.md](questionnaire.md)** - Structured decision tree for gathering user requirements
88
- **[examples/](examples/)** - 15 pre-configured environment examples demonstrating common deployment scenarios
9+
- **[outputs/](outputs/)** - Rendered deployment artifacts for all examples (complete input/output pairs)
910

1011
## Purpose
1112

@@ -16,6 +17,50 @@ These materials enable AI agents to:
1617
3. **Explain configuration tradeoffs** between different deployment options
1718
4. **Help users customize** example configurations for their specific needs
1819

20+
## Dataset Structure: Input/Output Pairs
21+
22+
This directory contains a **complete AI training dataset** with both inputs and outputs:
23+
24+
### Input: Environment Configurations
25+
26+
- Location: [`examples/`](examples/)
27+
- Format: JSON configuration files
28+
- Content: High-level deployment requirements (provider, database, domains, etc.)
29+
30+
### Output: Rendered Deployment Artifacts
31+
32+
- Location: [`outputs/`](outputs/)
33+
- Format: Rendered templates (Ansible, Docker Compose, configuration files)
34+
- Content: Concrete artifacts ready for deployment
35+
36+
### The Deployer as a Function
37+
38+
```text
39+
config.json → [Deployer] → Rendered Templates
40+
(IN) (OUT)
41+
```
42+
43+
The deployer transforms:
44+
45+
- **FROM**: Desired infrastructure state in custom domain language
46+
- **TO**: Executable deployment artifacts
47+
48+
### Example Mapping
49+
50+
| Input (Config) | Output (Artifacts) |
51+
| -------------------------------------------------------------- | ----------------------------------------------------- |
52+
| [`examples/01-minimal-lxd.json`](examples/01-minimal-lxd.json) | [`outputs/01-minimal-lxd/`](outputs/01-minimal-lxd/) |
53+
| SQLite + UDP + HTTP + LXD | Ansible playbooks, tracker.toml, docker-compose, etc. |
54+
55+
**Benefits for AI Agents:**
56+
57+
- **Few-shot learning**: Full input/output examples show transformation patterns
58+
- **Pattern recognition**: See how config changes affect rendered templates
59+
- **Diff analysis**: Compare outputs to understand configuration impact
60+
- **Template understanding**: Learn the structure of deployment artifacts
61+
62+
**Regenerating Outputs**: Run `./scripts/generate-ai-training-outputs.sh` to update artifacts when templates change.
63+
1964
## Using the Questionnaire
2065

2166
The [questionnaire.md](questionnaire.md) provides a systematic approach to gather user requirements across 14 key areas:

‎docs/ai-training/outputs/01-minimal-lxd/ansible/ansible.cfg‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# ============================================================================
2+
# Torrust Tracker Deployer - Generated Configuration
3+
# ============================================================================
4+
#
5+
# This file was generated by the Torrust Tracker Deployer.
6+
#
7+
# DOCUMENTATION:
8+
# Repository: https://github.com/torrust/torrust-tracker-deployer
9+
# Template: templates/ansible/ansible.cfg
10+
# API Docs: https://docs.rs/torrust-tracker-deployer/latest/
11+
#
12+
# DESCRIPTION:
13+
# Ansible global configuration settings for connecting to provisioned VMs.
14+
# Configures SSH connection settings and output formatting.
15+
#
16+
# For configuration options and valid values, see the API documentation link above.
17+
# ============================================================================
18+
19+
# 🔗 INFRASTRUCTURE WORKFLOW:
20+
# 1. OpenTofu (templates/tofu/lxd/) provisions VMs with cloud-init
21+
# 2. Cloud-init sets up users, SSH keys, and basic system configuration
22+
# 3. Ansible (this directory) connects to provisioned VMs for management tasks
23+
# 4. This config file ensures Ansible can reliably connect to OpenTofu-created VMs
24+
25+
[defaults]
26+
# Specify the default inventory file location
27+
# This tells Ansible where to find the list of hosts to manage
28+
# 🔗 The inventory.yml contains IPs of VMs created by OpenTofu
29+
inventory = inventory.yml
30+
31+
# Disable SSH host key checking for lab/development environments
32+
# This prevents SSH from asking "Are you sure you want to connect?" prompts
33+
# ⚠️ IMPORTANT: OpenTofu creates fresh VMs with new SSH host keys each time
34+
# WARNING: Only use this in trusted lab environments, not production
35+
host_key_checking = False
36+
37+
# Use debug callback plugin for better output formatting
38+
# This provides cleaner, more readable output from Ansible commands
39+
stdout_callback = debug
40+
stderr_callback = debug
41+
42+
# Enable progress display for long-running tasks
43+
# Shows task progress and timing information
44+
callback_enabled = timer, profile_tasks
45+
46+
# Display task timing information
47+
show_task_path_on_failure = True
48+
49+
# Set connection timeout in seconds
50+
# How long to wait for SSH connections before giving up
51+
timeout = 30
52+
53+
[ssh_connection]
54+
# SSH connection optimization arguments
55+
# These settings improve SSH connection performance and reliability:
56+
# - ControlMaster=auto: Reuse SSH connections for multiple commands
57+
# - ControlPersist=60s: Keep connections alive for 60 seconds after last use
58+
# - StrictHostKeyChecking=no: Don't verify SSH host keys (lab environment only)
59+
# - UserKnownHostsFile=/dev/null: Don't save host keys to known_hosts file
60+
ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null

‎docs/ai-training/outputs/01-minimal-lxd/ansible/configure-firewall.yml‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# ============================================================================
2+
# Torrust Tracker Deployer - Generated Configuration
3+
# ============================================================================
4+
#
5+
# This file was generated by the Torrust Tracker Deployer.
6+
#
7+
# DOCUMENTATION:
8+
# Repository: https://github.com/torrust/torrust-tracker-deployer
9+
# Template: templates/ansible/configure-firewall.yml
10+
# API Docs: https://docs.rs/torrust-tracker-deployer/latest/
11+
#
12+
# DESCRIPTION:
13+
# Ansible playbook to configure UFW firewall rules for SSH access.
14+
# Ensures safe SSH connectivity before enabling firewall.
15+
#
16+
# For configuration options and valid values, see the API documentation link above.
17+
# ============================================================================
18+
19+
---
20+
# IMPORTANT SECURITY NOTE:
21+
# =======================
22+
# This playbook ONLY configures SSH firewall rules. Application service ports
23+
# (tracker, grafana, prometheus, etc.) are NOT controlled by UFW because Docker
24+
# bypasses UFW rules when publishing container ports.
25+
#
26+
# Docker Security Model:
27+
# - Docker manipulates iptables NAT table directly, bypassing UFW's INPUT/OUTPUT chains
28+
# - Service exposure is controlled via docker-compose port bindings, not UFW
29+
# - Internal services (MySQL, Prometheus) have NO port bindings - Docker network only
30+
# - Public services (Tracker, Grafana) have explicit port bindings in docker-compose
31+
#
32+
# For details, see ADR: docs/decisions/docker-ufw-firewall-security-strategy.md
33+
#
34+
# This playbook configures UFW with restrictive policies while preserving SSH access.
35+
# CRITICAL: SSH access is allowed BEFORE enabling firewall to prevent lockout.
36+
#
37+
# Variables are loaded from variables.yml for centralized management.
38+
39+
- name: Configure UFW firewall safely (SSH access only)
40+
hosts: all
41+
become: yes
42+
gather_facts: yes
43+
vars_files:
44+
- variables.yml
45+
46+
tasks:
47+
- name: Install UFW (should already be present on Ubuntu)
48+
ansible.builtin.apt:
49+
name: ufw
50+
state: present
51+
update_cache: yes
52+
tags:
53+
- security
54+
- firewall
55+
- packages
56+
57+
- name: Reset UFW to clean state
58+
community.general.ufw:
59+
state: reset
60+
tags:
61+
- security
62+
- firewall
63+
- reset
64+
65+
- name: Set UFW default policy - deny incoming
66+
community.general.ufw:
67+
default: deny
68+
direction: incoming
69+
tags:
70+
- security
71+
- firewall
72+
- policy
73+
74+
- name: Set UFW default policy - allow outgoing
75+
community.general.ufw:
76+
default: allow
77+
direction: outgoing
78+
tags:
79+
- security
80+
- firewall
81+
- policy
82+
83+
# CRITICAL: Allow SSH BEFORE enabling firewall to prevent lockout
84+
- name: Allow SSH access on configured port (BEFORE enabling firewall)
85+
community.general.ufw:
86+
rule: allow
87+
port: "{{ ssh_port }}"
88+
proto: tcp
89+
comment: "SSH access (configured port {{ ssh_port }})"
90+
tags:
91+
- security
92+
- firewall
93+
- ssh
94+
95+
- name: Enable UFW firewall (AFTER SSH rules are in place)
96+
community.general.ufw:
97+
state: enabled
98+
tags:
99+
- security
100+
- firewall
101+
- enable
102+
103+
- name: Verify UFW status
104+
ansible.builtin.command:
105+
cmd: ufw status numbered
106+
register: ufw_status
107+
changed_when: false
108+
tags:
109+
- security
110+
- firewall
111+
- verification
112+
113+
- name: Display UFW status
114+
ansible.builtin.debug:
115+
var: ufw_status.stdout_lines
116+
tags:
117+
- security
118+
- firewall
119+
- verification
120+
121+
- name: Verify SSH port is allowed
122+
ansible.builtin.shell:
123+
cmd: "ufw status | grep -E '{{ ssh_port }}/tcp.*ALLOW'"
124+
register: ssh_port_check
125+
changed_when: false
126+
failed_when: ssh_port_check.rc != 0
127+
tags:
128+
- security
129+
- firewall
130+
- verification
131+
- ssh
132+
133+
- name: Confirm firewall configuration complete
134+
ansible.builtin.debug:
135+
msg:
136+
- "UFW firewall configured successfully"
137+
- "SSH access preserved on port {{ ssh_port }}"
138+
- "Default policy: deny incoming, allow outgoing"
139+
- "Active rules protect against unauthorized access"
140+
tags:
141+
- security
142+
- firewall

‎docs/ai-training/outputs/01-minimal-lxd/ansible/configure-security-updates.yml‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# ============================================================================
2+
# Torrust Tracker Deployer - Generated Configuration
3+
# ============================================================================
4+
#
5+
# This file was generated by the Torrust Tracker Deployer.
6+
#
7+
# DOCUMENTATION:
8+
# Repository: https://github.com/torrust/torrust-tracker-deployer
9+
# Template: templates/ansible/configure-security-updates.yml
10+
# API Docs: https://docs.rs/torrust-tracker-deployer/latest/
11+
#
12+
# DESCRIPTION:
13+
# Ansible playbook to configure automatic security updates using unattended-upgrades.
14+
# Schedules automatic reboots at 2:00 AM when updates require restart.
15+
#
16+
# For configuration options and valid values, see the API documentation link above.
17+
# ============================================================================
18+
19+
---
20+
- name: Configure automatic security updates
21+
hosts: all
22+
gather_facts: true
23+
become: true
24+
25+
tasks:
26+
- name: 🔐 Starting automatic security updates configuration
27+
ansible.builtin.debug:
28+
msg: "🚀 Configuring unattended-upgrades on {{ inventory_hostname }}"
29+
30+
- name: Install unattended-upgrades package
31+
ansible.builtin.apt:
32+
name: unattended-upgrades
33+
state: present
34+
update_cache: true
35+
force_apt_get: true
36+
when: ansible_os_family == "Debian"
37+
38+
- name: Enable automatic security updates
39+
ansible.builtin.lineinfile:
40+
path: /etc/apt/apt.conf.d/20auto-upgrades
41+
regexp: "^APT::Periodic::Unattended-Upgrade"
42+
line: 'APT::Periodic::Unattended-Upgrade "1";'
43+
create: true
44+
backup: true
45+
when: ansible_os_family == "Debian"
46+
47+
- name: Enable automatic reboot for security updates
48+
ansible.builtin.lineinfile:
49+
path: /etc/apt/apt.conf.d/50unattended-upgrades
50+
regexp: "^Unattended-Upgrade::Automatic-Reboot"
51+
line: 'Unattended-Upgrade::Automatic-Reboot "true";'
52+
backup: true
53+
when: ansible_os_family == "Debian"
54+
55+
- name: Set automatic reboot time to 2:00 AM
56+
ansible.builtin.lineinfile:
57+
path: /etc/apt/apt.conf.d/50unattended-upgrades
58+
regexp: "^Unattended-Upgrade::Automatic-Reboot-Time"
59+
line: 'Unattended-Upgrade::Automatic-Reboot-Time "02:00";'
60+
backup: true
61+
when: ansible_os_family == "Debian"
62+
63+
- name: Enable and start unattended-upgrades service
64+
ansible.builtin.systemd:
65+
name: unattended-upgrades
66+
enabled: true
67+
state: started
68+
when: ansible_os_family == "Debian"
69+
ignore_errors: true # Ignore in container environments where systemd might not work
70+
71+
- name: Verify unattended-upgrades configuration
72+
ansible.builtin.command:
73+
cmd: unattended-upgrade --dry-run --debug
74+
register: unattended_upgrades_test
75+
changed_when: false
76+
failed_when: false # Don't fail on verification errors in test environments
77+
when: ansible_os_family == "Debian"
78+
79+
- name: Display verification result
80+
ansible.builtin.debug:
81+
msg: "✅ Unattended-upgrades dry-run completed with exit code {{ unattended_upgrades_test.rc }}"
82+
83+
- name: Configuration summary
84+
ansible.builtin.debug:
85+
msg: |
86+
✅ Automatic security updates configuration completed!
87+
📦 Installed: unattended-upgrades
88+
🔄 Automatic updates: Enabled
89+
🕐 Automatic reboot time: 02:00
90+
ℹ️ Security updates will be installed automatically with scheduled reboots

0 commit comments

Comments
 (0)