Skip to content

Commit 442de27

Browse files
author
Kristopher Turner
committed
docs: restructure tools section for standalone test clarity
- Add tools/index.md overview page with selection flowchart and lifecycle table - Add per-tool index.md pages (vmfleet, fio, iperf, hammerdb, stress-ng) with standalone 'Run This Test' instructions — no CI/CD required - Rewrite docs/index.md: standalone callout, correct script paths, link to per-tool pages - Reframe operations/ci-cd.md as optional automation with 'When to Use' comparison table - Update mkdocs.yml nav: Tools section now has Overview index, Getting Started expanded - Update overview.md files with redirect notices pointing to new index.md pages - Fix all broken/stale overview.md cross-references across getting-started docs - Add docs/assets/images/ directory for draw.io diagram PNG exports
1 parent 93cddcb commit 442de27

20 files changed

Lines changed: 826 additions & 78 deletions

File tree

docs/assets/images/.gitkeep

Whitespace-only changes.

docs/getting-started/architecture.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -65,11 +65,11 @@ Each phase is checkpoint-tracked via `StateManager`, enabling resume after failu
6565

6666
See individual tool guides for tool-specific workflow details:
6767

68-
- [VMFleet Workflow](../tools/vmfleet/overview.md)
69-
- [fio Workflow](../tools/fio/overview.md)
70-
- [iPerf3 Workflow](../tools/iperf/overview.md)
71-
- [HammerDB Workflow](../tools/hammerdb/overview.md)
72-
- [stress-ng Workflow](../tools/stress-ng/overview.md)
68+
- [VMFleet Workflow](../tools/vmfleet/index.md)
69+
- [fio Workflow](../tools/fio/index.md)
70+
- [iPerf3 Workflow](../tools/iperf/index.md)
71+
- [HammerDB Workflow](../tools/hammerdb/index.md)
72+
- [stress-ng Workflow](../tools/stress-ng/index.md)
7373

7474
## Network Topology
7575

docs/getting-started/configuration.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,6 @@ See [Credential Management](../operations/credential-management.md) for full det
155155

156156
Choose a tool to get started:
157157

158-
- [VMFleet Guide](../tools/vmfleet/overview.md) — Storage load testing
159-
- [fio Guide](../tools/fio/overview.md) — Flexible I/O benchmarking
160-
- [iPerf3 Guide](../tools/iperf/overview.md) — Network testing
158+
- [VMFleet Guide](../tools/vmfleet/index.md) — Storage load testing
159+
- [fio Guide](../tools/fio/index.md) — Flexible I/O benchmarking
160+
- [iPerf3 Guide](../tools/iperf/index.md) — Network testing

docs/getting-started/installation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,4 +123,4 @@ All tests should pass, confirming modules load correctly and configuration is va
123123
## Next Steps
124124

125125
- [Configuration](configuration.md) — Understand the configuration system
126-
- [VMFleet Guide](../tools/vmfleet/overview.md) — Start with VMFleet storage testing
126+
- [VMFleet Guide](../tools/vmfleet/index.md) — Start with VMFleet storage testing

docs/getting-started/introduction.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -17,11 +17,13 @@ Azure Local clusters represent significant infrastructure investments. Before co
1717

1818
| Tool | Purpose | Status |
1919
|------|---------|--------|
20-
| [VMFleet](../tools/vmfleet/overview.md) | Storage load generation via VM fleet running DiskSpd workloads | ✅ Fully Implemented |
21-
| [fio](../tools/fio/overview.md) | Fine-grained storage I/O benchmarking (Linux VMs) | ✅ Implemented |
22-
| [iPerf3](../tools/iperf/overview.md) | Network bandwidth and latency testing | ✅ Implemented |
23-
| [HammerDB](../tools/hammerdb/overview.md) | Database workload benchmarking (OLTP/OLAP) | ✅ Implemented |
24-
| [stress-ng](../tools/stress-ng/overview.md) | CPU, memory, and system stress testing | ✅ Implemented |
20+
| [VMFleet](../tools/vmfleet/index.md) | Storage load generation via VM fleet running DiskSpd workloads | :white_check_mark: Fully Implemented |
21+
| [fio](../tools/fio/index.md) | Fine-grained storage I/O benchmarking (Linux VMs) | :construction: Structure Ready |
22+
| [iPerf3](../tools/iperf/index.md) | Network bandwidth and latency testing | :construction: Structure Ready |
23+
| [HammerDB](../tools/hammerdb/index.md) | Database workload benchmarking (OLTP/OLAP) | :construction: Structure Ready |
24+
| [stress-ng](../tools/stress-ng/index.md) | CPU, memory, and system stress testing | :construction: Structure Ready |
25+
26+
Each tool is standalone — pick one and follow the "Run This Test" instructions on its page. No CI/CD pipeline is required.
2527

2628
## Key Features
2729

docs/getting-started/prerequisites.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -81,10 +81,10 @@ This will check:
8181
Each tool may have additional requirements beyond the common set above:
8282

8383
- [VMFleet Prerequisites](../tools/vmfleet/prerequisites.md) — Base VHD, Collect volume, VMFleet module
84-
- [fio Prerequisites](../tools/fio/overview.md) — Linux VMs, fio package
85-
- [iPerf3 Prerequisites](../tools/iperf/overview.md) — iPerf3 binaries, RDMA configuration
86-
- [HammerDB Prerequisites](../tools/hammerdb/overview.md) — SQL Server, HammerDB installation
87-
- [stress-ng Prerequisites](../tools/stress-ng/overview.md) — Linux VMs, stress-ng package
84+
- [fio Prerequisites](../tools/fio/index.md) — Linux VMs, fio package
85+
- [iPerf3 Prerequisites](../tools/iperf/index.md) — iPerf3 binaries, RDMA configuration
86+
- [HammerDB Prerequisites](../tools/hammerdb/index.md) — SQL Server, HammerDB installation
87+
- [stress-ng Prerequisites](../tools/stress-ng/index.md) — Linux VMs, stress-ng package
8888

8989
## Next Steps
9090

docs/index.md

Lines changed: 38 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -2,50 +2,69 @@
22

33
Automated performance and load testing for Azure Local clusters — storage, network, database, and system stress — with standardised reporting.
44

5+
!!! tip "Each test is standalone"
6+
Every tool in this framework runs independently from a PowerShell terminal. No CI/CD pipeline is required. Pick a tool, follow the steps on its page, and run your test.
7+
58
## Quick Start
69

710
```powershell
811
# 1. Clone and configure
912
git clone https://github.com/AzureLocal/azurelocal-loadtools
13+
cd azurelocal-loadtools
1014
Copy-Item config\variables.example.yml config\variables.yml
1115
# Edit config\variables.yml with your cluster details
1216
13-
# 2. Run a storage benchmark
14-
.\scripts\Start-FioTest.ps1 -RunId "fio-$(Get-Date -f yyyyMMddHHmm)" -Profile "sequential-read"
15-
.\scripts\Collect-FioResults.ps1 -RunId "fio-$(Get-Date -f yyyyMMddHHmm)"
17+
# 2. Run a VMFleet storage benchmark (fully implemented)
18+
.\tools\vmfleet\Invoke-VMFleetPipeline.ps1 `
19+
-ConfigPath "config\variables.yml" `
20+
-Profiles @("General") `
21+
-CredentialSource Interactive `
22+
-GenerateReports
1623
17-
# 3. Generate a report
18-
.\scripts\New-LoadReport.ps1 -RunId "fio-$(Get-Date -f yyyyMMddHHmm)" -Tool fio
24+
# Or run individual steps for any tool:
25+
.\tools\vmfleet\scripts\Install-VMFleet.ps1 -ClusterName "hci01.corp.infiniteimprobability.com" -Nodes @("hci01-node1")
26+
.\tools\vmfleet\scripts\Start-VMFleetTest.ps1 -Profile "General" -DurationSeconds 600
27+
.\tools\vmfleet\scripts\Collect-VMFleetResults.ps1 -Nodes @("hci01-node1") -RunId "<run-id>"
1928
```
2029

21-
## Architecture at a Glance
30+
## Pick a Test
31+
32+
Each tool targets a specific performance domain. Click a tool to see exactly how to run it.
33+
34+
| Tool | What It Tests | Target OS | Status |
35+
|------|--------------|-----------|--------|
36+
| [VMFleet](tools/vmfleet/index.md) | Storage IOPS, throughput, latency via DiskSpd VM fleet | Windows (HCI host) | :white_check_mark: Fully Implemented |
37+
| [fio](tools/fio/index.md) | Fine-grained storage I/O benchmarking | Linux VMs | :construction: Structure Ready |
38+
| [iPerf3](tools/iperf/index.md) | Network bandwidth, jitter, packet loss | Linux / Windows | :construction: Structure Ready |
39+
| [HammerDB](tools/hammerdb/index.md) | SQL Server OLTP / OLAP benchmarking | Windows VMs | :construction: Structure Ready |
40+
| [stress-ng](tools/stress-ng/index.md) | CPU, memory, and system stress testing | Linux VMs | :construction: Structure Ready |
2241

23-
The framework is organised into five layers:
42+
For a detailed comparison and selection guide, see [Tools Overview](tools/index.md).
43+
44+
## How It Works
45+
46+
Every tool follows the same three-step pattern:
2447

2548
```
26-
ConfigurationAutomation → Execution → Monitoring → Reporting
49+
InstallRun Test → Collect Results
2750
```
2851

29-
All scripts consume ConfigManager-generated JSON (never the raw YAML). Results flow from target nodes via SSH/SCP or WinRM to JSON aggregates, then into AsciiDoc report templates. See the [Architecture Overview](architecture/overview.md) for the full breakdown.
52+
All scripts are in `tools/<tool>/scripts/` and can be called directly from PowerShell. No pipeline setup required.
53+
54+
For the full architecture breakdown, see [Architecture Overview](architecture/overview.md).
3055

31-
## Tools
56+
## Want to Automate?
3257

33-
| Tool | Target OS | Category | Status | Profiles |
34-
|------|-----------|----------|--------|---------|
35-
| [fio](tools/fio/overview.md) | Linux | Storage I/O | ![Implemented](https://img.shields.io/badge/-Implemented-brightgreen?style=flat-square) | 5 |
36-
| [iPerf3](tools/iperf/overview.md) | Linux / Windows | Network | ![Implemented](https://img.shields.io/badge/-Implemented-brightgreen?style=flat-square) | 3 |
37-
| [HammerDB](tools/hammerdb/overview.md) | Windows | Database | ![Implemented](https://img.shields.io/badge/-Implemented-brightgreen?style=flat-square) | 2 |
38-
| [stress-ng](tools/stress-ng/overview.md) | Linux | CPU / Memory / I/O | ![Implemented](https://img.shields.io/badge/-Implemented-brightgreen?style=flat-square) | 3 |
39-
| [VMFleet](tools/vmfleet/overview.md) | Windows (HCI) | VM fleet | ![Implemented](https://img.shields.io/badge/-Implemented-brightgreen?style=flat-square) ||
58+
CI/CD pipelines are available as an **optional addition** if you want to run tests on a schedule, trigger them from pull requests, or integrate into your deployment workflow. See [CI/CD Pipelines](operations/ci-cd.md).
4059

4160
## Navigation
4261

4362
| Section | Description |
4463
|---------|-------------|
4564
| [Getting Started](getting-started/introduction.md) | Prerequisites, installation, and first run |
65+
| [Tools Overview](tools/index.md) | All tools at a glance with selection guide |
4666
| [Architecture](architecture/overview.md) | Five-layer stack, tool selection, data flow |
47-
| [Tools](tools/fio/overview.md) | Per-tool installation, profiles, monitoring, reporting, troubleshooting |
48-
| [Operations](operations/ci-cd.md) | CI/CD pipelines, runner setup, troubleshooting |
67+
| [Operations](operations/ci-cd.md) | CI/CD pipelines (optional), runner setup, troubleshooting |
4968
| [Reference](reference/cmdlet-reference.md) | Cmdlet reference, variables, tool comparison |
5069
| [Roadmap](roadmap.md) | Milestone tracker and planned features |
5170

docs/operations/ci-cd.md

Lines changed: 51 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,48 @@
1-
# CI/CD Pipelines
1+
# CI/CD Pipelines (Optional Automation)
22

33
![Category: Operations](https://img.shields.io/badge/Category-Operations-9B59B6?style=flat-square)
44

5-
The framework includes pipeline definitions for GitHub Actions (primary), Azure DevOps, and GitLab CI.
5+
!!! info "Pipelines are optional"
6+
Every test in this framework can be run standalone from a PowerShell terminal. CI/CD pipelines are an **optional addition** for teams that want to automate, schedule, or integrate load testing into their deployment workflows.
67

7-
## GitHub Actions (Primary)
8+
The framework includes pipeline definitions for GitHub Actions (primary), Azure DevOps, and GitLab CI. These pipelines call the **same scripts** you would run manually — they are not a separate system.
89

9-
### Documentation Build (`build-docs.yml`)
10+
## When to Use CI/CD
1011

11-
Triggers on push to `main` and pull requests. Builds HTML and PDF documentation, exports draw.io diagrams, and publishes artifacts.
12+
| Use Case | Standalone | CI/CD |
13+
|----------|-----------|-------|
14+
| One-off test run from your workstation | :white_check_mark: | |
15+
| Scheduled nightly performance regression | | :white_check_mark: |
16+
| Triggered by infrastructure changes (PR) | | :white_check_mark: |
17+
| Automated reporting after every test | | :white_check_mark: |
18+
| Learning or debugging the framework | :white_check_mark: | |
19+
| Production validation before go-live | :white_check_mark: | :white_check_mark: |
1220

13-
### Configuration Validation (`validate-config.yml`)
21+
## Running Tests Without CI/CD
22+
23+
Every pipeline action can be run manually from a PowerShell terminal. See each tool's page for exact commands:
24+
25+
- [VMFleet — Run This Test](../tools/vmfleet/index.md#run-this-test)
26+
- [fio — Run This Test](../tools/fio/index.md#run-this-test)
27+
- [iPerf3 — Run This Test](../tools/iperf/index.md#run-this-test)
28+
- [HammerDB — Run This Test](../tools/hammerdb/index.md#run-this-test)
29+
- [stress-ng — Run This Test](../tools/stress-ng/index.md#run-this-test)
30+
31+
Or use the common utility scripts directly:
32+
33+
```powershell
34+
# Validate your configuration
35+
.\common\helpers\Initialize-Environment.ps1 -ValidateOnly
36+
37+
# Run Pester unit tests
38+
Invoke-Pester tests/ -Output Detailed
39+
```
40+
41+
---
1442

15-
Triggers on changes to `config/`. Validates `master-environment.yml` against the JSON Schema and verifies solution config generation.
43+
## GitHub Actions (Primary)
44+
45+
If you choose to automate, the following workflows are available:
1646

1747
### VMFleet Pipeline (`run-vmfleet.yml`)
1848

@@ -30,47 +60,36 @@ Requires a self-hosted runner with network access to the Azure Local cluster.
3060

3161
Runs Pester unit tests for all core PowerShell modules on every pull request.
3262

33-
### Linting (`lint.yml`)
34-
35-
Runs PSScriptAnalyzer for PowerShell, yamllint for YAML, and AsciiDoc syntax checks.
63+
### Configuration Validation (`validate-config.yml`)
3664

37-
## Azure DevOps (Placeholder)
65+
Triggers on changes to `config/`. Validates `variables.yml` against the JSON Schema and verifies solution config generation.
3866

39-
Pipeline definitions in `.azuredevops/pipelines/` mirror the GitHub Actions workflows, adapted for Azure DevOps syntax. Use Azure DevOps Service Connections for credential management.
67+
### Documentation Build (`deploy-docs.yml`)
4068

41-
## GitLab CI (Placeholder)
69+
Triggers on push to `main` and pull requests. Builds MkDocs HTML documentation and publishes to GitHub Pages.
4270

43-
Pipeline definition in `.gitlab/.gitlab-ci.yml` provides the same workflow stages, adapted for GitLab CI syntax.
71+
### Linting (`lint.yml`)
4472

45-
## Manual Execution
73+
Runs PSScriptAnalyzer for PowerShell, yamllint for YAML files.
4674

47-
Every pipeline action can also be run manually from a workstation:
75+
## Azure DevOps
4876

49-
```powershell
50-
# Build documentation
51-
asciidoctor-pdf docs/main.adoc -o output/azurelocal-loadtools.pdf
77+
Pipeline definitions in `.azuredevops/` mirror the GitHub Actions workflows, adapted for Azure DevOps syntax. Use Azure DevOps Service Connections for credential management.
5278

53-
# Validate configuration
54-
.\src\core\powershell\helpers\Initialize-Environment.ps1 -ValidateOnly
79+
## GitLab CI
5580

56-
# Run VMFleet pipeline
57-
.\src\solutions\vmfleet\orchestrator\Invoke-VMFleetPipeline.ps1 `
58-
-ClusterConfig "config/clusters/my-cluster.yml" `
59-
-Profiles @("General", "Peak") `
60-
-CredentialSource Interactive
81+
Pipeline definition in `.gitlab/.gitlab-ci.yml` provides the same workflow stages, adapted for GitLab CI syntax.
6182

62-
# Run unit tests
63-
Invoke-Pester tests/unit/ -Output Detailed
64-
```
83+
---
6584

66-
## Self-Hosted Runner Setup
85+
## Self-Hosted Runner Requirements
6786

68-
The VMFleet pipeline requires a self-hosted runner on the cluster management network:
87+
The VMFleet and tool-specific pipelines require a self-hosted runner on the cluster management network:
6988

7089
1. Install GitHub Actions runner on a Windows management station
7190
2. Register with the repository as a self-hosted runner
7291
3. Tag with labels: `self-hosted`, `windows`, `hci-management`
7392
4. Ensure PowerShell 7.2+ and required modules are installed
7493
5. Configure WinRM access to cluster nodes
7594

76-
See [GitHub Actions self-hosted runners documentation](https://docs.github.com/en/actions/hosting-your-own-runners) for setup instructions.
95+
See [Runner Setup](runner-setup.md) for detailed instructions.

docs/tools/fio/index.md

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# fio — Flexible I/O Tester
2+
3+
![Tool: fio](https://img.shields.io/badge/Tool-fio-6C3483?style=flat-square)
4+
![Status: Structure Ready](https://img.shields.io/badge/Status-Structure%20Ready-yellow?style=flat-square)
5+
6+
fio provides fine-grained storage I/O benchmarking with precise control over block sizes, queue depths, I/O engines, and workload patterns. It runs inside Linux VMs on the Azure Local cluster via Ansible deployment.
7+
8+
---
9+
10+
## Run This Test
11+
12+
### Prerequisites
13+
14+
- Linux VMs deployed on your Azure Local cluster
15+
- Ansible 2.14+ installed on your management workstation
16+
- SSH key-based access to Linux VMs
17+
- [Framework prerequisites](../../getting-started/prerequisites.md) completed
18+
19+
### Step 1 — Install fio on target VMs
20+
21+
```powershell
22+
.\tools\fio\scripts\Install-Fio.ps1 `
23+
-ClusterName "hci01.corp.infiniteimprobability.com" `
24+
-Nodes @("hci01-node1", "hci01-node2")
25+
```
26+
27+
This triggers an Ansible playbook that installs the `fio` package on the specified Linux VMs.
28+
29+
### Step 2 — Run a workload profile
30+
31+
```powershell
32+
.\tools\fio\scripts\Start-FioTest.ps1 `
33+
-ClusterName "hci01.corp.infiniteimprobability.com" `
34+
-Nodes @("hci01-node1", "hci01-node2") `
35+
-Profile "random-read" `
36+
-DurationSeconds 300
37+
```
38+
39+
### Step 3 — Collect results
40+
41+
```powershell
42+
.\tools\fio\scripts\Collect-FioResults.ps1 `
43+
-ClusterName "hci01.corp.infiniteimprobability.com" `
44+
-Nodes @("hci01-node1", "hci01-node2") `
45+
-RunId "<run-id-from-previous-step>"
46+
```
47+
48+
---
49+
50+
## When to Use fio vs VMFleet
51+
52+
| Scenario | Use fio | Use VMFleet |
53+
|----------|---------|-------------|
54+
| Linux VM storage performance | :white_check_mark: | |
55+
| Fine-grained I/O parameter control | :white_check_mark: | |
56+
| Fleet-scale Windows HCI storage load | | :white_check_mark: |
57+
| Data integrity verification | :white_check_mark: | |
58+
| I/O scheduler / queue depth testing | :white_check_mark: | |
59+
60+
## Key Capabilities
61+
62+
- **Job file-based** workload definition for reproducible tests
63+
- **Multiple I/O engines** — libaio, io_uring, Windows IOCP
64+
- **JSON output** for automated parsing and reporting
65+
- **Verification mode** for data integrity testing
66+
- **Client/server mode** for distributed I/O testing across nodes
67+
68+
---
69+
70+
## What You Get
71+
72+
After a test completes, you will have:
73+
74+
- **JSON output** per node with IOPS, bandwidth (KB/s), and latency percentiles
75+
- **Aggregated metrics** across all nodes
76+
- **Reports** in PDF, DOCX, and/or XLSX format
77+
78+
---
79+
80+
## File Locations
81+
82+
| Component | Path |
83+
|-----------|------|
84+
| Scripts | `tools/fio/scripts/` |
85+
| Workload Profiles | `tools/fio/config/profiles/` |
86+
| Ansible Playbooks | `tools/fio/playbooks/` |
87+
| Alert Rules | `tools/fio/monitoring/alert-rules.yml` |
88+
| Report Templates | `tools/fio/reports/templates/` |
89+
| Tests | `tools/fio/tests/` |
90+
| Logs | `tools/fio/logs/` |
91+
92+
---
93+
94+
## Deep Dive
95+
96+
| Topic | Page |
97+
|-------|------|
98+
| Ansible deployment details | [Installation](installation.md) |
99+
| Profile parameters and tuning | [Workload Profiles](workload-profiles.md) |
100+
| Alert rules during test runs | [Monitoring](monitoring.md) |
101+
| Report formats and templates | [Reporting](reporting.md) |
102+
| Common errors and fixes | [Troubleshooting](troubleshooting.md) |
103+
104+
---
105+
106+
## Automate This Test
107+
108+
To run fio tests on a schedule or as part of a CI/CD workflow, see [CI/CD Pipelines](../../operations/ci-cd.md). The pipeline calls the same scripts listed above.

docs/tools/fio/overview.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
# fio — Flexible I/O Tester
22

3+
!!! note "This page has moved"
4+
The main fio landing page with run instructions is now at [fio](index.md). This page contains additional technical details.
5+
36
![Tool: fio](https://img.shields.io/badge/Tool-fio-6C3483?style=flat-square)
47
![Status: Implemented](https://img.shields.io/badge/Status-Implemented-brightgreen?style=flat-square)
58

0 commit comments

Comments
 (0)