Prevent docs drift

[anupddas]

Describe the issue

Summary

The AWS CLI repository contains many user-facing command examples. Over time, these examples can drift from the actual CLI behavior, command names, parameter names, or output formats.

I would like to propose adding a lightweight validation step in CI that checks example files for basic consistency and catches stale or broken examples earlier.

Why this matters

Examples are often the first thing users copy and run. When an example becomes stale, the failure is confusing and creates unnecessary support burden.

A small automated check would help ensure that examples remain:

• syntactically valid
• aligned with current command structure
• consistent with the repository’s documented example conventions
• less likely to regress silently after unrelated changes

Proposed improvement

Add an automated test or validation script that checks example coverage in the repository, for example:

1. Verify example files exist where expected.
2. Verify documented commands still resolve to real subcommands/options.
3. Flag examples that are likely stale, incomplete, or no longer match current CLI behavior.
4. Allow explicit exceptions where an example is intentionally partial or sync-only.

Example approach

A simple first pass could validate file presence and structure.

from pathlib import Path

EXAMPLES_DIR = Path("awscli/examples")

def test_example_files_exist():
    assert EXAMPLES_DIR.exists(), "examples directory is missing"

    for path in EXAMPLES_DIR.rglob("*.rst"):
        assert path.is_file(), f"Missing example file: {path}"

A more useful follow-up would be to validate command examples against the CLI parser or a curated list of known-good example commands.

Test strategy

A useful regression test would:

• scan the example directory
• identify command blocks
• ensure the command being documented still exists
• run a lightweight parse/validation step against the command shape
• fail with a clear message when an example is stale

Example structure:

def test_documented_examples_are_valid():
    examples = load_documented_examples("awscli/examples")
    for example in examples:
        assert command_is_valid(example.command), example.path

If full execution is too expensive or environment-dependent, even parser-level validation would be a good improvement over no validation at all.

Criteria

This issue could be considered resolved when at least one of the following is in place:

• a CI test that detects stale or broken examples
• a documented process for validating example files before merge
• clear guidance for when an example is intentionally excluded from validation

Additional context

This request is intended as a documentation-quality and developer-experience improvement rather than a functional CLI bug.

I would be happy to help refine the scope if the maintainers prefer:

• a minimal file-existence check first, or
• a stronger command-validation test from the start.

Links

.

[adev-code]

Thanks for raising this @anupddas. Could you share one or more example files that have broken, ideally with the file path and a note on what's wrong with it? Thanks.