Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/ado-task-ir-contributor.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

170 changes: 120 additions & 50 deletions .github/workflows/ado-task-ir-contributor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
on:
schedule: every 4h
workflow_dispatch: {}
description: Crawls Azure DevOps built-in task docs, contributes typed IR helper functions for uncovered tasks, converts compiler-generated Step::RawYaml usages to typed steps, and opens a focused PR per run.
description: Crawls Azure DevOps built-in task docs, contributes typed IR builder structs for uncovered tasks, converts compiler-generated Step::RawYaml usages to typed steps, and opens a focused PR per run.
permissions:
contents: read
issues: read
Expand Down Expand Up @@ -42,9 +42,9 @@ The `ado-aw` compiler transforms agent markdown into Azure DevOps YAML through a
- `Step::Checkout` / `Step::Download` / `Step::Publish` — special-purpose steps
- `Step::RawYaml` — **escape hatch** for user-authored YAML that the IR cannot model

The `TaskStep` struct is already fully generic — it accepts any `task: String` and an `inputs: IndexMap`. The gap is:
1. **No typed factory functions** for most ADO built-in tasks — every place that uses a known ADO task must hand-craft `TaskStep::new("Foo@1", "display").with_input(...)`.
2. **Compiler-generated `Step::RawYaml`** — occasionally, extension/runtime code emits `Step::RawYaml` for steps that could be expressed as `Step::Task` with a typed helper; these should be migrated.
The `TaskStep` struct is fully generic — it accepts any `task: String` and an `inputs: IndexMap`. Typed coverage of specific ADO tasks lives in `src/compile/ir/tasks/` as **builder structs** (one submodule per task). Each builder exposes `new(<required>)`, one typed chained setter per optional input, and `into_step() -> TaskStep`; only inputs that were set are emitted. Constrained input values are typed enums (each with `as_ado_str()`), bool-string inputs take a Rust `bool`, and command/mode-dispatch tasks (`Docker@2`, `DotNetCoreCLI@2`, `NuGetCommand@2`, `PowerShell@2`) use a command enum with per-variant data so invalid input/command combinations are unrepresentable. `src/compile/ir/tasks/docker.rs` is the canonical template. The gaps are:
1. **No typed builder** for most ADO built-in tasks — code that uses an uncovered task must hand-craft `TaskStep::new("Foo@1", "display").with_input(...)` with raw string keys.
2. **Compiler-generated `Step::RawYaml`** — occasionally, extension/runtime code emits `Step::RawYaml` for steps that could be expressed as `Step::Task` with a typed builder; these should be migrated.

## Step 1 — Load Previous State

Expand All @@ -67,10 +67,13 @@ If the prior PR is still open, emit `noop` with "Waiting on open PR #N before ad

## Step 2 — Audit Existing Typed Coverage

Examine what typed factory functions already exist:
Examine which tasks already have a typed builder under `src/compile/ir/tasks/`:

```bash
# Existing TaskStep factory functions in runtimes and extensions
# One submodule per covered task; the struct + into_step are the builder.
ls src/compile/ir/tasks/
grep -rn "pub struct \|pub fn into_step" src/compile/ir/tasks/ --include="*.rs"
# Other typed task factories that live outside tasks/ (runtimes / extensions)
grep -rn "TaskStep::new\|fn.*task_step" src/runtimes src/compile/extensions --include="*.rs"
```

Expand All @@ -87,7 +90,7 @@ Focus on:

Ignore `push_raw_yaml_if_nonempty`, `step_to_raw_yaml_string`, and any function that explicitly handles user-authored YAML (these are legitimately `Step::RawYaml`).

Build a set of **already-typed tasks**: task identifiers that already have typed factory functions (e.g. `UseNode@1`, `UseDotNet@2`, `UsePythonVersion@0`, `NpmAuthenticate@0`, `PipAuthenticate@1`, `NuGetAuthenticate@1`).
Build the set of **already-typed tasks**: the union of the task identifiers covered by `src/compile/ir/tasks/` builder structs and the runtime/extension factories (e.g. `UseNode@1`, `UseDotNet@2`, `UsePythonVersion@0`, `NpmAuthenticate@0`, `PipAuthenticate@1`, `NuGetAuthenticate@1`).

## Step 3 — Fetch ADO Built-In Task Catalog

Expand Down Expand Up @@ -121,10 +124,12 @@ grep -rh "task:\|TaskStep::new" src tests --include="*.rs" --include="*.yml" --i

From the catalog, select **one** task that:
1. Is **not** already in `completed_tasks` from state
2. Is **not** already typed (no existing factory function)
2. Is **not** already typed (no existing builder struct under `src/compile/ir/tasks/` and no runtime/extension factory)
3. Has a stable, widely-used task identifier (prefer tasks in the `Utility`, `Build`, or `Test` categories)
4. Has clear documentation on `learn.microsoft.com`

The chosen task will become a new submodule `src/compile/ir/tasks/<task_snake>.rs`.

Priority order (highest first):
1. A `Step::RawYaml` in compiler-generated code that maps to a known ADO task — converting it to `Step::Task` removes tech debt directly.
2. A task used frequently in the `tests/` fixtures as raw YAML strings.
Expand All @@ -149,67 +154,130 @@ Document the task:
- Optional inputs with defaults
- Relevant use cases in ado-aw

## Step 5 — Implement the Typed Helper
## Step 5 — Implement the Typed Builder

Decide where to place the factory function:
Decide where to place the builder:

- **If the task is tied to a specific runtime** (language install, package auth): add to `src/runtimes/<name>/mod.rs`.
- **If the task is a general-purpose ADO built-in**: add to `src/compile/ir/step.rs` as a standalone constructor function alongside the existing step types, or create a new `src/compile/ir/tasks.rs` module if multiple new tasks are being added.
- **If the task is tied to a specific runtime** (language install, package auth): add to `src/runtimes/<name>/mod.rs` (these remain free factory functions).
- **If the task is a general-purpose ADO built-in**: create a **new submodule** `src/compile/ir/tasks/<task_snake>.rs` and declare it in `src/compile/ir/tasks/mod.rs` with `pub mod <task_snake>;` (alphabetical order). One file per task.

For a new `tasks.rs` module, create it at `src/compile/ir/tasks.rs` and add `pub mod tasks;` to `src/compile/ir/mod.rs`.
### Builder struct shape

### Helper function shape
Model after `src/compile/ir/tasks/copy_files.rs` (a single-mode task) and, for command/mode-dispatch tasks, `src/compile/ir/tasks/docker.rs` (the canonical command-enum template). Shared helpers (`bool_input`, `push_opt`, `push_bool`) live in `src/compile/ir/tasks/common.rs`.

Model after existing patterns in `src/runtimes/`:
A builder is a struct with the required inputs as fields, each optional input as an `Option<…>` field, and a `display_name: Option<String>` override:

```rust
/// Returns a [`TaskStep`] for `CopyFiles@2`.
///
/// Copies files matching `contents` from `source_folder` to `target_folder`.
/// All parameters map directly to the ADO task inputs.
pub fn copy_files_step(
source_folder: impl Into<String>,
contents: impl Into<String>,
target_folder: impl Into<String>,
) -> TaskStep {
TaskStep::new("CopyFiles@2", "Copy Files")
.with_input("SourceFolder", source_folder)
.with_input("Contents", contents)
.with_input("TargetFolder", target_folder)
use super::common::bool_input;
use crate::compile::ir::step::TaskStep;

/// Builder for a [`TaskStep`] invoking `CopyFiles@2`.
#[derive(Debug, Clone)]
pub struct CopyFiles {
contents: String,
target_folder: String,
source_folder: Option<String>,
clean_target_folder: Option<bool>,
display_name: Option<String>,
}

impl CopyFiles {
/// Required inputs are positional parameters of `new`.
pub fn new(contents: impl Into<String>, target_folder: impl Into<String>) -> Self {
Self {
contents: contents.into(),
target_folder: target_folder.into(),
source_folder: None,
clean_target_folder: None,
display_name: None,
}
}

/// One typed chained setter per optional input.
pub fn source_folder(mut self, value: impl Into<String>) -> Self {
self.source_folder = Some(value.into());
self
}

/// Bool-string inputs take a Rust `bool`.
pub fn clean_target_folder(mut self, value: bool) -> Self {
self.clean_target_folder = Some(value);
self
}

/// Always provide a `displayName` override.
pub fn with_display_name(mut self, value: impl Into<String>) -> Self {
self.display_name = Some(value.into());
self
}

/// `into_step` emits required inputs always, optionals only when set.
pub fn into_step(self) -> TaskStep {
let mut t = TaskStep::new(
"CopyFiles@2",
self.display_name.unwrap_or_else(|| "Copy Files".into()),
)
.with_input("Contents", self.contents)
.with_input("TargetFolder", self.target_folder);
if let Some(v) = self.source_folder {
t = t.with_input("SourceFolder", v);
}
if let Some(v) = self.clean_target_folder {
t = t.with_input("CleanTargetFolder", bool_input(v));
}
t
}
}
```

Guidelines:
- Function name: snake_case, derived from the task display name, without the version suffix (e.g. `copy_files_step`, `publish_test_results_step`).
- Required inputs: positional parameters.
- Optional inputs with common defaults: keyword-style builder on the returned `TaskStep` (use `.with_input` after the initial construction).
- Include a doc comment with the task identifier and a one-line description.
- Do NOT add a new `Step` enum variant for standard tasks — `Step::Task(TaskStep)` is the correct representation.
- Struct name: PascalCase of the task display name, without the version suffix (e.g. `CopyFiles`, `PublishTestResults`).
- **Required inputs** → positional parameters of `new`.
- **Optional inputs** → `Option<…>` fields with one typed chained setter each; only emit them in `into_step` when set.
- **Constrained values** (a fixed set of string tokens) → a typed enum with `as_ado_str(&self) -> &'static str` returning the exact ADO token. Colocate the enum with the task (reusable shared enums may live in `common.rs`).
- **Bool-string inputs** → `Option<bool>`; lower via `bool_input`.
- **Command/mode-dispatch tasks** (each command exposes a different optional-input set) → a command **enum with per-variant data** so invalid input/command combos are unrepresentable. Wrap it in a builder struct (`new(<Command>)` plus per-command constructors) and match the variant in `into_step`. Model on `src/compile/ir/tasks/docker.rs`.
- Include a doc comment with the task identifier and the ADO task reference URL.
- Do NOT add a new `Step` enum variant for standard tasks — `Step::Task(<builder>.into_step())` is the correct representation.

### Convert RawYaml if applicable

If Step 4 identified a `Step::RawYaml` in compiler code that this task covers, replace it now:

```rust
// Before:
steps.push(Step::RawYaml(format!("- task: CopyFiles@2\n inputs:\n SourceFolder: {src}\n TargetFolder: {dst}")));
steps.push(Step::RawYaml(format!("- task: CopyFiles@2\n inputs:\n Contents: '**'\n TargetFolder: {dst}")));

// After:
use crate::compile::ir::tasks::copy_files_step;
steps.push(Step::Task(copy_files_step(&src, "**", &dst)));
use crate::compile::ir::tasks::copy_files::CopyFiles;
steps.push(Step::Task(CopyFiles::new("**", &dst).into_step()));
```

## Step 6 — Add Tests

Add at least one unit test to the same file (or `tests/compiler_tests.rs` for integration tests):
Add at least one `#[cfg(test)] mod tests` unit test to the new task submodule (or `tests/compiler_tests.rs` for integration tests), building via the struct and asserting on `task` / `inputs`:

```rust
#[test]
fn copy_files_step_creates_task_with_inputs() {
let t = copy_files_step("$(Build.SourcesDirectory)", "**/*.rs", "$(Build.ArtifactStagingDirectory)");
assert_eq!(t.task, "CopyFiles@2");
assert_eq!(t.inputs.get("SourceFolder").map(|s| s.as_str()), Some("$(Build.SourcesDirectory)"));
assert_eq!(t.inputs.get("TargetFolder").map(|s| s.as_str()), Some("$(Build.ArtifactStagingDirectory)"));
#[cfg(test)]
mod tests {
use super::*;

#[test]
fn creates_task_with_inputs() {
let t = CopyFiles::new("**/*.rs", "$(Build.ArtifactStagingDirectory)")
.source_folder("$(Build.SourcesDirectory)")
.into_step();
assert_eq!(t.task, "CopyFiles@2");
assert_eq!(t.inputs.get("Contents").map(String::as_str), Some("**/*.rs"));
assert_eq!(
t.inputs.get("TargetFolder").map(String::as_str),
Some("$(Build.ArtifactStagingDirectory)")
);
assert_eq!(
t.inputs.get("SourceFolder").map(String::as_str),
Some("$(Build.SourcesDirectory)")
);
}
}
```

Expand Down Expand Up @@ -260,27 +328,29 @@ jq \
If changes were made, open a PR with:

**Title** — conventional commits format:
- `feat(ir): add typed helper for <TaskName@version>` — for new factory functions
- `feat(ir): add typed builder for <TaskName@version>` — for a new builder struct
- `refactor(ir): replace RawYaml with typed TaskStep for <TaskName@version>` — for RawYaml conversions
- `feat(ir): add tasks module with typed helpers for <TaskA> and <TaskB>` — if a new module is created

**Body**:

```markdown
## Summary

Adds a typed factory function for `<TaskName@version>` to the ado-aw IR.
Adds a typed builder struct for `<TaskName@version>` to the ado-aw IR.

## Motivation

Previously, any code that needed to emit this ADO task step had to hand-craft
`TaskStep::new(...)` with raw string inputs. This PR introduces a well-typed
helper that validates required inputs at the call site and provides a clear API.
`TaskStep::new(...)` with raw string input keys. This PR introduces a typed
builder struct (`new(<required>)` + typed optional setters + `into_step()`) so
required inputs are positional, optional inputs and their constrained values are
type-checked, and call sites stop using stringly-typed keys.

## Changes

- `src/compile/ir/tasks.rs` (or relevant file): `<fn_name>()` factory function
- `tests/...`: unit tests for the new helper
- `src/compile/ir/tasks/<task_snake>.rs`: new `<TaskName>` builder struct (+ any
typed enums) and its `#[cfg(test)] mod tests`
- `src/compile/ir/tasks/mod.rs`: `pub mod <task_snake>;` declaration
- (if applicable) `src/compile/...`: converted `Step::RawYaml` → `Step::Task`

## ADO Task Reference
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Every compiled pipeline runs as three sequential jobs:
│ │ │ ├── mod.rs # IR module entry point and shared types
│ │ │ ├── ids.rs # Stable IDs for jobs/steps/outputs in the IR
│ │ │ ├── step.rs # Step declarations and typed step variants
│ │ │ ├── tasks.rs # Typed factory helpers for built-in ADO TaskStep creation
│ │ │ ├── tasks/ # Typed builder structs for built-in ADO tasks (one file per task; new()+typed setters+into_step(); command-enum dispatch for Docker/DotNet/NuGet/PowerShell; docker.rs canonical template)
│ │ │ ├── job.rs # Job declarations and typed job graph nodes
│ │ │ ├── stage.rs # Stage declarations and typed stage graph nodes
│ │ │ ├── env.rs # Typed environment and variable modeling
Expand Down
Loading
Loading