Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 16 additions & 101 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,107 +13,7 @@ PipeForge is a lightweight, composable pipeline framework for .NET. It makes ste

PipeForge is available on [NuGet.org](https://www.nuget.org/packages/PipeForge/) and can be installed using a NuGet package manager or the .NET CLI.

## Usage

Pipelines are designed to operate on specific class, referred to as the **context**. Multiple pipeline steps are created in code to operate on that context. Steps are annotated with an attribute indicating the order in which they should be executed. Finally, the pipeline runner is given an instance of the context to run against.

The following example uses dependency injection, and is the recommended approach to using PipeForge. For more advanced scenarios, see the full [documentation](https://scottoffen.github.io/pipeforge).

> [!NOTE]
> I'm suffixing my context class with the word `Context`, and my steps with the word `Step` for demonstration purposes only.

### Create Your Context

```csharp
public class SampleContext
{
private readonly List<string> _steps = new();

public void AddStep(string stepName)
{
if (string.IsNullOrWhiteSpace(stepName))
{
throw new ArgumentException("Step name cannot be null or whitespace.", nameof(stepName));
}

_steps.Add(stepName);
}

public int StepCount => _steps.Count;

public override string ToString()
{
return string.Join("", _steps);
}
}
```

### Create Your Steps

```csharp
[PipelineStep(1)]
public class HelloStep : PipelineStep<SampleContext>
{
public override async Task InvokeAsync(SampleContext context, PipelineDelegate<SampleContext> next, CancellationToken cancellationToken = default)
{
context.AddStep("Hello");
await next(context, cancellationToken);
}
}

[PipelineStep(2)]
public class WorldStep : PipelineStep<SampleContext>
{
public override async Task InvokeAsync(SampleContext context, PipelineDelegate<SampleContext> next, CancellationToken cancellationToken = default)
{
context.AddStep("World");
await next(context, cancellationToken);
}
}

[PipelineStep(3)]
public class PunctuationStep : PipelineStep<SampleContext>
{
public override async Task InvokeAsync(SampleContext context, PipelineDelegate<SampleContext> next, CancellationToken cancellationToken = default)
{
context.AddStep("1");
await next(context, cancellationToken);
}
}
```

### Use Dependency Injection

The extension method will discover and register all steps for the given `T` context, as well as register an instance of `IPipelineRunner<T>` that can be injected into services.

```csharp
services.AddPipelineFor<SampleContext>();
```

### Execute Pipeline

Get an instance of `IPipelineRunnere<SampleContext>` from your dependency injection container.

```csharp
public class SampleService
{
private readonly IPipelineRunner<SampleContext> _pipelineRunner;

public SampleService(IPipelineRunner<SampleContext> pipelineRunner)
{
_pipelineRunner = pipelineRunner;
}

public async Task RunPipeline(SampleContext context, CancellationToken? token = default)
{
await _pipelineRunner.ExecuteAsync(context, token);
var result = context.ToString();
// result = "HelloWorld!"
}
}
```

## Support
## Usage and Support

- Check out the project documentation https://scottoffen.github.io/pipeforge.

Expand All @@ -123,6 +23,21 @@ public class SampleService

- **Issues created to ask "how to" questions will be closed.**

## Use Cases

While PipeForge is fundamentally a pipeline framework, it can also serve as the foundation for higher-level workflows. These workflows are built by composing individual pipeline steps that handle branching, retries, fallbacks, and decision logic - making it ideal for orchestrating complex processes like AI chains, data enrichment, or multi-stage validation.

| | |
|-|-|
| Game Loop and Simulation Ticks | Model turn-based or tick-based game logic using structured steps for input handling, state updates, AI, and rule enforcement. Ideal for simulations, server-side logic, or deterministic turn resolution. |
| Middleware-Style Request Processing | Build lightweight, modular request pipelines similar to ASP.NET middleware, without requiring a full web host. |
| DevOps and Automation Pipelines | Express deployment checks, file transforms, and system hooks as repeatable, testable steps. |
| Security and Auditing Pipelines | Enforce policies, redact sensitive data, and log events in a structured, traceable flow. |
| ETL and Data Processing Pipelines | Break down validation, transformation, and persistence into clean, maintainable processing steps. |
| LLM and AI Workflows | Orchestrate prompt generation, model calls, fallback handling, and response parsing using composable pipelines. |
| Business Logic and Domain Orchestration | Replace brittle `if` chains and nested logic with clearly structured rule execution and orchestration flows. |


## Contributing

We welcome contributions from the community! In order to ensure the best experience for everyone, before creating an issue or submitting a pull request, please see the [contributing guidelines](CONTRIBUTING.md) and the [code of conduct](CODE_OF_CONDUCT.md). Failure to adhere to these guidelines can result in significant delays in getting your contributions included in the project.
Expand Down
89 changes: 64 additions & 25 deletions docs/docs/describe-pipelines.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,31 @@
---
sidebar_position: 6
sidebar_position: 8
title: Describing Pipelines
---

# Describing Pipelines
PipeForge provides built-in support for inspecting pipeline structure and behavior at runtime. This is especially useful for observability, testing, documentation, and UI tooling.

The `Describe()` method on `IPipelineRunner<T>` is used to inspect and document the pipeline configuration at runtime. It returns a JSON string describing each registered pipeline step.
## Describe()

This method is useful for diagnostics, tooling, and dynamically displaying pipeline behavior in user interfaces or logs - but only if you've taken the time to add the necessary metadata to your step classes.
The `Describe()` method on `IPipelineRunner<T>` outputs a JSON array representing each registered step. It includes key metadata such as name, order, and short-circuit configuration.

## Output Format
:::warning[Side Effects]

The JSON output contains an array of objects, each representing a pipeline step with the following fields:
This method **instantiates all steps**, triggering constructor injection and service resolution. Use it only when such side effects are acceptable.

* `Order`: The zero-based order in which the step appears in the pipeline
* `Name`: The value of the step's `Name` property
* `Description`: The step's `Description`, if defined
* `MayShortCircuit`: A boolean indicating whether the step might short-circuit execution
* `ShortCircuitCondition`: The value of the step's `ShortCircuitCondition`, if any
:::

### Output Format

Example output:
| Field | Description |
| ----------------------- | ------------------------------------------------------- |
| `Order` | Step's position in the execution sequence |
| `Name` | Value of the step’s `Name` property |
| `Description` | Value of the step’s `Description`, if provided |
| `MayShortCircuit` | Indicates if the step may halt pipeline execution |
| `ShortCircuitCondition` | Explanation of the short-circuit trigger, if applicable |

### Example Output

```json
[
Expand All @@ -40,26 +46,59 @@ Example output:
]
```

:::warning Warning

The `Order` value in the JSON output represents the execution order of the steps. This value is assigned based on the order in which the steps will be executed, and it may differ from the `Order` specified in each step's `PipelineStep` attribute.
:::info[A Note About Order]

For example, if you have only two steps with `PipelineStep(Order = 3)` and `PipelineStep(Order = 4)`, the JSON output will show `Order` values of `0` and `1`, respectively - reflecting their relative execution sequence, not their original attribute values.
The `Order` shown in the output reflects the **actual runtime execution sequence**, not the numeric value assigned via the `[PipelineStep]` attribute. For example, two steps with attributes `Order = 3` and `Order = 4` will appear in the output as `Order = 0` and `Order = 1` if they are the only steps registered.

:::

## Instantiation Behavior
### Use Cases

Calling `Describe()` **will instantiate all steps** in the pipeline by accessing their `Lazy<T>` wrappers. This may result in constructor injection or other side effects associated with instantiating the step class. Use this method only when you are prepared for that overhead.
* Logging pipeline structure for observability
* Generating runtime or admin UI documentation
* Verifying step order and metadata in unit tests

## Use Cases
If you need to inspect step configuration without triggering instantiation, consider capturing step metadata during registration or design time.

* Logging pipeline structure for observability
* Generating runtime documentation or UI
* Verifying step order and metadata during tests or builds
## DescribeSchema()

The `DescribeSchema()` method returns a [JSON Schema v7](https://json-schema.org/specification.html) document that describes the metadata shape of a pipeline step. This is ideal for tools that visualize or validate pipeline structures.

### When to Use

If you need to inspect step configuration without triggering instantiation, consider maintaining parallel metadata or restricting usage of `Describe()` to controlled environments.
* Exposing step definitions through APIs or dashboards
* Powering UI editors or metadata-driven configuration
* Documenting expected step shape for developers
* Validating configuration or orchestration input

## Conclusion
### Schema Fields

| Property | Type | Description |
| ----------------------- | ------- | --------------------------------------------------- |
| `Order` | integer | Execution order of the step (inferred at runtime) |
| `Name` | string | Display name of the step |
| `Description` | string | Optional summary of the step’s purpose |
| `MayShortCircuit` | boolean | Indicates if the step may halt execution early |
| `ShortCircuitCondition` | string | Optional explanation of the short-circuit condition |

Only `Order` is required. All other fields are optional and serve documentation, inspection, or visualization purposes.

### Example Output

```json
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "PipelineStep",
"type": "object",
"properties": {
"Order": { "type": "integer", "description": "Execution order of the step (inferred)" },
"Name": { "type": "string", "description": "Display name of the step" },
"Description": { "type": "string", "description": "Optional description of the step" },
"MayShortCircuit": { "type": "boolean", "description": "Whether the step may halt pipeline execution early" },
"ShortCircuitCondition": { "type": "string", "description": "Explanation of the short-circuit condition, if any" }
},
"required": ["Order"]
}
```

Use `Describe()` to introspect your pipeline at runtime, but be aware that it will force full resolution of every registered step.
Use this schema to build validation layers, generate dynamic forms, or publish step definitions in API documentation.
6 changes: 1 addition & 5 deletions docs/docs/diagnostics.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 8
sidebar_position: 9
title: Diagnostics
---

Expand Down Expand Up @@ -43,7 +43,3 @@ listener.Subscribe(new Observer((name, payload) =>
* You can use this mechanism to generate timing metrics, debug issues, or visualize step execution.
* Diagnostic events are low-overhead and safe to leave enabled in production.
* Combine this with the `Describe()` method for a full picture of pipeline structure and execution behavior.

## Conclusion

PipeForge diagnostics give you deep visibility into pipeline execution with minimal effort. Whether you're debugging a failing step or building runtime instrumentation, the diagnostics hooks are ready to help.
18 changes: 9 additions & 9 deletions docs/docs/index.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
---
sidebar_position: 1
sidebar_position: 0
title: Getting Started
---

# Welcome to PipeForge

PipeForge is a lightweight, composable, lazy-instantiation pipeline framework for .NET. It simplifies step-based processing while remaining discoverable and testable. Inspired by middleware pipelines and modern dependency injection patterns, PipeForge gives you structured control over sequential logic flows - without the ceremony.
PipeForge is a lightweight, composable pipeline framework for .NET that makes step-based workflows easy to build, test, and maintain. With lazy instantiation and modern dependency injection, it gives you structured control over execution flow - without the heavy scaffolding of base classes, rigid lifecycles, or tightly coupled framework logic. Inspired by the simplicity of middleware pipelines, PipeForge favors clear, minimal structure over hidden complexity.

Pipelines operate on a specific class known as the **context**. Each pipeline step is a discrete unit of work, written in code and annotated with metadata indicating its order and (optional) filter. These steps are lazily instantiated and executed in sequence by the pipeline runner.
Each pipeline operates on a specific class called the **context**, which flows through each step in sequence. Steps are discrete units of work, written as regular code and annotated with metadata to define their order and optional filters. They’re lazily instantiated - only created when needed - and executed by the pipeline runner.

At any point, the pipeline can **short-circuit**, halting execution - and preventing the instantiation of any remaining steps.
At any point, a step can **short-circuit** the pipeline, halting further execution and preventing the instantiation of remaining steps.

## Sample Context

Expand All @@ -18,7 +18,7 @@ For the purposes of this documentation, the following sample context will be use
```csharp title="SampleContext.cs"
public class SampleContext
{
private readonly List<string> _steps = new();
public readonly List<string> Steps = new();

public void AddStep(string stepName)
{
Expand All @@ -27,22 +27,22 @@ public class SampleContext
throw new ArgumentException("Step name cannot be null or whitespace.", nameof(stepName));
}

_steps.Add(stepName);
Steps.Add(stepName);
}

public int StepCount => _steps.Count;
public int StepCount => Steps.Count;

public override string ToString()
{
return string.Join(",", _steps);
return string.Join(",", Steps);
}
}
```

This context allows us to:
- Track pipeline progress via `AddStep()`
- Evaluate the order and number of step executions
- Print step execution history using `ToString()`
- Assert how many steps ran using `StepCount`
- Simulate errors by passing null or empty step names

## Installation
Expand Down
Loading