Skip to content

Commit 0a845e8

Browse files
aepfliCopilot
aepfli
and
Copilot
committed
docs: add ADR for modular flagd builder
Propose adopting an OTel Collector Builder-style approach for flagd, enabling users to compose custom binaries with only the sync providers, service endpoints, evaluators, and middleware they need. Key points: - Factory interfaces for all component types (sync, service, evaluator, middleware) - Each component becomes a separate Go module with its own go.mod - YAML manifest-driven code generation and compilation - Standard distributions (minimal, cloud, kubernetes, full) - User extensibility: custom components via own Go modules, no forking required Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent d9b5aa2 commit 0a845e8

1 file changed

Lines changed: 239 additions & 0 deletions

File tree

‎docs/architecture-decisions/modular-builder.md‎

Lines changed: 239 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,239 @@
1+
---
2+
status: draft
3+
author: Simon Schrottner
4+
created: 2026-04-01
5+
updated: 2026-04-01
6+
---
7+
8+
# ADR: Modular flagd Builder
9+
10+
This document proposes a modular build system for flagd inspired by the [OpenTelemetry Collector Builder (ocb)](https://github.com/open-telemetry/opentelemetry-collector/tree/main/cmd/builder). The goal is to allow users to compose custom flagd binaries containing only the sync providers, service endpoints, evaluators, and middleware they need — and to enable users to contribute their own implementations without forking flagd.
11+
12+
## Background
13+
14+
flagd's [multi-sync architecture](./multiple-sync-sources.md) was a foundational decision that decoupled flag storage from the evaluation engine. The `ISync` interface enabled the community to add support for file, HTTP, gRPC, Kubernetes CRDs, and cloud blob storage (GCS, Azure Blob, S3). This extensibility has been a strength, but it has come at a cost: every flagd binary includes every provider and every dependency, regardless of what the user actually needs.
15+
16+
Today, the `core` Go module pulls in approximately 700 transitive dependencies through its `go.mod`, including the full AWS SDK v2, Google Cloud SDK, Azure SDK, Kubernetes client-go, the wazero WebAssembly runtime, and gocloud.dev with all three cloud blob drivers registered via side-effect imports. The `SyncBuilder` in `core/pkg/sync/builder/syncbuilder.go` unconditionally imports all sync provider packages, and `blob_sync.go` registers all cloud drivers via blank imports (`_ "gocloud.dev/blob/s3blob"`, etc.). There are no build tags or conditional compilation — everything is always compiled in.
17+
18+
Similarly, flagd's three service endpoints — the ConnectRPC/gRPC flag evaluation service, the OFREP REST service, and the gRPC flag sync service — are hardcoded in `flagd/pkg/runtime/from_config.go`. All three are always instantiated and started. Users cannot selectively disable endpoints, nor can they add custom endpoints (e.g., a WebSocket adapter, an admin API, or a custom protocol) without forking flagd.
19+
20+
The OpenTelemetry Collector project faced a nearly identical problem and solved it with the [OpenTelemetry Collector Builder](https://github.com/open-telemetry/opentelemetry-collector/tree/main/cmd/builder): a CLI tool that reads a YAML manifest specifying which components to include, generates Go source code from templates, and compiles a custom binary with only the selected dependencies. Each component is a separate Go module exposing a `NewFactory()` function. The builder generates a `components.go` that imports and registers only the selected factories, a `main.go` entrypoint, and a `go.mod` with only the required dependencies. Users can reference their own Go modules in the manifest to add custom components. This approach is proven at scale across hundreds of OTel Collector distributions.
21+
22+
## Requirements
23+
24+
* **Selective compilation**: Users must be able to choose which sync providers, service endpoints, evaluators, and middleware are compiled into their flagd binary, eliminating unused dependencies entirely (not just disabling them at runtime).
25+
* **User extensibility**: Users must be able to implement custom sync providers, service endpoints, evaluators, or middleware in their own Go modules and include them in a flagd build without forking the project.
26+
* **Backward compatibility**: The current monolithic flagd binary must remain available as a "full" distribution. Existing users should not be forced to adopt the builder.
27+
* **Standard distributions**: The project must ship pre-configured distributions for common use cases (minimal, cloud, Kubernetes, full) so that most users never need to run the builder themselves.
28+
* **Clean interfaces**: Each component type (sync, evaluator, service, middleware) must have a well-defined factory interface that all implementations — official and user-provided — implement.
29+
* **Standard Go tooling**: The build process must use standard Go modules and `go build`. No custom package managers, no dynamic linking, no Go plugins.
30+
31+
## Considered Options
32+
33+
* **Go build tags**: Use `//go:build` tags to conditionally compile sync providers and services. For example, `//go:build with_s3` would gate the S3 provider.
34+
* **OTel-style builder with code generation**: A builder CLI tool that generates Go source files from a YAML manifest and compiles a custom binary. Each component is a separate Go module.
35+
* **Go plugin system**: Use Go's `plugin` package to dynamically load sync providers and services at runtime as `.so` files.
36+
37+
## Proposal
38+
39+
We propose adopting the **OTel-style builder with code generation** approach, extended to cover not just sync providers but also service endpoints, evaluators, and middleware as first-class selectable components.
40+
41+
### Why not build tags?
42+
43+
Build tags are fragile and hard to compose. More critically, they do not eliminate dependencies from `go.mod` — the modules are still referenced even if their code is gated by build tags, and Go's module system still downloads and resolves them. Build tags also require maintaining `_tag.go` and `_notag.go` file pairs, which increases complexity and the surface area for bugs.
44+
45+
### Why not Go plugins?
46+
47+
Go's `plugin` package has severe limitations: it only works on Linux (and limited other Unix-like systems), requires both the host and plugin to be compiled with the exact same Go version and build flags, and does not support Windows or macOS. The OTel Collector project explicitly evaluated and rejected this approach for these reasons.
48+
49+
### Component taxonomy
50+
51+
The builder operates on four component types:
52+
53+
| Component Type | Factory Interface | Current Implementations | Example User Extension |
54+
|---------------|------------------|------------------------|----------------------|
55+
| **Sync Provider** | `SyncFactory` | file, kubernetes, http, grpc, gcs, azblob, s3 | Consul, etcd, Vault |
56+
| **Service Endpoint** | `ServiceFactory` | flag-evaluation (ConnectRPC), ofrep (REST), flag-sync (gRPC) | WebSocket adapter, admin API, custom protocol |
57+
| **Evaluator** | `EvaluatorFactory` | jsonlogic, wasm | Custom evaluation engine |
58+
| **Middleware** | `MiddlewareFactory` | cors, h2c, metrics | Rate limiting, auth, logging |
59+
60+
### Factory interfaces
61+
62+
Each component type gets a factory interface in the `core` module. The `core` module becomes lightweight (interfaces only, no external dependencies beyond stdlib):
63+
64+
```go
65+
// core/pkg/sync/factory.go
66+
type SyncFactory interface {
67+
Type() string
68+
Schemes() []string
69+
Create(cfg SourceConfig, logger *logger.Logger) (ISync, error)
70+
}
71+
72+
// core/pkg/service/factory.go
73+
type ServiceFactory interface {
74+
Type() string
75+
Create(deps ServiceDependencies) (Service, error)
76+
}
77+
78+
type Service interface {
79+
Start(ctx context.Context) error
80+
Shutdown(ctx context.Context) error
81+
}
82+
83+
type ServiceDependencies struct {
84+
Evaluator evaluator.IEvaluator
85+
Store store.IStore
86+
Logger *logger.Logger
87+
Config Configuration
88+
}
89+
90+
// core/pkg/evaluator/factory.go
91+
type EvaluatorFactory interface {
92+
Type() string
93+
Create(store store.IStore, logger *logger.Logger) (IEvaluator, error)
94+
}
95+
```
96+
97+
Each component module exposes a `NewFactory()` function:
98+
99+
```go
100+
// In github.com/open-feature/flagd/sync/file
101+
func NewFactory() sync.SyncFactory { ... }
102+
103+
// In github.com/open-feature/flagd/service/ofrep
104+
func NewFactory() service.ServiceFactory { ... }
105+
106+
// User-provided: github.com/mycompany/flagd-consul-sync
107+
func NewFactory() sync.SyncFactory { ... }
108+
```
109+
110+
### Module structure
111+
112+
The monolithic `core` module is split so that each component is a separate Go module with its own `go.mod`:
113+
114+
```
115+
core/ # Interfaces only (lightweight)
116+
sync/file/ # depends on core + fsnotify
117+
sync/kubernetes/ # depends on core + k8s.io/client-go
118+
sync/http/ # depends on core + net/http
119+
sync/grpc/ # depends on core + google.golang.org/grpc
120+
sync/gcs/ # depends on core + gocloud.dev/blob/gcsblob
121+
sync/azblob/ # depends on core + azure-sdk-for-go
122+
sync/s3/ # depends on core + aws-sdk-go-v2
123+
evaluator/jsonlogic/ # depends on core + jsonlogic
124+
evaluator/wasm/ # depends on core + wazero
125+
service/flag-evaluation/ # depends on core + ConnectRPC
126+
service/ofrep/ # depends on core + net/http
127+
service/flag-sync/ # depends on core + grpc
128+
```
129+
130+
Blob providers are split individually (not sharing a gocloud.dev base module) so users can include a single cloud provider without pulling all three SDKs.
131+
132+
### Builder manifest
133+
134+
The `flagd-builder` CLI reads a YAML manifest that references components as Go module paths:
135+
136+
```yaml
137+
dist:
138+
module: github.com/mycompany/custom-flagd
139+
name: flagd
140+
version: "0.12.0"
141+
output_path: ./build
142+
143+
syncs:
144+
- gomod: "github.com/open-feature/flagd/sync/file v0.12.0"
145+
- gomod: "github.com/open-feature/flagd/sync/http v0.12.0"
146+
- gomod: "github.com/mycompany/flagd-consul-sync v1.2.0" # user extension
147+
148+
evaluators:
149+
- gomod: "github.com/open-feature/flagd/evaluator/jsonlogic v0.12.0"
150+
151+
services:
152+
- gomod: "github.com/open-feature/flagd/service/ofrep v0.12.0"
153+
- gomod: "github.com/mycompany/flagd-admin-api v1.0.0" # user extension
154+
155+
middleware:
156+
- gomod: "github.com/open-feature/flagd/middleware/cors v0.12.0"
157+
- gomod: "github.com/open-feature/flagd/middleware/metrics v0.12.0"
158+
159+
replaces: []
160+
```
161+
162+
### Build process
163+
164+
The builder executes three steps (any of which can be skipped):
165+
166+
1. **Generate**: Execute Go templates to produce `components.go`, `main.go`, and `go.mod`
167+
2. **Get modules**: Run `go mod tidy` to resolve dependencies
168+
3. **Compile**: Run `go build` to produce the binary
169+
170+
### Standard distributions
171+
172+
The project ships pre-configured manifests for common use cases:
173+
174+
| Distribution | Syncs | Evaluators | Services | Target Use Case |
175+
|-------------|-------|------------|----------|----------------|
176+
| `flagd-minimal` | file, http | jsonlogic | ofrep | Smallest binary, CI/testing, REST-only |
177+
| `flagd-cloud` | file, http, gcs, s3, azblob | jsonlogic | flag-evaluation, ofrep | Cloud storage backends |
178+
| `flagd-kubernetes` | file, http, kubernetes, grpc | jsonlogic | flag-evaluation, ofrep, flag-sync | K8s with OFO |
179+
| `flagd-full` | ALL | ALL | ALL | Current behavior (backward compat) |
180+
181+
### Runtime changes
182+
183+
The `Runtime` struct changes from hardcoded service fields to a dynamic registry:
184+
185+
```go
186+
// Current (hardcoded):
187+
type Runtime struct {
188+
EvaluationService service.IFlagEvaluationService
189+
OfrepService ofrep.IOfrepService
190+
SyncService flagsync.ISyncService
191+
...
192+
}
193+
194+
// Proposed (dynamic):
195+
type Runtime struct {
196+
Services []service.Service
197+
Syncs []sync.ISync
198+
Evaluator evaluator.IEvaluator
199+
...
200+
}
201+
```
202+
203+
### Consequences
204+
205+
* Good, because flagd binaries can be dramatically smaller (a file+http+ofrep build eliminates all cloud SDKs, Kubernetes client, gRPC, ConnectRPC, and wazero)
206+
* Good, because users can extend flagd with custom sync providers, service endpoints, evaluators, and middleware without forking
207+
* Good, because the builder pattern is proven at scale by the OpenTelemetry Collector community
208+
* Good, because it uses standard Go modules and tooling — no custom package managers or dynamic linking
209+
* Good, because backward compatibility is maintained via the `flagd-full` distribution
210+
* Bad, because it is a breaking change for anyone importing `core/pkg/sync/builder` or other internal packages directly
211+
* Bad, because it adds build complexity — users who want custom builds need to learn the builder tool
212+
* Bad, because component modules require coordinated releases (or independent versioning, which adds its own complexity)
213+
* Bad, because the module split creates many more Go modules to maintain in the repository
214+
215+
### Timeline
216+
217+
1. ADR review and acceptance
218+
2. Factory interface design and implementation in core
219+
3. Module split (sync providers, evaluators, service endpoints, middleware)
220+
4. Runtime refactoring (dynamic service registry)
221+
5. Builder CLI tool implementation
222+
6. Standard distribution manifests
223+
7. CI/CD pipeline updates, Dockerfile changes, documentation
224+
225+
### Open questions
226+
227+
* **Module path structure**: Should component modules live at top-level (`sync/file`, `service/ofrep`) or under existing paths (`core/pkg/sync/file` with separate `go.mod`)?
228+
* **flagd-proxy**: Should `flagd-proxy` also be buildable via the builder, or does it remain as-is?
229+
* **Release versioning**: Should component modules version independently (like OTel contrib) or stay in lockstep with flagd releases?
230+
* **Community component registry**: Should there be a curated list or repository of community-contributed components (analogous to `opentelemetry-collector-contrib`)?
231+
* **Default service set**: Can a build have zero service endpoints (library/embedded mode), or should at least one always be required?
232+
* **Configuration compatibility**: How does the builder interact with the existing `-f`/`--sources` CLI flags? The runtime needs to know which URI schemes are available from the selected sync factories.
233+
234+
## More Information
235+
236+
* [OpenTelemetry Collector Builder (ocb)](https://github.com/open-telemetry/opentelemetry-collector/tree/main/cmd/builder) — the reference implementation this proposal is based on
237+
* [ADR: Multiple Sync Sources](./multiple-sync-sources.md) — the prior ADR establishing flagd's multi-sync architecture and the `ISync` interface
238+
* [OTel Collector component model](https://opentelemetry.io/docs/collector/custom-collector/) — documentation on building custom collectors
239+
* [gocloud.dev](https://gocloud.dev/) — the cloud abstraction library currently used for blob sync providers

0 commit comments

Comments
 (0)