FastConf — strongly typed, lock-free, Kustomize-style configuration for Go

Language: English · 中文

fastconf layers YAML / JSON / TOML files, environment variables, CLI flags, remote KV stores, and on-the-fly generators into a single strongly typed Go struct. A single-writer reload loop publishes new snapshots atomically via atomic.Pointer; the hot read path is one atomic.Pointer.Load().

Status: public beta. The API still moves where semantics demand it. pkg.go.dev and this README track the current truth of the codebase.

---

Table of contents

1. Quick start
2. Why FastConf
3. Installation
4. Core model
5. Manager API
6. Options reference
7. Reload pipeline
8. Profiles & overlays
9. Provider system
10. Transformers & migration
11. Watch, Subscribe, and Plan
12. Provenance, history & rollback
13. Observability
14. Multi-tenant & presets
15. Sub-module ecosystem
16. CLI tools
17. Performance
18. Development
19. License

---

Quick start

package main

import (
    "context"
    "log"

    "github.com/fastabc/fastconf"
    "github.com/fastabc/fastconf/providers/env"
)

type AppConfig struct {
    Server struct {
        Addr string `json:"addr" yaml:"addr"`
    } `json:"server" yaml:"server"`
    Database struct {
        DSN  string `json:"dsn"  yaml:"dsn"`
        Pool int    `json:"pool" yaml:"pool"`
    } `json:"database" yaml:"database"`
}

func main() {
    mgr, err := fastconf.New[AppConfig](context.Background(),
        fastconf.WithDir("conf.d"),
        fastconf.WithProfile(fastconf.ProfileOptions{
            EnvVar:  "APP_PROFILE",
            Default: "dev",
        }),
        fastconf.WithProvider(env.NewEnv("APP_")),
        fastconf.WithWatch(fastconf.WatchOptions{Enabled: true}),
    )
    if err != nil {
        log.Fatal(err)
    }
    defer mgr.Close()

    cfg := mgr.Get() // *AppConfig — lock-free, O(1), zero-alloc
    log.Println(cfg.Server.Addr, cfg.Database.Pool)
}

Directory layout:

conf.d/
  base/
    00-app.yaml
  overlays/
    prod/
      50-overrides.yaml
      _patch.json

# conf.d/base/00-app.yaml
server:
  addr: ":8080"
database:
  dsn: "postgres://localhost/app"
  pool: 10

Run with an environment override:

APP_PROFILE=prod APP_DATABASE_POOL=20 go run .

APP_DATABASE_POOL=20 maps to database.pool (single _ separator, Viper/Spring Boot style). With APP_PROFILE=prod, FastConf merges base/* first, then overlays/prod/*.

Recommended entry points

Scenario	Recommended combo	Read more	Runnable example
Local file config	New + WithDir + Get	Quickstart	examples/basic
Kubernetes hot-reload	PresetK8s + Subscribe + Errors	k8s cookbook	examples/sidecar
Remote source / GitOps	WithProvider + Plan + Provenance	Vault / Consul	examples/external_source

---

Why FastConf

• Strong typing on the read path. mgr.Get().Server.Addr is checked by the compiler. No dotted-path strings, no reflection, no interface{}.
• Lock-free hot reads. Get() is an atomic.Pointer.Load() — O(1), zero-alloc, safe from any number of goroutines.
• Fail-safe reload. Any pipeline stage that errors out keeps the old *State[T] live; a broken config never reaches your read path.
• Kustomize-style layering. base / overlays, RFC 6902 patches, and strategic merge for lists of objects.
• Opt-in extensions. Providers, transformers, secret resolvers, validators, policies, metrics, and tracing are all optional.

---

Installation

go get github.com/fastabc/fastconf@latest

# Optional sub-modules:
go get github.com/fastabc/fastconf/observability/otel@latest
go get github.com/fastabc/fastconf/observability/metrics/prometheus@latest
go get github.com/fastabc/fastconf/cue@latest
go get github.com/fastabc/fastconf/policy/opa@latest
go get github.com/fastabc/fastconf/providers/s3@latest

Command-line tools (Go ≥ 1.22):

go install github.com/fastabc/fastconf/cmd/fastconfd@latest
go install github.com/fastabc/fastconf/cmd/fastconfctl@latest
go install github.com/fastabc/fastconf/cmd/fastconfgen@latest

Compatibility

Item	Supported
Go toolchain	1.22, 1.23, 1.24, 1.25, 1.26 (no toolchain pin in go.mod)
OS / arch	linux/amd64, linux/arm64, darwin/amd64, darwin/arm64, windows/amd64 (binaries published on each tag)
Module form	one root module + independent sub-modules (cue, policy/opa, validate/playground, observability/{otel,metrics/prometheus}, providers/s3, integrations/{cli/pflag,log/phuslu,log/zerolog})
Pre-release contract	semantic-version tags follow vMAJOR.MINOR.PATCH. The current public line is v0.19; release-to-release API changes are documented in CHANGELOG.md and focused migration recipes under docs/cookbook/.

Versioning

• Tags follow vMAJOR.MINOR.PATCH. The root module and every sub-module receive the same tag through tools/tag-release.sh vX.Y.Z.
• Major-version 0 is reserved for the pre-1.0 cycle. Breaking changes may still land between minor versions until v1.0, but each release ships with an explicit migration recipe under docs/cookbook/ so the call-site delta is mechanical.
• The internal package set under internal/* is implementation detail and not covered by the SemVer contract — root re-exports (type aliases or wrappers) are the only stable surface.
• Reusable primitives live in domain packages such as codec, confmap, overlay, transform, feature, and providers/*. The old pkg/* import paths were removed in v0.20.
• When sub-modules tag independently the tag is module-path-prefixed (e.g. cue/vX.Y.Z); the README mostly hides this because a single release pushes the same version across the root and every sub-module.
• Before each release we run make test plus five guard scripts under tools/{check-layout,check-doc-symbols,audit-phase-comments, check-cjk-comments,loc-budget}.sh, so directory layout, public symbols, comment archaeology and LOC budgets are all enforced before a tag is pushed.

---

Core model

sources / generators / providers
              │
              ▼
       assemble preflight
              │
              ▼
 merge → migration → transform → secret → typed-hooks
      → decode → field-meta → validate → policy
              │
      fail ───┴─── keep old State[T]
              │
           success
              ▼
 canonical hash → atomic swap → history → audit → subscribers

Property	What it means
Typed read path	mgr.Get().Server.Addr, checked by the compiler
Single-writer reload	fsnotify, provider events, and manual Reload all serialize through one writer
Fail-safe	Any stage error keeps the old *State[T]; bad config never reaches business code
Kustomize-style layering	base / overlay, RFC 6902 patches, strategic merge with mergeKeys
Opt-in extensions	providers, transformers, secret resolvers, policies, metrics, tracer

---

Manager API

// Construction (first reload runs synchronously)
func New[T any](ctx context.Context, opts ...Option) (*Manager[T], error)

// Read path — lock-free, O(1), zero-alloc
func (m *Manager[T]) Get() *T

// Trigger a reload; ctx controls the full pipeline.
func (m *Manager[T]) Reload(ctx context.Context, opts ...ReloadOption) error

// Dry-run — never updates the live pointer
func (m *Manager[T]) Plan() *PlanBuilder[T]

// Current snapshot (State[T] + Sources + Origins)
func (m *Manager[T]) Snapshot() *State[T]

// Async failure stream — buffered 16, drop-on-full, closed by Close()
func (m *Manager[T]) Errors() <-chan ReloadError

func (m *Manager[T]) Watcher() *Watcher[T]  // .Pause() / .Resume()
func (m *Manager[T]) Replay()  *Replay[T]   // .List() / .Rollback(*State[T])
func (m *Manager[T]) Close() error

Package-level generics:

// Per-field subscribe; fires only when the extracted value actually changes.
// Pass WithEqual(eq) to override the default reflect.DeepEqual comparator.
func Subscribe[T, M any](m *Manager[T], extract func(*T) *M, fn func(old, new *M), opts ...SubscribeOption[M]) (cancel func())
func WithEqual[M any](equal func(old, new *M) bool) SubscribeOption[M]

// Typed feature-flag evaluation.
func Eval[T, V any](m *Manager[T], key string, ctx feature.EvalContext, def V) V

State[T] carries Value *T, Hash [32]byte, Generation uint64, Sources []SourceRef, and provenance helpers (Explain, Diff, Redacted).

---

Options reference

All WithXxx options return Option and may be composed in any order. The full reference is in docs/readme/02-core-model.md.

Key options

Option	Purpose	Default
WithDir(dir)	Config root directory	"conf.d"
WithFS(fs.FS)	Alternate fs.FS (testing)	—
WithWatch(WatchOptions{...})	Enable fsnotify; bundles Enabled / Paths / Coalesce / CoalesceProfile	Enabled:false
WithProfile(ProfileOptions{...})	Profile selection bundle: Single, Multi, Expr, EnvVar, Default	—
WithCoalesce(CoalesceOptions{...})	Tune watcher Quiet / MaxLag / SwapHint independently of WithWatch	—
WithProvider(p)	Register a structured provider	—
WithSource(src, parser)	Byte-blob source + parser	—
WithMigrations(fn)	Schema migration callback	—
WithTransformers(t...)	Post-merge transform chain	—
WithSecretResolver(r)	Decrypt leaves before decode	—
WithValidator[T](fn)	Typed validation after decode	—
WithPolicy[T](p)	Policy evaluation after validate	—
WithHistory(n)	Keep last n successful states	—
WithProvenance(level)	Off / TopLevel / Full	Off
WithMetrics(sink)	Metrics sink	—
WithAuditSink(sink)	Audit on each successful reload	—
WithTracer(tracer)	OTel-compatible tracer	—
WithLogger(*slog.Logger)	Inject a logger	io.Discard
WithStructDefaults[T]()	Populate zero values via fc:"default=…" tags	—

---

Reload pipeline

Stage sequence

reloadCh.recv(req)
  ├─ stageMerge:      overlay.Scan(dir) → decode files → confmap merge(layers)
  │                   apply _meta.yaml (appendSlices / profileEnv / match)
  │                   apply _patch.json (RFC 6902)
  ├─ stageAssemble:   for each provider: Load(ctx) → merge by Priority
  ├─ stageMigrate:    opts.migrationRun(merged)
  ├─ stageTransform:  for each transformer: t.Transform(merged)
  ├─ stageDecode:     json.Marshal(merged) → json.Unmarshal(→ *T)
  ├─ stageFieldMeta:  range / enum / required checks
  ├─ stageValidate:   for each validator: v(*T)
  ├─ stagePolicy:     for each policy: p.Evaluate(ctx, *T, reason, tenant)
  └─ commit:
       canonical SHA-256 dedup
       atomic.Pointer.Store(newState) → history → audit → subscribers

When any stage errors: atomic.Pointer is not updated, Generation is not incremented, the error surfaces on Errors(), no AuditSink fires.

---

Profiles & overlays

conf.d/
  base/                     # applied for every profile
    00-defaults.yaml
  overlays/
    prod/
      50-prod.yaml
      _meta.yaml            # profile match expression
      _patch.json           # RFC 6902 patch

_meta.yaml

schemaVersion: "1"
profileEnv: "APP_PROFILE"
defaultProfile: "dev"
appendSlices: true
match: "prod | staging"     # &, |, !, () supported

RFC 6902 JSON Patch

[
  { "op": "replace", "path": "/server/addr",      "value": ":8443" },
  { "op": "add",     "path": "/feature/darkMode", "value": true    },
  { "op": "remove",  "path": "/obsolete/key"                       }
]

Multi-profile mode: WithProfile(ProfileOptions{Multi: []string{"prod", "eu-west", "canary"}}) — each overlay's _meta.yaml.match decides whether it applies.

Per-directory file limit. Each overlay directory holds < 100 config files and base/ holds < 1000, so file priorities stay inside their band. Exceeding the limit fails the scan loud instead of silently interleaving layers across directories.

---

Provider system

Built-in structured providers (providers/*)

Provider	Constructor	Notes
Env	env.NewEnv("APP_") (providers/env)	APP_FOO_BAR → foo.bar; chain .WithReplacer, .At, .WithCoerce
CLI	cliflag.NewCLI(map) (providers/cliflag)	Pass only explicitly changed flags; files/env stay authoritative
DotEnv	dotenv.NewDotEnv("APP_", paths...) (providers/dotenv)	.env fallback; process env wins
Labels	labels.NewDottedLabels(labels, opts) / NewRoutingLabels(labels, opts) (providers/labels)	Config and routing DSL labels
K8s Downward	k8s.NewDefault()	/etc/podinfo/{labels,annotations}

First-party KV providers (root module, trim via build tag):

vp, _ := vault.New("https://vault.svc", "kv/data/myapp", os.Getenv("VAULT_TOKEN"))
cp, _ := consul.New("http://consul.svc:8500", "config/myapp")
hp, _ := httpprov.New("remote", "https://example.com/cfg.yaml", yamlCodec{})
// Build tag to exclude: -tags no_provider_vault,no_provider_consul,no_provider_http

Root-versioned event providers: NATS (providers/nats) and Redis Streams (providers/redisstream). Independent provider sub-module (go get as needed): S3 (providers/s3).

Priority constants

Merge order follows Priority() ascending — higher values overwrite lower:

Constant	Value	Use
PriorityDotEnv	5	.env fallback
PriorityStatic	10	Static / file layers
PriorityKV	30	Vault / Consul / HTTP / S3
PriorityK8s	40	Kubernetes ConfigMap / Secret
PriorityEnv	50	Process environment variables
PriorityCLI	60	Command-line flags (highest)
contracts.PriorityOrderedBase	160	Reserved base used by WithProviderOrdered

Use WithProviderOrdered(p1, p2, p3) to auto-assign priorities in call order.

contracts.Provider interface

type Provider interface {
    Name()     string
    Priority() int
    Load(ctx context.Context) (map[string]any, error)
    Watch(ctx context.Context) (<-chan Event, error)
}

---

Transformers & migration

Built-in transformers (transform)

fastconf.WithTransformers(
    transform.Defaults(map[string]any{"server": map[string]any{"timeout": "30s"}}),
    transform.SetIfAbsent("server.timeout", "30s"),
    transform.EnvSubst(),                           // ${VAR} / ${VAR:-default}
    transform.DeletePaths("internal.debug"),
    transform.Aliases(map[string]string{"db.url": "database.dsn"}),
)

Struct tags

type AppConfig struct {
    Server struct {
        Addr    string        `json:"addr"    fc:"default=:8080"`
        Timeout time.Duration `json:"timeout" fc:"default=30s"`
    } `json:"server"`
    Database struct {
        DSN string `json:"dsn" fc:"secret"` // redacted in logs/snapshots
    } `json:"database"`
}

Migration

fastconf.WithMigrations(func(root map[string]any) error {
    if v, ok := root["db_url"]; ok {
        db, _ := root["database"].(map[string]any)
        if db == nil { db = map[string]any{}; root["database"] = db }
        if _, has := db["dsn"]; !has { db["dsn"] = v }
        delete(root, "db_url")
    }
    return nil
})

For multi-step schema migrations use transform.New.

---

Watch, Subscribe, and Plan

Field-level Subscribe

Subscribe fires the callback only when the extracted value actually changes (DeepEqual on the dereferenced values). Pass WithEqual for a custom comparator — e.g. ignore noisy fields, hash-compare large structs, or force a side effect on every committed reload.

cancel := fastconf.Subscribe(mgr,
    func(app *AppConfig) *DatabaseConfig { return &app.Database },
    func(old, neu *DatabaseConfig) {
        reconnect(neu.DSN) // guaranteed: DB config actually changed
    },
)
defer cancel()

// Custom equality (ignore Pool).
fastconf.Subscribe(mgr,
    func(app *AppConfig) *DatabaseConfig { return &app.Database },
    func(_, neu *DatabaseConfig) { warmCache(neu) },
    fastconf.WithEqual(func(a, b *DatabaseConfig) bool { return a.DSN == b.DSN }),
)

// Fire on every reload regardless of value.
fastconf.Subscribe(mgr,
    func(app *AppConfig) *AppConfig { return app },
    func(_, neu *AppConfig) { auditEveryReload(neu) },
    fastconf.WithEqual(func(_, _ *AppConfig) bool { return false }),
)

Manual reload with one-shot override

err := mgr.Reload(ctx,
    fastconf.WithReloadReason("admin-cli"),
    fastconf.WithSourceOverride(map[string]any{
        "server": map[string]any{"addr": ":9999"},
    }),
)

Plan (dry-run)

result, err := mgr.Plan().WithHostname("ci-runner-7").Run(ctx)
// result.Validators — validation errors
// result.Policies   — policy violations (SeverityError downgraded to warning in dry-run)

Pause / Resume

mgr.Watcher().Pause()
applyBatchUpdate()
mgr.Watcher().Resume()

---

Provenance, history & rollback

Provenance

mgr, _ := fastconf.New[AppConfig](ctx,
    fastconf.WithDir("conf.d"),
    fastconf.WithProvenance(fastconf.ProvenanceFull),
)

origins := mgr.Snapshot().Explain("server.addr")
// each Origin: Source.Name, Source.Priority, Value

Level	Cost	What you can trace
ProvenanceOff	zero	nothing
ProvenanceTopLevel	O(top-level keys)	which layer set each top-level field
ProvenanceFull	O(leaves)	full override chain per leaf

History & rollback

mgr, _ := fastconf.New[AppConfig](ctx,
    fastconf.WithHistory(10),
)
history := mgr.Replay().List()     // []*State[T], oldest → newest
_ = mgr.Replay().Rollback(history[len(history)-2])

Errors stream

go func() {
    for re := range mgr.Errors() {
        slog.Error("reload failed", "reason", re.Reason, "err", re.Err)
    }
}()

---

Observability

// JSON-lines audit on each successful reload
mgr, _ := fastconf.New[AppConfig](ctx,
    fastconf.WithAuditSink(fastconf.NewJSONAuditSink(os.Stderr)),
    fastconf.WithDiffReporter(fastconf.DiffReporterFunc(
        func(ctx context.Context, ev fastconf.DiffEvent) error {
            return slack.Post(ctx, ev.Diff) // async, never blocks reload
        },
    )),
)

Prometheus metrics and OpenTelemetry tracing live in sub-modules:

import prommetrics "github.com/fastabc/fastconf/observability/metrics/prometheus"
import fastconfotel "github.com/fastabc/fastconf/observability/otel"

fastconf.WithMetrics(prommetrics.New())
fastconf.WithTracer(fastconfotel.NewTracer(otel.GetTracerProvider()))

Policy violations abort reload at SeverityError; SeverityWarning logs and continues. CUE and OPA implementations in cue/policy and policy/opa.

---

Multi-tenant & presets

// Multi-tenant: each tenant is a fully isolated Manager[T]
tm := fastconf.NewTenantManager[AppConfig]()
mgrA, _ := tm.Add(ctx, "tenant-a", fastconf.WithDir("/etc/config/tenant-a"))
app, err := tm.Get("tenant-a")  // fastconf.ErrUnknownTenant if absent
tm.Close()

// Presets
fastconf.PresetK8s(fastconf.K8sOpts{Dir: "/etc/config", Watch: true})
fastconf.PresetSidecar(fastconf.SidecarOpts{Dir: "/etc/fastconfd", HistoryN: 16})
fastconf.PresetTesting(fastconf.TestingOpts{FS: memFS, Profile: "testing"})

---

Sub-module ecosystem

Shipped with the root module

Package	Path
contracts	contracts — public interfaces
reusable primitives	codec, confmap, overlay, transform, feature, providers/{env,cliflag,dotenv,labels,source}
http / vault / consul	providers/{http,vault,consul} — build tags: no_provider_{http,vault,consul}
nats / redis-streams	providers/{nats,redisstream} — caller injects the transport client
policy	policy — Func adapter
sidecar service	cmd/fastconfd
CLI tools	cmd/{fastconfctl,fastconfgen}
integrations	integrations/{bus,openfeature,render}

Independent sub-modules (go get as needed)

Sub-module	Path	Primary dependency
validate/playground	validate/playground	go-playground/validator
prometheus	observability/metrics/prometheus	prometheus/client_golang
otel	observability/otel	OpenTelemetry SDK
cue (validation + policy)	cue	cuelang.org/go
opa-policy	policy/opa	open-policy-agent/opa
cli/pflag	integrations/cli/pflag	spf13/pflag
s3 provider	providers/s3	AWS SDK v2

Tag all sub-modules at once: ./tools/tag-release.sh vX.Y.Z [--push]

---

CLI tools

fastconfd — sidecar service

fastconfd --dir=/etc/config --profile=prod --addr=:8081

Endpoint	Method	Description
/healthz	GET	Plain text ok once the first reload succeeded
/version	GET	Version, generation, hash, load time, reason
/config	GET	Current config JSON; pass ?redact=true for masking
/dump	GET	Deterministic YAML (?format=json for JSON)
/reload	POST	Trigger a manual reload
/events	GET	SSE stream of ReloadCause on each successful reload

fastconfctl — admin CLI

fastconfctl dump     -dir conf.d -profile prod
fastconfctl diff     -dir conf.d -from dev -to prod --json
fastconfctl validate -dir conf.d -profile prod
fastconfctl explain  -dir conf.d -profile prod database.dsn

fastconfgen — code generator

fastconfgen -in conf.d/base/00-app.yaml -pkg config -type Config -out config/config_gen.go

---

Performance

Most recent benchmark run: Apple M2 / darwin-arm64 / Go 1.26.2.

Benchmark	median
BenchmarkGet	0.52 ns/op
BenchmarkReloadNoop	15.1 µs/op
BenchmarkReloadCommitSmall	16.5 µs/op
BenchmarkReloadManySubscribers/50	17.5 µs/op

Full baseline: docs/design/perf.md.

---

Development

go mod tidy
make build
make test        # go test -race -count=1 ./...
make test-all    # includes sub-modules
make lint        # requires golangci-lint

go test ./... -run '^Example' -v
go test -bench=BenchmarkGet -benchmem ./...

---

Documentation

Doc	Purpose
docs/readme/	In-depth chapters: core model, pipeline, extensions, operations
docs/cookbook/README.md	Ready recipes ordered by user journey
docs/design/spec.md	Runtime model, concurrency, module boundaries
docs/cookbook/migration-v0.19.md	v0.19 Subscribe diff-aware migration notes
GitHub Releases	Release notes and prebuilt CLI binaries
pkg.go.dev	godoc and runnable examples

Common recipes: k8s · vault · consul · secrets · features · policy · otel · tenant · sidecar · plan

---

License

MIT License, See LICENSE.

Copyright (c) 2026 FastAbc