diff --git a/.agents/skills/observability-and-env/SKILL.md b/.agents/skills/observability-and-env/SKILL.md new file mode 100644 index 0000000..3b5e5a1 --- /dev/null +++ b/.agents/skills/observability-and-env/SKILL.md @@ -0,0 +1,317 @@ +--- +name: observability-and-env +description: 'Use when adding structured logging (pino), centralized environment + validation (Zod), or Sentry initialization to a TanStack Start app. Teaches + the three-file bootstrap pattern (instrument.env.mts → instrument.shared.mts → + instrument.server.mts; emitted as .mjs for production), the src/env/ schema + split (server vs public), and the createModuleLogger / createServerLogger + factory pattern that eliminates scattered process.env access from application + code. Project: TanStack AI-Promptable Full-Stack Template. Triggers on "add + logging", "set up pino", "pino logger", "sentry init", "instrument server", + "instrument.server.mts", "instrument.server.mjs", "env schema", "environment + validation", "centralize observability", "createModuleLogger", + "createServerLogger", "webEnv", "webServerEnv", "LOG_LEVEL", "SENTRY_DSN".' +--- + +> This file is generated from `skills/src/*.skill.yaml`. Do not edit manually. +# Observability and Environment Setup + +**Purpose:** Establish a clean observability stack — validated env schemas, +structured pino logging, and centralized Sentry bootstrap — following the +patterns proven in production TanStack Start apps. Keeps `process.env` access +confined to two files; application code receives typed, validated values as +arguments. + +## Key invariants (do not violate) + +1. `process.env` is read **only** in `src/env/*.ts` (schema parse) and in a + single `BootstrapEnvSchema.parse(process.env)` call inside + `instrument.env.mts` (Sentry bootstrap — runs before the app entry). +2. Logger options (`logLevel`, `environment`) are **passed as arguments** to + `createModuleLogger` — the factory never reads `process.env`. +3. **Browser public env** — no `window.__ENV__`; use a route `loader` that calls a GET server function returning `webPublicEnv`. Do not import `webEnv` from client-shared route files. +4. The root pino logger is created **once** per process (lazy singleton); all + module loggers are `child()` instances of it. + +## File layout + +``` +src/env/ + runtimeEnvSchema.ts # DeploymentEnv, LogLevel, shared preprocessors + webEnv.ts # WebServerEnvSchema + WebPublicEnvSchema; parsed once + +src/utils/ + logger.ts # createModuleLogger(name, { environment, logLevel? }) + serverLogger.ts # createServerLogger(name) — binds webServerEnv + +src/middleware/ + webEnv.ts # webEnvMiddleware: injects ctx.context.publicEnv + +instrument.env.shared.mts # shared DeploymentEnvSchema for bootstrap + TS callers +instrument.env.mts # resolveSentryBootstrapEnv() +instrument.shared.mts # initSentry({ dsn, environment, serverName, release }) +instrument.server.mts # bootstrap entry: resolve + init (dev); emitted .mjs in .output/server for prod +tsconfig.instrument.json +``` + +## src/env/runtimeEnvSchema.ts + +Shared Zod enums and preprocessors used by web env (and any future pipeline env). + +```typescript +import { z } from 'zod' +import { DEPLOYMENT_ENV_VALUES, type DeploymentEnv, DeploymentEnvSchema } from '../../instrument.env.shared.mjs' + +/** Empty / whitespace-only strings → undefined (Node process.env values are strings). */ +export function envStringToUndefined(val: unknown): unknown { + if (val === undefined || val === null) return undefined + const s = String(val).trim() + return s === '' ? undefined : s +} + +export type { DeploymentEnv } +export { DEPLOYMENT_ENV_VALUES, DeploymentEnvSchema } + +export const LOG_LEVEL_VALUES = ['fatal', 'error', 'warn', 'info', 'debug', 'trace', 'silent'] as const +export const LogLevelSchema = z.enum(LOG_LEVEL_VALUES) +export type LogLevel = z.infer + +export const OptionalDeploymentEnvSchema = z.preprocess(envStringToUndefined, DeploymentEnvSchema.optional()) +export const OptionalLogLevelSchema = z.preprocess(envStringToUndefined, LogLevelSchema.optional()) +export const OptionalTrimmedStringSchema = z.preprocess(envStringToUndefined, z.string().optional()) +``` + +## src/env/webEnv.ts + +Parsed once when first imported. `WebPublicEnvSchema` is the browser-safe +slice — only non-secret fields. `WebServerEnvSchema` adds secrets. + +```typescript +import { z } from 'zod' +import { + OptionalDeploymentEnvSchema, + OptionalLogLevelSchema, + OptionalTrimmedStringSchema, +} from './runtimeEnvSchema' + +export const WebPublicEnvSchema = z.object({ + ENV: OptionalDeploymentEnvSchema.describe('Deployment name: development, staging, or production.'), + LOG_LEVEL: OptionalLogLevelSchema.describe('Minimum pino log level.'), + SENTRY_DSN: OptionalTrimmedStringSchema.describe('Sentry DSN for both server and browser.'), +}) +export type WebPublicEnv = z.infer + +export const WebServerEnvSchema = WebPublicEnvSchema.extend({ + // ... app-specific required/optional vars + AUTH_HEADER_NAME: z.string().optional(), +}) +export type WebServerEnv = z.infer + +export const webServerEnv: WebServerEnv = WebServerEnvSchema.parse(process.env) + +export const webPublicEnv: WebPublicEnv = { + ENV: webServerEnv.ENV, + LOG_LEVEL: webServerEnv.LOG_LEVEL, + SENTRY_DSN: webServerEnv.SENTRY_DSN, +} +``` + +**Add required secrets** (API keys, DB URIs) to `WebServerEnvSchema` only — +they must never appear in `WebPublicEnvSchema`. + +## src/utils/logger.ts + +Pino factory. No `process.env` access — env values come from the caller. + +```typescript +import pino, { type Logger } from 'pino' +import type { DeploymentEnv, LogLevel } from '../env/runtimeEnvSchema' + +export type ModuleLoggerOptions = { + environment: DeploymentEnv // validated by caller's env schema + logLevel?: LogLevel // validated by caller's env schema +} + +let rootLogger: Logger | null = null + +function getRootLogger(environment: DeploymentEnv): Logger { + if (rootLogger) return rootLogger + // Server-only — not safe for client bundles. + const isNodeTty = typeof process !== 'undefined' && process.stdout != null && Boolean(process.stdout.isTTY) + const useTtyPretty = isNodeTty && environment !== 'production' + // Root at 'trace' so child level overrides are never filtered out + rootLogger = useTtyPretty + ? pino({ level: 'trace' }, pino.transport({ + target: 'pino-pretty', + options: { colorize: true, singleLine: true, translateTime: 'HH:MM:ss.l' }, + })) + : pino({ level: 'trace' }) + return rootLogger +} + +export function createModuleLogger(name: string, options: ModuleLoggerOptions): Logger { + const { environment } = options + return getRootLogger(environment).child({ name, environment }, { level: options.logLevel ?? 'info' }) +} +``` + +## src/utils/serverLogger.ts + +Thin bound factory for server-side modules — eliminates repeated +`{ environment: webServerEnv.ENV, logLevel: webServerEnv.LOG_LEVEL }` boilerplate. + +```typescript +import { webServerEnv } from '../env/webEnv' +import { createModuleLogger } from './logger' + +/** Server-side logger factory pre-bound to webServerEnv options. */ +export const createServerLogger = (name: string) => + createModuleLogger(name, { environment: webServerEnv.ENV ?? 'development', logLevel: webServerEnv.LOG_LEVEL }) +``` + +Usage in any server module: +```typescript +import { createServerLogger } from '../utils/serverLogger' +const log = createServerLogger('myServerFn') +``` + +## instrument.env.mts + +TypeScript bootstrap module compiled to ESM for production. In dev, preload +`tsx` and import `instrument.server.mts` directly. Use `.mjs` extensions on +**relative imports between instrument files** so `moduleResolution: NodeNext` +maps to the emitted `.mjs` output. Bootstrap env stays validated in one place; +deployment enum is imported from `./instrument.env.shared.mjs` (source is +`.mts`). Keep strict: invalid `NODE_ENV` values fail at +`BootstrapEnvSchema.parse(process.env)`, and Sentry uses only `SENTRY_DSN`. + +```typescript +import { z } from 'zod' +import { DeploymentEnvSchema } from './instrument.env.shared.mjs' + +const BootstrapEnvSchema = z.object({ + NODE_ENV: DeploymentEnvSchema.optional(), + SENTRY_DSN: z.string().optional(), +}) + +export function resolveSentryBootstrapEnv() { + const env = BootstrapEnvSchema.parse(process.env) + return { + dsn: env.SENTRY_DSN, + environment: env.NODE_ENV ?? 'development', + } +} +``` + +## instrument.shared.mts + +Receives all values pre-resolved — no `process.env` reads inside. + +```typescript +import * as Sentry from '@sentry/tanstackstart-react' + +export type InitSentryOptions = { + serverName: string + dsn: string | undefined + environment: 'development' | 'staging' | 'production' + release?: string +} + +export function initSentry({ serverName, dsn, environment, release }: InitSentryOptions): void { + if (!dsn) return + Sentry.init({ + dsn, + environment, + serverName, + ...(release ? { release } : {}), + sendDefaultPii: true, + tracesSampleRate: environment === 'production' ? 0.1 : 1.0, + }) +} +``` + +## instrument.server.mts (bootstrap entry) + +```typescript +import { resolveSentryBootstrapEnv } from './instrument.env.mjs' +import { initSentry } from './instrument.shared.mjs' +import pkg from './package.json' with { type: 'json' } + +const { dsn, environment } = resolveSentryBootstrapEnv() +initSentry({ serverName: 'my-app', release: pkg.version, dsn, environment }) +``` + +**Dev** — preload `tsx` then this file, e.g. `NODE_OPTIONS='--import tsx --import ./instrument.server.mts'`. + +**Production** — `tsc -p tsconfig.instrument.json` emits `.mjs` beside the Vite server bundle; copy `package.json` into `.output/server` so the import above resolves. + +Update the `build` script: +```json +"build": "vite build && tsc -p tsconfig.instrument.json && cp package.json .output/server/package.json" +``` + +## webEnvMiddleware + +Injects `ctx.context.publicEnv` for server functions. For React, fetch public +env in a route loader via a GET server function (do not import `webEnv` in +client bundles). + +```typescript +// src/middleware/webEnv.ts +import { createMiddleware } from '@tanstack/react-start' +import { webPublicEnv } from '../env/webEnv' + +export const webEnvMiddleware = createMiddleware().server(({ next }) => + next({ context: { publicEnv: webPublicEnv } }) +) +``` + +Register in `src/start.ts`: +```typescript +import { webEnvMiddleware } from './middleware/webEnv' +// add to requestMiddleware array +``` + +Example server fn + loader pattern when a client component needs the slice: +```typescript +// serverFns.ts +export const getWebPublicEnv = createServerFn({ method: 'GET' }).handler(async () => webPublicEnv) + +// route loader +loader: async () => ({ publicEnv: await getWebPublicEnv() }) +``` + +## Updating call sites + +**observability/index.ts** — replace `process.env.SENTRY_DSN` with the +validated value from `webPublicEnv`: + +```typescript +import { webPublicEnv } from '../../env/webEnv' + +export function getObservability(options: GetObservabilityOptions): ObservabilityService { + const dsn = options.publicEnv?.SENTRY_DSN ?? webPublicEnv.SENTRY_DSN + if (!instance || instanceKey !== dsn) { + instance = dsn ? new SentryObservability() : new NoopObservability() + instanceKey = dsn + } + return instance +} +``` + +**middleware/auth.ts** — replace `process.env.AUTH_HEADER_NAME` with +`webServerEnv`: + +```typescript +import { webServerEnv } from '../env/webEnv' +const AUTH_HEADER_NAME = webServerEnv.AUTH_HEADER_NAME ?? 'Authorization' +``` + +## Checklist + +- [ ] `process.env` appears only in `src/env/*.ts` and one bootstrap schema parse in `instrument.env.mts` +- [ ] `createModuleLogger` / `createServerLogger` never call `process.env` +- [ ] `instrument.server.mts` uses `resolveSentryBootstrapEnv()` + `initSentry()`, and `pnpm build` emits `.output/server/instrument.*.mjs` +- [ ] `package.json` is copied next to the emitted instrument bundle so version import works +- [ ] Public env for the browser uses a GET server fn from route loaders (not `window.__ENV__`) +- [ ] `SENTRY_DSN` / `LOG_LEVEL` / `ENV` documented in `.env.example` diff --git a/.agents/skills/tanstack-promptable-fullstack-app-template/SKILL.md b/.agents/skills/tanstack-promptable-fullstack-app-template/SKILL.md index cbf5dad..0a6db2d 100644 --- a/.agents/skills/tanstack-promptable-fullstack-app-template/SKILL.md +++ b/.agents/skills/tanstack-promptable-fullstack-app-template/SKILL.md @@ -17,6 +17,8 @@ description: 'Use when scaffolding a new TanStack Start project, adding domain **Purpose:** Capture the **interface-first, schema-layered, AI-promptable** contract for TanStack Start apps from this template. Day-to-day conventions (UI kit, chat wiring, logging, tests) live in the repo’s **AGENTS.md** — use this skill for **architecture**, AGENTS.md for **operations**. > **Companion handbook:** [AGENTS.md](https://github.com/carlosvin/tanstack-fullstack-ai-template/blob/main/AGENTS.md) — structure, styling, auth snippets, Biome, testing/E2E, validation checklist, AI chat setup. +> +> **Companion skill:** `observability-and-env` — use it when changing logging, Sentry, environment schemas, or runtime config plumbing. Keep this skill focused on the architecture contract rather than the observability setup recipe. ## How to use this skill diff --git a/.env.example b/.env.example index c66f751..9c51039 100644 --- a/.env.example +++ b/.env.example @@ -18,11 +18,25 @@ # AZURE_OPENAI_DEPLOYMENT=gpt-4o # ============================================================================= -# Observability (Sentry) +# Observability (Sentry + pino) # ============================================================================= # Sentry DSN. Observability is disabled (no-op) when omitted. -# VITE_SENTRY_DSN=https://examplePublicKey@o0.ingest.sentry.io/0 +# SENTRY_DSN=https://examplePublicKey@o0.ingest.sentry.io/0 +# +# Deployment name — controls trace sampling and log routing. +# Allowed values: development | staging | production # ENV=development +# +# NOTE: Two separate env vars control deployment identity: +# NODE_ENV — used by Sentry's bootstrap (instrument.env.mts) to tag all events. +# ENV — used by the application (logger, sampling) for runtime behavior. +# Keep them consistent to avoid Sentry grouping events under the wrong environment +# (e.g. NODE_ENV=production + ENV=staging tags Sentry events as "production" while +# the app behaves as staging). In production, set both to the same value. +# +# Minimum pino log level. +# Allowed values: fatal | error | warn | info | debug | trace | silent +# LOG_LEVEL=info # ============================================================================= # Authentication diff --git a/.github/workflows/skills.yml b/.github/workflows/skills.yml index 16b3e10..c25370e 100644 --- a/.github/workflows/skills.yml +++ b/.github/workflows/skills.yml @@ -28,15 +28,15 @@ jobs: steps: - name: Checkout repository - uses: actions/checkout@v4 + uses: actions/checkout@v5 - name: Setup pnpm - uses: pnpm/action-setup@v4 + uses: pnpm/action-setup@v6 with: version: 10 - name: Setup Node.js - uses: actions/setup-node@v4 + uses: actions/setup-node@v6 with: node-version: 22 cache: pnpm diff --git a/AGENTS.md b/AGENTS.md index bd2f099..2d53bc0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -431,12 +431,46 @@ The chat endpoint is a TanStack Start file-based route at `/api/chat` with two s ## 9. Observability +### ObservabilityService interface + - **Interface**: `src/services/observability/types.ts` defines the `ObservabilityService` interface. -- **Implementations**: Sentry (`sentry.ts`) and no-op (`noop.ts`). Factory in `index.ts`. -- **Server Init**: `instrument.server.mjs` conditionally initializes the server-side SDK. +- **Implementations**: Sentry (`sentry.ts`) and no-op (`noop.ts`). Factory in `index.ts` selects based on `webPublicEnv.SENTRY_DSN`. - **Client Init**: `src/router.tsx` conditionally initializes client-side tracing. -- **Usage**: Wrap server function internals with `getObservability().startSpan('name', fn)`. -- **To swap providers**: Implement `ObservabilityService`, update the factory, and replace `instrument.server.mjs`. +- **Usage**: Wrap server function internals with `getObservability({}).startSpan('name', fn)`. +- **To swap providers**: Implement `ObservabilityService`, update the factory, and replace `instrument.server.mts` (and the emitted `instrument.server.mjs` in `.output/server`). + +### Sentry bootstrap + +Server-side Sentry is initialized via the `--import` hook before the app entry loads: + +| File (source) | Responsibility | +|------|---------------| +| `instrument.env.shared.mts` | Shared deployment env schema used by bootstrap and TypeScript callers (`runtimeEnvSchema.ts` re-exports it) | +| `instrument.env.mts` | Bootstrap resolver — parses `process.env` once; invalid `NODE_ENV` values fail schema validation | +| `instrument.shared.mts` | `initSentry({ serverName, dsn, environment, release })` — no `process.env` inside | +| `instrument.server.mts` | Bootstrap entry: resolve bootstrap env + init | + +**Build**: `pnpm build` runs `tsc -p tsconfig.instrument.json`, which emits `instrument.*.mjs` into `.output/server/` (alongside `cp package.json` so `instrument.server` can read the app version). **Production start** uses `node --import ./.output/server/instrument.server.mjs`. **Dev** preloads `tsx` then `./instrument.server.mts` via `NODE_OPTIONS`. + +From `src/**/*.ts`, import the shared schema module as `../../instrument.env.shared.mjs` (extension matches the compiled bootstrap output); with `moduleResolution: bundler`, TypeScript resolves it to the `instrument.env.shared.mts` source. The `instrument.*.mts` entrypoints import each other with `.mts` extensions so `tsx` can preload them in dev; `tsc -p tsconfig.instrument.json` emits `.mjs` and rewrites those specifiers via `rewriteRelativeImportExtensions`. + +**Key invariant**: `instrument.shared.mts` never reads `process.env`. All values are pre-resolved by the caller, bootstrap `NODE_ENV` is strictly schema-validated in `instrument.env.mts`, and Sentry is configured only through `SENTRY_DSN`. + +**NODE_ENV vs ENV**: Two separate variables control deployment identity. `NODE_ENV` drives Sentry's environment tag (set by the bootstrap in `instrument.env.mts`). `ENV` drives application behavior — logger sampling, pretty-printing, and log routing. A mismatch (e.g. `NODE_ENV=production` + `ENV=staging`) will silently tag all Sentry events as "production" while the app behaves as staging. Keep them consistent across deployments. + +### pino structured logging + +Application modules use the pino factory from `src/utils/logger.ts`: + +```typescript +import { createServerLogger } from '../utils/serverLogger' +const log = createServerLogger('myModule') // binds webServerEnv.ENV + LOG_LEVEL +``` + +- `createModuleLogger(name, { environment, logLevel? })` — core factory; no `process.env` access inside. +- `createServerLogger(name)` — bound factory for server modules; closes over `webServerEnv.ENV` and `webServerEnv.LOG_LEVEL`. +- Root pino instance is lazily created once per process; all module loggers are `child()` instances. +- TTY pretty-printing is auto-enabled for interactive Node terminals outside of production. ### App Version @@ -462,15 +496,7 @@ export default defineConfig({ declare const __APP_VERSION__: string ``` -**Sentry** — pass as `release` in `instrument.server.mjs` and in client init so errors are grouped by version: - -```javascript -Sentry.init({ - dsn: sentryDsn, - release: process.env.npm_package_version ?? 'unknown', - // ...existing config -}) -``` +**Sentry** — pass as `release` in `instrument.server.mts` (from `package.json`) and in client init so errors are grouped by version. For the client-side Sentry init, use the Vite-injected constant: @@ -489,51 +515,6 @@ export const logger = pino({ **Other tools** — any new observability integration should read `__APP_VERSION__` for the same purpose. The pattern ensures a single source of truth (`package.json`) with no manual version strings. -### Logging - -This project uses [`pino`](https://getpino.io/) as the default server-side structured logger. Do **not** use `console.log` / `console.error` / `console.warn` in server code — use the logger instead. - -**Setup**: The logger singleton lives in `src/services/logger.ts`. It conditionally adds the [`@sentry/pino-transport`](https://docs.sentry.io/platforms/javascript/guides/node/configuration/integrations/pino/) so that error-level logs (`logger.error(...)`) are automatically captured by Sentry when `VITE_SENTRY_DSN` is set. When no DSN is configured, pino logs to stdout with pretty-printing in development. - -```tsx -import { logger } from '../services/logger' - -logger.info({ repo: type }, 'Using repository') -logger.error({ err, taskId }, 'Failed to update task') -``` - -**Installation**: - -```bash -pnpm add pino -pnpm add -D pino-pretty # pretty-print in development -pnpm add @sentry/pino-transport # optional: forward errors to Sentry -``` - -**Configuration pattern** (`src/services/logger.ts`): - -```typescript -import pino from 'pino' - -const transport = process.env.VITE_SENTRY_DSN - ? pino.transport({ - targets: [ - { target: '@sentry/pino-transport', level: 'error' }, - { - target: process.env.NODE_ENV === 'production' ? 'pino/file' : 'pino-pretty', - level: 'info', - }, - ], - }) - : process.env.NODE_ENV === 'production' - ? undefined - : pino.transport({ target: 'pino-pretty' }) - -export const logger = pino({ level: 'info' }, transport) -``` - -When `VITE_SENTRY_DSN` is set, the Sentry transport receives error-level logs alongside the regular output target. When Sentry is not configured, the logger simply writes to stdout (pretty in dev, JSON in production). No code changes are needed when toggling Sentry on or off. - ## 10. Testing ### Unit Tests (Vitest) @@ -551,6 +532,7 @@ When `VITE_SENTRY_DSN` is set, the Sentry transport receives error-level logs al - **Auth fixture**: `e2e/auth.ts` provides `authedPage` / `authedContext` fixtures using unsigned JWTs sent via `extraHTTPHeaders`. - **Convention**: Spec files in `e2e/` as `*.spec.ts`. - **Running**: `pnpm test:e2e` (reuses existing dev server or starts one with seed data). +- **NODE_ENV**: The Playwright `webServer` config forces `NODE_ENV=development`. `instrument.env.mts` rejects values outside `development | staging | production`, so never start the dev server with `NODE_ENV=test`. ## 11. Linting and Formatting @@ -565,53 +547,47 @@ This project uses [Biome](https://biomejs.dev/) as the default linter and format ## 13. Public Runtime Config -Some config values differ per deployment (Sentry DSN, environment name, feature flags) but are **not secrets** and the client needs them immediately. Do not bake them into the Vite bundle via `import.meta.env` — that ties the built artifact to one environment. Instead, expose them through a GET server function and inline the result into the HTML document as `window.__ENV__` so the client can read them synchronously before any module runs. +Config values that differ per deployment (Sentry DSN, environment name, feature flags) and are safe for the browser **must not** be baked into the Vite bundle via `import.meta.env` — that ties the built artifact to one environment. They also **must not** be injected via `window.__ENV__` — that is insecure and fragile. -### Server function +Instead, expose them through the validated `WebPublicEnvSchema` slice and deliver them to the browser from a route `loader` that calls a GET server function when React actually needs that data. -```tsx -// src/services/api/serverFns.ts -export const getPublicEnv = createServerFn({ method: 'GET' }).handler(async () => { - return { - sentryDsn: process.env.VITE_SENTRY_DSN ?? null, - environment: process.env.ENV ?? 'development', - featureFlags: { - /* flags that are safe to expose */ - }, - } -}) -``` +### Environment schemas (`src/env/`) -### Inline into the document +`process.env` is read **only once**, at module import time: -Call `getPublicEnv()` in the root loader and inject the result via a `