CSPL-4577 Codex Agentic DX & Harness Foundation

.agents/skills/sok-ci-fixer/SKILL.md

@@ -0,0 +1,49 @@
+---
+name: sok-ci-fixer
+description: Analyze CI failures, map them to local repro commands, and propose fixes. Use when CI logs or pipeline failures need a local reproduction and patch.
+---
+
+# SOK CI Fixer
+
+## Overview
+Turn CI failures into a local repro and a minimal fix with a validation plan.
+
+## Scope
+Allowed paths:
+- `scripts/**`
+- `test/**`
+- `kuttl/**`
+- `config/**`
+- `api/**`
+- `internal/**`
+- `pkg/**`
+- `Makefile`, `go.mod`, `go.sum`
+
+Forbidden paths:
+- `vendor/**`
+- `bin/**`
+- `.git/**`
+
+If changes are needed outside the allowed paths, stop and propose a follow-up plan.
+
+## Workflow
+1. Summarize the CI failure (job name, step, error).
+2. Map to a local command (prefer `scripts/dev/pr_check.sh` or `./scripts/verify_repo.sh`).
+3. Reproduce locally if possible and capture the failing output.
+4. Implement the smallest safe fix.
+5. Re-run the local repro command.
+
+## Commands
+- PR gate: `scripts/dev/pr_check.sh`
+- Repo verify: `./scripts/verify_repo.sh`
+
+## Definition of Done
+- CI failure is reproducible locally or explained why not.
+- Fix is minimal and validated with a local repro command.
+- Regression risk is noted.
+
+## Output Contract
+- Changed files
+- Commands run
+- Results
+- PR-ready summary

.agents/skills/sok-ci-fixer/agents/openai.yaml

@@ -0,0 +1,3 @@
+interface:
+  display_name: "SOK CI Fixer"
+  short_description: "Triage and fix CI failures"

.agents/skills/sok-doc-updater/SKILL.md

@@ -0,0 +1,44 @@
+---
+name: sok-doc-updater
+description: Update Splunk Operator docs and examples for a change. Use when a change requires docs, CR examples, or user-facing guidance updates.
+---
+
+# SOK Doc Updater
+
+## Overview
+Keep docs, examples, and user-facing guidance in sync with code changes.
+
+## Scope
+Allowed paths:
+- `docs/**`
+- `README.md`
+- `config/samples/**`
+- `helm-chart/**`
+
+Forbidden paths:
+- `api/**`
+- `internal/**`
+- `pkg/**`
+- `test/**`
+- `kuttl/**`
+- `bundle/**`
+- `vendor/**`
+
+If product code changes are required, stop and hand off to the appropriate skill.
+
+## Workflow
+1. Identify the user-facing change and affected docs.
+2. Update spec fields, examples, and any compatibility notes.
+3. Verify examples are consistent with current CRD schema.
+4. Provide a short summary and a test/validation note if applicable.
+
+## Definition of Done
+- Docs and examples are consistent with current CRD schema.
+- User-facing behavior is clearly described.
+- Any required follow-up (tests or validation) is documented.
+
+## Output Contract
+- Changed files
+- Commands run
+- Results
+- PR-ready summary

.agents/skills/sok-doc-updater/agents/openai.yaml

@@ -0,0 +1,3 @@
+interface:
+  display_name: "SOK Doc Updater"
+  short_description: "Update docs and examples"

.agents/skills/sok-feature-scaffold/SKILL.md

@@ -0,0 +1,111 @@
+---
+name: sok-feature-scaffold
+description: Add or change Splunk Operator behavior by introducing a new field in a CRD spec/status, wiring it into reconciliation, and updating tests and docs. Use for changes to Standalone, IndexerCluster, SearchHeadCluster, ClusterManager, LicenseManager, MonitoringConsole, or shared CRD config. Do not use for pure refactors, dependency bumps, or formatting-only changes.
+---
+
+# SOK Feature Scaffold
+
+## Overview
+Implement CRD-driven features end-to-end with code, tests, and docs in this repository.
+
+## Scope
+Allowed paths:
+- `api/**`
+- `internal/controller/**`
+- `pkg/**`
+- `config/**`
+- `docs/**`
+- `test/**`
+- `kuttl/**`
+- `bundle/**`
+- `helm-chart/**`
+- `scripts/**`
+- `Makefile`, `README.md`, `PROJECT`, `go.mod`, `go.sum`
+
+Forbidden paths:
+- `vendor/**`
+- `bin/**`
+- `.git/**`
+
+If changes are needed outside the allowed paths, stop and propose a follow-up plan.
+
+## Workflow
+1. Print the files you plan to change and the test commands you will run before editing.
+2. Identify the target CRD kind and API version. Confirm the mapping in `PROJECT` and `docs/agent/CRD_MAP.md`.
+3. Locate the API types under `api/v*/` and update the spec/status struct.
+4. Add JSON tags, `omitempty` rules, and kubebuilder markers consistent with adjacent fields.
+5. Update any defaulting or validation logic that applies to the new field.
+6. Regenerate CRD/RBAC artifacts via the operator-sdk workflow.
+7. Wire the field into reconciliation with idempotent logic.
+8. Update or add unit tests, and add an integration test stub when relevant.
+9. Update docs and examples to expose the new field.
+10. Produce a PR-ready summary with tests and risks.
+
+## Implementation Details
+
+### 1) Find the right API types
+- `PROJECT` is the source of truth for kind and version mapping.
+- Use `docs/agent/CRD_MAP.md` for fast navigation to types, controllers, and enterprise logic.
+- Prefer the latest stable API version (typically `api/v4`).
+- Legacy kinds still use `api/v3` (legacy cluster manager and legacy license manager).
+- Use `rg "type .*Spec" api -g "*_types.go"` to locate the spec struct.
+- If the field is shared across CRDs, check `api/v4/common_types.go` (and `api/v3/common_types.go` for legacy kinds).
+
+### 2) Schema, CRD, and RBAC generation (operator-sdk workflow)
+- Use `operator-sdk create api` when introducing a new API or controller (scaffolding).
+- Add the field with a clear JSON name and `omitempty` as appropriate.
+- Follow nearby kubebuilder markers for validation, defaults, and list/map behavior.
+- Regenerate code and manifests with the repo targets (operator-sdk scaffolding uses controller-gen under the hood).
+`make generate` for deepcopy code.
+`make manifests` for CRDs and RBAC.
+Run `make bundle` to refresh `bundle/manifests/*` and `helm-chart/splunk-operator/crds` when bundle or Helm CRDs are tracked.
+- For verification, use `./scripts/verify_crd.sh` and optionally `./scripts/verify_bundle.sh` or `make verify VERIFY_BUNDLE=1`.
+- If you add new RBAC needs, update kubebuilder RBAC markers in the controller and re-run `make manifests` to refresh `config/rbac/role.yaml`.
+
+### 3) Reconcile wiring
+- Locate the controller in `internal/controller` and shared logic in `pkg/splunk/enterprise` or `pkg/splunk/common`.
+- Read the new field from the spec and apply it in a single, idempotent reconciliation path.
+- Update status only when the desired state is reached and avoid hot-looping.
+
+### 4) Tests
+- Add or update unit tests near the logic you touched (often under `internal/controller` or `pkg/splunk/*`).
+- If the behavior is user-visible or multi-resource, add a minimal integration test stub in `test/` or `kuttl/` to document coverage intent.
+- Prefer helper scripts when available: `scripts/dev/unit.sh`, `scripts/dev/lint.sh`, `scripts/dev/pr_check.sh`.
+
+### 5) Docs
+- Update `docs/CustomResources.md` for spec fields.
+- Update any feature-specific doc under `docs/` and add an example manifest if needed.
+
+## Definition of Done
+- CRD/schema generation is updated and verified.
+- Reconcile logic is idempotent and status updates are gated.
+- Tests added/updated for the new behavior.
+- Docs/examples reflect the new field or behavior.
+
+## Assets
+- Use `assets/pr-template.md` for the PR summary format.
+- Use `assets/crd-change-checklist.md` as a guardrail for CRD edits.
+
+## Key Paths
+- API types: `api/v*/` (e.g. `api/v4/*_types.go`)
+- Controllers: `internal/controller/`
+- Shared logic: `pkg/splunk/enterprise`, `pkg/splunk/common`
+- CRDs: `config/crd/bases/`
+- RBAC output: `config/rbac/role.yaml`
+- Bundles: `bundle/manifests/`, `helm-chart/splunk-operator/crds`
+- Docs: `docs/CustomResources.md`, `docs/Examples.md`
+
+## Repo Map (Common Cases)
+- Standalone: `api/v4/standalone_types.go`, `internal/controller/standalone_controller.go`, `pkg/splunk/enterprise/standalone.go`
+- IndexerCluster: `api/v4/indexercluster_types.go`, `internal/controller/indexercluster_controller.go`, `pkg/splunk/enterprise/indexercluster.go`
+- SearchHeadCluster: `api/v4/searchheadcluster_types.go`, `internal/controller/searchheadcluster_controller.go`, `pkg/splunk/enterprise/searchheadcluster.go`
+- ClusterManager: `api/v4/clustermanager_types.go`, `internal/controller/clustermanager_controller.go`, `pkg/splunk/enterprise/clustermanager.go`
+- LicenseManager: `api/v4/licensemanager_types.go`, `internal/controller/licensemanager_controller.go`, `pkg/splunk/enterprise/licensemanager.go`
+- MonitoringConsole: `api/v4/monitoringconsole_types.go`, `internal/controller/monitoringconsole_controller.go`, `pkg/splunk/enterprise/monitoringconsole.go`
+- Legacy v3 control-plane types/controllers: search under `api/v3/`, `internal/controller/`, and `pkg/splunk/enterprise/`
+
+## Output Contract
+- Changed files
+- Commands run
+- Results
+- PR-ready summary

.agents/skills/sok-feature-scaffold/agents/openai.yaml

@@ -0,0 +1,3 @@
+interface:
+  display_name: "Sok Feature Scaffold"
+  short_description: "Help with Sok Feature Scaffold tasks"

.agents/skills/sok-feature-scaffold/assets/crd-change-checklist.md

@@ -0,0 +1,11 @@
+## CRD Change Checklist
+- [ ] Update the correct `api/v*/` spec or status struct
+- [ ] Add JSON tags and `omitempty` consistently
+- [ ] Add kubebuilder markers for validation or defaults
+- [ ] Run `make manifests`
+- [ ] Verify `config/rbac/role.yaml` if RBAC markers changed
+- [ ] If bundle/Helm CRDs are tracked, run `make bundle`
+- [ ] Run `make verify` (optionally `VERIFY_BUNDLE=1`) to confirm outputs
+- [ ] Verify generated YAML under `config/crd/bases/` and `bundle/manifests/`
+- [ ] Update docs under `docs/`
+- [ ] Add or update tests

.agents/skills/sok-feature-scaffold/assets/pr-template.md

@@ -0,0 +1,8 @@
+## Summary
+- 
+
+## Tests
+- 
+
+## Risks / Follow-ups
+- 

.agents/skills/sok-issue-triage/SKILL.md

@@ -0,0 +1,79 @@
+---
+name: sok-issue-triage
+description: Turn a Splunk Operator issue report into scope, impacted components, proposed change list, test plan, and risks. Use when asked to triage a GitHub issue, bug report, or feature request into a PR plan.
+---
+
+# SOK Issue Triage
+
+## Overview
+Convert issue context into a PR-ready plan with scope, changes, tests, and risks.
+
+## Scope
+Allowed paths:
+- `.agents/**`
+- `docs/**`
+- `templates/**`
+
+Forbidden paths:
+- `api/**`
+- `internal/**`
+- `pkg/**`
+- `test/**`
+- `kuttl/**`
+- `config/**`
+- `bundle/**`
+- `helm-chart/**`
+- `vendor/**`
+
+This skill should not change product code. If code changes are required, stop and hand off to the appropriate skill.
+
+## Workflow
+1. Extract the problem statement and expected behavior.
+2. Identify impacted CRDs, controllers, and packages.
+3. Determine the minimal scope for a safe fix.
+4. Propose the change list in implementation order.
+5. Define a concrete test plan.
+6. Call out risks, migrations, and backward compatibility.
+
+## Details
+
+### 1) Parse the issue
+- Capture user intent, repro steps, and current behavior.
+- Identify the CR kind and any referenced fields.
+
+### 2) Map to code
+- Find the spec/status types under `api/v*/`.
+- Locate controllers in `internal/controller`.
+- Identify shared helpers in `pkg/splunk/enterprise` or `pkg/splunk/common`.
+- Use `PROJECT` to confirm CRD kind and version mapping.
+- Use `docs/agent/CRD_MAP.md` for a fast file map.
+- Use `docs/agent/RECONCILE_FLOW.md` for flow and phase context.
+
+### 3) Build a PR plan
+- List files or directories to touch.
+- Keep the change list ordered: schema, reconcile logic, tests, docs.
+
+### 4) Test plan
+- Prefer `make test` for unit coverage.
+- Propose an integration test or minimal stub when behavior is user-visible.
+
+## Definition of Done
+- Scope, impacted components, and change list are explicit.
+- Test plan is concrete and executable.
+- Risks and open questions are called out.
+
+## Key Paths
+- API types: `api/v*/`
+- Controllers: `internal/controller/`
+- Shared logic: `pkg/splunk/enterprise`, `pkg/splunk/common`
+- Docs: `docs/`
+- Project mapping: `PROJECT`
+- Agent docs: `docs/agent/CRD_MAP.md`, `docs/agent/RECONCILE_FLOW.md`
+
+## Output Contract
+- Changed files
+- Commands run
+- Results
+- PR-ready summary
+
+Use `assets/issue-triage-template.md` for the final structure and include open questions if any context is missing.

.agents/skills/sok-issue-triage/agents/openai.yaml

@@ -0,0 +1,3 @@
+interface:
+  display_name: "Sok Issue Triage"
+  short_description: "Help with Sok Issue Triage tasks"

.agents/skills/sok-issue-triage/assets/issue-triage-template.md

@@ -0,0 +1,17 @@
+## Scope
+- 
+
+## Impacted Components
+- 
+
+## Proposed Changes
+- 
+
+## Test Plan
+- 
+
+## Risks / Compatibility
+- 
+
+## Open Questions
+- 

.agents/skills/sok-new-crd-controller/SKILL.md

@@ -0,0 +1,57 @@
+---
+name: sok-new-crd-controller
+description: Create a new CRD and controller skeleton (operator-sdk), wire RBAC, add sample YAML, and add tests/docs. Use when introducing a brand-new custom resource to Splunk Operator.
+---
+
+# SOK New CRD + Controller
+
+## Overview
+Scaffold and wire a new CRD + controller end-to-end, with RBAC, samples, tests, and docs.
+
+## Scope
+Allowed paths:
+- `api/**`
+- `internal/controller/**`
+- `cmd/**`
+- `config/**`
+- `docs/**`
+- `test/**`
+- `kuttl/**`
+- `bundle/**`
+- `helm-chart/**`
+- `scripts/**`
+- `PROJECT`, `Makefile`, `go.mod`, `go.sum`
+
+Forbidden paths:
+- `vendor/**`
+- `bin/**`
+- `.git/**`
+
+If changes are needed outside the allowed paths, stop and propose a follow-up plan.
+
+## Workflow
+1. Print the files you plan to change and test commands you will run.
+2. Run `operator-sdk create api` (if a brand-new API) and confirm entries in `PROJECT`.
+3. Implement the spec/status types and kubebuilder markers in `api/v*/`.
+4. Wire the controller under `internal/controller/` and register in `cmd/main.go`.
+5. Update RBAC markers and regenerate manifests: `make generate manifests` or `./scripts/verify_crd.sh`.
+6. Add a sample CR under `config/samples/`.
+7. Add unit tests (and an integration stub if user-visible behavior).
+8. Update docs and examples.
+
+## Notes
+- Prefer scripts when available: `scripts/dev/unit.sh`, `scripts/dev/lint.sh`, `scripts/dev/pr_check.sh`.
+- Ensure CRD output is updated in `config/crd/bases/` and, if tracked, `bundle/` and `helm-chart/`.
+
+## Definition of Done
+- New CRD types are generated and registered in `PROJECT`/`cmd/main.go`.
+- RBAC and CRD manifests are regenerated and verified.
+- Sample YAML exists under `config/samples/`.
+- At least one test (unit or integration stub) is added.
+- Docs/examples updated for the new CRD.
+
+## Output Contract
+- Changed files
+- Commands run
+- Results
+- PR-ready summary