Skip to content

feat: add webhook support for GitHub events (AIE-13)#15

Merged
knechtionscoding merged 23 commits intoprodfrom
tim-aie-13
Mar 26, 2026
Merged

feat: add webhook support for GitHub events (AIE-13)#15
knechtionscoding merged 23 commits intoprodfrom
tim-aie-13

Conversation

@tmarshall
Copy link
Copy Markdown
Member

@tmarshall tmarshall commented Mar 25, 2026

What type of PR is this?

/kind feature

What this PR does / why we need it:

Adds webhook support for GitHub events to Kelos. TaskSpawners can now receive push notifications when issues or pull requests are created or updated, providing an alternative to API polling.

Which issue(s) this PR is related to:

Related to kelos-dev#687 (upstream Kelos webhook support request)

Linear ticket: AIE-13

Special notes for your reviewer:

Architecture: CRD-Based Queue

Webhook payloads are stored as WebhookEvent custom resources in Kubernetes. This is consistent with Kelos' existing architecture where all state lives in Kubernetes CRDs backed by etcd.

Alternative approaches considered:

  • In-memory queue with ConfigMap snapshots: Faster but loses events on crash between snapshots
  • External queue (Redis/RabbitMQ): Adds infrastructure dependency and is not Kubernetes-native

The CRD-based approach provides:

  • Persistence across pod restarts (etcd-backed)
  • Visibility via kubectl (kubectl get webhookevents)
  • Built-in RBAC and audit logging
  • Consistency with existing Kelos patterns

Components:

  1. WebhookEvent CRD (api/v1alpha1/webhookevent_types.go)

    • Custom resource for storing webhook payloads
    • Spec: source, payload, receivedAt
    • Status: processed flag, processedAt timestamp
  2. Webhook Receiver (cmd/kelos-webhook-receiver/main.go)

    • HTTP server listening on /webhook/github
    • Validates GitHub webhook signatures (HMAC-SHA256)
    • Creates WebhookEvent CRD instances
    • Returns 202 Accepted
  3. GitHub Webhook Source (internal/source/github_webhook.go)

    • Implements Source interface
    • Lists unprocessed WebhookEvent resources with source=github
    • Parses GitHub issue and pull_request payloads
    • Converts to WorkItem format
    • Marks events as processed
    • Supports label-based filtering (client-side)
  4. TaskSpawner CRD Updates (api/v1alpha1/taskspawner_types.go)

    • Adds githubWebhook field to When struct
  5. Unit Tests

    • Webhook receiver: signature validation, HTTP handling
    • GitHub webhook source: payload parsing, label filtering, discovery flow
  6. Documentation & Examples

    • docs/webhooks.md: Architecture overview and setup instructions
    • examples/taskspawner-github-webhook.yaml: Complete example with receiver deployment

Usage example:

apiVersion: kelos.dev/v1alpha1
kind: TaskSpawner
metadata:
  name: my-webhook-spawner
spec:
  when:
    githubWebhook:
      namespace: default
      labels: ["kelos-task"]
      excludeLabels: ["skip"]
  taskTemplate:
    type: claude-code
    credentials:
      type: api-key
      secretRef:
        name: anthropic-api-key
    workspaceRef:
      name: my-workspace

Compatibility with main:

This implementation is compatible with the upstream main branch:

  • No changes to existing API polling sources
  • Webhook support is opt-in via githubWebhook field
  • No breaking changes to existing TaskSpawner specs

TODO before merging:

  • Run make update in Go environment to generate:
    • Deep copy code (zz_generated.deepcopy.go)
    • CRD manifests
  • Add unit tests for webhook receiver signature validation
  • Add unit tests for GitHub webhook source payload parsing
  • Add RBAC permissions for WebhookEvent resources
  • Add integration test for end-to-end flow (requires live cluster)
  • Test in development cluster

Does this PR introduce a user-facing change?

Adds webhook support for GitHub events. TaskSpawners can now use `when.githubWebhook` to receive push notifications for issues and pull requests. Requires deploying the new `kelos-webhook-receiver` component. See `docs/webhooks.md` for setup instructions.

Defines WebhookEvent custom resource for storing incoming webhooks.
Uses CRD-based queue pattern consistent with Kelos architecture.

Note: Requires 'make update' in Go environment to generate deepcopy
and CRD manifests.
Implements HTTP endpoint at /webhook/:source that:
- Receives webhook payloads
- Validates GitHub signatures via HMAC-SHA256
- Creates WebhookEvent CRD instances
- Returns 202 Accepted

Supports multiple sources (github, slack, linear, etc.) via URL path.
Implements Source interface for webhook-based discovery:
- Reads unprocessed WebhookEvent CRDs with source=github
- Parses GitHub issue and pull_request webhook payloads
- Converts to WorkItem format
- Marks events as processed after discovery
- Supports label-based filtering (client-side)
Adds githubWebhook field to TaskSpawner CRD allowing webhook-based
discovery as an alternative to API polling. The spawner will watch
WebhookEvent resources in the specified namespace and convert GitHub
webhook payloads to tasks.
Updates spawner to create GitHubWebhookSource instances when
TaskSpawner has githubWebhook configured. Passes k8s client to
source so it can list and update WebhookEvent resources.
Includes:
- Complete TaskSpawner example using githubWebhook
- Webhook receiver deployment manifests
- Architecture documentation explaining CRD-based queue
- GitHub webhook setup instructions
- Comparison with API polling approach
Marks completed phases and notes what still needs to be done:
- CRD manifest generation (needs Go environment)
- Unit and integration tests
- Live cluster testing
tmarshall and others added 11 commits March 25, 2026 22:29
Webhook receiver tests:
- Valid/invalid GitHub signature validation
- Missing signature header handling
- No secret configured (dev mode)
- HTTP method validation
- Missing source in path

GitHub webhook source tests:
- Issue payload parsing
- Pull request payload parsing
- Closed issues/PRs are skipped
- Label filtering (required, excluded, multiple)
- End-to-end Discover with fake k8s client
- Events marked as processed after discovery
- Fix logger type in webhook receiver (logr.Logger)
- Add missing parameters to buildSource calls in tests
- Remove field selectors for fake client compatibility

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Fix HMAC-SHA256 signature in webhook receiver test
- Enable status subresource in fake client for webhook source test

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add webhookevents permissions to kelos-spawner-role
- Create kelos-webhook-receiver-role for webhook receiver

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Remove mentions of future sources (Slack, Linear, etc.)
- Remove comparison table with API polling
- Keep documentation focused on current implementation

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Test WebhookEvent discovery and processing
- Test label filtering (required and excluded labels)
- Test issue and pull request payloads
- Test source filtering (github vs other sources)
- Test skipping closed issues/PRs
- Verify events are marked as processed after discovery

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Mark all tasks as completed:
- RBAC permissions added
- Integration tests implemented
- All tests passing

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@tmarshall tmarshall marked this pull request as ready for review March 26, 2026 00:00
- Change from &eventList.Items[i] to DeepCopy() to fix status updates
- Mark events as processed even when filtered out or invalid
- Ensures all processed events have status updated in integration tests

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator

@knechtionscoding knechtionscoding left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I think this is fine. Might need to think through the routing part of it (i.e. different ingresses pointing at different services)?

We may also want the taskspawner, if it is a webhook source, to create the service with specific overrides to allow for clusterIP vs load balancer, annotations and labels, etc.

Comment thread docs/webhooks.md
@knechtionscoding knechtionscoding merged commit 5bd12a7 into prod Mar 26, 2026
8 checks passed
@knechtionscoding knechtionscoding deleted the tim-aie-13 branch March 26, 2026 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants