[pull] main from hookdeck:main #60
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* refactor: logstore interface & test suite & memlogstore implementation * chore: fix noop logstore * feat: cursor * test: cursor * chore: cursor support backward compat * chore: remove default time window query * feat: pglogstore implementation * test: idempotent insert * feat: chlogstore implementation * chore: update logstore usage with updated interface * chore: configure logstore with ch driver * test: e2e suite with ch * feat: implement delivery retrieve * feat: log api & retry implementation * chore: backward-compat support * test: e2e log api test cases * chore: gofmt * chore: api event * feat: add ListEvent to logstore layer Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add ClickHouse events table and implement ListEvent Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add ListEvents API handler and route Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add ListEvents e2e tests and fix path prefixes - Add TestListEventsAPI test function with comprehensive coverage: - List all events - Filter by destination_id - Filter by topic - Pagination with limit and next cursor - Time range filter - Fix all log_test.go tests to use correct /tenants/ prefix: - TestLogAPI - TestRetryAPI - TestLegacyLogAPI All 871 e2e tests pass. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: merge events migration into init and remove SupportsListEvent Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: correct API path prefixes in apirouter tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: reorganize e2e test suites and fix testinfra race condition - Fix race condition in testinfra.suiteCounter using atomic operations - Reorganize e2e suites with consistent naming convention: - TestE2E, TestE2E_WithDeploymentID (default, parallel) - TestE2E_Compat_* (TESTCOMPAT=1 only, sequential) - TestE2E_Regression_* (regression tests) - Default suites now use Dragonfly + ClickHouse configuration - Reduces default e2e test time from ~82s to ~28s via parallel execution * perf: replace e2e setup sleeps with health check polling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * perf: allow immediate log batch flush with threshold=0 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * perf: replace fixed sleeps with polling in log tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: replace fixed delay with polling in alert test Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: wait for all 3 deliveries in TestListEventsAPI The test publishes 3 events but was only waiting for 1 delivery, causing a race condition where the events query could return before all events were written to the log store. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * perf: replace fixed delays with polling in destwebhook tests Add MockServerPoll struct and WaitFor field to APITest for polling the mock server before running verification tests. This replaces fixed 1s delays with efficient polling at 100ms intervals. Changes: - Add waitForMockServerEvents helper for polling mock server - Replace 6x Delay:1s in destwebhook tests with WaitFor polling - Replace time.Sleep(3s) in retry test with polling for 2+ events - Reduce filter non-delivery check from 1s to 500ms delay E2E tests now run in ~10s (down from ~27s, target was 10-15s). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * perf: parallelize ListTenant tests in internal/models Enable t.Parallel() for ListTenant test suites. Tests with different deployment IDs use different RediSearch indexes, so they can safely run in parallel. Reduces internal/models test time from ~63s to ~30s. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * perf: reduce test timeouts in idempotence, mqinfra, and mqs packages - Make AWS SQS WaitTime configurable (defaults to 20s for prod, 1s for tests) - Reduce idempotence test timeouts from 3s to 1s and fix channel races - Use buffered channels in idempotence tests to prevent blocking - Replace fixed time.After waits with completion-based polling in mqinfra - Reduce visibility timeout from 5s to 2s in idempotence integration tests Test times improved: - mqs: 32s → 1.6s - mqinfra: 14.8s → ~11s - idempotence: 13.4s → 7.4s * perf: speed up Kinesis stream waiter in destawskinesis tests Configure AWS SDK StreamExistsWaiter with faster polling intervals (100ms min, 1s max) instead of using defaults (~10s). Also removes the unnecessary 2s sleep after stream creation. Test time: 13.6s → 2s * feat: add Manual field to APIDelivery response Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add parseQueryArray helper for comma-separated params Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add conditional response_data expansion for deliveries Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add sort_by and sort_order params to ListDeliveries Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add event_start and event_end params to ListDeliveries Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add sort_order param to ListEvents Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add handler tests for delivery API changes Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add e2e test for expand=response_data Add e2e test verifying response_data expansion works correctly. Remove manual field from API response (deferred to separate spec). * chore: gofmt * test: add e2e tests for sort_order and event_time_filter params - Add tests for events sort_order (asc/desc) - Add tests for deliveries sort_by=event_time - Add tests for event_start/event_end filtering - Add pagination test (10 events, limit=3, verify 4 pages) - Use parseTime helper for proper timestamp comparison - Skip delivery_time sort tests (delivery order not guaranteed) * refactor: update TypeScript types for deliveries-first model Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: replace Events view with Deliveries view Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: use SWR for tenant fetch to fix double fetch * feat: add delivery ID column to deliveries table Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: remove unused Delivery tab from sidebar header Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add icon prop to retry button for proper sizing in sidebar Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: remove count dependency from pagination UI Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: prettier portal * chore: portal ui * chore: .env.dev example * fix: guard nil pointer dereference in toAPIDelivery Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add max limit validation on pagination endpoints Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * perf: remove FINAL from ClickHouse ListEvent query Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: add deliveries API to OpenAPI schema Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: rename expand query parameter to include Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: redesign delivery details sidebar with unified view Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: make gotestsum rerun-fails configurable via env var Allow overriding the number of test retries via GOTESTSUM_RERUN_FAILS environment variable, defaulting to 2. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: update e2e tests to use include instead of expand The query parameter was renamed from expand to include in c4d7877. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add FlushWrites before deduplication test assertion ClickHouse deduplication is eventually consistent, so we need to flush writes before querying to ensure deduplication has occurred. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: improve waitForDeliveries error messages Include last count, status code, and any errors in the timeout message to help debug flaky test failures. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add 10s timeout to e2e HTTP client Prevents tests from hanging indefinitely if the server doesn't respond. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: simplify delivery details UI and remove event-centric fields - Remove Event ID column from deliveries table - Remove Event Time from delivery details - Flatten delivery details into single section (no separate Event section) - Rename data sections: 'Data', 'Metadata', 'Response' - Reorder fields: Status, Response Code, Attempt, Topic, Delivered at, IDs * feat: make TenantID optional in LogStore interface Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add admin GET /api/v1/events and /api/v1/deliveries endpoints Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: add OpenAPI schema for admin events and deliveries endpoints Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add cross-tenant query tests for logstore and admin API Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract shared logic between tenant and admin log handlers Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: use parameterized queries for cursor-based pagination Convert buildEventCursorCondition and buildCursorCondition to use parameterized queries with ? placeholders instead of string interpolation to prevent SQL injection vulnerabilities. Add parseTimestampMs helper to validate timestamp strings before use. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: remove SortBy from ListDeliveryEvent across all layers The SortBy option (event_time vs delivery_time) added complexity to cursor pagination and query logic without significant benefit. This change simplifies the API by always sorting by delivery_time, which is the natural ordering for delivery-focused queries. Changes: - Remove SortBy field from ListDeliveryEventRequest in driver.go - Simplify memlogstore, pglogstore, chlogstore to always use delivery_time sort - Remove sort_by query param validation from API handlers - Remove testCursorMismatchedSortBy and related tests from drivertest - Update log_handlers_test.go to remove sort_by validation tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: remove EventStart/EventEnd and rename DeliveryStart/DeliveryEnd to Start/End Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: remove e2e tests for removed sort_by and event_start/event_end params Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: update ClickHouse schema with time-sorted ordering and manual/attempt fields Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add Postgres migration and pglogstore support for manual/attempt fields Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add driver conformance tests for manual and attempt fields Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: optimize ClickHouse schema partitioning and index granularity - Change partition strategy from daily (toYYYYMMDD) to monthly (toYYYYMM) to reduce part count and merge pressure - Reduce bloom filter granularity from 4 to 1 for better index selectivity on low-cardinality columns like tenant_id and destination_id Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: remove unnecessary comments from logstore and API handlers Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: add deployment ID support to ClickHouse migrator Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add deployment ID support to chlogstore Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add deployment ID tests for chlogstore and migrator Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )