Skip to content

fix(e2e): log-generator always-on, 5× more volume for stable pattern clustering #360

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
szibis merged 3 commits into from
May 14, 2026
+16 −2
Merged

fix(e2e): log-generator always-on, 5× more volume for stable pattern clustering #360

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b8e13b6
fix(e2e): log-generator always-on, 5× more volume for stable pattern …
szibis May 14, 2026
e04a883
Merge branch 'main' into fix/e2e-patterns-volume
szibis May 14, 2026
a13d5b7
fix(e2e): restore profiles:[ui] gate on log-generator, add changelog
szibis May 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Fixed

- fix(e2e): log-generator now starts by default (`profiles: ["ui"]` removed) — cold stacks no longer have zero ingestion, enabling pattern clustering in Drilldown. `LOG_INTERVAL` 10→2, `LOG_BATCH` 8→15 raises throughput from ~8.6 to ~75 lines/s so the pattern miner sees enough volume per time-step bucket for stable clustering and a continuous Drilldown timeline. Added `GOMEMLIMIT=2GiB` to `loki-vl-proxy-patterns-autodetect` consistent with other proxy variants.

## [1.32.0] - 2026-05-13

### Added
Expand Down
14 changes: 12 additions & 2 deletions test/e2e-compat/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,8 @@ services:
- "13110:3100"
volumes:
- patterns-cache:/cache
environment:
- GOMEMLIMIT=2GiB
command:
- "-listen=:3100"
- "-backend=http://victorialogs:9428"
Expand Down Expand Up @@ -492,6 +494,14 @@ services:

# Continuous log generator — dual-writes realistic multi-service logs to Loki + VictoriaLogs.
# Provides live data for Grafana Explore UI tests and Logs Drilldown pattern detection.
#
# profile: ui — intentionally kept so this does NOT start during automated e2e-compat test
# runs (which use plain "docker compose up -d"). High-volume non-OTel data from the generator
# dilutes detected_fields results and breaks OTel tests. UI/Drilldown manual testing uses:
# docker compose --profile ui up -d
#
# LOG_INTERVAL=2 / LOG_BATCH=15: ~75 lines/s total — enough for stable pattern clustering
# across time-step buckets so Drilldown shows a continuous timeline rather than sparse points.
log-generator:
image: python:3.12-alpine
container_name: e2e-log-generator
Expand All @@ -502,8 +512,8 @@ services:
environment:
- LOKI_URL=http://loki:3100
- VL_URL=http://victorialogs:9428
- LOG_INTERVAL=10
- LOG_BATCH=8
- LOG_INTERVAL=2
- LOG_BATCH=15
depends_on:
loki:
condition: service_started
Expand Down