Skip to content

perf: update nethvoice-reports with CDR query optimizations#710

Merged
edospadoni merged 6 commits intomainfrom
perf-cdr-query-optimization
Mar 17, 2026
Merged

perf: update nethvoice-reports with CDR query optimizations#710
edospadoni merged 6 commits intomainfrom
perf-cdr-query-optimization

Conversation

@edospadoni
Copy link
Copy Markdown
Member

@edospadoni edospadoni commented Feb 26, 2026

Summary

  • Update nethvoice-report commit reference to include CDR query performance optimizations

See nethesis/nethvoice-report#207 for details.

Reference:

@edospadoni edospadoni self-assigned this Feb 26, 2026
@edospadoni edospadoni force-pushed the perf-cdr-query-optimization branch 2 times, most recently from fc5495d to 60a7b3b Compare March 4, 2026 10:12
edospadoni and others added 5 commits March 5, 2026 11:43
Points to nethesis/nethvoice-report#207 which includes:
- Indexes on source cdr table and generated cdr_YYYY/cdr_YYYY-MM tables
- Range scans replacing date_format() anti-patterns
- Pre-computed geographic columns eliminating correlated subqueries
- LEFT JOIN for trunk type detection instead of per-row subqueries
- REGEXP replaced with LIKE for disposition filters
- Parallel view execution with goroutines
Add idempotent index creation for queue_log, queue_log_history,
report_queue, and cdr.

These tables are filtered and joined repeatedly by time range, event,
queue, callid, agent, and linkedid during each consolidation run. Without
proper indexes, MySQL falls back to wide scans, making the job slow and
increasing lock time on busy systems.

Creating targeted composite indexes improves selectivity for the hottest
predicates, reduces I/O and temporary work, and keeps report generation
stable as data grows.
This reverts commit 4cab95e04546406e33a3f68d8d6fe6cffb03dce9.
@edospadoni edospadoni force-pushed the perf-cdr-query-optimization branch from 2043425 to 459805f Compare March 5, 2026 10:43
@viktec
Copy link
Copy Markdown
Contributor

viktec commented Mar 10, 2026

Tested on this machine: https://helpdesk.nethesis.it/a/tickets/199090 — test OK.

[nethvoice1@nethserver state]$ systemctl --user status reports-scheduler
○ reports-scheduler.service - Generate reports
     Loaded: loaded (/home/nethvoice1/.config/systemd/user/reports-scheduler.service; static)
     Active: inactive (dead) since Tue 2026-03-10 00:52:43 CET; 8h ago
TriggeredBy: ● reports-scheduler.timer
    Process: 1152777 ExecStart=/usr/bin/podman exec freepbx /opt/nethvoice-report/scripts/queue-miner.php (code=exited, status=0/SUCCESS)
    Process: 1159079 ExecStart=/usr/bin/podman exec reports-api tasks cdr (code=exited, status=0/SUCCESS)
    Process: 1159380 ExecStart=/usr/bin/podman exec reports-api tasks cost (code=exited, status=0/SUCCESS)
    Process: 1159426 ExecStart=/usr/bin/podman exec reports-api tasks views (code=exited, status=0/SUCCESS)
    Process: 1161478 ExecStart=/usr/bin/podman exec reports-api tasks queries (code=exited, status=0/SUCCESS)
    Process: 1161536 ExecStart=/usr/bin/podman exec reports-api tasks phonebook (code=exited, status=0/SUCCESS)
    Process: 1161580 ExecStart=/usr/bin/podman exec reports-api tasks values (code=exited, status=0/SUCCESS)
   Main PID: 1161580 (code=exited, status=0/SUCCESS)
        CPU: 2.839s

@edospadoni edospadoni merged commit df76823 into main Mar 17, 2026
9 of 10 checks passed
@edospadoni edospadoni deleted the perf-cdr-query-optimization branch March 17, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants