pgcollector-ingest

Ingest pganalyze PostgreSQL snapshots and expose as Prometheus metrics

Overview

pgcollector-ingest is a Go-based service that bridges the pganalyze collector with Prometheus/Grafana monitoring stack. It receives PostgreSQL performance data via WebSocket, decodes protobuf snapshots, and exposes them as Prometheus metrics for visualization and alerting.

Features

• WebSocket ingestion - Receives compressed protobuf snapshots from pganalyze collector
• Prometheus metrics - Exposes 50+ metric families covering system, database, table, index, query, and activity statistics
• Delta tracking - Maintains state to compute deltas for cumulative counters
• Pre-built dashboards - 6 Grafana dashboards included for immediate visibility
• Lightweight - Minimal dependencies, containerized deployment

Architecture

pganalyze-collector → WebSocket → pgcollector-ingest → Prometheus → Grafana
                     (protobuf)                        (/metrics)

Components

• Server (internal/server/) - HTTP/WebSocket server with authentication
• Snapshot Decoder (internal/snapshot/) - Protobuf decompression and decoding
• Metrics Processor (internal/metrics/) - Snapshot-to-Prometheus transformation

Quick Start

Prerequisites

• Go 1.25+
• PostgreSQL with pg_stat_statements extension enabled
• pganalyze collector configured
• Prometheus/VictoriaMetrics
• Grafana (optional, for dashboards)

Installation

# Clone repository
git clone https://github.com/crisog/pgcollector-ingest.git
cd pgcollector-ingest

# Build
go build -o pgcollector-ingest

# Run
export LISTEN_ADDR=:8080
./pgcollector-ingest

Docker

docker build -t pgcollector-ingest .
docker run -p 8080:8080 -e LISTEN_ADDR=:8080 pgcollector-ingest

Configuration

Environment Variables

Variable	Default	Description
LISTEN_ADDR	:8080	HTTP server listen address
PGANALYZE_API_KEY	-	Expected API key from collectors (header validation)

pganalyze Collector Setup

Configure your pganalyze collector to send snapshots to this service:

# collector.conf
api_base_url: http://localhost:8080
api_key: your-api-key-here

Prometheus Configuration

Add scrape config to prometheus.yml:

scrape_configs:
  - job_name: "pgcollector-ingest"
    static_configs:
      - targets: ["localhost:8080"]
    scrape_interval: 15s

Or use the included vmagent configuration in vmagent/.

API Endpoints

Endpoint	Method	Purpose
/v2/snapshots/grant	GET	Authenticates collector and returns feature flags
/v2/snapshots/websocket	GET	WebSocket endpoint for receiving snapshot streams
/metrics	GET	Prometheus metrics exposition format

Metrics

Exposed metrics include:

System Metrics (pga_system_*)

• CPU usage, memory, load average
• WAL usage and checkpoints
• Replication lag and status

Database Metrics (pga_db_*)

• Transaction counts (commits, rollbacks)
• Cache hit rates
• Freeze age and autovacuum metrics

Table Metrics (pga_table_*)

• Table sizes and tuple counts
• Sequential vs index scans
• Vacuum and analyze timestamps

Index Metrics (pga_index_*)

• Index sizes
• Index scan counts
• Block reads

Query Metrics (pga_query_*)

• Call counts and execution time
• Rows returned/affected
• I/O statistics
• Latency histograms

Activity Metrics (pga_activity_*)

• Active sessions by state
• Wait event tracking

Grafana Dashboards

Six pre-built dashboards are included in grafana/:

1. System Overview - CPU, memory, WAL, replication
2. Database Performance - Transactions, cache hits, connections
3. Replication - Lag monitoring and replica status
4. Tables and Indexes - Size, scans, maintenance
5. Activity - Session states and wait events
6. Query Performance - Top queries by time, calls, I/O

Import these JSON files into Grafana to get started immediately.

Related Projects

• pganalyze/collector - PostgreSQL statistics collector
• VictoriaMetrics - Monitoring solution and time series database