Skip to content

Service Metrics

The service exposes Prometheus metrics at /metrics for monitoring request traffic, latency, and Go runtime health.

HTTP request metrics

Metric Type Labels Description
http_requests_total Counter method, path, status Total number of HTTP requests
http_request_duration_seconds Histogram method, path Request duration in seconds

Duration buckets are scaled logarithmically at 5× intervals: 100µs, 500µs, 2.5ms, 12.5ms, 62.5ms, 312ms, 1.56s, 7.8s, 39s.

These let you track request rates, error ratios (via the status label), and build latency SLOs from histogram percentiles.

Unmatched routes (404s, OPTIONS preflight) are not recorded — the middleware returns early when no route matches, preventing label explosion from random scanner traffic.

Go Runtime Metrics

The Prometheus Go client library registers default collectors automatically. Key interesting metrics exposed by the service:

Metric What it measures Performance signal
go_goroutines Number of goroutines currently running Spikes may indicate connection leaks or stuck handlers
go_memstats_alloc_bytes Total allocated heap objects (cumulative) Monotonic increase — watch the rate for allocation bursts
go_memstats_heap_alloc_bytes Currently live heap memory Primary memory pressure indicator
go_memstats_heap_inuse_bytes Heap memory in use by allocated spans Higher than heap_alloc if the GC hasn't released spans
go_memstats_stack_inuse_bytes Stack memory in use across all goroutines Grows with goroutine count — correlated with go_goroutines
go_memstats_gc_cpu_fraction Fraction of CPU time spent in garbage collection (since start) Values above 0.05 (5%) indicate the GC is struggling to keep up
go_gc_duration_seconds Duration of individual GC pauses (summary) Tail latency can cause request timeouts
process_resident_memory_bytes Resident set size of the process Whole-process memory pressure, not just Go heap
process_cpu_seconds_total Cumulative CPU time consumed (user + system) Rate over time gives CPU utilisation
process_open_fds Number of open file descriptors Leaks from unclosed connections or files

The full list of default Go collectors is documented in the prometheus/client_golang package. The go_gc_duration_seconds summary follows the Go runtime's GC metrics convention.

Performance Checks

The following PromQL queries can be used to assess service health from these metrics:

Memory pressure — heap growing without GC releasing it:

go_memstats_heap_alloc_bytes

Goroutine leak — sustained upward trend over minutes:

rate(go_goroutines[1m])

GC overhead — values above 5% warrant investigation:

go_memstats_gc_cpu_fraction

Request error ratio — percentage of non-2xx responses:

sum(rate(http_requests_total{status!~"2.."}[5m])) / sum(rate(http_requests_total[5m])) * 100

p99 latency — the histogram bucket at which 99% of requests fall below:

histogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket[5m])) by (le, method, path))

CPU usage — CPU cores consumed on average:

rate(process_cpu_seconds_total[1m])