Monitor Signals with Prometheus and Grafana

A how-to for watching the Signals collector's own operational health — whether collection cycles are running, how long they take, and whether exports succeed — with Prometheus and Grafana.

This describes Signals' signals_* metrics. It does not expose your PostgreSQL data or credentials; the collected diagnostics stay in Signals' local store and the snapshot ZIP. The unified Elevarq deployment's elevarq_* metrics are a separate surface — see Observability.

1. Enable the metrics endpoint

The /metrics endpoint is opt-in and off by default. Turn it on with an environment variable:

SIGNALS_METRICS_ENABLED=true

or in signals.yaml:

signals:
  metrics_enabled: true
  metrics_path: /metrics

The endpoint is protected by the same bearer token as the control API and binds to loopback by default, so a scraper must send Authorization: Bearer <token> and reach it over loopback or a private network. See Signals configuration for the API token settings.

2. Scrape it with Prometheus

Point Prometheus at the endpoint, supplying the bearer token from a file. Put the token in a 0600 file next to the config:

printf '%s' "<your-signals-api-token>" > signals-token
chmod 600 signals-token

prometheus.yml:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: signals
    metrics_path: /metrics
    authorization:
      type: Bearer
      credentials_file: /etc/prometheus/secrets/signals-token
    static_configs:
      - targets: ["signals:8081"]   # host:port of the Signals API

3. View it in Grafana

Add Prometheus as a data source, then build panels on the metrics below. Every collection metric carries a target label, so with several monitored databases each one shows up separately — for example, collection rate per database:

sum by (target) (rate(signals_collection_cycles_total[5m]))

and collection-freshness (seconds since the last successful collection):

time() - signals_last_successful_collection_timestamp

4. Try it locally (complete, copy-paste)

A self-contained stack — Signals, a PostgreSQL target, Prometheus, and a pre-provisioned Grafana dashboard — to see the metrics end to end. Save as docker-compose.yml alongside the prometheus.yml and signals-token above (use the dev token dev-local-only-replace-in-prod-32chars in the token file for this trial):

services:
  signals:
    image: ghcr.io/elevarq/signals:latest
    environment:
      SIGNALS_ENV: dev
      SIGNALS_API_TOKEN: dev-local-only-replace-in-prod-32chars
      SIGNALS_LISTEN_ADDR: "0.0.0.0:8081"
      SIGNALS_METRICS_ENABLED: "true"
    volumes:
      - ./signals.yaml:/etc/signals/signals.yaml:ro
    ports: ["8081:8081"]

  prometheus:
    image: prom/prometheus:v3.1.0
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - ./signals-token:/etc/prometheus/secrets/signals-token:ro
    ports: ["9090:9090"]

  grafana:
    image: grafana/grafana:11.4.0
    environment:
      GF_SECURITY_ADMIN_USER: admin
      GF_SECURITY_ADMIN_PASSWORD: admin
    ports: ["3000:3000"]

docker compose up -d

Then open Prometheus at http://localhost:9090/targets (the signals target should be UP) and Grafana at http://localhost:3000 (admin/admin). Because /metrics requires the token, a browser hit to http://localhost:8081/metrics returns 401 — view the data through Grafana, or curl it with the Authorization: Bearer header.

Reference — what you can monitor

All metrics are prefixed signals_.

Security

• Keep the token file at mode 0600.
• Bind /metrics to loopback or a private interface and control access at the network layer; never expose it publicly.
• The metrics carry only bounded, low-cardinality labels — no query text, no table data, no credentials.

Next: Operate a running Signals.