> ## Documentation Index > Fetch the complete documentation index at: https://docs.vexlo.ai/llms.txt > Use this file to discover all available pages before exploring further. # The relay daemon > Run vexlo relay run on each host: what it reads from the spool directory, what it sends to the server, and how to watch and reload it. The SDK never opens a network connection. It appends records to a spool directory on its own host, and the relay daemon, `vexlo relay run`, is the one process that talks to the Vexlo server: it tails the spool directory, batches and gzips what it finds, posts it to `POST /v1/records`, sends a heartbeat per live writer (one that has written in the current or previous UTC hour), and writes the capture policy the server answers with back into the spool directory for the SDK to read. An instrumented process therefore never blocks on network I/O to get a record captured, and an unreachable server becomes a backlog on disk, up to the spool directory's byte budget (`-spool-budget-bytes`, default 2 GiB). One daemon serves one spool directory, one destination, and one token, so a host runs one daemon unless it splits services or environments across spool directories. The ingest route's own limits are on [Ingest](https://docs.vexlo.ai/api/ingest.md). Flags are spelled with a single dash, as `-h` prints them; the binary accepts both spellings. ## Start it Two flags are required: `-destination` (or `VEXLO_ENDPOINT`), the server's base URL, and `-token` (or `VEXLO_INGEST_TOKEN`), an ingest token. Use the environment variable for the token outside a one-off manual run; a flag value is visible in the process list. ```text theme={null} $ vexlo relay run -spool-dir host-spool -destination http://127.0.0.1:18100 \ -token "$INGEST_TOKEN" -environment dev -health-addr 127.0.0.1:19100 \ -destination-file destination.txt -token-file token.txt time=2026-09-14T01:51:10.707-07:00 level=INFO msg="relay: corrected directory permissions" dir=host-spool old_mode=-rwxr-xr-x new_mode=448 time=2026-09-14T01:51:10.708-07:00 level=INFO msg="relay run: starting" spool_dir=host-spool destination=http://127.0.0.1:18100 environment=dev poll_interval=250ms heartbeat_interval=30s retention_interval=1m0s health_addr=127.0.0.1:19100 ... 3 lines omitted ``` The first line appears when the daemon finds the spool root at a mode other than `0700` and corrects it (the directory here was created at `0755` first). The second names seven settings: the spool directory, the destination, the environment, the three intervals, and the health address. The other eight flags, `-token` and `-spool-budget-bytes` among them, are not logged. `-destination-file` and `-token-file` change nothing at startup; they name the two files a SIGHUP reload re-reads, covered in [Reload the destination or token](#reload-the-destination-or-token). Ctrl-C (SIGINT) or SIGTERM stops the daemon; the last line is `msg="relay run: shutting down"`. Without a token the daemon does not start: ```text theme={null} $ vexlo relay run -spool-dir host-spool -destination http://127.0.0.1:18100 vexlo: -token (or VEXLO_INGEST_TOKEN) is required ``` ### The destination must be https or loopback A plaintext `http://` destination is refused unless its host is `127.0.0.1`, `::1`, or `localhost`, matched as a string once it is lowercased and a trailing dot is dropped, never resolved through DNS. The refusal names the ways forward: ```text theme={null} $ vexlo relay run -spool-dir host-spool -destination http://vexlo.internal:8080 -token "$INGEST_TOKEN" ... 1 line omitted vexlo: relay run: new forwarder: destination "http://vexlo.internal:8080" is plaintext http:// and not loopback: use https://, point at loopback (127.0.0.1, ::1, localhost) for local dev, or set -allow-insecure-http/VEXLO_ALLOW_INSECURE_HTTP ``` `-allow-insecure-http` (or `VEXLO_ALLOW_INSECURE_HTTP=1`) accepts any `http://` destination and is meant for a same-network development server. The same rule governs `vexlo policy set` and `vexlo policy show`. ### `-environment` `-environment ` (or `VEXLO_ENVIRONMENT`) is sent as `X-Vexlo-Environment` on every request the daemon makes. Environment is declared here, once per daemon, never by the SDK: a host that serves two environments runs two daemons against two spool directories. The value is 1 to 64 printable ASCII characters with no comma, checked at startup: ```text theme={null} $ vexlo relay run -spool-dir host-spool -destination http://127.0.0.1:18100 -token "$INGEST_TOKEN" \ -environment dev,staging vexlo: -environment (or VEXLO_ENVIRONMENT): "dev,staging" contains a comma, reserved as the multi-value separator ``` Unset, the daemon sends no header and the server records the rows with no environment. An ingest token pinned with `-environment` rejects a batch that carries no header, so a daemon forwarding under a pinned token runs with `-environment` set to the pinned value; see [Authentication and tokens](https://docs.vexlo.ai/server/authentication.md). ## The spool directory The daemon and the SDK share one directory, `-spool-dir` (or `VEXLO_SPOOL_DIR`), default `~/.vexlo`. After one demo run and one daemon pass it holds: ```text theme={null} $ find host-spool | sort host-spool host-spool/cursor host-spool/cursor/2026-09-14T08 host-spool/cursor/2026-09-14T08/1789375859-57501-446ef338.cursor host-spool/policy host-spool/policy/delivery-service.json host-spool/spool host-spool/spool/2026-09-14T08 host-spool/spool/2026-09-14T08/1789375859-57501-446ef338.jsonl host-spool/writers host-spool/writers/1789375859-57501-446ef338.counters.json host-spool/writers/1789375859-57501-446ef338.identity.json ``` | Path | Written by | Holds | | - | - | - | | `spool//.jsonl` | the SDK | records, append-only, one file per writer per hour | | `cursor//.cursor` | the daemon | the spool cursor: the offset up to which that file has been accepted by the server | | `writers/.identity.json` | the SDK | `service` and `instance_id`, written once | | `writers/.counters.json` | the daemon | `records_sent` and `dropped`, monotonic for the life of the writer id | | `policy/.json` | the daemon | the resolved capture policy from the last heartbeat | Every directory is created at `0700` and every file at `0600`, whichever side creates it first. At startup, and again on every retention sweep (`-retention-interval`, default 1 minute), the daemon re-asserts `0700` on the spool root and on `spool/`, `cursor/`, `writers/`, and `policy/`, and logs `relay: corrected directory permissions` for each one it changed. A `chmod` that moves one of those five off `0700` is reverted within one sweep. The modes after this run: ```text theme={null} $ stat -f '%Sp %N' host-spool host-spool/spool host-spool/cursor host-spool/writers host-spool/policy \ host-spool/spool/*/* host-spool/writers/* host-spool/policy/* host-spool/cursor/*/* drwx------ host-spool drwx------ host-spool/spool drwx------ host-spool/cursor drwx------ host-spool/writers drwx------ host-spool/policy -rw------- host-spool/spool/2026-09-14T08/1789375859-57501-446ef338.jsonl -rw------- host-spool/writers/1789375859-57501-446ef338.counters.json -rw------- host-spool/writers/1789375859-57501-446ef338.identity.json -rw------- host-spool/policy/delivery-service.json -rw------- host-spool/cursor/2026-09-14T08/1789375859-57501-446ef338.cursor ``` ## What it sends The daemon polls the spool directory every 250 ms (`-poll-interval`) and sends what is ready as one batch of at most 512 records or 1 MiB, whichever binds first. A batch is a gzip-compressed JSON array of records with `Authorization: Bearer ` and, when `-environment` is set, `X-Vexlo-Environment`. The server answers `204` when every record is accepted, or `200` with a `rejected` list naming the records it refused. A record in that list is terminal: it is counted as dropped and never resent. A connection failure, a `429`, and `500`, `502`, `503`, or `504` are retried, 3 attempts in all, with a full-jitter backoff starting from a 200 ms ceiling and doubling per attempt; a `429`'s `Retry-After` is added to the delay and capped at 60 seconds. Every other status is terminal on the first response: the whole batch is counted `daemon_terminal_reject`, never resent, and the daemon logs `send batch: terminal rejection` with the status. `400`, `413`, and `422` fall here, and so do `401` and `403`, so a revoked or mistyped token drops batches rather than building a backlog. Rotate a token in that order: write the new one into `-token-file`, SIGHUP the daemon, then revoke the old one. Every batch the daemon sends while it holds a revoked token is dropped. Every 30 seconds (`-heartbeat-interval`) the daemon sends one `POST /v1/heartbeat` per live writer, carrying that writer's `records_sent` and `dropped` counters. A writer is live when it has a spool file in the current or the previous UTC hour and an identity file on disk. A writer that stops writing stops heartbeating, and once every writer of a service has stopped, nothing refreshes that service's capture policy file until one of its processes writes again. The response is the capture policy for that writer's service, which the daemon writes to `policy/.json`; [Capture policy](https://docs.vexlo.ai/server/capture-policy.md) covers what the SDK does with it. ## Reload the destination or token SIGHUP reloads the destination and the token, and nothing else. The daemon cannot re-read its own flags or environment (a running process's copy of both is fixed at start), so a reload reads files: `-destination-file` (or `VEXLO_ENDPOINT_FILE`) and `-token-file` (or `VEXLO_INGEST_TOKEN_FILE`) each name a file holding one bare string, trimmed of surrounding whitespace. Neither affects startup; `-destination` and `-token` still start the daemon. Write the current destination and token into those files before the first signal: a reload that cannot read a configured file is refused at `ERROR` and the live destination and token stay in place. On SIGHUP the daemon re-reads whichever files are configured, validates the result (the https-or-loopback rule included), and swaps the new values in. A batch already in flight finishes against the old values. The log names which value changed, never the value. Two signals, the second after `token.txt` was rewritten: ```text theme={null} $ kill -HUP $RELAY_PID ... 2 lines omitted time=2026-09-14T01:52:25.967-07:00 level=INFO msg="relay: sighup reload applied" destination_changed=false token_changed=false time=2026-09-14T01:52:26.983-07:00 level=INFO msg="relay: sighup reload applied" destination_changed=false token_changed=true ... 1 line omitted ``` A daemon started with neither file refuses the signal and keeps running: ```text theme={null} ... 1 line omitted time=2026-09-14T01:52:44.006-07:00 level=ERROR msg="relay: sighup reload refused: no -destination-file or -token-file configured; SIGHUP has no reloadable source (env and the exact flags this process started with can never change for an already-running process)" ... 1 line omitted ``` A reload whose result fails validation is refused the same way, and the old destination and token stay live. ## Health and metrics The daemon serves `GET /healthz` and `GET /metrics` on `-health-addr` (or `VEXLO_HEALTH_ADDR`), default `127.0.0.1:9101`. Neither route is authenticated, and neither carries a service name, an instance id, a path, or a value: every field is an aggregate count. The daemon below bound `127.0.0.1:19100`. `/healthz` is `200` with `{"status":"healthy"}`, or `503` with a `reason`, when one of three conditions holds: unsent records exist and the server has accepted nothing in the last 5 minutes (a daemon that has never had a send accepted counts as stale from its first backlog), the spool directory holds more than 80% of its byte budget, or reading the spool directory's own state failed or ran past 5 seconds. An idle daemon with no backlog is healthy. Match an alert on the status, not on the `reason` text: the first two reasons carry the measured numbers, the third is the fixed string `spool status unavailable`. ```text theme={null} $ curl -s -i http://127.0.0.1:19100/healthz HTTP/1.1 200 OK Content-Type: application/json Date: Mon, 14 Sep 2026 08:51:28 GMT Content-Length: 21 {"status":"healthy"} ``` The degraded body, from a daemon pointed at a port with nothing listening: ```text theme={null} HTTP/1.1 503 Service Unavailable Content-Type: application/json Date: Mon, 14 Sep 2026 08:53:07 GMT Content-Length: 101 {"status":"degraded","reason":"backlog present (65 records) with no accepted send in the last 5m0s"} ``` The third body, from a daemon whose spool directory had been made unreadable. The daemon logs `relay: healthz spool status failed` with the underlying error at the same moment: ```text theme={null} HTTP/1.1 503 Service Unavailable Content-Type: application/json Date: Mon, 19 Oct 2026 08:20:30 GMT Content-Length: 58 {"status":"degraded","reason":"spool status unavailable"} ``` `/metrics` is Prometheus text exposition with eight metrics. The three counters (`records_sent_total`, `records_dropped_total`, `panics_recovered_total`) and `last_send_success_timestamp_seconds` are process-lifetime state that a restart resets to zero, so a zero there means "nothing since this process started", never "nothing ever". The backlog, backoff, and spool-byte gauges are read from the spool directory on each scrape, and `spool_budget_bytes` reports the configured `-spool-budget-bytes`. Every metric is present from the first scrape, at zero if nothing has happened yet. ```text theme={null} $ curl -s -i http://127.0.0.1:19100/metrics HTTP/1.1 200 OK Content-Type: text/plain; version=0.0.4 Date: Mon, 14 Sep 2026 08:51:28 GMT Content-Length: 1734 # HELP vexlo_relay_records_sent_total Records the destination has accepted. # TYPE vexlo_relay_records_sent_total counter vexlo_relay_records_sent_total 77 # HELP vexlo_relay_records_dropped_total Records permanently dropped, by cause. # TYPE vexlo_relay_records_dropped_total counter vexlo_relay_records_dropped_total{cause="daemon_torn_record"} 0 vexlo_relay_records_dropped_total{cause="daemon_spool_expired"} 0 vexlo_relay_records_dropped_total{cause="daemon_terminal_reject"} 0 # HELP vexlo_relay_backlog_records Unsent records across every tracked writer. # TYPE vexlo_relay_backlog_records gauge vexlo_relay_backlog_records 0 # HELP vexlo_relay_writers_backoff Writers currently excluded from Load by an active advance-failure backoff. # TYPE vexlo_relay_writers_backoff gauge vexlo_relay_writers_backoff 0 # HELP vexlo_relay_spool_bytes Bytes currently on disk under the spool root. # TYPE vexlo_relay_spool_bytes gauge vexlo_relay_spool_bytes 41718 # HELP vexlo_relay_spool_budget_bytes The configured spool budget. # TYPE vexlo_relay_spool_budget_bytes gauge vexlo_relay_spool_budget_bytes 2147483648 # HELP vexlo_relay_last_send_success_timestamp_seconds Unix time of the last send the destination accepted at least one record from. # TYPE vexlo_relay_last_send_success_timestamp_seconds gauge vexlo_relay_last_send_success_timestamp_seconds 1789375870 # HELP vexlo_relay_panics_recovered_total Panics recovered at a goroutine boundary, by loop. # TYPE vexlo_relay_panics_recovered_total counter vexlo_relay_panics_recovered_total{loop="poll"} 0 vexlo_relay_panics_recovered_total{loop="retention"} 0 vexlo_relay_panics_recovered_total{loop="heartbeat"} 0 vexlo_relay_panics_recovered_total{loop="health_server"} 0 ``` ## `vexlo relay status` `vexlo relay status` reads the same files a running daemon would and prints one summary per writer. No daemon has to be running. Before the daemon's first pass over this spool directory: ```text theme={null} $ vexlo relay status -spool-dir host-spool spool: 40.6KiB / 2.0GiB (0.0%) writer 1789375859-57501-446ef338 service: delivery-service instance: 0c0809d5012a92b247a6b4fb872aada0 state: active backlog: 77 records, 40.5KiB oldest unsent age: 51m11s records sent: 0 dropped: none ``` After the daemon forwarded the backlog: ```text theme={null} $ vexlo relay status -spool-dir host-spool spool: 40.7KiB / 2.0GiB (0.0%) writer 1789375859-57501-446ef338 service: delivery-service instance: 0c0809d5012a92b247a6b4fb872aada0 state: active backlog: 0 records, 0B records sent: 77 dropped: none ``` `oldest unsent age` is printed only while a backlog exists, and is measured from the start of the oldest hour that still holds unsent records (records carry no parsed timestamp), so it overstates the oldest record's age by up to an hour. A writer with no spool file in the current or the previous UTC hour shows `state: paused`, the condition that also stops its heartbeat; a writer with no identity file shows `identity: missing (no heartbeat can be attributed)`. The command repeats the daemon's own startup warning when the configured spool budget exceeds the free space on the spool filesystem. ## Flag reference ```text theme={null} $ vexlo relay run -h Usage of relay run: -allow-insecure-http accept a plaintext http:// destination even when it is not loopback (env VEXLO_ALLOW_INSECURE_HTTP) -destination string destination server base URL (env VEXLO_ENDPOINT), required -destination-file string path to a file whose contents a SIGHUP reload re-reads as the destination (env VEXLO_ENDPOINT_FILE); does not affect startup, only reload — see -token-file -environment string deployment environment name, sent as X-Vexlo-Environment on every outbound request (env VEXLO_ENVIRONMENT); optional, 1-64 printable-ASCII characters excluding comma -finality-grace duration how long past an hour's end the daemon waits before treating it as final (default 5m0s) -health-addr string bind address for /healthz and /metrics (env VEXLO_HEALTH_ADDR), default 127.0.0.1:9101 -heartbeat-interval duration how often each live writer instance's heartbeat is sent (default 30s) -max-writers int cap on concurrently-tracked writer ids, past which the oldest safely-retireable ones are evicted (default 10000) -poll-interval duration how often the spool is polled for a batch ready to send (default 250ms) -retention-interval duration how often the spool's retention sweeps and writer retirement run (default 1m0s) -spool-budget-bytes int total on-disk spool size budget, in bytes, before pressure retention starts evicting (default 2147483648) -spool-dir string spool root directory (env VEXLO_SPOOL_DIR, default ~/.vexlo) -token string bearer token for the destination (env VEXLO_INGEST_TOKEN), required -token-file string path to a file whose contents a SIGHUP reload re-reads as the bearer token (env VEXLO_INGEST_TOKEN_FILE); does not affect startup, only reload — without at least one of -destination-file/-token-file set, a SIGHUP has nothing it can safely re-read and is refused -writer-ttl duration how long an idle writer's identity and counters survive with nothing left to attribute (default 24h0m0s) vexlo: flag: help requested ``` Eight of the fifteen flags have an environment-variable form. The five timing knobs, `-spool-budget-bytes`, and `-max-writers` do not. | Flag | Environment variable | Default | | - | - | - | | `-destination` | `VEXLO_ENDPOINT` | none; required | | `-token` | `VEXLO_INGEST_TOKEN` | none; required | | `-spool-dir` | `VEXLO_SPOOL_DIR` | `~/.vexlo` | | `-environment` | `VEXLO_ENVIRONMENT` | unset (no header sent) | | `-health-addr` | `VEXLO_HEALTH_ADDR` | `127.0.0.1:9101` | | `-allow-insecure-http` | `VEXLO_ALLOW_INSECURE_HTTP` | `false` | | `-destination-file` | `VEXLO_ENDPOINT_FILE` | none | | `-token-file` | `VEXLO_INGEST_TOKEN_FILE` | none | | `-poll-interval` | none | `250ms` | | `-heartbeat-interval` | none | `30s` | | `-retention-interval` | none | `1m0s` | | `-finality-grace` | none | `5m0s` | | `-writer-ttl` | none | `24h0m0s` | | `-max-writers` | none | `10000` | | `-spool-budget-bytes` | none | `2147483648` (2 GiB) | `vexlo relay status` takes `-spool-dir` and `-spool-budget-bytes` only. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.