The slots
Astep-attempt record’s slots object holds a handle per captured value:
An attempt carries
output or error, never both, so it holds at most three slots, or two with
capture_logs off.
A handle carries no bytes. When bytes were held it carries the slot’s state, the content_id
(SHA-256 of the bytes), bytes_count, truncated, redacted, encoding, and preview, the
first 64 Unicode code points of the rendered value. redacted is true when the SDK redacted or
withheld anything inside that slot, so a reader at shape level can tell either from an absent
value. The bytes travel in value records, one per distinct content_id per trace, shipped when
the root returns.
Three handles carry less than that. A slot with state: "not_captured" carries nothing else,
which is what capture_values=False and a marked generator’s output
ship. A logs slot on an attempt that logged nothing has no content_id and no preview. So
read state first and the rest only where the state says there are bytes.
encoding is json for every value except a bare top-level str (text) and a bare
top-level bytes (base64).
How a value is rendered
The SDK dispatches on the exact type of a value,type(x), against a fixed table. It never
calls isinstance. For a type the table does not list it reads no attribute and calls neither
repr() nor str() nor iteration: the value is rendered from its type object alone. A type the
table does list is rendered the way the table says, which for an Enum member means reading
.value and for a Decimal or a UUID means calling str().
Exact means exact. A
frozenset is not a set, so the demo’s Dispatch.serviced_zones, which
returns a frozenset, records {"$type": "builtins.frozenset"}. A subclass of dict is not a
dict and records its type marker, because a subclass can override iteration.
Adapters
Without help, a dataclass, a namedtuple, or a Pydantic model would also arrive as a bare type marker. Adapters, on by default, convert those three shapes into a dict through exactly one conversion, then render the dict under every bound below with"captured_via": "adapter"
merged in, so a reader can tell that host code ran:
A conversion that raises, or returns anything whose exact type is not
dict (a dict subclass
included, by the same rule as the walk itself), increments adapter_errors, one of the counters
vexlo.diagnostics() reads, and the value
falls back to its type marker. init(adapters=False) or VEXLO_ADAPTERS=0 turns adapters off
for the process; there is no per-type setting.
This script passes one value of most rows through a step:
values.py
value field of the three value records, one per line, in the order they shipped: the root’s
input, then the returned dict (the same bytes for plan’s output and the root’s output, so one
record), then plan’s input:
plan’s input: the Order dataclass went through the adapter, its Decimal became the
string "10.83" and its Enum member became "picked_up"; the Connection instance became a
type marker; and api_key inside the credentials dict was redacted in place. In the output:
the tuple and the set became arrays, the datetime and UUID became strings, and the members
are in key order, not the order the dict was built in.
Where capture runs your code
Adapters are on by default, so the default configuration runs one method of yours per unrecognised type. That matters for a value whose attributes are lazy: a dataclass-based ORM row with an unloaded relationship emits its query inside the adapter’s field reads, in the request that was being captured. Withadapters=False the walk over your values runs no code of yours
at all, and such a value renders as a bare type marker.
Two paths run your code whatever adapters is set to, both guarded, both counted on a raise:
- An exception’s message is
str(exc), rendered by the exception’s own__str__on every failed attempt.capture_error_message=Falseis the one way to stop that call being made. - With
capture_logs=True, a captured log record is rendered withrecord.getMessage(), which formats whatever arguments were logged.
adapters=False is the setting that keeps capture out of your attributes. It is not a
setting that keeps capture out of your code entirely.
Bounds
Bounds are structural and temporal. Structural caps alone do not terminate a walk over an aliased or self-referential structure, so a node budget and a wall-clock deadline sit beside them.
Any cut sets
truncated: true on the handle and on the value record. bytes_count records
the size before the cut only when the 16 KiB slot cap made it; a cut by one of the structural
bounds cannot know the size of a walk it stopped, so bytes_count is then the emitted length.
This script hits both: render_label returns one 20,000-character string, render_manifest
returns 64 strings of 2,000 characters.
truncation.py
output handles, from the two steps’ step-attempt records:
render_label’s string was cut to 2,048 characters by the string bound, so its bytes_count
of 2,080 is the emitted length. render_manifest’s 64 strings each fit the string bound, the
rendered object came to 128,833 bytes, and the slot cap cut it to 16,384: bytes_count keeps
the pre-cut size.
The frame bound is the one a reader meets without a large value in sight. A stack deeper than
64 frames keeps its first 64 and sets truncated on a handle whose message is short and whose
payload is nowhere near 16 KiB. Deep recursion is the ordinary way to reach it. This script
raises from 100 frames of it, then reads its own segment file back:
deep.py
Redaction
Redaction matches a name, never a value. A dict member whose key matches a redaction rule is replaced in place with{"$redacted":true}. The real value is never rendered, never touched,
never type-checked. The match runs at any depth on every str dict key, and the top level of
the input slot is a dict of parameter names, so a parameter called password is redacted too.
The slot keeps its state and its other members, and its handle carries redacted: true.
The default set has 12 names, which vexlo.DEFAULT_REDACT_KEYS holds: access_token,
api_key, apikey, authorization, credit_card, passwd, password, private_key,
refresh_token, secret, ssn, and token.
init(redact_keys=[...]) replaces the set; it does not extend it. A process that passes
redact_keys=["session_cookie"] no longer redacts password. Extend it by naming the default
list: redact_keys=[*vexlo.DEFAULT_REDACT_KEYS, "email"].
Each name matches as you wrote it, and also matches its lower-, upper- and capitalized
spellings, so authorization covers Authorization and AUTHORIZATION. No other convention is
covered: api_key does not match apiKey, Api-Key, or X-Api-Key. Name the spellings your
own code uses. The match is of a whole name, never a substring, so user_password is not
password.
One rule covers all three places a record gains your data under a name: the walk’s dict keys,
the tag_keys harvest, and vexlo.tag(). A tag() pair whose key matches is dropped rather
than shipped, and counted in redacted_tags; see tags.
A name rule cannot reach inside text the host already rendered, and two places are text:
- An exception’s message.
ValueError(f"invalid token {token}")from a marked function ships that token whateverredact_keysholds, and an error attempt keeps its values at every sample rate. - A captured log message.
log.info("auth %s", token)ships as written, which is whycapture_logsdefaults to off.
Withholding a value
Redaction replaces a value whose name matches. The two settings here stop a value being captured at all, whatever it is named. Both areinit() arguments with an environment form, and both are
lowerable by the relay daemon’s policy file, so an
operator who finds credentials in a service’s exception messages turns them off without a
deploy.
One script covers both. It takes the setting to turn off as its argument, and raises from a
marked charge whose message carries the card number:
withhold.py
capture_error_message=False withholds a failed attempt’s message. str(exc) is never called,
so the text never enters the process. The class and the stack still ship, so a trace still says
which step failed and with what. The host’s own except block still sees the real message,
because this governs what is captured, not what the exception is:
capture_values=False is the limiting case: input, output, and error all ship
not_captured, no walk runs, and no value byte of a slot leaves the process. The same script,
the same failure:
redacted: true and the message reads {"$redacted":true}, as the record above shows. Shape
survives either setting: every step, attempt, status, timing, and path is still recorded.
Two things do not. capture_values=False covers no log text, so capture_logs, if it is on,
still ships a captured log message as written. And grouping two identical consecutive calls as
retries of one step works off the captured input, so with nothing captured the attempts do not
group; @vexlo.step(retries=True) still groups them. Declared tags are harvested either way, so
traces still correlate across services.
The error slot
An attempt that raises recordsinput and error, both stored whatever the trace’s
sampling rate, and never output. The error value is an object with the exception’s message
and its stack, one string per frame as file:line in function. The file is a basename, so a
record never carries the directory a process was checked out in, and the function is its
qualified name. A frame carries no source line: fetching one means reading that frame’s file off
disk, inside the request that just failed.
error_slot.py
fetch_order record and the error value behind its handle:
error_class on the record.
Value sampling
value_sample_rate, from init() or VEXLO_VALUE_SAMPLE_RATE, is a number from 0.0 to
1.0, default 1.0. It decides whether the values of a trace that closes ok are kept. It
never touches shape: every step, attempt, status, timing, and handle is recorded at any rate.
It never touches an error: an attempt with status: "error" keeps its values at any depth and
any rate. The decision is made once per trace, deterministically from the trace_id, at the
moment the root returns, and a slot’s state reports the outcome:
sampled_out appears only on the root, the one attempt whose completion coincides with knowing
the trace’s fate. An excluded pending slot is never revised: it keeps that state and never
gets a value record.
Two further states reach the wire. not_captured is what
withholding ships, and what a marked generator’s output ships.
too_large appears on a logs slot whose first entry alone did not fit the 16 KiB cap. Those
five are what this version of the SDK writes; the wire format defines two more, redacted and
dropped, which it never writes, and the server adds expired and missing at read time.
The rate is a volume control, not a privacy control. It changes only whether captured values are
kept, never whether capture runs, so 0.0 costs the same per call as 1.0, a trace that ends
in an error keeps its values regardless, and an excluded slot still carries its content_id and
its 64-character preview. Withholding is the control for keeping
values inside the process.
The same script under value_sample_rate=0.0, once with an order that succeeds and once with
one that makes the inner step raise:
sampling.py
kind, name, status,
and slot state lines.
- o-5001: both attempts ok
- o-9999: the inner step raises
value record. find_order’s slots are pending; the root’s are
sampled_out: