What You Can Ask

Every signal is one question about one event. The shape of the input decides the shape of the answer, and the shape of the answer decides what you can do with the column.

Flag: a yes/no with a probability

The shortest form. The model returns P(yes), and the column reads value: true, confidence: 0.9. criteria is optional and tells the model what each answer looks like, which raises confidence on borderline events:

import { defineSignal } from '@evlog/signals'

export const retryable = defineSignal({
  name: 'retryable',
  when: e => (e.status ?? 0) >= 500,
  ask: 'Would the same request most likely succeed if retried in a few seconds?',
  criteria: {
    true: 'Timeout, connection reset, rate limit, transient upstream error',
    false: 'Bug, bad data, missing resource',
  },
})

"retryable": { "value": true, "confidence": 0.9 }

This is the shape for keep: a retry policy, a "did the customer get what they came for", a "does this deserve a reviewer". The verdict is a boolean, so keep: v => v.value && v.confidence > 0.8 reads as written.

Categorise: one option out of a list

Name the options, each with the description the model matches against. The column's value is one of the keys, which makes it the shape to reach for when the question is "which kind":

export const fault = defineSignal({
  name: 'fault',
  when: e => (e.status ?? 0) >= 400,
  ask: 'Who is responsible for this failure?',
  choice: {
    client: 'Bad input, expired session, missing permission, client mistake',
    app: 'A bug, a misconfiguration or a validation error in our own code',
    upstream: 'A third-party dependency failed, timed out or rate-limited us',
  },
})

"fault": { "value": "upstream", "confidence": 0.93 }

In keep, v.value autocompletes to the option names, so v.value === 'app' is checked by the compiler. Add an unknown option when the event may not carry the answer; without it the model picks the least wrong of the others, and you lose the signal that your events are missing a field.

Rank: a position on a rubric

Ordered levels, lowest first. The column carries the most likely level as value and the probability-weighted position as score, so severity.score > 1.5 reads as "closer to page than to watch":

export const severity = defineSignal({
  name: 'severity',
  when: e => (e.status ?? 0) >= 500,
  ask: 'How urgent is this failure for the on-call engineer?',
  score: ['noise', 'watch', 'page'],
})

"severity": { "value": "watch", "score": 1.2, "confidence": 0.6 }

Use score to sort and value to bucket. The confidence of a score signal is the probability of the top level, so a 0.6 here says the model was between watch and page. A model that answers without a distribution leaves confidence off choice and score columns; value and score are still there.

Scope with when, always

when is a plain predicate and runs before anything else. No model call happens when it returns false, which is what keeps a signal cheap: fault above costs nothing on the 99% of requests that succeed. It is required on any signal with keep.

The event a signal sees is the wide event in enrich, or the request context plus status, path, method and durationMs in keep. The fields evlog sets are typed; what your handlers logged with log.set() is there too, as unknown:

when: e => e.status === 200 && (e.payment as { provider?: string } | undefined)?.provider === 'stripe'

Write a question the model can answer

The model reads the event and the question, nothing else. Three checks before you ship one:

1. The answer is in the event. "Would a retry succeed" works because the error name and status are there. "Is this user about to churn" does not.
2. The answer is one value. "Was it slow, and was it our fault" is two signals.
3. A rule could not do it. If when alone answers the question, you do not need the model. status >= 500 is a rule. "Transient or permanent" is not.

Recipes has twelve questions that pass, and the ones that do not.