Middleware

A middleware intercepts a call site rather than occupying a slot in a spec. It is the one kind an operator installs and a flow does not mention, which is the honest arrangement for something that runs on every node whether that flow asked or not.

That is also why it cannot be spec-named. A component a flow does select is a transform, which already exists. A spec that writes a middleware's component type is refused, saying so.

heddle run flow.json \
  --plugin ./retry-policy.json \
  --plugin-config RetryPolicy='{"maxAttempts":3}'

Four worked policies, one seam each (a retry, an approval gate, a per-node audit and a rate limit), ship in examples/policies, with flows that run them without a credential.

Seams

A seam is a place in the engine where a middleware may be consulted, and each names the verdicts it will honour. The set is closed, declared as data, and read by the manifest validator, the verdict reader and the handshake alike.

All five are consulted. The set is closed and nothing is held in reserve: a manifest subscribing to a name that is not in the table is refused at load as a name heddle does not have, listing the ones it does.

Note what toolCall does not admit: retry. By the time a tool call is decided, the assistant message that asked for it is already in the conversation, so re-issuing the call is not re-entering a clean state.

Gating a tool call

toolCall is the seam an approval gate hangs off, and it is the first with a before half, the first place a middleware decides what happens rather than reacting to what did.

serve({
  ApprovalGate: {
    toolCall: {
      before: ({ subject, input }, ctx) => {
        if (subject.toolName !== 'shell') return { action: 'proceed' };
        if (/rm|sudo/.test(String(input.cmd))) {
          return { action: 'reject', reason: 'destructive command' };
        }
        return { action: 'proceed' };
      },
    },
  },
});

modify does not end the walk. A verdict that replaces, rejects or fails settles what happens; a modification only changes what the next middleware is deciding about, so the chain carries on with the new arguments. That is what lets a redactor and a gate compose: the redactor rewrites the arguments and the gate then sees what would actually run rather than what the model first asked for.

A middleware subscribes per half, so one that declared only after is never asked. The call site checks before it consults, which matters more here than at nodeError: this runs on every tool call of every round rather than only on a failure.

After the call

The same seam has an after half, consulted whether the tool returned or threw. A middleware that truncates a large result and one that turns a failure into a canned answer are the same hook seen from two sides.

serve({
  Auditor: {
    toolCall: {
      after: ({ subject, outcome }) => {
        if (outcome.ok && JSON.stringify(outcome.value).length > 10_000) {
          return { action: 'replace', value: { note: 'result too large, omitted' } };
        }
        return { action: 'pass' };
      },
    },
  },
});

pass lets the outcome stand, replace substitutes a result the tool did not produce and is reported as a warning naming the middleware, and fail ends the run. retry is not admitted, for the reason above.

A middleware may subscribe to either half or both, and each is asked only if it declared it.

Writing one

A middleware subscribes to seams in its manifest and serves a handler per seam. The handler is keyed by seam name, so one component can subscribe to several and each gets its own function rather than a switch every author has to remember to write.

{
  "name": "retry-policy",
  "version": "1.0.0",
  "capabilities": [],
  "components": [
    {
      "componentType": "RetryPolicy",
      "kind": "middleware",
      "seams": { "nodeError": ["after"] },
      "schema": {
        "type": "object",
        "properties": { "maxAttempts": { "type": "number" } }
      }
    }
  ]
}

serve({
  RetryPolicy: {
    nodeError: {
      after: ({ subject, outcome }, ctx) => {
        if (ctx.attempt >= (ctx.component.maxAttempts ?? 3)) return { action: "pass" };
        if (!/timed out|429/.test(outcome.error.message)) return { action: "pass" };
        return { action: "retry", delayMs: 500 * ctx.attempt };
      },
    },
  },
});

ctx.component is the operator's configuration: {} when they supplied none, never undefined, because a middleware is host-configured and this is the only channel its settings arrive on.

ctx.admits carries the verdicts this seam will honour, sent with the handshake. A middleware that wants to work at more than one seam can read it and fall back rather than send a verdict it will be refused for, which under the fatal policy would cost the run.

A middleware is given the thing it is deciding about, and nothing wider. At nodeError and toolCall that is metadata: subject names which node or tool, outcome says how it went. At agentRound it is narrower still: a round number, and the names of the tools that round ran. At modelCall it is the request, conversation included, because a policy that may modify what is sent has to see what is being sent. At node it is that node's input state and its output, for the same reason, and it is the widest view any seam offers.

That is worth being deliberate about on a server, where a middleware is named nowhere in the caller's document. An installed node policy sees every caller's data flowing through every flow. It is the operator's own code, trusted the way --tools-dir is trusted, but a policy that only needs to know that a node ran should subscribe to after and read subject, not log outcome.value.

A component that needs the data in order to do its job is a plugin node, which the flow names and the caller therefore chose.

Verdicts

The chain is consulted in the order the operator loaded the plugins, the sequence of --plugin flags, and the first non-pass verdict wins. That makes the chain's behaviour something an operator can change without touching a plugin or a flow.

A retry spends an iteration, so --max-iterations remains the real bound on how many times anything executes. --max-node-attempts caps one arrival at a node and defaults to 3; the budget is per arrival, so a node inside a loop gets a fresh one each visit.

A granted retry and a substituted result are each reported as a warning event naming the middleware that decided it, so "the flow returned something odd" is traceable.

Around a model call

modelCall wraps every request heddle sends, an agent's rounds and an LlmNode's single call alike. It is the only seam that admits retry, and the reason is what has already been said: a failed tool call leaves an assistant message asking for it in the conversation, so re-issuing is not re-entering a clean state, while a failed model call has changed nothing. That makes this the seam a 429 policy hangs off.

serve({
  RateLimitPolicy: {
    modelCall: {
      after: ({ outcome }) => {
        if (outcome.ok) return { action: 'pass' };
        if (!/429|timed out/.test(outcome.error.message)) return { action: 'pass' };
        return { action: 'retry', delayMs: 1000 };
      },
    },
  },
});

before sees the request and may proceed, modify it, replace it with an answer (a cache hit, where nothing is sent at all), or reject it, which fails the node with the stated reason.

Retries are bounded at three attempts by heddle, not by the chain. A ceiling a middleware can raise is not a ceiling; a refused retry is reported as a warning and the outcome stands.

Configuration

--plugin-config <ComponentType>=<json> passes the operator's settings, once per component type. The value may be @path to read it from a file instead of the command line.

The manifest's schema is applied to it before anything is built, and the check runs even when the operator supplied nothing. That is the case that matters, since a middleware declaring a required field and receiving {} is exactly the one that would otherwise read undefined on the first node that failed.

On a server

heddle-server takes the same two flags, and they are the only way to install middleware on it. A plugin submitted with a request is refused with a 400 if it declares any, whether or not --allow-request-code is on.

heddle-server \
  --plugin ./policies/spend-limit.json \
  --plugin-config SpendLimit=@/etc/heddle/spend-limit.json

Everything loads before the port opens, so a plugin that will not start, or a --plugin-config that fails its schema, is a server that does not start rather than a 500 on whichever request first needs it. GET /v1/capabilities reports what is installed under middleware, since a caller cannot select one but can have their tool call rejected by it.

The difference from the CLI is lifetime. One process per plugin serves the whole server, not one run: that is what makes an MCP session or a connection pool worth holding, and it is why an installed plugin must not keep per-run state in a module variable, since concurrent runs share the process and nothing separates them. The chain itself is still built per run, because a middleware holds that run's dependencies; only the processes behind it are shared.

One consequence reaches plugin authors directly: a shared plugin's runTool must name the call it was made inside. There is no server-wide fallback, because the tools reachable from a call are the ones its own run brought, and answering an unattributed request would hand one caller's tools to another caller's plugin. The plugin runtime does this for you; a plugin speaking the protocol by hand has to.

Around every node

node is the widest seam. It wraps an execution of a node, any node of any type in any flow, rather than a failure or a call, which is what makes a cache, a dry run or an audit possible at all.

serve({
  NodeAudit: {
    node: {
      before: ({ subject }, ctx) =>
        (ctx.component.dryRun ?? []).includes(subject.nodeType)
          ? { action: 'replace', value: ctx.component.stub ?? {} }
          : { action: 'proceed' },

      after: ({ subject, outcome }, ctx) => {
        ctx.emitEvent('node', { node: subject.nodeName, ok: outcome.ok });
        return { action: 'pass' };
      },
    },
  },
});

before may proceed, modify the node's input state, replace its output without running it, or reject it and end the run. after sees what the execution came to, { ok: true, value } or { ok: false, error }, and may pass, replace or fail.

node and nodeError sit at the same position, and the nesting is what keeps them apart. nodeError is inside node, because it is the seam that owns retries:

• A retry abandons this execution for another, so after is not consulted for it and the next attempt starts again at before, with ctx.attempt moved on.
• A settled execution reaches after exactly once, whether it produced a result or failed, which is why an auditor subscribing only to node sees every node once.
• node's after admits no retry. The seam that does is the one nested inside it.

A replaced node takes the unbranched edge, for the same reason a nodeError substitution does: a supplied result is a result and never a route. A node whose every edge is labelled reports that the run cannot continue past it.

Around an agent's rounds

A round is one model call plus the tool calls it asked for. agentRound is consulted before each round and after each round that ran tools, and it is the narrowest seam heddle has: it may stop a round or let it happen, and nothing else.

The narrowness is the design. The two seams inside a round already own what is sent (modelCall) and what runs (toolCall), so the one thing neither of them can say is that there should not be another round, which is the thing an agent looping on tools costs money for. heddle's own ceiling is ten rounds and an operator cannot move it; this is where a lower one hangs.

serve({
  RoundBudget: {
    agentRound: {
      before: ({ input }, ctx) => {
        // Never above the engine's own ceiling, and usually far below it.
        const cap = Math.min(ctx.component.maxRounds ?? 3, input.maxRounds);
        return input.round > cap
          ? { action: 'reject', reason: `${cap} rounds is the budget` }
          : { action: 'proceed' };
      },

      after: ({ outcome }, ctx) => {
        ctx.emitEvent('round', outcome.value);
        return { action: 'pass' };
      },
    },
  },
});

before is given { round, maxRounds }: the round about to start, counted from 1, and heddle's own ceiling, so a policy tightening it need not write 10 down and drift when it moves. after is given { round, toolCalls }, the names of the tools that round ran. reject ends the node before the model is called and fail ends it after the round; both errors name the middleware, the round and the stated reason.

Nothing is reserved

Every seam in the table is consulted, and a name outside it is refused as a name heddle does not have rather than one it has not reached.

toolResult was the last reserved name, and it was dropped rather than built. It was written down before toolCall had an after half; now that it has one, that half is shown the call, its arguments, its id and the result, so a seam shown the result without the call that asked for it would be strictly less. Leaving it reserved was worse than deleting it, because a load-time refusal saying heddle does not consult it yet promises something that should never ship.

One smaller gap worth knowing: heddle run -i builds its own runner from the same options, so the chain is installed and a retry works, but the chat interface reads four event types and has no warning arm, so the retry happens silently. heddle run renders it.