There’s a specific moment when a clean architectural invariant collides with production reality. For me it was staring at a scan result showing 442 processed research jobs and realizing that 27 of them — roughly 6% — had been silently truncated at exactly 4096 tokens. Not failed. Not flagged. Truncated and returned as if complete, corrupting the downstream record with no indication anything had gone wrong.

That’s where this post starts.

The Invariant, Stated Precisely

DeepSeek’s harness architecture — the one that’s attracted considerable attention since the repository crossed 137k GitHub stars — is built on what sounds like a simple guarantee: everything the model sees must be reconstructible byte-for-byte from an append-only event log. Every tool call, every response, every injected context blob gets appended as an immutable record. Replay the log and you get the exact same model-visible context. The architecture’s appeal is that this makes the system auditable, debuggable, and — critically for prefix caching — economically efficient.

The invariant sounds simple. It’s not. What “model-visible ↔ durably referenced” actually requires is that nothing between your application code and the model’s input can silently modify, truncate, or discard content without that modification being itself logged as an event. That’s a much stronger claim than “we write logs.” It means every library default, every streaming cut-off, every session boundary is a potential violation point.

DeepSeek’s research materials — which I’ve been working through as background for our own architecture decisions — describe this thoroughly in the context of their training harness. Reading through those transcripts, what stood out to me was how cleanly they handle this in a controlled research environment, where the event producers are instrumented code they control end-to-end. The harder question is what happens when you try to apply the same invariant to a live, multi-source personal AI pipeline with third-party library dependencies that have their own opinions about token limits.

Violation One: The pydantic-ai Silent Truncation Bug

This one we found ourselves. We were running a retrospective scan across completed research jobs — 442 total — looking for output quality regressions. What the scan surfaced was a cluster of jobs where the model’s output ended mid-thought, cleanly, at what turned out to be exactly 4096 tokens every time.

pydantic-ai’s default max_tokens cap is 4096. When a response hits that ceiling, the library returns the truncated output as a completed response — no error, no warning in the return value, no indication in the standard logging path that truncation occurred. From the perspective of our append-only log, those 27 jobs looked like successful completions. The log faithfully recorded what the model returned. The problem was that what the model would have returned, if the cap hadn’t been hit, was silently discarded at the library layer — below the level our logging saw.

This is a canonical invariant violation: content that was model-visible (the full response was generated) became permanently unavailable because a library boundary silently discarded it before we could append it. The log is append-only, but it’s recording the wrong thing.

The bug had been running undetected for weeks across those 442 jobs before the scan caught it. The fix is straightforward — set max_tokens explicitly to something appropriate for your workload, and instrument the response metadata to detect stop_reason == "max_tokens" as an error condition — but the deeper lesson is about where invariants actually live. We were treating our logging layer as the invariant enforcement point, but pydantic-ai’s default was violating the invariant upstream.

What you can’t recover: those 27 truncated outputs. The sessions are gone. You can re-run the jobs, but the log entry that claimed completion is now a lie in the historical record.

Violation Two: Session Compaction Breaks the Reconstruction Guarantee

The second failure mode is structural rather than accidental.

Long-running Claude Code sessions eventually hit context limits. Compaction is the standard response: summarize the current context, discard the full history, continue with the summary as the new base. From a user experience standpoint, this is reasonable. From an append-only log standpoint, it’s a discontinuity. The context that existed before compaction cannot be reconstructed from the events after it; the summary is a lossy compression, not a replay-faithful record.

The research I’ve been working from — external write-ups on Claude Code’s compaction behavior, including community posts and the GitHub issue tracker — describes a variety of approaches people have tried, including PostToolUse hooks and session lifecycle hooks to inject context before and after compaction events. Reading through these, what stood out to me was how many of the proposed solutions have a subtle problem: they hook on events that don’t exist, or that fire at the wrong point in the lifecycle.

We built a context-restore hook for our own sessions, and the first version made exactly this mistake. The hook was registered on a PostToolUse+compact matcher — a pattern that would be elegant if it existed. It doesn’t, at least not in a form that fires reliably at the point where the pre-compaction context is still injectable. We caught this before it hit production only because we had a test session that deliberately triggered compaction, and the hook simply didn’t fire.

The corrected architecture uses SessionStart as the injection point — restoring relevant context at the beginning of each new session segment rather than trying to intercept the compaction event itself. This is less elegant (it means the restored context is always injected, not just post-compaction), but it’s reliable. The external sources I reviewed, including a few production-oriented hook guides, describe similar patterns — inject-at-start rather than intercept-mid-compaction — and our experience confirms that’s the right tradeoff.

What this costs in terms of the append-only invariant: the restored context is a first-class event in the log (it’s injected as a tool result at session start), but it’s not a replay of the original events. It’s a reconstruction from a separate memory store. The log remains auditable, but it no longer supports byte-for-byte reconstruction of the pre-compaction context.

The Prefix Cache Consequence

This matters economically, not just architecturally.

DeepSeek’s harness documentation — and the broader literature on prefix caching — makes the case that consistent, append-only context enables very high token reuse. The claims I’ve seen cited are on the order of 120x reuse ratios in well-structured workloads. The mechanism is that if the prefix of your context is always the same (system prompt, pinned memory, recent events, in that order, byte-for-byte), the KV cache can be reused across calls without recomputation.

Both violation types we’ve described destroy this. Silent truncation means the log doesn’t accurately represent what the model saw, so when you reconstruct context from the log for a follow-up call, the prefix doesn’t match. Compaction means the session context changes shape discontinuously, breaking any prefix that depended on prior session history.

In practice, we’ve seen cache hit rates vary significantly with session continuity. We don’t have clean A/B data to attribute this specifically to invariant violations versus other factors, but the correlation with session boundaries and the truncation-affected jobs is suggestive.

A Practical Audit Checklist

Based on what we’ve found, here’s what I’d check in any pipeline claiming append-only semantics:

Library defaults that silently violate the invariant:

  • max_tokens: check every LLM client initialization; the default is rarely what you want, and hitting it produces silent truncation, not errors
  • Streaming cutoffs: some streaming implementations truncate at a byte limit before token limits; check whether your response aggregation handles mid-stream termination
  • Session ID assumptions: if your logging layer assumes session continuity and a new session silently starts (after a crash, timeout, or compaction), you’ll have events logged against the wrong session context

Detection before corruption:

  • Instrument stop_reason on every completion; max_tokens or length stop reasons should be treated as errors and trigger a re-run or explicit flag, not silent success
  • Log context length at each call alongside the response; sudden drops in context length are a compaction or discontinuity signal
  • Run periodic reconstruction tests: pick a random completed session, attempt to reconstruct its model-visible context from the log, and verify the token count matches what was sent

What an invariant checker actually looks like: We have a CI artifact that replays a sample of logged sessions and checks that the reconstructed context matches what the log claims was sent. Currently it covers about 30% of sessions (sampling, not full replay — full replay is too expensive). It catches truncation violations and session discontinuities. What it doesn’t catch is compaction events, because those are by design — the post-compaction context is intentionally different from the pre-compaction context.

That last part is honest about what “independent invariant checker” means in practice versus aspiration. The aspirational version checks everything. The real version checks what you can afford to check and flags the rest as known exceptions.

What This Means for System Design

The append-only invariant is worth pursuing. It makes debugging tractable, it enables prefix caching, and it gives you a substrate for auditing model behavior across time. But the invariant doesn’t live in your architecture document — it lives in every library call, every session boundary, and every configuration default you inherit from dependencies.

The truncation bug we found in pydantic-ai isn’t a pathological edge case; it’s exactly the kind of thing that happens when you compose multiple systems, each with their own defaults, and assume the composition preserves properties that none of the components were designed to guarantee jointly.

What we’re building toward is a system where invariant violations are first-class events in the log rather than silent corruptions. A truncation isn’t a failed job — it’s a logged truncation event that downstream systems can reason about and flag. A compaction isn’t a discontinuity — it’s a logged compaction event with a pointer to the memory store snapshot that was used to reconstruct context.

That’s harder to build than an append-only log. But it’s what “append-only” actually requires in a live system, as opposed to a controlled research environment where you wrote all the event producers yourself.