Verify Behavior, Not Status

1Introduction

Agentic systems increasingly run long-horizon workflows — deployments, migrations, multi-source research — where a failure at step N can be masked by a success signal and discovered much later. The dangerous failure mode is not the loud error; it is the silent success: a process that returns a green status while its actual objective was not met.

This paper is an experience report. Over one working session we deployed the open-source Long Horizon harness to Cloud Run, shipped a feature to it, then turned the FCoT 3.0 protocol inward — auditing our own engineering rather than a research question. The contribution is threefold:

1. A methodological transfer. FCoT 3.0 was designed to discipline LLM synthesis; it transfers cleanly to engineering audit, its instruments mapping naturally onto system scopes.
2. A defect taxonomy. “Assert-vs-verify” is a fractal defect — the same bug in different vocabulary at macro/meso/micro. Naming its self-similarity is what made the fix generalizable.
3. Two verified interventions, released as pull requests, each closing one instance of the defect, with verification evidence.

2Background

2.1 · The Long Horizon harness

Long Horizon is a reference agent harness on Google’s ADK and Agent Platform. Relevant subsystems: an ADK backend on Cloud Run; a per-user Vertex Agent Engine sandbox for tool execution; a web-research sub-agent that calls Gemini’s native google_search grounding; and a workspace the agent reads and writes. The deploy path (make deploy) provisions infrastructure via Terraform — including two Cloud Run services seeded with a placeholder image — then rolls the real images in later steps.

2.2 · FCoT 3.0 in brief

FCoT 3.0 treats reasoning as observation at a chosen resolution. Its instruments, as used here:

• Context apertures — macro forces → meso subsystems → micro code → nano evidence. Levels are self-similar re-projections of one substrate; moving down is a change of vocabulary that exposes the commitments the level above rested on.
• Aperture isolation (Invariant Zero). A scope crosses a boundary only via an output contract — never its interior reasoning. A leak invalidates every downstream verdict.
• Gated hill-climb. The producer generates; it never terminates. An independent verifier with fresh context and an adversarial mandate returns PASS / PASS-WITH-FLAGS / FAIL. In synthesis the gate is arithmetic (J⁺ ≥ 10/12, J⁻ ≤ 2/10, ΔJ⁺ ≥ +2); in engineering it is a behavioral assertion.
• Instrumented dual objectives. f_max (value) and f_min (degradation), each with a measurable proxy declared up front. An unmeasured objective is not an objective.
• Silent-failure detection. A self-reported success is a hypothesis to test, never ground truth.

3Method

We ran FCoT 3.0’s audit machinery over one subject: the harness deployment and the feature we shipped to it.

1. Declare f_min proxies for the engagement.
2. Decompose the subject across macro / meso / micro apertures.
3. Verify each aperture’s contract adversarially, asking: if this step failed right now, would any signal tell us?
4. Trace boundary leaks (Invariant Zero).
5. Rank interventions by slope (impact / effort) and implement the top-ranked, independent ones in parallel.

4Findings

4.1 · The fractal defect

The central finding: one defect — assert-vs-verify — appeared at all three scopes, each time dressed in the local vocabulary.

The macro instance was not hypothetical: the deploy-backend recipe chains commands with ; rather than &&, so an agents-cli deploy failure (a Cloud Run env-var type conflict — LHA_REMINDER_DB_URL set as a secret but re-sent as a literal) was masked by a trailing gcloud run services update that succeeded. The pipeline reported success; the backend ran cloudrun/container/hello.

4.2 · The Invariant-Zero diagnosis

Aperture isolation localizes where the defect entered. A micro signal (“exit code 0”) leaked upward and was consumed as a macro claim (“deploy succeeded”). By Invariant Zero, that leak invalidated the macro verdict — which is precisely why the placeholder slipped through. The correct macro contract is behavioral (“a real image is serving”), not a status code.

4.3 · Instrumented objectives (baseline)

5Interventions

The fake-edge test found no data dependency between the two top-slope fixes, so they were built as independent pull requests.

5.1 · B-1 — Deploy self-verification gate macro

A post-deploy gate (scripts/verify_deploy.sh, wired as make verify-deploy and chained as deploy step [4/4]) reads the live Cloud Run services and fails unless each is (a) not on the placeholder image and (b) serving a Ready revision. It replaces a status signal (exit code) with a behavioral contract (what is actually serving).

==> Verifying deploy (behavioral gate) — deepcontextgraph / us-central1
  ✓ lha:     real image, revision Ready
  ✓ lha-web: real image, revision Ready
==> Deploy verified: both services serving real images and Ready.

5.2 · B-2 — Fractal, grounded research log meso nano

The harness auto-saves every web search to research_log.md. The audit flagged it as write-only, entropy-additive, and ungrounded. B-2 restructures each entry to carry its warrant — the nano aperture, where a claim earns the right to have been saved:

• Grounding (nano): extract the source URLs from each result and list them; a result with no traceable source is flagged ⚠ UNGROUNDED rather than silently trusted.
• Entropy control: a repeated (normalized) query is marked (repeat) instead of logged as fresh knowledge.
• Structured entry: query + answer + sources — evidence, not transcript.

Both interventions are best-effort and non-blocking: a logging or verification failure can never turn a successful search or deploy into an error.

Verify behavior, not status. A status signal is a claim the producer makes about itself; the discipline is to test it against the behavior it implies.

6Results

We report only outcomes observed this session. This is a single case study, not a controlled experiment; the values below are existence/behavior evidence, not statistical deltas.

Keep-rate movement. The most directly moved proxy: backend deploy keep-rate went 0% → 100% — the first make deploy shipped the placeholder (0% of the intended image surviving); after the source redeploy and with B-1 guarding future rolls, the real image is what serves and what the gate certifies.

Defect-visibility movement. Two of the three Table-1 instances now emit a signal where they previously emitted silence: the deploy gate turns a masked roll into a hard failure, and the research log turns an ungrounded claim into a visible flag. The third (the swallow-all callback) remains instrumented-as-future-work.

7Discussion

7.1 · The general principle

The result generalizes to one sentence: verify behavior, not status. A status signal (exit code, “saved”, “no exception”) is a claim by the producer about itself; FCoT 3.0’s discipline is to treat that claim as a hypothesis and install an independent check of the behavior it implies.

7.2 · Fractal bug, fractal fix

The audit was efficient because the defect was self-similar. Recognizing the same bug at three scopes meant one fix pattern — an independent behavioral gate at the boundary — applied three times, in three vocabularies. That is the practical payoff of the fractal framing: three seemingly unrelated issues become three instances of one.

7.3 · Comparison to the informal pass

An informal fractal analysis correctly named the self-similar bug but could not locate or measure it. The full protocol added the Invariant-Zero localization (the exit-code→success boundary leak), the proxies that made “better” a number, and the bounded-recursion rule that identified the single node (prod tool-exec verification) warranting a deeper sub-aperture — the difference between a good metaphor and an instrumented finding.

7.4 · Limitations

• Single case study, no controlled baseline. We cannot claim FCoT 3.0 caused the improvements versus a careful ad-hoc review — only that it made the defect legible and the fixes systematic.
• Self-audit bias. Producer and auditor shared a model; fresh-context adversarial passes mitigate but don’t eliminate shared blind spots. Cross-model verification would strengthen it.
• Grounding is shallow. B-2 treats “has a traceable source URL” as grounded; it does not fetch the URL, to avoid blocking the hot path.
• Prod tool-exec verified by proxy. The prod UI is IAP-gated; end-to-end sandbox execution was confirmed via an identical local Agent-Engine path, not in prod directly.

8Future work

• B-3 — research-log read-back. Let the agent consult the log so repeated searches are answered from it and knowledge compounds. Now unblocked because entries are grounded.
• Behavioral deploy canary. Extend B-1 from image identity to tool execution by wiring the dormant RUN_SANDBOX_PROBE hook — refuse a revision unless a canary sandbox call actually runs.
• Callback observability. Emit a dropped-save counter so the swallow-all path becomes visible — closing the last silent instance.
• IaC drift capture. Move the runtime-image repo and IAM bindings into Terraform so the reproducibility proxy reaches zero drift.

9Conclusion

Applying FCoT 3.0 as an audit discipline turned a vague “what can we do better?” into a precise, instrumented finding: a fractal silent-success defect recurring at three system scopes, entering through a single boundary leak. The corresponding fractal fix — verify behavior, not status — was implemented as two independent, tested, released gates and verified against a live system. The broader lesson for long-horizon agentic engineering: success signals must be earned behaviorally at every scope, and a fractal reasoning protocol is an efficient way to find where they are merely asserted.

Appendix A · Protocol-to-scope mapping