Deep Context Graph · Advancements Brief

From Navigation to Projection

How the Deep Context Graph evolved from an agentic memory-navigator into an information-theoretic engine that computes the minimal sufficient context — and measures that it is enough.

Author · Dr. Ali Arsanjani Assistant · Claude (Claude Code) Date · 2026-08-21 Scope · ~/deepcontextgraph → ~/deep-context-graph

~600× fewer

Tokens collapse — 96,043 → 164 for the reference task

~300× denser

Density rises — 0.016 → 5.07 bits/token

0→100%

Compliance — hard constraints satisfied vs the raw dump

The one-sentence advancement

The original DCG navigated a graph and let an LLM judge the trace; the advanced DCG projects the graph through a formal operator and lets an external verifier measure the result. It is a change of kind — from retrieval you hope is right to computation you can certify.

1Where it started — the original

The original DCG (~/deepcontextgraph) is a multidimensional “Active Memory Space”: nodes at coordinates over three axes — Knowledge, Temporal, Causal — retrieved by pivoting along one axis at a time (relational, directional retrieval, not flat vector lookup). On top sits a Fractal Chain-of-Thought navigator: per step it inspects Micro → Meso → Macro, runs gap analysis (“what am I missing?”), hill-climbs to the next pivot, and halts on a confidence threshold. Its enemy was the silent over-claim — declaring an investigation complete while the root-cause node was never retrieved.

Evolution of the concept lab

version1

Deterministic FCoT navigator over a hand-built K/T/C graph + a passive trajectory visualizer.

version2

A real multi-agent framework — FIPA-ACL messages, async broker, versioned blackboard, hub-and-spoke; investigative-reporting framing.

version3

LLM-backed K/T/C agents + an Evaluator scoring J⁺/J⁻ per hop; introduces the macro / meso / micro / nano apertures.

real_dcg

Ground-truth + planted decoys / early-stop traps, honest self-grading, a decoupling emit() hook, and the --reconcile Cognitive Reconciliation guardrail against silent completeness failure.

Phase shift · navigation → projection

deep-context-graph

Information-theoretic re-foundation. The aperture becomes a formal operator; quality becomes measured entropy, density, and externally-verified fidelity. Everything below is net-new here.

2The re-foundation — the aperture operator

Instead of a navigation trace, the advanced engine computes context as a composition of four typed layers, projected at a point in time τ:

𝒜(τ) = σ( 𝒢_D ∘ 𝒢_C ∘ 𝒢_T ∘ 𝒢_K ) │ τ_active

Compose right-to-left: 𝒢_K structural extraction → 𝒢_T temporal mask at τ → 𝒢_C Pearlian do(X) invariant chains → σ = 𝒢_D policy projection emitting hard constraints. Budget ≤ 1500 tokens, < 50 ms.

The decisive additions over the original K/T/C navigator: a fourth typed layer 𝒢_D (Decision/Policy) that acts as the projection σ — SOC2/HIPAA/PCI rules as machine-checkable predicates — and genuine Pearlian causality (do(X) graph mutilation severs a node’s causes and drops correlational edges). The original code contained no entropy, do(X), or counterfactual operator at all; these are new.

3Original vs advanced

Table 1 — What changed, dimension by dimension.
Dimension	Original (navigator)	Advanced (engine)
Context production	pivot + hill-climb (a trace)	a formal operator 𝒜(τ), deterministic, <50 ms
Layers	K / T / C axes	+ 𝒢_D governance-as-projection
Causality	correlational edges	Pearlian `do(X)` graph mutilation
Quality signal	LLM’s J⁺/J⁻ opinion	measured H(C), density & non-circular fidelity
Verification	model self-assessment	independent AST verifier — SQL / Python / TS / Go
Temporal	chronology axis	prunes expired state → “don’t reintroduce the legacy PK”
Deploy	local demos / traces	single Cloud Run image + React/Cytoscape dashboard

4The result — Conditions 0–4

Reference task: a payments-microservice UUIDv7 primary-key migration at τ=150. Fidelity is measured — an agent generates SQL from each context; an independent parser checks it against four constraints. Numbers quoted exactly from the engine’s benchmark output.

Table 2 — Adding each layer makes one more buried constraint legible.
Condition	Tokens	H(C) bits	Density b/tok	Fidelity	Constraint made legible
0 · raw 93k dump	96,043	571	0.016	0%	all cues drowned
1 · 𝒢_K	50	75	3.25	25%	UUIDv7 target type
2 · +𝒢_T	66	135	3.86	50%	legacy bigserial expired
3 · +𝒢_C	99	241	4.46	75%	index build → CONCURRENTLY
4 · full 𝒢_DCG	164	466	5.07	100%	SOC2 `lock_timeout ≤ 50ms`

Intellectual honesty (the repo’s own caveat)

Total Shannon entropy is not where the win is — the 93k dump’s H(C) (571 bits) is only ~1.2× the 164-token aperture’s (466 bits), because a repetitive dump is highly compressible. The real, defensible wins are tokens and density only.

Don’t hand the model raw text. Hand it the minimal dense context this step actually needs — and a certificate that it’s enough.

5New capabilities

Two research augmentations exist only because the engine has the entropy + causality machinery — impossible in the original navigator.

NEW A1 — Epistemic-Entropy Firewall

Confidence-tiered nodes + a min_confidence gate. On a graph polluted with 40 causally-invariant inference-noise nodes:

Table 3 — The firewall is orthogonal to causal pruning.
Arm	Tokens	Density	Fidelity	Gated
firewall OFF	1,492	2.12	0.0	0
firewall ON	164	5.07	1.0	40

𝒢_C keeps all 40 noise nodes (they’re “invariant”); only the confidence gate removes them. → the engine can separate trustworthy from merely relevant — a distinction the original couldn’t draw.

NEW A2 — Aperture Sufficiency Certificate

Turns “smallest connected set” into a measured certificate: sufficiency (F_aperture ≥ F_full) + tightness (load-bearing / total, via per-entity ablation). Payments task: sufficient=True, tightness=0.5. → the aperture can now prove it’s enough and quantify its slack.

Also net-new

Pearlian do(X) interventional projection — counterfactual “what does this change actually cause.”
Governance-as-projection — hard constraints emitted as predicates an external verifier checks.
Measured H(C) from real Gemini logprobs (offline order-2 Markov fallback) + density.
Multi-language external verifier (SQL/Python/TS/Go), comment- and literal-stripped.
Live dashboard — Cytoscape hypergraph, τ slider + H(C) toggle, WebSocket runner, size-vs-fidelity chart.
One-image Cloud Run deploy + the EE runtime monitor for silent-failure detection.

6Implications

Context engineering becomes computation, not retrieval. You compute the minimal sufficient projection under a token budget, with a certificate that it’s enough.
Compliance is enforceable by construction. Governance moves from “rules pasted in the prompt” to “predicates emitted and externally verified” — the 0% → 100% jump.
Trust becomes a first-class axis. The firewall strips confident-but-unwarranted inferences that are causally relevant yet epistemically junk.
Evaluation is defensible. Non-circular, external, multi-language verification is publishable where J⁺/J⁻ self-scoring is not.
One lineage with FCoT 3.0. The original’s --reconcile guardrail against silent completeness failure is the ancestor of “verify behavior, not status”; the DCG apertures are the fractal apertures. Measurement + gating + grounding replacing assertion — the same move, at the context layer.

7Honest limitations (from the code itself)

Hand-authored graph — no extractor yet; the research brief calls this the largest gap.
Single-shot benchmark — the “turn-1 rule violated at turn 45” thesis is asserted, not yet measured (needs a multi-turn drift harness).
Offline H(C) is a Markov surrogate; sufficiency is an ablation proxy, not exact mutual information.
Total Shannon entropy is not the win — always frame it as tokens + density.
Docs say “50 tests”; the suite is actually 59 (the A1/A2 additions post-date the doc).