Claude Code Uncertainty Heatmap for AI Code Review

An AI agent can produce a 1,400-line diff across 23 files in a single sitting. A human reviewer still reads at human speed. The usual response is to read uniformly — start at line 1, give every hunk the same scrutiny, and hope the one genuinely dangerous change isn't buried in the middle. That worked when humans wrote code at the same rate humans review it. It does not scale to machine-speed authorship.

What you probably already know: AI-generated diffs need targeted review, and reviewer attention is the scarce resource. What you may not have considered: the agent already told you where it was nervous — just not in the diff. It told you in its reasoning. Somewhere in the session trace it wrote "I'm not sure this is the right migration path," or "this might break if the buffer is partially written." Then it kept going, and that flag disappeared into a .jsonl file you have never opened.

The uncertainty heatmap recovers those flags. It is a reviewable output of an auditable workspace: the agent's own hedges, mined from its reasoning during the run and ranked into a per-file review agenda. The key claim is narrow and worth stating precisely up front — this is a router for attention, not a bug detector, and the signal lives in the trace, not the diff.

Uncertainty signals inside AI-assisted edits

When Claude Code works on your codebase, it narrates as it goes — thinking blocks and assistant text that explain what it is doing and why. h5i reads that narration. Inside it you find sentences like:

"I'm not sure this is the right migration path for the foreign key constraint…"

"This might break the async flush if the buffer is partially written, need to verify."

"Tricky, the lock order here assumes single-threaded initialization, which may not hold."

The agent is hedging — waving a small flag that says a human should look here — and then burying it in the log. This is different from a code comment. The agent did not write // not sure about this into the file; it said so to itself while working, and that sentence never makes it into the diff a reviewer sees. It is recoverable only from the trace.

h5i recall notes uncertainty surfaces those flags for review.

What it looks like in practice

Say Claude just refactored your database layer. You run two commands:

$ h5i recall notes analyze    # parses the session log, links to HEAD
$ h5i recall notes uncertainty

You get this:

── Uncertainty Heatmap ─────────────────────────────────────────────
  9 signals  ·  session a3f8c12d  ·  4 files

  Risk Map
  ──────────────────────────────────────────────────────────────────────────
  src/db/migrations.rs        ████████████░░░░  ●●●  3 signals  avg  28%
  src/db/connection_pool.rs   ██████░░░░░░░░░░  ●●   2 signals  avg  38%
  src/repository.rs           ████░░░░░░░░░░░░  ●    1 signal   avg  45%
  tests/integration_test.rs   ░░░░░░░░░░░░░░░░  ●    1 signal   avg  62%

  Timeline
  t:12 ──────────────────────────────────────────────────────────── t:87
  ··············█···············▓·········█··█·····················░·····
               ↑t:18           ↑t:34     ↑t:52↑t:55

  Signals
  ──────────────────────────────────────────────────────────────────────────
  ⚠ HIGH  src/db/migrations.rs · t:18 · "might break" · 20% confidence
       "…might break the rollback if the constraint was partially applied
        on a previous failed run…"

  ⚠ HIGH  src/db/migrations.rs · t:52 · "not sure" · 25% confidence
       "…not sure whether the index creation here should be inside or
        outside the transaction boundary…"

  ▲ MOD   src/db/connection_pool.rs · t:34 · "assuming" · 50% confidence
       "…assuming the pool max is set before the first acquire call,
        but I should verify this…"

In under three seconds, you know exactly where to focus your review.

• migrations.rs is where the AI was genuinely nervous, two high-risk signals, average confidence 28%
• The timeline shows signals clustered around turns 18, 52, and 55, a burst of uncertainty mid-session, not at the end
• The individual snippets give you the reason for the doubt, quoted verbatim from the AI's thinking

You didn't have to read a single line of diff to know where to start.

Why this changes code review

The standard review process is symmetric: every line costs the same amount of reviewer attention. But AI-generated code isn't symmetric. Some of it the model wrote with total confidence, patterns it has seen ten thousand times. Some of it the model wrote while silently hedging, reaching into unfamiliar territory, making assumptions it flagged to itself as unverified.

Treating those two categories identically is a waste of your most limited resource: focused human attention.

h5i recall notes uncertainty makes the review process asymmetric in the right direction. Code the agent wrote without hedging gets a normal pass; the spots the agent itself flagged get read first and read hardest.

The result is not "fewer bugs" — that depends on your review, not on a tool. It is better ordering: your first and freshest attention lands where the agent's own doubt is concentrated, instead of being spent uniformly and running out before you reach the dangerous hunk.

How the signal is detected

There is no model-in-the-loop here, and it helps to be exact about that. h5i walks the session .jsonl line by line, pulls out the assistant's reasoning (thinking blocks where they are present, and assistant text blocks — which matter because Claude Code often redacts the raw thinking content), and runs a case-insensitive substring match against a fixed table of roughly thirty hedge phrases. Each phrase carries a hand-assigned weight between 0.20 and 0.50 that h5i renders as a "confidence" percentage. Lower weight, brighter red.

That is the whole mechanism: a lexical scan with a lookup table. It is deterministic, runs offline in milliseconds, costs no inference, and every signal it emits can be traced to the exact phrase that triggered it plus a verbatim ±150-character snippet. The price of that simplicity is everything a substring match cannot do — covered honestly below.

The signal table

The vocabulary of self-doubt phrases h5i matches, each mapped to its assigned weight:

The lower the weight, the brighter the red in the heatmap. The higher the density of signals on a given file, the more its risk bar fills. The Risk Map sorts files by lowest average weight first, so the file the agent hedged about most rises to the top.

Uncertainty is not the same as confidence

The single most important thing to understand about this table is what the percentage is not. It is not the model's probability of being correct. It is not a logprob, not a self-reported confidence the model emitted, and not a statistical calibration. It is a fixed weight a human picked for each phrase. Calling it "confidence" is a display convenience.

That distinction has teeth. A model can be confidently wrong — write a subtly broken lock ordering with no hedge at all — and that change produces zero signals. The heatmap measures expressed doubt, which is correlated with risk but is a different quantity. Expressed doubt is a leading indicator the agent generously volunteered; the absence of it tells you only that the agent did not say it was unsure, which is not the same as the code being safe. Treat a clean heatmap as "no volunteered flags," never as "reviewed and approved."

Why is expressed doubt still worth surfacing? Because it is nearly free to collect and it points at exactly the places where the agent was reaching beyond familiar patterns — the migration it had not seen before, the concurrency assumption it could not check. Those are disproportionately where real bugs in AI-generated code live, and a reviewer who starts there spends scarce attention well even though the score itself is only a heuristic.

The false-positive problem

A substring match does not understand sentences. "assuming" fires on "assuming the pool is initialized" (genuine doubt) and on "assuming good faith from the caller" (a figure of speech). "maybe" and "perhaps" are weak hedges that often mean nothing. The detector is deliberately high-recall and low-precision: it would rather show you a harmless hedge than miss a real one.

Two design choices keep that from becoming noise. First, detection reads only the reasoning trace, never the diff — so a // this is risky comment in the code is never a false positive, because the matcher never looks at code. The classic "marker in a comment vs. genuine doubt" confusion does not arise; the only text scanned is the agent's own narration. Second, every signal ships with its verbatim snippet and turn number, so triage is a glance: you read the quoted sentence, decide in two seconds whether it is real, and move on. The heatmap is a router for attention, not an adjudicator. A false positive costs you one sentence of reading; a false negative — a real doubt you never saw — is the expensive failure, which is why the recall bias is the right one.

Mapping signals to files

Each signal is tagged with the file the agent was editing at that moment in the session, so the Risk Map can group doubt by path. This association is temporal, not semantic: h5i attributes the hedge to whatever file was most recently being touched, not by parsing which file the sentence is about. Usually those coincide — the agent hedges while working on the thing it is unsure of. Sometimes they don't, and a signal shows up against a neighbouring file or as (no file) when the doubt was expressed during pure reasoning with no edit in flight. Read the file column as "where the agent was when it hedged," which is a strong prior for "where to look," not a guarantee.

Compared with LLM-scoring the diff and sentiment analysis

Two adjacent approaches are worth giving their due, because they catch things lexical mining cannot.

These are complementary, not competing. Lexical mining is a cheap pre-filter that runs on every session and routes a human's eyes; an LLM judge is a deeper, costlier second pass you might reserve for the files the heatmap already flagged. The honest framing: the heatmap raises the probability that you look at the right file first, for almost no cost. It does not promise the file is buggy, and it cannot promise the unflagged files are clean.

Filter by file

Already know which module worries you? You can filter signals to a specific path:

$ h5i recall notes uncertainty --file src/db/migrations.rs

Useful when you're reviewing a PR and want to jump straight to the model's doubts about the file you're responsible for, without wading through signals from unrelated parts of the diff.

The workflow

Here's how it fits into a natural AI-assisted development cycle:

# 1. Claude finishes a task and you're ready to review
$ h5i recall notes analyze              # parse the session, link to HEAD

# 2. Get the 10,000-foot view of what the AI touched
$ h5i recall notes show                 # footprint: consulted vs. edited files

# 3. Find the risky spots
$ h5i recall notes uncertainty          # heatmap + timeline + verbatim snippets

# 4. Now open your diff viewer — but start with migrations.rs, turn 18

The whole thing takes ten seconds. What you get back is a prioritized review agenda, assembled from uncertainty signals captured during the AI-assisted edit.

Under the hood

h5i reads Claude Code's session logs, .jsonl files that record every message, tool call, and thinking block in a session. It auto-detects the latest session for your repository, so you rarely need to pass a path manually.

After analysis, the results are persisted in .git/.h5i/ and linked to the commit OID, so you can revisit the uncertainty map for any past commit:

$ h5i recall notes uncertainty --commit a3f8c12

Because the analysis is content-addressed and lives under refs/h5i/*, it does not travel with a plain git push. Team members who pull your h5i data (h5i share pull) can run the same query on sessions they weren't present for — the reviewer reads the author's agent's doubts without having watched the session.

Failure modes and honest limits

A tool you trust to route attention deserves a clear account of when it goes quiet or misleads:

• Redacted or absent reasoning. Claude Code frequently redacts raw thinking content; h5i falls back to assistant text blocks, but a model that works silently — extended thinking off, terse narration — leaves little to match. Fewer words, fewer signals, regardless of actual risk.
• Absence of signals is not safety. The single most important caveat, worth repeating: a clean heatmap means "no volunteered hedges," not "reviewed and correct." Confidently-wrong code is invisible here.
• Fixed English vocabulary. Doubt phrased in a way the table doesn't list, or in another language, is missed. The list is a floor, not a model of how doubt can be expressed.
• Substring matching is literal. It cannot tell a real hedge from a figure of speech; the verbatim snippet is your filter for that.
• File attribution is temporal. A signal is tagged with the file in flight, which is a strong prior but not proof of what the sentence is about.

None of these are bugs to be fixed away — they are the shape of what a fast, deterministic lexical pass can and cannot do. Knowing the edges is what lets you use the signal without over-trusting it.

Getting started

# Install h5i (see github.com/h5i-dev/h5i for build instructions)
$ curl -fsSL https://raw.githubusercontent.com/h5i-dev/h5i/main/install.sh | sh

# Initialize in your repo
$ h5i init

# Install the hooks that capture every session (idempotent)
$ h5i hook setup --write

# After your next Claude Code session:
$ h5i recall notes analyze
$ h5i recall notes uncertainty

The h5i hook setup --write step wires in the SessionStart, PostToolUse, Stop, and UserPromptSubmit hooks that passively capture the prompts, traces, and test evidence the heatmap reads — so you never have to run those captures by hand.

Conclusion

The uncertainty heatmap rests on one observation: AI agents leak their doubt in plain text while they work, and that text is review signal. A grep-grade lexical pass over the reasoning trace is enough to recover it, rank it by file, and put the agent's nervous moments at the top of a reviewer's queue — for no inference cost and with full traceability back to the exact sentence.

What it is honestly worth: a cheap, deterministic router for scarce attention. What it is not: a measure of correctness, a calibrated probability, or a guarantee that the unflagged files are safe. Expressed doubt correlates with risk; it does not define it, and silence is not a clean bill of health. Used with that framing — first pass to order your review, not last word on whether the code is right — it changes where your freshest attention lands, which is most of what review triage is. Pair it with an agent-aware review pass and the broader risk framework, and the heatmap becomes the cheapest of several signals rather than a single thing you over-trust.

The practical takeaway is small and durable: when the agent told you it was nervous, don't make that the one thing you never read.

Frequently asked questions