Auditing AI-generated code: risk-rank what to review

When an agent can write a thousand lines in a minute, line-by-line review stops scaling. Reviewers either rubber-stamp AI diffs or burn out trying to read everything with equal care. Both fail the same way: attention is spread evenly across changes that are very much not equally risky.

The problem: volume without prioritization

The hard question isn't "is this diff correct" — it's "of the 40 commits on this branch, which three should a human actually scrutinize?" Git gives you no signal to answer that. Diff size is a poor proxy; a one-line change to an auth path beats a 400-line rename.

How h5i solves it

h5i derives risk signals from the AI session itself — moments of expressed uncertainty, blind edits (a file edited with no prior read in that session), edit churn, and the scope of files touched — and ranks commits so the riskiest surface first. It's deterministic: no model sits in the audit path.

Commands

Triage a branch before merging — riskiest commits first:

$ h5i audit review --limit 50

Suggested Review Points — 2 commits flagged (scanned 50)
─────────────────────────────────────────────────────────
  #1  a3f8c12  score 0.74  ████████░░
     add retry logic to HTTP client
     ⚠ high uncertainty · 5 edits · 4 files · 1 blind edit
  #2  9e21b04  score 0.45  ████░░░░░░
     refactor parser  ·  moderate complexity

See which files Claude edited without reading first (memory edits, not state-aware edits):

$ h5i recall notes coverage --max-ratio 0.5
  src/auth.rs        2 blind edit(s)   coverage 0.33
  src/http_client.rs 0 blind edit(s)   coverage 1.00

Export a date-ranged report for a regulated workflow, and surface it where reviewers already are:

$ h5i audit compliance --since 2026-01-01 --until 2026-03-31 \
    --format html --output audit.html
✔  wrote audit.html — 1,204 commits, 38% AI-assisted

$ h5i share pr post      # sticky PR comment with provenance per AI commit

Worked example: a triage funnel

Before merging an AI-heavy branch, run the funnel: h5i audit review to rank, open the top two or three by score, and check their attention coverage. Everything below the threshold got a normal pass; your deep review went where the signals pointed. Pair the run with h5i audit compliance when you need an auditable trail.

Frequently asked questions

How does h5i decide which AI commits are risky?

It scores commits from signals captured during the AI session: expressed uncertainty, blind edits (a file edited with no prior read), edit churn, and how many files the change touched. h5i audit review ranks commits by that combined score, riskiest first.

What is a 'blind edit' and why does it matter?

A blind edit is a Write or Edit with no preceding Read of the same file in that session — the agent changed the file from memory rather than its current state. High blind-edit counts (visible via h5i recall notes coverage) are a strong signal a change deserves a closer look.

Is a language model used to do the audit?

No. The ranking and prompt-injection scans are deterministic — derived from recorded session signals — so there's no model in the audit path and results are reproducible.

Can I produce a report for compliance or an external auditor?

Yes. h5i audit compliance --since/--until --format html|json|text generates a date-ranged report of AI-assisted activity. Combine it with h5i share pr post to leave an auditable trail on the pull request itself.

Can I enforce review rules automatically?

h5i audit policy manages .h5i/policy.toml, where you declare rules such as blocking on credential leaks or requiring an audit on authentication changes, so the gates apply at commit time.