From Patch Generation to Evidence: Ela Avrahami on Trusting AI-Written Code ([un]prompted II)

A patch generator is a demo. A patch generator somebody merges is a trust architecture. Ela Avrahami has the [un]prompted II talk — people type it unprompted — about the second half of that sentence, and it is the quietest genuinely important slot in our guide to the conference. The disputable claim: the model's reasoning is never the evidence. It is the thing evidence has to be collected about.

Avrahami is listed on the agenda as an AI & Software Advisor, with no company given, and speaks on Day 3, Stage 2, at 15:30, per the official agenda as published on 2026-10-12. This brief is written from that published abstract, before the talk. We are independent coverage — not the organiser, not a sponsor, not a media partner — and we are following remotely. Nothing here is a quote, and the agenda can change.

What the talk says it will show

The abstract describes the trust architecture behind the OpenAnt Auto Patcher: which evidence helped, which ideas failed, and how controlled experiments refined the line between LLM reasoning and deterministic policy.

Three phrases there are rare enough to name. "Which ideas failed" is a negative result, volunteered. And "the line between LLM reasoning and deterministic policy" is the architectural decision most of this field is still deferring.

The boundary is the real subject

Every agentic patching system answers one question twice: what may the model decide, and what must policy decide. Lean too far towards the model and your reviewer is reading a rationale the model wrote about its own work. Lean too far towards policy and the system only fixes bugs that already had a rule, which is a linter with a larger bill.

Two other sessions put that line in different places.

Session	Where the line falls
Tim de Waal (Member of Technical Staff, OpenAI), "Prompt, Patch, Prove", Day 1, Stage 1, 13:30	An agentic coding model patched two published CVEs on an authorized ASUS router. The abstract commits to what the model got wrong and to the verification gates that caught it. The gates are deterministic; the repair is not.
Gabe Bello (Twilio), "Building Security Review Agents: the Models are the Reliable Part", Day 3, Stage 2, 14:45	Several models assess third-party app permission grants, a stronger model referees disagreements, and a human decides. Models assess; policy and a person adjudicate. The title says which part surprised them.
Ela Avrahami, Day 3, Stage 2, 15:30	Controlled experiments on where the line should sit, rather than one team's taste. Nobody else on the programme claims to have measured it.

Why this is the hard part of the unprompted programme

Evidence is not a technical property. It is whatever a specific reviewer will accept before putting their name on a merge, and that bar moves with the reviewer, the blast radius and the hour of the day.

The agenda prices the consequence. Tobias Heldt (Co-Founder and CEO, XOR) reports frontier agents producing 12 working exploits and 0 accepted patches from 54 attempts on his company's own privately disclosed zero-days — including a fix that also blocked valid messages. David Molnar (Meta) reports over 40% of critical vulnerabilities fixed by AI-written diffs and no automatic drop in time-to-fix. One result is a trust failure, the other a throughput failure, and both are downstream of the same missing artifact.

We made the oversight argument in human oversight in production agentic systems, and heard the practitioner version from Dan Guido on Trail of Bits going AI-native: review is where AI-assisted security work compounds or stalls. The full speaker list has more sessions on the same seam.

What we will be watching for

• The evidence bundle, itemised. Reproduction, a failing-then-passing check, a diff rationale, provenance — and which of those a reviewer actually read.
• Which ideas failed. A rejected evidence type is worth more than a working one, because everyone else is about to build it.
• How the controlled experiments were set up: what was held fixed, and what the reviewers were blind to.
• Whether the deterministic side can veto, or only annotate. A policy that cannot block is documentation.

If this holds, what changes on Monday

Write down your own boundary before you buy a patching tool. One column for decisions the model may make, one for decisions a deterministic gate must make, and a named owner for the second column. Most teams never write the second column down.

Then define the merge artifact. If a patch arrives without a check that failed before it and passes after it, reject it rather than reading the rationale harder.

Three questions we would ask Ela Avrahami

1. Which single piece of evidence changed a reviewer's decision most often — and was it something the model produced, or something the pipeline produced about the model?
2. Which evidence ideas failed, and did they fail because reviewers ignored them or because they were wrong?
3. Where did the deterministic line finally land, and which decision were you most surprised to find the model could not be trusted with?