--- title: "From Patch Generation to Evidence: Ela Avrahami on Trusting AI-Written Code ([un]prompted II)" description: "Generating a patch is solved. Getting one merged is not. Ela Avrahami's Day 3 talk on the OpenAnt Auto Patcher is about the evidence that moves a reviewer, and the line between what an LLM may decide and what deterministic policy must." author: "Mara Voss" category: "AI Security Conferences" date: 2026-10-20T05:40:04.714Z canonical: "https://agenticcyber.co/blog/unprompted-ii-ela-avrahami-patch-evidence" --- # From Patch Generation to Evidence: Ela Avrahami on Trusting AI-Written Code ([un]prompted II) ![Photograph illustrating calibrating precision instrument.](https://hsppuvezyxmkpzkgfkho.supabase.co/storage/v1/object/public/media/conference/5bc07ae2-9ee0-46b9-9820-b0704936f742/unprompted-ii-ela-avrahami-patch-evidence.jpg) > Generating a patch is solved. Getting one merged is not. Ela Avrahami's Day 3 talk on the OpenAnt Auto Patcher is about the evidence that moves a reviewer, and the line between what an LLM may decide and what deterministic policy must. A patch generator is a demo. A patch generator somebody merges is a trust architecture. Ela Avrahami has the **[un]prompted II** talk — people type it *unprompted* — about the second half of that sentence, and it is the quietest genuinely important slot in our [guide to the conference](/blog/unprompted-ii-conference-guide). The disputable claim: the model's reasoning is never the evidence. It is the thing evidence has to be collected *about*. Avrahami is listed on the agenda as an AI & Software Advisor, with no company given, and speaks on **Day 3, Stage 2, at 15:30**, per the [official agenda as published on 2026-10-12](https://unpromptedcon.org/agenda-10-2026/). This brief is written from that published abstract, before the talk. We are independent coverage — not the organiser, not a sponsor, not a media partner — and we are following remotely. Nothing here is a quote, and the agenda can change. ## What the talk says it will show The abstract describes the trust architecture behind the **OpenAnt Auto Patcher**: which evidence helped, which ideas failed, and how controlled experiments refined the line between LLM reasoning and deterministic policy. Three phrases there are rare enough to name. "Which ideas failed" is a negative result, volunteered. And "the line between LLM reasoning and deterministic policy" is the architectural decision most of this field is still deferring. ## The boundary is the real subject Every agentic patching system answers one question twice: what may the model decide, and what must policy decide. Lean too far towards the model and your reviewer is reading a rationale the model wrote about its own work. Lean too far towards policy and the system only fixes bugs that already had a rule, which is a linter with a larger bill. Two other sessions put that line in different places. SessionWhere the line falls Tim de Waal (Member of Technical Staff, OpenAI), "Prompt, Patch, Prove", Day 1, Stage 1, 13:30An agentic coding model patched two published CVEs on an authorized ASUS router. The abstract commits to what the model got wrong and to the verification gates that caught it. The gates are deterministic; the repair is not. Gabe Bello (Twilio), "Building Security Review Agents: the Models are the Reliable Part", Day 3, Stage 2, 14:45Several models assess third-party app permission grants, a stronger model referees disagreements, and a human decides. Models assess; policy and a person adjudicate. The title says which part surprised them. Ela Avrahami, Day 3, Stage 2, 15:30Controlled experiments on where the line should sit, rather than one team's taste. Nobody else on the programme claims to have measured it. ## Why this is the hard part of the unprompted programme Evidence is not a technical property. It is whatever a specific reviewer will accept before putting their name on a merge, and that bar moves with the reviewer, the blast radius and the hour of the day. The agenda prices the consequence. Tobias Heldt (Co-Founder and CEO, XOR) reports frontier agents producing **12 working exploits and 0 accepted patches** from 54 attempts on his company's own privately disclosed zero-days — including a fix that also blocked valid messages. David Molnar (Meta) reports over 40% of critical vulnerabilities fixed by AI-written diffs and no automatic drop in time-to-fix. One result is a trust failure, the other a throughput failure, and both are downstream of the same missing artifact. We made the oversight argument in [human oversight in production agentic systems](/blog/human-oversight-agentic-systems-governance-production), and heard the practitioner version from Dan Guido on [Trail of Bits going AI-native](/blog/inside-trail-of-bits-ai-native-transformation-dan-guido-on-building-a-security-f-6niv): review is where AI-assisted security work compounds or stalls. The [full speaker list](/blog/unprompted-ii-speakers-list) has more sessions on the same seam. ## What we will be watching for - The evidence bundle, itemised. Reproduction, a failing-then-passing check, a diff rationale, provenance — and which of those a reviewer actually read. - Which ideas failed. A rejected evidence type is worth more than a working one, because everyone else is about to build it. - How the controlled experiments were set up: what was held fixed, and what the reviewers were blind to. - Whether the deterministic side can veto, or only annotate. A policy that cannot block is documentation. ## If this holds, what changes on Monday Write down your own boundary before you buy a patching tool. One column for decisions the model may make, one for decisions a deterministic gate must make, and a named owner for the second column. Most teams never write the second column down. Then define the merge artifact. If a patch arrives without a check that failed before it and passes after it, reject it rather than reading the rationale harder. ## Three questions we would ask Ela Avrahami - Which single piece of evidence changed a reviewer's decision most often — and was it something the model produced, or something the pipeline produced about the model? - Which evidence ideas failed, and did they fail because reviewers ignored them or because they were wrong? - Where did the deterministic line finally land, and which decision were you most surprised to find the model could not be trusted with? ## FAQ ### What counts as evidence for an AI-written patch? Not the model's explanation. The useful set is reproduction of the original finding, a check that fails before the patch and passes after it, the provenance of the change, and a record of which deterministic gates ran. Ela Avrahami's [un]prompted II abstract promises which of those helped and which ideas failed. ### Where should the line between model reasoning and deterministic policy sit? The agenda shows no consensus, which is the point. Tim de Waal (OpenAI) reports verification gates that caught the model's errors on firmware CVEs. Gabe Bello (Twilio) has models assess and a human decide. Avrahami's abstract is the only one claiming controlled experiments on where the line belongs. ### Is this coverage from inside the conference? No. We are independent coverage - not the organiser, not a sponsor, not a media partner - and we are following remotely. Everything here comes from the official [un]prompted II agenda as published on 2026-10-12, plus our own analysis. The talk has not happened. --- Source: https://agenticcyber.co/blog/unprompted-ii-ela-avrahami-patch-evidence