How well it reads an Office Action
The record reader against public USPTO file histories: the rejections each Office Action states, the references each applies, and the reasons a Notice of Allowance gives, scored against the same papers read by hand.
Read this first
This is a small study: 2 file histories. Both applications are public and named below so you can check them. Nothing on this page is a recommendation about any application, and none of it is a legal determination: what a record means for a particular matter is the practitioner's reading.
Reading an Office Action: a first check on two file histories
The app reads a Patent Center "download all documents" file, scanned pages included, and reports each rejection an Office Action states and the reasons a Notice of Allowance gives, with the quotation each came from. (Since version 0.6.5 it also reports objections, requirements and statements of claim status; this study predates them and does not measure them.) We scored that reading against the record, read by hand, on the file histories of 18/541,216 and 19/180,870: their Office Actions and their Notice of Allowance. One rejection is one of the examiner's own headings: a ground, the claims it covers and the references it applies, as the Office Action groups them.
| Model | Run | Rejections found | References found | References named that the paper does not contain | Other citations listed as references | Examiner's stated reasons found |
|---|---|---|---|---|---|---|
Claude Sonnet 5.5 | remote | 10 of 10 | 13 of 13 | 0 | 0 | 2 of 2 |
kimi-k2.5 | remote | 10 of 10 | 13 of 13 | 0 | 0 | 2 of 2 |
qwen3.5:9b | on a laptop | 10 of 10 | 12 of 13 | 0 | 0 | 2 of 2 |
qwen3.5:9b | through OpenRouter | 9 of 10 | 13 of 13 | 0 | 3 | did not finish |
A 9-billion-parameter model running on a laptop found every rejection, which is the configuration that keeps a file on the machine. The same model reached through OpenRouter did worse on this run and some of its calls ran to the time limit: a result for that route rather than for the model, since OpenRouter chooses the host and hosts differ in how they serve it.
No model named a reference the paper does not contain, so the app's own check, which
withholds any reference it cannot find in the paper, had nothing to withhold.
The qwen3.5:9b run through OpenRouter listed 3 citations that are not
references of record, a statute and a court decision among them; they are in the paper, so
that check passed them, and the app now withholds statutes and case law as a separate step,
a change made because of this run.
Scored against the record as read by hand; a patent practitioner's check of that reading is under way, and this table will change if it does. One entry is open: the Final Rejection in the second file states its obviousness rejection over two references, then applies a third to later claims, and the hand reading lists all three. Read as two, the local model's one missed reference disappears; the rejection counts are the same either way. The last column counts the reasons a Notice of Allowance states, where it states them; the reasons for allowance are not always written out.
Drafting the reply is measured separately, against the replies practitioners filed: drafted replies, measured.
What this does not measure: how much time the reading saves, how it does on a file whose scans are poor, or how it does across the range of art units and Office Action styles. Two files is a first check, not a survey. One of the readings is printed unedited, quotations and all.
The figures on this page are published under CC BY 4.0: reuse them, with credit to Obviously Not.
The papers, the action, and the round before
Three more checks of what 0.9.0 reads before a reply is drafted.
- Each paper on its own. Read with the Office's label hidden and compared with it, the kind of paper was right for 344 of 348 papers, finality for 121 of 121 Office Actions, and the date for 107 of 108 papers that carry their own.
- The claim table. On file histories it was not tuned on, each claim and the items the action states against it agreed with the majority of three AI judges on 293 of 302 claim-and-ground pairs.
- The round before, compared. Over 20 pairs of consecutive Office Actions, the maintained, modified, new and withdrawn labels agreed with the same judges' majority on 166 of 171.
The judges are AI models from three vendors; no practitioner labeled anything. The labels are facts about the papers, and none is a view of whether a rejection holds.
Two more measurements, about continuations
The same study measured two things about drafting continuations from these records: what real continuations left out of their parents' allowed claims, split by whether a limitation was filed with the application or added during prosecution, and where the app looks when it proposes drafting directions, including a finding against the app. Both are published with their method on patentcontinuation.com, where the drafting side of the app is described.