Living Software: How Adjudicate Watches and Fixes Itself

A partner at the law firm that runs our software told me last year that his team’s bugs tend to be fixed before anyone at the firm has noticed them. He meant the remark as a compliment. I treat it as a requirement.

Enterprise software usually ages the same way. A firm buys the system, small things break, tickets get filed, and the waiting begins. The vendor’s roadmap and the customer’s reality drift apart, until the system everyone depends on has become the system everyone works around.

Adjudicate, case management and decision-writing software for document-heavy law, has resisted that pattern for eight years. The product has processed more than 20,000 appeals end to end for a veterans law practice with more than sixty reviewers. Ari, our CEO, still writes the code. He engineered the software to observe itself in production and hand issues and bugs to AI agents that repair it.

We call the result living software: it fixes most bugs before users notice them. Four systems make the process work.

The Instrument Panel

The Product Knows When It Is Failing

A reviewer tells us the queue felt broken yesterday. In most support relationships, that sentence starts an argument about what actually happened. Here nobody has to argue, because yesterday is already in the data. Commercial aviation stopped relying on pilots’ memories once the flight recorder became standard equipment, and Adjudicate records on the same principle.

Every deployment streams everything a user does, unsampled, into one telemetry pipeline: every click, every data call with its latency and error type, every stall of the main thread longer than 200 milliseconds. When a user closes the tab, the session writes its own summary. The pipe also counts rage clicks, four or more taps on an unresponsive control inside a second: as close as software comes to measuring frustration directly.

The product handles legal records, so the recorder never records content: events carry identifiers, shapes, and numbers. No names, no typed input, no rendered text. Nothing from a case file can reach a log. Every event passes through one point in the code, and that point holds a banned-key list and a length limit — so a leak fails the build instead of living on as a note in a policy doc.

It never slows the user. Nothing on the reviewer’s path waits on the network: events accumulate in memory and dispatch in the background every five seconds, and the telemetry client loads only after the first paint, when the browser is idle. Recording everything costs the reviewer nothing they can feel, and a heavy session costs about two cents.

Every event records the exact version of the software that produced it, so when something breaks, the data already shows which release did it. The moment error thresholds trip in production, the product pages the team over an alert path that is wired and tested end to end — and only a real deployment can trigger it. A single request identifier ties a user’s telemetry, the network’s access log, and the backend’s stack trace together, so one fault shows up as one story instead of three separate logs to line up by hand.

The instrument panel closes the gap most support contracts quietly bill for: the weeks between “something is wrong” and “we know what happened.”

The Repair Loop

Found by the Software, Fixed by Agents, Proven by Machines

AI agents browse the live product in each user role, looking for dead ends before a human hits one. They work on synthetic accounts and see no client case files, and what they find lands in the same pipeline, tagged so it never pages a person. The telemetry adds errors with their stack traces, rage clicks with their targets, and slow calls with their timings. Any user can send a one-tap note that arrives with its context attached: the screen, the role, the last dozen routes visited, the build, and an optional screenshot. And a standing internal audit looks for anything that merely appears to work, because the most dangerous bugs look fine.

Toyota’s production system gave manufacturing the andon cord, the pull that lets any worker stop the line over a defect so the line can learn from it. Adjudicate’s cord is pulled by the software itself.

Ari wrote the triage contract for AI agents, not a support desk. An agent reads the report, reconstructs the session from the telemetry, locates the offending code, and writes a candidate fix.

The gate makes this safe: no change, human or agent, lands on its own authority. Each candidate enters a merge queue, where a dedicated machine rebases the change onto the live codebase, rebuilds the project as it would exist after the merge, and runs the type checks, the test suite, and a real browser that loads every screen and catches what simulated browsers miss: blank routes, white flashes, a command palette that fails to reopen. The fix lands only if everything passes and the codebase has not moved in the meantime. The team’s rule is to repair the failure, never to route around it. The gate also answers the question every buyer should ask about a team this small: the safety of a change does not live in one engineer’s head. The gate judges Ari’s commits exactly as it judges an agent’s.

In June, the team pointed dozens of agents at every screen of the product simultaneously. Each finding then went to a second agent with the opposite job: read the cited code and prove the finding wrong. Ninety findings entered that cross-examination and eighty-nine survived it. Ten shipped to production as immediate fixes; the rest went onto a tracked backlog, logged as lower-severity items rather than dropped. No customer filed a ticket at any point.

The same machinery enforces qualities that never appear in a bug report. Contract tests in the build ban a button that only scrolls, forbid a preview panel from modifying data, and require every clickable element to be reachable by keyboard, so accessibility is tested on every merge rather than audited once a year. People still review every change and decide what ships; what they no longer do is the waiting, the rebasing, and the regression-proofing.

The Document Engine

Four Thousand Pages, Zero Blanks

A single appeal’s record holds hundreds of documents, and some of those documents are more than a hundred pages long: decades of medical history, hearing transcripts, and forms scanned on machines long out of production. The deepest file in a recent sweep contained 4,658 pages. A reviewer spends the working day inside that file, and the worst failure a document viewer can commit is to display a blank page where evidence belongs and say nothing about it.

Ari rewrote the viewer’s rendering engine to close off that failure. He rewrote the hardest image codecs in the PDF standard — the compression formats government scanners have used since the 1990s — in Rust, then compiled them to WebAssembly. The new engine ships only if it beats the old one on two counts: pixel-for-pixel parity on a library of reference pages, and speed. If it renders a page slower, the build fails.

Then he went after blank pages directly. A test tool opens complete case files, draws every page as a reviewer’s screen would, and flags any page that comes up empty. One sweep checked ninety thousand pages and found failures nobody had reported. The worst was a corrupt page that could freeze the viewer for four and a half hours. For an attorney working inside that file, a freeze like that is not a rendering defect. It is a lost afternoon. The fix came out of an annex of the PDF specification, an end-of-stream rule most implementations skip. The sweep also caught the opposite mistake: a legitimately blank scanned page that an overcautious safety check kept rejecting as an error. The page was innocent, the check was wrong, and both got fixed.

The 4,658-page file now displays every one of its pages. When a document genuinely cannot be served, the viewer says so plainly, never substitutes a wrong record, and files its own report into the telemetry.

The AI Layer

The AI Proposes, a Person Decides

The agents that repair Adjudicate work behind the scenes, and reviewers never see them. The AI inside the product is different. Reviewers use it all day, so the rules are stricter. It runs in thirty-seven places across the product, and every one follows the same rule: the AI proposes, and a person decides. A model may rank a queue, flag the pages that matter to an issue, or draft the language a decision needs. It may not act. Every suggestion arrives as a proposal that a person accepts, edits, or rejects, and the outcome is stamped into the record together with the proposal that prompted it.

The product watches its AI features the way it watches everything else. A feature stays in the product only while it keeps passing a standing test: we run it on real cases with known-correct answers, re-run it with the same inputs to catch drift, and pull it the moment it stops beating a simple rule-based baseline. This is living software applied to its own AI.

When a model consults public case law, the request sends out the legal citation and nothing from the case file. Case content cannot leave the firm’s boundary, and the block is enforced in the network, not by a policy.

What This Means

What to Ask Before You Buy Software

The traditional model prices software as a construction project and treats everything after launch as decay to be managed: twenty people to build the system, a help desk to defend it. Adjudicate treats the product as ongoing work: instrumented so failures cannot stay silent, and repaired by agents as fast as problems are found.

A small senior team has sustained a system a law practice bets its casework on, and eight years of production is the proof. The watching, the triaging, the regression-testing, and the staging that once justified the twenty people is now the machines’ work. People keep the judgment: deciding what the product should do next, and reviewing what ships.

The running costs are on the record in this study: recording everything a user does costs about two cents for a heavy session.

I tell buyers that the demo is the least informative part of any product, because anyone can make software look right for twenty minutes. The quieter questions decide the next five years: whether the product knows when it is failing, and who fixes it, how fast, and with what proof. This product knows, through its instrument panel; agents fix it within days, with the merge gate’s evidence attached. And if you would rather hear all of this from a customer than from a vendor, the firm quoted at the top of this study has run its casework on Adjudicate for eight years. Ask them.