← Vociferous AI · Research · Law
Legal AI, where a wrong citation is the whole problem
Three strands
This page describes a capability and how I approach it. It names no client and no matter, and where work is still in progress it is described as a capability rather than a finished result.
Research. I audited a widely used legal-reasoning benchmark — a full re-derivation of the gold labels for every item in its rule-governed tasks, from primary authority, with the published answer key sealed off from the derivation path. The result is largely a confirmation, which is the harder outcome to publish and the more useful one to have. It also locates precisely where deterministic validation stops and trained legal judgment has to take over. The paper is under peer review; title, venue, and full findings stay private until the decision lands.
Build. Source-grounded review systems for legal teams, and private model training for work that cannot go to a commercial API.
Enterprise delivery for law firms. From 2013 to 2021 I was the long-term consulting lead at Sia/Addison, supporting complex digital, technical, and security-sensitive engagements for leading law firms, alongside Fortune 50 companies and major financial institutions. Architecture, application development, technical strategy, and cybersecurity-related work, with senior stakeholders, across multiple years of repeat engagements. Client names and matters stay confidential — which is the same constraint that shapes everything else on this page.
Why legal AI fails, and it is not the model
Legal work is document-heavy and bound by confidentiality. Three constraints bind at once, and a general chatbot fails all three.
The answer has to be tied to an authority a person can open and check. The sensitive material often cannot be sent to a commercial model provider at all. And the cost of a confident wrong answer is not a bad user experience — it is a sanctionable filing, a missed limitation period, or privileged material in someone else's training set.
So the hard parts of legal AI are confidentiality, traceability, evaluation, and a clear human-in-the-loop boundary. Get those right and the technology becomes trustworthy enough to actually use on a real matter. Get them wrong and no amount of model quality rescues it.
What I build
Source-grounded review and citation finding. The first useful version helps a legal team move faster without giving up control: AI-assisted citation finding, authority recommendations, and document review where every answer points back to the document or authority it came from. The system finds, organizes, and grounds. The lawyer decides.
Private model training, on your own terms. When the work requires a model trained on legal documents rather than calls to a public API, I take that end to end — supervised training and fine-tuning, evaluation against precision, recall, and F1 benchmarks, and delivery of the trained weights. Smaller models with intelligent routing keep inference affordable, and the pipeline can be built from open components with no proprietary lock-in.
Privacy by deployment, not by promise. For privileged or confidential material, privacy has to be structural. The pipeline can run self-hosted, on-premises, or fully isolated, so training data and client documents never leave an environment the client controls and are never sent to a third-party model provider. The guarantee lives in where the system runs, not in a vendor's policy.
What the benchmark audit taught me that applies to your evaluation
A confirmation is a result. Most audit work is published only when it finds something damaging. That selection pressure is itself a problem: it means the field has almost no record of which benchmarks are sound. Re-deriving every label and reporting that the key largely holds is less dramatic and more useful to anyone deciding whether to rely on it.
The boundary is the finding. The most valuable output was not a defect count. It was the precise line where a rule can be applied deterministically and where it stops abbreviating doctrine and requires a licensed attorney. An evaluation that does not state that boundary is implicitly claiming there isn't one.
Additional model readers are not a substitute for a lawyer. They can map where disagreement lives and how stable it is. They cannot adjudicate it. Expert review is not a compliance step appended to the end — it is the mechanism by which the work's ceiling moves.
The full method is on the Research page.
Public-interest fit
This matters most where the stakes are high and the budgets are thin: civil legal aid, nonprofits, and access-to-justice organizations that hold sensitive client information and can afford neither a data leak nor a hallucinated citation. The same patterns that protect a firm's privileged files protect a legal-aid client's, and the cost structure that makes self-hosted small models viable matters far more to the second group than the first.
Where AI helps, and where it must not
AI here is an assistant to judgment, not a substitute for it. It does not give legal advice and it does not decide. It makes the record searchable, keeps answers tied to sources, and surfaces what a person should check — so an attorney's time goes to the judgment only a person can make.
If this is your work
If you run a legal or public-interest team weighing AI — where confidentiality and source-grounding are non-negotiable — or you are about to rely on a legal benchmark nobody has checked, send the workflow.