Mastodon

TechLex

The automation worked perfectly, until one day it didn't

the-automation-worked-perfectly-until-one-day-it-didnt A partner at a small firm — four lawyers and a paralegal — hired an automation expert to build an intake workflow: a new enquiry form, an automatic conflict check, a matter opened in the practice system, an engagement letter sent. He showed it to the team on a Monday morning, and it worked perfectly.

Nine days later, a prospective client rang to ask why nobody had been in touch. The form had been submitted, but the conflict check had not run, the matter had not opened, and the engagement letter had never gone out. The practice system had quietly changed its API and renamed a field the workflow depended on — something engineers call schema drift. The workflow kept running and kept returning a success status on every attempt, while processing nothing.

The same failure plays out in other forms: a document template that switches clause variants after a library update, an NDA routing rule that stops firing when a new email domain appears, a billing sync that posts nothing for a week behind a healthy green status light. Automation fails quietly, and the first signal is usually a confused client or a gap in the accounts.

The visibility problem nobody sells you

The promise of automation is efficiency. The risk that never appears in the demo is the loss of visibility — being able to tell, at any moment, whether the system is doing what it should.

A paralegal doing intake by hand notices when something is off. Volume drops, replies slow, a call goes unreturned. That awareness comes with the person. Automated systems carry out instructions and stop, and from the outside, stopping because the work is done looks identical to stopping because something broke.

Well-built automation plans for this: if a step cannot complete, it retries, sends an alert, then routes the matter to manual review. Much of what gets built today does not include any of that. It runs, it stops succeeding, and the first to find out is the client.

So before asking what you can automate, ask who will notice when it breaks, and how quickly.

Where automation earns its place

Not everything carries the same risk. Automation delivers where the process is genuinely repetitive, the inputs are structured, and a brief failure can be recovered.

Client intake sits at the top of that list. Routing enquiries, triggering conflict checks, opening matters, sending acknowledgements — done dozens of times a week in a consistent form. The failure shows itself quickly, because a client follows up, and the fix is fast. A benchmark that ran eleven firms through their own intake, posing as a real client, scored them from 17 to 90 out of 100 and found almost none clean the whole way; the worst were a form that silently broke and enquiries that drew no reply.

NDA and signing workflows fit for the same reasons: the process is defined, the parties are known, the documents are standard, and a human reviews before anything is signed. Document assembly for routine instruments — engagement letters, standard NDAs, terms of business — is the most mature use in practice: the tools are reliable, the gains are real, and a lawyer still reviews before anything leaves the building.

These all sit at the edge of the firm's work, not at its centre. They are the administrative envelope around legal practice rather than the practice itself.

Where automation creates more risk than it removes

Contract review, legal research and billing analysis are pitched for automation most often. They are also where a quiet error costs the most and the checking takes the longest.

Automating contract review does not remove the partner's liability for the result. It moves time from reading to checking, and checking what the model flagged still takes enough grasp of the document to catch what it missed. For a large team with a legal operations function, that can pay off. For a small firm, the senior lawyer often spends hours verifying the output.

Research carries the same trap. An AI that finds relevant authority quickly is useful. When it confidently returns the wrong authority, cleanly formatted and convincingly cited, AI becomes a liability.

Sometimes the right answer is not to automate

There is a category of process where the human in the loop is the part that matters most. Conflict checks, onboarding conversations, the decision whether to open a matter, billing review: a brief slowdown here is exactly where something gets caught. The paralegal who pauses on an odd intake form and asks a question is doing something no rule can.

For work where the firm's reputation rides on each instance, a well-run manual process is a deliberate choice, worth making on purpose rather than by default.

The KPI that actually matters

Automation is sold on time saved. But a firm's reputation rides on every matter, not on the average — and one missed engagement letter that sends a client to a competitor is not offset by the forty enquiries that routed correctly. One missed clause that later becomes a dispute is not cancelled out by the nine that were caught.

A good question to ask your automation expert: what can currently go wrong that this automation would prevent?

Before you build anything

The automations that last begin with someone walking the process by hand, many times, before a single rule is written — not once in a test environment, but in real conditions, with real inputs, until every edge case has been met and written down.

The assumptions that break automation are invisible until you do the work yourself. The intake form that sometimes arrives without a phone number. The conflict check that behaves differently for a returning client. The matter type that needs a field no template allows for. Those cases are the process itself, and an automation that skips them automates a simplified version — the gap between the two is where the failures live.

Find the process that is genuinely repetitive, structured, and recoverable. Walk it by hand until you know exactly what it contains. Then decide whether to automate it. The answer may still be yes — and when it is, it will be a better yes.