What Teams Had to Fix Before They Could Automate

The processes these teams could not automate were not short of tools. They were short of a written version of themselves. The rules sat in one person's head, a key word had never been defined, or nobody owned the step where someone decides what is true. Several contributors found that out by automating anyway, and the automated version did not break. It ran cleanly and gave wrong answers that nobody questioned.

Pattern 1: The rules lived in one person's head

The most common account was a process that worked because one experienced person quietly corrected it. Ace Zhuo, CEO of TradingFXVPS, connected billing straight to server provisioning and learned what his senior support engineer had been doing by hand:

In reality though, all of this was managed by one senior support guy, holding in his head over a dozen or so unique exceptions to normal setup.

Within a week, new servers were landing in the wrong region for the customer's broker. "A trader 80ms from their brokers' servers instead of 2ms ends up with a refund and a really annoyed customer." He spent two days with the engineer writing every exception down, cut the plan catalog to a short clean list, and made broker location a required field at checkout. His rule now: "If a rule only exists in one person's head then your automation will skip it every single time."

Jimi Patel, co-founder of SellerQI, hit the same wall with an analyst's judgement. Diagnosing a sales drop on Amazon meant reasoning across three reports that each identify a product differently, and the analyst carried the mapping between them from memory:

When we first tried to automate the diagnosis, the logic was fine but the answers were wrong, because the same product looked like three different products across three reports. A stockout on one SKU would hide behind healthy inventory on another SKU for the same ASIN.

The fix was a single product mapping table, rebuilt daily, before any diagnosis rule was written. His lesson: "before automating a judgement process, find the reference data the expert keeps in their head. That is usually the real project, and the tool is the easy part."

Sometimes the missing knowledge is a single step. Richard Meadows, Head of Content at Streamrise, has AI agents draft, review and deploy articles:

What was broken was a step nobody had written down. Publishing a page on our site takes four files, not one: the article content, a small page file that loads it, and entries in two registry files. The written brief for the agent listed only three of the four.

The review passed and the build reported success. The page returned "not found", one 404 among 413 pages in the deploy log. His team caught it only because the routine ends by checking the live URL. "Done" now means the live page loads, not that the build succeeded.

Pattern 2: A word nobody had defined

A second group had the steps written down but not the terms. A person reading "find the price" or "a mention of" fills in the meaning without noticing. A script cannot.

Edgars Lapins, founder of TrustedAIAgencies, wanted a script to record the starting price each of about 280 agencies publishes:

As it stood, the process was too vague to automate. "Find the price" sounds simple. But nobody had written down what counts as a price. When we ran the scraper on 31 October, it pulled 755 price lines from 119 sites. A lot of them were not offers at all.

The lines included salary tables, ChatGPT seat prices and budget dropdowns on contact forms, and 121 rows were dropped on review. The fix was a written rule for what a starting price is, a fixed list of price types, and a keep-or-drop decision on every row.

Ed Brancheau, founder of Elevyr, was scoring care homes for his own sales list. The plan said a mention of Medi-Cal, California's Medicaid program, should lower a home's score. It never said what counts as a mention:

A Node script read each home's site and rejected the home outright if it found the term, with the hyphen optional. Take the hyphen out of Medi-Cal and you get "medical." Almost every care home website says "medical."

The script threw out 33 homes. At least 24 were false alarms, dropped for phrases like "medical needs" and "medical care."

Nothing looked wrong, because the mistake sat in what the script discarded. "People check what an automation sends. Rejects never complain."

Aziz Chaabane, founder of Groundwork, tried to write his own editorial judgement down as rules for choosing what to publish next. He tested the rules before switching them on:

Before switching it on, we tested those rules against 355 topics I had already approved by hand. They failed. The word list would have thrown out 158 of them, 45%, including one with 33,100 searches a month.

"The rules described what I thought I did, not what I did." He deleted the hand-written filters. The agent now ranks the full list on measured data and he makes the pick. His advice to anyone automating a judgement call: "test the written rules against decisions you have already made. If they disagree, the rules are wrong, not the decisions."

Pattern 3: Nobody owned the truth

The third group had a gap in ownership. The automation did what it was told, using a record that nobody was responsible for keeping right.

Dhwani Shah, Assistant Manager Human Resources at Qubit Capital, automated interview scheduling from a shared sheet:

A candidate once sat alone on a video call for 20 minutes because our interview scheduling had invited her and nobody else. I had automated it for our recruitment team a month earlier, a Google Form feeding a Google Sheet that Zapier read to send invites. It did exactly what it was told. Hiring managers rescheduled in Discord messages to whichever recruiter they caught first, which the sheet never saw.

The team switched it off for six weeks and gave each open role one recruiter who owns every reschedule. "You could call it bad data, though with 60 of us working remotely it felt more like nobody owning the row."

Zac Hunter, CEO of Thelemata, handed project tracking to an AI agent and found the gap on the usage bill:

What was broken: the job had no edges. "Keep the tracker current" sounded like data entry, but the real status of a project lived in email threads, Slack messages and files. Nobody owned the step of deciding what was actually true before it went into the master tracker. So when we handed tracking to the agent, it inherited that judgment work too.

Almost all of the agent's cost came from reasoning its way to a status, not from logging one. The team split the work: an audit step settles what is true, and only then does the agent log it.

Jens Hagel, founder and owner of kjello, connected seven of his IT firm's systems so its key figures would build themselves, and found the figures did not agree with each other:

The data wasn't ready: the order backlog showed 17 percent too high because bundle items counted twice, the ERP's contribution margin sat a factor of two away from gross profit, and 54 Microsoft accounts for shared mailboxes and services threw staff cost per head off by more than double. Once we corrected that, 95 of 109 open orders matched to the cent. I'd long thought we had a reporting problem, and it turned out to be a connection problem.

The through-line

In these accounts the manual process had one advantage: it was visibly unreliable, so people checked it. Luke Sandelands, founder of Stack Architect, built a monthly profit report for a Shopify store where nobody could say what anything cost:

I found this out by building the automation anyway. It worked. Every week it produced a clean, confident report that was badly wrong on any product whose cost had changed during the year, and nothing in it looked wrong. The manual version at least made people uneasy. The automated one was quiet, fast and authoritative enough that the owner would have priced against it.

Fixing it took two weeks of data entry, giving each product a cost with a date attached. The build took two days once the data was right. His warning applies to every account in this report:

What I would tell anyone scoping an automation project is that broken manual processes are loud. Somebody is always complaining about them, which is how they get on the roadmap in the first place. Automate one without fixing it and the complaints stop while the errors continue, which is a far worse place to be than where you started.

The repairs these contributors describe were small and unglamorous: write down every exception, define the terms a script will have to apply, name one owner for the record, and test the written rules against decisions already made.