PRAPI Research · 2026-11-19
What the First Weeks of an AI Rollout Found That the Scoping Call Missed
We asked people who have put an AI system into a working team what the first weeks turned up that the sales or scoping call had missed. The most common find was a spreadsheet: one experienced person kept it, the official system did not contain it, and the real decisions ran through it. Others found records that were wrong, logins nobody held, and work done by someone the org chart did not name. Plans slipped by weeks, and these contributors now ask to see the file before they agree a timeline.
12 contributors cited
The scoping call describes the process a team believes it has. The people who answered this brief found the one it actually runs, usually within two weeks and usually by watching someone work. The most common find was a spreadsheet that one experienced person kept and nobody had mentioned. Others found records that were wrong, logins nobody held, and work done by someone the org chart did not name.
Pattern 1: The process ran on a spreadsheet nobody mentioned
Bogdan Nicholas, co-founder of VINspectorAI, deployed an AI vehicle inspection system with a used car dealership group. The scoping calls described photos going in and condition reports coming out. Week two showed something else:
Turns out the actual grading decisions lived in a single Excel file maintained by one senior appraiser. He'd built his own depreciation logic over eight years. Nobody mentioned it because nobody else fully understood it.
His team found the file by shadowing the appraiser and noticing that his assessments kept diverging from the documented criteria. Rebuilding his logic into the model pushed the timeline back three weeks. "Now I always ask: who on your team would you trust over any written policy? That person holds the real process."
Panayot Kalinov, senior software developer at Casinoreviews.net, was adding an AI layer to help editors check licence status and bonus terms against operator pages and regulator databases. In the first week he noticed that nobody opened the regulator sites first:
They opened a shared sheet one senior editor had kept for years. It had colour codes for operators that had quietly changed terms before, notes on reissued licence numbers, and a column that just said "check manually." That column was the real process.
The team turned the sheet into structured data, with the editor explaining every colour and note, and changed the model's job from making decisions to drafting flags for a person to review.
At Qubit Capital, which works with founders raising money, the sheet belonged to an investor researcher and recorded which funds had said not now, and why. The new tool built investor lists from the company database. Sahil Agrawal, founder and head of marketing at Qubit Capital:
In week 1 she checked every list against that sheet by hand, which is how we found it. The CRM showed one fund as open. Her sheet said a partner wanted to wait for their next fund.
The rollout moved back about three weeks while her notes were loaded in.
Loc Dang, digital marketing specialist at Cricket One, found his on a shared Google Sheet kept by a freelance sourcing assistant who was not on the org chart, with free-text columns in Vietnamese and English. The timeline went from two weeks to six, and the scope shrank to one task.
Before any build I now ask one thing: which spreadsheet does this actually run on, and who owns it? The org chart says who's supposed to do the work. The sheet says who does.
Sometimes the sheet shows what the process loses. Lilach Bullock, AI implementation consultant and fractional CMO, worked with a B2B client whose six salespeople tracked follow-up in a spreadsheet they colour-coded by hand, and who believed marketing was failing:
I pulled the last 90 days of form submissions and matched them against that sheet line by line. There were 43 leads that had never been contacted, lost somewhere between the form submission and someone typing the lead into the sheet by hand.
Her advice was to fix the capture step before automating follow-up, "or you'll end up scoring leads nobody is calling."
Pattern 2: The records were not what the system said
A second group found the official system itself was wrong. Anthony Pinto, founder of Veteran Vectors, exported a three-person firm's CRM and read the raw data before building anything:
The pipeline was empty. Zero records in the deal and proposal tables. Nobody had been logging deals in the old system. About 2,000 company records, and most weren't companies.
An earlier automation had filled the company name field with email addresses, phone numbers and email footers. The connection that logged contacts from the owner's email had been dead for weeks after a security reset. One of the questions he now asks first: "What breaks if one person's password changes?"
Daniel Tupczynski, head of marketing at Staffly and founder of MOAT Marketing, built prospecting agents for a team of three salespeople. He found the same company entered under three different names, and cards nobody had touched in a year still counted as open pipeline.
I found it by exporting everything into BigQuery and counting rows instead of reading the dashboard.
Of the plan, he wrote that "the first weeks went into data hygiene instead of agents. The agents were the fast part. Scoping had it the other way round."
Olam Sule, founder of Dolphin Analytics, works on marketing measurement. At a private aviation company, the brief assumed web analytics could explain where deals came from. When the tracking was mapped end to end, 443 of 500 contacts with deals had no web attribution at all.
The real sales process lived in the CRM and in people's conversations, not on the website.
A model fed the web data alone, he wrote, "would have learned from the minority of deals."
Abhijit Singh, founder of Seven Billion Analytics, built plain-English search for a wholesale shipping container marketplace where every purchase had gone through a sales rep. The rep pieced each answer together from depot spreadsheets, scanned condition reports and prices changed by hand in email threads.
The data behind the search box didn't exist as one thing. Inventory was almost always in the network somewhere, but no single system knew where.
It became a data engineering project first. The search went on top only after listings from hundreds of sources had been cleaned and kept in sync with the depots.
John Gwin, CEO of The Auctus Group, a medical billing company, found that even the written procedures could not be trusted. Over the years staff had done what worked last time, and the procedures had quietly followed them.
So we couldn't build from what the team was doing. If we had, we'd have automated the drift.
The question he asks before starting now is whether a step is the requirement or just the habit.
Pattern 3: Nobody held the login, and the org chart was wrong
Victor Smushkevich, founder of Tested Media, keeps meeting home service companies with a clean CRM account and a Google Sheet that actually runs the business, usually the office manager's. Access is his other surprise:
The Google Business Profile often sits under a former web designer's personal Gmail, and nobody remembers. Ownership transfer requests with Google can eat a week or two on their own, so I pad the plan with a discovery week before any automation gets built.
The org chart is off too. The owner says the estimator handles follow-up, and reading a few weeks of the real inbox says otherwise: "In practice, whoever is nearest their phone answers texts at night, often the owner's spouse."
Kalinov met a version of the access problem that had no fix. Some of the most reliable data sat in partner dashboards behind personal logins with two-factor authentication and no API. "We weren't going to hand those credentials to an automated agent, so that part stayed with people."
A different order: pilot first
One contributor argued against cleaning up before starting. Carlos Correa, chief operating officer of Ringy, was building a tool to answer RFPs, and the inputs were in people's heads.
If we tried to collect all the tribal knowledge before deploying the tool, we'd be at it forever.
His team launched a pilot within three months and ran it on an RFP of about 1,600 lines that would have taken five weeks by hand. It answered in 20 minutes with 86 percent accuracy, and the misses showed where the unwritten process was. The people on the team now review the exceptions each day and log what the tool did not know.
The through-line
Every contributor quoted here found the gap by watching the work or reading the raw data, not by asking about it. Nicholas shadowed an appraiser. Kalinov watched which file the editors opened first. Tupczynski counted rows. Pinto exported everything before he built anything. Nobody on the scoping call was hiding the spreadsheet. People leave it out because, to them, it is not a system.
The questions these contributors now ask before agreeing a plan are short:
- Which spreadsheet does this actually run on, and who owns it? (Loc Dang)
- Who on your team would you trust over any written policy? (Bogdan Nicholas)
- Which file do you open first every morning? (Panayot Kalinov)
- What breaks if one person's password changes? (Anthony Pinto)
- Is this step the requirement, or just the habit? (John Gwin)
Kalinov's test for the answers:
If a team can't answer those, you've scoped the demo, not the project.
Contributors
Turns out the actual grading decisions lived in a single Excel file maintained by one senior appraiser. He'd built his own depreciation logic over eight years. Nobody mentioned it because nobody else fully understood it.
They opened a shared sheet one senior editor had kept for years. It had colour codes for operators that had quietly changed terms before, notes on reissued licence numbers, and a column that just said "check manually." That column was the real process.
In week 1 she checked every list against that sheet by hand, which is how we found it. The CRM showed one fund as open. Her sheet said a partner wanted to wait for their next fund.
Before any build I now ask one thing: which spreadsheet does this actually run on, and who owns it? The org chart says who's supposed to do the work. The sheet says who does.
I pulled the last 90 days of form submissions and matched them against that sheet line by line. There were 43 leads that had never been contacted, lost somewhere between the form submission and someone typing the lead into the sheet by hand.
The pipeline was empty. Zero records in the deal and proposal tables. Nobody had been logging deals in the old system. About 2,000 company records, and most weren't companies.
I found it by exporting everything into BigQuery and counting rows instead of reading the dashboard.
The real sales process lived in the CRM and in people's conversations, not on the website.
The data behind the search box didn't exist as one thing. Inventory was almost always in the network somewhere, but no single system knew where.
So we couldn't build from what the team was doing. If we had, we'd have automated the drift.
The Google Business Profile often sits under a former web designer's personal Gmail, and nobody remembers. Ownership transfer requests with Google can eat a week or two on their own, so I pad the plan with a discovery week before any automation gets built.
If we tried to collect all the tribal knowledge before deploying the tool, we'd be at it forever.