All posts

The objection that the data is too messy, and why it argues for a different method, not against automation

Messy data is real and expensive, and it defeats rules that require clean input. That is exactly why the case for automation rests on a method that reads the data as it is and routes the doubtful cases to a person.

Javier Leguina

Javier Leguina

Co-founder & CTO of flowscope, previously a founding engineer at ModelML (YC W24).

· Buyer objections, answered

The most common reason an automation effort never starts is the belief that the data is too messy to automate, and the belief is usually correct about the data. In Forrester's 2023 Data Culture and Literacy Survey, more than a quarter of the data and analytics employees who called poor data quality an obstacle to data literacy at their organization estimated the resulting losses at more than $5 million a year, and 7 percent put them at $25 million or more. So when an operator says the data is a mess, the operator is describing a measured, expensive condition, not making an excuse. The error is in the next step, the move from "the data is messy" to "therefore we cannot automate," because that conclusion only follows if there is one method of automation, and there is more than one.

What "messy" actually means in an operating business

Messy data is variance with a structure that nobody wrote down. A vendor name appears as "Acme Corp" on one invoice, "ACME CORPORATION" on the next, and "Acme Inc." on a third, all referring to the same account. A date is formatted four ways across three systems. A purchase-order number lives in a field on some documents and in the body of an email on others. Quantities are entered with units on some lines and without on others. Experian, in its 2019 Global Data Management research, found that companies believe, on average, 29 percent of their customer and prospect data is inaccurate in some way. None of those defects stops a person from doing the work, because a person reads the document, infers what was meant, and continues. The defects only become fatal when the consumer of the data is a machine that was told to expect one exact shape.

Why "clean it first, automate later" rarely finishes

The standard advice for messy data is sequential: clean the data, structure it, then automate against the clean version. This is the right advice for a rules-based or template-based system, because such a system has no tolerance for variance. A rule that extracts the invoice total from cell C14 breaks the moment a vendor moves the total to C16, and a template keyed to a fixed layout fails on the next layout. So before that kind of automation can run, every input has to be forced into the shape the rules assume, which is the cleaning project.

The cleaning project rarely completes, for a reason that has nothing to do with effort. The variance that makes the data messy is generated continuously by the outside world. New vendors send new formats, customers fill in fields their own way, and a system migration introduces a fresh batch of inconsistencies. By the time a team has normalized last quarter's documents, this quarter's have arrived in shapes the cleaning rules did not anticipate. The cleaning is real work and worth doing in part, but staking the automation on its completion ties the automation to a finish line that keeps moving.

A method that reads the data in the state it is in

The alternative is to choose a method that does not require clean input in the first place. A model that reads unstructured and inconsistent text does what a person does with a messy document: it interprets "ACME CORPORATION" and "Acme Inc." as the same account, finds the total wherever it sits on the page, and reconciles the four date formats without a rule for each one. This is not a claim that the mess stops mattering. It is a claim that the mess is handled at read time, by interpretation, rather than at write time, by forcing every input into a fixed schema before anything can run. The method reads the data in the only state the data is ever actually in. The long tail of document variability is precisely the part that a fixed-template approach cannot reach and an interpretive one can.

Interpretation is not infallible, which is why the method has a second part that matters as much as the first. Every reading carries a confidence, and the cases where confidence is low, the genuinely ambiguous document, the figure that could be read two ways, the account that matches two records, get routed to a person rather than guessed. A redesign worth the name does not promise to read everything correctly. It promises to read the high-confidence majority without human touch and to hand a person a short, ranked list of the cases that actually need judgment. That exception path is what makes the rest safe, and it is the reason a serious effort starts by baselining the process so the rate of exceptions can be measured rather than assumed.

The objection answered, and where it still holds

A reasonable counter is that the messy-data objection is sometimes not about format variance at all but about something deeper: data that is not merely inconsistent but wrong, missing entirely, or contradicted by another system of record. That distinction is real and worth keeping. An interpretive method handles inconsistent representation of correct information very well, and it handles missing or contradictory information no better than a person does, because the information is not there to be read. In a 2017 MIT Sloan Management Review article, Thomas Redman estimated the cost of bad data at 15 to 25 percent of revenue for most companies, and a meaningful share of that cost lives in absent or conflicting records that no method can interpret into existence.

This is why the response to the messy-data objection is a method choice paired with an honest scope, not a promise that the mess does not matter. The objection correctly rules out the rules-first, clean-first approach. It does not rule out automation, because the method that reads variance and routes the doubtful cases to a person was built for the condition the objection describes. The remaining judgment is which workflows have inconsistent-but-recoverable data, where interpretation plus an exception path does the job, and which have genuinely absent or contradictory data, where the right answer is not to automate yet and to fix the source instead. Deciding which is which is the work the discovery phase exists to do, and it is a better use of three weeks than another quarter spent cleaning data that the world keeps making messy again.

Common questions

Our data is too messy to automate. Isn't that a real blocker?
The data is usually genuinely messy, but that argues for a different method, not against automation. Rules and templates break on the variance that messy data carries, which is why the advice to clean the data first and automate later rarely completes. A method that reads unstructured and inconsistent input and routes low-confidence cases to a person meets the data in its current state instead of waiting for it to be cleaned.
Should we clean our data before we automate anything?
A full clean-up before automating tends not to finish, because the data keeps changing and the variance is unbounded. The more reliable path is to automate against the data as it is, with a human handling the cases the system flags as uncertain, and to let the corrections improve the system over time.
How expensive is poor data quality, really?
In Forrester's 2023 Data Culture and Literacy Survey, more than a quarter of the data and analytics employees who called poor data quality an obstacle to data literacy at their organization estimated the annual losses at more than five million dollars, and seven percent put them at twenty-five million or more. Experian's Global Data Management research finds companies believe about 29 percent of their customer and prospect data is inaccurate.