Decision Science — on evidence, and the discipline of deleting it.
A number we had printed in one of our books turned out to be true the way a rumour is true: everyone repeats it, and no one has met the source. The number was a tidy one. Reverse-chronological resumes, it said, pass an automated screen about 2.3 times more often than the skills-first kind. It had the shape of a fact — a decimal, a multiplier, a clean comparison. We had cited it the way everyone cites it, to a couple of industry write-ups. This week, checking a sibling book, we went looking for the actual study behind the decimal. It isn’t there. The write-ups point to “a 2022 global survey.” The survey has no name, no author, no link, no sample size. The chain of citation dead-ends at a sentence that begins studies show and ends nowhere.
So we cut it.
The honest move cost us the impressive number — and that is exactly why it was the right one. In its place went a smaller, duller, true claim: a dated, chronological layout parses more reliably because each keyword sits tied to a role, and recruiters read it as straightforward rather than evasive. No multiplier. No decimal. Just the direction the evidence actually supports.
Worth sitting with why that trade feels like a loss when it isn’t.
A statistic that is precise, flattering to the advice it supports, and easy to repeat will out-compete a vague true one every time — in exactly the way a sharp anecdote beats a careful study at a dinner party. The 2.3× was useful. It made a soft recommendation (“use this format”) sound like a measured one (“use this format; here is the multiplier”). Usefulness is what gets a number copied from one blog to the next until the original — if there ever was one — is ten links upstream and nobody checks.
This is not a failing unique to career advice. It is the default failure mode of any field where the demand for confident guidance outruns the supply of real measurement. The number isn’t fabricated, exactly. It’s orphaned — severed from whatever did or didn’t produce it, and adopted by everyone who finds it convenient.
The tell is always the same: the citation points to a description of the evidence, never the evidence. A study found. Research shows. Surveys suggest. When the trail ends at a paraphrase, the number is a rumour wearing a decimal point.
The orphan number’s disguise is its specificity. 2.3 times is doing two jobs at once: it carries a direction — chronological beats functional — and it asserts a precision: not twice, not three times, but 2.3. The direction may well be sound. The precision almost never is, because precision is expensive. It takes a named sample, a stated method, a margin of error — and orphan numbers have none of that machinery behind them. They borrow the authority of the decimal without paying for it.
This is the old distinction between accuracy and precision, and it matters more in decisions than people allow. An accurate-but-vague claim — this format helps — keeps you honest about the size of what you don’t know. A precise-but-unfounded one — this format helps 2.3 times as much — invites you to plan as if the magnitude were settled. And magnitude is what you act on. You weight a 2.3× far more heavily than a “somewhat,” as you should, if it’s real. When it isn’t, the false precision doesn’t merely mislead — it mis-weights, quietly, every decision sitting downstream of it. The vague claim leaves room for judgement. The hollow precise one crowds judgement out, which is the more dangerous error, because it doesn’t feel like an error at all. It feels like rigour.
Here is the part I keep turning over. Anyone can add a footnote. Footnotes are cheap, and a dense bibliography is one of the easier kinds of credibility to fake — pile up enough references and most readers assume someone checked. The thing that actually separates trustworthy work from confident noise is the opposite move: the willingness to remove a number you can’t stand behind, even when it’s load-bearing, even when the smaller true claim is less impressive.
Trust isn’t built at the point of citation. It’s built at the point of deletion — at the number you were willing to lose.
That move has a cost, which is why it’s rare. You give up the clean multiplier. You hand the reader a claim that sounds less authoritative precisely because it’s more honest about its own evidence. In a market where everyone else still quotes the 2.3×, you look, briefly, like you know less than they do. You don’t. You know one more thing than they do: that the number is hollow. But that knowledge is invisible, and the missing decimal is not.
This is the quiet asymmetry of evidence-led work. The discipline that makes it trustworthy is the same discipline that makes it look, on the surface, less sure of itself.
I’ve come to think the evidence you keep matters less than the evidence you’re willing to lose. A body of work is only as honest as its weakest cited number, and the only way to find that number is to go looking for the study behind the decimal — and to actually cut what isn’t there, rather than leave it because it reads well and everyone else uses it.
We re-check every number in every book on a schedule, and we’ll keep finding orphans, because the whole genre is full of them. When we cut one, we’ll say so — like this. Not because the admission is good marketing, but because the willingness to make it is the entire difference between advice you can build a decision on and advice that merely sounds like you can.
So if you take one thing from this: next time a tidy statistic helps you decide something, follow it upstream one link further than feels necessary. See if you reach a study or a paraphrase. It’s a thirty-second habit, and it quietly changes which numbers you’re allowed to trust.
Pick this up in the next one, where I want to look at the opposite problem — the true numbers we ignore because they’re inconvenient.
- Jon.
First shared with our subscribers on Substack · read it there →
← All Field Notes