How we built identity resolution that refuses to merge

Identity resolution has a seductive failure mode: the more aggressively you merge, the better your data looks.

Fewer duplicate customers. Higher average lifetime value. Cleaner cohorts. Every dashboard improves. And the merges that were wrong don't announce themselves — they quietly attach one person's behaviour to another person's record and then sit there, correct-looking, forever.

We built ours to under-merge on purpose. Here's the reasoning and what it costs.

The two errors are not symmetrical

A missed merge leaves one customer as two records. Your customer count is inflated, average LTV is understated, and a cohort analysis splits one person's behaviour across two rows. It's a visible, measurable, correctable error. Someone will eventually notice that jsmith@gmail.com and john.smith@work.com are the same person, and the merge can be applied later.

A false merge combines two people into one record. Their purchase histories interleave. Their support tickets share a thread. Their consent states — marketing permissions, data deletion requests, communication preferences — get combined, and there is no correct way to combine them.

The first error is recoverable. The second is a data breach with extra steps.

Causal finding this asymmetry is structural. A missed merge preserves the information needed to fix it later: two clean records that can be joined. A false merge destroys that information by interleaving two behavioural histories, and no amount of downstream cleverness reconstructs which event belonged to whom.

Given that, the correct posture is obvious and most systems don't take it, because under-merging makes your metrics look worse and vendors are evaluated on how good the metrics look in the demo.

The thresholds

Three bands, and the middle one is the important one.

Above 0.97 — auto-merge. Strong deterministic evidence. A matching verified email, a matching payment instrument fingerprint, an authenticated account link. These are cases where the alternative explanation requires a genuinely improbable coincidence.

0.90 to 0.97 — human review. Enough signal to suspect a match, not enough to act. These queue for a person, with both records shown side by side and the specific matching signals highlighted. Most teams process these in batches weekly.

Below 0.90 — never merge. Not queued. Not flagged. Not merged with a warning label. The records stay separate and the possibility is not surfaced, because a queue full of low-confidence candidates trains reviewers to approve without looking, which is worse than not asking.

That last decision was contested internally. The argument against it is that we're discarding real matches. That's true. We accepted it, because a review queue that's mostly noise stops being a review.

What we count as evidence

Signals are weighted differently depending on how forgeable and how shared they are.

High weight: verified email match, payment instrument fingerprint, authenticated account linkage, government-ID-verified fields where the customer's business collects them.

Medium weight: phone number, shipping address with name, device identifier persisting across sessions.

Low weight, never sufficient alone: name similarity, IP address, coarse geolocation, browser fingerprint.

The low-weight signals cause most false merges in naive systems. Two people at the same company on the same office IP with similar names is a common configuration in the real world, not an edge case. Household members sharing a device and a shipping address is another. A system that weights shared-context signals highly will merge families and coworkers routinely, and the resulting records look perfectly plausible.

Every merge is reversible

This is the part I'd argue is non-negotiable regardless of your threshold choices.

Source records are never deleted. A merge creates a link, not a replacement. The underlying records from Stripe, Shopify, Zendesk, or wherever remain intact and independently addressable.

Merges are versioned with their evidence. Every merge records what matched, what the confidence score was, which rule version produced it, and who or what approved it.

Unmerge restores prior state exactly. Not approximately. If we can't guarantee a clean unmerge, we don't perform the merge.

This matters more than the thresholds do, because it converts an irreversible decision into a reversible one — which is the same principle we apply to agent permissions. Wherever an action can be made reversible, make it reversible, and then you can afford to be slightly wrong.

What it costs us

Our numbers look worse in evaluations. A prospect comparing customer counts across vendors will see a higher count from us. That reads as worse resolution. It's actually more conservative resolution, and explaining the difference in a bake-off is genuinely hard.

Someone has to work the review queue. The 0.90–0.97 band doesn't clear itself. For a large customer that's real recurring effort, and "you'll need a person to spend twenty minutes a week on this" is not a feature bullet anyone enjoys.

We miss real matches. Below 0.90, definitely. Those customers stay split, their LTV stays understated, and their cohorts stay fragmented. We've decided that's the right error to make.

Cross-device resolution is weak by design. Systems relying heavily on probabilistic device graphs achieve much higher match rates than we do. They also merge households routinely. We think that trade is wrong for a product where the output is a money decision, but I acknowledge it's a trade and not a free win.

Where this connects to everything else

Identity resolution isn't a standalone feature. It's the substrate every other claim rests on.

A cohort analysis is only meaningful if cohort membership is correct. A causal finding about a deploy's effect requires knowing which customers were exposed. A holdout measurement is invalid if a customer appears in both treatment and control because their records didn't merge — or if two different people appear as one and their outcomes get averaged.

So a false merge doesn't produce one wrong record. It produces a confidently wrong finding, in a Room, with evidence attached, that a human then approves — because it looks exactly like every correct finding they've approved before.

That's the same failure shape as treating missing data as zero, and it's why both decisions got made the same way: make the error visible rather than making it invisible and favourable.

What we'd change if we're wrong

The strongest counterargument is that our conservatism pushes work onto customers who'd rather have the match. If a team's alternative is no resolution at all, our refusing a 0.88 match isn't rigour — it's leaving them with two records and no help.

If we find that customers routinely approve high volumes of 0.90–0.97 candidates without a meaningful rejection rate, that tells me our threshold is set too low and the band should shift upward. Conversely, a high rejection rate in that band means we're surfacing noise.

We instrument the review queue's approval-to-rejection ratio for exactly this reason. It's one of the few numbers where I don't have a strong prior about what the right answer is, and I'd rather find out from data than defend the guess we shipped.