Matching OSM Features to External Datasets

The matcher is where conflation is won or lost, and almost every bad matcher has the same shape: a distance threshold, a fuzzy name comparison, and a single number that decides. It performs well on the easy cases everybody tests with, and fails silently on exactly the cases that matter.

The four stages of a matcher and what each one is allowed to discard Four stages. Normalisation cleans names and categories on both sides without discarding anything, keeping the originals. Candidate generation uses a spatial index to reduce the search from every feature to a handful per record, and must be generous because a candidate discarded here can never be recovered. Scoring evaluates each candidate pair on independent signals and discards nothing. Classification is the only stage that decides, sorting pairs into confident matches, review cases and non-matches. Only the last stage is allowed to decide normalise clean both sides keep the originals generate spatial, generous discards permanently score independent signals discards nothing classify three-way outcome the only decision The second stage is the only one that loses information irreversibly, which is why it should always err towards too many candidates.
Separating scoring from deciding is what makes a matcher tunable without re-running the expensive stages.

The Problem This Topic Solves

You have an external dataset of places — assets, premises, addresses, facilities — and you need to know which OSM feature each record corresponds to. The result feeds either an enrichment or an import, and both depend entirely on the matches being right.

The failure scenario is a matcher that looks excellent in testing. It is developed against a hundred records in a town the developer knows, tuned until every one matches, and run over fifty thousand records nationally. The match rate is 82 percent, which sounds good. Nobody notices that in dense city centres the matcher is picking the nearest of six plausible candidates essentially at random, or that in rural areas where the external coordinate is a postcode centroid it matches almost nothing. Both failures are invisible in an aggregate percentage.

Prerequisites

Understand the section’s framing in OSM Conflation & Data Enrichment, particularly the three-way classification. Know your spatial index options from Spatial Index Selection: R-tree vs H3 vs Quadkey. And have OSM tags normalised already — matching against raw values means every casing variant is a separate category.

Normalisation: What to Clean and What to Keep

Normalisation prepares both sides for comparison and must be non-destructive: the cleaned value is used for comparison, the original is kept for display and review.

For names, the useful operations are Unicode normalisation, case folding, punctuation removal, whitespace collapsing, and — carefully — the removal of a small set of legal and generic suffixes. “Careful” matters: stripping “Ltd” is usually safe, stripping “Church” is not, because for a church it is the entire distinguishing content of the name.

For categories, both sides need mapping onto a shared vocabulary. An external dataset’s “PHARMACY” and OSM’s amenity=pharmacy should both become one token. Where no shared vocabulary exists, the category signal simply cannot be used, and pretending otherwise produces a signal that is noise.

Candidate Generation: Be Generous

Candidate generation is the only stage that discards information permanently. A correct pairing not proposed here can never be recovered, however good the scoring is.

The radius should come from the positional characteristics of the external source, not from a round number. A surveyed asset register might need 25 metres; a geocoded address file might need 150; a postcode-centroid file might need 500 or might be unusable for point matching at all. Measuring that distribution from a labelled sample is a day’s work that determines everything downstream.

A cap on candidates per record keeps the pair count bounded in dense areas. Ten is usually generous; if a record legitimately has more than ten plausible candidates, that record is a review case regardless of what the scoring says.

Typical positional error by external source type, which sets the candidate radius Five source types with the radius that covers most of their positional error. A surveyed asset register with recorded coordinates needs about twenty five metres. A rooftop-level geocode needs about eighty metres. A street-interpolated geocode needs about one hundred and fifty metres. A locality or settlement centroid needs several hundred metres. A postcode centroid can be a kilometre or more and is often unusable for point matching at all. The radius comes from the source, not from a round number Surveyed coordinates about 25 m Rooftop geocode about 80 m Street interpolation about 150 m Locality centroid about 400 m Postcode centroid 1 km or more A source at the bottom of this table cannot be point-matched reliably at all; it can only constrain a match made on other evidence.
Using one radius across sources means over-matching the accurate ones and missing everything in the inaccurate ones.

A Note on Blocking Keys

Where a spatial index is unavailable or the geometry is unreliable, the classical alternative to a spatial candidate stage is blocking: grouping both datasets by a cheap key and comparing only within a group. A postcode, a settlement name, or the first few characters of a normalised street name all work.

Blocking trades recall for speed in a very specific way: any true pair whose two sides fall in different blocks can never be found. That makes the key choice critical — a postcode block is useless if one dataset’s postcodes are missing or wrong, which is exactly the situation that made the geometry unreliable in the first place. Where both a spatial index and a usable blocking key exist, running both and taking the union of candidates costs little and recovers pairs that either alone would miss.

Scoring on Independent Signals

Four signals cover most real conflation, and their value comes from being independent.

Distance is already computed by the candidate join and costs nothing. It is a weak discriminator — something is always nearby — but a strong disqualifier at the far end of the radius.

Name similarity is usually the strongest signal where names exist. Token-based measures handle word reordering better than character-based ones for place names; character-based ones handle typos better. Using both and keeping them separate is more informative than blending them.

Category agreement is genuinely independent of position and name, which makes it disproportionately valuable. A pharmacy matching a pharmacy is meaningful evidence even at a moderate distance with a middling name score.

Shared identifier — a ref:* tag, an operator’s own code, a Wikidata link — is the strongest signal of all when present, and should short-circuit the rest. Linking OSM Features to Wikidata Identifiers covers establishing such links deliberately.

The combination should be a weighted sum with the components retained, not a single opaque number — the argument developed in Scoring Conflation Candidates with Multiple Signals.

Classification: Three Groups, Not Two

The output is three groups.

Confident pairs have a high combined score and no close runner-up. That second condition is what most matchers omit: a pair scoring 0.9 with a runner-up at 0.88 is not confident, it is ambiguous, and the gap between the best and second-best candidate is often more informative than the best score itself.

Review covers everything in the middle, plus every ambiguous case regardless of score. This group should be small enough that a human can work through it and large enough that it contains the genuinely uncertain cases.

No match covers records with no candidate scoring above a floor. These are not failures — an external record with no OSM counterpart is a normal and common outcome, and treating it as a failure pushes the matcher towards accepting bad pairs.

Turning a scored candidate set into one of three outcomes A decision node taking the best score and the gap to the runner-up, with three outcomes. A high best score with a clear gap to the second-best candidate is a confident match and proceeds automatically. A high score with a close runner-up, or a middling score of any kind, is ambiguous and goes to a reviewer regardless of the absolute value. A best score below the floor means no candidate is plausible and the record is recorded as unmatched, which is a legitimate result rather than a failure. Best score and runner-up gap, together How good, and how clear? Two numbers, not one The gap is the tie-break Confident match High score and a clear gap to the second-best Needs review Middling score, or a close runner-up at any score No match Nothing above the floor; a normal, expected outcome A matcher that looks only at the best score sends its densest-area mistakes straight through as confident matches.
Two numbers instead of one is the entire difference between a reviewable matcher and an opaque one.

Validation and Error-Handling Matrix

Condition Root cause Detection Remediation
High match rate, poor precision Nearest candidate accepted without discrimination Manual sample shows wrong matches Require a gap between best and second-best
Almost no matches in rural areas Radius calibrated on urban data Match rate varies sharply by density Calibrate the radius per source and region
Name signal contributes nothing Names absent on one side Score distribution identical with and without it Drop the signal rather than letting it add noise
Category signal is noise No shared vocabulary between sources Agreement rate near chance Build an explicit mapping, or drop the signal
Review queue unmanageable Middle band too wide Most pairs land in review Improve signals; do not just move thresholds
Matches differ between runs Unstable tie-breaking Same input, different output Sort candidates deterministically before selecting
One record matched to many features Cardinality rule not enforced Duplicate feature identifiers in output Enforce the agreed cardinality explicitly

Performance and Scale

The pair count governs everything. With \(n\) external records and an average of \(k\) candidates each, scoring is \(O(nk)\), and \(k\) is controlled entirely by the radius and the cap.

Build the spatial index once over the OSM side and query it per record; rebuilding per batch is the most common accidental quadratic in a conflation pipeline. Order the signal computations by cost, with distance first — it is already available — and anything requiring a network call outside the loop entirely.

Conflation partitions cleanly by area, provided partitions overlap by at least the candidate radius so a record near a boundary can still see its match. That makes it embarrassingly parallel at whatever granularity your infrastructure prefers.

Failure Modes and Gotchas

  • Aggregate match rate hides everything. Report it by region, by density and by source type, or it will mislead you.
  • A missing counterpart is not a failure. Many external records have no OSM feature; a matcher that must match everything will match them wrongly.
  • Signals must be independent. Three signals derived from position are one signal counted three times.
  • The runner-up matters. The gap to the second-best candidate is often a better confidence indicator than the best score.
  • Suffix stripping is dangerous. Removing a generic word that happens to be the distinguishing part of a name destroys the signal.
  • Ties must break deterministically. Otherwise two runs over identical input produce different matches and nothing is reproducible.
  • Matches decay. Store the date and the OSM version alongside every match, per OSM Feature Identity & ID Stability.

Integration Points

Upstream, both datasets need normalising — OSM through Parsing & Tag Normalization Workflows, and the external side through equivalent cleaning. Where the external source has addresses rather than coordinates, geocoding comes first, using Nominatim Geocoding Pipelines — and the place rank it returns is itself a useful input to the candidate radius.

Downstream, confident matches feed either enrichment or import preparation, and every match feeds the audit described in Conflation QA & Rollback.

Guides in This Topic

Frequently Asked Questions

What radius should I use for candidate generation?

One derived from the positional characteristics of the external source, measured rather than assumed. A surveyed register with recorded coordinates needs tens of metres; a street-interpolated geocode needs a couple of hundred; a postcode centroid may not be point-matchable at all. Take a labelled sample, measure the distance distribution between true matches, and set the radius to cover the great majority of it — then cap the candidates per record so dense areas stay bounded.

Why does the runner-up score matter?

Because it distinguishes a confident match from an ambiguous one. A pair scoring highly with nothing else close is strong evidence; the same score with a near-identical alternative means the matcher is choosing between two plausible answers and has no basis for preferring one. Requiring a meaningful gap between the best and second-best candidate catches exactly the dense-area failures that an absolute threshold lets through.

Should every external record end up matched?

No, and a matcher built on that assumption will produce bad matches to satisfy it. External datasets routinely contain records with no OSM counterpart: things not yet mapped, things that closed, things outside the extract. Treating no-match as a legitimate and expected outcome keeps the thresholds honest, and the no-match rate is itself a useful signal about the two datasets’ coverage.

How do I know whether a signal is contributing anything?

Compare score distributions for known matches and known non-matches with and without that signal, using a labelled sample. A signal that shifts the two distributions apart is contributing; one that shifts them equally is adding noise that makes the total harder to interpret. Category agreement is worth this check in particular, because it depends on a shared vocabulary that may not actually exist between your sources.

Can I reuse one matcher across several external datasets?

The structure, yes; the parameters, no. Candidate radius, which signals are available, and how much weight each deserves all depend on the source’s positional accuracy, whether it has names, and whether its categories map onto anything. Keep the pipeline shared and the configuration per source, and calibrate each new source against its own labelled sample before running it at scale.

Up one level: OSM Conflation & Data Enrichment.