Normalizing OSM Phone Numbers to E.164

A phone tag in OSM is free text, and the range of what appears there is wider than any regular expression usefully covers. The thing that makes normalization tractable is not a better pattern — it is knowing which country the feature is in.

Prerequisites

Conceptual minimum

E.164 is the international canonical form: a plus sign, a country calling code, and the national number, with no spaces or punctuation and a maximum of fifteen digits. Two numbers in E.164 are equal if and only if they are the same number, which is what makes it worth converting to.

Three facts about OSM’s phone values shape the work.

Most values are national. A number written as it would be dialled locally has no country code, so converting it requires knowing the country — and the only reliable source of that is the feature’s location, not the string.

Several keys carry phone numbers. phone, contact:phone, contact:mobile, fax and contact:fax all appear, sometimes on the same feature with different values.

Multi-valued is common. A business with two lines writes both, separated by a semicolon, and occasionally by a slash or the word “or”.

The rule that keeps this honest is that an unparseable value is not dropped and not guessed at. It is preserved and flagged, because a phone number that cannot be resolved is still information a human can use.

Three kinds of phone value and what each needs to resolve Three panels. An international value already carrying a plus and a country code resolves without any context and is the easy case. A national value written as it would be dialled locally carries no country code and can only be resolved using the feature's country, which comes from its location rather than from the string. An ambiguous value, such as one with a country code but no plus, or an internal extension alone, cannot be resolved reliably at all and belongs in a review queue rather than being guessed at. Three kinds of value, three levels of certainty International Already has plus and code Resolves with no context Validate and reformat The easy case National As dialled locally Needs the feature country From location, not text The common case Ambiguous Code without a plus Or an extension alone Cannot resolve reliably Flag, never guess The second panel is the majority, which is why containment must be resolved before phone normalisation rather than after.
Guessing a country for the third panel produces a valid-looking number that dials somewhere else entirely.

Runnable solution

python 
from __future__ import annotations

import logging
import re
from dataclasses import dataclass

import phonenumbers
from phonenumbers import NumberParseException, PhoneNumberFormat

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
logger = logging.getLogger("osm.clean.phone")

PHONE_KEYS = ("phone", "contact:phone", "contact:mobile", "mobile",
              "fax", "contact:fax")
# Separators people actually use, beyond the conventional semicolon.
SPLIT = re.compile(r"\s*(?:;|/|\bor\b|,(?=\s*\+))\s*", re.I)
EXTENSION = re.compile(r"\s*(?:ext\.?|x|extension|durchwahl)\s*(\d{1,6})\s*$", re.I)


@dataclass(frozen=True)
class Phone:
    e164: str | None
    extension: str | None
    original: str
    key: str
    status: str        # 'ok' | 'invalid' | 'unparseable' | 'no_country'


def split_values(value: str) -> list[str]:
    return [p.strip() for p in SPLIT.split(value) if p.strip()]


def normalize_one(raw: str, country: str | None, key: str) -> Phone:
    """Parse one value. `country` comes from the feature's location."""
    text = raw.strip()
    extension = None
    match = EXTENSION.search(text)
    if match:
        # Extensions are not part of E.164; keep them separately rather than
        # letting the parser silently absorb or reject them.
        extension = match.group(1)
        text = text[:match.start()].strip()

    is_international = text.startswith("+") or text.startswith("00")
    if not is_international and not country:
        # A national number with no country is genuinely unresolvable. Guessing
        # produces a valid-looking number that dials somewhere else.
        return Phone(None, extension, raw, key, "no_country")

    try:
        parsed = phonenumbers.parse(text, None if is_international else country)
    except NumberParseException:
        return Phone(None, extension, raw, key, "unparseable")

    if not phonenumbers.is_valid_number(parsed):
        # Parsed but not a real number for that country: a typo or a
        # placeholder. Keep the original for review rather than discarding it.
        return Phone(None, extension, raw, key, "invalid")

    return Phone(phonenumbers.format_number(parsed, PhoneNumberFormat.E164),
                 extension, raw, key, "ok")


def normalize_tags(tags: dict[str, str], country: str | None) -> list[Phone]:
    out: list[Phone] = []
    for key in PHONE_KEYS:
        value = tags.get(key)
        if not value:
            continue
        for part in split_values(value):
            out.append(normalize_one(part, country, key))
    return out


def audit(results: list[Phone]) -> dict[str, int]:
    counts: dict[str, int] = {}
    for phone in results:
        counts[phone.status] = counts.get(phone.status, 0) + 1
    total = sum(counts.values()) or 1
    logger.info("phone normalisation: %s (%.1f%% resolved)", counts,
                100 * counts.get("ok", 0) / total)
    if counts.get("no_country", 0):
        logger.warning("%d value(s) had no country: resolve administrative "
                       "containment before phone normalisation",
                       counts["no_country"])
    return counts


if __name__ == "__main__":
    tags = {"phone": "+48 12 345 67 89; 012 345 67 90 ext. 12",
            "contact:fax": "12 345 67 91"}
    for phone in normalize_tags(tags, country="PL"):
        logger.info("%-12s %-16s %-8s from %r", phone.key, phone.e164 or "-",
                    phone.status, phone.original)

Step-by-step walkthrough

  1. Split on more than semicolons. Slashes and the word “or” appear regularly, and a comma before a plus sign is almost always a separator rather than punctuation.
  2. Strip extensions before parsing. E.164 has no place for them, and leaving one attached causes the parser either to reject the number or to absorb the digits into it.
  3. Detect international form from the prefix. A value starting with a plus or a double zero carries its own country code and needs no context, which is worth checking before reaching for the feature’s country.
  4. Refuse to guess a country. A national number with no known country is unresolvable, and defaulting to a likely one produces a syntactically valid number that reaches a different country entirely.
  5. Distinguish unparseable from invalid. A string the parser cannot make sense of and a well-formed number that does not exist in that country are different problems with different fixes.
  6. Keep the original always. Every outcome retains the source value, so a human reviewing the flagged ones has something to work with.
  7. Report the status distribution. A high proportion lacking a country means the containment step has not run, which is a pipeline ordering problem rather than a data problem.
Four outcomes, what each means and what to do with it A grid of four statuses against their meaning and the appropriate action. An OK status means the value parsed and is a valid number for its country, and the canonical form can be used directly. An invalid status means the value parsed but is not a real number in that country, usually a typo or a placeholder, and belongs in a review queue. An unparseable status means the parser could make no sense of the string at all, which is usually a note or an address in the wrong field. A no-country status means a national number arrived without the feature's country, which is a pipeline ordering fault rather than a data fault. Four outcomes, four different responses Means Do ok valid for its country use the canonical form invalid typo or placeholder review the original unparseable not a number at all review, likely wrong field no_country containment not resolved fix the pipeline order The last row is the only one that indicates a fault in your pipeline rather than in the data, which makes it the first to check.
Collapsing these four into a simple success-or-failure loses the distinction between bad data and a bad pipeline.
The order the stages must run in, and why Four stages in a required order. Administrative containment must run first, because it produces the country that national numbers cannot be resolved without. Multi-value splitting comes next, so each number is handled independently rather than the whole tag failing on one bad part. Extension stripping follows, since the canonical form cannot hold one and the parser will otherwise absorb or reject it. Parsing and validation come last, with each outcome carrying its original value forward. Four stages, and the first one is not about phones containment resolve the country before anything else split one number at a time one bad part, one failure strip extension store it separately E.164 has no slot parse validate per country keep the original Running phone normalisation before containment is the most common ordering fault, and it shows up as a wall of unresolvable values.
Only the last stage is about phone numbers; the first three are about getting the input into a shape that can be parsed.

Verification

  • A known number round-trips. Take a number you can verify and confirm the canonical form matches its published international form.
  • National numbers resolve. With a country supplied, a locally-written number should produce the right country code.
  • No country means no guess. Without a country, a national number must return the no-country status rather than any number at all.
  • Extensions survive. A value with an extension should produce both a canonical number and the extension separately.
  • Multi-values split. A tag with two numbers should produce two results, both resolved.

Common errors and fixes

Symptom Root cause One-line fix
Numbers resolve to the wrong country A default country assumed Return a no-country status rather than guessing
Extensions lost or mangled Extension left attached during parsing Strip and store it before parsing the number
Only the first number kept Value not split Split on semicolons, slashes and the word “or”
Valid numbers rejected Written in international form with a double zero Treat a leading double zero as international
Everything reported unresolvable Country not resolved before this stage Run administrative containment first
Typos silently dropped Invalid numbers discarded Keep the original and flag it for review
Fax numbers missed Only the primary key read Read every key that carries a phone number

Specification reference

E.164 defines the international public telecommunication numbering plan: a number consists of a country code followed by a national number, with a maximum of fifteen digits in total and no formatting characters. Parsing a national-format number into this form requires knowing the region it belongs to. See the libphonenumber documentation for the parsing and validation semantics the library implements.

Frequently Asked Questions

Why can a national number not be parsed without a country?

Because the same digits mean different numbers in different countries, and nothing in the string says which. A number written as it would be dialled locally omits the country code precisely because the caller is assumed to know it. Supplying a default country produces a syntactically valid number that reaches somewhere else entirely, which is worse than returning nothing because it looks correct.

Where does the country come from?

From the feature’s location, through the administrative containment that a warehouse schema materialises anyway. That makes phone normalisation dependent on containment having run first, which is a pipeline ordering constraint worth stating explicitly — a large proportion of unresolvable phone values is usually a symptom of the two stages being in the wrong order.

What should happen to an extension?

Keep it in its own field. E.164 has no representation for an extension, so leaving it attached either causes the parse to fail or lets its digits be absorbed into the number, producing something that dials the wrong place. Splitting it out before parsing preserves both pieces, and a consumer that needs the extension has it while one that does not can ignore it.

Should invalid numbers be dropped?

No. A value that parses but is not a real number for its country is usually a typo, a transposed digit or a placeholder, all of which a human can recognise and often correct. Dropping it discards the evidence; keeping the original with an invalid flag routes it somewhere useful. The same applies to values the parser cannot make sense of at all, which are frequently addresses or notes entered in the wrong field.

Up one level: Value Standardization & Regex Cleaning.