Splitting Semicolon-Separated OSM Tag Values

The semicolon is OSM’s conventional multi-value separator, and it is also an ordinary character that appears inside real values. A global split turns one opening-hours string into two meaningless fragments; not splitting at all leaves a cuisine tag nobody can filter on.

Prerequisites

Conceptual minimum

Three facts decide the design.

Splitting is per key, never global. cuisine=pizza;italian is two values. opening_hours=Mo-Fr 09:00-17:00; Sa 10:00-14:00 is one value whose syntax uses semicolons internally. Applying one rule to both is guaranteed to break one of them.

Some keys are multi-valued and some are not. cuisine, sport, ref, alt_name and several others conventionally hold lists. name, opening_hours, description and addr:street do not, and a semicolon inside one of them is either literal or a data error — but not a separator.

Order sometimes matters. For cuisine the order is arbitrary and deduplication is safe. For ref on a route it may reflect signage order. Sorting a list to make comparison easier can destroy information, so it should be a per-key decision too.

Which keys split, which do not, and what happens when the rule is wrong A grid of four key families against whether they split and the consequence of getting it wrong. Multi-valued keys such as cuisine and sport should split, and failing to split leaves values nobody can filter on. Structured-syntax keys such as opening hours and conditional restrictions must not split, because the semicolon is part of their grammar and splitting produces meaningless fragments. Free-text keys such as name and description must not split, because a semicolon there is literal punctuation. Reference keys such as route numbers split but must preserve order, because the sequence can carry meaning. Four key families, three different rules Split? If you get it wrong cuisine, sport yes unfilterable values opening_hours never grammar destroyed name, description never punctuation split ref on a route yes, keep order signage order lost There is no rule that is right for all four rows, which is why a global split or a global refusal both cause damage.
The second row is the one a global splitter destroys most visibly, because the fragments are individually meaningless.

Runnable solution

python 
from __future__ import annotations

import logging
from dataclasses import dataclass

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
logger = logging.getLogger("osm.tags.multivalue")

# Keys whose values are conventionally lists. Anything absent is NOT split.
MULTI_VALUED: dict[str, bool] = {
    # key -> whether the order carries meaning
    "cuisine": False, "sport": False, "religion": False, "diet": False,
    "payment": False, "fuel": False, "recycling": False, "service": False,
    "alt_name": True, "ref": True, "operator": True, "brand": True,
    "route_ref": True, "network": True, "surface": False,
}
# Keys whose syntax uses semicolons internally. Splitting these is destructive.
NEVER_SPLIT = {
    "opening_hours", "service_times", "collection_times", "name", "description",
    "note", "fixme", "inscription", "addr:street", "addr:full", "conditional",
}
MAX_PARTS = 24          # a value with more parts than this is probably an error


@dataclass(frozen=True)
class MultiValue:
    key: str
    values: tuple[str, ...]
    original: str
    split: bool
    suspicious: bool


def split_value(key: str, value: str) -> MultiValue:
    """Split only where the key's convention says a list is meant."""
    base = key.split(":", 1)[0]     # addr:street -> addr, name:en -> name

    if key in NEVER_SPLIT or base in NEVER_SPLIT:
        return MultiValue(key, (value,), value, split=False, suspicious=False)
    if key not in MULTI_VALUED and base not in MULTI_VALUED:
        # Unknown key: do NOT split. An unsplit list is recoverable; a split
        # structured value is not.
        suspicious = ";" in value
        return MultiValue(key, (value,), value, split=False,
                          suspicious=suspicious)

    parts = [p.strip() for p in value.split(";")]
    parts = [p for p in parts if p]
    ordered = MULTI_VALUED.get(key, MULTI_VALUED.get(base, True))
    if not ordered:
        # Order is arbitrary for this key: deduplicate and sort for comparison.
        parts = sorted(set(parts))
    else:
        seen: set[str] = set()
        parts = [p for p in parts if not (p in seen or seen.add(p))]

    suspicious = len(parts) > MAX_PARTS
    if suspicious:
        logger.warning("%s has %d part(s); likely a data error", key, len(parts))
    return MultiValue(key, tuple(parts), value, split=True, suspicious=suspicious)


def split_tags(tags: dict[str, str]) -> dict[str, MultiValue]:
    return {k: split_value(k, v) for k, v in tags.items()}


def audit(tag_stream) -> dict[str, dict[str, int]]:
    """Which keys actually contain semicolons, and are we splitting them?"""
    stats: dict[str, dict[str, int]] = {}
    for tags in tag_stream:
        for key, value in tags.items():
            if ";" not in value:
                continue
            entry = stats.setdefault(key, {"seen": 0, "split": 0, "kept": 0})
            entry["seen"] += 1
            result = split_value(key, value)
            entry["split" if result.split else "kept"] += 1

    unsplit = {k: v for k, v in stats.items() if v["kept"] and v["seen"] > 20}
    if unsplit:
        logger.info("keys with semicolons that are NOT split (review these): %s",
                    sorted(unsplit, key=lambda k: -unsplit[k]["seen"])[:10])
    return stats


if __name__ == "__main__":
    tags = {
        "cuisine": "pizza;italian;pizza",
        "opening_hours": "Mo-Fr 09:00-17:00; Sa 10:00-14:00",
        "name": "Smith; Jones and Co",
        "ref": "A1;M25",
    }
    for key, result in split_tags(tags).items():
        logger.info("%-14s split=%-5s -> %s", key, result.split, result.values)

Step-by-step walkthrough

  1. Default to not splitting. An unsplit list can be split later; a structured value split into fragments cannot be reassembled reliably. When the key is unknown, leaving it alone is the recoverable choice.
  2. Check the namespace base as well as the full key. name:en inherits name’s rule, and enumerating every language variant is not practical.
  3. List the never-split keys explicitly. Keys whose grammar uses semicolons are few and well known, and naming them is what makes the default safe rather than merely cautious.
  4. Trim and drop empties. A trailing separator produces an empty part, which downstream becomes a mysterious blank category.
  5. Decide ordering per key. Sorting makes comparison easy and destroys signage order; doing it only where order is genuinely arbitrary keeps both properties where they belong.
  6. Deduplicate in both branches. A repeated value is noise in either case; the difference is only whether the surviving order is the original one.
  7. Flag implausible part counts. A tag with dozens of parts is almost always a data error rather than a genuine list, and the warning surfaces it rather than propagating it.
  8. Audit what is not being split. A key containing semicolons often and never split is either correctly excluded or a gap in the vocabulary, and the audit is what tells the two apart.
The four decisions the splitter makes for each tag Four decisions in order. The first checks whether the key or its namespace base is on the never-split list, returning the value untouched if so. The second checks whether the key is known to be multi-valued, and defaults to not splitting when it is not, since an unsplit list is recoverable. The third splits, trims and drops empty parts. The fourth applies ordering: sorting and deduplicating where order is arbitrary, and preserving order while removing repeats where it is not. Four decisions, and the default is to leave it alone never split? grammar uses semicolons return untouched known list? unknown means no recoverable choice split and trim drop empty parts no blank categories order sort or preserve per key, deliberately Defaulting to no split makes a missing vocabulary entry a minor inconvenience rather than an irreversible loss.
Every step except the third is a lookup, which is what keeps this cheap enough to run on every tag of every element.
What a global split and a global refusal each destroy Three panels. Splitting everything breaks structured values such as opening hours into fragments that are individually meaningless and cannot be reassembled, and turns literal punctuation in a name into two separate names. Splitting nothing leaves genuine lists as single opaque strings, so a consumer cannot filter on one cuisine or match one alternative name. Splitting per key handles both correctly at the cost of maintaining a vocabulary, which an audit of the corpus keeps current. Two global rules, both wrong; one per-key rule Split everything Opening hours in fragments Names split at punctuation Fragments are meaningless Cannot be reassembled Split nothing Cuisines stay opaque Alternative names unmatched Filters cannot work At least it is recoverable Split per key Both cases handled Needs a vocabulary Audit keeps it current Default to not splitting The middle panel is the safer of the two global choices precisely because nothing it does is irreversible.
That asymmetry is the argument for defaulting unknown keys to unsplit rather than to split.

Verification

  • Structured values survive intact. An opening-hours string with internal semicolons must come through as a single value.
  • List keys are split. A cuisine value with two parts must produce two values.
  • Unknown keys are not split. Introduce an unfamiliar key with a semicolon and confirm it passes through whole and flagged.
  • Empty parts do not appear. A trailing separator must not produce a blank value.
  • Order is preserved where it matters. A route reference list must keep its original sequence.

Common errors and fixes

Symptom Root cause One-line fix
Opening hours in fragments Global split applied Split per key, with an explicit never-split list
Cuisine values unfilterable Nothing split Add the key to the multi-valued vocabulary
Blank categories downstream Empty parts kept Trim and drop empty parts after splitting
Route references reordered Sorting applied to an ordered key Sort only where order is genuinely arbitrary
Language variants not split Only the full key checked Check the namespace base as well
Dozens of parts on one tag A data error propagated Warn above a plausible maximum part count
Vocabulary never improves Unsplit semicolon values unaudited Report keys that frequently contain separators

Specification reference

The semicolon is the conventional separator for multiple values in a single OSM tag, though its use is not universal and several keys define a syntax in which semicolons appear as part of the value rather than as a separator. Consumers are expected to apply the convention per key. See the semi-colon value separator documentation for the convention and its exceptions.

Frequently Asked Questions

Why not split every value containing a semicolon?

Because several keys use the semicolon inside their own grammar. An opening-hours value separates its rules with semicolons, and splitting it produces fragments that are individually meaningless and cannot be reassembled reliably. The same applies to conditional restrictions and to free-text keys where a semicolon is ordinary punctuation. Splitting has to be a per-key decision, driven by an explicit vocabulary.

What should an unknown key default to?

Not splitting. The asymmetry is the whole argument: a list left unsplit can be split later once the key is added to the vocabulary, while a structured value already split into fragments has lost information that is difficult to recover. Flagging unknown keys that contain semicolons, so they can be reviewed and classified, gives the benefit of the split without the risk.

Should the split values be sorted?

Only where the order is genuinely arbitrary. Sorting a cuisine list makes two features with the same cuisines compare equal regardless of how they were entered, which is useful. Sorting a route reference list destroys the signage order, which a consumer may depend on. The ordering decision belongs in the same vocabulary as the split decision, one flag per key.

How do I discover which keys need splitting?

Audit the corpus. Count, per key, how often its values contain a semicolon and whether the splitter currently splits it. A key that frequently contains separators and is never split is either a correct exclusion or a gap, and reviewing the top few by frequency covers almost all the value. Regional tagging conventions differ, so the audit is worth repeating on each new area.

Up one level: Tag Taxonomy & Key-Value Standards.