Tag & Attribute Consistency Checks Jump to heading#

An OSM element can carry perfect geometry, sit in the right place, and still be wrong — because its tags say two contradictory things at once. An element tagged both highway=residential and building=yes is not a road or a house; it is a data error that renders twice, routes as a street, and imports as a structure depending on which consumer reads it first. Tag defects are insidious precisely because nothing about a free-form key-value map forces internal consistency: the format accepts maxspeed=fixme, a layer of bridge, or a road with no name and no complaint. This guide, part of the broader OSM Data Quality & Validation discipline, treats the tag dictionary as a schema to be enforced rather than a bag of strings to be trusted, and it lays out the rule classes and the engine that check every element against them.

Prerequisites Jump to heading#

Tag QA only makes sense against a reference vocabulary, so anchor these first. The controlling document is Tag Taxonomy & Key-Value Standards: it defines which keys are enumerations, which expect numeric or integer values, and which combinations are semantically legal — the ground truth every rule below encodes. If your rules will grow beyond hard-coded checks into a maintained rule language, the sibling Authoring OSM Validation Rules reference covers expressing them as reusable presets rather than bespoke Python. Finally, consistency checking assumes the values are already clean of casing and whitespace noise — run the passes in Value Standardization & Regex Cleaning upstream, or a rule that expects maxspeed=50 will trip over 50 or 50 mph before it ever tests the real logic.

Rule Classes: A Taxonomy of Tag Defects Jump to heading#

Tag QA is tractable because every defect falls into one of five rule classes, each with a distinct evaluation shape.

• Deprecation — a key or value the community has retired in favour of a successor. These are lookups against a deprecated→replacement map: highway=ford moved to ford=yes on the crossing node, and many barrier sub-values were consolidated. Deprecation is usually informational: the data still parses, but it should be migrated.
• Conflict — two keys that must not co-occur on the same element because they assign incompatible primary types. highway and building on one element is the canonical case; natural=water with building=yes is another. Conflicts are errors — a consumer cannot know which type wins.
• Value-type — a key whose value must match a datatype. maxspeed must be a positive number (optionally with a unit suffix), layer and level must be integers, lanes a positive integer, width a number. A value like maxspeed=fixme or layer=1.5 is a type violation regardless of geometry.
• Required-tag — a feature class that mandates a companion tag. A highway should carry a name or an explicit noname=yes; an amenity=parking benefits from access. Missing-required rules are keyed on the feature class the element belongs to, so they fire only when the primary tag is present.
• Cross-tag — a consistency rule between two present tags. oneway=yes on a junction=roundabout is redundant but harmless; bridge=yes without a layer is suspect; maxspeed present with highway=footway is contradictory. These encode the implications that the taxonomy leaves implicit.

The value of naming the classes is that each maps to a reusable evaluator, so adding a new check means adding data (a key, a pair, a datatype) rather than code. The distinction between conflict and cross-tag is worth holding: a conflict says two tags may never share an element, while a cross-tag rule says that given one tag, another must (or must not) also appear or hold a value.

Rule Specification Reference Jump to heading#

A declarative rule set makes the checks auditable and lets a non-programmer add a rule. The schema below drives the engine in the next section; every rule carries a severity so the report can be triaged.

Two encoding rules keep the set unambiguous. First, value_type speed accepts a bare number, a number plus mph/knots, or the special token none/walk that OSM defines for maxspeed; anything else is a violation. Second, required rules match on a specific when_value (e.g. highway=residential) or on the key alone (when_key: highway, any value), which controls how narrowly the requirement applies. Keeping the rule set in version control alongside the pipeline means every change to what “consistent” means is reviewable, exactly as the Authoring OSM Validation Rules reference recommends.

Step-by-Step: A Declarative Tag-Consistency Engine Jump to heading#

The engine below loads a rule set and evaluates one element’s tags against every rule, returning structured findings. It uses Python 3.10+ type hints and the project logger convention, and it separates detection (this engine) from remediation so a report never silently rewrites data.

1. Model a finding. Every rule produces the same shape — element reference, rule id, severity, message, and optional suggested replacement — so downstream triage is uniform.
2. Dispatch by rule class. A single evaluator per class keeps the code linear; the rule’s class field selects the function.
3. Guard the antecedent. Required and cross-tag rules only fire when their trigger is present, so an element without a highway never trips a highway-required rule.
4. Parse values defensively. Value-type checks must treat a missing or empty value as “no opinion,” not a violation, and must strip unit suffixes before testing numerics.
5. Collect, do not abort. One bad tag never stops the scan; findings accumulate and are returned for the caller to rank and route.

from __future__ import annotations

import logging
import re
from dataclasses import dataclass

logger = logging.getLogger(__name__)

_INT_RE = re.compile(r"^-?\d+$")
_NUM_RE = re.compile(r"^-?\d+(\.\d+)?$")
_SPEED_RE = re.compile(r"^(\d+(\.\d+)?)(\s?(mph|knots))?$")
_SPEED_TOKENS = {"none", "walk", "signals"}


@dataclass(frozen=True)
class Finding:
    element: str
    rule_id: str
    severity: str
    message: str
    replacement: str | None = None


def _check_value_type(key: str, value: str, dtype: str) -> bool:
    """Return True if value is well-formed for dtype."""
    if dtype == "int":
        return bool(_INT_RE.match(value))
    if dtype == "number":
        return bool(_NUM_RE.match(value))
    if dtype == "speed":
        return value in _SPEED_TOKENS or bool(_SPEED_RE.match(value))
    logger.warning("unknown dtype %r for key %r", dtype, key)
    return True


def evaluate(tags: dict[str, str], ref: str, rules: list[dict]) -> list[Finding]:
    """Evaluate one element's tags against every rule, returning findings."""
    findings: list[Finding] = []
    for rule in rules:
        cls, rid = rule["class"], rule["id"]
        sev = rule.get("severity", "warning")

        if cls == "deprecated":
            key = rule["key"]
            if key in tags and (rule.get("value") in (None, tags[key])):
                findings.append(Finding(
                    ref, rid, rule.get("severity", "info"),
                    f"deprecated {key}={tags[key]}", rule.get("replacement")))

        elif cls == "conflict":
            if all(k in tags for k in rule["keys"]):
                findings.append(Finding(
                    ref, rid, sev, "conflicting tags: " + ", ".join(rule["keys"])))

        elif cls == "value_type":
            key = rule["key"]
            val = tags.get(key, "").strip()
            if val and not _check_value_type(key, val, rule["dtype"]):
                findings.append(Finding(
                    ref, rid, sev, f"{key}={val} is not a valid {rule['dtype']}"))

        elif cls == "required":
            trigger = rule["when_key"]
            want_val = rule.get("when_value")
            if trigger in tags and want_val in (None, tags[trigger]):
                missing = [k for k in rule["require"] if k not in tags]
                if missing:
                    findings.append(Finding(
                        ref, rid, sev, f"{trigger} missing required: {', '.join(missing)}"))

        elif cls == "cross_tag":
            ik, iv = rule["if_key"], rule.get("if_value")
            if ik in tags and iv in (None, tags[ik]):
                tk, tv = rule["then_key"], rule.get("then_value")
                actual = tags.get(tk)
                if (tv is None and actual is None) or (tv is not None and actual != tv):
                    findings.append(Finding(
                        ref, rid, sev, f"{ik}={tags[ik]} expects {tk}={tv}, found {actual}"))

    return findings

An example rule set the engine consumes:

rules:
  - {id: dep-ford, class: deprecated, key: highway, value: ford, replacement: "ford=yes", severity: info}
  - {id: conf-hw-bldg, class: conflict, keys: [highway, building], severity: error}
  - {id: vt-maxspeed, class: value_type, key: maxspeed, dtype: speed, severity: error}
  - {id: vt-layer, class: value_type, key: layer, dtype: int, severity: error}
  - {id: req-hw-name, class: required, when_key: highway, require: [name], severity: warning}
  - {id: xt-bridge-layer, class: cross_tag, if_key: bridge, if_value: "yes", then_key: layer, severity: warning}

Validation & Detection Matrix Jump to heading#

Each rule class has a signature failure, and each fix belongs upstream in the source data or in a normalization pass — never as a silent rewrite inside the checker.

Performance & Scale Considerations Jump to heading#

The engine is O(elements × rules), and both factors are controllable. The rule count stays small — a mature set is dozens of rules, not thousands — so the dominant cost is the element scan. On a streaming source the checks add negligible overhead because they touch only the tag dictionary, which is already in memory for each element; there is no geometry reconstruction and no spatial query. The one trap is recompiling regular expressions: precompile every value_type pattern once at rule-load time, as the module above does, rather than inside the per-element loop, or a planet-scale scan pays the compile cost hundreds of millions of times.

Two levers help at scale. First, short-circuit on primary tag: index rules by their trigger key so an element with no highway skips every highway rule instead of testing each one — a dictionary from key to relevant rules turns the inner loop from “all rules” into “applicable rules.” Second, batch findings to columnar output: emit findings as rows and write them to Parquet or a database in chunks, so a multi-million-finding report on a dirty continental extract never materializes as one giant Python list. For frame-based inputs, the same rules vectorize cleanly over pandas columns, which is the path the child Flagging Deprecated OSM Tags in a Pipeline walkthrough takes for the deprecation class.

Failure Modes & Gotchas Jump to heading#

• Empty is not invalid. A missing or empty value should register no opinion, not a type violation. Test presence before datatype, or every absent maxspeed floods the report with false errors.
• Units are legal in some keys. maxspeed=30 mph is valid; a naive integer check rejects it. Encode the unit grammar in the datatype (the speed type) rather than forcing bare numbers.
• Deprecation is a moving target. The deprecated→replacement map changes as the community evolves conventions. Pin it to a dated snapshot and refresh deliberately, or a rule that flagged nothing last month starts flagging valid current tags.
• Conflicts have legitimate exceptions. A few tag pairs that look conflicting are occasionally valid on multipolygon relations versus ways. Scope conflict rules to element type where needed so a valid relation is not flagged as a broken way.
• Required does not mean universal. name is expected on most roads but not on highway=service driveways or highway=steps. Encode the exceptions with when_value so the requirement fires only where it applies.
• Regex compiled per element. Building the same re pattern inside the element loop silently multiplies runtime. Compile once at load; the cost is invisible until you profile a large run.

Integration Points: Feeding Downstream Stages Jump to heading#

The engine’s output is a stream of Finding rows, and the clean boundary is that consumers subscribe to a severity rather than to the check internals. The wiring below scans an iterable of elements and partitions findings by severity so a pipeline can hard-fail on errors while merely logging deprecations:

from collections import defaultdict


def scan_elements(elements, rules: list[dict]) -> dict[str, list[Finding]]:
    """Evaluate many elements and bucket findings by severity."""
    buckets: dict[str, list[Finding]] = defaultdict(list)
    for ref, tags in elements:
        for finding in evaluate(tags, ref, rules):
            buckets[finding.severity].append(finding)
    logger.info(
        "scan complete: %d errors, %d warnings, %d info",
        len(buckets["error"]), len(buckets["warning"]), len(buckets["info"]),
    )
    return buckets

Errors gate a publish; warnings feed a review queue; info-level deprecations feed a migration backlog. This tag layer runs beside the geometry and topology checks in the wider OSM Data Quality & Validation suite, and the rule set itself is best maintained as the presets described in Authoring OSM Validation Rules so the same definitions drive both an ad-hoc scan and a continuous pipeline.

Examine Tag Consistency in Depth Jump to heading#

This reference expands into a focused, runnable treatment of its most common rule class:

• Flagging Deprecated OSM Tags in a Pipeline — a complete deprecated→replacement mapping applied over a pyosmium stream or a pandas frame, emitting a findings report with suggested successors for each obsolete tag.

Frequently Asked Questions Jump to heading#

Related Jump to heading#

• Flagging Deprecated OSM Tags in a Pipeline — the runnable deprecation-class walkthrough this section frames.
• Tag Taxonomy & Key-Value Standards — the reference vocabulary every consistency rule encodes.
• Authoring OSM Validation Rules — expressing these checks as maintainable, reusable presets.
• Value Standardization & Regex Cleaning — the upstream pass that cleans values before consistency logic runs.
• OSM Data Quality & Validation — the parent section spanning geometry, topology, and tag QA.

Up one level: OSM Data Quality & Validation.