Tracking an OSM Feature Across Versions

A version number says an object changed. The history says what changed, and that is the difference between knowing a stored reference needs review and knowing whether it needs anything at all.

Prerequisites

Conceptual minimum

Every OSM object retains all its versions, and the API serves them individually or as a list. Each version carries its tags, its geometry or member references, the changeset that produced it, and a timestamp.

Comparing consecutive versions classifies each edit into one of a small number of kinds, and the classification is what makes a history readable.

A tag change alters the tag dictionary and nothing else. Usually benign for a stored reference, occasionally meaning-changing.

A geometry change alters a node’s coordinate or a way’s node list. For a way, a shortened node list is the signature of a split, which is the case a version number cannot reveal.

A membership change alters a relation’s member list, which affects anything that resolved the relation into geometry.

A deletion marks the version as not visible, which the API represents by an absent object in the current state and a final version flagged accordingly.

What a diff between consecutive versions reveals A grid of four change kinds against what the diff shows and what it implies for a stored reference. A tag-only change shows a different tag dictionary with an identical node list, and usually leaves a reference valid though its meaning may have moved. A geometry change on a node shows a moved coordinate and leaves a reference valid but possibly relocated. A shortened way node list is the signature of a split and means the reference now names a fragment. A member list change on a relation means anything derived from its geometry needs recomputing. Four diffs, four implications The diff shows Implication Tags only different dictionary meaning may differ Node moved new coordinate reference relocated Way shortened fewer node refs a split: now a fragment Members changed different member list recompute geometry The third row is the only one a version number alone cannot distinguish from the first, and it is the one that matters most.
Classifying the change is what turns a version bump from an alarm into information.

Runnable solution

python 
from __future__ import annotations

import logging
import xml.etree.ElementTree as ET
from dataclasses import dataclass
from enum import Enum

import requests

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
logger = logging.getLogger("osm.identity.history")

API = "https://api.openstreetmap.org/api/0.6"
HEADERS = {"User-Agent": "osm-pipeline-example/1.0 (contact@example.org)"}
SPLIT_RATIO = 0.75          # a node list losing a quarter looks like a split


class ChangeKind(str, Enum):
    TAGS = "tags"
    GEOMETRY = "geometry"
    MEMBERS = "members"
    LIKELY_SPLIT = "likely_split"
    DELETED = "deleted"
    NONE = "none"


@dataclass(frozen=True)
class Version:
    version: int
    visible: bool
    timestamp: str
    changeset: int
    user: str | None
    tags: dict[str, str]
    nodes: tuple[int, ...]           # ways only
    members: tuple[tuple[str, int, str], ...]   # relations only

    @property
    def node_count(self) -> int:
        return len(self.nodes)


def parse_version(element: ET.Element) -> Version:
    return Version(
        version=int(element.get("version")),
        visible=element.get("visible", "true") != "false",
        timestamp=element.get("timestamp", ""),
        changeset=int(element.get("changeset", 0)),
        user=element.get("user"),
        tags={t.get("k"): t.get("v") for t in element.findall("tag")},
        nodes=tuple(int(n.get("ref")) for n in element.findall("nd")),
        members=tuple((m.get("type"), int(m.get("ref")), m.get("role", ""))
                      for m in element.findall("member")),
    )


def history(osm_type: str, osm_id: int) -> list[Version]:
    response = requests.get(f"{API}/{osm_type}/{osm_id}/history",
                            headers=HEADERS, timeout=60)
    response.raise_for_status()
    root = ET.fromstring(response.content)
    versions = [parse_version(e) for e in root.findall(osm_type)]
    versions.sort(key=lambda v: v.version)
    logger.info("%s/%d has %d version(s)", osm_type, osm_id, len(versions))
    return versions


def classify(before: Version, after: Version) -> set[ChangeKind]:
    kinds: set[ChangeKind] = set()
    if not after.visible:
        return {ChangeKind.DELETED}
    if before.tags != after.tags:
        kinds.add(ChangeKind.TAGS)
    if before.nodes != after.nodes:
        kinds.add(ChangeKind.GEOMETRY)
        # A node list that lost a substantial share, with the remainder still a
        # prefix or suffix of the original, is what a split looks like.
        if (before.node_count and
                after.node_count < before.node_count * SPLIT_RATIO and
                (after.nodes == before.nodes[:after.node_count]
                 or after.nodes == before.nodes[-after.node_count:])):
            kinds.add(ChangeKind.LIKELY_SPLIT)
    if before.members != after.members:
        kinds.add(ChangeKind.MEMBERS)
    return kinds or {ChangeKind.NONE}


def walk(osm_type: str, osm_id: int) -> list[tuple[int, set[ChangeKind]]]:
    versions = history(osm_type, osm_id)
    timeline: list[tuple[int, set[ChangeKind]]] = []
    for before, after in zip(versions, versions[1:]):
        kinds = classify(before, after)
        timeline.append((after.version, kinds))
        marker = "  <-- SPLIT" if ChangeKind.LIKELY_SPLIT in kinds else ""
        logger.info("v%-3d %-19s %-22s by %s%s", after.version,
                    after.timestamp, ",".join(sorted(k.value for k in kinds)),
                    after.user or "(anonymous)", marker)
    return timeline


def changed_since(osm_type: str, osm_id: int,
                  stored_version: int) -> set[ChangeKind]:
    """What has happened to this object since we last looked at it?"""
    versions = history(osm_type, osm_id)
    relevant = [v for v in versions if v.version >= stored_version]
    if len(relevant) < 2:
        return {ChangeKind.NONE}
    kinds: set[ChangeKind] = set()
    for before, after in zip(relevant, relevant[1:]):
        kinds |= classify(before, after)
    kinds.discard(ChangeKind.NONE)
    return kinds or {ChangeKind.NONE}


if __name__ == "__main__":
    logger.info("changes since v3: %s", changed_since("way", 4305800, 3))

Step-by-step walkthrough

  1. Sort by version explicitly. The API returns versions in order in practice, and relying on that rather than asserting it is how a subtle ordering bug survives.
  2. Parse all three shapes in one record. Nodes have coordinates, ways have node lists, relations have members. One dataclass with empty tuples for the inapplicable fields keeps the comparison logic uniform.
  3. Check visibility first. A deleted version has no meaningful tag or geometry diff, and classifying it as a tag change would be nonsense.
  4. Detect a split structurally. A shortened node list where the remainder is a prefix or suffix of the original is the signature. Checking the prefix or suffix condition is what distinguishes a split from a way that simply had nodes removed.
  5. Return a set, not a single kind. One edit can change tags and geometry together, and collapsing that to one label loses information.
  6. Compare from the stored version forward. changed_since accumulates the kinds across every intervening version, which is the question a pipeline with a stored reference actually has.
  7. Log the user and timestamp. When a stored reference breaks, the next question is always who changed it and when, and having it in the same output saves a second lookup.
Three ways a way's node list changes and how each looks in a diff Three panels. A split leaves the node list shortened, with the remaining nodes forming a contiguous prefix or suffix of the original, and is the case a stored reference must detect. A node insertion leaves the list longer with the original nodes still present in order, which is ordinary geometry refinement and harmless to a reference. A reroute leaves the list a similar length but with different nodes in the middle, which means the geometry moved substantially even though nothing about the identifier changed. Three node-list changes, three different meanings Split List gets shorter Remainder is a prefix Or a suffix Reference now a fragment Insertion List gets longer Originals still in order Geometry refined Harmless to a reference Reroute Similar length Middle nodes differ Geometry moved Check before trusting Only the prefix or suffix test separates the first case from the third, which is why a length comparison alone is not enough.
All three increment the version identically, which is precisely why the version number is not a sufficient signal.
What kinds of edit a long-lived OSM way typically accumulates Five edit kinds with their approximate share of versions across a sample of long-lived road ways. Tag-only changes dominate, covering refinements to names, surfaces and access. Geometry refinement through node insertion or adjustment is the next largest. Relation membership changes follow. Splits are a small but significant share and are the ones that break stored references silently. Deletions are the smallest share, and they at least fail loudly. Edit kinds by share of versions, on long-lived ways Tags only about 48% Geometry refinement about 27% Membership change about 13% Split about 9% Deletion about 3% Roughly one edit in eleven is a split, which over a few years is enough to affect a substantial share of any stored reference set.
The two smallest categories are the ones that break references, and only the smaller of them does so loudly.

Verification

  • A known split is detected. Find a way you know was split and confirm the classifier flags that version.
  • An insertion is not flagged. A way that gained nodes should be a plain geometry change.
  • Deleted objects classify as deleted. Fetch the history of a deleted object and confirm the last transition is a deletion.
  • Relations classify on members. A relation whose members changed should report a membership change, not a geometry one.
  • changed_since accumulates. An object with three intervening edits of different kinds should report all of them.

Common errors and fixes

Symptom Root cause One-line fix
Splits not detected Only version numbers compared Diff the node lists and test for a prefix or suffix
Insertions reported as splits Length compared without the prefix test Require the remainder to be contiguous from one end
Deleted objects misclassified Visibility checked after the diff Test visibility first and return early
Relations report geometry changes Members compared as if they were nodes Compare member tuples separately from node lists
One kind returned per edit Classification collapsed to a single label Return a set; edits routinely change several things
History requests rate limited Per-object API calls at volume Use a history file for anything beyond a handful
Ordering assumed Versions used as returned Sort by version number explicitly

Specification reference

The API serves an object’s full version history at the /history path, returning every version with its tags, geometry or members, changeset, timestamp and author, and marking deleted versions as not visible. Individual versions are also retrievable by number. See the OSM API v0.6 documentation for the history endpoints and the version representation.

Frequently Asked Questions

How do I tell a split from a way that just lost nodes?

By checking whether the remaining nodes form a contiguous run from one end of the original list. In a split, one part keeps the original identifier and its node list becomes a prefix or a suffix of what it was. A way that had nodes removed from the middle, or simplified, leaves a list that is shorter but not contiguous in that way. The structural test is what distinguishes the two, and a length comparison alone cannot.

Should I use the API or a history file?

The API for a handful of objects, a history file for anything else. Each history request is a separate call against the same shared infrastructure everything else uses, so investigating a few hundred objects one at a time is both slow and impolite. A history extract answers the same questions locally at whatever rate your hardware allows, which is the only practical route for a scheduled re-resolution job.

Can one edit change several things at once?

Routinely. A single changeset can retag a way, adjust its geometry and alter its relation memberships, and a classifier that returns one label per edit will report whichever it checked first. Returning a set costs nothing and preserves the distinction between an edit that only fixed a spelling and one that also moved the road.

What does a version tell me that a timestamp does not?

The version is a monotonic counter, so it answers “has this changed since I looked” exactly, while timestamps can be equal for edits in the same changeset and are recorded with limited resolution. The timestamp is what you want for human-facing questions about when something happened; the version is what you want for the machine-facing question of whether anything happened at all.

Up one level: OSM Feature Identity & ID Stability.