Reconstructing OSM Features at a Past Date Jump to heading#

Given a full-history .osh.pbf, produce the map exactly as it existed at a chosen instant T — every node, way, and relation in the version it held then, and nothing that had been deleted by then.

Prerequisites Jump to heading#

Check each item; a missing one is the usual reason a “historical” snapshot still contains present-day edits.

• A full-history source file (*.osh.pbf) whose header sets HistoricalInformation — see Full-History .osh.pbf Processing for how the flag and version chains work.
• osmium command-line tool ≥ 1.14 on PATH for the time-filter subcommand.
• pyosmium ≥ 3.6 installed (pip install osmium) for the two-pass Python approach.
• Python 3.10+ for the dict[tuple[str, int], ...] typing used below.
• The target timestamp T fixed in UTC with an explicit Z suffix, e.g. 2023-01-01T00:00:00Z.
• Enough disk for a second output file roughly the size of one current-state extract of the same region.

Conceptual minimum Jump to heading#

A point-in-time snapshot is a per-object reduction, not a filter over the file as a whole. For each distinct (type, id), walk that object’s version chain, keep only versions whose timestamp is at or before T, and take the one with the highest version number among them. If that surviving version is a deletion — its visible flag is false — the object did not exist at T and is dropped entirely; otherwise it is emitted in exactly that version. This is the same fold introduced in the full-history processing overview, applied once per object and then written back out. The reason version rather than timestamp decides the winner is that timestamps have one-second resolution and two rapid edits can share one, whereas the version counter is strictly monotonic and never ties.

The one subtlety beyond that rule is consistency between primitives. A way at T references node ids, and those nodes have their own histories described by the node, way, and relation data model; a faithful snapshot must pair the way version live at T with the node versions live at T, or the geometry will be reconstructed from present-day coordinates. Both approaches below preserve that consistency by selecting the live version of every primitive against the same cutoff.

Runnable solution Jump to heading#

The fast path is one command. osmium time-filter streams the history file once and emits the state at the given instant:

# Snapshot the map as it stood at midnight UTC on 2023-01-01.
osmium time-filter history.osh.pbf 2023-01-01T00:00:00Z -o snapshot.osm.pbf

# Verify the result carries no history and reports its object counts.
osmium fileinfo -e snapshot.osm.pbf | grep -E "Multiple versions|Number of"

When you need the reduction inside a Python pipeline — to fold custom logic in, or to avoid shelling out — the two-pass approach below is the pyosmium equivalent. The first pass records, per (type, id), the version number live at T and whether it is visible; the second pass streams the file again and writes exactly those live, visible versions.

from __future__ import annotations

import logging
from datetime import datetime, timezone

import osmium

logger = logging.getLogger("osm.reconstruct")


class LiveVersionScanner(osmium.SimpleHandler):
    """Pass 1: find the version live at the cutoff for every object."""

    def __init__(self, cutoff: datetime) -> None:
        super().__init__()
        self.cutoff = cutoff
        # (type, id) -> (winning version number, is that version visible)
        self.live: dict[tuple[str, int], tuple[int, bool]] = {}

    def _scan(self, kind: str, obj: osmium.osm.OSMObject) -> None:
        if obj.timestamp > self.cutoff:
            return  # a future edit; it cannot win the snapshot
        key = (kind, obj.id)
        current = self.live.get(key)
        if current is None or obj.version > current[0]:
            self.live[key] = (obj.version, obj.visible)

    def node(self, n: osmium.osm.Node) -> None:
        self._scan("node", n)

    def way(self, w: osmium.osm.Way) -> None:
        self._scan("way", w)

    def relation(self, r: osmium.osm.Relation) -> None:
        self._scan("relation", r)


class SnapshotWriter(osmium.SimpleHandler):
    """Pass 2: write the live, visible version of each object."""

    def __init__(self, live: dict[tuple[str, int], tuple[int, bool]],
                 writer: osmium.SimpleWriter) -> None:
        super().__init__()
        self.live = live
        self.writer = writer
        self.written = 0

    def _emit(self, kind: str, obj: osmium.osm.OSMObject) -> None:
        target = self.live.get((kind, obj.id))
        if target is None:
            return
        version, visible = target
        if obj.version == version and visible:
            if kind == "node":
                self.writer.add_node(obj)
            elif kind == "way":
                self.writer.add_way(obj)
            else:
                self.writer.add_relation(obj)
            self.written += 1

    def node(self, n: osmium.osm.Node) -> None:
        self._emit("node", n)

    def way(self, w: osmium.osm.Way) -> None:
        self._emit("way", w)

    def relation(self, r: osmium.osm.Relation) -> None:
        self._emit("relation", r)


def reconstruct(src: str, dst: str, cutoff: datetime) -> int:
    """Materialize the state of *src* at *cutoff* into *dst*; return objects written."""
    scanner = LiveVersionScanner(cutoff)
    scanner.apply_file(src)
    logger.info("scanned %d distinct objects", len(scanner.live))

    writer = osmium.SimpleWriter(dst)
    emitter = SnapshotWriter(scanner.live, writer)
    try:
        emitter.apply_file(src)
    finally:
        writer.close()  # flush and finalize the output PBF
    logger.info("wrote %d live objects to %s", emitter.written, dst)
    return emitter.written


if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    T = datetime(2023, 1, 1, tzinfo=timezone.utc)
    reconstruct("history.osh.pbf", "snapshot.osm.pbf", T)

Step-by-step walkthrough Jump to heading#

1. Fix the cutoff in UTC. datetime(2023, 1, 1, tzinfo=timezone.utc) builds a timezone-aware instant. pyosmium exposes obj.timestamp as tz-aware UTC, so a naive datetime would raise on comparison — the tzinfo is not optional.
2. Pass 1 records only the winner. LiveVersionScanner skips any version dated after T and, for the rest, keeps the highest version number per (type, id). Because versions arrive in ascending order within each object, the last one it accepts is the survivor, and it stores that version’s visible flag alongside it.
3. Pass 2 emits by exact match. SnapshotWriter re-reads the file and writes an object only when its version equals the recorded winner and that winner is visible. A tombstone winner (visible=false) matches no output, so deleted objects vanish from the snapshot.
4. Two passes, not one. The file is streamed twice because you cannot know an object’s final surviving version until you have seen its whole chain, and buffering every version in memory would defeat the point on a large history. The scan keeps only two integers per object, so its footprint is a fraction of the file.
5. The writer is closed in finally. SimpleWriter.close() flushes the final block; skipping it on an exception leaves a truncated, unreadable PBF.

Verification Jump to heading#

• Header carries no history. osmium fileinfo -e snapshot.osm.pbf should report Multiple versions of objects: no — the output is a plain current-state file, not another history.
• Counts are plausible. The snapshot’s object count should be lower than the history’s distinct-id count by roughly the number of objects deleted before T. A count equal to the distinct-id count means the visible=false filter never fired.
• Spot-check a known deletion. Pick an id you know was deleted before T and confirm it is absent: osmium getid snapshot.osm.pbf n123456 should return nothing.
• Log lines agree. The wrote N live objects line should match osmium fileinfo object totals, confirming pass 2 emitted exactly the scanned survivors.
• Cross-check the CLI. Run osmium time-filter on the same T and compare object counts; large divergence points to a timezone or tie-breaking bug in the Python fold.

Common errors and fixes Jump to heading#

Specification reference Jump to heading#

The osmium time-filter command “copies all objects that are valid at the given point in time” from a history file to a snapshot output; an object is valid at T when its newest version with a timestamp at or before T is visible. See the osmium-tool manual for the command reference, and Planet.osm/full on the OSM Wiki for how full-history planet files are produced and distributed.

Related Jump to heading#

• Full-History .osh.pbf Processing — the format, the version chains, and the fold this page implements end to end.
• Extracting Changeset Metadata from History Files — the provenance-oriented sibling that reads the same versions for audit data.
• Node-Way-Relation Data Model — why a consistent snapshot must pair way versions with the node versions live at T.
• Applying .osc Change Files with osmium — moving a current extract forward through diffs, the inverse of rewinding one.
• How to Decode OSM PBF Headers in Python — confirming a file is historical before you reconstruct from it.

Up one level: Full-History .osh.pbf Processing.

Frequently Asked Questions Jump to heading#