Finding the Replication Sequence for a Timestamp

You have a wall-clock datetime — the moment a base extract was cut, or the point you want a catch-up to begin — and you need the replication sequence number to start fetching diffs from, because the replication stream is addressed by integer, not by time.

Prerequisites

Conceptual minimum

The replication server does not offer a “give me the sequence for this time” endpoint, so the mapping from timestamp to sequence has to be discovered. The reliable method exploits the one invariant that always holds: sequence numbers and their timestamps are jointly monotonic, so the state.txt files form a sorted-by-time array indexed by sequence. Finding the sequence whose timestamp is at-or-before your target is therefore a binary search over that array, where each “array read” is an HTTP fetch of one state.txt. pyosmium wraps exactly this search in ReplicationServer.timestamp_to_sequence, and understanding the underlying probe is what lets you reimplement it when pyosmium is unavailable or you are talking to a non-standard feed.

The number you want is the sequence whose data is current at or just before your target instant, because applying that diff and everything after it brings the data up through your target without ever skipping a change. Picking the sequence after the target would leave a gap; that is why the search rounds down. This lookup is the seeding step referenced by the parent guide, Replication Sequence Numbers & State Tracking — you run it once to find where a sync should begin, then hand the result to the diff-applying loop.

Binary search from timestamp to replication sequence Sequences are sorted by timestamp; the search probes a midpoint state.txt, compares its timestamp to the target T, and halves the range until it lands on the greatest sequence at or before T. Sequences are sorted by time — find the last one at or before T low bound earlier current seq now (root cursor) probe mid state.txt target T if mid.timestamp < T → answer is right of mid else → answer is left of mid; halve and repeat converges on greatest sequence with timestamp ≤ T

Runnable solution

The primary path uses pyosmium; the fallback reimplements the binary search directly against state.txt files for environments where pyosmium’s client is unavailable or points at a feed it does not recognise.

python 
from __future__ import annotations

import logging
from datetime import datetime, timezone

import osmium.replication.server as rserv

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
logger = logging.getLogger("osm.replication.seqlookup")

MINUTE_URL = "https://planet.openstreetmap.org/replication/minute"


def sequence_for_timestamp(target: datetime, base_url: str = MINUTE_URL) -> int:
    """Return the sequence whose data is current at or before ``target``.

    ``target`` must be timezone-aware UTC. Applying this sequence and every
    later diff brings a snapshot up through ``target`` with no skipped change.
    """
    if target.tzinfo is None:
        raise ValueError("target datetime must be timezone-aware (UTC)")
    target = target.astimezone(timezone.utc)

    server = rserv.ReplicationServer(base_url)
    try:
        seq = server.timestamp_to_sequence(target)
    finally:
        server.close()

    if seq is None:
        raise LookupError(f"no published sequence at or before {target.isoformat()}")
    logger.info("timestamp %s -> sequence %d", target.isoformat(), seq)
    return seq


if __name__ == "__main__":
    when = datetime(2026, 7, 11, 0, 0, tzinfo=timezone.utc)
    print(sequence_for_timestamp(when))

The manual fallback fetches only the state.txt files the search actually probes — typically around log2(range) requests, roughly two dozen even for a multi-year span:

python 
from __future__ import annotations

import logging
from datetime import datetime, timezone

import requests

logger = logging.getLogger("osm.replication.seqlookup.manual")


def _fragment(seq: int) -> str:
    p = f"{seq:09d}"
    return f"{p[0:3]}/{p[3:6]}/{p[6:9]}"


def _state_timestamp(seq: int, base_url: str) -> datetime:
    url = f"{base_url.rstrip('/')}/{_fragment(seq)}.state.txt"
    resp = requests.get(url, timeout=30)
    resp.raise_for_status()
    for line in resp.text.splitlines():
        if line.startswith("timestamp="):
            raw = line.split("=", 1)[1].strip().replace("\\:", ":")
            return datetime.fromisoformat(raw.replace("Z", "+00:00"))
    raise ValueError(f"no timestamp field in {url}")


def _current_sequence(base_url: str) -> int:
    resp = requests.get(f"{base_url.rstrip('/')}/state.txt", timeout=30)
    resp.raise_for_status()
    for line in resp.text.splitlines():
        if line.startswith("sequenceNumber="):
            return int(line.split("=", 1)[1].strip())
    raise ValueError("no sequenceNumber in root state.txt")


def sequence_for_timestamp_manual(target: datetime, base_url: str, lo: int = 1) -> int:
    """Binary-search the state.txt array for the greatest sequence <= target."""
    target = target.astimezone(timezone.utc)
    hi = _current_sequence(base_url)
    if _state_timestamp(lo, base_url) > target:
        raise LookupError("target predates the low bound; widen lo or reset base")

    best = lo
    while lo <= hi:
        mid = (lo + hi) // 2
        if _state_timestamp(mid, base_url) <= target:
            best = mid        # mid is a candidate; look for a later one
            lo = mid + 1
        else:
            hi = mid - 1      # mid is too new; discard the upper half
    logger.info("manual search resolved %s -> sequence %d", target.isoformat(), best)
    return best

Step-by-step walkthrough

  1. Require an aware UTC datetime. sequence_for_timestamp rejects a naive datetime outright — comparing naive and aware datetimes raises, and silently assuming local time is how a lookup lands an hour off. astimezone(timezone.utc) normalizes any aware input.
  2. Delegate to pyosmium first. ReplicationServer(base_url).timestamp_to_sequence(target) performs the same binary search internally and is the maintained, correct implementation; always close the server to release its HTTP session.
  3. Handle the empty result. timestamp_to_sequence returns None when the target is older than anything the feed still publishes; surface that as a clear LookupError rather than passing None into path arithmetic.
  4. Fall back to explicit probing. _current_sequence reads the root cursor for the upper bound, then the loop halves the range, reading one state.txt per iteration through _state_timestamp, which unescapes the backslash-colons before parsing.
  5. Round down deliberately. The best variable keeps the greatest sequence whose timestamp is ≤ the target, so the caller applies from best forward and never skips a change straddling the boundary.

Verification

Confirm the resolved sequence is the right one before feeding it to a sync loop:

  • Bracket the answer. Fetch best.state.txt and (best + 1).state.txt; the first timestamp must be ≤ your target and the second strictly greater. If both are ≤ target, the search rounded down too far.
  • Cross-check the two methods. For the same target, sequence_for_timestamp and sequence_for_timestamp_manual should return the identical integer; a mismatch of one usually means a naive-datetime comparison slipped in.
  • Watch the log line. The timestamp … -> sequence line records exactly what was resolved; keep it so a later audit can reconstruct where a sync was seeded.
  • Count the probes. The manual search should touch roughly log2(current - lo) state files — about 24 for a minutely feed several years deep. Far more requests than that means the range bounds are wrong.

Common errors and fixes

Symptom Root cause One-line fix
TypeError: can't compare offset-naive and offset-aware Naive target datetime Attach tzinfo=timezone.utc before calling
Result an hour off Local-time datetime assumed UTC Convert with astimezone(timezone.utc) first
LookupError: no published sequence Target predates the feed’s retained history Seed from a fresh base extract instead
ValueError parsing timestamp Backslash-escaped colons left in the field Replace \: with : before fromisoformat
Wrong sequence entirely Regional target against the planet feed Pass the extract’s own base_url
Hundreds of HTTP requests lo/hi bounds wrong, search not converging Read the root cursor for hi; keep lo ≥ 1

Specification reference

The pyosmium replication client exposes timestamp_to_sequence(timestamp) on osmium.replication.server.ReplicationServer, which locates the sequence covering a given point in time by probing the server’s state.txt files — see the pyosmium replication documentation. The state.txt format, the sequenceNumber/timestamp fields, and the minutely/hourly/daily layout are described on the OSM wiki under Planet.osm/diffs. Timestamp parsing follows the ISO 8601 rules implemented by datetime.fromisoformat.

Frequently Asked Questions

Should the resolved sequence be applied inclusively or exclusively?

The function rounds down to the greatest sequence whose timestamp is at or before your target, so that sequence’s diff is the first you apply. Applying it and every later diff brings the data up through the target with no gap. If your base snapshot is already current as of that sequence, start from the next one to avoid re-applying it.

Why does pyosmium sometimes return None?

timestamp_to_sequence returns None when the target instant is older than the earliest state the feed still serves, or newer than the latest published sequence. Treat None as a signal to reseed from a fresh base extract for old targets, or to wait and retry for targets in the immediate future that upstream has not yet published.

How many network requests does the lookup make?

The search is logarithmic in the number of sequences between the low bound and the current cursor, so it fetches roughly log2 of that range in state.txt files — about two dozen requests even for a feed several years deep. pyosmium’s built-in search has the same complexity, so neither method downloads the whole history.

Does this work for regional Geofabrik feeds?

Yes, as long as you pass that feed’s own replication base URL. Regional feeds number their sequences independently of the planet stream, so a sequence resolved against the planet feed is meaningless for a regional extract and vice versa. Match the base URL to the extract’s osmosis_replication_base_url header.

Up one level: Replication Sequence Numbers & State Tracking.