Dry-Running a Bulk Edit Against the Dev API

Rehearse an automated edit on a throwaway copy of OpenStreetMap so that the first time your script touches the real map, nothing about its behaviour is a surprise to you or to the people reviewing it.

Prerequisites

Conceptual minimum

The development instance is a complete, separate deployment of the OpenStreetMap API. It has its own database, its own accounts, and — critically — its own object identifiers. An object with id 12345 there has no relationship to id 12345 on the live map.

That single fact shapes the whole rehearsal. You cannot simply point your script at the development endpoint and run it against live ids: the ids will either not exist or, worse, will exist and refer to something entirely unrelated. A meaningful dry run therefore has three parts: seed fixtures that resemble the real objects, run the genuine pipeline against them, and diff the result against what you predicted.

The four phases of a dry run and what each one proves Four phases. The extract phase pulls the real objects the edit would touch from live data, without modifying them. The seed phase recreates equivalent objects on the development instance and records the mapping from live identifiers to development identifiers. The run phase executes the genuine pipeline against the development endpoint, exercising the same code paths including conflict handling. The diff phase compares the before and after states of every seeded object and produces the counts and samples a reviewer will ask for. Four phases, and only the third one is your normal code extract real objects, read only the live map is untouched seed recreate on dev record the id mapping run the genuine pipeline same code, same paths diff before versus after counts and samples Seeding is the phase teams skip, and skipping it reduces the rehearsal to proving that HTTP works.
Without realistic fixtures a dry run tests the client library; with them it tests the edit.

The fourth thing a dry run produces is not technical at all: evidence. A community discussion about a bulk edit goes very differently when you can say “here is the exact object count, here are twenty representative before-and-after pairs, and here is the changeset comment every upload will carry” than when you can only describe your intentions.

Runnable solution

python 
from __future__ import annotations

import json
import logging
import os
import xml.etree.ElementTree as ET
from dataclasses import dataclass, asdict
from pathlib import Path

import requests

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
logger = logging.getLogger("osm.api.dryrun")

LIVE = "https://api.openstreetmap.org/api/0.6"
DEV = "https://master.apis.dev.openstreetmap.org/api/0.6"
DEV_TOKEN = os.environ["OSM_DEV_TOKEN"]
UA = "osm-pipeline-example/1.0 (contact@example.org)"


@dataclass
class SeedResult:
    live_id: int
    dev_id: int
    before_tags: dict[str, str]


def read_live_node(node_id: int) -> ET.Element:
    """Read only — the live map is never written during a rehearsal."""
    response = requests.get(f"{LIVE}/node/{node_id}",
                            headers={"User-Agent": UA}, timeout=30)
    response.raise_for_status()
    return ET.fromstring(response.content).find("node")


def seed_node(changeset_id: int, source: ET.Element) -> int:
    """Recreate an equivalent node on the dev instance; return its dev id."""
    root = ET.Element("osm")
    node = ET.SubElement(root, "node", {
        "changeset": str(changeset_id),
        "lat": source.get("lat"), "lon": source.get("lon"),
    })
    for tag in source.findall("tag"):
        ET.SubElement(node, "tag", {"k": tag.get("k"), "v": tag.get("v")})
    response = requests.put(
        f"{DEV}/node/create",
        headers={"Authorization": f"Bearer {DEV_TOKEN}", "User-Agent": UA},
        data=ET.tostring(root), timeout=30)
    response.raise_for_status()
    return int(response.text.strip())


def seed_fixtures(live_ids: list[int], changeset_id: int,
                  out: Path) -> list[SeedResult]:
    results: list[SeedResult] = []
    for live_id in live_ids:
        source = read_live_node(live_id)
        tags = {t.get("k"): t.get("v") for t in source.findall("tag")}
        dev_id = seed_node(changeset_id, source)
        results.append(SeedResult(live_id, dev_id, tags))
        logger.info("seeded live node %d as dev node %d", live_id, dev_id)
    out.write_text(json.dumps([asdict(r) for r in results], indent=2),
                   encoding="utf-8")
    return results


def diff_after_run(results: list[SeedResult]) -> dict[str, int]:
    """Compare every seeded object's current dev state against its before state."""
    counts = {"changed": 0, "unchanged": 0, "unexpected": 0}
    for result in results:
        response = requests.get(f"{DEV}/node/{result.dev_id}",
                                headers={"User-Agent": UA}, timeout=30)
        response.raise_for_status()
        node = ET.fromstring(response.content).find("node")
        after = {t.get("k"): t.get("v") for t in node.findall("tag")}
        if after == result.before_tags:
            counts["unchanged"] += 1
            continue
        removed = set(result.before_tags) - set(after)
        # Any key removed that the edit did not intend to remove is a red flag.
        if removed - {"shop"}:
            logger.error("dev node %d lost unexpected keys: %s",
                         result.dev_id, sorted(removed - {"shop"}))
            counts["unexpected"] += 1
        else:
            counts["changed"] += 1
    logger.info("dry-run diff: %s", counts)
    return counts


if __name__ == "__main__":
    logger.info("seed fixtures, run the real pipeline against DEV, then diff")

Step-by-step walkthrough

  1. Read live, write dev. The extraction reads the production API without authentication and never writes to it. Every write in the script targets the development endpoint.
  2. Recreate rather than assume. Each fixture is created on the development instance from the live object’s coordinates and tags, so the pipeline encounters realistic input including the messy tags real objects carry.
  3. Persist the id mapping. The live-to-development id mapping is written to disk, because without it the diff cannot connect a development object back to the real one it stands for.
  4. Snapshot the before state. Tags are captured at seed time. Reading them back after the run is the only way to know what the edit actually did rather than what it reported doing.
  5. Run the unmodified pipeline. The point of the rehearsal is to exercise the real code, including its changeset metadata, its batching and its conflict path. A special dry-run mode inside the pipeline tests the dry-run mode, not the pipeline.
  6. Classify the diff three ways. Changed as intended, unchanged, and — the important one — changed in a way that was not intended. The third bucket is where a partial-modify bug that strips tags shows up, and it must be zero.
  7. Publish the numbers. The counts and a sample of before-and-after pairs are the evidence for the community discussion. They are also the baseline you compare the real run against.
What a dry run can and cannot tell you about a bulk edit A grid of four questions against whether the dry run answers them. Whether the code works mechanically is fully answered. Whether the edit preserves untouched tags is fully answered by the before and after diff. Whether the tagging decision is correct for the region is not answered at all and needs community review. Whether concurrent editors will conflict is only partly answered, because the development instance has almost no other activity. A rehearsal answers two of these four questions Dry run answers it? What closes the gap Does the code work? fully nothing else needed Are other tags preserved? fully the before/after diff Is the tagging correct? not at all community review Will edits conflict? partly a small live pilot The bottom two rows are why a dry run is a prerequisite for a discussion rather than a substitute for one.
A green dry run proves the script is safe to run; it says nothing about whether the edit is a good idea.
Where a rehearsal typically finds its defects, by category Five categories of defect ranked by how often a first dry run surfaces them. Incomplete modify elements that strip untouched tags are the most common finding. Selection queries that include or exclude objects the author did not expect come next. Changeset metadata templates that render empty or uninformative follow. Non-idempotent edits that keep changing objects on every run come next. Conflict handling that bumps the version is rarest but the most serious when present. What a first dry run actually catches Modify strips other tags most common Selection off by a filter very common Empty changeset comment common Edit is not idempotent occasional Conflict handler bumps version rare, worst The bottom row is rare because it only fires under concurrency, which is exactly why it survives into production when there is no rehearsal.
Four of these five produce no error at all when they happen — they simply change more, or less, than intended.

Verification

  • The unexpected-change count is zero. Any object that lost a key the edit did not intend to remove is a blocking defect, not a curiosity.
  • The changed count matches the prediction. Predict the number before running; a mismatch means the selection query and the edit disagree.
  • A second run changes nothing. Re-running against the same fixtures should report everything unchanged, proving the edit is idempotent.
  • Changeset metadata is present on the dev changesets. Fetch one and read the comment as a stranger would; if it does not explain the edit, fix the template now.
  • The conflict path fires. Edit a seeded object through the development web interface mid-run and confirm the pipeline raises rather than overwriting.

Common errors and fixes

Symptom Root cause One-line fix
Object not found on dev Live ids used directly against the dev API Seed fixtures and use the recorded id mapping
HTTP 401 against dev Live token used on the development instance Register separately and issue a dev-instance token
Dry run passes, live run fails Pipeline has a special dry-run branch Exercise the real code path; change only the endpoint
No difference detected Before state never captured Snapshot tags at seed time and diff after the run
Unexpected key removals Partial tag set in the modify element Rebuild elements from the complete merged tag set
Second run keeps editing Edit is not idempotent Skip objects whose merged state equals the current one
Reviewers still object Evidence never published Publish counts and before-and-after samples with the proposal

Specification reference

The OpenStreetMap development API instance is a separate deployment intended for testing editing software. It requires its own user account and API tokens, its database is unrelated to the production database, and object identifiers assigned there have no correspondence to production identifiers. See the OSM development API documentation and the community’s automated edits code of conduct for the testing and discussion expectations that apply to bulk edits.

Frequently Asked Questions

Can I use my normal OSM account on the development API?

No. The development instance has an entirely separate user database, so you must register an account there and issue tokens from it. This trips people up because the login page looks identical, and the resulting error is an unhelpful authentication failure rather than anything that names the cause. Keep the two sets of credentials in clearly distinct environment variables so the wrong one cannot be picked up by accident.

Why not just add a dry-run flag to the pipeline instead?

Because a flag that skips the upload tests everything except the part that actually changes data. The bugs that matter in a bulk edit — a modify element missing tags, a conflict handler that bumps the version, a changeset comment template that renders empty — all live in the code the flag would skip. Changing only the endpoint keeps every code path live while directing the consequences somewhere harmless.

How many fixtures should I seed?

Enough to cover the variety in the real selection rather than enough to be statistically large. Include the ordinary case, the objects with unusual extra tags, the ones with tags in non-Latin scripts, and any object your selection query only just includes or only just excludes. Fifty carefully chosen fixtures find more problems than a thousand copies of the same easy case.

Does a clean dry run mean the edit is ready?

It means the script is safe to run, which is necessary and not sufficient. A dry run cannot tell you whether the tagging change is correct for that region, whether local mappers have a convention you are about to flatten, or whether the source data is good enough to import. Those questions are answered by publishing the evidence the dry run produced and letting people who know the area respond.

Up one level: The OSM Editing API & Changeset Upload.