Importing Nominatim from an OSM Extract

Stand up a private geocoder from a regional .osm.pbf so a batch of a hundred thousand addresses becomes a twenty-minute job instead of a day of throttled requests against a shared service.

Prerequisites

Conceptual minimum

A Nominatim import is not a copy of the extract; it is a purpose-built search database derived from it. The import reads OSM elements, decides which are places worth indexing, computes an address hierarchy by working out which administrative and named areas contain each place, and writes a token index used to match query text.

Three parts of that pipeline dominate the runtime and the failure modes.

Node location storage. Building way and relation geometry requires the coordinates of every referenced node, and there are far more nodes than places. The import can either keep them in the database — slow and enormous — or in a flatnode file, a fixed-layout file indexed directly by node id. The flatnode file is essentially mandatory for anything country-sized and above, and it must be on fast local disk because access to it is random.

Address computation. Each place is assigned its containing hierarchy: the street, the suburb, the city, the state, the country. This is the stage that makes structured queries work, and it is the most compute-heavy part of the import.

Interpolation. Address ranges mapped as interpolation ways are expanded into individual addressable points. Skipping this pass makes the import faster and leaves entire streets ungeocodable in regions where interpolation is the dominant address mapping style.

The stages of a Nominatim import and what each one produces Four stacked stages. Loading reads the extract and writes raw place rows, using a flatnode file to hold node coordinates outside the database. Ranking classifies each place by how specific it is, from country down to individual address point. Address computation assigns every place its containing hierarchy of street, suburb, city, state and country, and is the most compute-heavy stage. Indexing builds the token index that turns query text into candidate matches. A note observes that only the last stage is quick. Four stages, and the third one dominates the clock Load Read the extract, write place rows needs a flatnode file Rank Classify places by specificity country down to address Address Assign the containing hierarchy the longest stage by far Index Build the query token index comparatively quick Interrupting during the address stage means restarting it, which is why the import wants a machine nobody is going to reboot.
The address stage is where a country import spends most of its hours, and it is also the stage that makes structured queries work.

Runnable solution

bash 
#!/usr/bin/env bash
# Import a private Nominatim instance from a regional extract.
set -euo pipefail

EXTRACT="/data/poland-latest.osm.pbf"
PROJECT_DIR="/srv/nominatim"
FLATNODE="/fast-local/nominatim.flatnode"   # MUST be local, random-access disk
THREADS="$(nproc)"

mkdir -p "$PROJECT_DIR" "$(dirname "$FLATNODE")"

cat > "$PROJECT_DIR/.env" <<'ENV'
NOMINATIM_DATABASE_DSN=pgsql:dbname=nominatim
NOMINATIM_FLATNODE_FILE=/fast-local/nominatim.flatnode
# Keep interpolation: in many countries it is the dominant address style.
NOMINATIM_USE_US_TIGER_DATA=false
# The replication source must match the region of the extract, exactly.
NOMINATIM_REPLICATION_URL=https://download.geofabrik.de/europe/poland-updates/
ENV

cd "$PROJECT_DIR"
nominatim import --osm-file "$EXTRACT" --threads "$THREADS" 2>&1 | tee import.log

# Index freshness state so replication can start from the extract's timestamp.
nominatim replication --init

echo "import finished; run the verification script before trusting results"

Verification belongs in code, not in a manual spot check:

python 
from __future__ import annotations

import logging
from dataclasses import dataclass

import requests

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
logger = logging.getLogger("osm.nominatim.verify")

LOCAL = "http://localhost:8080"
TOLERANCE_DEG = 0.01          # roughly a kilometre; an import check, not a precision one


@dataclass(frozen=True)
class Known:
    street: str
    city: str
    country: str
    lat: float
    lon: float


def lookup(sample: Known) -> dict | None:
    params = {
        "street": sample.street, "city": sample.city,
        "countrycodes": sample.country, "format": "jsonv2",
        "addressdetails": 1, "limit": 1,
    }
    response = requests.get(f"{LOCAL}/search", params=params, timeout=30)
    response.raise_for_status()
    results = response.json()
    return results[0] if results else None


def verify(samples: list[Known]) -> bool:
    ok = True
    for sample in samples:
        hit = lookup(sample)
        if hit is None:
            logger.error("no match for %s, %s — import may be incomplete",
                         sample.street, sample.city)
            ok = False
            continue
        dlat = abs(float(hit["lat"]) - sample.lat)
        dlon = abs(float(hit["lon"]) - sample.lon)
        rank = int(hit.get("place_rank", -1))
        if dlat > TOLERANCE_DEG or dlon > TOLERANCE_DEG:
            logger.error("%s: matched %.4f,%.4f, expected %.4f,%.4f",
                         sample.street, float(hit["lat"]), float(hit["lon"]),
                         sample.lat, sample.lon)
            ok = False
        elif rank < 26:
            # A coarse rank means the street did not match; the hierarchy
            # (address computation) stage probably did not complete.
            logger.warning("%s: matched at place_rank %d — street-level or coarser",
                           sample.street, rank)
        else:
            logger.info("%s: ok at place_rank %d", sample.street, rank)
    return ok


if __name__ == "__main__":
    known = [
        Known("Rynek Główny", "Kraków", "pl", 50.0617, 19.9373),
        Known("Krupówki", "Zakopane", "pl", 49.2969, 19.9490),
    ]
    logger.info("import verification %s", "PASSED" if verify(known) else "FAILED")

Step-by-step walkthrough

  1. Put the flatnode file on local disk. It is accessed randomly by node id throughout the load stage. On network storage the import can take an order of magnitude longer, and no amount of extra CPU compensates.
  2. Set the replication URL to match the extract’s region. A country database updated from a continent’s diff directory receives changes for territory it does not contain. Nothing warns you.
  3. Give the import all the cores. The load and address stages parallelise reasonably well; the index stage less so. Expect the machine to be busy for hours on a country extract.
  4. Initialise replication immediately after the import. The state is derived from the extract’s own timestamp, so doing it later means guessing, and guessing means either a gap or a redundant replay.
  5. Verify against coordinates you already know. The verification script checks three things at once: that the address matched at all, that it matched near the right place, and that it matched at a fine enough rank to be a real street match rather than a fallback.
  6. Treat a coarse place rank as a failure signal. A result that resolves to a city when a street was supplied usually means the address computation stage did not complete, which is a far more common outcome than a completely failed import.
  7. Keep the import log. When results look wrong three weeks later, the stage timings in the log are the fastest way to see whether a stage was skipped or cut short.
Choosing between replication updates and periodic re-import for a private geocoder A decision node about how fresh the geocoder must be, with three outcomes. If results only need to reflect the map within a month, a periodic re-import from a fresh extract is simplest and gives predictable performance. If daily freshness is needed, attaching the replication update loop keeps the database current at the cost of continuous background work. If the results must be reproducible for audit, pinning one import and never updating it is the only approach that guarantees the same input gives the same output. How fresh, and how reproducible, must the answers be? Freshness or reproducibility? The two pull in opposite directions Pick one deliberately Periodic re-import Monthly freshness, predictable performance, simple to operate Replication updates Daily or better freshness, continuous background work Pinned, never updated Audit reproducibility: same input, same output, forever A geocoder used to produce numbers somebody will defend later belongs in the third branch, whatever the freshness argument says.
Most teams default to the middle branch without noticing that it makes last quarter's results impossible to reproduce.
Roughly how a Nominatim database grows relative to the source extract Five bars showing storage multiples relative to the source extract size. The raw place rows are about three times the extract. The computed address hierarchy adds the largest single increment, taking it to around seven times. The search token index takes it to around ten. The flatnode file, sized by the highest node identifier rather than by the region, adds a further couple of multiples. Working headroom for the import itself takes the practical requirement to around fifteen. Size the disk on the database, not on the file Source extract 1x baseline Place rows about 3x Plus address hierarchy about 7x Plus search index about 10x Plus flatnode and headroom plan for 15x The flatnode file is sized by the highest node id in the planet, so it is surprisingly large even for a small country extract.
Sizing a volume from the extract alone is the most common way an overnight import dies at four in the morning.

Verification

  • Known addresses resolve near their known coordinates. Within a kilometre is a generous tolerance that still catches a wholesale import failure.
  • Street-level queries return a fine place rank. A coarse rank across the board means address computation did not finish.
  • Interpolated addresses resolve. Pick a house number known to exist only as part of an interpolation range; if it fails, that pass was skipped.
  • Reverse geocoding returns a sensible hierarchy. A reverse lookup at a known city centre should return the city, not just the country.
  • Replication state is initialised. The stored sequence must correspond to the extract’s timestamp, not to the present moment.

Common errors and fixes

Symptom Root cause One-line fix
Import runs for days Flatnode file on network storage Move it to local disk and restart the import
Every result is city-level Address computation did not complete Check the log for the stage; rerun the indexing step
House numbers never match Interpolation pass skipped Re-enable interpolation and re-run the import
Out of disk near the end Sized on the extract, not the database Provision eight to fifteen times the extract size
Results drift from the public instance Replication points at the wrong region Match the replication URL to the extract exactly
Replication replays months of diffs State initialised after the import, from “now” Initialise replication immediately post-import
Queries slow after months of updates Index bloat from continuous updates Schedule a periodic re-import

Specification reference

The Nominatim import reads an OSM file into a PostgreSQL/PostGIS database in distinct stages — loading, ranking, address computation and indexing — and uses an optional flatnode file to store node coordinates outside the database, which is recommended for imports of a country or larger. Replication state is initialised from the imported data’s timestamp. See the Nominatim installation and import documentation for stage-by-stage requirements and the current sizing guidance.

Frequently Asked Questions

How much disk does a Nominatim import really need?

Between eight and fifteen times the source extract, depending on how densely the region is addressed. The place and address tables dominate, the search index adds substantially, and the flatnode file is sized by the highest node identifier rather than by the region, so it is surprisingly large even for a small country. Provision the upper end: running out during the address stage means restarting a multi-hour job from the beginning.

Do I need a flatnode file for a single country?

In practice yes. Without it, node coordinates are stored in the database and accessed through it, which turns the load stage into a database-bound crawl and inflates the database size considerably. The flatnode file is indexed directly by node identifier and is much faster, provided it sits on local random-access storage. Putting it on a network volume gives you the worst of both approaches.

Why do all my results come back at city level?

Because the address computation stage did not complete. That stage assigns every place its containing hierarchy, and without it the geocoder can still match a settlement name but has nothing finer to offer for a street or a house number. Check the import log for the stage’s timings, and re-run the indexing step rather than assuming the whole import failed — the earlier stages are usually intact.

Should I attach replication updates or re-import periodically?

It depends on whether freshness or reproducibility matters more. Replication keeps the database within hours of the live map but runs continuously and slowly degrades query performance. A periodic re-import gives predictable performance and a clean database, at the cost of being a batch job. If the geocoded results will ever be defended in an audit, consider pinning one import and never updating it, because that is the only way the same input keeps producing the same output.

Up one level: Nominatim Geocoding Pipelines.