Querying OSM: Overpass, Nominatim & APIs

The four OpenStreetMap services a pipeline talks to and what each returns A pipeline in the centre reaches four OpenStreetMap services. Overpass answers ad hoc element queries and returns JSON or XML. Nominatim answers place and address lookups and returns ranked candidates. The editing API reads single objects and accepts changeset uploads. Extract providers serve whole regional PBF files on a daily cadence. Each service returns a different shape of data and carries a different quota, so the pipeline chooses one per question rather than defaulting to whichever is easiest to call. Four services, four answer shapes, four quotas your pipeline asks one question Overpass API ad hoc element query Nominatim place and address search Editing API single object, upload Extract provider whole region, daily JSON or XML elements, tags, members ranked places with address detail object versions and changeset ids PBF file read locally, no quota Nothing here is interchangeable: asking Overpass for a country, or the editing API for a bulk read, is how a pipeline gets blocked.
Choosing the wrong service is rarely a performance problem first — it is a quota problem first, and a correctness problem second.

Everything on this site up to now assumes a file: an extract on disk, parsed with a streaming reader, normalized and written to a sink. That assumption is right most of the time, and it is why OSM Data Fundamentals & Architecture opens with the PBF container rather than with an HTTP client. But real pipelines also talk to the live OpenStreetMap service layer: to ask a question too small to justify downloading a continent, to geocode a column of addresses, to read the current version of one object before deciding whether a cached copy is stale, or — rarely and carefully — to write an edit back.

This section covers that service layer as an engineering surface. It serves mapping engineers who need a query answered today, ETL developers wiring an external call into a scheduled job, and GIS analysts who want a bounded dataset without standing up a parsing stack. The unifying theme is that every one of these services is a shared public resource with a quota, and the difference between a pipeline that works for years and one that gets its address blocked in a week is almost entirely about respecting those quotas by design rather than by apology.

What Each Service Actually Is

The four services differ in what they store, not just in what they return, and confusing them is the root of most misuse.

Overpass API is a read-only query engine over a continuously updated copy of the OSM database, exposed through its own language, Overpass QL. It is not a REST API over objects; it is closer to a database with a query planner, and it will happily accept a query that scans a continent and then refuse to finish it. Its unit of work is a set of elements produced by filters and set operations, and its cost model is dominated by how many elements a filter must touch before the bounding box narrows the search.

Nominatim is a geocoder built on an imported OSM database with its own indexing of places, addresses and administrative hierarchies. It answers two questions — “where is this text?” and “what is at this coordinate?” — and it answers them with a ranking, not a lookup. Treating a Nominatim result as a deterministic key is the classic mistake; it is a search result, and search results change when the underlying map changes.

The editing API (the /api/0.6/ endpoints) is the authoritative interface to the live database. It serves single objects and small bounding boxes, and it accepts changeset uploads. It is deliberately hostile to bulk reads, because it is the same machinery every editor depends on. Its read half is useful for checking one object’s current version; its write half is the only legitimate way to put data back into the map, and it is covered with the care it deserves in The OSM Editing API & Changeset Upload.

Extract providers are not an API at all — they are file servers publishing pre-cut regional PBF files, usually daily. They have no quota worth worrying about, they are the cheapest possible source per element, and they are the right answer far more often than the other three. OSM Extract Providers & Automated Downloads covers how to consume them reproducibly.

The four OSM services compared on freshness, volume ceiling, cost and failure mode A grid comparing Overpass, Nominatim, the editing API and extract providers across four questions. Freshness ranges from minutes for Overpass and the editing API to daily for extracts. The practical volume ceiling is thousands of elements for Overpass, one request per second for Nominatim, one object per call for the editing API, and unlimited for a local extract. The cost falls on a shared public server for the first three and on your own disk for extracts. The typical failure mode is a timeout for Overpass, a block for Nominatim, a rejected upload for the editing API, and a stale file for extracts. Pick the service from the row that matters to your job Overpass Nominatim Editing API Extracts Freshness minutes hours to days live daily Volume ceiling thousands 1 req/sec one object none Who pays shared server shared server shared server your disk Fails as timeout HTTP 429 rejected upload stale data The bottom row is the one to design against: each service fails in a different way, and only the extract's failure is silent.
A stale extract is the most dangerous failure here precisely because nothing errors — the pipeline succeeds against yesterday's map.

Overpass QL: the Model Behind the Syntax

Overpass QL reads like a filter language, but it is a set algebra. Each statement produces a set of elements; ->.name binds a set to a variable, a bare statement writes into the default set _, and the final out statement serialises whatever is in the set you point it at. Understanding this is what turns “why does my query return nothing” into a five-second diagnosis, because an empty result almost always means a set was overwritten rather than that the data is missing.

Three primitives carry most real work. A tag filter like node["amenity"="pharmacy"] narrows by key and value, with regular-expression forms (~), negations (!=) and existence tests (["amenity"]). A spatial filter — a bounding box (s,w,n,e), an (around:radius,lat,lon) clause, or an (area) reference resolved from an administrative relation — narrows by geography. And a recursion operator — > for “and their members and nodes”, < for “and the parents that reference them” — completes a partial result into something geometrically usable. The overwhelming majority of Overpass queries in production are one tag filter, one spatial filter, and one recursion, and the ones that time out are usually the ones that got the order wrong. Overpass API Query Language develops the full model with worked queries.

The output format matters as much as the filter. out body; emits elements with tags but no geometry for ways; out geom; inlines coordinates onto each way and relation member, which is far larger but removes the need to resolve node references yourself; out center; collapses each way or relation to a single representative point, which is exactly what you want for a point-of-interest layer and a fraction of the bytes. Choosing out geom when out center would do is one of the easiest order-of-magnitude savings available, and it costs one word.

Quotas, Timeouts and the Etiquette That Is Actually a Spec

Every public instance of these services publishes usage limits, and they are enforced, not aspirational. Overpass applies a per-query timeout (settable with [timeout:n], capped server-side) and a memory ceiling ([maxsize:n]), and it queues requests per client with a slot system that returns HTTP 429 when you exceed your share. Nominatim’s public instance permits roughly one request per second from a single source, requires an identifying User-Agent, and blocks sources that ignore either. The editing API rate-limits writes and rejects changesets that are too large or that lack a meaningful comment.

The engineering consequence is that retry logic is not optional garnish, it is the main event. A client that retries immediately on a 429 converts a soft throttle into a hard block; a client that retries with exponential backoff and honours a Retry-After header rides the throttle out and finishes. Equally important is the shape of the workload: a hundred small queries issued back-to-back are much harder on a shared server than one query that asks for the same data in a single pass, and they are also slower for you. Handling Overpass Timeouts and Rate Limits turns this into a concrete client, and Batch Geocoding with Nominatim Without Getting Blocked does the same for the geocoder.

What a well-behaved client does when a shared OSM service pushes back A four-step response to a rate limit. First, read the status: a 429 with a Retry-After header is an instruction, not an error. Second, wait the full interval with jitter so a fleet of workers does not retry in lockstep. Third, halve the concurrency for the rest of the run rather than restoring it immediately. Fourth, cache the successful response so a rerun of the same job never asks twice. A note adds that retrying immediately is what converts a soft throttle into a hard block. A 429 is an instruction, not an error read 429 plus Retry-After the server said how long wait full interval, jittered never in lockstep back off halve concurrency for the rest of the run cache store the response a rerun asks once Caching is the step teams skip, and it is the one that removes most of the traffic: reruns during development dominate real production volume.
Three of these four steps are about the next request; the fourth is about never needing to make it.

Geocoding Is a Ranking Problem, Not a Lookup

Nominatim answers with candidates ordered by an importance score derived from place rank, address completeness and, where present, external popularity signals. A forward geocode of “Springfield” returns many real answers, and the right one depends entirely on context your query did not supply. This has two practical consequences for a pipeline.

First, always constrain. A countrycodes parameter, a viewbox with bounded=1, or a structured query that supplies street, city and postalcode separately narrows the candidate space far more effectively than any post-hoc filtering of a free-text result. Structured queries also sidestep the parser’s guesswork about which token is a street and which is a suburb.

Second, never treat a geocode as stable. The coordinate returned for an address can move when a mapper improves the data, and the osm_id attached to a result can change when a way is replaced by a relation. If you need stability, store the returned identifier and the coordinate and the date, and re-resolve on a schedule rather than assuming yesterday’s answer. The identity question underneath this is the same one OSM Feature Identity & ID Stability works through for the file-based pipeline.

For anything beyond a few thousand lookups, the honest answer is to stop calling the public service and run your own. A Nominatim import from a regional extract is a well-trodden path, it removes the rate limit entirely, and it makes the geocoder reproducible — the same input gives the same output until you choose to re-import. Importing Nominatim from an OSM Extract covers the sizing and the import itself.

Writing Back: the Editing API and the Duty of Care

Reading OSM is a technical decision; writing to it is a social one. The editing API will accept a changeset from any authenticated account, and the community’s tolerance for automated edits is conditional on those edits being discussed, documented, reversible and small. The mechanics are straightforward — open a changeset, upload an osmChange document, close the changeset — and they are covered in Uploading an OSM Changeset from Python. The discipline around them is what matters.

Three rules carry most of the weight. Every automated edit needs a changeset comment that names the source, the script and a contact point, because the first thing a reviewer does with a suspicious edit is look for a human to ask. Every bulk edit needs a dry run against the development API before it touches the live database, which costs an hour and has saved countless reverts; Dry-Running a Bulk Edit Against the Dev API shows the setup. And every edit needs a version check immediately before upload, because the API rejects a change to an object whose version has moved on, and a client that blindly retries with a bumped version silently overwrites somebody else’s work.

Three ways an automated OSM edit goes wrong and what prevents each Three panels describing failure modes of automated editing. A conflict happens when the object version moved between read and write, and is prevented by re-reading the version immediately before upload. An unreviewable edit happens when the changeset comment names no source, script or contact, and is prevented by a comment template enforced in code. An irreversible edit happens when one changeset carries thousands of unrelated objects, and is prevented by splitting the work into small, single-purpose changesets that a reviewer can revert independently. Three failure modes of automated editing Version conflict Object changed between read and write API returns HTTP 409 Blind retry overwrites a mapper Fix: re-read version before upload Never bump the version yourself Unreviewable edit Comment names no source or script Reviewer has nobody to ask Trust cost lands on everyone Fix: template the comment in code Include source, script, contact Irreversible edit One changeset, thousands of objects Mixed unrelated changes Revert takes the good with the bad Fix: small single-purpose changesets Group by what a revert should undo All three are prevented by code you write once, and all three are normally discovered by somebody else after the fact.
The changeset is the unit of review and the unit of revert, so its size and scope are engineering decisions, not incidentals.

When Not to Query at All

The most valuable judgement in this section is knowing when the whole service layer is the wrong tool. A live query is right when the question is small, ad hoc, and needs current data. It is wrong when any of those three fail, and it is wrong in a way that gets worse the more successful your pipeline becomes, because a nightly job that queries a public server does not degrade gracefully — it works until somebody notices and blocks it.

The alternative is almost always a regional extract filtered locally. An osmium tags-filter pass over a country extract answers the same question as a large Overpass query, runs in seconds on a laptop, costs a shared server nothing, and is reproducible because the input file is a fixed artefact you can archive. The migration path from one to the other is mechanical, and Replacing an Overpass Query with an osmium Filter walks it end to end. Choosing Between Overpass and a Local Extract frames the decision itself.

The inverse case is real too. If your question covers a handful of features in a city and must reflect an edit made ten minutes ago, downloading a daily extract answers the wrong question no matter how efficiently you parse it. Freshness is a requirement like any other, and the replication machinery in OSM Replication & Diff Sync exists precisely because “current” and “local” are not mutually exclusive — they just cost more engineering than either one alone.

Client Engineering: the Parts Every Integration Needs

Whichever service you talk to, the client-side requirements are nearly identical, which is why it pays to build them once.

  • An identifying User-Agent. A string naming your project and a contact address. Anonymous traffic is the first thing an operator blocks, and a named client is the one that gets an email instead.
  • A response cache keyed on the exact request. Development reruns dominate real traffic in most projects; a disk cache keyed on the normalized query text removes nearly all of it and makes your test suite deterministic.
  • Bounded concurrency with a single shared limiter. One semaphore for the whole process, not one per worker, so adding workers never multiplies your request rate.
  • Retry with jittered exponential backoff, capped. Honour Retry-After when present; give up after a bounded number of attempts and surface the failure rather than looping.
  • A hard result-size guard. Refuse to materialise a response larger than an explicit ceiling, so a mis-scoped query fails fast rather than exhausting memory.
  • Structured logging of query, duration and element count. The three numbers that let you find the query that got you throttled, weeks later.

That list is short enough to implement in an afternoon and is the difference between an integration that survives contact with production and one that has to be rewritten the first time it is scheduled.

Validation and Error Handling

Service responses need the same defensive treatment as parsed files, and a few failure classes recur often enough to deserve named handling.

Condition Root cause Detection Remediation
Empty Overpass result A later statement overwrote the default set Result has zero elements but no error Bind sets explicitly with ->.name and union at the end
Overpass HTTP 504 Query exceeded the server timeout Gateway timeout after the declared [timeout:n] Narrow the spatial filter first, then raise the timeout
Overpass HTTP 429 Too many concurrent slots for your address Rate-limit response with Retry-After Back off, halve concurrency, cache successes
Nominatim returns the wrong country Unconstrained free-text search Result country_code differs from expectation Use a structured query plus countrycodes
Nominatim blocked Missing User-Agent or over one request per second Persistent 403 or 429 Add an identifying agent, throttle, or self-host
Changeset upload 409 Object version moved since you read it Conflict response naming the object Re-read the object, re-apply intent, never bump blindly
Downloaded extract truncated Interrupted transfer, no integrity check Parser fails at a blob boundary Verify the published checksum before use
Extract silently stale Mirror stopped updating File timestamp older than the expected cadence Assert freshness on the file date as a pipeline gate

The last two rows are worth dwelling on because they fail quietly. An HTTP error is loud and a monitoring system will catch it; a daily extract that stopped refreshing two weeks ago produces perfectly valid output derived from stale input, and the only defence is an explicit assertion on the file’s publication date before parsing begins.

Performance and Scale

Throughput on the service layer is governed by round trips, not by bytes. A query that returns ten megabytes in one response is almost always faster and cheaper than a hundred queries returning a hundred kilobytes each, because per-request overhead — queueing, planning, TLS, and the server’s own slot accounting — dominates. The practical patterns that follow from this are: batch spatially (one bounding box covering a work area rather than one per feature), batch by tag (one query with a union of filters rather than one query per amenity type), and prefer out center over out geom whenever a representative point is enough.

For geocoding, the corresponding rule is to deduplicate before you call. Address lists from real systems are heavily repetitive, and normalising case, whitespace and punctuation before the lookup routinely removes a third of the calls without changing a single result. Parsing Nominatim Address Details into Columns covers the structured response that makes this deduplication reliable on the way back out.

Beyond a few thousand requests an hour, the answer stops being tuning and starts being hosting. Running your own Overpass instance, described in Running a Local Overpass Instance for Bulk Queries, converts a quota problem into a capacity-planning problem, which is a much better problem to have.

Licensing and Attribution

Data retrieved through any of these services is OpenStreetMap data and carries the Open Database Licence exactly as a downloaded extract does. The service you used to fetch it changes nothing about your attribution obligation or about whether a derived database triggers share-alike. Because API-sourced data often arrives in small pieces and gets blended into other datasets, it is easier to lose track of its provenance than with a single downloaded file — which makes recording provenance at fetch time more important, not less. The obligations themselves are worked through in OSM Licensing & ODbL Compliance, and the mechanics of carrying provenance through a pipeline are in Recording OSM Data Provenance in a Pipeline.

Topics in This Section

Frequently Asked Questions

Is Overpass a REST API I can just call in a loop?

No. Overpass is a query engine with a planner, a per-query timeout and a memory ceiling, and public instances allocate concurrent slots per client address. A loop of small queries is both slower for you and far harder on the server than a single query that asks for the same data in one pass. Design the workload as few large queries with explicit set bindings, cache every successful response, and back off when the server returns a rate-limit status.

How many Nominatim requests per second is acceptable?

The public instance’s usage policy allows roughly one request per second from a single source, and requires a User-Agent that identifies your application. Above that, the correct answer is not a faster client but your own instance imported from a regional extract, which removes the limit entirely and makes results reproducible. Deduplicating addresses before lookup typically removes a large fraction of calls at no cost to correctness.

When should I download an extract instead of querying?

Whenever the question is repeated, covers a whole region, or will run on a schedule. A local extract filtered with osmium answers most large Overpass queries in seconds, costs a shared server nothing, and is reproducible because the input is a fixed file you can archive. Keep live queries for questions that are small, one-off, and genuinely need data fresher than the extract’s daily cadence.

Do I need permission to upload automated edits?

The API will accept an authenticated changeset without asking, but the community expects automated and bulk edits to be discussed beforehand, documented in the changeset comment with a source and contact, and kept small enough to revert independently. Dry-run against the development API first, re-read each object’s version immediately before upload, and never bump a version to force a conflicting change through.

Does data fetched from an API carry different licence terms?

No. Data retrieved through Overpass, Nominatim or the editing API is OpenStreetMap data under the Open Database Licence exactly as a downloaded file is. Because API responses are small and get blended into other datasets, provenance is easier to lose, so record the source, the query and the fetch time alongside the data at the moment you receive it rather than reconstructing it later.

Up one level: OSM Data Processing & QA Pipelines.