Serving & Invalidating OSM Tiles

Generating a tile set is a batch problem with a clear end. Serving one is an operational problem with no end, and the decisions that make it cheap or expensive are made at packaging time, before a single request arrives.

The layers a tile request passes through and where each one can hold a stale copy Five layers between the archive and the reader. The browser holds tiles in its own HTTP cache keyed on the URL. A content delivery network holds copies at edge locations worldwide. An origin cache or reverse proxy may hold a copy in front of the server. The tile server or object store holds the archive itself. The generator produced the archive from an extract at a point in time. Any of the first four can serve a stale tile after the archive is updated, which is why invalidation has to address each of them. Five layers, four of which can be stale Browser cache Per reader, keyed on the URL cleared by nobody Edge network Copies at edge locations purge by key or by tag Origin cache Reverse proxy in front purge or short TTL Archive The tiles you generated the source of truth The browser layer is the one you cannot purge, which is why a cache-busting element in the URL is worth building in from the start.
Every layer above the archive is a place a reader can be shown yesterday's map after you have fixed it.

The Problem This Topic Solves

You have an archive of tiles and readers who need to see them, on a map that must reflect the data without being regenerated from scratch every time an edit lands. The serving decision determines your ongoing cost and your operational surface; the invalidation decision determines whether the map is ever actually current.

The failure scenario is a map that is correct in the archive and wrong in the browser. A diff is applied, the affected tiles are regenerated, and readers continue to see the old ones — from a browser cache, from an edge location, or from a proxy nobody remembered was in the path. Nothing errors, the archive is demonstrably right, and the bug report says “the map is wrong” with a screenshot that cannot be reproduced.

Prerequisites

Understand the tile pyramid from OSM Vector Tiles & Rendering Pipelines, especially the ancestor relationship between zooms. Know how change files identify what moved, from Applying .osc Change Files with osmium. And have an archive to serve, from either Building OSM Tiles with Tippecanoe or Planetiler & Tilemaker Workflows.

Three Packaging Formats

MBTiles is a SQLite database holding tiles and metadata. It is the natural output of most generators, it is easy to inspect with ordinary tools, and it requires a process that can open the database to serve from it. Updating individual tiles is a straightforward row update, which makes it the friendliest format for incremental re-rendering.

PMTiles is a single file with a hierarchical index at its head, designed so a client can find and fetch one tile with a couple of HTTP range requests. That removes the server entirely: the archive sits in object storage behind a content delivery network and clients read it directly. The trade is that the file is immutable in practice — updating individual tiles means rewriting it — so it suits a tile set rebuilt as a unit.

Loose directories store one file per tile. Conceptually simple and trivially cacheable, but a continent at zoom 14 is hundreds of millions of objects, and most storage systems handle that number of small files badly. It remains reasonable for a small area or for a tile set with a shallow maximum zoom.

The three packaging formats compared on serving, updates and operational cost A grid of four properties against three formats. MBTiles needs a server process, supports updating individual tiles easily, produces one file, and suits a tile set that is re-rendered incrementally. PMTiles needs no server because clients read it by range request, is effectively immutable so updates mean a rewrite, produces one file, and suits a tile set rebuilt as a unit. Loose directories need only a static file server, allow trivial per-tile updates, produce hundreds of millions of objects, and suit small or shallow tile sets. Three formats, and updateability is the dividing line MBTiles PMTiles Loose files Needs a server yes no static only Update one tile easy rewrite trivial Object count one file one file hundreds of millions Natural fit incremental rebuilt whole small or shallow Choosing PMTiles for a tile set updated from minutely diffs means rewriting a multi-gigabyte file every few minutes.
The update row is what ties the packaging choice to the invalidation strategy, which is why they belong in one topic.

Invalidation: the Dirty Tile Computation

When a change file arrives, the great majority of tiles are unaffected. Working out which ones are not is the whole game, and it has three parts.

Locate the changes. Every created, modified or deleted element in the change file has a position — directly for a node, through its members for a way or relation. Those positions map to tile coordinates at the maximum zoom. The mechanics are in Computing a Dirty Tile List from an .osc File.

Expand to ancestors. A tile at zoom 14 has one parent at 13, one grandparent at 12, and so on to zoom 0. A change visible at 14 also changes the generalized representation at every lower zoom, so the dirty set must include the whole ancestor chain. Forgetting this produces the characteristic bug of a map that is right zoomed in and wrong zoomed out.

Expand for the buffer. A change near a tile boundary affects the neighbouring tile too, because that tile retains geometry past its edge. Including the eight neighbours at the deepest zoom is a cheap over-approximation that avoids a whole class of edge artefacts.

The result is a set of tile addresses to re-render and, separately, to purge from every cache layer.

How the dirty tile set grows as each expansion rule is applied Four counts for one minutely change file over a country. The raw changed elements map to a few hundred tiles at the deepest zoom. Adding the eight immediate neighbours of each, to account for buffer geometry, roughly quadruples that once overlaps are removed. Adding the full ancestor chain up to zoom zero adds about a third again, because each level contributes only a quarter as many tiles as the one below. The final set remains a vanishing fraction of the whole pyramid. Expansion is cheap; the pyramid is not Changed elements about 320 tiles Plus neighbours about 1,150 Plus ancestors about 1,520 Whole pyramid millions Even fully expanded the dirty set is a tiny fraction of the pyramid, which is why re-rendering everything is never the right answer.
Both expansions together add less than five times the base set, and both prevent a class of visible artefact.

Cache Layers and What Each Needs

Every layer between the archive and the reader needs its own invalidation, and they do not all support the same operations.

An edge network typically supports purging by URL or by a tag attached at response time. Tagging responses with the tile’s zoom and a coarse spatial key makes bulk purges possible without enumerating millions of URLs.

An origin cache — a reverse proxy in front of a tile server — usually supports purge by URL or a short time to live. For a tile set updated every few minutes, a time to live shorter than the update interval is simpler than wiring purges and costs little.

A browser cache cannot be purged at all. The only reliable control is the URL: including a version token in the tile path, incremented when the tile set is rebuilt, makes every tile a new URL and sidesteps the problem entirely. That is worth building in from the start, because retrofitting it means changing every style document.

Validation and Error Handling

Condition Root cause Detection Remediation
Map right zoomed in, wrong zoomed out Ancestor tiles not invalidated Low-zoom tiles predate the change Expand the dirty set up the whole ancestor chain
Stale tiles after a purge A cache layer nobody knew about Response headers name an unexpected cache Trace the full request path and purge each layer
Artefacts near changed features Neighbouring tiles not invalidated Seams appear only next to edits Include neighbours at the deepest zoom
Readers see old tiles for days Browser cache with a long time to live Only a hard refresh fixes it Put a version token in the tile URL
Rewrite takes longer than the update interval Immutable format with frequent updates Each rebuild overlaps the next Use an updateable format, or update less often
Purge API rate limited Enumerating millions of URLs Purge requests throttled Tag responses and purge by tag
Archive updated, server serves old data Server holds the file open Restart fixes it Swap archives atomically and signal the server

Performance and Scale

Tile serving is an unusually favourable workload: requests are read-only, responses are immutable for their lifetime, and the access distribution is extremely skewed — a small fraction of tiles serve the overwhelming majority of requests. That skew is what makes edge caching so effective and what makes origin capacity planning far less alarming than the tile count suggests.

Two consequences matter. Cache hit ratio is the metric, not origin throughput; a ratio in the high nineties is normal and achievable, and a drop in it is the earliest signal that something is wrong with cache keys or invalidation. And the long tail is cheap to serve slowly: tiles nobody looks at can be rendered on demand rather than pre-generated, which for deep zooms saves enormous amounts of generation time.

Compression matters too. Vector tiles compress well and clients accept gzipped payloads, so tiles should be stored compressed and served with the appropriate header rather than compressed per request.

Failure Modes and Gotchas

  • Ancestors are not optional. A dirty set without the ancestor chain leaves low zooms permanently behind.
  • Buffers make neighbours dirty. A change just inside one tile alters the geometry retained by the tile next door.
  • Browser caches are unpurgeable. Only a URL change reaches them, which is why versioned paths are worth the effort.
  • Immutable formats and frequent updates conflict. Pick one; a minutely-updated single-file archive is a rewrite treadmill.
  • Open file handles hide updates. A server holding an archive open keeps serving the old contents after the file is replaced.
  • Range requests need range support. Serverless delivery depends on the storage and every cache in front of it honouring range requests correctly.
  • An empty tile is not a missing tile. Returning a not-found status where a legitimately empty tile exists makes clients retry endlessly.

Integration Points

Upstream, the dirty tile list comes from the replication stream — Incremental Updates for Derived Datasets covers the general pattern and tiles are one instance of it. Downstream, the style document consumes the archive’s metadata, so the layer names and zoom range recorded at build time are part of the serving contract.

The two guides develop each half: Serving PMTiles from Object Storage for delivery, and Invalidating Tile Caches After an OSM Diff for keeping it current.

Guides in This Topic

Frequently Asked Questions

Why is my map correct when zoomed in and stale when zoomed out?

Because the invalidation computed the tiles a change touches at the deepest zoom and stopped there. A feature that changed at zoom 14 also appears, generalized, in the zoom 13 tile containing that one, and in its parent, and so on to zoom 0. The dirty set must include the whole ancestor chain of every touched tile — which is cheap, since each level contributes only a quarter as many tiles as the one below.

Do I need a tile server at all?

Not for a static base map. A single-file archive with an embedded index can be read directly by clients using HTTP range requests, so the whole serving stack becomes an object store and a content delivery network. A server earns its place when you need per-request behaviour — access control, filtering by user, dynamically composed layers — or when the tile set is updated incrementally and rewriting a large immutable file is impractical.

How do I invalidate a browser cache?

You cannot, which is why the answer has to be in the URL. Including a version token in the tile path — incremented whenever the tile set is rebuilt — makes every tile a new URL that no browser has cached, and lets you set a long cache lifetime on the old URLs without ever needing them to expire. Retrofitting this means updating every style document that references the tiles, so it is worth building in before the first reader arrives.

Should empty tiles be stored or omitted?

Omit them from storage and return a valid empty response for them, rather than a not-found status. Storing hundreds of millions of zero-length objects is a real cost in any storage system, and clients handle a missing tile as a transient failure to be retried rather than as an answer. An explicit empty tile — or a not-found the client is configured to treat as empty — avoids both problems.

What cache hit ratio should I expect?

High nineties for a map with ordinary usage, because tile requests are extremely skewed towards a small number of popular areas and zoom levels. That is also why the ratio is the metric to watch: a sudden drop usually means cache keys changed, a version token was introduced without the caches being warmed, or invalidation purged far more than it needed to.

Up one level: OSM Vector Tiles & Rendering Pipelines.