Serving PMTiles from Object Storage

Put one file in a bucket, point a map at it, and have no tile server to operate — provided the storage, the edge network and the headers all cooperate on one thing: HTTP range requests.

Prerequisites

Conceptual minimum

A PMTiles archive is one file laid out so that a client can find any tile with a small number of byte-range reads. Its head holds a header and a hierarchical directory; tile data follows. A client reads the header once, walks the directory — usually one or two additional range reads, cached thereafter — and then fetches the tile’s bytes directly.

Everything that makes this work is a property of the transport rather than of the file. Range requests must be honoured end to end: by the object store, by the edge network, and by any proxy in between. A layer that answers a range request with the whole file turns each tile fetch into a multi-gigabyte download. CORS headers must permit the range header from browser clients, which is a separate allowance from permitting the request itself.

The consequence of the single-file design is immutability in practice. Updating one tile means rewriting the archive, so PMTiles fits a tile set rebuilt as a unit and fits a minutely-updated one badly.

What a client does to fetch one tile from a single remote archive Four steps. The client first requests a small byte range covering the header, learning the archive's zoom range, bounds and the location of the root directory. It then requests the root directory range, which maps tile addresses to either leaf directories or tile locations. For deep archives it requests one leaf directory range. Finally it requests the byte range holding the tile itself. The first three responses are cached, so subsequent tiles from the same area cost a single range request each. Three reads to warm up, then one per tile header a few hundred bytes zoom range and bounds root directory one range read cached thereafter leaf directory deep archives only also cached tile bytes one range read the steady state Once the directories are cached the steady-state cost is identical to a conventional tile server: one request per tile.
The warm-up is three requests for the whole session, which is why the format is viable at all.

Runnable solution

bash 
#!/usr/bin/env bash
# Convert an MBTiles archive to PMTiles and publish it.
set -euo pipefail

SRC="${1:?usage: publish.sh <archive.mbtiles>}"
BASE="$(basename "$SRC" .mbtiles)"
VERSION="$(date -u +%Y%m%dT%H%M%SZ)"       # a version token in the object key
DST="${BASE}-${VERSION}.pmtiles"
BUCKET="s3://tiles.example.org"

pmtiles convert "$SRC" "$DST"
pmtiles show "$DST"                         # prints zoom range, bounds, layers

# Tiles are immutable per version, so they can be cached effectively forever.
aws s3 cp "$DST" "$BUCKET/$DST" \
  --content-type application/octet-stream \
  --cache-control "public, max-age=31536000, immutable"

echo "https://tiles.example.org/$DST"
python 
from __future__ import annotations

import logging
import sys

import requests

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
logger = logging.getLogger("osm.pmtiles.verify")

ORIGIN = "https://map.example.org"      # the site that will embed the map


def check(url: str) -> bool:
    ok = True

    head = requests.head(url, timeout=30)
    head.raise_for_status()
    size = int(head.headers.get("Content-Length", 0))
    logger.info("archive is %.1f MiB", size / (1 << 20))
    if head.headers.get("Accept-Ranges") != "bytes":
        logger.error("origin does not advertise byte ranges")
        ok = False

    # The decisive test: ask for 16 bytes and insist on getting 16 bytes.
    ranged = requests.get(url, headers={"Range": "bytes=0-15"}, timeout=30)
    if ranged.status_code != 206:
        logger.error("range request returned %d, expected 206 Partial Content",
                     ranged.status_code)
        ok = False
    elif len(ranged.content) != 16:
        logger.error("range request returned %d bytes, expected 16 — a layer "
                     "in the path is collapsing ranges", len(ranged.content))
        ok = False
    else:
        logger.info("range requests honoured; magic bytes %r", ranged.content[:7])

    # Browsers send a preflight for the Range header; it must be allowed.
    pre = requests.options(url, headers={
        "Origin": ORIGIN,
        "Access-Control-Request-Method": "GET",
        "Access-Control-Request-Headers": "range",
    }, timeout=30)
    allowed = pre.headers.get("Access-Control-Allow-Headers", "").lower()
    if "range" not in allowed:
        logger.error("CORS preflight does not allow the Range header (got %r)",
                     allowed or "nothing")
        ok = False
    expose = pre.headers.get("Access-Control-Expose-Headers", "").lower()
    if "content-length" not in expose and "*" not in expose:
        logger.warning("Content-Length not exposed to the browser; some "
                       "readers need it")
    return ok


if __name__ == "__main__":
    logger.info("verification %s", "PASSED" if check(sys.argv[1]) else "FAILED")

Step-by-step walkthrough

  1. Put the version in the object key. A dated file name makes every rebuild a new URL, which lets the archive be cached indefinitely and removes the browser-cache problem entirely.
  2. Set an immutable cache policy. Because the key changes on every rebuild, the object behind it never changes, so a long lifetime with an immutable directive is both safe and the cheapest possible configuration.
  3. Inspect after converting. The conversion reports the zoom range, bounds and layer list; comparing that against what the build intended catches a truncated or wrong archive before it is uploaded.
  4. Test the range request, not the header. Advertising byte-range support and honouring it are different things. Asking for sixteen bytes and counting what arrives is the only test that distinguishes them.
  5. Insist on a partial-content status. A layer that answers a range request with the whole file returns a normal success status, which is exactly the failure this check exists to find.
  6. Check the CORS preflight for the range header. Browsers send a preflight naming Range; a configuration that allows the origin but not that header fails only in a browser, never in a command-line test.
  7. Watch for exposed headers. Some client readers need Content-Length visible to the page, which requires it to be explicitly exposed in the CORS response.
Three configuration failures that only appear in a browser Three panels. A collapsed range happens when a layer in the path answers a range request with the whole file, which returns a success status rather than partial content and downloads gigabytes per tile. A blocked preflight happens when the CORS configuration allows the origin but not the range header, which works from a command line and fails silently in a browser. A missing exposed header happens when content length is not made visible to page scripts, which some readers require to locate directories. Three failures a command-line test will not find Collapsed range Layer returns the whole file Status 200, not 206 Gigabytes per tile fetched Test: ask for 16 bytes Count what arrives Blocked preflight Origin allowed, header not Browsers preflight Range Works from the shell Fails silently in a page Test: send a preflight Hidden headers Content-Length not exposed Reader cannot locate data Map loads then stalls Test: check expose header Add it to the CORS policy All three produce a map that fails to load with no server-side error: every request involved succeeded from the origin's point of view.
These are the reasons a serverless tile set is a configuration problem rather than a code problem.
What each layer in the delivery path must do for a serverless tile set to work A grid of four delivery layers against the two capabilities each must provide. Object storage must honour byte ranges and must attach the CORS policy. The content delivery network must forward and cache ranged responses correctly and must not strip CORS headers. Any reverse proxy in the path must pass range headers through untouched. The browser must be permitted to send the range header by the CORS preflight. A note observes that a single layer failing either capability breaks the whole arrangement. Four layers, and every one of them can break it Must honour ranges Must pass CORS Object storage yes, natively policy attached here Edge network forward and cache must not strip Reverse proxy pass through pass through Browser sends the header preflights it first Testing against the origin alone proves nothing: the failure is almost always introduced by a layer added between the origin and the reader.
Because one broken layer breaks everything, the verification has to run against the public URL a reader will actually use.

Verification

  • A sixteen-byte range returns sixteen bytes. Anything else means a layer in the path is collapsing ranges.
  • The status is partial content. A plain success status on a range request is the signature of the same problem.
  • The preflight allows the range header. Test with an options request carrying the origin and the requested header.
  • The archive’s metadata matches the build. Zoom range, bounds and layer list should be exactly what the generator produced.
  • A cold client loads in a handful of requests. Watch the network panel: header, directory, then tiles. A long sequence of directory reads means the archive is deeply nested and may benefit from reclustering.

Common errors and fixes

Symptom Root cause One-line fix
Whole archive downloaded per tile A layer collapses range requests Test with a small range and require partial content
Map loads from the shell, not the browser CORS preflight rejects the range header Allow Range in the allowed-headers policy
Map loads then stalls Content-Length not exposed to scripts Expose it in the CORS response
Readers see a stale archive Object key reused across rebuilds Put a version token in the object key
Archive rewritten every few minutes Immutable format with incremental updates Use an updateable archive format instead
Client reports an unsupported version Archive written by an older tool Convert with a current toolchain and re-inspect
High cost per request Cache lifetime too short for immutable objects Set a long lifetime and rely on versioned keys

Specification reference

A PMTiles archive is a single file beginning with a fixed-size header followed by a hierarchical directory structure and then tile data, designed so that a client can locate and retrieve an individual tile using HTTP range requests without a server-side component. Clients require the storage layer and any intermediaries to honour range requests and, for browser use, to permit the Range header through CORS. See the PMTiles specification for the header layout and directory format.

Frequently Asked Questions

Why does my map work from the command line but not in a browser?

Almost always CORS. A browser sends a preflight request naming the Range header before it will issue a ranged fetch to another origin, and a configuration that permits the origin but not that specific header rejects the preflight. Command-line clients do not preflight, so the same archive works perfectly from a shell. Test with an explicit options request carrying the origin and the requested header.

How do I know range requests are really being honoured?

Ask for a small range and count the bytes that come back. A layer that ignores ranges returns the entire file with an ordinary success status, which looks like success to every naive check. Requiring a partial-content status and exactly the requested number of bytes distinguishes the two unambiguously, and it is worth running against the public URL rather than the origin, because an edge network is a common place for ranges to be lost.

Can I update a single tile in a published archive?

Not practically. The format is laid out for read efficiency rather than in-place modification, so changing a tile means rewriting the file. That is fine for a tile set rebuilt as a unit on a daily or weekly cadence, and unworkable for one updated from minutely diffs. If your tile set changes continuously, use an updateable archive behind a server instead.

What cache lifetime should I set?

A very long one, provided the object key carries a version token. Because a rebuild produces a new key, the object behind any given URL never changes, so there is no reason for a cache anywhere in the path ever to revalidate it. That combination — versioned keys plus an immutable, long-lived cache policy — is what makes serverless tile delivery cheap.

Up one level: Serving & Invalidating OSM Tiles.