Extract Clipping & Boundary Polygons

Almost no production pipeline processes the planet. It processes a region — a country, a metropolitan area, a service territory drawn by a business rather than by a border — and the first stage of that pipeline is a cut: taking a larger .osm.pbf and producing a smaller one covering exactly the area of interest. The operation sounds like a spatial filter and is routinely implemented as one, which is where the trouble starts. OpenStreetMap objects are not independent geometries that can be tested individually against a polygon; they are a reference graph in which a way is nothing but a list of node identifiers, and a relation is nothing but a list of member identifiers. Cutting that graph with a polygon means deciding what to do with every edge that crosses the line, and the decision is not made for you.

This topic covers the mechanics of that cut. It defines the boundary formats OSM tooling accepts, walks the four osmium extract strategies and what each one keeps, quantifies the cost of choosing the wrong one, and sets out how to verify that the extract you produced is actually self-contained. It sits under OSM Data Fundamentals & Architecture because clipping is a format-level operation on the reference model described in the Node, Way & Relation Data Model — if the way-references-node relationship is unfamiliar, read that first, because every decision below is about preserving it.

How a clipping run decides what crosses the boundary A four-stage chain. A continent or planet extract is read. Each node is tested for membership of the boundary polygon, which is the only cheap test available. A reference policy such as complete_ways or smart then decides what happens to ways and relations that straddle the line. The output is either a self-contained extract or a referentially broken one, depending on that policy. What crosses the cut line, and what each strategy does about it input extract continent .osm.pbf or a planet file boundary test node inside the polygon? the only cheap test reference policy complete_ways · smart decides the edge cases output extract self-contained .osm.pbf or a referentially broken one Every clipping decision reduces to one question asked at the boundary: when a way has nodes on both sides, which of them come with it?
Clipping is not a spatial operation with a single right answer. The node test is unambiguous; everything interesting happens to the objects that reference nodes on both sides.

Prerequisite concepts

Three ideas from elsewhere on this site do most of the work here. The first is the reference model: ways store node identifiers, not coordinates, so a way is only renderable if every node it names is present in the same file. The second is the PBF File Structure Deep Dive view of a file as ordered blocks of nodes, then ways, then relations, which is what makes a two-pass clip possible without holding the file in memory. The third is the bounding box in the file header, described in the same page, which lets a pipeline reject an extract that cannot possibly cover the area requested before spending twenty minutes proving it.

Boundary formats: bbox, .poly and GeoJSON

The cheapest boundary is a bounding box, given as four numbers in the order left, bottom, right, top. It is exact, it costs one comparison per node, and it is almost always the wrong shape. Countries are not rectangles, and a bounding box around a country with an awkward outline pulls in large parts of its neighbours — a bbox around Norway includes most of Sweden, and a bbox around Chile includes most of Argentina.

The Osmosis .poly format is the format OSM tooling has settled on for real boundaries. It is a plain-text file: a name line, then one or more polygon sections, each opened by an identifier line, followed by coordinate pairs as longitude then latitude, one pair per line, closed by END, with the whole file closed by a final END. A section identifier prefixed with ! marks a hole rather than an outer ring, which is how enclaves are expressed. The format has no CRS declaration; coordinates are always WGS 84 degrees, the same convention described in Coordinate Reference Systems in OSM.

text 
ireland
1
   -1.06E+01   5.14E+01
   -5.30E+00   5.14E+01
   -5.30E+00   5.55E+01
   -1.06E+01   5.55E+01
   -1.06E+01   5.14E+01
END
END

Recent osmium versions also accept GeoJSON directly, which is usually easier to produce because it comes straight out of the tools that already hold your boundaries. A GeoJSON boundary must be a single Feature or FeatureCollection containing Polygon or MultiPolygon geometry; a collection of separate features is interpreted as a multi-region request rather than as one boundary, which is a useful behaviour and an easy accident.

Vertex count matters more than the format. Every node tested against the boundary pays a point-in-polygon cost proportional to the number of edges, so a boundary simplified from an administrative relation with 180 000 vertices to one with 2 000 vertices runs roughly two orders of magnitude faster and, for a clipping operation, describes the same region to well within the accuracy anyone needs. Simplify boundaries with a tolerance of around 100 metres before using them to cut.

The four strategies

osmium extract takes a --strategy flag with four values, and this flag decides what “inside” means.

The four osmium extract strategies compared A grid comparing simple, complete_ways, smart and referenced strategies on which ways are kept, which nodes are kept, and what the output looks like. Simple keeps only fully-inside ways and only inside nodes, producing the smallest file with geometry clipped at the edge. complete_ways keeps any way with at least one inside node together with all of that way nodes, producing geometrically complete features. Smart adds the relations those ways belong to and all their referenced members, producing the largest and relationally complete output. Referenced keeps the ways that chosen relations need. Four strategies, four different definitions of "inside" ways kept nodes kept output is simple only fully-inside ways only inside nodes smallest, clipped at the edge complete_ways any way with one inside node all nodes of those ways geometrically complete smart complete_ways + their relations all referenced nodes relationally complete, largest referenced ways referenced by kept relations all referenced nodes complete for chosen relations The difference between the first and the third is roughly 8 percent of file size on a country cut, and the difference between a usable coastline and a shredded one.
The strategy is the single most consequential flag in a clipping run, and the default is not the one most pipelines want.

The simple strategy keeps a node if it is inside the polygon and a way only if every one of its nodes is inside. It is the fastest and produces the smallest file, and it truncates every feature that crosses the boundary. A coastline way running along the edge of the region disappears entirely; a road crossing the border is dropped rather than clipped. Use it only when the consumer genuinely does not care about edge features — a point-of-interest count, say.

The complete_ways strategy keeps a way if any of its nodes is inside, and then keeps every node that way references, including the ones outside the polygon. This is the strategy most pipelines actually want: features are geometrically complete, no way has a dangling reference, and the file is a few percent larger than simple. The output extends slightly beyond the boundary wherever a long way crosses it, which is a property to know about rather than a defect.

The smart strategy does everything complete_ways does, then additionally keeps relations that reference the kept objects and, in turn, the members those relations need. This is what makes multipolygon buildings, route relations and boundary relations survive a cut intact. It produces the largest output and takes the longest, and it is the correct choice whenever relations matter — which includes any routing, boundary or landuse work.

The referenced strategy inverts the question: it starts from relations you want and pulls in the ways and nodes they need. It is the right tool for extracting a specific set of features, such as one bus network, rather than a geographic area.

Output size and referential integrity by clipping strategy A bar chart of output size cutting Ireland from a 28.4 gigabyte Europe extract. The simple strategy produces 190 megabytes with 4100 broken way references. complete_ways produces 205 megabytes with no broken way references. Smart produces 224 megabytes with no broken references of any kind. The unclipped Europe input is shown at 28.4 gigabytes for scale. What each strategy costs on the same boundary Ireland cut from the Europe extract (28.4 GB input) simple 190 MB · 4 100 broken way references complete_ways 205 MB · 0 broken way references smart 224 MB · 0 broken references at all no clip (Europe) 28.4 GB · the input, for scale The gap between the cheapest and the correct strategy is 15 MB on a 200 MB output — under 8 percent, for the difference between broken and complete geometry.
The economics are not close. Referential completeness costs single-digit percentages of output size and removes an entire class of downstream failure.

The numbers make the choice straightforward. Cutting Ireland out of a Europe extract, simple saves fifteen megabytes over complete_ways on a two-hundred-megabyte output and hands the next stage four thousand broken way references. Nothing downstream benefits from that trade.

Running the cut

A single-region cut is one command. The --strategy flag is the important one; the rest is plumbing.

bash 
osmium extract \
  --polygon boundaries/ireland.poly \
  --strategy=smart \
  --output-header="osmosis_replication_base_url=https://planet.osm.org/replication/minute/" \
  --overwrite \
  -o extracts/ireland.osm.pbf \
  europe-latest.osm.pbf

Two details in that command are worth dwelling on. --overwrite is required because osmium refuses to clobber an existing output, which is correct behaviour and inconvenient in an automated pipeline; set it deliberately rather than discovering it at 03:00. And --output-header is how replication metadata survives the cut. By default an extract carries the header fields of its parent, and if the parent was itself an extract with no replication anchor, the child has none either, which means the resulting file cannot be caught up by the diff-sync workflow described in OSM Replication & Diff Sync. Set the base URL and, where you know it, the sequence number, at cut time.

Cutting many regions from the same input should be one pass, not many. osmium extract accepts a JSON configuration listing every output and its boundary, and reads the input once:

json 
{
  "directory": "extracts",
  "extracts": [
    { "output": "ireland.osm.pbf",  "polygon": { "file_name": "boundaries/ireland.poly",  "file_type": "poly" } },
    { "output": "scotland.osm.pbf", "polygon": { "file_name": "boundaries/scotland.geojson", "file_type": "geojson" } },
    { "output": "wales.osm.pbf",    "bbox": [-5.35, 51.35, -2.63, 53.44] }
  ]
}
bash 
osmium extract --config extracts.json --strategy=smart europe-latest.osm.pbf

The saving is not marginal. Twelve separate runs over a 28 GB input read 336 GB; one configured run reads 28 GB. On network-backed storage that is the difference between a nightly job and a job that does not finish overnight.

Validation and error-handling matrix

Condition Root cause How it surfaces Action
Ways render as straight lines between distant points simple strategy dropped intermediate nodes Visible only when rendered Re-cut with complete_ways
Multipolygon buildings missing their holes Relation members outside the polygon were dropped Courtyards filled in Re-cut with smart
Output is empty but the command succeeded Boundary coordinates in latitude, longitude order Zero objects written, exit code 0 Swap to longitude, latitude
Extract cannot be caught up with diffs Replication header not carried through osmium fileinfo shows no sequence Re-cut with --output-header
Cut takes hours on a small region Boundary has tens of thousands of vertices CPU-bound, single core pinned Simplify the boundary first
Coastline broken along the edge Coastline ways truncated at the boundary Rendering shows land bleeding into sea Use complete_ways and re-run the coastline build

The empty-output row deserves emphasis because it is silent. A .poly file written in latitude, longitude order describes a polygon somewhere off the coast of Somalia for most European regions, and osmium will happily cut it, write a valid PBF containing nothing, and exit zero. The check is trivial — assert that the output object count is non-zero — and it is not done by default.

Performance and scale considerations

Clipping is dominated by reading the input, not by the geometric test. A well-simplified boundary costs a few hundred nanoseconds per node; reading and inflating a 28 GB input costs minutes. Three consequences follow. First, cut from the smallest input that contains your region: cutting a German state out of the Germany extract rather than out of Europe reads 3 GB instead of 28 GB. Second, batch every region you need into one configured run. Third, spend effort on boundary simplification only when a profile shows the point-in-polygon test actually mattering, which it does only for pathological boundaries.

Memory behaviour differs sharply by strategy. simple streams in constant memory. complete_ways and smart need two passes and an identifier set: the first pass determines which ways and relations are wanted, and the second collects the nodes they reference. That identifier set is the memory cost, and it scales with the number of objects near the boundary rather than with the size of the input, so a long, convoluted boundary costs more than a compact one of the same area.

Failure modes and gotchas

The subtle failure is the one that produces a file that looks right. An extract cut with simple opens fine, renders mostly correctly, and passes a naive object count. Its damage shows up two stages later, when a routing graph built from it has roads that stop at the boundary and a connectivity check reports components that do not exist in the real network — a defect the Routing-Graph Topology QA topic treats as a data-quality problem when it is actually a clipping problem.

A second gotcha concerns re-cutting. Applying diffs to a clipped extract does not keep the clip honest: an object edited upstream so that it now falls inside your boundary will not appear in your extract, because the diff stream contains the edit but your file never had the object. Over months, a diff-updated regional extract drifts away from what a fresh cut of the same boundary would contain. The fix is to re-cut periodically from a current parent rather than to trust the diff stream to maintain the boundary.

Third, boundaries that cross the antimeridian are handled by neither .poly nor GeoJSON in a way tools agree on. Split such a region into two polygons either side of 180 degrees and merge the outputs.

Integration points

The output of a clip is the input to everything else on this site. Feed it to the parsers surveyed in Choosing an OSM Parser; anchor it to a replication stream using the header fields covered in Replication Sequence Numbers & State Tracking; index it using one of the schemes compared in Spatial Index Selection.

The wiring that matters is a verification step between the cut and everything after it:

python 
import logging
import subprocess
import json

logger = logging.getLogger(__name__)

def verify_extract(path: str, min_nodes: int = 1) -> dict[str, int]:
    """Assert a freshly cut extract is non-empty and carries a replication anchor."""
    raw = subprocess.run(
        ["osmium", "fileinfo", "--extended", "--json", path],
        capture_output=True, text=True, check=True,
    ).stdout
    info = json.loads(raw)
    counts = info["data"]["count"]
    if counts["nodes"] < min_nodes:
        raise ValueError(f"{path}: {counts['nodes']} nodes — boundary is probably inverted")
    header = info["header"]["options"]
    if "osmosis_replication_base_url" not in header:
        logger.warning("%s: no replication anchor — this extract cannot be caught up", path)
    logger.info("%s: %d nodes, %d ways, %d relations",
                path, counts["nodes"], counts["ways"], counts["relations"])
    return counts

In this section

Frequently Asked Questions

Which strategy should I use if I am not sure?

Use smart. It is the only strategy that guarantees no dangling references of any kind, including relation members, and on a country-sized cut it costs under twenty percent more output size than the cheapest option. The cases where a smaller strategy is genuinely better are narrow — a point-of-interest count that ignores geometry, or a disk budget so tight that a few percent matters — and in those cases you will know.

Why does my extract contain objects outside the boundary polygon?

Because complete_ways and smart deliberately keep them. A way with one node inside the polygon is kept entire, which means every node of that way comes too, including the ones several kilometres outside. This is what makes the feature renderable and routable. If you need output strictly bounded, clip the resulting geometries in your own processing rather than asking the extractor for it.

Can I clip a file that has already been clipped?

Yes, and it is often the fastest route — cutting a city out of its country extract reads far less data than cutting it out of a continent. The caveat is referential: if the parent was cut with simple, it already has dangling references, and no strategy applied to it can restore the nodes that are not there. Cut from parents produced with complete_ways or smart.

Does clipping preserve the replication sequence number?

Only if you ask for it. osmium extract copies the parent header, so the sequence survives when the parent had one, but many published extracts do not carry one. Set --output-header explicitly with the base URL and, where known, the sequence number and timestamp, so the extract can be caught up later rather than only re-cut.

How do I clip to an administrative boundary from OSM itself?

Extract the boundary relation, assemble it into a polygon, simplify it, and write it out as GeoJSON. The assembly is the multipolygon problem described in Understanding OSM Multipolygon Relations for GIS — administrative boundaries are frequently open or mis-roled and need the same containment-based ring classification as any other multipolygon.

Up one level: OSM Data Fundamentals & Architecture.