Choosing complete_ways vs smart in osmium extract

Two of the four osmium extract strategies produce referentially sound output, so the choice between them is not about correctness in the abstract — it is about which kind of completeness your consumers need, and what the safer option costs.

Prerequisites

Conceptual minimum

Both strategies handle ways identically. A way with at least one node inside the boundary is kept, and every node it references comes with it, including the nodes outside. That is what makes both of them safe for geometry: no way in the output has a reference the file cannot resolve.

They diverge on relations, and only on relations.

complete_ways against smart, row by row A grid comparing the two strategies. Both keep any way with a node inside the boundary and all of that way nodes. complete_ways keeps a relation only if one of its members happens to be inside and does not pull in the rest of its members. Smart keeps such relations whole and pulls in every member they reference. complete_ways needs two passes and produces a smaller file; smart needs three passes and is typically nine percent larger. The difference is relations, and only relations complete_ways smart ways with a node inside kept kept nodes of those ways all of them all of them relations referencing them only if a member is inside kept whole members of those relations not pulled in pulled in passes required two three output size smaller 9% larger typically Both produce renderable ways. Only smart produces complete multipolygons, route relations and boundary relations.
Everything below the second row is the decision. If nothing downstream reads relations, complete_ways is enough; if anything does, smart is the only correct answer.

complete_ways keeps a relation when one of its members happens to have been kept for its own reasons, but it does not go looking for that relation’s other members. smart does: it treats a partially-included relation as something to complete, and pulls in whatever ways and nodes are needed to make it whole. The extra pass is where the extra time goes.

What complete_ways actually breaks

Three feature classes that complete_ways leaves incomplete Three panels of failure. Multipolygon buildings: the outer ring is inside and an inner ring outside, so the hole member is dropped, the courtyard fills in solid and area is over-reported, while the result renders plausibly. Route relations: a bus route crossing the boundary loses its member ways beyond the edge, so the route appears to terminate mid-street and the itinerary is silently truncated. Boundary relations: an administrative boundary whose member ways are partly outside no longer closes, cannot be assembled into a polygon, and breaks the next clip that uses it. What breaks under complete_ways, concretely Multipolygon buildings Outer ring inside, inner ring outside Relation kept, hole member dropped Courtyard fills in solid Area over-reported Renders plausibly — no error Route relations Bus route crossing the boundary Member ways beyond the edge dropped Route appears to terminate mid-street Itinerary silently truncated Valid geometry, wrong data Boundary relations Admin boundary as a relation Member ways partly outside Boundary no longer closes Cannot be assembled to a polygon Breaks the next clip that uses it All three failures produce valid, renderable output. None of them raises an error anywhere in the pipeline.
The common shape is that the damage is to data, not to structure — which is why no validator catches it and the map still draws.

The three failure classes share a property that makes them expensive to discover: the output is structurally valid. Every reference resolves, every way has its nodes, the file passes osmium check-refs cleanly, and a renderer draws it without complaint. The damage is semantic, and it surfaces two or three stages downstream — as an over-reported building area, a routing itinerary that stops at a boundary, or a boundary relation that cannot be assembled into the polygon you wanted to use for the next clip.

The multipolygon case is worth being concrete about because it is the most common. A building with a courtyard is a relation with an outer ring and an inner ring. If the outer ring has a node inside your boundary and the inner ring does not, complete_ways keeps the outer way, keeps the relation because one member survived, and does not fetch the inner way. Assembling that relation with the containment logic from Understanding OSM Multipolygon Relations for GIS then yields a solid polygon where a courtyard should be.

What smart costs

Size and time penalty of smart over complete_ways on four regions A bar chart of the percentage size increase from choosing smart. Ireland cut from Europe grows from 205 to 224 megabytes, 9.3 percent, taking 41 seconds longer. Bavaria from Germany grows from 412 to 441 megabytes, 7.1 percent, 26 seconds longer. Greater London from Great Britain grows from 118 to 132 megabytes, 11.8 percent, 14 seconds longer. Kenya from Africa grows from 188 to 200 megabytes, 6.4 percent, 33 seconds longer. The cost of the safer strategy, measured four regions cut from their parent, smart against complete_ways Ireland from Europe 205 → 224 MB · +9.3% · +41 s Bavaria from Germany 412 → 441 MB · +7.1% · +26 s Greater London from GB 118 → 132 MB · +11.8% · +14 s Kenya from Africa 188 → 200 MB · +6.4% · +33 s Bar length is the percentage size increase. The largest observed penalty is under twelve percent, and under a minute of extra wall-clock.
This is the entire cost of never having to explain a filled-in courtyard or a truncated bus route.

Across four regions of very different shape and density, the penalty for smart is between six and twelve percent of output size and well under a minute of wall-clock. There is no case in that spread where the saving would change a capacity decision.

Deciding

The question that settles it is not about the data but about the consumers. Ask what reads the extract:

python 
NEEDS_RELATIONS = {
    "routing graph build",        # turn restrictions are relations
    "administrative boundaries",  # boundaries are relations
    "landuse / natural areas",    # frequently multipolygons
    "public transport",           # routes are relations
    "buildings with courtyards",  # multipolygons
}
NO_RELATIONS_NEEDED = {
    "point-of-interest counts",   # nodes only
    "address geocoding",          # nodes and simple ways
    "street-name extraction",     # ways only
}

If any consumer is in the first set — now or plausibly in the next year — use smart. If every consumer is in the second set, complete_ways is a legitimate saving, and it is worth documenting in the pipeline why, so that the person who later adds a routing build knows to revisit it.

bash 
# The comparison, if you want to see it on your own region
for s in complete_ways smart; do
  /usr/bin/time -f "$s: %e s" osmium extract \
    --polygon boundaries/ireland.poly --strategy="$s" \
    --overwrite -o "extracts/ireland-$s.osm.pbf" europe-latest.osm.pbf
done
osmium fileinfo -e extracts/ireland-complete_ways.osm.pbf | grep -E 'Number of|Size'
osmium fileinfo -e extracts/ireland-smart.osm.pbf         | grep -E 'Number of|Size'

The relation counts are the interesting line in that output. A smart cut of a country typically carries between two and five percent more relations than a complete_ways cut, and those are precisely the ones that straddle the boundary.

Verification

Check that relations survived rather than that the file is bigger. osmium check-refs with the relation flag reports members that are referenced but absent:

bash 
osmium check-refs --check-relations extracts/ireland-smart.osm.pbf

On a smart cut this reports no missing members. On a complete_ways cut of the same boundary it reports the incomplete relations, and the count it gives is a direct measure of what you would have been shipping.

Common errors and fixes

Symptom Root cause Fix
Courtyards filled in on buildings Inner ring member dropped Re-cut with smart
Bus routes stop at the region edge Route members outside not pulled in Re-cut with smart
check-refs reports missing relation members complete_ways used Re-cut with smart, or accept and document
smart run runs out of memory Third pass identifier set on a planet input Cut from a smaller parent, or raise the memory limit
No size difference between the two Region has no boundary-straddling relations Nothing wrong — a small or interior region

The memory row is the one genuine argument against smart: the extra pass holds an identifier set proportional to the relations near the boundary, and on a planet-sized input with a long boundary that set is large. Cutting from a continent rather than the planet removes the problem.

Specification reference

complete_ways — “Include all nodes referenced by ways that have at least one node in the region.” smart — “Like complete_ways, but also include all relations that have at least one member in the region, and all members of those relations.” The strategies differ only in their treatment of relations; both guarantee that every way in the output has all of its nodes.

Frequently Asked Questions

Is smart ever the wrong choice?

Rarely, and for reasons of resources rather than correctness. Its third pass holds an identifier set proportional to the relations near the boundary, so cutting a long, convoluted boundary directly out of the planet can exhaust memory where complete_ways would not. The fix is almost always to cut from a smaller parent rather than to drop to a weaker strategy — a country cut from its continent costs a fraction of the same cut from the planet.

How do I tell whether an existing extract was cut with complete_ways?

Run osmium check-refs --check-relations over it. A smart cut reports no missing relation members; a complete_ways cut of a region with any boundary-straddling relations reports them, and the count tells you how much was truncated. The file header does not record the strategy, so this behavioural test is the only reliable way to find out after the fact.

Does the strategy affect whether diffs can be applied later?

Not directly — applying a diff needs a replication anchor in the header, not a particular strategy. Indirectly it does matter, because a diff that modifies a relation whose members your extract never had will apply cleanly and leave the relation still incomplete. Starting from a smart cut means later edits land on complete objects.

What about the referenced strategy — where does it fit?

It answers a different question. referenced starts from a set of relations you name and pulls in the ways and nodes those relations need, which is the right tool for extracting one transport network or one set of administrative boundaries regardless of geography. It is not an alternative to the two strategies here, which are both geographic.

Can I mix strategies across regions in one run?

No. --strategy is a property of the run, not of an entry in the config, so a batch that needs smart for some regions must use it for all of them. Given the measured penalty is under twelve percent, the practical answer is to use smart for the whole batch rather than splitting the run in two.

Recording the decision

Whichever strategy a pipeline settles on, the choice belongs in the extract’s own header rather than only in the script that produced it. osmium extract --output-header accepts arbitrary key-value pairs, and writing the strategy into one of them means that six months later a colleague looking at an unfamiliar .osm.pbf can find out how it was cut without behavioural testing. It costs one flag and removes an entire category of archaeology.

The same argument applies to the boundary. Recording a hash or a version of the boundary file alongside the strategy makes an extract fully reproducible: strategy plus boundary plus parent sequence number is enough to recreate the file byte for byte, and any two of the three is not.

Up one level: Extract Clipping & Boundary Polygons.