Geometry Validation & Repair Jump to heading#

A way in OpenStreetMap carries no coordinates of its own — it is an ordered list of node references, and its geometry exists only after a parser resolves those references and closes the ring. That reconstruction step is where geometry goes wrong. A building traced as a figure-eight, a coastline way whose final node drifted a centimetre off its first, a multipolygon whose inner ring was digitised outside its outer: none of these are visible in the tag dictionary, and all of them pass straight through parsing to detonate later. The failure is rarely loud. An invalid polygon silently returns the wrong answer from a contains test, inflates or zeroes an area computation, or makes a PostGIS ST_Union abort an hours-long load with TopologyException: side location conflict. This guide sits inside the OSM Data Quality & Validation section and treats geometry as a gate: every reconstructed feature is classified as valid, repairable, or quarantine-worthy before it is allowed downstream.

Prerequisites Jump to heading#

This guide assumes you can already turn OSM primitives into candidate geometries and that you have somewhere to send the ones that fail. Three foundations matter. Reconstruction correctness comes first: the ring-assembly and right-hand-rule winding described in Understanding OSM Multipolygon Relations for GIS determine whether a relation even has a coherent geometry to validate. Second, you need a spatial index — the containment tests that decide whether an inner ring truly sits inside its outer, or whether two members overlap, run in acceptable time only against the Spatial Indexing for OSM Extracts structures rather than an all-pairs scan. Third, geometry validation is a producer of failures, so it needs the quarantine and dead-letter discipline from Error Handling in Large OSM Extracts: a defective polygon must never abort the run, and a silently mangled repair is worse than an honest rejection.

The Geometry Defects OSM Reconstruction Produces Jump to heading#

Six defect classes account for nearly all invalid OSM geometry. They are worth naming precisely, because the correct remediation differs sharply between them and a blanket repair call papers over distinctions that matter.

• Self-intersections. A ring whose boundary crosses itself — the classic “bowtie” from a mapper who traced vertices out of order, or a “figure-eight” where a building outline loops back through its own edge. The ring has a well-defined vertex list but no consistent interior, so area and point-in-polygon tests are meaningless.
• Unclosed rings. A way tagged as an area (or used as a multipolygon member) whose first and last node references are not identical. Often the last node sits a nanodegree or two off the first — a near-miss from an editor that failed to snap — so the ring looks closed on screen but fails the closure invariant a polygon constructor demands.
• Spikes and duplicate consecutive nodes. Two identical consecutive coordinates create a zero-length segment; three collinear nodes with a back-and-forth create a zero-area spike. Neither changes the visible shape much, but both break topology operations and inflate vertex counts.
• Degenerate polygons. A linear ring needs at least four coordinates (three distinct corners plus the repeated closing node) to bound any area. A way with fewer collapses to a line or a point and cannot legally become a Polygon.
• Wrong ring winding. The OGC and GeoJSON conventions expect a specific orientation — exterior rings counter-clockwise, holes clockwise in the right-hand-rule world that GIS engines assume. OSM imposes no winding, so a reconstructed outer ring may arrive clockwise, which some consumers read as a hole.
• Inner rings outside their outer. In a multipolygon, an inner member must lie within an outer member to punch a hole. When role tags are wrong or ring assignment during reconstruction pairs the wrong rings, an “inner” ends up beside or overlapping its outer, producing an invalid polygon-with-hole.

These are structural defects in the reconstructed shape, distinct from attribute problems. Bad tags — a deprecated key, a malformed maxspeed — are handled elsewhere; here the concern is purely whether the coordinate sequence describes a lawful geometry under the OGC Simple Features model.

Validity Specification: What “Valid” Actually Means Jump to heading#

Shapely implements the OGC Simple Features definition of validity, and it pays to know the exact predicate each method answers rather than treating them interchangeably.

The distinction between is_valid and is_simple trips people up. is_simple applies to curves and asks whether a line self-intersects; is_valid applies to areal geometry and additionally enforces ring nesting and orientation-independent interior consistency. A Polygon built from a self-crossing shell is invalid; the same coordinates as a LineString are merely non-simple. For a polygon the OGC rules are strict: the exterior and every interior ring must be closed LinearRings, rings may touch only at single points (never along a segment), and interior rings must lie inside the exterior and not nest inside one another. Any violation makes is_valid return False, and explain_validity names which rule broke and where.

$$A_{\text{ring}} = \tfrac{1}{2}\left| \sum_{i=0}^{n-1} \left( x_i\,y_{i+1} - x_{i+1}\,y_i \right) \right|$$

The signed form of that shoelace sum — dropping the absolute value — also gives winding: a positive signed area means counter-clockwise, negative means clockwise, which is exactly how a re-orientation step decides whether an outer ring needs flipping.

Step-by-Step: Detect, Classify, Then Decide Jump to heading#

The pattern below streams reconstructed geometries, classifies each one, and routes it to pass, repair, or quarantine. It uses Python 3.10+ type hints, Shapely 2.x, and the project logger convention. The key design choice is that classification precedes repair — you decide what is wrong before reaching for a fix, so the fix is targeted rather than a hopeful buffer(0) on everything.

1. Fast-path the valid. Check is_valid first; a valid geometry needs no work and most do not.
2. Explain the invalid. Call explain_validity to get the OGC reason string, and parse the leading token (Self-intersection, Ring Self-intersection, Too few points, Interior is disconnected) to classify the defect.
3. Repair only the unambiguous. Re-close near-closed rings, drop duplicate consecutive nodes, and run make_valid for self-intersections — then re-validate the result.
4. Guard against lossy repair. Compare pre- and post-repair area; if a repair drops or merges a lobe beyond a tolerance, or if it changes the geometry type, quarantine instead of accepting the silently altered shape.

import logging
from dataclasses import dataclass, field

from shapely import make_valid
from shapely.geometry import Polygon
from shapely.geometry.base import BaseGeometry
from shapely.validation import explain_validity

logger = logging.getLogger("osm.geometry.validate")

AREA_TOLERANCE = 0.02  # 2% relative area change is the auto-repair ceiling


@dataclass
class GeometryReport:
    accepted: list[BaseGeometry] = field(default_factory=list)
    repaired: list[BaseGeometry] = field(default_factory=list)
    quarantined: list[dict] = field(default_factory=list)


def classify_reason(geom: BaseGeometry) -> str:
    """Reduce Shapely's explain_validity string to a defect class token."""
    reason = explain_validity(geom)
    head = reason.split("[", 1)[0].strip().lower()
    if "too few points" in head:
        return "degenerate"
    if "ring self-intersection" in head or "self-intersection" in head:
        return "self_intersection"
    if "interior is disconnected" in head or "holes are nested" in head:
        return "ring_nesting"
    if "nested shells" in head or "self-touching ring" in head:
        return "ring_nesting"
    return "other"


def repair(geom: BaseGeometry) -> BaseGeometry | None:
    """Attempt an unambiguous repair; return None if none is safe."""
    fixed = make_valid(geom)
    if fixed.is_empty:
        return None
    # make_valid can return a collection; keep only the dominant polygon.
    if fixed.geom_type == "GeometryCollection":
        polys = [g for g in fixed.geoms if g.geom_type in ("Polygon", "MultiPolygon")]
        if not polys:
            return None
        fixed = max(polys, key=lambda g: g.area)
    return fixed


def validate_stream(geoms: list[BaseGeometry]) -> GeometryReport:
    report = GeometryReport()
    for i, geom in enumerate(geoms):
        if geom.is_valid and not geom.is_empty:
            report.accepted.append(geom)
            continue

        defect = classify_reason(geom)
        if defect == "degenerate":
            report.quarantined.append({"index": i, "defect": defect, "reason": explain_validity(geom)})
            continue

        fixed = repair(geom)
        if fixed is None or not fixed.is_valid:
            report.quarantined.append({"index": i, "defect": defect, "reason": explain_validity(geom)})
            continue

        # Reject repairs that silently change area or geometry type.
        base_area = geom.area if isinstance(geom, Polygon) else fixed.area
        if base_area > 0 and abs(fixed.area - base_area) / base_area > AREA_TOLERANCE:
            report.quarantined.append({"index": i, "defect": defect, "reason": "area drift on repair"})
            continue

        report.repaired.append(fixed)

    logger.info(
        "geometry validation: %d accepted, %d repaired, %d quarantined",
        len(report.accepted), len(report.repaired), len(report.quarantined),
    )
    return report

Validation & Remediation Matrix Jump to heading#

Each row pairs a defect with its root cause in OSM editing, the predicate that surfaces it, and the safe remediation. The rule threading through the table is that repair is warranted only when the mapper’s intent is unambiguous; everything else is a review item.

The area-drift guard in the code deserves emphasis: buffer(0) and make_valid are powerful precisely because they can restructure a geometry, and that same power lets them quietly delete a lobe of a pathological polygon. Comparing area before and after — the discipline detailed for defective records in Error Handling in Large OSM Extracts — turns a silent corruption into an explicit quarantine decision.

Repair Strategies and Their Trade-offs Jump to heading#

Three repair tools cover the field, and choosing between them is not arbitrary.

buffer(0) exploits a side effect of the buffer algorithm: buffering a polygon by zero distance re-runs the overlay engine, which reconstructs a valid geometry from the input’s segments. It is fast and available in every Shapely version, and for a simple bowtie it produces the two triangles most people expect. Its weakness is unpredictability on complex rings — it can dissolve slivers, merge touching lobes, or return an empty geometry, and it gives no explanation. Treat it as a heuristic, never as a guarantee.

make_valid (Shapely 2.x, backed by GEOS MakeValid) is the principled successor. It is designed to return a valid geometry that preserves as much of the input as possible, and crucially it is honest about ambiguity: a self-intersecting polygon may come back as a MultiPolygon or a GeometryCollection mixing polygons and lines. That collection is the signal to inspect — it means the original shape had no single unambiguous interpretation. The code above extracts the dominant polygon and quarantines when nothing polygonal survives, rather than pretending the collection is a clean fix.

Ring re-closing and re-orientation are surgical, not heuristic. When the only defect is a near-miss closure or a wrong winding, you do not need the overlay engine at all — snap the last coordinate to the first, or reverse the ring, and rebuild. These operations are lossless and reversible, so they are always preferable to a general repair when the defect class permits them. The full ring-assembly and closure workflow for multipolygon members is walked through in repairing unclosed ways and broken multipolygons.

When to Quarantine Instead of Auto-Repair Jump to heading#

Auto-repair is appropriate for defects with a single obvious correct answer: a near-miss closure has one intended ring, a wrong winding has one correct orientation, a duplicate node has one shape without it. Quarantine is the right call whenever the fix requires guessing what the mapper meant. A degenerate polygon has no lawful area to recover. A make_valid that returns three disconnected polygons from one “building” means the source geometry was nonsense, and picking the largest lobe discards real information. An inner ring that overlaps rather than nests inside its outer could be a mis-tagged role or a genuinely broken relation, and only a human — or a downstream rule that consults the tags — can tell which. The governing principle is that a rejected feature is recoverable (someone can fix the source in OSM), whereas a silently mangled feature poisons every downstream computation invisibly. Route quarantined geometries to a dead-letter table keyed on (type, id) with the explain_validity reason attached, and surface them in the review queue rather than dropping them.

Performance & Scale Considerations Jump to heading#

Validity checking is cheap per feature but runs across every reconstructed geometry in a continental extract, so constant factors matter. is_valid is O(n log n) in the vertex count for the sweep-line intersection test, which dominates on dense coastline and boundary polygons with tens of thousands of vertices. Two levers help. First, short-circuit: call is_valid before explain_validity, because the explanation is markedly more expensive and most geometries are valid. Second, prepare geometries you test repeatedly: when validation involves containment checks — inner-in-outer, member-in-relation — use Shapely’s prepared geometries or the spatial index from Spatial Indexing for OSM Extracts so you filter candidate pairs by bounding box before running the exact predicate. Validating an unindexed all-pairs containment across a country’s buildings is the classic accidental quadratic that turns a ten-minute job into an overnight one.

Failure Modes & Gotchas Jump to heading#

• buffer(0) can return empty. On a fully degenerate input the zero-buffer collapses to an empty polygon. Always test is_empty after repair and treat empty as a quarantine, not a success.
• make_valid changes type. A valid repair of a self-touching polygon is often a MultiPolygon; downstream code expecting a single Polygon will break. Normalise the type explicitly and decide whether a multi-part result is acceptable.
• Winding is not validity. A clockwise exterior ring is still valid under OGC SF — orientation does not affect is_valid. If a downstream consumer (GeoJSON, some tile pipelines) requires right-hand-rule winding, you must re-orient explicitly; validation alone will not flag it.
• Coordinate precision creates phantom self-intersections. Two segments that “should” meet at a vertex can cross by a floating-point epsilon after reprojection. Snap-round coordinates to a consistent grid before validating so precision noise does not masquerade as a real defect.
• Relations are not free geometry. A multipolygon is only a geometry after its member ways are assembled into rings; validate the assembled polygon, not the individual member ways, or you will flag correctly-open rings as broken.
• Empty geometries are silently valid. An empty Polygon returns True from is_valid. Guard is_empty separately, because an empty geometry is almost never what you want downstream.

Integration Points: Feeding the Next Stage Jump to heading#

Validated geometry is the contract every later stage depends on. The accepted and repaired outputs flow into the spatial index — an R-tree insert assumes geometry.bounds is meaningful, which it is not for an invalid ring — and from there into topology and attribute checks. The wiring below routes a validation report into an index build and a quarantine sink, keeping the two streams cleanly separated:

def route_report(report: GeometryReport, index, quarantine_sink) -> None:
    """Feed valid + repaired geometries to the index; defective ones to quarantine."""
    fid = 0
    for geom in (*report.accepted, *report.repaired):
        index.insert(fid, geom.bounds, obj=geom)
        fid += 1
    for item in report.quarantined:
        quarantine_sink.write(item)  # dead-letter table keyed on (type, id)
    logger.info("indexed %d geometries; quarantined %d", fid, len(report.quarantined))

Downstream of a clean geometry set, routing-graph topology and tag-consistency checks can assume every feature is a lawful shape, which is why geometry validation runs first in the quality gate. For the specific mechanisms, the two references below drill into detection and repair respectively.

Geometry Validation & Repair in Depth Jump to heading#

• Detecting Self-Intersecting OSM Polygons with Shapely — batch-flag bowtie and figure-eight polygons across reconstructed areas, classify each with explain_validity, and emit a report before any repair.
• Repairing Unclosed Ways and Broken Multipolygons — assemble relation members into closed rings, close near-miss ways, and fix outer/inner assignment and winding.

Frequently Asked Questions Jump to heading#

Related Jump to heading#

• OSM Data Quality & Validation — the section this geometry gate belongs to, alongside topology and tag-consistency checks.
• Understanding OSM Multipolygon Relations for GIS — ring assembly and winding that produce the geometries validated here.
• Spatial Indexing for OSM Extracts — the index that keeps containment checks out of accidental-quadratic territory.
• Error Handling in Large OSM Extracts — the quarantine and dead-letter discipline for geometries this stage rejects.
• Detecting Self-Intersecting OSM Polygons with Shapely — the focused detection walkthrough for bowtie polygons.
• Repairing Unclosed Ways and Broken Multipolygons — the focused repair walkthrough for rings and relations.

Up one level: OSM Data Quality & Validation.