A technical reference for engineers building and operating production OpenStreetMap data pipelines. Cover the full ETL surface — from PBF and XML parsing, through tag normalization and topology cleaning, all the way to routing graph extraction, diff-based updates, and automated validation rules.
Every page is written for mapping engineers, OSM contributors, GIS analysts, and Python ETL developers who need deterministic, reproducible workflows at continental scale. Expect deep binary-format dives, memory-aware streaming patterns, and rule-driven QA you can wire into Dask, Ray, or plain old multiprocessing.
The content is organised in two pillars: a data fundamentals track covering the OSM schema and serialization formats, and a workflow track covering parsing, normalization, and routing-graph conversion.
Schema, serialization, spatial indexing, tag taxonomy, and licensing foundations for production-grade OSM pipelines.
Async PBF parsing, batch attribute mapping, regex-driven value cleaning, error handling, and routing-graph conversion.
Byte-level dissection of the PBF container, block framing, StringTable deduplication, and delta encoding for nodes, ways, and relations.
Async ingestion with Pyrosm, chunked memory-aware processing, regex-driven value standardization, and reproducible schema alignment.
OSMnx graph conversion, turn restrictions, edge contraction, one-way handling, and accessibility constraints for routing engines.
Error isolation, structured logging, quarantine workflows, automated tag taxonomy validation, and ODbL compliance checks.