OSMnx vs Pyrosm performance benchmarks for routing Jump to heading#

Task: decide whether to build a routing graph straight from OSMnx or to parse the .osm.pbf with Pyrosm first, by measuring parse-plus-build wall time and peak resident memory on the same regional extract — then wire the faster path into a single reproducible function.

Prerequisites Jump to heading#

• Python 3.10+ (the snippets use str | None union syntax and tuple[...] generics)
• osmnx>=2.0 — the graph_from_xml / graph_from_gdfs and add_edge_speeds API surface used here
• pyrosm>=0.6.2 for the Cython/libosmium PBF reader that returns GeoDataFrames
• networkx>=3.2 (optionally igraph>=0.11 or rustworkx>=0.14 for the low-RAM adjacency fallback)
• osmium-tool>=1.14 on PATH for osmium fileinfo pre-validation
• A regional extract such as us-california-latest.osm.pbf from Geofabrik, plus psutil>=5.9 to sample RSS
• Environment: enough free RAM for the OSMnx path (budget ~5× the PBF size); confirm with free -m first

What the two libraries actually do differently Jump to heading#

The performance gap is structural, not incidental. Pyrosm wraps libosmium through Cython and reads a PBF in one sequential pass into GeoDataFrames backed by Apache Arrow arrays; it never materializes a networkx graph until you explicitly call get_network() with nodes=True, so peak memory stays close to the on-disk feature volume. OSMnx instead builds a networkx.MultiDiGraph immediately and runs ox.simplify_graph() to merge degree-2 nodes during construction, trading RAM for an out-of-the-box, routing-ready topology. The decision is therefore ingestion velocity versus immediate routing readiness — exactly the trade this page measures and the reason it sits under OSMnx Graph Conversion Techniques. Because edge length and travel_time are meaningless until the graph is projected, both paths still depend on the projection rules covered in Coordinate Reference Systems in OSM, and both reuse the same deterministic value coercion from Value Standardization & Regex Cleaning so the benchmark compares construction cost, not tag-cleaning cost.

Runnable solution Jump to heading#

The harness below times and memory-profiles both construction paths against one extract, then returns a comparable record per path. Each builder shares the same add_edge_speeds/add_edge_travel_times weighting so only parsing and graph assembly differ.

import logging
import time
import tracemalloc
from typing import Callable

import networkx as nx
import osmnx as ox
from pyrosm import OSM

logger = logging.getLogger(__name__)


def build_via_osmnx(pbf_path: str) -> nx.MultiDiGraph:
    """Path A: OSMnx parses the PBF and simplifies topology in one step."""
    G = ox.graph_from_xml(pbf_path, simplify=True, retain_all=False)
    G = ox.add_edge_speeds(G)        # impute speed_kph from highway-class table
    G = ox.add_edge_travel_times(G)  # length / speed_kph -> travel_time (seconds)
    return G


def build_via_pyrosm(pbf_path: str) -> nx.MultiDiGraph:
    """Path B: Pyrosm parses to GeoDataFrames, OSMnx assembles + weights."""
    osm = OSM(pbf_path)
    nodes, edges = osm.get_network(network_type="driving", nodes=True)
    if nodes is None or edges is None:
        raise ValueError(f"Pyrosm returned no driving network for {pbf_path}")
    G = ox.graph_from_gdfs(nodes, edges)
    G = ox.add_edge_speeds(G)
    G = ox.add_edge_travel_times(G)
    return G


def benchmark(label: str, builder: Callable[[str], nx.MultiDiGraph],
              pbf_path: str) -> dict[str, float | str]:
    """Time one builder and capture peak Python-heap allocation."""
    tracemalloc.start()
    t0 = time.perf_counter()
    G = builder(pbf_path)
    wall = time.perf_counter() - t0
    _, peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()

    record = {
        "path": label,
        "seconds": round(wall, 1),
        "peak_mb": round(peak / 1024 ** 2, 1),
        "nodes": G.number_of_nodes(),
        "edges": G.number_of_edges(),
    }
    logger.info("%(path)s: %(seconds)ss, peak %(peak_mb)s MB, "
                "%(nodes)d nodes / %(edges)d edges", record)
    return record


def compare(pbf_path: str) -> list[dict[str, float | str]]:
    """Run both paths and return one comparable record each."""
    return [
        benchmark("osmnx", build_via_osmnx, pbf_path),
        benchmark("pyrosm+graph_from_gdfs", build_via_pyrosm, pbf_path),
    ]

tracemalloc captures the Python heap; for true process RSS (which includes the C arenas libosmium and GEOS allocate) sample psutil.Process().memory_info().rss in a background thread instead — the matrix below reports RSS, which is the number that decides whether a job survives on a given host.

Step-by-step walkthrough Jump to heading#

1. build_via_osmnx is the one-call path. graph_from_xml(..., simplify=True, retain_all=False) parses, simplifies, and prunes disconnected islands in a single step, so the returned graph is immediately routable. The cost is that the whole MultiDiGraph plus its dict-of-dict adjacency lives in RAM at once.
2. build_via_pyrosm splits parsing from assembly. OSM(pbf_path).get_network(network_type="driving", nodes=True) returns Arrow-backed GeoDataFrames; only then does ox.graph_from_gdfs build the graph. Peak memory tracks the feature tables, not a fully expanded graph, so it stays far lower on large extracts.
3. The None guard matters. Pyrosm’s get_network() returns None for an empty result (a clipped extract with no driving ways), so the explicit check stops a confusing TypeError inside graph_from_gdfs and surfaces the real cause.
4. Both builders end identically. add_edge_speeds imputes speed_kph from the highway-class lookup and add_edge_travel_times derives travel_time in seconds, so the routing weights are byte-for-byte comparable and the benchmark isolates construction.
5. benchmark wraps timing and allocation. time.perf_counter() brackets the build and tracemalloc reports peak Python-heap bytes; emitting nodes/edges confirms both paths produced the same graph rather than one silently dropping features.
6. compare returns structured records, not printed text, so you can assert on them in CI or write them to a Parquet log and watch the trend as extracts grow.

Verification Jump to heading#

Run compare("us-california-latest.osm.pbf") and check the returned records against the reference matrix below. Measurements were taken on Ubuntu 22.04 LTS, AMD EPYC 7763 (64-core), 128 GB DDR4, Python 3.11.7, NetworkX 3.2.1, on the 4.1 GB California extract, with a routing workload of 10,000 randomized origin-destination pairs using A* on travel_time weights.

Sanity checks that the numbers are trustworthy:

• Equal node/edge counts. Both records should report the same nodes and edges (within simplification rounding). A large divergence means one path applied a different network filter or skipped simplification.
• Routing parity. Once weighted, an identical A* query (nx.shortest_path(G, o, d, weight="travel_time")) must return the same path on both graphs — construction speed should not change routing results.
• RSS plateau, not climb. Watch psutil RSS during the build; the Pyrosm path should plateau near the feature-table size, while OSMnx peaks during simplify_graph. A monotonic climb past the matrix figure signals a runaway extract or a missing retain_all=False.
• Pre-validate the file. osmium fileinfo -e california.osm.pbf should report a clean bounding box and nonzero way count before you trust any timing; a corrupt block inflates parse time unpredictably.

Common errors & fixes Jump to heading#

When the OSMnx path is OOM-killed and re-tiling is impractical, swapping NetworkX for an igraph or rustworkx adjacency structure sustains routing queries under heavy memory pressure — igraph stores adjacency as contiguous C arrays and routes multi-million-edge graphs in under 2 GB. Files that fail validation or crash a parser should flow to the triage path in Error Handling in Large OSM Extracts rather than aborting the whole run.

Spec reference Jump to heading#

Pyrosm reads PBF data through libosmium and exposes it as GeoDataFrames; its get_network() filters ways by a network type before returning, and returns None when no matching features exist. OSMnx encodes a street network as a directed multigraph and derives travel_time from edge length (metres) and speed_kph. See the Pyrosm documentation for reader configuration, the OSMnx documentation for graph simplification parameters, and the OpenStreetMap maxspeed key for the value conventions both libraries must normalize.

Frequently Asked Questions Jump to heading#

Related Jump to heading#

• OSMnx Graph Conversion Techniques — the full extract → normalize → project → weight → validate conversion this page benchmarks.
• Async PBF Parsing with Pyrosm — overlapping disk reads with compute to push the Pyrosm path even faster.
• Memory-Efficient Chunk Processing — tiling and spill-to-disk patterns when even the Pyrosm path exceeds RAM.
• Value Standardization & Regex Cleaning — the deterministic tag coercion both builders share before weighting.
• Error Handling in Large OSM Extracts — triaging corrupt PBFs and empty network clips.
• Coordinate Reference Systems in OSM — why projection must precede any length or travel-time weight.

This guide is part of OSMnx Graph Conversion Techniques; return to that overview to follow the conversion from raw ways through normalization into a routing-ready graph.