RouteMatching

Clipping OSM Extracts to a Fleet Operating Area with osmium

A continental OpenStreetMap extract is around 30 GB. A metropolitan operating area with a generous buffer is a few hundred megabytes. That ratio is why clipping is worth doing carefully: it turns a graph build from a forty-minute job into a four-minute one, and a four-minute build is one you can afford to run weekly, which is what actually keeps the map current.

This page is the mechanical companion to map data and graph preparation for fleet routing: the exact osmium invocations, the flag that matters, and the checks that catch a clip that has quietly broken the network at its edges.

Compatibility and Configuration Requirements

Requirement Value Notes
osmium-tool ≥ 1.16 --strategy complete_ways and tags-filter semantics assumed here
Polygon format GeoJSON, EPSG:4326 osmium also accepts .poly; GeoJSON is easier to produce from GeoPandas
Geometry type Polygon or MultiPolygon A FeatureCollection with several features is accepted; a GeometryCollection is not
Disk ~2× the source PBF, free The extract is streamed but the index is not
Memory 4 GB is comfortable for a country-sized source complete_ways holds a node-id index

The Clip

#!/usr/bin/env bash
set -euo pipefail

SRC=${1:?source pbf}
POLY=${2:?operating area geojson}
OUT=${3:-region-routing.osm.pbf}

# 1. Spatial clip. complete_ways keeps every node of any way that touches the
#    polygon, so a road crossing the boundary stays traversable end to end.
osmium extract \
    --polygon "$POLY" \
    --strategy complete_ways \
    --overwrite \
    --output /tmp/clipped.osm.pbf \
    "$SRC"

# 2. Tag filter. Keep highways, plus the relations that carry turn restrictions
#    and the nodes that carry barriers — everything else is dead weight.
osmium tags-filter \
    /tmp/clipped.osm.pbf \
    w/highway \
    n/barrier \
    r/type=restriction \
    r/type=route \
    --overwrite \
    --output "$OUT"

osmium fileinfo -e "$OUT"

The two steps are separate on purpose. Running the tag filter first is faster but wrong: filtering before clipping discards the relations osmium would have needed to keep boundary-crossing ways whole.

What the strategy flag does at the boundary A road crosses the operating-area boundary. Under the simple strategy the way is cut at the polygon edge and the remaining stub ends in a node with no continuation, which the graph builder turns into a dead end. Under complete_ways the whole way is retained, including the nodes outside the polygon. One road, one boundary, two strategies --strategy simple polygon edge dangling stub — a dead end in the graph --strategy complete_ways way retained whole — the crossing still routes The failure the simple strategy causes is silent: traces crossing the boundary stop matching, and nothing raises.

Execution and Tuning Guidelines

Produce the polygon from GeoPandas and write it as one feature. osmium accepts a FeatureCollection, but a file with several features is easy to produce accidentally when a hull operation returns a multipart geometry, and the resulting clip covers only the first part on some versions.

area = operating_area(fixes, pad_km=25)          # GeoSeries in EPSG:4326
area.to_file("operating_area.geojson", driver="GeoJSON")

Keep the r/type=route relations if you carry public transport or ferry routes. Ferries in particular matter for fleet matching: a trace that crosses water with no ferry way in the graph produces an unreachable transition and a broken trellis, which is the failure described in building an HMM-based map matcher.

Filter barriers, do not drop them. n/barrier keeps bollards, gates and lift barriers, which are what stop a router sending a truck through a pedestrianised centre. They are a rounding error in file size and a large part of whether the graph is realistic.

Measure the saving, then decide whether to clip further. For most fleets the metropolitan clip plus tag filter gets the build under five minutes, at which point tightening the polygon further buys nothing and increases the chance of clipping away a route somebody needs.

What each reduction step is worth Size and build time for a continental extract, a country extract, a metropolitan clip and the same clip after tag filtering. The country extract removes most of the bytes; the metropolitan clip removes most of the remainder; the tag filter halves what is left and takes the build under five minutes. Reduction steps, PBF size and OSRM build time europe-latest 29 GB · 9 h country 4.1 GB · 48 min metropolitan clip 860 MB · 9 min + tag filter 410 MB · 4 min — weekly rebuilds are now cheap The last row is the one that changes behaviour: a four-minute build is one a team will actually run on a schedule. A nine-hour build gets run when someone remembers, which in practice means a map that is months old.

Verifying the Clip

Three checks catch nearly every clipping mistake, and all three run in seconds.

Way count against the previous clip. A drop of more than a few percent between weekly runs is a truncated download or a polygon that changed. Assert on it rather than eyeballing it.

Known-route traversability. Keep a handful of origin–destination pairs that cross the operating area, including at least one that crosses the boundary, and route them against the freshly built graph. A route that returns no path is a clip that has severed the network.

Boundary node degree. Count nodes on the polygon edge whose degree is one. Under complete_ways that count should be near zero; a large number means the strategy flag did not take effect.

PREV=$(cat graph/prev_way_count)
NOW=$(osmium fileinfo -e -g data.count.ways region-routing.osm.pbf)
python3 - "$PREV" "$NOW" <<'EOF'
import sys
prev, now = int(sys.argv[1]), int(sys.argv[2])
drop = (prev - now) / prev
assert drop < 0.05, f"way count fell {drop:.1%} — truncated source or changed polygon"
EOF

That assertion has to be able to fail, so test it: point the script at a deliberately truncated PBF and confirm it raises. A build-time check that has never rejected anything is decoration.

Common Pitfalls Specific to This Technique

Clipping with a polygon in the wrong CRS. osmium reads coordinates as longitude and latitude. A GeoJSON written from a projected GeoDataFrame contains metres, and osmium will interpret the first value as a longitude of 628 437 degrees — producing an empty extract rather than an error.

Reusing an old polygon after the fleet expanded. The extract quietly stops covering the new depot, matching fails only for its vehicles, and the symptom looks like a device problem. Re-derive the polygon on the same cadence as the rebuild.

Filtering before clipping. Tag filtering removes the relations that complete_ways needs to keep boundary ways intact, so the order in the script above is load-bearing rather than stylistic.

Handling Multi-Region Fleets

A fleet with separated operating areas — a national carrier with three regional hubs, say — should not be served by one polygon covering everything between them. The convex hull of three distant hubs includes vast areas nobody drives, and the resulting extract is close to the national one it was meant to replace.

The right shape is a MultiPolygon, one part per operating area, in a single GeoJSON:

import geopandas as gpd
from shapely.ops import unary_union

def multi_area(fixes: gpd.GeoDataFrame, hub_col: str = "hub", pad_km: float = 25.0):
    """One padded hull per hub, merged into a single MultiPolygon."""
    projected = fixes.to_crs(fixes.estimate_utm_crs())
    parts = [
        grp.union_all().convex_hull.buffer(pad_km * 1000)
        for _, grp in projected.groupby(hub_col)
    ]
    merged = unary_union(parts)          # overlapping hulls dissolve into one part
    return gpd.GeoSeries([merged], crs=projected.crs).to_crs(4326)

Give osmium the multipolygon in one run rather than running three extracts and merging the results. Merging afterwards loses any way that spans two parts, and — more subtly — produces three copies of every node in the overlap regions, which some builders accept and others reject with an error that does not mention duplication.

unary_union is doing quiet work here: where two hubs are close enough that their padded hulls overlap, it dissolves them into a single part. Without it the multipolygon has self-intersecting parts, and osmium’s behaviour on those is version-dependent.

When the parts are far apart

Long-haul work between hubs raises a question the polygon cannot answer: does the fleet need the motorway corridor between them in the graph? If vehicles drive it, yes — and the corridor should be a fourth part, a buffered line rather than a hull.

corridor = hub_a_point.shortest_line(hub_b_point).buffer(15_000)

A fifteen-kilometre buffer along the straight line between hubs is crude and usually sufficient, because motorway corridors are straight enough that the great-circle line stays inside the buffer. Where they are not — mountains, coastlines — route the corridor on a coarse graph first and buffer the result.

Three hubs: hull against multipolygon plus corridor Three separated operating areas. A single convex hull spans everything between them, producing an extract nearly as large as the national one. A multipolygon of three padded hulls plus a buffered motorway corridor covers what the fleet drives at a fraction of the area. Three hubs, two ways to cover them one convex hull covers 41 000 km² buffered corridor multipolygon + corridor covers 7 400 km² A fifth of the area, and it includes the one thing the hull was needed for: the corridor the trucks actually drive.

Related

Up to section Map Data and Graph Preparation for Fleet Routing Related Adding Truck Restrictions to an OSM Extract for Routing Related Scheduling OSM Extract Refreshes for Routing Engines Related Pinning Map Versions for Reproducible Fleet Matching