Map Data and Graph Preparation for Fleet Routing

Every page about routing engine integration starts from the same unstated assumption: that there is a routing graph, that it covers where the fleet drives, and that it is current. That assumption is where a surprising share of production matching problems actually live. A matcher that suddenly starts failing on one depot’s traces is usually not a matcher problem — it is a road layout that changed eight months ago in a graph that has not been rebuilt since.

This page covers the data stage that sits underneath the engines: choosing what to extract, clipping it, checking that the tags the profile depends on are actually present, building the graph, and versioning the whole thing so that a match performed in March can still be explained in September.

Prerequisites

• osmium-tool 1.16+ for clipping and filtering. The older osmosis still works and is considerably slower on modern extract sizes.
• A regional PBF source. Geofabrik’s country and region extracts are the usual starting point; a planet file is rarely necessary and always painful.
• Disk. Budget roughly six times the PBF size for the intermediate artefacts of a build chain, and keep the previous graph while the new one is validated.
• A defined operating area. This is the input everything else depends on, and it is a business question before it is a technical one.
• The engine’s own build tooling — osrm-extract and friends, valhalla_build_tiles, or GraphHopper’s import mode. See choosing a routing engine for which chain you will be running.

Defining the Operating Area

The instinct is to use an administrative boundary — a country, a set of provinces — because those are easy to obtain. For most fleets that is both too much and too little. Too much, because a national extract carries millions of ways the fleet will never touch; too little, because vehicles cross the border to a customer twenty kilometres beyond it, and the graph stops dead at the line.

A better polygon comes from the traces themselves:

import geopandas as gpd

def operating_area(fixes: gpd.GeoDataFrame, pad_km: float = 25.0) -> gpd.GeoSeries:
    """Convex operating polygon derived from where the fleet actually drove."""
    projected = fixes.to_crs(fixes.estimate_utm_crs())
    hull = projected.union_all().convex_hull
    return gpd.GeoSeries([hull.buffer(pad_km * 1000)], crs=projected.crs).to_crs(4326)

A convex hull with a generous buffer is deliberately crude. A concave hull hugging the observed traces looks tidier and breaks the first time a driver takes a legitimate route the fleet has not used before — and the failure mode is a matching gap, not an error, so nobody notices for a week.

Clipping and Filtering

Two reductions matter, and they are independent. Spatial clipping removes ways outside the polygon. Tag filtering removes features the routing profile will never consult — buildings, landuse, waterways — which is often the larger saving.

# Spatial clip, keeping ways that cross the boundary complete.
osmium extract --polygon operating_area.geojson --strategy complete_ways \
    --output region-clipped.osm.pbf europe-latest.osm.pbf

# Tag filter: keep only what a routing profile reads.
osmium tags-filter region-clipped.osm.pbf \
    w/highway nwr/restriction=* r/type=restriction \
    --output region-routing.osm.pbf

--strategy complete_ways is the important flag. The default strategy cuts ways at the boundary, which leaves dangling segments the graph builder will happily turn into dead ends — and a dead end at the edge of your operating area is a place where matching silently fails for every vehicle that crosses it.

Keep the turn-restriction relations. They are a small fraction of the file and they are the difference between a graph that knows a left turn is banned and one that routes a truck through it every night.

Auditing Restriction Coverage

A truck profile that filters on maxweight is only as good as the tagging. Before shipping a graph, measure what the tags actually cover — the answer varies enormously by country and road class, and the gaps are almost never where you would guess.

import polars as pl

def restriction_coverage(ways: pl.DataFrame) -> pl.DataFrame:
    """Share of network length carrying each restriction tag, by road class."""
    return (
        ways.group_by("highway")
        .agg(
            pl.col("length_m").sum().alias("total_m"),
            pl.col("length_m").filter(pl.col("maxweight").is_not_null()).sum().alias("weight_m"),
            pl.col("length_m").filter(pl.col("maxheight").is_not_null()).sum().alias("height_m"),
        )
        .with_columns(
            (pl.col("weight_m") / pl.col("total_m")).alias("weight_coverage"),
            (pl.col("height_m") / pl.col("total_m")).alias("height_coverage"),
        )
        .sort("total_m", descending=True)
    )

The typical result is uncomfortable: motorways and trunk roads carry near-complete tagging, and residential and service roads carry almost none. Those are exactly the roads where a 4-metre truck meets a 3.8-metre bridge, so the profile is most confident precisely where it has least information.

The practical response is not to abandon the profile but to treat missing as unknown rather than as permitted, and to record which decisions rested on an absent tag. A route that depends on ten untagged residential segments is a different kind of answer from one that stayed on tagged trunk roads, and a fleet planning system can use that distinction even when the router cannot.

Building and Versioning

Once the extract is prepared, the build chain is engine-specific and well documented. What is usually missing is the versioning discipline around it.

Every graph artefact should carry three pieces of metadata: the checksum of the source PBF, the timestamp of the OSM data it was derived from (not the download time), and the profile version used to build it. The first makes rebuilds idempotent, the second makes reproducibility possible, and the third catches the case where the map is identical but the profile changed underneath it.

osmium fileinfo -e -g data.timestamp.last.out region-routing.osm.pbf
sha256sum region-routing.osm.pbf | cut -d' ' -f1 > graph/SOURCE_SHA256
git rev-parse HEAD:profiles/truck.lua > graph/PROFILE_REV

Store the matched output with a reference to that graph version. It costs one small column and it is the difference between “we can show you exactly what the map said when this trip was matched” and a shrug — see pinning map versions for reproducible fleet matching.

Refresh Cadence and the Fitness Gate

Rebuilding often is good. Promoting every rebuild automatically is not: an upstream extract can be truncated, a tagging schema can shift, and a vandalised area can survive long enough to reach a download. The protection is a fitness gate — a check that the new graph matches yesterday’s real traces at least as well as the old one before it becomes live.

def promote_if_fit(new_graph, current_graph, sample_traces, tolerance_pts: float = 1.0) -> bool:
    """Promote only if the new graph does not lose more than `tolerance_pts` of match rate."""
    new_rate = match_rate(new_graph, sample_traces)
    old_rate = match_rate(current_graph, sample_traces)
    if new_rate < old_rate - tolerance_pts / 100:
        raise RuntimeError(f"new graph match rate {new_rate:.3f} < current {old_rate:.3f}; not promoting")
    return True

The sample should be yesterday’s traces, not a frozen benchmark set. A frozen set stops representing the fleet within a few months, and a gate that no longer represents the workload is a gate that will approve the change that breaks it.

Weekly is the right default cadence for most fleets. Daily rebuilds are mostly churn: the changes that matter to a fleet accumulate over weeks, and a daily promotion cycle gives the fitness gate too little signal to distinguish a real regression from sampling noise.

Sizing the Build Host

Graph builds are memory-bound rather than CPU-bound, and the memory they need is set by the extract rather than by the machine. Getting the sizing wrong produces one of two failures: a build killed by the out-of-memory reaper, or a machine three times larger than necessary running for four minutes a week.

As a working rule, the peak resident set of a build chain is roughly twelve to twenty times the clipped PBF size, depending on the engine. A 400 MB metropolitan clip therefore wants 6–8 GB available, and a 4 GB country extract wants 50–80 GB — which is the point at which clipping stops being an optimisation and becomes a requirement.

Three practical consequences follow.

Build on a separate host from the one serving. A build that exhausts memory on a serving node takes the routing service down with it, and the failure arrives during the quiet window when nobody is watching. Separate hosts also let the build machine be ephemeral: it exists for ten minutes a week and costs accordingly.

Give the build fast local disk, not network storage. The intermediate artefacts are written and re-read several times, and a build chain on network storage frequently spends more time in I/O than in computation. A local SSD scratch volume, discarded afterwards, is usually the single largest speed-up available.

Watch the peak, not the average. Build memory is spiky: the partition step in a multi-level Dijkstra chain, or the tile-writing step in Valhalla, will briefly use several times the steady-state figure. Monitoring that reports averages will show a comfortable build right up until the run that gets killed.

Where the graph must be rebuilt for several regions, run them sequentially on one adequately sized host rather than concurrently on a larger one. The builds do not share work, and running them in parallel multiplies the peak without shortening the critical path by much — the longest single region still sets the wall clock.

Operational Troubleshooting

Match rate falls in one city only. Almost always a local road layout change — a new one-way system, a pedestrianised centre. Check the OSM changeset history for the area before suspecting the matcher.

Build fails after months of working. Usually an upstream schema or tooling change, or a disk that filled with intermediate artefacts. Both are why the previous graph must still be on disk.

Graph is much smaller than last week. The extract was truncated — a download that failed partway through still produces a valid PBF. Assert on file size and way count relative to the previous build, not just on the build succeeding.

Trucks routed down banned streets. Turn restrictions or access tags were dropped by the tag filter. Re-check the filter expression; it is easy to keep w/highway and lose the relations that carry the restrictions.

Every trace matches but the routes look wrong. The profile changed and the graph did not, or the reverse. This is what the profile revision in the metadata is for.

Deployment Checklist

• Operating-area polygon derived from real traces and padded, not from an administrative boundary
• Extract clipped with --strategy complete_ways
• Turn-restriction relations retained through tag filtering
• Restriction coverage audited per road class and recorded
• Source PBF checksum, OSM data timestamp and profile revision stored with the graph
• Previous graph retained on disk for rollback
• Fitness gate runs against recent traces before promotion
• Build asserts on way count and file size relative to the previous build
• Matched output carries the graph version it was produced against
• Refresh cadence documented, with the reasoning for the interval chosen

Frequently Asked Questions

Related

• Clipping OSM extracts to a fleet operating area with osmium — the clipping step in detail
• Adding truck restrictions to an OSM extract for routing — filling the coverage gaps this page measures
• Scheduling nightly OSM extract refreshes for routing engines — the refresh job and its fitness gate
• Choosing a routing engine: OSRM vs Valhalla vs GraphHopper — which build chain you will be running
• Routing Engine Integration for Fleet Telematics — parent section