Interpreting OSRM Match Confidence Scores

OSRM’s /match service returns a confidence between zero and one on every matching. It looks like exactly what a fleet pipeline wants: a per-trace quality number that can be thresholded, aggregated and reported. It is used that way constantly, and the interpretation is wrong in a way that matters.

Confidence measures how well the observations fit the path the matcher selected. It says nothing about whether that path is the one the vehicle drove. Those coincide most of the time and diverge precisely where matching is difficult — parallel carriageways, service roads, dense junctions — which means the metric is least informative exactly where a quality signal would be most useful.

This page sets out what the number does measure, how to calibrate it so a threshold means something, and the triage policy it genuinely supports. It extends OSRM integration for fleet map matching.

Compatibility and Configuration Requirements

What the Number Is

OSRM derives confidence from the likelihood of the selected path relative to the alternatives it considered, normalised into a zero-to-one range. Three consequences follow directly and explain most of the surprises.

It falls when candidates are ambiguous, not when the answer is wrong. A long motorway run with no parallel roads scores near one because there was nothing else to choose. A city-centre trace scores lower because there were many plausible paths — even when the chosen one is correct.

It is per matching. A trace that OSRM splits into three matchings returns three values. Averaging them produces a number that hides the split, which is usually the more important signal.

It is not comparable across engines or across major versions. The normalisation is an implementation detail, and it has changed between OSRM releases.

Calibrating It Once

The relationship above is specific to a fleet’s network and sampling rate, so it is worth measuring rather than assuming. Take the labelled set from building a ground-truth set, bin by reported confidence, and record observed correctness per bin.

import polars as pl

def calibrate(scored: pl.DataFrame, bins: int = 10) -> pl.DataFrame:
    """Observed correctness per confidence decile — run once, store the table."""
    return (
        scored.with_columns((pl.col("confidence") * bins).floor().alias("bin"))
        .group_by("bin")
        .agg(
            pl.col("correct_edge_rate").mean().alias("observed"),
            pl.len().alias("n"),
        )
        .sort("bin")
    )

The output is a small lookup table, and it is what makes a threshold defensible. “We review anything below 0.55” is arbitrary; “we review anything below 0.55, where observed correctness falls under 80 percent” is a policy with a reason.

Re-derive it after any change to sampling rate, receiver hardware or operating area. All three move the curve, and a threshold calibrated on last year’s fleet quietly stops meaning what it did.

A Triage Policy That Works

The defensible use of confidence is routing attention, not accepting results.

Note what the table does not do: it does not treat a high value as evidence. Everything above the middle band is accepted because there is nothing better to go on, not because confidence has certified it.

The other half of the policy is the matching count. A trace returned as four separate matchings has been broken three times, and that is a stronger negative signal than any of the four confidence values. Check it first.

def triage(matchings: list[dict]) -> str:
    if len(matchings) > 1:
        return "review"                       # the trace was split — that is the finding
    c = matchings[0]["confidence"]
    return "reject" if c < 0.4 else ("sample" if c < 0.7 else "accept")

Using It as a Change Detector

Where confidence genuinely earns its place is in monitoring. Its absolute level is uninformative; a change in the shape of its distribution is a reliable signal that something upstream moved — a map refresh, a firmware rollout, a new operating area.

Track the full histogram daily rather than the mean, and alert on divergence from a trailing baseline. The mean is insensitive to exactly the change that matters: a bimodal distribution developing a second peak at low confidence can leave the mean almost unchanged.

This is the same signal used in detecting map-matching drift, and it works there for the same reason: it needs no labels and it responds within a day.

Common Pitfalls Specific to This Technique

Averaging confidence across matchings. It produces a number that looks better the more broken the trace is, as the figure above shows.

Reporting mean confidence as a quality metric to stakeholders. It will be read as accuracy, it will be trended, and the first time somebody labels a sample the two will disagree publicly.

Thresholding on a value taken from documentation. The distribution is fleet-specific. A threshold of 0.5 might reject two percent of traces on one fleet and thirty on another.

Storing the triage decision instead of the raw value. Thresholds change. A stored boolean cannot be re-evaluated; a stored float can.

What to Store Alongside It

Confidence is most useful in combination with three other values the same response already contains, and storing all four costs almost nothing.

The matching count. How many separate matchings the trace produced. As established above, this outranks confidence as a quality signal and is a single integer.

The unmatched tracepoint count. OSRM reports null for tracepoints it could not place. A trace with a high confidence and eleven null tracepoints is not a good match; it is a good match of the sixty percent of the trace that was placed.

The matched distance against the input’s straight-line distance. A ratio far above one indicates the matcher routed a long way round to explain the observations, which is the signature of a parallel-road or wrong-carriageway failure.

def quality_row(response: dict, trace) -> dict:
    m = response["matchings"]
    tps = response["tracepoints"]
    return {
        "confidence": m[0]["confidence"] if len(m) == 1 else None,
        "n_matchings": len(m),
        "n_unmatched_points": sum(1 for t in tps if t is None),
        "distance_ratio": sum(x["distance"] for x in m) / max(trace.straight_line_m, 1.0),
    }

Together these four make a usable triage score, and none of them requires a label. The confidence on its own is the weakest of the four, which is worth knowing given how often it is the only one stored.

Reporting to people outside the team

The temptation is to publish mean confidence as a quality metric because it is available daily and needs no labelling. Resist it. It will be read as accuracy, trended in a dashboard, and eventually contradicted publicly when somebody labels a sample.

Publish the unmatched rate and the split rate instead. Both are unambiguous, both are things a non-specialist can reason about, and neither invites a claim the data cannot support.

Related

• OSRM integration for fleet map matching — parent topic
• Confidence scoring for stop detection — the same calibration argument applied to stops
• Map matching accuracy validation and benchmarking — what to use instead when you need an accuracy figure
• Detecting map-matching drift after an OSM update — confidence as a monitoring signal
• Routing Engine Integration for Fleet Telematics — parent section