RouteMatching

Detecting Unclosed Dwell Events with Timeout Rules

A dwell event opens when a vehicle stops and closes when it leaves. The second half fails more often than anybody expects: the tracker’s battery dies in the yard, the vehicle is loaded onto a transporter, the device is unplugged for maintenance, or the last fix of a shift simply never arrives.

The event then never closes. Its duration runs to whatever the pipeline last saw, and it enters the warehouse as a nineteen-hour stop that looks exactly like a genuine overnight park. The flat tail in the dwell histogram described in time-window based dwell calculation is made almost entirely of these.

This page covers closing them deliberately, marking them so they can be excluded, and using the rate at which they occur as a device-health signal.

Compatibility and Configuration Requirements

Requirement Value Notes
Vehicle class on every event Required The timeout is class-specific and one global value is wrong for everybody
Closure reason column Enum, not boolean observed_exit, timeout, feed_end, shift_end
Feed watermark Available to the job Needed to distinguish “not yet closed” from “never closed”
Aggregate definitions Filter on the reason Otherwise the exclusion silently does not happen
Reprocessing Idempotent per partition A timeout applied twice must not shift the timestamp

The Closure Rule

from __future__ import annotations

import polars as pl

MAX_DWELL_S = {
    "cargo_bike": 6 * 3600,
    "van": 14 * 3600,
    "rigid": 20 * 3600,
    "articulated": 72 * 3600,     # weekend parking is normal
}
DEFAULT_MAX_DWELL_S = 14 * 3600


def close_dwells(open_events: pl.DataFrame, watermark_utc, classes: pl.DataFrame) -> pl.DataFrame:
    """Close dwell events that have exceeded their class maximum, with a reason.

    ``watermark_utc`` is how far the feed has been processed. Events younger
    than their timeout are simply still open and are not touched.
    """
    limits = classes.select(
        "vehicle_id",
        pl.col("vehicle_class")
        .replace_strict(MAX_DWELL_S, default=DEFAULT_MAX_DWELL_S)
        .alias("max_dwell_s"),
    )

    return (
        open_events.join(limits, on="vehicle_id", how="left")
        .with_columns(
            (pl.lit(watermark_utc) - pl.col("enter_utc")).dt.total_seconds().alias("open_for_s")
        )
        .with_columns(
            pl.when(pl.col("open_for_s") >= pl.col("max_dwell_s"))
            .then(pl.col("enter_utc") + pl.duration(seconds=pl.col("max_dwell_s")))
            .otherwise(None)
            .alias("exit_utc"),
            pl.when(pl.col("open_for_s") >= pl.col("max_dwell_s"))
            .then(pl.lit("timeout"))
            .otherwise(pl.lit("still_open"))
            .alias("closed_by"),
        )
    )

Two details carry the correctness. The exit timestamp is set to enter plus the limit, not to the watermark — so re-running the job later produces the same value rather than a duration that grows with each reprocessing. And events that have not yet exceeded their limit are marked still_open rather than closed, so a stop in progress is never mistaken for a completed one.

Three ways a dwell event ends An observed exit closes the first event at its real end. The second is still within its class limit and remains open. The third exceeded the fourteen-hour van limit with no exit observed and is closed at enter plus fourteen hours, flagged as a timeout. Van class, 14-hour limit, feed processed to the dashed line watermark observed exit exit seen — duration is a measurement still open 3 h inside the limit — leave it alone timed out closed here — enter + 14 h Closing at enter + limit rather than at the watermark makes the value stable across reprocessing. The dashed remainder is time the vehicle may or may not have spent there — that is the point of the flag.

Setting the Limit Per Class

A single fleet-wide timeout is wrong in both directions at once. Set it at fourteen hours and every articulated tractor parked over a weekend is flagged as a fault; set it at seventy-two and a van whose tracker died on Monday morning is not caught until Thursday.

Derive each class’s limit from its own dwell distribution, taking a percentile well above the genuine maximum rather than a round number:

limits = (
    dwells.filter(pl.col("closed_by") == "observed_exit")
    .group_by("vehicle_class")
    .agg((pl.col("dwell_s").quantile(0.999) * 1.5).alias("suggested_max_s"))
)

Using only observed-exit events is the important part. Including timeout-closed events in the calculation makes the limit drift upward every time it is recomputed, because the previous limit is baked into the data it is derived from — a feedback loop that ends with a limit of several days and a detector that catches nothing.

The 1.5 multiplier is deliberate headroom. The aim is a limit that a genuine stop essentially never reaches, because every false timeout removes a real dwell from the duration statistics.

One limit per class, derived from that class Observed dwell durations for cargo bikes, vans and articulated tractors. The bike distribution ends by four hours, the van by eleven, and the tractor extends past sixty because weekend parking is routine. A single fleet-wide limit cannot serve all three. Observed dwell durations by class, and the derived limit cargo bike 6 h van 14 h articulated 72 h 0 36 h 80 h A fleet-wide 14-hour limit flags every tractor's weekend; a fleet-wide 72-hour one hides a dead van tracker for three days. Derive each limit from observed-exit events only, or it drifts upward every time it is recomputed.

Keeping Them Out of the Aggregates

The flag only helps if the aggregates read it, and the default behaviour of every reporting tool is to include everything. Make the exclusion structural rather than a convention somebody has to remember:

CREATE VIEW dwell_durations AS
SELECT *
FROM dwell_events
WHERE closed_by = 'observed_exit';      -- durations only from observed exits

CREATE VIEW dwell_occurrences AS
SELECT *
FROM dwell_events;                       -- counts include every stop that happened

Two views, two questions. How long do stops last may only be answered from observed exits. How many stops happened may be answered from everything, because the stop itself was observed even when its end was not.

Pointing the reporting layer at views rather than at the base table is what makes this stick. A convention that analysts must filter on closed_by survives exactly as long as the person who wrote it down.

The same distinction matters for billing. A timeout-closed dwell has an unknown duration, so it cannot support a duration-based charge — but it is evidence that the vehicle was at the site, which may support a visit-based one.

The Timeout Rate as a Health Signal

The share of dwell events closed by timeout is a direct measure of how often the fleet stops reporting while stationary. It should be small and stable, and when it moves it is almost never because driving changed.

Track it per device family and per depot. A rise in one device family is a hardware or firmware problem — batteries, a wiring loom, a sleep-mode change. A rise in one depot is usually environmental: an underground yard, a new site with poor coverage, a shielded loading bay.

SELECT device_family,
       date_trunc('week', enter_utc) AS week,
       avg((closed_by = 'timeout')::int) AS timeout_rate
FROM dwell_events
GROUP BY 1, 2
ORDER BY 2;

Alert on the change rather than the level. Some fleets legitimately run at two or three percent because of where they park; what matters is a doubling, which is a fault that has just started.

The timeout rate is a device signal Weekly share of dwell events closed by timeout for three device families. Two stay near one percent. The third rises from 1.2 to 7.4 percent after a firmware update changed the sleep threshold, so the tracker stops reporting sooner after the engine is switched off. Weekly timeout-closure rate by device family 8 % 4 % 0 family C — 7.4 % families A and B firmware 5.1 The new firmware sleeps sooner after ignition-off, so the exit fix is never sent and the dwell never closes. Without the closure reason on every event, this appears as a rise in long dwells and looks like a change in operations. Alert on a doubling rather than on a level; the baseline legitimately differs between fleets.

Common Pitfalls Specific to This Technique

Capping the duration instead of flagging it. Produces a spike at exactly the cap that looks like a real behavioural pattern, and leaves no way to identify the affected rows afterwards.

Closing at the watermark rather than at enter-plus-limit. Makes the duration depend on when the job ran, so a reprocess changes historical numbers.

Deriving the limit from data that includes previous timeouts. A feedback loop that inflates the limit on every recomputation until the rule stops firing at all.

Treating a still-open event as closed. A vehicle currently at a customer has not finished its stop, and reporting it as a completed short dwell understates every duration statistic in the current day.

Related

Up to section Time-Window Based Dwell Calculation Related Calculating Accurate Dwell Times Across Timezone Shifts Related Geofence-Intersection Dwell Detection in Python Related Splitting Trips on Ignition Cycles in Python