Sliding-Window Variance Stop Detection

A stop is not a point — it is an interval during which a vehicle stays put. DBSCAN-based stop clustering reconstructs those intervals after the fact by grouping spatially dense pings, which works well for a completed day of history but is fundamentally a batch, spatial-only operation: it holds no notion of time ordering, it re-scans the whole point set, and it cannot answer “is this vehicle stopped right now?” without re-clustering everything. For a live dashboard, a geofence trigger, or a driver-facing “you have arrived” prompt, you need a detector that consumes fixes in order and emits a decision within one sampling interval.

Sliding-window variance detection is that alternative. Instead of density in space, it measures spread over a short window of time: compute the rolling variance of projected position (and, optionally, the rolling mean and variance of speed), then threshold. When a vehicle is genuinely parked, the fixes inside the window collapse to a tight blob and the positional variance drops toward the receiver’s noise floor; when it moves, the window stretches across metres of travel and variance climbs sharply. Because the window only ever holds the last few dozen samples, the detector runs in constant memory per vehicle and slots directly into a streaming pipeline behind Kalman filtering for GPS noise reduction.

This page builds the detector end to end: resampling to a fixed cadence, the two rolling signals, threshold-plus-hysteresis state logic, and segmentation of contiguous stationary runs into stop events with arrival and departure timestamps. The variance signal also separates idling from travel far more cleanly than raw speed, which is what the child guide on engine idle vs true stops builds on.

Why Naive Approaches Fail

Thresholding raw speed alone is brittle. GPS speed derived by differencing consecutive fixes carries the receiver’s positional noise amplified by the sampling rate: a van parked under a bridge routinely reports 3-8 km/h of phantom motion, while a truck crawling in stop-and-go traffic dips below any fixed speed cutoff for tens of seconds without ever completing a stop. A single scalar threshold on instantaneous speed therefore produces both false stops (jitter while parked) and false motion (creep in traffic) on the same vehicle within minutes.

DBSCAN answers a different question. Density clustering is excellent for the retrospective task — take a day of pings, find the places the vehicle lingered, co-locate the 09:15 and 17:40 visits to the same depot. But it is inherently batch (it needs the whole point set), spatial-only (it discards time ordering and cannot split a morning and evening visit without a bolt-on temporal rule), and its cost grows with the number of pings rather than staying bounded. None of that suits a streaming consumer that must decide per fix.

Variance over a time window fixes both. By measuring how spread out the last W seconds of fixes are, the detector becomes:

• Online — it only needs the trailing window, so it runs incrementally as fixes arrive.
• Noise-robust — variance of position over a window averages out per-fix jitter instead of reacting to it, so a parked vehicle stays below threshold even when individual speed samples spike.
• Idle-aware — a vehicle idling at a red light has low positional variance but non-trivial speed variance over the approach, and fusing the two signals separates “waiting to move” from “arrived and parked”. That fusion is the subject of the engine idle vs true stop child page.

The trade-off is that variance windows do not natively co-locate stops across trips the way DBSCAN does; if you need “how many times this week did any vehicle visit this address”, run this detector for the online decision and reconcile emitted stop centroids downstream.

Prerequisites

Python environment: Python 3.10+, pandas ≥ 2.0 (for DataFrame.rolling with a time-based window offset), numpy ≥ 1.24, pyproj ≥ 3.5 for projection.

Coordinates in a metric CRS. Rolling positional variance is expressed in metres-squared, so easting and northing must be projected before the window runs. Apply coordinate reference system mapping to a local UTM zone. Computing variance on raw WGS84 degrees is invalid: a degree of longitude shrinks from ~111 km at the equator to 0 at the poles, so the same physical spread yields wildly different variance depending on latitude, and the threshold stops meaning anything.

Monotonic, deduplicated timestamps. Both the resampling step and the time-based rolling window assume a strictly increasing UTC index with no duplicates. Establish this with timestamp synchronisation upstream; out-of-order packets corrupt the window contents and produce spurious variance spikes.

Math background. Familiarity with sample variance and the relationship between window length and sampling interval (below). No probability model is required — the detector is deterministic given its thresholds.

Minimum fields per record: synchronised UTC timestamp, easting and northing in metres, and either a speed column or enough consecutive fixes to derive one. A Kalman velocity estimate, if you already run the filter, is a cleaner speed source than finite differencing.

Step-by-Step Workflow

Step 1 — Project and resample to a fixed cadence

The rolling window must span a constant number of seconds, which is only equivalent to a constant number of samples when the cadence is uniform. Real telematics arrives at irregular intervals (1 s on motorways, 30 s on cheap plans, gaps in tunnels), so resample each vehicle track to a fixed grid first.

import numpy as np
import pandas as pd
from pyproj import Transformer

CADENCE = "2s"          # fixed grid; match to your densest realistic sampling
MAX_GAP = pd.Timedelta("30s")  # do not interpolate across gaps longer than this

transformer = Transformer.from_crs("EPSG:4326", "EPSG:32632", always_xy=True)


def prepare_track(df: pd.DataFrame) -> pd.DataFrame:
    """Project to UTM and resample one vehicle's track to a fixed cadence."""
    df = df.sort_values("utc_ts").drop_duplicates("utc_ts").copy()
    df["easting"], df["northing"] = transformer.transform(
        df["longitude"].values, df["latitude"].values
    )
    df = df.set_index(pd.to_datetime(df["utc_ts"], utc=True))

    grid = df[["easting", "northing"]].resample(CADENCE).mean()
    # Linear interpolation, but only across short gaps
    gap = grid["easting"].isna()
    grid = grid.interpolate(method="time", limit=int(MAX_GAP / pd.Timedelta(CADENCE)))
    grid["interpolated"] = gap & grid["easting"].notna()
    return grid.dropna(subset=["easting", "northing"])

Expected output shape: a DataFrame indexed by a uniform 2s UTC grid with easting, northing, and an interpolated flag. Gaps longer than MAX_GAP remain as NaN rows that later break stop runs rather than bridging them.

Step 2 — Compute the rolling positional variance signal

The core signal is the variance of position over the trailing window. Sum the variance of easting and northing to get a single scalar spread in m²; taking the sum (equivalently the trace of the 2×2 covariance) keeps the signal rotation-invariant, so it does not matter which way the road runs.

WINDOW = "60s"   # dwell you must resolve; see the math section on window length

def rolling_position_variance(grid: pd.DataFrame, window: str = WINDOW) -> pd.Series:
    """Trace of the positional covariance over a trailing time window (m²)."""
    var_e = grid["easting"].rolling(window, min_periods=4).var()
    var_n = grid["northing"].rolling(window, min_periods=4).var()
    return (var_e + var_n).rename("pos_var")

Expected output shape: a pos_var Series aligned to the grid. For a parked vehicle it sits near the receiver noise floor (typically 4-25 m² for consumer GPS); while driving it runs into the thousands. min_periods=4 suppresses meaningless variance from a window that is barely populated after a gap.

Step 3 — Compute rolling speed statistics

Speed is the second, complementary signal. Derive it from the projected step distance divided by the cadence (or reuse a Kalman velocity), then take its rolling mean.

def rolling_speed(grid: pd.DataFrame, window: str = WINDOW) -> pd.DataFrame:
    step_m = np.hypot(grid["easting"].diff(), grid["northing"].diff())
    dt_s = grid.index.to_series().diff().dt.total_seconds()
    speed_ms = (step_m / dt_s).replace([np.inf, -np.inf], np.nan)
    speed_kmh = (speed_ms * 3.6).rename("speed_kmh")
    roll = pd.DataFrame({"speed_kmh": speed_kmh})
    roll["speed_mean"] = speed_kmh.rolling(window, min_periods=4).mean()
    roll["speed_var"] = speed_kmh.rolling(window, min_periods=4).var()
    return roll

Expected output shape: columns speed_kmh, speed_mean, speed_var. The mean gates obvious motion cheaply; speed_var is what later separates a steady idle-creep from a true stationary hold in the idle-vs-stop extension.

Step 4 — Threshold to a stationary boolean

A sample is stationary when the window around it is both spatially tight and slow. Requiring both conditions is what rejects the two failure modes of speed-only thresholding.

POS_VAR_ENTER = 30.0    # m²; below this the window is tight enough to be a stop
SPEED_ENTER = 3.0       # km/h; rolling mean speed must also be low

def stationary_mask(pos_var: pd.Series, speed: pd.DataFrame) -> pd.Series:
    tight = pos_var < POS_VAR_ENTER
    slow = speed["speed_mean"] < SPEED_ENTER
    return (tight & slow).fillna(False).rename("stationary_raw")

Expected output shape: a boolean Series. At this point it still chatters near the thresholds — a fix or two flipping the state — which Step 5 cleans up.

Step 5 — Apply hysteresis to debounce

A single threshold produces a stream of short false transitions whenever the signal hovers near it. Hysteresis uses a lower threshold to enter the stationary state than to leave it, plus a minimum-duration guard, so noise cannot toggle the state.

POS_VAR_EXIT = 60.0     # must exceed this (m²) to leave a stop — wider than ENTER
SPEED_EXIT = 6.0        # km/h to leave
MIN_STOP = pd.Timedelta("45s")

def apply_hysteresis(pos_var, speed, cadence=CADENCE):
    """Two-threshold debounced stationary state, then min-duration filter."""
    state = np.zeros(len(pos_var), dtype=bool)
    stationary = False
    for i in range(len(pos_var)):
        pv, sp = pos_var.iat[i], speed["speed_mean"].iat[i]
        if np.isnan(pv) or np.isnan(sp):
            stationary = False                      # gap breaks the run
        elif not stationary and pv < POS_VAR_ENTER and sp < SPEED_ENTER:
            stationary = True                       # enter on the tight thresholds
        elif stationary and (pv > POS_VAR_EXIT or sp > SPEED_EXIT):
            stationary = False                      # leave only on the wide thresholds
        state[i] = stationary
    out = pd.Series(state, index=pos_var.index, name="stationary")

    # Drop runs shorter than MIN_STOP (jitter that squeezed through)
    run_id = (out != out.shift()).cumsum()
    for _, idx in out.index.to_series().groupby(run_id):
        block = out.loc[idx.index]
        if block.iloc[0] and (idx.index[-1] - idx.index[0]) < MIN_STOP:
            out.loc[idx.index] = False
    return out

Expected output shape: a clean boolean stationary Series with no runs shorter than MIN_STOP and no single-sample flips. The gap between ENTER and EXIT thresholds is the debounce band.

Step 6 — Segment contiguous runs into stop events

Finally, collapse each contiguous stationary run into one stop event with arrival, departure, centroid, and dwell.

def segment_stops(grid, stationary, min_dwell=MIN_STOP):
    grid = grid.assign(stationary=stationary)
    run_id = (grid["stationary"] != grid["stationary"].shift()).cumsum()
    events = []
    for _, run in grid.groupby(run_id):
        if not run["stationary"].iloc[0]:
            continue
        arrival, departure = run.index[0], run.index[-1]
        dwell = departure - arrival
        if dwell < min_dwell:
            continue
        events.append({
            "arrival_time": arrival,
            "departure_time": departure,
            "dwell_seconds": dwell.total_seconds(),
            "centroid_easting": run["easting"].mean(),
            "centroid_northing": run["northing"].mean(),
            "ping_count": len(run),
        })
    return pd.DataFrame(events)

Expected output: one row per stop event. Hand these arrival/departure timestamps to time-window-based dwell calculation for timezone-correct dwell reporting, and back-project the centroid to WGS84 for POI matching.

The Variance Model

Sample variance over the window

For a window holding easting samples e₁ … e_n with mean ē, the sample variance is s²_e = (1/(n−1)) · Σ (eᵢ − ē)², and likewise for northing. The signal used above is their sum:

pos_var = s²_e + s²_n = trace(Σ)

where Σ is the 2×2 positional covariance of the window. The trace is invariant to rotation of the coordinate axes, so a stop is detected identically whether the vehicle was travelling north-south or east-west before halting — an important property a per-axis threshold would lack.

Window length vs sampling interval

The window spans W seconds; at cadence Δt it holds n = W / Δt samples. Two constraints bracket a good W:

• Lower bound (statistical). Variance estimated from very few samples is itself high-variance. Keep n ≥ 8-15 samples so the estimate is stable; at Δt = 2 s that means W ≥ 16-30 s.
• Upper bound (resolution). A window of length W cannot resolve two stops separated by less than W, and it delays the detected arrival by up to W/2 because the window must fill with tight fixes before variance drops. Keep W at or below the shortest dwell you must detect.

For last-mile delivery drops of 30-90 s, W = 60 s is a common compromise. If you must catch 20 s curbside handoffs, shorten the window and the cadence together.

Why variance beats mean speed near GPS jitter

Consider a genuinely parked vehicle whose true position is fixed but whose fixes scatter with per-axis standard deviation σ metres. The finite-difference speed between two consecutive fixes has expected magnitude on the order of σ·√2 / Δt; at σ = 4 m and Δt = 2 s that is ~2.8 m/s ≈ 10 km/h of phantom speed on individual samples, easily tripping a naive speed threshold. The rolling positional variance, by contrast, converges to ≈ 2σ² (the sum of both axes’ noise variance) — a small, stable number near the receiver noise floor — regardless of how the per-fix speed spikes. Averaging spread over the window is exactly the operation that cancels zero-mean jitter, which is why the variance signal is the primary gate and mean speed only a secondary sanity check.

Integration Notes

Place the detector after smoothing, before routing. Feed it the output of Kalman filtering: a smoothed track lets you lower POS_VAR_ENTER (the noise floor is smaller) without fragmenting stops, and the filter’s velocity estimate is a cleaner speed source than finite differencing. Do not, however, over-smooth — an aggressive filter that drags a parked vehicle’s estimate around will raise apparent variance.

Streaming deployment. The trailing window maps directly onto a per-vehicle_id stateful operator. In a Kafka consumer, key the stream by vehicle so each partition holds one vehicle’s ordered fixes, keep the last W seconds of (easting, northing, speed) in a bounded deque, and recompute pos_var incrementally on each arrival. Reset window and hysteresis state on a trip boundary or a gap longer than MAX_GAP. Out-of-order packets within a partition must be dropped or reordered — they inject false spread into the window.

Routing-engine coordinate order. Stop centroids sent onward to OSRM or Valhalla must be back-projected to WGS84 and ordered [longitude, latitude]; a silent lat/lon swap places the centroid in the wrong hemisphere with no error. This is the same gotcha covered in the GPS preprocessing fundamentals and applies identically here.

Confidence, not just a boolean. The margin by which pos_var sits below its threshold, the run length, and the fraction of interpolated samples in the run are all inputs to confidence scoring for stop detection. Emit those alongside each event rather than discarding them.

Operational Troubleshooting

Deployment Checklist

• Coordinates projected to a metric CRS (UTM) before any variance is computed
• Timestamps sorted, monotonic, de-duplicated; per-vehicle stream keying in place
• Track resampled to a fixed cadence; gaps longer than MAX_GAP left unbridged
• Rolling positional variance uses trace (var_e + var_n) for rotation invariance
• Window length W set to the shortest dwell you must resolve and ≥ 8-15 samples
• Both gates active: positional-variance AND rolling mean-speed thresholds
• Hysteresis configured with ENTER < EXIT thresholds plus MIN_STOP guard
• Hysteresis and window state reset on trip boundaries and long gaps
• Contiguous stationary runs segmented into events with arrival, departure, centroid, dwell
• Interpolated-sample fraction recorded per event and surfaced to confidence scoring
• Centroids back-projected to WGS84 in [lon, lat] order before routing / POI calls
• Idle vs true-stop signal (ignition / RPM) wired in where intersection false positives matter

Parent topic: Stop Detection & Dwell Time Analytics

Related

• Detecting engine idle vs true stops with variance windows — fuse ignition/RPM with the variance signal to reject red-light idling
• DBSCAN for fleet stop clustering — the batch, spatial-only counterpart for retrospective co-location of visits
• Time-window-based dwell calculation — turn arrival/departure timestamps into timezone-correct dwell durations
• Kalman filtering for GPS noise reduction — upstream smoothing that lowers the variance noise floor and supplies a clean speed estimate
• Confidence scoring for stop detection — grade each emitted stop using variance margin, run length, and interpolated fraction