Time-Window Based Dwell Calculation

Raw speed-below-threshold flags tell you a vehicle is stationary; they do not tell you for how long, which stop is which, or whether a telemetry dropout inflated the measurement. Time-windowed dwell calculation solves that: it segments a continuous GPS stream into discrete, auditable stationary events with deterministic start timestamps, end timestamps, and durations. Without it, billing systems overcharge for detention, compliance reports mis-attribute breaks, and asset-utilization dashboards produce garbage from overnight blackouts.

This page describes a production-grade Python implementation for time-window dwell calculation. It covers the stationary-classification math, window-segmentation logic, gap-tolerance rules, timezone-safe arithmetic, and a vectorized pandas reference that handles all four common edge cases: midnight crossing, device reboot, DST transition, and half-hour timezone offsets.

Prerequisites

Your telemetry schema must expose these columns before the dwell pipeline runs:

• vehicle_id — string key, partition boundary
• timestamp — datetime64[ns, UTC], monotonically increasing per vehicle
• speed_kmh — float; NaN is treated as moving (conservative default)
• latitude, longitude — WGS84 decimal degrees
• message_id — unique string per packet (used to count points per window)

Upstream GPS noise reduction and outlier removal should already be applied. Points with speed_kmh < 0 or velocity jumps that imply impossible accelerations (> 7 m/s²) must be filtered or flagged before windowing begins.

Mathematical Model

A dwell window is a maximal contiguous sequence of telemetry records satisfying:

1. speed_kmh ≤ θ_v for at least n_min consecutive records (stationary condition)
2. No inter-record gap exceeds τ_gap seconds (continuity condition)

Let t_i be the UTC timestamp of record i. The raw dwell duration for a window spanning records [a, b] is:

D = t_b − t_a

This works for clean streams. For streams with dropout gaps, a naïve last − first over a window that includes a gap of g > τ_gap seconds will overstate dwell by up to g. The fix: split at every gap exceeding τ_gap, producing two sub-windows instead of one inflated one.

The minimum valid dwell filter then removes any resulting window where D < τ_min (default: 300 seconds / 5 minutes), which eliminates traffic-light waits and brief parking events that are too short to carry operational meaning.

Stationary hysteresis

A simple threshold speed_kmh ≤ 3.0 triggers false positives near intersections where GPS drift registers phantom 1–3 km/h motion on a stationary vehicle. A rolling minimum-points requirement adds hysteresis: require n_min consecutive points below threshold before opening a window, and require n_min consecutive points above threshold before closing it.

In practice, n_min = 3 at 30-second sampling means 90 seconds of continuous low speed before a window opens — enough to filter red lights but still capture loading bay stops.

For vehicles where CAN bus or OBD-II ignition state is available, bypass the speed threshold entirely when ignition = OFF. Cold-weather diesel trucks idle at 600 RPM with near-zero GPS speed, but the ignition signal is unambiguous.

Step-by-Step Implementation Workflow

Step 1 — Ingest and normalize telemetry

Load your partitioned Parquet dataset, cast timestamps, sort, and deduplicate:

import pandas as pd
import pyarrow.dataset as ds

dataset = ds.dataset("s3://fleet-raw/telemetry/", partitioning="hive")
df = dataset.to_table(
    filter=ds.field("date") >= "2024-01-01",
    columns=["vehicle_id", "timestamp", "speed_kmh", "latitude",
             "longitude", "message_id", "ignition"]
).to_pandas()

# Ensure UTC-aware timestamps
df["timestamp"] = pd.to_datetime(df["timestamp"], utc=True)

# Sort and deduplicate per vehicle
df = (
    df
    .sort_values(["vehicle_id", "timestamp"])
    .drop_duplicates(subset="message_id")
    .reset_index(drop=True)
)

Out-of-order records are common when cellular modems buffer and flush on reconnect. Sorting before any windowing is mandatory; any downstream diff() call on unsorted data produces negative gaps that break window-change detection.

Step 2 — Classify stationary segments

SPEED_THRESHOLD = 3.0   # km/h
MIN_STATIONARY_POINTS = 3

# Ignition-off override: if ignition=OFF treat as definitely stationary
df["speed_eff"] = df["speed_kmh"].where(df["ignition"] != "OFF", 0.0)

# Boolean stationary flag
df["is_stationary"] = df["speed_eff"].fillna(999.0) <= SPEED_THRESHOLD

# Hysteresis: require N consecutive stationary points
df["stationary_confirmed"] = (
    df.groupby("vehicle_id")["is_stationary"]
    .transform(
        lambda x: x.rolling(window=MIN_STATIONARY_POINTS, min_periods=MIN_STATIONARY_POINTS)
                   .min()
                   .astype(bool)
    )
)

fillna(999.0) treats missing speed as moving — a conservative default that avoids creating phantom dwell events from bad sensor reads. Adjust to fillna(0.0) only if your data contract guarantees that NaN speed always means ignition-off.

Expected output shape: original row count preserved, two new boolean columns added (is_stationary, stationary_confirmed).

Step 3 — Compute inter-record gaps

MAX_GAP_SECONDS = 300   # 5-minute dropout tolerance

df["time_diff_sec"] = (
    df.groupby("vehicle_id")["timestamp"]
    .diff()
    .dt.total_seconds()
    .fillna(0.0)
)

df["gap_exceeded"] = df["time_diff_sec"] > MAX_GAP_SECONDS

The gap check runs at the raw-record level before window grouping. Any record where the delta to its predecessor exceeds MAX_GAP_SECONDS carries gap_exceeded = True. This flag is then OR-combined with the stationary-state-change flag to force a new window ID.

Step 4 — Assign window IDs

# New window begins whenever: state changes OR gap exceeded
df["state_change"] = (
    df.groupby("vehicle_id")["stationary_confirmed"]
    .transform(lambda x: x != x.shift(1))
)

df["window_boundary"] = df["state_change"] | df["gap_exceeded"]
df["window_id"] = (
    df.groupby("vehicle_id")["window_boundary"]
    .transform("cumsum")
)

The cumsum pattern is the canonical vectorized window-ID assignment. It produces a monotonically increasing integer per vehicle that increments at every boundary. Each unique (vehicle_id, window_id) pair now identifies a contiguous stationary or moving segment.

Step 5 — Aggregate and filter

MIN_DWELL_SECONDS = 300   # minimum reportable dwell: 5 minutes

stationary = df[df["stationary_confirmed"]].copy()

dwell = (
    stationary
    .groupby(["vehicle_id", "window_id"], sort=False)
    .agg(
        start_time=("timestamp", "min"),
        end_time=("timestamp", "max"),
        point_count=("message_id", "count"),
        avg_speed_kmh=("speed_kmh", "mean"),
        centroid_lat=("latitude", "median"),
        centroid_lon=("longitude", "median"),
    )
    .reset_index()
)

dwell["dwell_sec"] = (
    dwell["end_time"] - dwell["start_time"]
).dt.total_seconds()

dwell["dwell_min"] = dwell["dwell_sec"] / 60.0

# Drop windows below minimum reportable dwell
dwell = dwell[dwell["dwell_sec"] >= MIN_DWELL_SECONDS].copy()

Using median for the centroid instead of mean makes the spatial center robust against the single GPS outlier that occasionally appears at the start of a stop when the device is still settling after a moving-to-stationary transition.

Expected output shape: one row per discrete dwell event, columns vehicle_id, window_id, start_time, end_time, dwell_sec, dwell_min, point_count, centroid_lat, centroid_lon.

Timezone-Safe Dwell Arithmetic

All dwell arithmetic runs in UTC. The UTC-only rule prevents two classes of silent errors:

DST spring-forward: clocks advance one hour at 02:00 local time. A vehicle stopped from 01:45 to 02:15 local spans the transition. Naïve local-time subtraction returns −45 minutes; UTC subtraction correctly returns +30 minutes.

Half-hour offsets: India (UTC+5:30), Nepal (UTC+5:45), and Newfoundland (UTC−3:30) are common in logistics networks serving South Asia or transatlantic shipping. Any pipeline step that assumes integer-hour offsets silently corrupts dwell for vehicles operating in these regions.

Convert to local time only at the reporting layer, using zoneinfo:

from zoneinfo import ZoneInfo

# timezone_map: vehicle_id -> IANA timezone string
# Example: {"VH-001": "Asia/Kolkata", "VH-002": "America/St_Johns"}

def localize_dwell(row, tz_map: dict) -> pd.Series:
    tz = ZoneInfo(tz_map.get(row["vehicle_id"], "UTC"))
    return pd.Series({
        "start_local": row["start_time"].astimezone(tz),
        "end_local": row["end_time"].astimezone(tz),
    })

tz_map = load_vehicle_timezone_map()   # from your config store
dwell[["start_local", "end_local"]] = dwell.apply(
    localize_dwell, axis=1, tz_map=tz_map
)

For detailed patterns covering cross-border fleet timezone assignment and shift-boundary alignment, see Calculating accurate dwell times across timezone shifts.

Routing and Downstream Integration

Dwell events do not live in isolation. Two integrations are immediate after window aggregation:

Spatial facility join: join centroid_lat/centroid_lon against your warehouse polygon layer using geopandas.sjoin. This is the gateway to location typing and POI matching for stops, which enriches each event with a stop_type label (warehouse_arrival, customer_delivery, unauthorized_idle).

DBSCAN re-clustering for unknown stops: when a stop centroid does not fall inside any known facility polygon, feed it into DBSCAN for fleet stop clustering as a candidate for emergent location discovery. DBSCAN groups recurring centroids at the same curbside or parking bay across multiple trips, producing discovered locations without manual polygon definition.

For driver compliance (HOS), billing detention, or SLA tracking, join start_time/end_time against appointment windows using a range join — available natively in pandas >= 2.2 with merge_asof or a direct SQL-style BETWEEN in DuckDB.

Operational Troubleshooting

Deployment Checklist

• Timestamps cast to datetime64[ns, UTC] before any arithmetic
• Records sorted by ["vehicle_id", "timestamp"] before diff()
• message_id deduplication applied before windowing
• MAX_GAP_SECONDS configured and OR-combined with state-change flag
• MIN_STATIONARY_POINTS calibrated to sampling interval (target: ≥ 90 s open time)
• MIN_DWELL_SECONDS set to at least 300 s to suppress traffic-signal noise
• Ignition-off override applied where CAN/OBD-II data is available
• Post-aggregation assertions in place: non-negative durations, end ≥ start
• Unit test covers midnight-crossing, device-reboot, DST transition, and half-hour offset scenarios
• Downstream spatial join uses median-derived centroid, not mean
• UTC → local conversion deferred to reporting layer using IANA timezone strings
• CI pipeline fails if dwell calculations deviate > 2 s from a known-good baseline fixture

Quality Assurance Metrics

Automated validation should run on every pipeline release:

• Duration monotonicity: assert (dwell["end_time"] >= dwell["start_time"]).all()
• Arithmetic consistency: dwell["dwell_sec"] must equal (dwell["end_time"] - dwell["start_time"]).dt.total_seconds() within floating-point tolerance (< 0.001 s)
• Gap coverage audit: log windows where time_gap_coverage = point_count * expected_interval / dwell_sec < 0.5 — these had significant telemetry dropout and may need manual review
• Timezone regression: use IANA Time Zone Database test vectors to confirm UTC-to-local conversions round-trip correctly through two DST boundaries per year
• Boundary alignment: no dwell event should span a vehicle shift change or a maintenance_flag record in your operational database

Scaling Dwell Computation Across a Fleet

Everything above describes one vehicle’s day. A production run processes several thousand of them every night, and the shape of that run decides whether the job finishes before the morning reports are due.

The saving grace is that dwell windowing is perfectly partitionable. No dwell event can span two vehicles, and — with one caveat covered below — none spans two operating days either. That makes the natural unit of work a (vehicle_id, operating_date) partition holding one to three thousand fixes, which windows in well under a second on a single core. A fleet of eight hundred vehicles over a thirty-day reprocessing window is twenty-four thousand such partitions, and they parallelise with no coordination whatsoever.

import polars as pl

def dwell_for_partition(frame: pl.DataFrame) -> pl.DataFrame:
    """Window one (vehicle, operating_date) partition. Pure — no shared state."""
    return (
        frame.sort("ts_utc")
        .pipe(classify_stationary)
        .pipe(assign_window_ids)
        .pipe(aggregate_windows)
    )

results = (
    fixes.lazy()
    .group_by(["vehicle_id", "operating_date"])
    .map_groups(dwell_for_partition, schema=DWELL_SCHEMA)
    .collect(streaming=True)
)

The caveat is the overnight stop. A vehicle parked at a depot from 18:40 on Tuesday to 05:20 on Wednesday produces two half-events, one closing at the partition boundary and one opening at it. Handle this by carrying a two-hour overlap into each partition and merging any two windows whose boundary fixes share an identifier — not by widening the partition, which reintroduces the very skew the partitioning removed.

Partition on the vehicle’s local operating day, not on UTC midnight. A fleet spread across two time zones that partitions on UTC will cut a night shift in half for one of them, and every downstream report that groups by day will quietly disagree with the depot’s own paperwork.

The second scaling concern is state. A naive implementation accumulates a dictionary of open windows keyed by vehicle and never evicts it, which is fine for a day and fatal for a backfill. Bound the structure explicitly: evict any window whose last fix is older than the maximum plausible dwell for that vehicle class, emit it as closed, and record that it closed on a timeout rather than on an observed departure. That distinction matters when someone later asks why a truck appears to have spent nineteen hours at a customer site.

Reconciling Dwell With Ignition and Job Data

Dwell computed from position alone is an inference. Most fleets also hold two other records of the same event — the ignition or CAN-bus feed, and the job or delivery management system — and the three rarely agree exactly. Reconciling them is not a data-quality chore; it is how the dwell figure earns the right to appear on an invoice.

The three sources fail in characteristically different ways:

A workable reconciliation rule is to take the ignition feed as the boundary authority where it exists, use position windowing to decide whether a stop happened at all, and treat the job system purely as the label for what the stop was for. Where ignition is absent, fall back to position and widen the published uncertainty rather than pretending to the same precision.

def reconcile(dwell: pl.DataFrame, ignition: pl.DataFrame) -> pl.DataFrame:
    """Snap dwell boundaries to ignition events within a tolerance window."""
    return dwell.join_asof(
        ignition.sort("ts_utc"),
        left_on="start_time",
        right_on="ts_utc",
        by="vehicle_id",
        strategy="nearest",
        tolerance="120s",
    ).with_columns(
        # Keep the ignition timestamp where one was found; else keep our own.
        pl.coalesce(["ignition_off_ts", "start_time"]).alias("start_time_final"),
        pl.col("ignition_off_ts").is_not_null().alias("boundary_from_ignition"),
    )

Publish boundary_from_ignition alongside the duration. Two dwell figures that differ by ninety seconds are not in conflict if one of them is flagged as position-derived and the other as ignition-derived — but they look like a bug if the flag is missing, and someone will spend a fortnight chasing it.

Finally, reconcile in one direction only. It is tempting to feed the job system’s timestamps back into the position pipeline to “improve” it, but that creates a loop in which the analytics validate the paperwork and the paperwork validates the analytics. Keep the derived pipeline independent, and let the disagreement between the two remain a signal you can measure.

Frequently Asked Questions

Related

• Calculating accurate dwell times across timezone shifts — deep dive on DST, half-hour offsets, and shift-boundary alignment
• DBSCAN for fleet stop clustering — spatial preprocessing that groups GPS points into stop candidates before temporal windowing
• Location typing and POI matching for stops — how dwell event centroids are matched to facility polygons and commercial POI databases
• Outlier removal in raw telematics streams — upstream filtering that prevents speed outliers from corrupting stationary classification
• Stop Detection & Dwell Time Analytics — parent overview covering the full pipeline from ingestion to enrichment