Time-Window Based Dwell Calculation

Raw speed-below-threshold flags tell you a vehicle is stationary; they do not tell you for how long, which stop is which, or whether a telemetry dropout inflated the measurement. Time-windowed dwell calculation solves that: it segments a continuous GPS stream into discrete, auditable stationary events with deterministic start timestamps, end timestamps, and durations. Without it, billing systems overcharge for detention, compliance reports mis-attribute breaks, and asset-utilization dashboards produce garbage from overnight blackouts.

This page describes a production-grade Python implementation for time-window dwell calculation. It covers the stationary-classification math, window-segmentation logic, gap-tolerance rules, timezone-safe arithmetic, and a vectorized pandas reference that handles all four common edge cases: midnight crossing, device reboot, DST transition, and half-hour timezone offsets.

Prerequisites

Your telemetry schema must expose these columns before the dwell pipeline runs:

• vehicle_id — string key, partition boundary
• timestamp — datetime64[ns, UTC], monotonically increasing per vehicle
• speed_kmh — float; NaN is treated as moving (conservative default)
• latitude, longitude — WGS84 decimal degrees
• message_id — unique string per packet (used to count points per window)

Upstream GPS noise reduction and outlier removal should already be applied. Points with speed_kmh < 0 or velocity jumps that imply impossible accelerations (> 7 m/s²) must be filtered or flagged before windowing begins.

Mathematical Model

A dwell window is a maximal contiguous sequence of telemetry records satisfying:

1. speed_kmh ≤ θ_v for at least n_min consecutive records (stationary condition)
2. No inter-record gap exceeds τ_gap seconds (continuity condition)

Let t_i be the UTC timestamp of record i. The raw dwell duration for a window spanning records [a, b] is:

D = t_b − t_a

This works for clean streams. For streams with dropout gaps, a naïve last − first over a window that includes a gap of g > τ_gap seconds will overstate dwell by up to g. The fix: split at every gap exceeding τ_gap, producing two sub-windows instead of one inflated one.

The minimum valid dwell filter then removes any resulting window where D < τ_min (default: 300 seconds / 5 minutes), which eliminates traffic-light waits and brief parking events that are too short to carry operational meaning.

Stationary hysteresis

A simple threshold speed_kmh ≤ 3.0 triggers false positives near intersections where GPS drift registers phantom 1–3 km/h motion on a stationary vehicle. A rolling minimum-points requirement adds hysteresis: require n_min consecutive points below threshold before opening a window, and require n_min consecutive points above threshold before closing it.

In practice, n_min = 3 at 30-second sampling means 90 seconds of continuous low speed before a window opens — enough to filter red lights but still capture loading bay stops.

For vehicles where CAN bus or OBD-II ignition state is available, bypass the speed threshold entirely when ignition = OFF. Cold-weather diesel trucks idle at 600 RPM with near-zero GPS speed, but the ignition signal is unambiguous.

Step-by-Step Implementation Workflow

Step 1 — Ingest and normalize telemetry

Load your partitioned Parquet dataset, cast timestamps, sort, and deduplicate:

import pandas as pd
import pyarrow.dataset as ds

dataset = ds.dataset("s3://fleet-raw/telemetry/", partitioning="hive")
df = dataset.to_table(
    filter=ds.field("date") >= "2024-01-01",
    columns=["vehicle_id", "timestamp", "speed_kmh", "latitude",
             "longitude", "message_id", "ignition"]
).to_pandas()

# Ensure UTC-aware timestamps
df["timestamp"] = pd.to_datetime(df["timestamp"], utc=True)

# Sort and deduplicate per vehicle
df = (
    df
    .sort_values(["vehicle_id", "timestamp"])
    .drop_duplicates(subset="message_id")
    .reset_index(drop=True)
)

Out-of-order records are common when cellular modems buffer and flush on reconnect. Sorting before any windowing is mandatory; any downstream diff() call on unsorted data produces negative gaps that break window-change detection.

Step 2 — Classify stationary segments

SPEED_THRESHOLD = 3.0   # km/h
MIN_STATIONARY_POINTS = 3

# Ignition-off override: if ignition=OFF treat as definitely stationary
df["speed_eff"] = df["speed_kmh"].where(df["ignition"] != "OFF", 0.0)

# Boolean stationary flag
df["is_stationary"] = df["speed_eff"].fillna(999.0) <= SPEED_THRESHOLD

# Hysteresis: require N consecutive stationary points
df["stationary_confirmed"] = (
    df.groupby("vehicle_id")["is_stationary"]
    .transform(
        lambda x: x.rolling(window=MIN_STATIONARY_POINTS, min_periods=MIN_STATIONARY_POINTS)
                   .min()
                   .astype(bool)
    )
)

fillna(999.0) treats missing speed as moving — a conservative default that avoids creating phantom dwell events from bad sensor reads. Adjust to fillna(0.0) only if your data contract guarantees that NaN speed always means ignition-off.

Expected output shape: original row count preserved, two new boolean columns added (is_stationary, stationary_confirmed).

Step 3 — Compute inter-record gaps

MAX_GAP_SECONDS = 300   # 5-minute dropout tolerance

df["time_diff_sec"] = (
    df.groupby("vehicle_id")["timestamp"]
    .diff()
    .dt.total_seconds()
    .fillna(0.0)
)

df["gap_exceeded"] = df["time_diff_sec"] > MAX_GAP_SECONDS

The gap check runs at the raw-record level before window grouping. Any record where the delta to its predecessor exceeds MAX_GAP_SECONDS carries gap_exceeded = True. This flag is then OR-combined with the stationary-state-change flag to force a new window ID.

Step 4 — Assign window IDs

# New window begins whenever: state changes OR gap exceeded
df["state_change"] = (
    df.groupby("vehicle_id")["stationary_confirmed"]
    .transform(lambda x: x != x.shift(1))
)

df["window_boundary"] = df["state_change"] | df["gap_exceeded"]
df["window_id"] = (
    df.groupby("vehicle_id")["window_boundary"]
    .transform("cumsum")
)

The cumsum pattern is the canonical vectorized window-ID assignment. It produces a monotonically increasing integer per vehicle that increments at every boundary. Each unique (vehicle_id, window_id) pair now identifies a contiguous stationary or moving segment.

Step 5 — Aggregate and filter

MIN_DWELL_SECONDS = 300   # minimum reportable dwell: 5 minutes

stationary = df[df["stationary_confirmed"]].copy()

dwell = (
    stationary
    .groupby(["vehicle_id", "window_id"], sort=False)
    .agg(
        start_time=("timestamp", "min"),
        end_time=("timestamp", "max"),
        point_count=("message_id", "count"),
        avg_speed_kmh=("speed_kmh", "mean"),
        centroid_lat=("latitude", "median"),
        centroid_lon=("longitude", "median"),
    )
    .reset_index()
)

dwell["dwell_sec"] = (
    dwell["end_time"] - dwell["start_time"]
).dt.total_seconds()

dwell["dwell_min"] = dwell["dwell_sec"] / 60.0

# Drop windows below minimum reportable dwell
dwell = dwell[dwell["dwell_sec"] >= MIN_DWELL_SECONDS].copy()

Using median for the centroid instead of mean makes the spatial center robust against the single GPS outlier that occasionally appears at the start of a stop when the device is still settling after a moving-to-stationary transition.

Expected output shape: one row per discrete dwell event, columns vehicle_id, window_id, start_time, end_time, dwell_sec, dwell_min, point_count, centroid_lat, centroid_lon.

Timezone-Safe Dwell Arithmetic

All dwell arithmetic runs in UTC. The UTC-only rule prevents two classes of silent errors:

DST spring-forward: clocks advance one hour at 02:00 local time. A vehicle stopped from 01:45 to 02:15 local spans the transition. Naïve local-time subtraction returns −45 minutes; UTC subtraction correctly returns +30 minutes.

Half-hour offsets: India (UTC+5:30), Nepal (UTC+5:45), and Newfoundland (UTC−3:30) are common in logistics networks serving South Asia or transatlantic shipping. Any pipeline step that assumes integer-hour offsets silently corrupts dwell for vehicles operating in these regions.

Convert to local time only at the reporting layer, using zoneinfo:

from zoneinfo import ZoneInfo

# timezone_map: vehicle_id -> IANA timezone string
# Example: {"VH-001": "Asia/Kolkata", "VH-002": "America/St_Johns"}

def localize_dwell(row, tz_map: dict) -> pd.Series:
    tz = ZoneInfo(tz_map.get(row["vehicle_id"], "UTC"))
    return pd.Series({
        "start_local": row["start_time"].astimezone(tz),
        "end_local": row["end_time"].astimezone(tz),
    })

tz_map = load_vehicle_timezone_map()   # from your config store
dwell[["start_local", "end_local"]] = dwell.apply(
    localize_dwell, axis=1, tz_map=tz_map
)

For detailed patterns covering cross-border fleet timezone assignment and shift-boundary alignment, see Calculating accurate dwell times across timezone shifts.

Routing and Downstream Integration

Dwell events do not live in isolation. Two integrations are immediate after window aggregation:

Spatial facility join: join centroid_lat/centroid_lon against your warehouse polygon layer using geopandas.sjoin. This is the gateway to location typing and POI matching for stops, which enriches each event with a stop_type label (warehouse_arrival, customer_delivery, unauthorized_idle).

DBSCAN re-clustering for unknown stops: when a stop centroid does not fall inside any known facility polygon, feed it into DBSCAN for fleet stop clustering as a candidate for emergent location discovery. DBSCAN groups recurring centroids at the same curbside or parking bay across multiple trips, producing discovered locations without manual polygon definition.

For driver compliance (HOS), billing detention, or SLA tracking, join start_time/end_time against appointment windows using a range join — available natively in pandas >= 2.2 with merge_asof or a direct SQL-style BETWEEN in DuckDB.

Operational Troubleshooting

Deployment Checklist

• Timestamps cast to datetime64[ns, UTC] before any arithmetic
• Records sorted by ["vehicle_id", "timestamp"] before diff()
• message_id deduplication applied before windowing
• MAX_GAP_SECONDS configured and OR-combined with state-change flag
• MIN_STATIONARY_POINTS calibrated to sampling interval (target: ≥ 90 s open time)
• MIN_DWELL_SECONDS set to at least 300 s to suppress traffic-signal noise
• Ignition-off override applied where CAN/OBD-II data is available
• Post-aggregation assertions in place: non-negative durations, end ≥ start
• Unit test covers midnight-crossing, device-reboot, DST transition, and half-hour offset scenarios
• Downstream spatial join uses median-derived centroid, not mean
• UTC → local conversion deferred to reporting layer using IANA timezone strings
• CI pipeline fails if dwell calculations deviate > 2 s from a known-good baseline fixture

Quality Assurance Metrics

Automated validation should run on every pipeline release:

• Duration monotonicity: assert (dwell["end_time"] >= dwell["start_time"]).all()
• Arithmetic consistency: dwell["dwell_sec"] must equal (dwell["end_time"] - dwell["start_time"]).dt.total_seconds() within floating-point tolerance (< 0.001 s)
• Gap coverage audit: log windows where time_gap_coverage = point_count * expected_interval / dwell_sec < 0.5 — these had significant telemetry dropout and may need manual review
• Timezone regression: use IANA Time Zone Database test vectors to confirm UTC-to-local conversions round-trip correctly through two DST boundaries per year
• Boundary alignment: no dwell event should span a vehicle shift change or a maintenance_flag record in your operational database

Related

• Calculating accurate dwell times across timezone shifts — deep dive on DST, half-hour offsets, and shift-boundary alignment
• DBSCAN for fleet stop clustering — spatial preprocessing that groups GPS points into stop candidates before temporal windowing
• Location typing and POI matching for stops — how dwell event centroids are matched to facility polygons and commercial POI databases
• Outlier removal in raw telematics streams — upstream filtering that prevents speed outliers from corrupting stationary classification
• Stop Detection & Dwell Time Analytics — parent overview covering the full pipeline from ingestion to enrichment