Methodology, assumptions and limitations¶
This document states what gbfs-toolkit measures, the thresholds it uses, and, just as
importantly, what it cannot know. It exists so a reviewer can judge a study built on the
toolkit without reading the source. If you cite the toolkit, cite this page for the audit
definitions and the polling caveats.
1. The canonical data model¶
Ingestion is normalised once into version-independent pandas frames (see
gbfs_toolkit.core.models.SCHEMAS). Two contracts matter for reproducibility:
- Timestamps are tz-aware UTC (
datetime64[ns, UTC]). GBFS 2.x unix seconds and GBFS 3.x RFC-3339 strings are both parsed to the same dtype. Convert to local time explicitly (build_availability_panel(target_tz=…)) before any daily or diurnal aggregation. Otherwise "days" cut at UTC midnight, i.e. mid-afternoon in the Americas. - Nullable extension dtypes (
Int64,boolean,string). A field absent in an older feed version ispd.NA, never silently0orFalse. This survives the outer join injoin_availability(a station present in only one endpoint keepspd.NA, not a float-cast).
2. The static semantic audit (A1–A7)¶
The A1–A7 taxonomy is from Fossé & Pallares (gbfs-audit-catalogue). The thresholds here
are kept identical to the published catalogue so that audit_static reproduces the paper's
verdicts. They are deliberately not tuned per-feed; treat them as the published operational
definition, not as free parameters.
| Flag | Definition | Threshold | Level |
|---|---|---|---|
| A1 | Out-of-domain inclusion (car-share advertised as bike-share) | station_type == "carsharing" |
station |
| A2 | Placeholder capacity (one constant non-zero capacity across a docked system) | nunique == 1, median > 0, ≥ 20 docked stations |
system |
| A3 | Structural over-capacity (free-floating anchors) | station_type == "free_floating" |
station |
| A4 | Geospatial outlier (transposed / isolated coordinate) | nearest-neighbour distance > max(median + 3·MAD, 1000 m), ≥ 5 stations |
station |
| A5 | Out-of-perimeter coverage | system bounding box > 50 000 km² |
system |
| A6 | Zero-capacity docks | ≥ 1 % of docked stations declare capacity 0, ≥ 20 stations |
system |
| A7 | Null capacity field | ≥ 50 % of docked stations declare capacity NaN, ≥ 20 stations |
system |
Design choices worth knowing:
- A2 / A6 / A7 are docked-aware: they exclude free-floating and virtual stations, whose capacity is legitimately constant, zero, or null. Without this, a dockless system (Lime, Tier, Bird) would trip every capacity flag spuriously.
- A4 uses a robust 3σ rule (MAD-rescaled) on nearest-neighbour distance, with a 1 km floor. Formally, station \(i\) is flagged when its nearest-neighbour distance \(d_i\) satisfies
$\(d_i > \max\!\left(\operatorname{median}(d) + 3\,\mathrm{MAD}(d),\; 1000\text{ m}\right),\)$
evaluated only when at least 5 stations are present. Caveat: dock spacing is closer to a power law than Gaussian, so in a system with a dense core and sparse periphery the MAD envelope can over-flag genuinely remote suburban docks. The 1 km floor mitigates but does not eliminate this. We keep the published rule for reproducibility; if you need a different spatial-outlier model, run your own (e.g. DBSCAN noise) on the coordinates and treat A4 as one signal among several. - A5 is antimeridian-safe: the longitudinal extent uses the smallest covering arc, so a system straddling ±180° is not reported as Earth-spanning.
Why these thresholds¶
The constants are operational definitions inherited from the published catalogue, not free parameters. Their rationale is worth stating, since unjustified defaults are a common review objection.
- MAD rather than standard deviation (A4). The median absolute deviation has a 50 % breakdown point, so the outlier envelope is not itself dragged outward by the very transposed coordinates it is meant to catch. A mean-and-standard-deviation rule has a 0 % breakdown point and would mask exactly the gross errors of interest. The factor 3 is the conventional three-sigma equivalent after the MAD is rescaled to the standard deviation of a Gaussian.
- The 1 km floor (A4). In a dense urban core the MAD of nearest-neighbour distance can be a few tens of metres, which would flag ordinary spacing as anomalous. The floor sets a physically meaningful minimum below which a station is never called an outlier.
- A 20-station minimum (A2, A6, A7). System-level rate rules are unreliable on tiny systems, so the audit abstains rather than emit a high-variance verdict on fewer than 20 docked stations.
- The 1 % and 50 % rates (A6, A7). A6 fires on even a small contamination of zero-capacity docks, because a single block of placeholder capacities is enough to bias a capacity-weighted metric. A7 requires a majority of null capacities, the signature of a feed that does not populate the field at all rather than one with a few gaps.
- 50 000 km² (A5). This bounding-box area is larger than any single metropolitan bike-share service area, so exceeding it indicates merged feeds, out-of-jurisdiction stations, or mis-georeferenced coordinates rather than a genuine footprint.
Empirical validation¶
The taxonomy is validated empirically in the companion study
(gbfs-audit-catalogue, in
preparation), not only argued from first principles. Four results bear on whether the default
thresholds are defensible.
- Scale. The catalogue audits 46 307 stations across more than one hundred French operators, so the rules are exercised on real free-floating, docked and hybrid systems rather than a toy sample.
- Threshold sensitivity (A4). Sweeping the A4 cutoff from \(\sigma = 2\) to \(\sigma = 4\) moves the flagged share only from 1.29 % to 0.90 % of stations, and the systems most affected are essentially unchanged: the Kendall rank correlation of the ten most-flagged cities and operators stays at or above 0.956, and the Jaccard overlap of the top ten is 1.0 across the whole range. The published default \(\sigma = 3\) therefore sits in a flat, stable interior rather than on a knife-edge.
| \(\sigma\) | A4 stations | Flagged share | Top-10 Jaccard |
|---|---|---|---|
| 2.0 | 597 | 1.29 % | 1.0 |
| 2.5 | 543 | 1.17 % | 1.0 |
| 3.0 | 500 | 1.08 % | 1.0 |
| 3.5 | 459 | 0.99 % | 1.0 |
| 4.0 | 415 | 0.90 % | 1.0 |
- Generalisation. A leave-one-operator-out cross-validation on seven operators (Bird, Citiz, Dott, Pony, Voi, Vélib' Métropole, Vélo&Co) shows each rule firing on a held-out operator where its type predicts: the A3 free-floating rule fires on the dockless operators and not on the docked ones. This is evidence that the rules encode the intended structure rather than memorising the training systems.
- Robust A4 geometry. The topology-aware A4 detector removes the large blocks of false positives that a naive station-centroid method produces on dockless feeds (on the order of 2 700 spurious flags on a single Paris dockless operator alone), which is why A4 operates on nearest-neighbour distance rather than distance to a centroid.
A pre-registered, blind author annotation of a stratified sample (n = 422 stations) provides a
construct-validity check of the automated verdicts. Full protocols and per-operator results are in
the companion repository under experiments/.
3. The dynamic audit (D1–D3) and frozen stations¶
On a live availability snapshot:
- D1 negative:
num_bikes_available < 0ornum_docks_available < 0. - D2 over-capacity:
bikes + docks > capacity(only where capacity is a positive number). - D3 stale:
fetched_at − last_reportedexceeds the feed's advertisedttl(plus a 60 s clock-skew buffer), orstale_after_minutes(default 60) when nottlis given.
A7 is dockless-aware by default in the library: the rate is taken over physical docked
stations, so a mostly free-floating system is not flagged merely for leaving capacity null. The
original gbfs-audit-catalogue evaluated A7 over every station, under which a fully free-floating
system with null capacities is flagged. Pass audit_static(..., a7_scope="all") (or gbfs audit
--a7-scope all) to reproduce the published catalogue verdicts exactly.
detect_frozen_stations (longitudinal) is distinct from D3: a frozen station has a fresh
last_reported but a value that never changes over an active window, which is the signature of a
dead sensor rather than a stale fetch. It is also distinct from a legitimate stockout (handled by
stockout_episodes). Restrict it to active hours to avoid flagging the normal overnight
flatline.
4. Flows are observed, not inferred: the polling Nyquist limit¶
calculate_net_flow reports the observed Δ in available bikes between consecutive polls.
turnover sums |Δ|; flow_balance splits it into inflow and outflow. All three share one hard
limit:
Aliasing. Any activity that cancels within a polling interval is invisible. A bike rented and returned to the same station between two snapshots yields Δ = 0. These quantities are therefore a lower bound on true activity. Poll well below the timescale of the dynamics you want to measure, and report your polling cadence.
Formally, let \(T\) be the true number of vehicle movements at a station over a window and \(\hat{T}\) the turnover observed at polling interval \(\Delta t\). Then \(\hat{T} \le T\), with equality only if no station's count ever returns to a previous value within a single interval. Decreasing \(\Delta t\) can only raise \(\hat{T}\) toward \(T\), never overshoot it, so the estimator is conservative by construction.
The toolkit deliberately ships no rebalancing/OD attribution. Whether a Δ is a rebalancing van or organic demand is not identifiable from station-aggregate counts: even under system-wide mass conservation, an internal van move (A→B) is indistinguishable from coincident organic trips. Trip and OD reconstruction belong in dedicated research code, not here.
5. Spatial statistics: what is and isn't boundary-robust¶
coverage_statsreports density and the Clark–Evans dispersion index (boundary-robust enough for an overall verdict). The Clark–Evans aggregation index compares the observed mean nearest-neighbour distance to the value expected under complete spatial randomness:
$\(R = \frac{\bar{d}_{\text{obs}}}{\bar{d}_{\text{exp}}},\qquad \bar{d}_{\text{exp}} = \frac{1}{2\sqrt{\rho}},\)$
where \(\rho\) is the point density; \(R < 1\) indicates clustering, \(R \approx 1\) randomness (CSR),
and \(R > 1\) dispersion. Density is measured against the convex hull by default, or the real
geofencing service area when you pass the operator's zones, which is strongly preferred for
free-floating systems, where a hull badly overstates the footprint.
- morans_i (spatial autocorrelation) uses row-standardised k-NN weights; significance is the
analytic z-score under the normality assumption. Global Moran's I is
$\(I = \frac{n}{\sum_{i}\sum_{j} w_{ij}} \cdot \frac{\sum_{i}\sum_{j} w_{ij}(x_i-\bar{x})(x_j-\bar{x})}{\sum_{i}(x_i-\bar{x})^2},\)$
where the \(w_{ij}\) are the row-standardised k-NN weights.
- ripley_k has no edge correction. It is biased downward at radii approaching the
study-area size and is unreliable on irregular real boundaries (coastlines, rivers). Use it for
relative comparison at small radii; prefer Clark–Evans for an overall dispersion verdict.
- local_morans_i (LISA) gives the per-station hot-spot / cold-spot map behind the global
index, with conditional-permutation pseudo p-values. Its statistic is validated against
esda/PySAL (identical to machine precision under identical weights), and its variance follows
the same ddof=1 convention so the reported local_i is directly comparable.
Great-circle weights, not Euclidean degrees¶
All k-nearest-neighbour weights here are built on great-circle (haversine) distance via the
shared GeoKDTree, not on Euclidean distance in raw WGS-84 degrees. This is a correctness choice,
not a detail: one degree of longitude is about 111 km at the equator but only about 80 km at
Montpellier's latitude, so Euclidean-on-degrees (the default in several spatial libraries) distorts
the neighbour graph by latitude and biases every k-NN-based statistic (Moran's I, LISA, the
nearest-neighbour metrics). Computing exact great-circle distances keeps the weights valid
independent of the network's latitude. This is why a naive comparison against a library's default
Euclidean weights differs from gbfs-toolkit, while a comparison with matched weights agrees to
machine precision.
6. Inequality / equity metrics¶
concentration_metrics reports the Gini and Theil indices of capacity and the
top-decile share; lorenz_curve returns the curve points. For station capacities \(c_i\) with mean
\(\bar{c}\), the Gini index is
and the Theil index is
These are descriptive measures of how capacity is distributed across stations, deliberately kept outside the A1–A7 audit, because concentration is a property of a network's design, not a feed-quality defect.
7. Fleet reconciliation limits¶
reconcile_fleet_state excludes vehicles that carry a station_id from the deployed total so the
docked and free-floating feeds do not double-count, and reports the overlap explicitly. It relies
on the operator setting station_id correctly; a hybrid feed that lists dock-adjacent free bikes
without it will still double-count. detect_ghost_vehicles requires stable vehicle_ids.
GBFS 2.1+ rotates them for privacy, so on such feeds ghost detection is not meaningful.
8. Reproducibility & provenance¶
For a citable dataset, record generate_manifest(lake_dir) (a SHA-256 per Parquet partition plus
a system/date summary) alongside the deposit, and coverage_report(panel) to quantify
missingness. The toolkit never imputes: gaps stay NaN and are reported, not smoothed.
9. References¶
The audit taxonomy and the statistical estimators rest on the following sources.
- MobilityData. General Bikeshare Feed Specification (GBFS). github.com/MobilityData/gbfs
- Fossé, R. and Pallares, G. (2026). A certified, anomaly-flagged reference catalogue for GBFS bike-sharing feeds (
gbfs-audit-catalogue). In preparation. The A1–A7 taxonomy and thresholds originate here. - Clark, P. J. and Evans, F. C. (1954). Distance to nearest neighbor as a measure of spatial relationships in populations. Ecology, 35(4), 445–453.
- Moran, P. A. P. (1950). Notes on continuous stochastic phenomena. Biometrika, 37(1/2), 17–23.
- Ripley, B. D. (1977). Modelling spatial patterns. Journal of the Royal Statistical Society: Series B, 39(2), 172–212.
- Gini, C. (1912). Variabilità e mutabilità. Bologna.
- Theil, H. (1967). Economics and Information Theory. North-Holland, Amsterdam.