Changelog¶
All notable changes are documented here (Keep a Changelog, SemVer).
[1.7.2] - pandas 3.0 compatibility and benchmarks¶
Fixed¶
- pandas 3.0 compatibility.
offset_minutes(behindservice_reliability_index,temporal_concentration,turnover,fleet_turnover_proxy, ...) raised a spurious "not a fixed-width frequency" error for"1D"on pandas 3.0, becausepd.Timedelta(Day())now raises there. It uses the cross-versionoffset.nanosprimitive again, so daily frequencies work on both pandas 2.x and 3.0. This is the bug the new core-only CI job (the only one running on the latest pandas) caught.
Added¶
- A
benchmarks/suite (pytest benchmarks/ --benchmark-only) profiling the heaviest descriptive functions on realistic synthetic panels, run informationally in CI. The core-only CI job now also runs the dependency-free test files, so a pandas-major incompatibility is caught before release.
[1.7.1] - supply-chain and hygiene¶
Changed¶
- Supply chain. All GitHub Actions are now pinned by full commit SHA (not a moving
tag), and the CI workflow runs with least-privilege
contents: read. - Dependency policy. Documented (README): lower bounds only, following SPEC 0; speculative upper caps are avoided because they cause resolution conflicts without guarding against real breakage. Support tracks SPEC 0 / NEP 29.
- Internal: the GBFS timestamp parser is a single
parse_gbfs_timestamphelper incore(the byte-identical_utc/_utc_tscopies in the io layer were merged).
Added¶
- A
feature_requestissue template (the governance set is now complete). - A pytest
filterwarningsrule that silences the upstream pandas 2.3 / numpy 2.5pd.Timedeltageneric-unitDeprecationWarning(an internal-pandas issue, not our usage), so genuine warnings stay visible in the test output.
[1.7.0] - correctness, consistency and hardening (professionalism audit)¶
Deprecated¶
dynamic_gini_index(target_col=...)andtemporal_autocorrelation(column=...)are renamed tovalue_colfor consistency with the rest of the library. The old keywords still work for one release cycle and emit aFutureWarning.
Changed¶
- Uniform validation contract.
concentration_metrics,lorenz_curve,morans_i,ripley_k,coverage_statsandstation_statenow raise the didacticSchemaError(viarequire_columns) on a missing required column instead of a rawKeyError.system_profilestays deliberately lenient (a profile/describe works on any frame). - Internal consolidation. The equirectangular projection behind
ripley_k,coverage_statsand the clustering helpers now routes through the singleproject_metershelper; the A3 ratio threshold lives incore.modelswith the other audit thresholds. - io error taxonomy.
fetch_feed_jsonnow raisesGBFSFetchError(not a rawrequests.HTTPError) on a 4xx/5xx or a malformed body, andresolveraisesGBFSDiscoveryError(aKeyErrorsubclass) for an unknownsystem_id, soexcept GBFSErrorcatches both as the error hierarchy promises. - The
gbfsCLI now prints a one-line message to stderr and exits with code 2 on a missing file, malformed JSON or fetch failure, instead of dumping a Python traceback. - Release pipeline gates on tests.
release.ymlnow runs the suite (py3.10 + py3.12) and only builds/publishes to PyPI if it passes, so a tag whose tests are red can no longer ship a wheel. - Documentation cross-references render. ~120 Sphinx
:func:/:class:/:data:roles (which displayed as raw text under mkdocstrings) were rewritten to the mkdocstrings autoref form. - Internal hygiene: the panel-flatten idiom is now the single
panel_framehelper (10 call sites de-duplicated), the time-of-day bucket helper is reused instead of inlined, andrequire_columns(the validation primitive) is now public for extension authors.
Fixed¶
- Join-key dtype mismatch (silent wrong results / merge error).
to_canonical_vehiclesandto_canonical_station_infoleftvehicle_type_id/region_idnumeric while their lookup tables coerced them to strings, sojoin_vehicle_typesand region joins failed (all-NaNor a merge error) for any feed using integer ids. All foreign keys are nowstring-typed on both sides. spatial_outage_redundancygave wrong results on real sparse panels: an unobserved neighbour was counted as empty (so neighbourhood deserts were over-reported), and duplicate(station, timestamp)polls were summed rather than de-duplicated. A systemic outage now requires at least one observed neighbour and treats unobserved neighbours as unknown; repeated polls are de-duplicated (last kept).boundary_stressnulled the whole drop-off metric when acapacitycolumn was present but all-NaN(a common GBFS case). Only a finite non-positive capacity now means "no physical docks"; missing capacity is treated as unknown.- Non-fixed frequencies (
service_reliability_index,temporal_concentration) now raise a clearValueErrorfor offsets with no constant width ("ME","W", ...) instead of a cryptic internal error. spatial_outage_redundancyis now order-independent: a duplicate(station, timestamp)is resolved deterministically (highest reported count) rather than by input row order._hull_area_km2(behindripley_k/coverage_stats) usesnanmean, so a singleNaNcoordinate no longer poisons the projected area.vehicle_id_persistencecoerces timestamps witherrors="coerce"(a bad timestamp yieldsNaTrather than raising), matchingaudit_dynamic.
Added¶
- A CI job that installs the package core-only (no extras) and runs a smoke import plus the dependency-free test subset, so a code path silently needing an optional or transitive package fails CI instead of a user's environment.
- An input-purity test asserting that public functions never mutate their input
frames, and least-privilege
permissions: contents: readon the CI workflow.
[1.6.1] - patch¶
Fixed¶
format_paper_summary(fmt="latex")now generates the LaTeX table directly instead of delegating toDataFrame.to_latex, which on some pandas builds routes through the Styler and raisesImportErrorwhenjinja2is not installed. The function is now dependency-free for both Markdown and LaTeX output.
[1.6.0] - feed governance, service stress, panel ergonomics¶
Added¶
Four descriptive metrics (grounded in the bike-share literature and an external review) and a set of everyday panel utilities. All within the descriptive scope (no origin-destination, routing, prediction or imputation).
- Feed governance.
vehicle_id_persistencecharacterises whether a feed rotates or keeps itsvehicle_id(the GBFS 2.1+ privacy guidance), via the rolling Jaccard overlap of the live id-set and the observed id lifespan. The inverse of this persistence is the ceiling on origin-destination identifiability, so it is the check that tells a study whether trip-level work is even admissible (a feed property, not trip inference). - Service stress.
boundary_stressreports the share of time a station sits near empty or full (absolute thresholds, default<= 2), the perceived-unreliability that the strictstation_outage_rates(== 0) undercounts; drop-off stress isNAfor free-floating / zero-capacity stations. - Spatial redundancy.
spatial_outage_redundancyseparates a station's local stockouts from systemic failures where every station within a walking radius is also empty, the rupture a user cannot walk around (uses the existingGeoKDTree). - System-level coverage.
coverage_report(level="system")summarises a feed's temporal completeness for a paper's data section: window, median cadence, cadence jitter and overall station-hours yield (the per-station report is unchanged,level="station"). - Panel ergonomics.
add_local_time(tz conversion that handles the index),resample_panel(dtype-safe step-function resampling onto a fixed grid),insert_explicit_gaps(mark collection outages withNaNrows so plots break honestly),extract_snapshot_asof(the city's state at one instant),to_wide_matrix(long to station-by-time matrix) andfilter_by_bbox(the missing rectangular spatial filter).
[1.5.0] - feed-first audit, research algorithms, generated reference¶
Added¶
Audit goes feed-first and batch, and the descriptive analysis surface gains the algorithms a quantitative study reports. All within the descriptive scope (no origin-destination, routing, prediction or imputation).
- Audit pipeline.
audit_staticnow exposes every policy threshold (a5_area_km2,a6_tau,a7_tau,n_min;a4_sigmaalready public), defaults unchanged.audit_sensitivity(threshold-robustness sweep with the Jaccard overlap of the flagged set) andflag_rate_ci(seeded cluster-bootstrap intervals on flag rates) make robustness and uncertainty reproducible from one call. - Feed-intrinsic classification.
classify_from_vehicle_types(A1 car-sharing from the GBFS v3form_factor),classify_from_virtual_station(free-floating fromis_virtual_station),overcapacity_ratio+reclassify_overcapacity(the A3 conditional-averaging signature),capacity_convention(the six capacity semantics) andflag_sentinel_coordinates(the (0,0) null-island filter). - Batch audit.
audit_cataloguefetches and audits many systems in one call, returning the per-station verdict and a per-system status. - Research algorithms.
fdr_adjustandlocal_morans_i(fdr=True)(Benjamini-Hochberg control for LISA),theil_index(between/within decomposable) andpalma_ratio(equity),two_step_fcaexponential (gravity) decay (E2SFCA),rebalancing_tension(minimum-work spatial fragmentation via the Wasserstein earth-mover distance),block_bootstrap_ci+effective_sample_size(honest uncertainty for autocorrelated series), andcensored_time_ratio(observability loss at saturation). - Human-validation helpers.
krippendorff_alpha,cohen_kappa,wilson_intervalfor construct-validity studies.
Fixed¶
resolvepreferred the operator websiteurlover the GBFS auto-discovery endpoint (fetched the homepage instead ofgbfs.json).ginicould return a tiny negative on all-equal inputs (float roundoff).
Documentation¶
- The API reference is now a generated per-function catalogue (one page per object
from
__all__, a thematic landing) with clickable pandas/numpy/scipy types and a source button; versioned withmike. DoctestedExampleson the analytical surface (run in CI), a docstring-coverage gate (interrogate), and a ten-page How-To scenario set (audit, collection, history, equity, rigour, free-floating & ghost fleets, reliability, context, macro-scale).
[1.4.0] - advanced descriptive analytics¶
Added¶
Five research-grade descriptive functions, all on the .gbfs accessor and within the descriptive
scope (no origin-destination, routing, prediction or imputation). Closes #2.
local_morans_i(LISA): per-station spatial-autocorrelation hotspots and cold spots with conditional-permutation pseudo p-values and HH/LL/HL/LH cluster labels, where the globalmorans_ionly gives one number.[geo]/scipy. Anselin (1995).availability_synchrony: pairwise correlation of station availability series over their common support, returned as an upper-triangle edge list for downstream network analysis (bring your own graph). Correlates observed availability only, never trips. O'Brien et al. (2014).diurnal_bimodality: Sarle's bimodality coefficient of each station's diurnal profile, a continuous scalar separating commuter (bimodal) from recreational (unimodal) stations.outage_survival: empirical survival function and median / P90 time-to-recovery of stockout episodes, with the right-censoring caveat stated and never imputed. Kaplan-Meier (1958).temporal_concentration: per-station Gini of activity across time-of-day bins (temporal peaking), the temporal analogue ofdynamic_gini_index.
[1.3.0] - descriptive research indicators¶
Added¶
Seventeen pure, descriptive indicators that turn the panel into publication-ready summary
statistics. All are strictly descriptive (no origin-destination, routing, prediction or
imputation), operate on the canonical frames, and are exposed on the .gbfs accessor.
- Service and equity:
service_reliability_index(level-of-service probability per station and time-of-day),station_outage_rates(stockout and saturation fractions),capacity_utilization(bikes over capacity, nullable),dynamic_gini_index(Gini of available bikes over time),two_step_fca(two-step floating catchment area accessibility,[geo]). - Observed dynamics:
flow_asymmetry_ratio(inflow over outflow),fleet_turnover_proxy(a lower-bound usage rate per vehicle),cumulative_imbalance(drift),docking_pressure(typical inflow over free docks),spatial_center_of_mass(fleet centre of gravity over time),spatial_entropy(Shannon entropy of the free-floating distribution). - Temporal and sampling:
temporal_autocorrelation(ACF at hour/day/week lags),aliasing_vulnerability(a Nyquist-limit diagnostic),diurnal_summary_stats(hour-of-day mean/median/P5/P95),temporal_context_features(is_weekend, time_block, optional is_holiday). - Fleet and exogenous:
vehicle_idle_time(zombie-fleet share over time),join_exogenous_timeseries(safemerge_asofof weather/traffic onto the panel, bring your own).
[1.2.0] - reproduce the audit catalogue from the library¶
Added¶
audit_staticgains two keyword options so the publishedgbfs-audit-catalogueverdicts can be reproduced exactly from the library:a7_scope="docked"(default) or"all". The default keeps A7 dockless-aware (the toolkit's behaviour since 1.0)."all"evaluates the null-capacity rate over every station, reproducing the catalogue's original A7, under which a fully free-floating system with null capacities is flagged.a4_sigma=3.0(default). Exposes the A4 nearest-neighbour outlier multiplier for sensitivity analysis, so a threshold sweep no longer needs to monkey-patch a module constant.- Verified byte-identical against the 46 307-station catalogue: with
a7_scope="all", all of A1 to A7 match the published flags, anda4_sigmareproduces the published 2.0 to 4.0 sweep. - CLI rendering adapts to context.
gbfs auditprints a coloured table whenrichis installed and the output is an interactive terminal, plain text otherwise. New flags:--json(machine-readable output for pipelines),--a7-scope {docked,all}, and--no-color.richlives behind the new optional[cli]extra; the core install is unchanged. fetch_multiple(progress=True)reports progress on a long multi-system pull: a tqdm bar when tqdm is installed (also in[cli]), otherwise periodic log lines. Feeds are now consumed as they complete, and a per-system failure is logged as well as recorded.- Doctests on
occupancyandstation_state, run in CI, so the reference examples cannot drift from the code.
[1.1.0] - conformance & robustness (from real-world migration)¶
Added¶
detect_frozen_stationsgainsstrict=True(a column counts as frozen only if it never changes across the whole observed window, not merely a long run) andcolumns=(...)(require all listed columns frozen, e.g. bikes and docks). Motivated by cross-validating thegbfs-dynamic-auditzombie detector against the toolkit.- Vehicle schema gains
current_fuel_percent(GBFS 3.0 battery fraction).
Fixed / Changed¶
- Language-nested feeds (old GBFS 1.x/2.x
data.<lang>.<key>) are now tolerated by every normalizer (to_canonical_station_info/_status/_vehicles/_vehicle_types/_pricing_plans/_system_regions/_alerts), not just discovery. Surfaced by migrating bikeshare-data-explorer, whose collectors kept hand-rolled flattening for this reason. to_canonical_station_statusalso reads GBFS 3.0vehicle_docks_availableas a fallback fornum_docks_available(companion to the 1.0.1num_vehicles_availablefix).- CI now runs the
load_exampledoctest.
[1.0.1]¶
Fixed¶
to_canonical_station_statusnow reads the GBFS 3.0num_vehicles_availablefield as a fallback fornum_bikes_available(3.0 renamed it). Surfaced while migrating a real consumer (bikeshare-data-explorer) onto the toolkit.
[1.0.0] - first stable release¶
Promotes 1.0.0rc1 unchanged (validated by a clean install from PyPI and the full test suite).
The canonical schema and public API are now stable under SemVer. Development status →
Production/Stable.
[1.0.0rc1] - first public release candidate¶
Frozen canonical schema and public API after three peer-review passes. Adds METHODOLOGY.md
(audit thresholds, the polling/aliasing limit, spatial-stat caveats), CONTRIBUTING.md, and a
PyPI Trusted-Publishing release workflow. The sections below list everything since 0.1.0.
Added (library-API conventions)¶
.gbfspandas accessor: fluent chaining over the pure functions:df.gbfs.audit(),av.gbfs.occupancy(),panel.gbfs.net_flow(),info.gbfs.join_status(status). Single-frame ops map directly; two-frame ops take the second frame as the argument.load_example(): a small, deterministic bundled GBFS snapshot (central Paris) for docs, doctests and offline tests; returns canonical(station_info, station_status).show_versions(): environment/dependency diagnostic for bug reports.validate_schema(df, schema)/coerce_schema(df, schema): public schema check/cast over the canonical contracts (SCHEMASregistry); assert or repair a mutated frame before writing it.GBFSFeed.__repr__/_repr_html_: readable repr in shells/Jupyter (cached state only, never triggers a network call).
Added (distilled from the lab's research code)¶
detect_frozen_stations(panel): flags a value stuck unchanged over an active window while the feed stays fresh (dead sensor), distinct from staleness (D3) and stockouts. Seen reimplemented across three dynamic-audit repos.flow_balance(panel): per-station inflow/outflow split + source↔sink balance ratio (the "Keq" several notebooks computed by hand).turnover(..., normalize="capacity"): capacity-normalised activity, comparable across station sizes.normalize_operator(name): canonical operator brand from a system id/name (smovengo→Vélib' Métropole); non-lossy. Lifted from the audit-catalogue'sdetect_operator.cyclical_time_features(timestamps): sin/cos calendar encoding (the single most duplicated helper across the lab's repos).
Added (ergonomic one-liners)¶
drop_flagged(stations): audit and keep the clean subset in one call.occupancy(availability): the bikes/(bikes+docks) ratio, vectorised, NaN-safe on the empty-and-no-docks case (everyone was recomputing it inconsistently).filter_vehicles(vehicles, types, form_factor=…, propulsion=…)andebikes(...): resolve vehicle types and filter in one call ("where are the e-bikes?").- In-process catalogue cache:
systems_catalognow memoises the parsed registry for the process (withrefresh=Trueto force), so resolving many systems in a loop downloads once.
Added (research convenience helpers)¶
stockout_episodes(panel): contiguous empty/full outage events per station (start, end, duration, n_obs); the service-quality complement tocoverage_report/availability_stats.turnover(panel, freq="1D"): per-station Σ|net_flow| activity proxy (a documented lower bound, by the aliasing argument).network_changes(old, new): diff two station inventories: stations added / removed / recapacitated / moved (with distance), for longitudinal studies that span network growth.stations_near(points, info, radius_m): accessibility primitive: per external POI, count of stations within a radius + nearest distance/id (the inverse offeatures_within, for equity).to_geojson(frame_or_gdf, path=...): export stations/zones to GeoJSON for QGIS / kepler.gl.join_vehicle_types/join_pricing: resolvevehicle_type_id/pricing_plan_idonto a vehicles frame so "where are the e-bikes?" / cost joins are one call.
Quality¶
- Input-validation guards:
join_availability,calculate_net_flow,coverage_report,detect_ghost_vehiclesandlink_transit_stopsnow raise a clearSchemaErrornaming the missing columns instead of a crypticKeyErrordeep in the call. - Test & coverage pass: 99 → 115 tests, coverage 83% → 94%; added a CI coverage gate
(
--cov-fail-under=85). New coverage for the dynamic audit,fetch_multiple/ module one-liners / feed delegators (offline), conditional GET, catalogue cache, the OSM geopandas path, panel resampling, manifests, and stats on empty/degenerate inputs.
Added¶
- Ghost-vehicle detection (
fleet.detect_ghost_vehicles(vehicle_panel, idle_days=14, move_threshold_m=50)): flags free-floating units advertised at the same spot for a long span (lost / broken / abandoned but still inflating availability), from a longitudinal vehicle panel. Returns per-vehiclefirst_seen, last_seen, n_obs, observed_days, max_displacement_m, is_ghost. Completes the dynamic fleet-health story alongside D1–D3.
[0.8.0] - v1.0-readiness pass (provenance, robustness, methodology)¶
Added¶
- Provenance / citability (
timeseries):coverage_report(panel, expected_freq)quantifies per-station uptime and longest gap (missingness without imputation);generate_manifest(lake_dir)emits a SHA-256-per-partition manifest + dataset summary for Zenodo/Dataverse deposits. - Polite networking (
fetch):build_session()(pooledrequests.Sessionwith retry/backoff on 429/5xx, now the default infetch_multiple);fetch_feed_json(url, etag=..., last_modified=...)does conditional GETs and raisesGBFSNotModifiedon HTTP 304; structured logging under thegbfs_toolkitlogger. - Exception hierarchy (
errors):GBFSErrorbase withGBFSFetchError,GBFSDiscoveryError(also aKeyErrorfor back-compat),GBFSValidationError,GBFSNotModified.SchemaErrornow subclassesGBFSValidationError(and stillValueError). - New canonical endpoints:
to_canonical_system_regions(region lookup) andto_canonical_alerts(system_alerts: disruptions that explain data anomalies), plusGBFSFeed.system_regions()/.alerts(). - Catalogue offline fallback:
systems_catalogcaches successful downloads and falls back to the cached copy (with a warning) when the registry is unreachable. - Documented methodology limits: the aliasing / polling-Nyquist caveat on
calculate_net_flow(net flow is a lower bound on activity) and a prominent edge-effect warning onripley_k. - e2e round-trip test (raw → canonical → parquet → panel → net_flow + audit_frames + coverage) and CLI tests; coverage 83% → 87%.
Changed¶
- Version
0.1.0→0.8.0; development status Alpha → Beta.
[1.0.0rc1] - pre-1.0 consolidation (second peer-review pass)¶
Changed / Removed¶
- Decoupled analysis from fetching. The join and audit logic are now pure functions on
canonical frames:
join_availability(info, status)andaudit_frames(info, status=..., ttl_seconds=..., system_id=...), so they work on frames read back from a Parquet lake, not only on a liveGBFSFeed.GBFSFeed.availability()/.audit()are kept as thin delegators (no behaviour change for online use). - The
availability()presencecolumn is now a fixed-categoryCategorical(both/info_only/status_only) instead of a free string. - Removed
osm.fetch_osm_around: fetching from OSM's rate-limited Overpass endpoint violated the no-HTTP / Bring-Your-Own-GeoDataFrame contract and was a CI/issue liability. Fetch withosmnxin your own script and pass the result toenrich_with_osm. calculate_net_flownow reports the observed Δ only. Removedaccount_for_system,system_net_flowandis_rebalancing_suspected: attributing a flow to rebalancing vs. organic demand is not identifiable from availability counts (even with system-wide mass conservation), so the library no longer ships a misleading cause label.
Changed / Fixed (hardening, peer-review pass)¶
- Nullable dtypes in
to_canonical_station_status(Int64counts,booleanflags) andto_canonical_vehicles, so theavailability()outer join insertspd.NAinstead of silently upcasting integer counts and boolean flags tofloat64(which corrupted equality and boolean logic on orphaned stations). - A7 (null capacity) is now dockless-aware: restricted to physical docked stations (excludes free-floating and virtual anchors, like A2/A6 already did). Mostly-dockless systems (Lime/Tier/Bird), whose capacity is null by design, no longer trip A7 spuriously.
- A5 (bounding box) is antimeridian-safe: longitudinal extent now uses the smallest covering arc, so a system straddling ±180° is no longer reported as Earth-spanning.
calculate_net_flow(account_for_system=True)adds mass-conservation context: asystem_net_flowcolumn and a corroboratedis_rebalancing_suspectedthat fires only when a station spike coincides with a same-sign system-wide change (fleet injection/removal). Internal van moves stay indistinguishable from organic demand at panel resolution (documented).fetch_multiple(..., session=...)accepts a sharedrequests.Sessionto pool connections across systems (avoids TCP/port exhaustion when polling many feeds on a schedule).
Schema (future-proofing before 1.0)¶
STATION_STATUS_COLUMNSgainsis_installed(hardware deployed vs.is_renting).VEHICLE_STATUS_COLUMNSgainscurrent_range_meters(e-bike/battery research) andpricing_plan_id(preserved, not parsed, for equity/pricing joins).
Added¶
- Descriptive stats (
stats):system_profile(one-glance numeric card of a snapshot: stations, capacity, occupancy, % empty/full/disabled/virtual, staleness),compare_systems(stacks profiles across cities into one table),concentration_metrics(capacity Gini + top-decile hub share, an equity lens kept outside the A1–A7 audit), andavailability_stats(per-station longitudinal scalars: occupancy, time empty/full, volatility, diurnal amplitude, peak hour), andcoverage_stats(station density, nearest-neighbour spacing, and the Clark–Evans dispersion index: density measured against the convex hull, or the real geofencing service area when zones are passed). Pure, pandas-only, strictly descriptive. - Standard spatial / inequality algorithms (
stats, numpy/scipy only, deterministic):morans_i(global Moran's I spatial autocorrelation + analytic z-score/p-value via k-NN weights),ripley_k(Ripley's K/L multi-scale clustering, density vs. hull or service area),lorenz_curve(inequality-curve points), and a Theil index added toconcentration_metrics. - Per-vehicle-type station counts (
to_canonical_station_vehicle_counts): melts GBFS 2.2+/3.xvehicle_types_availableinto a long frame (STATION_VEHICLE_COUNTS_COLUMNS), so "where are the e-bikes?" is a join: the aggregatenum_bikes_availablecan't answer it. - Pricing-plan lookup (
to_canonical_pricing_plans): parsessystem_pricing_plans.jsonintoPRICING_PLAN_COLUMNS, resolving thepricing_plan_idforeign key for cost/equity work. target_tzonbuild_availability_panel: convertsfetched_at/last_reportedto a local zone before dedup/resample, so daily aggregations cut at local midnight (UTC-midnight cuts silently corrupted diurnal analysis).- Parquet pushdown in
build_availability_panel(columns=..., filters=...): project only the columns you need (join/dedup keys always read) and AND an extrapyarrow.datasetpredicate with the built-in system/date filter, so multi-month / multi-city panels prune row-groups before materialising instead of OOM-ing. - Fleet reconciliation (
fleet):reconcile_fleet_state(station_status, vehicles)(andGBFSFeed.reconcile_fleet()) merge the docked aggregate counts and the per-vehicle feed into one labelled tally:available_in_stations,free_floating_available/_reserved/ _disabled,total_deployed,total_rentable. Vehicles carrying astation_idare excluded from the deployed total (so the two feeds don't double-count) and the overlap is reported asdocked_in_vehicle_feed/double_count_avoided.VEHICLE_STATUS_COLUMNSgainsstation_id(set when a vehicle is parked at a station, else NA → free-floating). - Geofencing / service areas (
geofencing, extra[geo]):to_canonical_geofencing(raw, system_id=...)parsesgeofencing_zones.jsoninto a canonicalGeoDataFrame(one row per zone, shapely geometry in EPSG:4326; v2.xride_allowedand v3.xride_start/ride_end_allowedreconciled; full per-vehicle-typerulespreserved).zones_for_pointsis the point-in-zone spatial join (which zone each station/vehicle sits in),zone_area_km2reprojects to an equal-area CRS for metric, latitude-comparable density, andGBFSFeed.geofencing_zones()fetches them live. Unlocks sound spatial-density / equity analysis for free-floating & hybrid systems (the real service area, not a station convex hull). NewGEOFENCING_COLUMNScontract. - Station surroundings / OSM (
osm, extra[osm]):features_within(points, features, radius_m=300, category_col=...): the generic "what's nearby" primitive (counts within a radius + nearest distance + per-categoryn_<cat>breakdown, onGeoKDTree).station_surroundings(info, transit=..., osm=..., radius_m=300): one-shot context frame combining transit feeders and OSM features.enrich_with_osm(reduces any GeoDataFrame geometry to representative points; Bring-Your-Own-GeoDataFrame) and the optional network-boundfetch_osm_around(osmnx). Routing/isochrones stay out of scope. - Multimodal (
multimodal):link_transit_stops(info, gtfs_stops_df, radius_m=200): flags first/last-mile feeder docks near rail/bus by spatial proximity (GeoKDTree; Bring-Your-Own GTFSstops, no transit API, no schedules). Addsnearest_stop_id,nearest_stop_dist_m,n_transit_within,is_transit_feeder. - Station clustering (
cluster, extra[cluster]):cluster_spatial(HDBSCAN/DBSCAN on projected metres),cluster_spectral(geographic-affinity spectral clustering), andcluster_diurnal_profiles(occupancy-profile clustering → typologies). Modern options: automatic k by silhouette (n_clusters="auto"), shape clustering (normalize="zscore"), soft GMM (method="gmm", withcluster_confidence), shape-aware DTW (method="dtw", extra[dtw]/tslearn), and weekday/weekend split. Plusdiurnal_profiles(reusable profile matrix) andlabel_diurnal_typology(human-readable station types: morning_origin / morning_destination / evening_origin / recreational / mostly_empty / mostly_full / stable). - Longitudinal data lake (
timeseries, extra[parquet]):append_to_parquet(Hive-partitioned bysystem_id/date, append-only, concurrent-safe),build_availability_panel(PyArrow partition-pruned read + dedup onstation_id+last_reported+ optional resample),calculate_net_flow(Δ bikes/station +is_rebalancing_suspected, NaN across unchangedlast_reported). GeoKDTree(coregeo): shared great-circle k-NN / radius index (scipy cKDTree over a 3-D unit sphere; EPSG:4326 contract).find_nearest_stationsnow uses it.- Fetch / scrape layer:
GBFSFeed(discover once, then.station_information(),.station_status(),.vehicles(),.availability(),.audit(),.snapshot(),.summary());parse_discovery; one-linersavailability(url)/audit_feed(url);fetch_multiple(system_ids)(threaded, per-system error isolation).get_jsonis dependency-injectable for offline use. - Dynamic audit
audit_dynamic(D1 negative counts, D2 bikes+docks > capacity, D3 staleness): the real-time counterpart to the static A1–A7 audit. - Derived metrics
station_state(empty/full/disabled/normal). - Geo helpers
find_nearest_stations,haversine_m,to_gdf(lazy geopandas,[geo]). - Catalogue
filter_catalog(country_code=, city=, name=): find a system by place.
Changed (canonical model, pre-1.0 schema hardening)¶
- StationInfo gains
is_virtual_station. - StationStatus gains
is_renting,is_returning. - VehicleStatus gains
vehicle_type_id(pedal / e-bike / scooter). last_reportedandfetched_atare now tz-aware UTCdatetime64[ns, UTC](previously unix ints) for unambiguous cross-city merges.
0.1.0 - 2026-06¶
Initial scaffold, consolidates GBFS tooling that was scattered across the lab's research repositories into one tested, installable package.
Added¶
- Canonical data model (
models): version-independentStationInfo,StationStatus,VehicleStatus,AuditVerdictschemas + the A1–A7 rule definitions and thresholds. - Static semantic audit (
audit.static.audit_static): the A1–A7 taxonomy, ported from the publishedgbfs-audit-cataloguepipeline, operating on the canonical frame (no I/O). - Cross-version normalisation (
normalize.to_canonical_station_info): GBFS 2.x string names and 3.x localized-array names; station-type inference. - Catalogue discovery (
catalog.systems_catalog,catalog.resolve): the MobilityData globalsystems.csv. - CLI:
gbfs audit <station_information.json>: the semantic counterpart to MobilityData's syntacticgbfs-validator. - PEP 561
py.typed, CI (py3.10–3.13 + ruff + build), packaging, 14 unit tests.
Notes¶
- Alpha: the public API may change before 1.0. Fetch/archive, the spatial and dynamic audits, and analysis panels are planned for 0.2–0.3.