Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
78e175e
#51 script to generate pre-processed data for GTFShift web dashboard …
gmatosferreira Jan 22, 2026
4b07d87
#51 #25 fix on osm_bus_lanes query when psv or bus osm tags do not exist
gmatosferreira Jan 28, 2026
10e15be
#51 #23 osm_shapes_to_routes improved feedback on routes not matches …
gmatosferreira Jan 28, 2026
5889298
#51 #33 get_way_frequency_hourly improved to return routes info
gmatosferreira Jan 28, 2026
bc71e26
Merge branch 'refs/heads/main' into 51-create-ui
gmatosferreira Feb 2, 2026
c783bf1
#51 web_version script improved to include rt-data
gmatosferreira Feb 2, 2026
1d5fe14
#51 #45 rt_collect methods renamed for better consistency
gmatosferreira Feb 3, 2026
2b098d7
#51 docs typo fix
gmatosferreira Feb 3, 2026
98cc905
#51 README rt simplified and key functions extended with latest devel…
gmatosferreira Feb 3, 2026
8e59693
#51 #38 article prioritize updated to refer network continuity + prio…
gmatosferreira Feb 3, 2026
80fa8cc
#51 #45 article rt map visualization improved
gmatosferreira Feb 3, 2026
de129a9
#51 #38 prioritize_lanes n_lanes_direction calculations fix to avoid …
gmatosferreira Feb 3, 2026
78cec3f
#51 dev/web_version script extended to produce metadata for dashboard…
gmatosferreira Feb 3, 2026
a34001e
#51 #42 fixing conceptual assumption that a trip's first stop has sto…
gmatosferreira Feb 3, 2026
00e7752
#51 dev/web_version metadata improved
gmatosferreira Feb 3, 2026
5176090
#51 README and get started reviewed and extended with web dashboard i…
gmatosferreira Feb 4, 2026
f776b88
pkgdown updated with GTFShift image
gmatosferreira Feb 4, 2026
afbc382
pkgdown navbar extended with dashboard entry
gmatosferreira Feb 4, 2026
9b8328b
small fixed after copilot revision
gmatosferreira Feb 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ export(osm_shapes_to_routes)
export(osm_trips_to_routes)
export(prioritize_lanes)
export(query_mobilitydatabase)
Copy link

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you add a deprecated rt_collect() wrapper for backwards compatibility, it also needs to remain exported here (currently only rt_collect_json is exported). Otherwise, existing code calling GTFShift::rt_collect() will fail after updating.

Suggested change
export(query_mobilitydatabase)
export(query_mobilitydatabase)
export(rt_collect)

Copilot uses AI. Check for mistakes.
export(rt_collect)
export(rt_collect_json)
export(rt_collect_protobuf)
export(rt_extend_prioritization)
export(unify)
Expand Down
11 changes: 5 additions & 6 deletions R/get_route_frequency_hourly.R
Original file line number Diff line number Diff line change
Expand Up @@ -92,13 +92,12 @@ get_route_frequency_hourly = function(
"stop_sequence"
)))

stop_times = stop_times |> # Only departures from origin (first stop)
filter(stop_sequence == 1)

stop_times = stop_times |>
mutate(
hour = lubridate::hour(departure_time)
)
arrange(stop_sequence) |>
group_by(trip_id) |>
slice(1) |> # Only departures from origin (first stop)
ungroup() |>
mutate(hour = lubridate::hour(departure_time))

freq_data = stop_times |>
group_by(across(any_of(c("route_id", "route_short_name", "direction_id", "hour")))) |>
Expand Down
17 changes: 10 additions & 7 deletions R/get_way_frequency_hourly.R
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
#' \item \code{way_osm_id}, the \code{osm_id} attribute from OSM way.
#' \item \code{hour}, the hour for which the frequency applies (24 hour format).
#' \item \code{frequency}, the number of services for the route that depart from the first stop for the corresponding 60 minutes period.
#' \item \code{routes}, the list of route_ids that use the way, separated by semicolon.
#' \item \code{geometry}, the route shape.
#' \item (if \code{keep_osm_attributes = TRUE}) all OSM way attributes.
#' }
Expand Down Expand Up @@ -75,13 +76,12 @@ get_way_frequency_hourly = function(
"stop_sequence"
)))

stop_times = stop_times |> # Only departures from origin (first stop)
filter(stop_sequence == 1)

stop_times = stop_times |>
mutate(
hour = lubridate::hour(departure_time)
)
arrange(stop_sequence) |>
group_by(trip_id) |>
slice(1) |> # Only departures from origin (first stop)
ungroup() |>
mutate(hour = lubridate::hour(departure_time))

freq_data = stop_times |>
group_by(across(any_of(c("route_id", "route_short_name", "direction_id", "hour")))) |>
Expand All @@ -107,7 +107,10 @@ get_way_frequency_hourly = function(
ways_freq = routes_freq |>
inner_join(ways |> sf::st_drop_geometry() |> select(shape_id, way_osm_id), by="shape_id", relationship = "many-to-many") |>
group_by(way_osm_id, hour) |>
summarize(frequency = sum(frequency)) |>
summarize(
frequency = sum(frequency),
routes = paste(unique(route_id), collapse = ";")
) |>
ungroup() |>
inner_join(ways_unique_geometry, by="way_osm_id") |>
st_as_sf()
Expand Down
5 changes: 4 additions & 1 deletion R/prioritize_lanes.R
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
#' \item \code{n_lanes}, the total number of lanes.
#' \item \code{n_directions}, the number of travel directions.
#' \item \code{n_lanes_direction}, the number of lanes per direction.
#' \item \code{routes}, the list of route_ids that use the way, separated by semicolon.
#' \item \code{geometry}, the route shape.
#' \item (if \code{keep_osm_attributes = TRUE}) all OSM way attributes.
#' }
Expand Down Expand Up @@ -84,19 +85,21 @@ prioritize_lanes <- function(
2 # NA_integer_
),
n_directions = case_when(
n_lanes == 1 ~ 1, # When only one lane, assume one direction
oneway %in% c("yes", "1", "-1", "true") ~ 1,
oneway %in% c("no", "0", "false") ~ 2,
TRUE ~ 2
),
n_lanes_direction = case_when(
n_lanes / n_directions < 1 ~ 1,
!is.na(n_lanes) & !is.na(n_directions) ~ n_lanes / n_directions,
TRUE ~ NA_real_
)
)

if (!keep_osm_attributes) {
lanes = lanes |>
select(way_osm_id, hour, frequency, is_bus_lane, n_lanes, n_directions, n_lanes_direction, geometry)
select(way_osm_id, hour, frequency, is_bus_lane, n_lanes, n_directions, n_lanes_direction, routes, geometry)
}

return(lanes)
Expand Down
4 changes: 2 additions & 2 deletions R/query_osm_bus_lanes.R
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ osm_bus_lanes <- function(bbox) {
# Based on https://wiki.openstreetmap.org/wiki/Bus_lanes
psv == "designated"
| highway == "busway"
| if_any(all_of(cols_to_check_access), ~ grepl("designated", .x))
| if_any(all_of(cols_to_check_count), ~ is.numeric(.x) & .x >= 1)
| (length(cols_to_check_access) & if_any(all_of(cols_to_check_access), ~ grepl("designated", .x)))
| (length(cols_to_check_count) & if_any(all_of(cols_to_check_count), ~ is.numeric(.x) & .x >= 1))
)

return(osm_lanes)
Expand Down
2 changes: 1 addition & 1 deletion R/query_osm_shapes_match_routes.R
Original file line number Diff line number Diff line change
Expand Up @@ -314,7 +314,7 @@ osm_shapes_match_routes <- function(gtfs, q, geometry = TRUE, gtfs_match = "rout
route_dist = st_length(geometry) |> units::drop_units(),
trip_id_copy = trip_id,

first_stop_id = gtfs$stop_times |> filter(trip_id == trip_id_copy & stop_sequence == 1) |> slice(1) |> pull(stop_id),
first_stop_id = gtfs$stop_times |> filter(trip_id == trip_id_copy) |> arrange(stop_sequence) |> slice(1) |> pull(stop_id),
last_stop_id = gtfs$stop_times |> filter(trip_id == trip_id_copy) |> arrange(desc(stop_sequence)) |> slice(1) |> pull(stop_id),
initial = stops_sf |> filter(stop_id == first_stop_id) |> slice(1) |> pull(geometry),
final = stops_sf |> filter(stop_id == last_stop_id) |> slice(1) |> pull(geometry),
Expand Down
24 changes: 18 additions & 6 deletions R/query_osm_shapes_to_routes.R
Original file line number Diff line number Diff line change
Expand Up @@ -137,12 +137,24 @@ osm_shapes_to_routes <- function(gtfs, q, ways = FALSE, ways_tags = c("lanes", "
pb$terminate()
}

# 4. Log missing shapes
message(sprintf("Matched %d shapes with OSM routes!", length(unique(result$shape_id))))
shapes_missing = shape_ids |> filter(!(shape_id %in% result$shape_id)) |> left_join(gtfs$trips, by="shape_id") |> left_join(gtfs$routes, by="route_id") |> distinct(shape_id, .keep_all = TRUE)
if (nrow(shapes_missing)>0) {
row_strings <- with(shapes_missing, sprintf("%s (%s)", shape_id, route_short_name))
warning(sprintf("Shapes missing (ignored in the result): %s", paste(row_strings, collapse = " ")))
# 4. Log missing shapes/routes
routes_shapes = gtfs$routes |> select(route_id, route_short_name, route_long_name) |>
right_join(gtfs$trips |> select(trip_id, route_id, shape_id), by="route_id") |>
distinct(route_id, shape_id, .keep_all = TRUE)

shapes_matched_n = result |> distinct(shape_id) |> nrow()
shapes_gtfs_n = gtfs$shapes |> distinct(shape_id) |> nrow()
routes_matched_n = routes_shapes |> filter(shape_id %in% result$shape_id) |> distinct(route_id) |> nrow()
routes_gtfs_n = gtfs$routes |> distinct(route_id) |> nrow()

message(sprintf("Matched %d shapes (%.2f%% of %d in GTFS) of %d routes (%.2f%% of %d in GTFS) with OSM routes!",
shapes_matched_n, shapes_matched_n/shapes_gtfs_n*100, shapes_gtfs_n,
routes_matched_n, routes_matched_n/routes_gtfs_n*100, routes_gtfs_n
))
routes_shapes_missing = routes_shapes |> filter(!(shape_id %in% result$shape_id))
if (nrow(routes_shapes_missing)>0) {
row_strings <- with(routes_shapes_missing, sprintf("| %s | %s | %s | %s |", route_id, shape_id, route_short_name, route_long_name))
warning(sprintf("Shapes missing (ignored in the result):\n| route_id | shape_id | route_short_name | route_long_name |\n%s", paste(row_strings, collapse = "\n")))
}

return(result)
Expand Down
8 changes: 4 additions & 4 deletions R/rt_collect.R → R/rt_collect_json.R
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#' Collect GTFS-RT data
#' Collect GTFS-RT data from a JSON feed at regular intervals
#'
#'
#' @param gtfs_rt_url String. URL of the GTFS-RT feed in JSON format.
Expand All @@ -12,19 +12,19 @@
#' @details
#' Downloads GTFS-RT data from the specified URL at regular intervals and saves them to the destination file.
#'
#' This function will run indefinitely until manually stopped.
#' This function will run indefinitely until manually stopped (CTRL + C).
#'
#'
#' @examples
#' \dontrun{
#' GTFShift::rt_collect("https://api.example.com/gtfs-rt", "gtfs_rt_data.csv")
#' GTFShift::rt_collect_json("https://api.example.com/gtfs-rt", "gtfs_rt_data.csv")
#' }
#'
#' @import jsonlite
#' @import progress
#'
#' @export
rt_collect <- function(
rt_collect_json <- function(
gtfs_rt_url, destination_file,
header_key="header", # Optional
entity_key="entity",
Expand Down
6 changes: 3 additions & 3 deletions R/rt_collect_protobuf.R
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#' Collect GTFS-RT data (with Protocol Buffers support)
#' Collect GTFS-RT data from a Protocol Buffers feed at regular intervals
#'
#'
#' @param gtfs_rt_url String. URL of the Protocol Buffers GTFS-RT feed.
Expand All @@ -10,7 +10,7 @@
#' @details
#' Downloads GTFS-RT data from the specified URL at regular intervals and saves them to the destination file.
#'
#' This function will run indefinitely until manually stopped. Each downloaded file is named with a timestamp to ensure uniqueness.
#' This function will run indefinitely until manually stopped (CTRL + C).
#'
#'
#' @examples
Expand Down Expand Up @@ -78,7 +78,7 @@ rt_collect_protobuf <- function(
)

suppressMessages({
rt_collect(
rt_collect_json(
gtfs_rt_url = temp_json,
destination_file = destination_file,
fields_collect = fields_collect,
Expand Down
6 changes: 3 additions & 3 deletions R/rt_extend_prioritization.R
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
#' @details
#' Extends the \code{lane_prioritization} data with speed metrics calculated from the GTFS-RT data points that fall within a buffer around each lane segment.
#'
#' Refer to \code{GTFShift::rt_collect()} for details on GTFS-RT data collection.
#' Refer to \code{GTFShift::rt_collect_json()} or \code{GTFShift::rt_collect_protobuf()} for details on GTFS-RT data collection.
#'
#'
#' @returns The \code{lane_prioritization} \code{sf} \code{data.frame}, extended with the following columns:
Expand All @@ -24,7 +24,7 @@
#' @examples
#' \dontrun{
#' rt_collect_file <- "gtfs_rt_data.csv"
#' GTFShift::rt_collect("https://api.example.com/gtfs-rt", rt_collect_file)
#' GTFShift::rt_collect_json("https://api.example.com/gtfs-rt", rt_collect_file)
#' lane_prioritization <- GTFShift::prioritize_lanes(gtfs, osm_query)
#'
#' rt_collection <- read.csv(rt_collect_file) |> sf::st_as_sf(coords = c("longitude", "latitude"), crs = 4326)
Expand All @@ -45,7 +45,7 @@ rt_extend_prioritization <- function(
lane_buffer = 15 # in meters
) {
# 1. Validate inputs
required_cols = c("way_osm_id", "geometry")
required_cols = c("way_osm_id")
missing_cols = setdiff(required_cols, colnames(lane_prioritization))
if (length(missing_cols) > 0) {
stop(paste("lane_prioritization is missing required columns:", paste(missing_cols, collapse = ", ")))
Expand Down
110 changes: 15 additions & 95 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# GTFShift
# GTFShift <img align="right" src="man/figures/logo.png" alt="logo" width="180">

<!-- badges: start -->
[![](https://github.com/U-Shift/GTFShift/actions/workflows/pkgdown.yaml/badge.svg)](https://github.com/U-Shift/GTFShift/actions/workflows/pkgdown.yaml)
Expand All @@ -8,48 +8,22 @@
overview of where bus lanes should be prioritized for a given territory,
using General Transit Feed Specification (GTFS) and OpenStreetMap (OSM) data.

It provides a simple bundle for an aggregated analysis, that with one execution
compiles in a few seconds the following indicators:
It provides a comprehensive bundle of methods that cover several dimensions of this
problem, namely:

- Frequency of buses (and trams) per hour and direction, at a peak hour;
- Number of lanes in the same direction.
- Number of lanes in the same direction;
- Existing traffic conditions;
- Existing bus lanes in the area (from a network continuity perspective).

Together, these can be used to identify road segments where bus lanes should be implemented,
enabling for a transparent and data-driven decision-making process, suitable to different contexts
and criteria.

```r
library(GTFShift)
library(osmdata)

data = read.csv(system.file("extdata", "gtfs_sources_pt.csv", package = "GTFShift"))
gtfs_id = "lisboa"
gtfs = GTFShift::load_feed(data$URL[data$ID == gtfs_id], create_transfers=FALSE)
osm_q = opq(bbox=sf::st_bbox(tidytransit::shapes_as_sf(gtfs$shapes))) |>
add_osm_feature(key = "route", value = c("bus", "tram")) |>
add_osm_feature(key = "network", value = "Carris", key_exact = TRUE)

lanes = prioritize_lanes(gtfs, osm_q)

mapview::mapview(
lanes |> filter((frequency<=4 | (is.na(n_lanes) | n_lanes_direction<=1)) & is_bus_lane),
layer.name="Bus lane with - 5 bus/h OR - 1 lane/dir",
color="#DAD887"
) + mapview::mapview(
lanes |> filter(frequency>=5 & !is.na(n_lanes) & n_lanes_direction>1 & is_bus_lane),
layer.name="Bus lane with 5 or + bus/h + 1 lane/dir",
color="#3BC1A8"
) + mapview::mapview(
lanes |> filter(frequency>=5 & !is.na(n_lanes) & n_lanes_direction>1 & !is_bus_lane),
layer.name="NO bus lane with 5 or + bus/h + 1 lane/dir",
color="#F63049"
)
```

![](man/figures/prioritization.png)

> Example of bus lane prioritization analysis for Lisbon city, considering road segments with
a minimum frequency of 5 buses/hour and more than 1 lane per direction.
a minimum frequency of 10 buses/hour, average commercial speed below 9.7 km/h and more than 1 lane per direction.

## Installation

Expand All @@ -67,72 +41,18 @@ remotes::install_github("U-Shift/GTFShift")
library(GTFShift)
```

## Key functions

**GTFShift** provides methods for the entire workflow of bus network
density analysis. For detailed examples on their functionality, refer to
the articles at <https://u-shift.github.io/GTFShift/>.

### Getting transit data

Starting with a valid GTFS feed is the key for a successful analysis.
**GTFShift** includes a method to load feeds that simultaneously scans
for any integrity errors and fixes them automatically.

If the feed location is unknown, it also provides a database listing
GTFS for Portugal and a method to query worldwide open catalogues by
city or country names or even a bounding box.

### Filter

GTFS feeds do not have a defined scope regarding its coverage of the
transportation system. Some can be bounded to one agency, whereas others
can aggregate several modes in the same city, or even national wise.

From the simpler to the most complex feeds, some analysis require to
narrow the perspective. **GTFShift** provides some to help in this
process.

### Aggregate

Public transit analysis takes advantage of the standardized GTFS format.
However, its provision by operator makes it difficult for network
aggregated analysis, considering connectivity and multimodality.

**GTFShift** includes a method to easily generate an aggregated GTFS
file given several instances.

![](man/figures/unify.png)

> Aggregated GTFS for Fertagus and Transportes Coletivos do Barreiro
> operators

### Analyse

Analyzing public transit feeds is important to understand its
territorial coverage and dynamics, both on its spatial and temporal
dimensions.

**GTFShift** provides several methods that encapsulate pre-defined
methodologies for them, for instance, analysing hourly frequency per
stop, route or road segment.

![](man/figures/analyse_aggregated_frequencies.png)

> Aggregated route frequency for Carris Lisboa operator, at 8:00
## Get started

### OSM Data
For more details on the package and how to get started, please visit the [Get started](https://u-shift.github.io/GTFShift/articles/GTFShift.html) page.

OpenStreetMaps (OSM) is an important data source for transit analysis,
due to its rich, open, and detailed geographic data.
## Dashboard

**GTFShift** includes some methods that allow to access its information
directly, namely to export bus lanes, get centerlines for the road
network and export the OSM transit routes.
**GTFShift** provides an interactive dashboard that allows users to explore and visualize results for real case studies,
aiming to illustrate its potential and capabilities to a non-technical audience, while disseminating the outputs of these real world scenarios.

![](man/figures/osm_buslanes.png)
Visit it at [ushift.pt/apps/gtfshift](https://ushift.pt/apps/gtfshift).

> OSM exported bus lanes for Lisbon
[![](man/figures/web.png)](https://ushift.pt/apps/gtfshift)

## Related packages

Expand All @@ -144,7 +64,7 @@ network and export the OSM transit routes.
## Acknowledgement

**GTFShift** is developed and maintained by
[U-shift](https://ushift.tecnico.ulisboa.pt) urban mobility research
[U-Shift](https://ushift.tecnico.ulisboa.pt) urban mobility research
group, part of [CERIS](https://ceris.pt/) research unit, at [Instituto
Superior Técnico](https://tecnico.ulisboa.pt/pt/), Lisbon, Portugal.

Expand Down
14 changes: 14 additions & 0 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,20 @@
url: https://u-shift.github.io/GTFShift/
template:
bootstrap: 5
light-switch: true
bslib:
primary: "#15AF97FF"

navbar:
structure:
left: [intro, reference, articles, dashboard]

components:
dashboard:
text: "Dashboard"
href: "https://ushift.pt/apps/gtfshift"
target: "_blank"


reference:
- title: Get GTFS feeds
Expand Down
Loading