Catenary Backend is a distributed system comprised of microservices operating in Kubernetes. The system is designed for fault tolerance, high-avaliability, and native execution speed in x86-64 using the Rust systems programming language.
- Maple: GTFS Downloader and ingestion engine into postgres
- Edelweiss: Routing execution engine (in progress, see also our routing testbed
- Avens: OSM Preprocessor and graph generator for routing (in progress)
- Alpenrose: Distributed system to ingest GTFS-rt and other realtime data (Rose des Alpes); successor to Kactus.
- Aspen: Processing of realtime data and dynamic insertion into other engines. Submodule Pando is used for distribution management
- Linnaea: Visualisation of the graphs for debugging and research paper purposes
- Gentian: Transit graph generation runner (in progress)
- Harebell: Map tile geometry generator creating line ordering optimised graph maps (LOOM) MVT files.
- Spruce: Websocket server for frontend to stream data to and from backend, including realtime locations, stop times (not started yet)
- Birch: HTTP API server
- OSM Station Import: Imports railway stations from OpenStreetMap PBF files for GTFS stop association
The kubernetes configuration is generated using Helm templates. See Helm's documentation for further information on that.
The code is heavily commented, go to each folder in src for more information.
This project is built off the work of researchers in Germany and many other places, vielen dank!
- Round-Based Public Transit Routing, Karlsruhe Institute of Technology, Delling, Daniel; Pajor Thomas, Werneck Renato. https://www.microsoft.com/en-us/research/wp-content/uploads/2012/01/raptor_alenex.pdf
- Motis Project https://github.com/motis-project Felix Gündling, innovation in in-memory data stores for graphs
- Hannah Bast, PhD at Freiburg Universitat: Transfer Patterns and Scalable Transfer Patterns
- Patrick Brosi: Pfaedle and Line optimised shapes
- DMFR dataset reader: reads data from transitland-atlas into raw structs https://docs.rs/dmfr-dataset-reader/latest/dmfr_dataset_reader/
- Château: Associates feeds with operators and vise versa using depth first search in knowledge graph
- Pfaedle-rs: Shape matching in rust using low memory.
- Amtrak GTFS rt: Conversion of proprietary realtime data from amtrak's website into gtfs-rt.
- Chicago GTFS Rt: conversion of proprietary Chicago CTA data to GTFS realtime
- Rtc Québec GTFS RT: conversion of proprietary app RTC Nomade to GTFS realtime
- Via Rail GTFS RT: Conversion of Via Rail tracking to GTFS Realtime.
sudo apt install -y postgresql-common
sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
sudo apt install libprotoc-dev protobuf-compiler build-essential gcc pkg-config libssl-dev unzip wget cmake openssl libpq-devsudo apt-get install coinor-cbc coinor-libcbc-devsudo apt install postgresql-18 postgresql-18-postgis-3 postgresql-contrib postgresqlYou may also use an external database if you prefer.
CREATE EXTENSION postgis;Good commit messages are required to contribute to this project.
See https://www.postgresql.org/download
PostGIS is also required like
sudo apt install postgresql-16-postgis-3See https://trac.osgeo.org/postgis/wiki/UsersWikiPostGIS3UbuntuPGSQLApt for more instructions
We've switched to diesel for our queries. Read the diesel documentation to learn how to use it. https://diesel.rs/guides/getting-started.html
Lib PQ is also required to install the diesel cli. Only postgres is required. Example
sudo apt-get install libpq-dev
cargo install diesel_cli --no-default-features --features postgresIs Postgis not installing? This page may be helpful: https://trac.osgeo.org/postgis/wiki/UsersWikiPostGIS3UbuntuPGSQLApt
git submodule update --rebase --remoteCode should be formatted with cargo fmt and be well documented.
The following cargo clippy rules are enforced.
#![deny(
clippy::mutable_key_type,
clippy::map_entry,
clippy::boxed_local,
clippy::let_unit_value,
clippy::redundant_allocation,
clippy::bool_comparison,
clippy::bind_instead_of_map,
clippy::vec_box,
clippy::while_let_loop,
clippy::useless_asref,
clippy::repeat_once,
clippy::deref_addrof,
clippy::suspicious_map,
clippy::arc_with_non_send_sync,
clippy::single_char_pattern,
clippy::for_kv_map,
clippy::let_unit_value,
clippy::let_and_return,
clippy::iter_nth,
clippy::iter_cloned_collect,
clippy::bytes_nth,
clippy::deprecated_clippy_cfg_attr,
clippy::match_result_ok,
clippy::cmp_owned,
clippy::cmp_null,
clippy::op_ref,
clippy::useless_vec,
clippy::module_inception
)]- Truffle reachability analysis
- Chinese plum (needs to pick better syonym for it)
The OSM Station Import system associates GTFS railway stops with OpenStreetMap stations. This provides additional metadata like multilingual names, UIC references, and station relationships.
First, obtain a pre-filtered PBF file containing railway stations (or use one from Geofabrik filtered with osmium):
cargo run --bin osmstationimport -- --file /path/to/railstations.osm.pbfmultiple imports can be done sequentially, and will overwrite old data.
cargo run --bin osmstationimport -- --file railstations-europe-latest.osm.pbf
cargo run --bin osmstationimport -- --file railstations-north-america-latest.osm.pbf
cargo run --bin osmstationimport -- --file railstations-asia-latest.osm.pbfThe importer:
- Computes SHA256 hash to skip duplicate imports
- Extracts rail, tram, and subway stations
- Parses multilingual names (
name:en,name:de, etc.) - Stores in
gtfs.osm_stationstable with spatial indexing
When Maple processes GTFS feeds, it automatically matches stops to OSM stations for rail/tram/subway routes:
cargo run --bin maple -- --transitland /path/to/transitland-atlasSee the Maple readme for more info
The routing engine provides point-to-point transit directions using three main microservices:
- Avens: OSM preprocessor to build walking/biking graphs.
- Gentian: Timetable compiler to create a zero-copy transit timetable.
- Edelweiss: The routing execution engine.
Avens takes OpenStreetMap PBF files and compiles them into a serialized routing graph format with an R-Tree spatial index.
To compile a batch of regions using a configuration file (recommended for managing multiple regions):
cargo run --bin avens --release -- batch --config avens_regions.tomlExample avens_regions.toml:
[global]
tmp_dir = "/tmp/avens_raw"
output_dir = "/path/to/graphs"
[[regions]]
name = "germany"
url = "https://download.geofabrik.de/europe/germany-latest.osm.pbf"
[[regions]]
name = "switzerland"
url = "https://download.geofabrik.de/europe/switzerland-latest.osm.pbf"Procedure for Managing and Updating OSM PBFs:
To manage your region endpoints, simply update the [[regions]] list in your TOML config file. Re-running the batch command will fetch the latest full PBFs from the listed URLs (overwriting old temp files) and regenerate the routing graphs in the output_dir seamlessly.
Alternatively, you can pull directly from a URL and compile a single region:
cargo run --bin avens --release -- pull --url https://download.geofabrik.de/europe/germany-latest.osm.pbf --output-dir /path/to/graphs/germanyTo extract a locally downloaded PBF file:
cargo run --bin avens --release -- extract --input /path/to/germany-latest.osm.pbf --output-dir /path/to/graphs/germanyYou can view statistics about a compiled graph directory using:
cargo run --bin avens --release -- info --graph-dir /path/to/graphs/germanyGentian builds a RAPTOR-compatible zero-copy transit routing timetable from data imported into PostgreSQL (e.g., via Maple). It integrates with the Avens graphs to accurately compute transfer footpaths.
cargo run --bin gentian --release -- --output /path/to/timetable.bin --avens-graphs /path/to/graphsEdelweiss provides the actual routing service via a Tarpc RPC server (listening on 0.0.0.0:9090). It uses the Avens graphs for OSM routing and dynamically loads timetables compiled by Gentian.
GRAPH_DIR=/path/to/graphs cargo run --bin edelweiss --release