jpo-cvmanager

US Department of Transportation (USDOT) Intelligent Transportation Systems (ITS) Joint Program Office (JPO) Connected Vehicle Manager

The JPO Connected Vehicle Manager is a web-based application that helps an organization manage their deployed CV devices (Roadside Units and Onboard Units) through an interactive, graphical user interface using Mapbox.

Features:

• Visualize devices on a Mapbox map
• Display the current statuses of devices
  • Latest online status
  • ISS SCMS certificate expiration
  • Other identifying values tracked on a PostgreSQL database
• jpo-ode supported message counts, sorted by RSU IP (BSM, MAP, SPaT, SRM, SSM, TIM)
• Visualize an RSU's currently active MAP message
• Visualize Basic Safety Messages (BSMs) relative to a specified geofence and time period
• Device configuration over SNMP (v3) for message forwarding including an option to include security headers for NTCIP-1218 devices
• Device firmware upgrade support for Kapsch, Commsignia and Yunex devices
• Admin controls for adding, modifying and removing devices and users
• Fully customizable theming with Material UI - cvmanager theming documentation
• Organizations allow fine tuned access control of users and devices. Multiple organizations can be run concurrently within 1 CV-Manager deployment.
• Role-based access supports three user roles: Admin, Operator, and User. See the chart below for a breakdown of role-based access within and between organizations

To provide feedback, we recommend that you create an "issue" in this repository (https://github.com/usdot-jpo-ode/jpo-cvmanager/issues). You will need a GitHub account to create an issue. If you don’t have an account, a dialog will be presented to you to create one at no cost.

Quick Start

This section is brief - for more detailed instructions, please see Getting Started

Requirements

• Docker and Docker Compose installed on your machine

Steps

1. Copy the sample.env file to a new file named .env in the root directory of the project.
2. Edit the .env file to set the required environment variables. At a minimum, you will need to set the following variables:
   • DOCKER_HOST_IP: The IP address of your Docker host. This can be found through linux/wsl through the command "ifconfig", or "localhost" if using Docker Desktop on Windows or Linux (not mac).
   • MAPBOX_TOKEN: Any valid mapbox token. Please see Creating a Mapbox Token for instructions on how to create and account/generate a new token
   • MAVEN_GITHUB_TOKEN: A GitHub access token used to access public GitHub Maven packages. See Github Token section for instructions on generating this token.
3. Initialize the jpo-utils submodule:

git submodule update --init --recursive

4. Run the following command to start the CV Manager:

docker compose up -d

5. Access the CV Manager webapp at http://localhost:3000 in your web browser.

Default Username: test@gmail.com
Default Password: tester

If you have any issues, try the following steps:

1. Ensure that the following required services are healthy in Docker (docker ps): i. cvmanager_postgres ii. cvmanager_keycloak iii. cvmanager_api iv. cvmanager_webapp
2. Bring down the system and re-build all containers

docker compose down -v
docker compose up --build -d

For more details on running the CV-Manager through Docker, see the Getting Started section below.

Release Notes

The current version and release history of the JPO CV Manager: Release Notes

Requirements and Limitations

The JPO CV Manager was originally developed for the Google Cloud Platform and a few of its GCP dependencies still remain. The GCP dependencies will eventually be streamlined to support other options. However, there are a handful of technologies to understand before attempting to utilize the CV Manager.

CV Manager Webapp

ReactJS with Redux Toolkit and Mapbox GL

• Supports OAuth2.0 through Keycloak for user authentication only. It can be configured for several different Identity Providers, including Google.

Creating a Mapbox Token

To generate a free mapbox access token:

1. navigate to https://www.mapbox.com/ and select "Get started for free"
2. Enter your name and email, and choose a unique username
3. Enter your payment information. Your account will remain free unless you surpass the free tier limits (the free tier is very extensive, it will cover everything up to a multi-user large scale deployment)
4. Confirm your email address
5. Enter your billing address
6. After logging in, select the "Tokens" tab under the Admin section
7. Press create a new token (it is easier to manage and re-create new tokens than the default public token) i. Enter a recognizable token name ii. No scopes are required, as this token will only be used for tile loading iii. Under Token Restrictions, enter the URL paths the CV-Manager will be hosted on. For local development, this is http://localhost:3000 and http://${DOCKER_HOST_IP}:3000. This is incredibly important. When the CV-Manager is deployed, the mapbox token can be extracted quite easily. The only way to protect the use of this token (and not incur additional access costs) is to restrict the allowed domains iv. Select "Create Token"
8. Copy the token value and paste it into the .env under MAPBOX_TOKEN=

CV Manager API

Python 3.12.2

• PostgreSQL database is required. Run the table creation script to create a to-spec database.
  • Follow along with the README to ensure your data is properly populated before running the CV Manager.
• GCP BigQuery is required to support J2735 message counts and BSM data. Message counts will be migrated to PostgreSQL eventually, however it is not recommended to store full J2735 messages in a PostgreSQL database. A noSQL database or a database that is specialized for storing big data is recommended. Support for MongoDB is planned to be implemented.
  • It is recommended to create a table for storing J2735 messages, one table per message type (BSM, MAP, SPaT, SRM, and SSM), before running the CV Manager.

Keycloak

• Keycloak is used for the CV Manager webapp's authentication.
• The Keycloak pod requires a realm.json file in the folder: ./resources/keycloak/ to startup with the proper configurations. It also requires a login theme that can be modified and generated using the keycloakify forked repository in resources/keycloak/keycloakify. The theme will be automatically generated when using the docker image provided but can also be built using instructions found in the keycloakify folder.
• To migrate an environment from a disconnected keycloak/postgres deployment to using the Keycloak postgres user provider, follow the steps described in resources/keycloak/README.md

Intersection Data + ConflictMonitor Integration

The CVManager now has the ability to manage, configure, and display data from connected intersections. Using the JPO-ODE ConflictMonitor and other JPO-ODE resources, intersection-specific data can be collected, processed, and analyzed. The CVManager has the ability to display the results of this analysis, show live message data, and configure intersection monitoring. This includes the following:

• Displaying live MAPs, SPATs, and BSMs on a Mapbox map
• Displaying archived MAPs, SPATs, and BSMs on a Mapbox map
• Querying, downloading, and displaying events created by the ConflictMonitor
• Querying, downloading, and displaying assessments of events created by the ConflictMonitor
• Querying, managing, and displaying notifications created by the ConflictMonitor
• Updating and managing configuration parameters controlling message analysis, assessments, and notifications

More information on the ConflictMonitor and other services described above can be found here:

• jpo-conflictmonitor
• jpo-geojsonconverter
• jpo-ode
• jpo-conflictvisualizer

Ongoing Efforts This feature is under active development. This is a joint effort involving combining the features of the existing CIMMS conflictvisualizer tools with the CVManager components, to enable connected vehicle and intersection analysis in one application.

One of the major features which is under active development is the combined CVManager - ConflictVisualizer API. This can currently be found under the services/intersection-api folder.

Github Token

A GitHub token is required to pull artifacts from GitHub repositories. This is required to obtain the jpo-ode jars and must be done before attempting to build this repository.

1. Log into GitHub.
2. Navigate to Settings -> Developer settings -> Personal access tokens.
3. Click "New personal access token (classic)".
   1. As of now, GitHub does not support Fine-grained tokens for obtaining packages.
4. Provide the name "jpo_conflictmonitor"
5. Set an expiration date
6. Select the read:packages scope.
7. Click "Generate token" and copy the token.
8. Set this token as the MAVEN_GITHUB_TOKEN environment variable in the .env file (root and ./services/intersection-api/.env)

Feature Flags

This application has the ability to disable certain features based on environment variables. For each of these variables, the feature will be enabled if the variable is anything but 'false'. These features include:

• ENABLE_RSU_FEATURES: if 'false', disable all RSU-specific features, including map, RSU data, RSU configuration, and RSU organization linking.
• ENABLE_INTERSECTION_FEATURES: if 'false', disable all intersection-specific features, including intersection map, intersection dashboard, and intersection admin pages.
• ENABLE_WZDX_FEATURES: if 'false', disable all wzdx-specific features, including WZDx data on the main map.

These variables will disable UI pages/visuals, UI features, and API endpoints.

Local Development

Ease of local development has been a major consideration in the integration of intersection data into the CVManager application. Through the use of public docker images and sample datasets, this process is relatively simple. The services required to show intersection data on the CVManager webapp are:

• kafka
  • Base kafka image used to supply required topics to the intersection api
• kafka-setup
  • Kafka topic creation image, to create required topics for the intersection api
• mongo
  • Base MongoDB image, with sample data, used to supply data to the intersection api
• mongo-setup
  • MongoDB collection creation image, to create required collections for the intersection api

It should be noted that the kafka, kafka-setup, mongo and mongo-setup services are provided by the jpo-utils repository.

Running a Simple Local Environment

Build the docker-compose:

docker compose up -d

If any issues occur, try re-building all images, with:

docker compose up --build -d

Re-generating ConflictMonitor Sample Data

A set of data dumps exists in the ./resources/mongodumps folder, which can each be automatically injected into MongoDB using the MONGO_SAMPLE_DATA_RELATIVE_PATH environment variable, such as MONGO_SAMPLE_DATA_RELATIVE_PATH=../resources/mongodumps/dump_2025_07_21.

To create a new sample dataset, simply follow the below steps, and see the sample commands below the steps.

1. Configure MongoDB to not restore any data on boot by leaving the env var MONGO_SAMPLE_DATA_RELATIVE_PATH blank
2. Clear all existing docker volumes

docker compose down -v

3. Bring up the cv-manager including the conflictmonitor components with the env var COMPOSE_PROFILES=basic,webapp,intersection,conflictmonitor,mongo_full,kafka_full,kafka_connect_standalone

docker compose up -d

4. Clone the ConflictMonitor repository and run the test-message-sender to generate sample data
   1. If data is not being sync'd to mongodb, ensure that you have set your env var CONNECT_URL=http://${DOCKER_HOST_IP}:8083

git clone https://github.com/usdot-jpo-ode/jpo-conflictmonitor
cd jpo-conflictmonitor/test-message-sender/script-runner
mvn clean install
cd ../
java -jar ./script-runner/target/script-runner-cli.jar ../jpo-conflictmonitor/scripts/IntegrationTestScripts/ConnectionOfTravel-u-turn.csv

5. Export the newly generated data to create a new dump of the MongoDB database

cd ../../jpo-cvmanager
# Update the username and password to match your MongoDB instance (MONGO_READ_WRITE_USER and MONGO_READ_WRITE_PASSWORD)
docker exec -it jpo-cvmanager-mongo-1 mongodump --db CV --out /dump --username=ode --password=replace_me --authenticationDatabase admin
# This command requires an unix shell - if running in powershell, replace `$(date +%Y_%m_%d)` with `$formattedDate` where `$formattedDate = Get-Date -Format "yyyy_MM_dd"`.
docker cp jpo-cvmanager-mongo-1:/dump ./resources/mongodumps/dump_$(date +%Y_%m_%d)

6. Update your MONGO_SAMPLE_DATA_RELATIVE_PATH in the .env file to point to the new dump folder and restart the docker compose to use the new data dump:
   1. e.x. MONGO_SAMPLE_DATA_RELATIVE_PATH=../resources/mongodumps/dump_2025_07_21

docker compose up -d

MongoDB

MongoDB is the backing database of the intersection api. This database holds configuration parameters, archived data (SPATs, MAPs, BSMs, ...), and processed data (notifications, assessments, events). For local development, a mongodump has been created in the conflictmonitor/mongo/dump_2024_08_20 directory. This includes notifications, assessments, events, as well as SPATs, MAPs, and BSMs. All of this data is available through the intersection api. To disable starting the mongo container with sample data, set the INSERT_SAMPLE_DATA environment variable to false in the .env file.

Kafka

Kafka is used by the intersection api to receive data from the ODE, GeoJSONConverter, and ConflictMonitor. These connections enable live intersection data to be streamed from the ConflictMonitor into the cvmanager intersection map.

Generating Sample Data

Some simple sample data is injected into the MongoDB instance when created. If more data is useful, the test-message-sender from the jpo-conflictmonitor can also be used to generate live sample data. This component should be cloned/installed separately, and is described here: jpo-conflictmonitor/test-message-sender

Getting Started

The following steps are intended to help get a new user up and running the JPO CV Manager in their own environment.

1. Follow the Requirements and Limitations section and make sure all requirements are met.

2. Run the following command to initialize submodules:

   git submodule update --init --recursive

3. Create a copy of the sample.env named ".env" and refer to the Environmental variables section below for more information on each variable.

   1. Make sure at least the DOCKER_HOST_IP, MAVEN_GITHUB_TOKEN, and MAPBOX_TOKEN are set for this.
   2. For other services or different configuration, please make a copy of the sample-full.env. Some of these variables, delineated by sections, pertain to the jpo-conflictmonitor, jpo-geojsonconverter, and jpo-ode. Please see the documentation provided for these projects when setting these variables.

4. The CV Manager has four core components that need to be built and run: the API, the PostgreSQL database, Keycloak, and the webapp. Ensure that both of the following profiles are specified in the COMPOSE_PROFILES variable of your .env file:

   • basic: brings up the API, PostgreSQL, and Keycloak
   • webapp: brings up the CV-Manager webapp component
   • intersection: Optional, brings up the Intersection API and enables visualization/management of connected intersections

5. Use the docker compose to start the required components:

   docker compose up -d

   If any issues occur, try:

   docker compose up --build -d

6. Access the website by going to http://localhost:3000

     Default Username: test@gmail.com
     Default Password: tester
   

7. To access keycloak go to http://localhost:8084/

     Default Username: admin
     Default Password: admin
   

   This should automatically redirect you to http://${DOCKER_HOST_IP}:8084/. If it does not, navigate to that URL directly.

• If you are looking to deploy in Kubernetes or on separate VMs, refer to the Kubernetes YAML deployment files to deploy the four components to your cluster. (Kubernetes YAML)

Docker Profiles

Docker compose profiles allow for the customization of services that are run. For more information on how this works, see the Docker Compose Profiles Documentation. Services and profiles are configured using the COMPOSE_PROFILES environment variable. Multiple profiles may be specified, like COMPOSE_PROFILES=basic,webapp,intersection

In addition to the groups defined in the table below, each service may also be activated independently by specifying the service name as a profile. This can be combined with other service names or profile groups to produce unique combinations of services. For example, the entry COMPOSE_PROFILES=kafka,kafka_init,basic would bring up the kafka services and the basic CV-Manager services. To avoid breaking name changes, the conflictmonitor service can be started individually using the "conflictmonitor_only" profile.

Profiles and Services

Service	basic	webapp	intersection	conflictmonitor	addons	obu_ota
cvmanager_api	✅	❌	❌	❌	❌	❌
cvmanager_webapp	❌	✅	❌	❌	❌	❌
cvmanager_postgres	✅	❌	❌	❌	❌	❌
cvmanager_keycloak	✅	❌	❌	❌	❌	❌
intersection_api	❌	❌	✅	❌	❌	❌
conflictmonitor	❌	❌	❌	✅	❌	❌
ode	❌	❌	❌	✅	❌	❌
aem	❌	❌	❌	✅	❌	❌
adm	❌	❌	❌	✅	❌	❌
geojsonconverter	❌	❌	❌	✅	❌	❌
deduplicator	❌	❌	❌	✅	❌	❌
connect	❌	❌	❌	✅	❌	❌
jpo_count_metric	❌	❌	❌	❌	✅	❌
rsu_status_check	❌	❌	❌	❌	✅	❌
jpo_iss_health_check	❌	❌	❌	❌	✅	❌
firmware_manager_upgrade_scheduler	❌	❌	❌	❌	✅	❌
firmware_manager_upgrade_runner	❌	❌	❌	❌	✅	❌
jpo_ota_backend	❌	❌	❌	❌	❌	✅
jpo_ota_nginx	❌	❌	❌	❌	❌	✅

Note on JPO-Utils Profiles

While kafka, kafka-setup, mongo, mongo-setup, and kafka-connect are not included in the table above, they are required for the intersection API to run. These services are provided by the jpo-utils repository. To enable these services, you must include the kafka_full, mongo_full, and kafka_connect_standalone profiles.

Debugging

Note that it is recommended to work with the Python API from a virtual environment.

Setting up a virtual environment from the command line

1. Verify that you have Python 3.12.2 installed on your machine by running the following command:

   python3.12 --version

   python --version

   If you have a different version installed, download and install Python 3.12.2 from the Python website.
2. Open a terminal and navigate to the root of the project.
3. Run the following command to create a virtual environment in the project root:

   python3.12 -m venv .venv

   python -m venv .venv

4. Activate the virtual environment:

   source .venv/bin/activate

   .venv\Scripts\activate

5. Install the required packages:

   pip3.12 install -r services/requirements.txt

   pip install -r services/requirements.txt

Environment Variables

Required Variables

• DOCKER_HOST_IP: Set with the IP address of the eth0 port in your WSL instance. This can be found by installing networking tools in wsl and running the command ifconfig
• MAPBOX_TOKEN: A token from Mapbox used to render the map in the Webapp. The free version of Mapbox works great in most cases.
• MAVEN_GITHUB_TOKEN: A GitHub access token used to access public GitHub Maven packages. See Github Token section for instructions on generating this token.

Generic Variables

• WEBAPP_HOST_IP: Defaults to DOCKER_HOST_IP value. Only change this if the webapp is being hosted on a separate endpoint.
• KC_HOST_IP: Defaults to DOCKER_HOST_IP value. Only change this if the webapp is being hosted on a separate endpoint.

Webapp Variables

• WEBAPP_DOMAIN: The domain that the webapp will run on. This is required for Keycloak CORS authentication.
• API_URI: The endpoint for the CV manager API, must be on a Keycloak Authorized domain.
• VIEWER_MSG_TYPES: List of CV message types to query geospatially.
• DOT_NAME: The name of the DOT using the CV Manager.
• MAPBOX_INIT_LATITUDE: Initial latitude value to use for MapBox view state.
• MAPBOX_INIT_LONGITUDE: Initial longitude value to use for MapBox view state.
• MAPBOX_INIT_ZOOM: Initial zoom value to use for MapBox view state.

API Variables

• MONGO_PROCESSED_BSM_COLLECTION_NAME: The collection name in MongoDB for processed BSM messages.
• MONGO_PROCESSED_PSM_COLLECTION_NAME: The collection name in MongoDB for processed PSM messages.
• MONGO_SSM_COLLECTION_NAME: The database name for SSM visualization data.
• MONGO_SRM_COLLECTION_NAME: The database name for SRM visualization data.
• FIRMWARE_MANAGER_ENDPOINT: Endpoint for the firmware manager deployment's API.
• CSM_EMAIL_TO_SEND_FROM: Origin email address for the API error developer emails.
• CSM_EMAILS_TO_SEND_TO: Destination email addresses for the API error developer emails.
• CSM_EMAIL_APP_USERNAME: Username for the SMTP server.
• CSM_EMAIL_APP_PASSWORD: Password for the SMTP server.
• CSM_TARGET_SMTP_SERVER_ADDRESS: Destination SMTP server address.
• CSM_TARGET_SMTP_SERVER_PORT: Destination SMTP server port.
• API_LOGGING_LEVEL: The level of which the CV Manager API will log. (DEBUG, INFO, WARNING, ERROR)
• CSM_TLS_ENABLED: Set to "true" if the SMTP server requires TLS.
• CSM_AUTH_ENABLED: Set to "true" if the SMTP server requires authentication.
• WZDX_ENDPOINT: WZDX datafeed endpoint.
• WZDX_API_KEY: API key for the WZDX datafeed.
• TIMEZONE: Timezone to be used for the API.
• GOOGLE_APPLICATION_CREDENTIALS: Path to the GCP service account credentials file. Attached as a volume to the CV manager API service.

PostgreSQL Variables

• PG_DB_HOST: The database host, must include the port (normally hostname:5432). Defaults to DOCKER_HOST_IP:5432 but can be configured to a separate endpoint.
• PG_DB_USER: The database user that will be used to authenticate the cloud function when it queries the database.
• PG_DB_PASS: The database user's password that will be used to authenticate the cloud function.
• INSTANCE_CONNECTION_NAME: The connection name for the Cloud SQL instance. (project-id:region:name)

MongoDB Variables

For Windows Users Only

If running on Windows, please make sure that your global git config is set up to not convert end-of-line characters during checkout.

Disable git core.autocrlf (One Time Only)

git config --global core.autocrlf false

• MONGO_DB_URI: URI for the MongoDB connections.
• INSERT_SAMPLE_DATA: If true, sample data will be inserted in the CVCounts, V2XGeoJson, and OdeSsmJson collections

Keycloak Variables

• KEYCLOAK_DOMAIN: Domain name that Keycloak will be served on.
• KEYCLOAK_ADMIN: Admin username for Keycloak configuration.
• KEYCLOAK_ADMIN_PASSWORD: Admin password for Keycloak configuration.
• KEYCLOAK_ENDPOINT: Keycloak base URL to send requests to. Reference the sample.env for the URL formatting.
• KEYCLOAK_REALM: Keycloak Realm name.
• KEYCLOAK_GUI_CLIENT_ID: Keycloak GUI client name (unauthorized client)
• KEYCLOAK_API_CLIENT_ID: Keycloak API client name.
• KEYCLOAK_API_CLIENT_SECRET_KEY: Keycloak API secret for the given client name.
• KEYCLOAK_LOGIN_THEME_NAME: Name of the jar file to use as the theme provider in Keycloak. For generating a custom theme reference the Keycloakify Github
• KC_LOG_LEVEL: The level of which the Keycloak instance will log. (ALL, DEBUG, ERROR, FATAL, INFO, OFF, TRACE, and WARN)
• GOOGLE_CLIENT_ID: GCP OAuth2.0 client ID for SSO Authentication within keycloak.
• GOOGLE_CLIENT_SECRET: GCP OAuth2.0 client secret for SSO Authentication within keycloak.

Environment variables from addon services can also be set in the main .env file. These variables are defined in their own README files in the services/addons/images location of this repository.

Line Endings

This project utilizes a .gitattributes file to normalize line endings for all files. This applies to all new files. To re-run this normalization on existing files, use the following command:

git add --renormalize .
git commit -m "Re-normalize line endings"

On Windows, Disable git core.autocrlf (One Time Only)

git config --global core.autocrlf false

CV Manager Common Issues

1. After logging into the webapp, you are presented with a page reading "Login Unsuccessful: Unknown Error Occurred" This indicates an issue within the cvmanager_api service, see the docker logs for more information. Common issues include: i. Unable to connect to PostgreSQL server (see postgres logs) ii. Keycloak authentication error (see keycloak logs)
2. The webapp needs to be re-build after each environment variable change This is due to the fact that environment variables are injected into the Docker image at BUILD time, not runtime.

docker compose up --build -d cvmanager_webapp

3. The Keycloak Hostname needs to be accessible from docker and a browser Keycloak needs to be accessible other docker containers (cvmanager_api, intersection_api) as well as the webapp (running in a browser). This means that using "localhost" will not work (not accessible from other docker containers), and using "", the docker container network name, will not work either (not accessible in the browser since it is outside of the docker network). This is why the suggested approach is to set your docker host IP address as your keycloak hostname. Keycloak will redirect any incomming connection to the hostname, therefore you cannot set the hostname to "localhost" and have docker services access it at cvmanager_keycloak:8080.
4. The Keycloak volume must be cleared if crucial parameters are changed If the keycloak admin user credentials, client id(s), client secret, or webapp endpoint are changed by environment variable, those changes will not be reflected in keycloak unless the volume is cleared and re-built. This can be done by:

docker compose down -v
docker compose up -d

SMTP Email Testing

For testing Intersection API email generation, a local SMTP server should be used. The recommended server is smtp4dev.

This can be used by enabling the smtp4dev profile in the docker-compose-intersection.yml file. This will start smtp4dev on ports 5000 (web UI), 25 (SMTP), 143 (IMAP), and 110 (POP3).

License Information

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either expressed or implied. See the License for the specific language governing permissions and limitations under the License.