Construction Safety Copilot

Construction Safety Copilot is a multimodal application for reviewing construction site images and generating structured safety observations.

It is designed for workflows such as:

• site safety review
• preliminary hazard identification
• PPE guidance
• supervisor follow-up preparation
• toolbox talk preparation
• report drafting from field images and notes

App Preview

Inline demo preview:

Full recording: View the demo video

Inspection Intake

Safety Analysis Results

Report Output

Core Capabilities

Upload a construction site image and provide light project context such as work activity and field notes. The application returns:

• overall site risk level
• scene summary
• likely hazards and recommended actions
• PPE recommendations
• supervisor follow-up questions
• toolbox talk points
• downloadable Markdown report

Operating Modes

The application supports two modes:

Demo Mode

Demo Mode is intended for product walkthroughs, UI review, and low-cost testing.

In Demo Mode, the app:

• does not call the OpenAI API
• generates a realistic sample safety analysis
• allows the full workflow to be demonstrated without live inference

Live API Mode

Live API Mode is intended for real image analysis.

In Live API Mode, the app:

• sends the uploaded image and user notes to the OpenAI API
• returns a structured safety observation
• uses API credits

Example Use Cases

• Review a site photo before a safety meeting.
• Generate a preliminary observation for field documentation.
• Draft toolbox talk points from visible conditions.
• Prepare supervisor follow-up questions from an image and short field notes.
• Create a report-ready summary for internal safety review.

How It Works

1. Upload a construction site image.
2. Add optional project context and work activity notes.
3. Run the safety analysis.
4. Review the generated hazards, recommendations, and follow-up questions.
5. Download a Markdown report.

Tech Stack

• Python
• Streamlit
• OpenAI Responses API
• Pydantic
• python-dotenv

Project Structure

.
|-- app.py
|-- requirements.txt
|-- .env.example
|-- README.md
|-- examples
|   |-- README.md
|   |-- demo_case_01.md
|   `-- demo_case_02.md
`-- src
    |-- __init__.py
    |-- openai_client.py
    |-- reporting.py
    `-- schemas.py

Setup

1. Create and activate a virtual environment.
2. Install dependencies:

pip install -r requirements.txt

3. Copy .env.example to .env and add your API key:

OPENAI_API_KEY=your_api_key_here
OPENAI_MODEL=gpt-4.1-mini

4. Start the app:

streamlit run app.py

Recommended Usage

Demo Mode

1. Start the app
2. Leave Demo mode (no API cost) turned on
3. Upload a construction image
4. Review the generated output and report

Live API Mode

1. Add a valid OpenAI API key to .env
2. Make sure API billing is active
3. Turn off Demo Mode in the sidebar
4. Upload an image and run analysis

Example Demo Cases

Use the example scenarios in the examples/ folder for repeatable walkthroughs:

• examples/demo_case_01.md
• examples/demo_case_02.md

Notes

• This tool provides an AI-assisted preliminary safety observation, not a certified inspection.
• Output quality depends on image clarity and the quality of the field context provided.
• The model may be uncertain when the image is incomplete, low-resolution, or visually ambiguous.
• Demo Mode is intended for walkthroughs; Live API Mode is what produces image-based model output.

Roadmap

Next

• export polished PDF reports
• add saved inspection history
• improve risk tagging visuals
• support multiple images per review

Future

• add object detection overlays
• map outputs to formal safety taxonomies
• compare observations across time
• support recurring site review workflows

Source Notes

The implementation uses the OpenAI Responses API with image input and structured outputs.