Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
194 changes: 194 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,194 @@
# System Architecture Overview

This document provides a high-level overview of the architectural
boundaries and dependency rules enforced across the codebase.

------------------------------------------------------------------------

# 1. High-Level Package Architecture

┌──────────────┐
│ public_api │
└───────┬──────┘
│ (via designated interface only)
┌────────────┐
│ metrics │
└────────────┘

┌────────────┐
│ cms │
└──────┬─────┘
│ (independent except via interface)
metrics


┌────────────┐
│ ingestion │
└──────┬─────┘
│ (via interface only)
metrics


┌────────────┐
│ validation │
└──────┬─────┘
│ (via interface only)
metrics


┌────────────┐
│ feedback │
└────────────┘
(isolated)

Key Principles:

- `metrics` is the core module.
- External modules depend on `metrics` only through designated
interfaces.
- CMS and Metrics are independent except for controlled integration
points.
- Feedback is isolated and behaves like a plugin module.

------------------------------------------------------------------------

# 2. Metrics Internal Layering

Enforced layered architecture:

metrics.api
metrics.domain
metrics.data

Allowed direction: **top → down**\
Forbidden direction: **bottom → up**

Examples:

- ✔ `api → domain`
- ✔ `domain → data`
- ✖ `data → domain`
- ✖ `domain → api`

This ensures clean separation of concerns.

------------------------------------------------------------------------

# 3. Metrics Domain Sibling Isolation

Domain submodules are isolated from one another.

metrics.domain
├── bulk_downloads
├── charts
├── headlines
├── tables
├── trends
└── weather_health_alerts

No cross-imports between sibling domains:

- `charts` ✖ `headlines`
- `tables` ✖ `trends`
- `trends` ✖ `weather_health_alerts`

This prevents hidden coupling and keeps business logic modular.

------------------------------------------------------------------------

# 4. Public API Isolation

The Public API module has strict boundaries.

public_api
│ (interface only)
metrics.interface

Forbidden direct dependencies:

- metrics.api
- metrics.domain
- metrics.data
- cms
- ingestion
- feedback
- validation

Only designated interfaces are allowed.

------------------------------------------------------------------------

# 5. Ingestion Isolation

ingestion
│ (interface only)
metrics.core_models

Ingestion cannot depend directly on:

- metrics.api
- cms
- public_api
- feedback
- validation

This maintains a clean ingestion boundary.

------------------------------------------------------------------------

# 6. Feedback Module Isolation

Feedback is intentionally isolated:

feedback

(no outward dependencies)

The rest of the system:

- Cannot import `feedback`
- Except for explicit URL wiring allowances

This allows feedback to behave like a replaceable subsystem.

------------------------------------------------------------------------

# 7. Source vs Test Boundary

Production code must not depend on test code.

Source Code ✖ tests

Prevents accidental coupling to test utilities.

------------------------------------------------------------------------

# Architectural Summary

This architecture represents:

- A modular monolith
- Strict layered architecture within the core
- Interface-based integration
- Strong dependency direction enforcement
- Domain isolation
- Plugin-style subsystems
- Architectural contracts enforced via Import Linter

The result is a disciplined, maintainable, and evolution-friendly
system.
4 changes: 2 additions & 2 deletions docs/development_standards.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ As such it is important to ensure the body of the test is signposted with enough

In this example, we follow the same structure as if we were writing a unit test but we add more comments as necessary.

> Note that we consider something to be a unit test if there is some I/O bound activity.
> Note that we consider something to be an integration test if there is some I/O bound activity.
> Think request over network or database query.

Generally, test time can be a good indication
Expand Down Expand Up @@ -320,7 +320,7 @@ to use key word arguments instead of positional arguments.

We shall avoid the use of `assert` statements in source code.

For Python processes, the application of th `assert` keyword can be easily switched off in production
For Python processes, the application of the `assert` keyword can be easily switched off in production
with the [PYTHONOPTIMIZE](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONOPTIMIZE) flag.
This is usually done for compilation optimization purposes.

Expand Down
2 changes: 1 addition & 1 deletion docs/ingestion_design_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ to the `processed/` or the `failed/` folder within the s3 bucket, depending on t
It should be noted that the `source_data/` directory at the root level of the project contains a number of
source data files which are used to populate databases in development environments.

These files are ran through and handled by the same `Consumer` class
These files are run through and handled by the same `Consumer` class
as the ingestion lambda function in deployed environments.
The only exception being, we consume the test dataset with multiprocessing.

Expand Down