Standardize country codes iso 3166 by ram-from-tvl · Pull Request #159 · openclimatefix/solar-consumer

ram-from-tvl · 2026-02-09T05:19:07Z

Pull Request

Description

This PR standardizes all country codes in the solar-consumer project to follow the ISO 3166-1 alpha-3 standard, as requested in issue #157.

Changes made:

Old Code	New Code	Country
`gb`	`gbr_gb`	Great Britain (excluding Northern Ireland)
`nl`	`nld`	Netherlands
`de`	`deu`	Germany
`be`	`bel`	Belgium
`ind_rajasthan`	`ind_rj`	India – Rajasthan

Files updated:

fetch_data.py — Updated country code dict keys, default parameter, docstring, and error message
app.py — Updated all country conditionals, default parameters, docstring, and os.getenv default
save_data_platform.py — Updated default parameter, docstring, and all country comparisons/comments
save_site_database.py — Updated all country comparisons, default parameters, docstrings, and error messages
test_fetch_data.py — Updated country= argument in GB historic tests
test_save_forecast.py — Updated country= argument for NL and India tests
test_save_nl_to_data_platform.py — Updated country= argument
README.md — Updated COUNTRY environment variable documentation
.example.env — Updated default COUNTRY value and available options comment

Only country code parameter strings were changed. No data-level identifiers (e.g. "nl_national" site names, "Belgium" region names from the Elia API) were modified — these are data values, not country code parameters.

Fixes #157

Checklist:

My code follows OCF's coding style guidelines
I have performed a self-review of my own code
I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works (updated existing tests with new country codes)
I have checked my code and corrected any misspellings

ram-from-tvl · 2026-02-09T05:20:19Z

Hi @peterdudfield
I have made changes for country codes standardization.
Please review it and let me know if we are good to go.
Thank you!

ram-from-tvl · 2026-02-09T05:20:59Z

All tests have passed.

Copilot

Pull request overview

This PR aims to standardize country code parameters across the solar-consumer project to ISO 3166-1 alpha-3-style codes (with special handling like gbr_gb), updating application defaults, dispatch logic, and tests/docs accordingly.

Changes:

Updated country-code parameters/defaults across fetch, save, and app orchestration paths (e.g. gb→gbr_gb, nl→nld, de→deu, be→bel, ind_rajasthan→ind_rj).
Updated unit/integration tests to use the new country code strings.
Updated user-facing documentation and example environment configuration to reflect the new codes.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`solar_consumer/fetch_data.py`	Updates default country and dispatch table keys to new codes; updates docstrings/messages.
`solar_consumer/app.py`	Updates default `country` and country→`model_tag` mapping to new codes.
`solar_consumer/save/save_data_platform.py`	Updates default `country` and NL/GB branching comparisons to new codes.
`solar_consumer/save/save_site_database.py`	Updates defaults and country comparisons/docstrings for site-db saving logic.
`solar_consumer/data/fetch_gb_data.py`	Adds fallback logic for missing GSPs (DB-backed) and adjusts handling of empty GSP frames.
`tests/unit/test_fetch_data.py`	Updates GB historic tests to call `fetch_data(..., country="gbr_gb")`.
`tests/unit/test_save_forecast.py`	Updates NL and India country args used in save tests.
`tests/integration/test_save_nl_to_data_platform.py`	Updates NL integration test to use `country="nld"`.
`README.md`	Updates `COUNTRY` env var docs to new codes.
`.example.env`	Updates default `COUNTRY` value and comment options list.

Comments suppressed due to low confidence (1)

solar_consumer/save/save_data_platform.py:56

save_generation_to_data_platform treats any country other than "nld" as the GB branch. Now that the application supports other country codes (e.g. "deu", "bel", "ind_rj"), calling this with those values (e.g. via app.py when SAVE_METHOD=data-platform) will incorrectly assume GB columns like gsp_id/regime exist and can fail or silently write bad data. Add explicit validation (raise for unsupported countries) or implement dedicated branches per supported country.

    # 0. Create the observers required if they don't exist already
    if country == "nld":
        required_observers = {"nednl"}
        id_key = "region_id"
        capacity_col = "capacity_kw"
        capacity_multiplier = 1000
    else:  # gbr_gb
        required_observers = {"pvlive_in_day", "pvlive_day_after"}
        id_key = "gsp_id"
        capacity_col = "capacity_mwp"
        capacity_multiplier = 1e6

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

solar_consumer/data/fetch_gb_data.py

Copilot · 2026-02-09T05:25:01Z

solar_consumer/save/save_site_database.py

    If `capacity_override_kw` is provided, that value will be used when creating the site. 
    For Germany, the TSO’s known capacity is used if no override is given. For NL, default 20GW applied

    Parameters:
        session (Session): CurrentSQLAlchemy session
        pvsite (PVSite): Pydantic model with site metadata
-        country (str): Country code ('nl' or 'de')
+        country (str): Country code ('nld' or 'deu')
        capacity_override_kw (Optional[int]): Force a specific capacity on creation


The get_or_create_pvsite docstring says country is ('nld' or 'deu'), but the function is also used for India Rajasthan via save_generation_to_site_db(... country="ind_rj"). Update the docstring (and/or the capacity-selection comment) to include the supported ind_rj code so callers know it’s valid.

@ram-from-tvl check can we please check this i think we can update the doc string here

solar_consumer/fetch_data.py

Copilot · 2026-02-09T05:25:01Z

README.md


 - `DB_URL=postgresql://postgres:postgres@localhost:5432/neso_solar` : Database Configuration
- `COUNTRY="gb"` : Country code for fetching data. Currently, other options are ["be", "ind_rajasthan", "nl"] 
+- `COUNTRY="gbr_gb"` : Country code for fetching data. Currently, other options are ["bel", "ind_rj", "nld"] 


README COUNTRY documentation lists other options as ["bel", "ind_rj", "nld"], but the application also supports Germany ("deu") (see app.py country→model_tag mapping and fetch_data dispatch table). Update the README options list to include "deu" (and any other supported codes) to avoid misconfiguration.

Suggested change

- `COUNTRY="gbr_gb"` : Country code for fetching data. Currently, other options are ["bel", "ind_rj", "nld"]

- `COUNTRY="gbr_gb"` : Country code for fetching data. Currently, other options are ["bel", "deu", "ind_rj", "nld"]

@ram-from-tvl can we update the README as wel?

solar_consumer/app.py

peterdudfield · 2026-02-16T13:02:25Z

@katyalmohit and @pavankulkarni007 could you review this?

ram-from-tvl · 2026-02-16T13:47:06Z

Hi @peterdudfield
Should i be commiting all the Copilot generated comments before review.
Thank you!

peterdudfield · 2026-02-16T14:29:02Z

Hi @peterdudfield Should i be commiting all the Copilot generated comments before review. Thank you!

Up to you, if you think they are good, use them. If you dont think they are good, perhaps reject them

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

solar_consumer/data/fetch_gb_data.py

Reverts fetch_gb_data.py to upstream state. The backup GSP data logic (DB_URL lookup, LocationSQL queries, national->GSP data synthesis) falls outside the scope of the country-code standardization issue (openclimatefix#157). This will be addressed in a separate PR.

ram-from-tvl · 2026-02-17T06:18:26Z

Hey @PavanRaghavendraKulkarni,

You're right. the backup GSP data logic was out of scope for this PR. I've now:

Removed the out-of-scope fetch_gb_data.py changes from this PR (commit 8e53981). This PR now only contains the country-code standardization changes as described.
Moved the backup GSP data feature to a separate branch: feature/backup-gsp-data-from-national in a separate PR Add backup GSP data synthesis from national data when GSPs are missing #163

Thanks for the review feedback!

PavanRaghavendraKulkarni · 2026-02-18T07:28:00Z

solar_consumer/save/save_site_database.py

    If `capacity_override_kw` is provided, that value will be used when creating the site. 
    For Germany, the TSO’s known capacity is used if no override is given. For NL, default 20GW applied

    Parameters:
        session (Session): CurrentSQLAlchemy session
        pvsite (PVSite): Pydantic model with site metadata
-        country (str): Country code ('nl' or 'de')
+        country (str): Country code ('nld' or 'deu')
        capacity_override_kw (Optional[int]): Force a specific capacity on creation


@ram-from-tvl check can we please check this i think we can update the doc string here

PavanRaghavendraKulkarni · 2026-02-18T07:30:19Z

README.md


 - `DB_URL=postgresql://postgres:postgres@localhost:5432/neso_solar` : Database Configuration
- `COUNTRY="gb"` : Country code for fetching data. Currently, other options are ["be", "ind_rajasthan", "nl"] 
+- `COUNTRY="gbr_gb"` : Country code for fetching data. Currently, other options are ["bel", "ind_rj", "nld"] 


@ram-from-tvl can we update the README as wel?

ram-from-tvl · 2026-02-18T11:38:41Z

@PavanRaghavendraKulkarni Thanks for the review! I've addressed both unresolved comments:

save_site_database.py - Updated the get_or_create_pvsite docstring to include 'ind_rj' alongside 'nld' and 'deu' as supported country codes.
README.md - Added "deu" to the COUNTRY environment variable options list.

- Updated get_or_create_pvsite docstring to include 'ind_rj' as supported country code - Added 'deu' to README COUNTRY environment variable options list

- Resolved conflicts in save_data_platform.py: accepted upstream's refactored config-driven structure, applied ISO 3166 country codes (nl→nld, be→bel, gb→gbr_gb) - Accepted upstream deletion of test_save_nl_to_data_platform.py (replaced by test_save_nl_and_be_to_data_platform.py) - Updated locations.csv country_code column: nl→nld, be→bel - Updated test_save_nl_and_be_to_data_platform.py with new country codes - Updated test_save_be_generation_unit.py with new country codes

PavanRaghavendraKulkarni · 2026-02-23T09:21:38Z

@ram-from-tvl PR lGTM! please resolve the merge conflicts

ram-from-tvl added 5 commits November 4, 2025 10:25

Implement backup logic for PVLive GSP data using national scaling

b66d612

Merge branch 'openclimatefix:main' into main

5f5a1ef

Merge branch 'openclimatefix:main' into main

30c04b1

Merge branch 'openclimatefix:main' into main

db18d0d

Standardize country codes to ISO 3166-1 alpha-3 format

41eeacb

Copilot AI review requested due to automatic review settings February 9, 2026 05:19

Copilot started reviewing on behalf of ram-from-tvl February 9, 2026 05:19 View session

Copilot AI reviewed Feb 9, 2026

View reviewed changes

ram-from-tvl and others added 2 commits February 16, 2026 22:05

Update solar_consumer/fetch_data.py

534c144

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update solar_consumer/app.py

d86bd37

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

PavanRaghavendraKulkarni requested changes Feb 17, 2026

View reviewed changes

solar_consumer/data/fetch_gb_data.py Outdated Show resolved Hide resolved

PavanRaghavendraKulkarni requested changes Feb 18, 2026

View reviewed changes

ram-from-tvl added 2 commits February 18, 2026 17:09

fix: address review comments - update docstring and README

a247487

- Updated get_or_create_pvsite docstring to include 'ind_rj' as supported country code - Added 'deu' to README COUNTRY environment variable options list

ram-from-tvl requested a review from PavanRaghavendraKulkarni February 18, 2026 11:49

	- `COUNTRY="gbr_gb"` : Country code for fetching data. Currently, other options are ["bel", "ind_rj", "nld"]
	- `COUNTRY="gbr_gb"` : Country code for fetching data. Currently, other options are ["bel", "deu", "ind_rj", "nld"]

Uh oh!

Conversation

ram-from-tvl commented Feb 9, 2026

Pull Request

Description

Changes made:

Files updated:

Checklist:

Uh oh!

ram-from-tvl commented Feb 9, 2026

Uh oh!

ram-from-tvl commented Feb 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

PavanRaghavendraKulkarni Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

PavanRaghavendraKulkarni Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

peterdudfield commented Feb 16, 2026

Uh oh!

ram-from-tvl commented Feb 16, 2026

Uh oh!

peterdudfield commented Feb 16, 2026

Uh oh!

Uh oh!

ram-from-tvl commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PavanRaghavendraKulkarni Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

PavanRaghavendraKulkarni Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

ram-from-tvl commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PavanRaghavendraKulkarni commented Feb 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ram-from-tvl commented Feb 17, 2026 •

edited

Loading

ram-from-tvl commented Feb 18, 2026 •

edited

Loading