fix: close file handles via context managers, fix Config string-concat and mutable default arg bugs#47
Conversation
|
Hi Suyash, great catch on the mutable default argument Also, since you are refactoring FileClass.py, I’ll drop the an optimization note here: Really clean fixes overall, especially the thread state correction. |
|
This is a good cleanup - switching to context managers definitely fixes the file handle leak issue but one thing I noticed is that we’re still opening files with Maybe it’s worth considering a temp-file + atomic rename approach here? That way the original file isn’t touched unless the new write fully succeeds. Since these JSON files hold core model data, that might make the writes a bit more resilient. |
Great point on json.load(f) vs json.loads(f.read()) — the streaming approach |
…t bug, fix mutable default arg in CustomThread
- FileClass.py: Replace bare open/close pairs with with context managers
in readFile, writeFile, writeFileUJson, readParamFile. Prevents file handle
leaks when exceptions are raised between open() and close().
- Config.py: Add missing comma in PARAMETERS_C['EmissionActivityRatio'].
'e''t' was silently parsed by Python as 'et' (implicit string concat),
giving this param only 4 dimensions instead of the required 5 (r, e, t, y, m).
- CustomThreadClass.py: Replace mutable default argument kwargs={} with
kwargs=None sentinel. All instances that omitted kwargs previously shared
the same dict object; now each call gets its own independent dict.
…d readParamFile Streaming directly from the file object avoids loading the entire JSON payload into a string before parsing, reducing peak RAM usage for large OG-CLEWS multi-country scenario files.
735b7d2 to
8923c99
Compare
Summary
Fixes three real bugs found in the API base classes during code review.
What changed
1. FileClass.py — Resource leak: unclosed file handles
All four methods (readFile, writeFile, writeFileUJson, readParamFile) used
bare
open()+close(). If an exception fired between them, the file handle wasnever closed — causing "Too many open files" OS errors under load and potential
data corruption on writes.
Fix: Replaced all four
open/closepairs withwithcontext managers, whichguarantee closure even on exception.
2. Config.py — Silent string concatenation in
PARAMETERS_CA missing comma between
'e'and't'in theEmissionActivityRatioentry: