initial working version of class to get data from Mass Save#3
Conversation
karpdave
changed the title
|
This addresses #1 |
karpdave
changed the title
|
|
||
| import json | ||
|
|
||
| import pandas as pd |
There was a problem hiding this comment.
Could we use polars rather than pandas?
There was a problem hiding this comment.
In masssave_explore.qmd as well! No pandas.
| @@ -0,0 +1,237 @@ | |||
| ######################################################################################## | |||
| ## masssave_reader.py | |||
| ## SwitchBox | |||
| @@ -0,0 +1,237 @@ | |||
| ######################################################################################## | |||
There was a problem hiding this comment.
Let's implement our standard data download pattern from reports:
Rename tmp_data_shared to data, add an ma subdir, and move the existing dataset in tmp_data_shared to data/ma.
Then, let's move this module to data/ma/, rename it masssave_downloader.py, and move the masssave_reader calls from massave_explore.md to a if __name__ == "__main__": block in masssave_downloader.py.
Then add a Makefile task in data/ma/ that calls masssave_downloader.py. If there are any natural parameters that should be defined in masssave_downloader.py and passed via the Makefile, let's do that as well.
The output file should follow the following naming convention: <data_provider>_<topic>_<download_date in YYYMMDD.
| @@ -70,3 +70,23 @@ df.xs("Total",level="municipality")["installed_hp_locations"].unstack() | |||
| ```{python} | |||
There was a problem hiding this comment.
Let's move this notebooks to notebooks/, since it doesn't need to ship with the package in hp_adoption to potential users of the model.
|
Unclear if the Next Steps were fully completed, but I'm merging this in the spirit of cleaning up our outstanding open PRs. |
2 participants
UPDATE: Below has been addressed
This needs a little work to be more general, but it "works".
Next steps:
Once that 2nd point is done, we should have a proper reproducible dataset to work from