feat: added support for Azure sensitive column in cur_sanitize.py

[TheSamStewart]

Added support to sanitize azure sensitive columns in cur_sanitize.py. Checked according to: https://learn.microsoft.com/en-us/cloud-computing/finops/focus/metadata.

• Supports FOCUS 1.0 and FOCUS 1.2
• Accounts for PascalCase and lowercase
• Checked against example Datasets

[jnioche]

Thanks @TheSamStewart

We won't support FOCUS, at least until it provides the information we need to do GreenOps. At the moment, it is in provider-specific columns, which defeats the object of having a standard.

Instead of lowercasing the values, maybe we can change the script so that it removes the column in a case insensitive way? This will make it easier to maintain in the long run

Using the data from https://microsoft.github.io/finops-toolkit/open-data

duckdb <<'EOF'
create table spruced as select * from 'azure-examples/EA-Cost-Actual.csv';
.mode line
select * from spruced limit 1;
EOF

will give you the list of column with example values.

These columns look like they should be removed as well

        AdditionalInfo = ""AMLResourceType"": ""Provisioner.Batch""
          ServiceInfo1 = ""env"": ""prod""
          ServiceInfo2 = ""org"": ""trey""

What do you think? Thanks!

[jnioche]

see comment

[jnioche]

thaks @TheSamStewart
Did you test the script on the csv files from https://microsoft.github.io/finops-toolkit/open-data ?