Add basic JSON support

[albertpastrana]

By applying this commit the anon tool will be able to read,
anonymise and output basic JSON files.

It currently only supports one level JSON fields.

Some small refactor has been done and it's more than probable
that the solution could be a bit DRYer, but don't have time
to do so.

Signed-off-by: Albert Pastrana albert.pastrana@intenthq.com

[codecov]

Codecov Report

Merging #25 into master will decrease coverage by 6.62%.
The diff coverage is 65.47%.

@@            Coverage Diff             @@
##           master      #25      +/-   ##
==========================================
- Coverage      85%   78.37%   -6.63%     
==========================================
  Files           3        5       +2     
  Lines         140      185      +45     
==========================================
+ Hits          119      145      +26     
- Misses         18       37      +19     
  Partials        3        3

Impacted Files	Coverage Δ
config.go	100% <ø> (ø)	⬆️
main.go	43.24% <0%> (-24.45%)	⬇️
anonymisations.go	100% <100%> (ø)	⬆️
csv_processor.go	70% <70%> (ø)
json_processor.go	72% <72%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fecd661...744ecb9. Read the comment docs.

[nathankleyn]

Starting with documentation and high-level review for now — a few comments from me! 👍

[nathankleyn]

I think it's a bit confusing to have both csv and json in the same example — should give a bit of preamble and have two examples. Maybe something like this:

At a high-level, the config looks like this:

{
  // Name of the format of the input file
  // Currently supports "csv" and "json"
  "formatName": {
    // Options for the format you have picked go here.
    // See the documentation for the format you choose below.
  },
  "sampling": {
    // FIXME: Steal from below.
  },
  "actions": [
    // FIXME: Steal from below.
  ]
}

Formats

You can use CSV or JSON files as input.

CSV

For a CSV file you will need a config like this:

"csv": {
  "delimiter": ",",
  // Specify in which column a unique ID exists on which the sampling can
  // be performed. Indices are 0 based, so this would sample on the first
  // column.
  "idColumn": "0"
}

JSON

For a JSON file you will need to define config like this:

"json": {
  // Specify in which field a unique ID exists on which the sampling can
  // be performed.
  "idField": "id"
}

[nathankleyn]

Can we camel-case this to match the rest? eg. jsonField instead of JsonField

[albertpastrana]

it's actually how it should be, that was a typo (probably because of copy&pasting from go code)

[nathankleyn]

Couple of final things here — and codecov is complaining about coverage, worth a look! 👍

[nathankleyn]

Suggested change

	Anon is a tool for taking delimited files and anonymising or transforming columns/fields until the output is useful for applications where sensitive information cannot be exposed. Currently this tools supports both CSV and JSON files (with one level of depth).
	Anon is a tool for taking delimited or JSON files and anonymising or transforming columns/fields until the output is useful for applications where sensitive information cannot be exposed. Currently this tools supports both CSV and JSON files (with one level of depth).

[nathankleyn]

This feels a bit out of place — it suddenly talks about JSON when above we've changed it all to talk about there being multiple formats. Worth breaking out into a little explanation about how CSVs will infer the column number whereas JSON needs the jsonField to tell it which one you wanted?