Parse yaml directories as key=value pairs

[katharinahafner]

Fix #258

[katharinahafner]

Minimal example:

# config.yml
vars:
  a: [1, 2, 3]
  b: "foo"
  c: "bar"

Add an action to parse dict-string to dict

import argparse
from configargparse import ArgParser, YAMLConfigFileParser

class StoreDict(argparse.Action):
    def __call__(self, parser, namespace, values, option_string=None):
        import json
        try:
            my_dict = json.loads(values)
        except json.JSONDecodeError:
            my_dict = json.loads(values.replace("'",'"'))
        setattr(namespace, self.dest, my_dict)

p = ArgParser(config_file_parser_class=YAMLConfigFileParser)
p.add('-c', '--config_file', default='config.yml', is_config_file=True)
p.add('--vars', dest="vars", action=StoreDict)
args = p.parse_args()

print(args.vars)

Output: {'a': [1, 2, 3], 'b': 'foo', 'c': 'bar'}

[bw2]

If other users would like to see support for dictionaries, please post your views here or in #258

[tbooth]

I see the original request in #258 for this came from Snakemake, which is also the project I'm personally interested in.

I don't think this PR is the right fix as it stands. In the PR we have:

        elif isinstance(value, dict):
            args.append( "%s=%s" % (command_line_key, value) )

So the entire Python dict is quoted into the argument string, which is equivalent to calling repr(value). Then in the example code above we have:

    my_dict = json.loads(values.replace("'",'"'))

Which replaces the quotes then treats the Python-quoted dict literal as JSON. This sometimes works, sometimes not.

I think for Snakemake what we are really after is:

        elif isinstance(value, dict):
            args.extend( [ "%s=%s" % (k, v) for k, v in value.items() ] )

Snakemake already has internal logic to turn these lists of strings back into dicts, and as long as you assume (or check!) that no dict key contains "=" there's no need for extra quoting or any JSON-shenanigans. My feeling is that this should be OK, but we probably should add an explicit option to activate the behaviour, because it's a change to existing behaviour (no longer raising an error) and not part of the argparse spec.

[tbooth]

I've written some demonstration code showing how this can now be implemented with minimal subclassing of the config parser, if a new tweak_value() hook is implemented in the main module.

See scenario 3 in this code:
https://github.com/tbooth/ConfigArgParse/blob/devel/tests/test_config_parser_subclasses.py

If there is enough enthusiasm for this change, hopefully it can be accepted into the next release of the module.