Data connector can acquire 3 versions of config specifications from td CLI because td API server normalizes all of them. But we should move server-side normalization code to td CLI side. td CLI should send latest (3rd) config only to td API server. And td CLI can show warning messages if users still use old versioned config spec.
The following is 3 types config specification:
- 1st version of config specification: doesn't include
in key like:
type: s3
bucket: my_bucket
path_prefix: path/to/prefix
...
- 2nd version: includes
config key like:
config:
in:
type: s3
bucket: my_bucket
path_prefix: path/to/prefix
...
in:
type: s3
bucket: my_bucket
path_prefix: path/to/prefix
...
Data connector can acquire 3 versions of config specifications from td CLI because td API server normalizes all of them. But we should move server-side normalization code to td CLI side. td CLI should send latest (3rd) config only to td API server. And td CLI can show warning messages if users still use old versioned config spec.
The following is 3 types config specification:
inkey like:configkey like:configkey #79 3rd version: Embulk config specification like: