Skip to content

Possible data loss when importing into DynamoDB! #139

@Dunedan

Description

@Dunedan

While trying to use the cli to import data into DynamoDB I wondered why no data appeared in DynamoDB.
After some digging I noticed that aggregator._flush isn't called when piping data into dyno. That causes up to 25 records not to be imported into DynamoDB and happens for put as well as for import!
In my case my test data had less than 25 records, that's why I noticed it.

Tested on Debian/unstable with NodeJS 7.10.0 & 8.1.3.

The internet seems to suggest (nodejs/node#53) that a .push(null) at the end of the stream is necessary, but as I'm not really familiar with NodeJS I don't know if that applies to reading from stdin as well.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions