Likely to be more memory efficient as the client need only hold and update one target's data at any time. Also more durable, as a failure will only affect one target.
This will allow the pipeline to scale out, with potentially multiple client instances running
The current approach of using a QTable does now work but it relies on reading in the whole file, working on all selected targets before overwriting the file with an updated copy at the end of the step. If anything goes wrong, we get none of the updates.
Likely to be more memory efficient as the client need only hold and update one target's data at any time. Also more durable, as a failure will only affect one target.
This will allow the pipeline to scale out, with potentially multiple client instances running
The current approach of using a QTable does now work but it relies on reading in the whole file, working on all selected targets before overwriting the file with an updated copy at the end of the step. If anything goes wrong, we get none of the updates.