Skip to content

Batch operation support: offer#69

Open
davidroberts63 wants to merge 1 commit intopaulomarquesc:masterfrom
davidroberts63:BatchOperationSupport
Open

Batch operation support: offer#69
davidroberts63 wants to merge 1 commit intopaulomarquesc:masterfrom
davidroberts63:BatchOperationSupport

Conversation

@davidroberts63
Copy link

This is providing batch capabilities for adding records. I have not built out tests or docs, but I will do so if this PR is of interest, as an initial contribution offer.

If this is of interest I also have a question about the Save-AzTableBatch name. The 'Save' verb sounds about right, but I'm easily convinced to use something else.

@paulomarquesc
Copy link
Owner

Hi @davidroberts63 , this seems very interesting, have you noticed significant performance improvements on this one?

@davidroberts63
Copy link
Author

Yes. In my situation I was processing about 1.9 million records. Doing them in batches of 100 (the limit for a TableBatch) took my processing from 10-20 records per second, for individual record operations, to 150-200 records per second for batching. It also reduces the number of transactions involved in the cost of the table storage account.

@paulomarquesc
Copy link
Owner

Cool, thanks for the info, I think we should move forward with this PR.

To move forward, a few comments:

  1. Regarding Save-AzTableBatch, the Invoke verb seems more appropriate in this case.
  2. Add the test case
  3. Let's add the cmdlet help at (https://github.com/paulomarquesc/AzureRmStorageTable/tree/master/docs)
  4. Update the main documentation (docs/README.md) to have an end-to-end example.

@paulomarquesc
Copy link
Owner

Hi @davidroberts63, are you still going to continue with this PR? If, yes, can you please provide the changes I asked in my previous comment?

@bszimmer bszimmer mentioned this pull request Feb 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants