Skip to content

Conversation

@jaschrep-msft
Copy link
Member

Implements two foundation components to implement partitioned upload and copy.

  • PartitionedStream: Consumes a Box<dyn SeekableStream> and converts it to a Stream<Item = Result<Bytes, Error>> where each Ok(Bytes) returned is a contiguously buffered partition to be used for a put block or equivalent request.
  • run_all_with_concurrency_limit(): Takes a sequence of async jobs (impl FnOnce() -> impl Future<Output = Result<(), Error>>). These will be sequences of put block operations or equivalent requests.

@github-actions github-actions bot added the Storage Storage Service (Queues, Blobs, Files) label Nov 12, 2025
@jaschrep-msft jaschrep-msft marked this pull request as ready for review November 19, 2025 22:01
Copilot AI review requested due to automatic review settings November 19, 2025 22:01
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements foundational components for partitioned upload and copy operations in Azure Storage Blob SDK, introducing stream partitioning and concurrent operation execution capabilities.

  • Adds PartitionedStream that converts a SeekableStream into partitioned Bytes chunks for block operations
  • Implements run_all_with_concurrency_limit() for executing async operations with configurable concurrency
  • Includes comprehensive test coverage for both components

Reviewed Changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 16 comments.

Show a summary per file
File Description
sdk/storage/azure_storage_blob/src/streams/partitioned_stream.rs New implementation of PartitionedStream with Stream and FusedStream traits, plus test suite
sdk/storage/azure_storage_blob/src/streams/mod.rs Module declaration for streams
sdk/storage/azure_storage_blob/src/partitioned_transfer/mod.rs New implementation of run_all_with_concurrency_limit() with concurrency control logic and tests
sdk/storage/azure_storage_blob/src/lib.rs Module declarations for new partitioned_transfer and streams modules
sdk/storage/azure_storage_blob/Cargo.toml Added bytes and futures dependencies (and rand for tests)
Cargo.lock Lock file updates reflecting new dependencies

jaschrep-msft and others added 5 commits November 19, 2025 22:21
Accept useful generated comments.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
generated docs tried to write a doctest for a non-public function.
@jaschrep-msft jaschrep-msft requested a review from heaths December 3, 2025 19:32
@github-actions
Copy link

github-actions bot commented Dec 3, 2025

API Change Check

APIView identified API level changes in this PR and created the following API reviews

azure_storage_blob

@jaschrep-msft
Copy link
Member Author

@heaths I think this is pretty stable now that your core comments are addressed. Can I get another review on this?

@heaths
Copy link
Member

heaths commented Dec 9, 2025

I will, but in general I was letting your team sign off on the rest of the changes and Storage PRs in general. I've only been taking a peek more recently since your team has been much more comfortable with Rust. I'm mainly just looking at more architectural concerns. But you can always mention me specifically in reviews or contact me offline too.

@jaschrep-msft jaschrep-msft merged commit 1bc0ef3 into Azure:main Dec 9, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Storage Storage Service (Queues, Blobs, Files)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants