feat: add cross-platform transferables support and lean transport benchmarks

[kartikey321]

Related issue

Closes #56

What this PR changes

1) Transferables support across platforms

Threads a transferables parameter end-to-end from IsolateManager.compute() → IsolateQueue → sendMessage() → sendIsolate() / postMessage(). Callers opt in explicitly — no implicit buffer detachment.

Supported transferable types:

• VM: ByteBuffer, Uint8List (wrapped via codec), or pre-built TransferableTypedData (O(1) send)
• Web (dart2js): ByteBuffer, Uint8List, or raw JSArrayBuffer — passed directly in the postMessage transfer list for true zero-copy

2) Native transferable codec

Extends the native envelope codec to support TransferableTypedData directly in the transferables list, so callers who pre-build their own TTD objects can pass them through without the codec re-wrapping them.

3) Web transfer correctness tests

Adds browser tests verifying:

• Source ArrayBuffer is detached after postMessage with transfer list on dart2js (confirms zero-copy)
• JSArrayBuffer is accepted directly in the transferables list
• WASM correctly relaxes detachment assertions (WASM linear memory is opaque to the JS engine)

4) Shared benchmark helpers

Extracts benchmark_helpers.dart with shared runBenchmarkCase, buildBytes, formatMs, benchmarkSizesKb to remove duplication between VM and web benchmark files.

---

Benchmarks

All runs use pre-allocated data — timing captures transport cost only, not allocation or fill. Reported as median over measured samples after warmup.

Commands:

# VM
dart test --platform vm test/transfer_performance_test.dart -r expanded

# Web (dart2js)
dart test -p chrome --compiler dart2js test/web_transfer_performance_test.dart -r expanded

# Web (dart2wasm)
dart test --platform chrome --compiler dart2wasm test/web_transfer_performance_test.dart -r expanded

VM (native isolate)

Size	without transferables	ByteBuffer transfer	pre-built TransferableTypedData¹
100 KB	0.10 ms	0.10 ms (1.0×)	0.12 ms
1 MB	0.47 ms	0.93 ms	0.25 ms (1.9× faster)
10 MB	3.85 ms	4.58 ms	2.92 ms (1.3× faster)

¹ Pre-built TTD is the fastest VM path. The ByteBuffer mode routes through the native transferable codec which calls TransferableTypedData.fromList() internally — at 1 MB that codec overhead is visible. If you build your TransferableTypedData ahead of time and pass it in transferables, the timed path skips that copy entirely and captures only the O(1) channel cost.

Web — dart2js (Chrome)

Size	without transferables	with ArrayBuffer transfer	speedup
100 KB	0.20 ms	0.20 ms	~1×
1 MB	0.70 ms	0.50 ms	1.4×
10 MB	8.20 ms	5.10 ms	1.6×

Transfer lists give real zero-copy wins on dart2js. After postMessage, the source ArrayBuffer is detached (lengthInBytes → 0), confirming no copy occurred. Benefit grows with payload size.

Web — dart2wasm (Chrome)

Size	without transferables	with ArrayBuffer transfer	note
100 KB	4.50 ms	6.30 ms	slower
1 MB	60.10 ms	63.50 ms	~same
10 MB	699.60 ms	780.80 ms	slower

dart2wasm cannot benefit from transfer lists. WASM linear memory is opaque to the JS engine, so the runtime must copy bytes from the WASM heap into a JS ArrayBuffer before postMessage regardless of whether a transfer list is provided — then transfer lists add interop overhead on top. Recommendation: omit transferables when targeting dart2wasm. The _isWasm compile-time flag in tests relaxes buffer-detachment assertions accordingly.

---

Notes

• All numbers are from a local machine (Apple Silicon). Results will vary by hardware, OS, and browser version.
• dart2js is the recommended compile target for workloads that move large typed data between workers.
• The transferables parameter is always optional and backward-compatible — existing code requires no changes.

[kartikey321]

Are there any issues from my side behind the ci cancelling/failing? On the local machine the tests are running without any issues

[lamnhan066]

The issue is currently being caused by the Dart formatter.

Run dart format --output=none --set-exit-if-changed .
Changed lib/src/utils/converter.dart
Changed lib/src/utils/native_transferable_codec.dart
Changed test/auto_transfer_test.dart
Changed test/native_transferable_codec_test.dart
Changed test/native_transferables_vm_test.dart
Changed test/transfer_performance_test.dart
Changed test/web_transfer_performance_test.dart
Formatted 64 files (7 changed) in 0.15 seconds.
Error: Process completed with exit code 1.

[kartikey321]

Hi @lamnhan066 — two issues showed up in CI that I've since fixed in this branch

1. dart test --platform=vm failed to load isolate_manager_test.dart

Root cause: processBytes in test/web_transfer_test.dart was annotated with @isolateManagerWorker, which caused the code generator (run as a CI step) to
inject import 'web_transfer_test.dart' into isolate_manager_test.dart. That pulled dart:js_interop (a web-only library) into a VM compilation, causing a
hard compile error.

Fix:

• Removed @isolateManagerWorker and @pragma('vm:entry-point') from processBytes — the pre-generated worker at test/workers/processBytes.js is already
  committed so the annotation is no longer needed
• Replaced the top-level import 'dart:js_interop' with a conditional import (web_js_interop_helpers.dart on web, web_js_interop_helpers_stub.dart on VM)
  so the file compiles cleanly on all platforms while preserving the JSArrayBuffer passthrough test on Chrome

2. [Chrome, Dart2Wasm] web_transfer_performance_test.dart: transport benchmark 10240KB timed out on Ubuntu

Root cause: The 10MB WASM benchmark pre-allocates 60MB of data (6 × 10MB arrays filled byte-by-byte in WASM linear memory) before timing starts. On
Ubuntu CI runners (2-core x86, headless Chrome without GPU acceleration) this pushed the test past the default 30-second timeout. The benchmark itself is
also less meaningful on WASM since dart2wasm must copy from linear memory to the JS heap regardless of transfer lists, so there's no benefit to measure
at large sizes.

Fix: Added an early return for _isWasm && sizeKb >= 10240 — the 10MB WASM numbers are already recorded in the PR description from local runs.

Sorry for the noise — everything passes cleanly now including the full coverage run. Please rerun when you get a chance!

[lamnhan066]

We should not remove the annotations because we need to regenerate them to ensure they are compatible with both old and new versions on each release.

[lamnhan066]

You can also run tool/run_test.sh to run the workflow-like locally. It will also regenerate the workers and clean up automatically after completion.

[codecov]

Codecov Report

❌ Patch coverage is 66.45161% with 52 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.43%. Comparing base (74a7b65) to head (d6627c8).

Files with missing lines	Patch %	Lines
lib/src/utils/native_transferable_codec.dart	66.34%	35 Missing ⚠️
lib/src/utils/converter.dart	0.00%	14 Missing ⚠️
lib/src/utils/auto_transfer.dart	86.66%	2 Missing ⚠️
..._controller/isolate_contactor_controller_stub.dart	90.90%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #57      +/-   ##
==========================================
- Coverage   91.81%   85.43%   -6.38%     
==========================================
  Files          23       25       +2     
  Lines         464      604     +140     
==========================================
+ Hits          426      516      +90     
- Misses         38       88      +50     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[lamnhan066]

The reports said that we were missing tests. Are there any code that we couldn’t write tests for?

[kartikey321]

Yeah @lamnhan066 , it was an issue from my side i thought the edge cases are covered , depending on how dart processes things
but coverage works differently for each code block, so i have expanded the tests. Now ig the test coverage will pass for it

[lamnhan066]

Could you also update the docs and note about the pros and cons of the new implementations?

[kartikey321]

@lamnhan066 I have added the docs, should i add changelogs with it and version bump to 6.2.0 with it or just update the docs with the pros and cons of using it with each platform and provide the changelogs separately?

[lamnhan066]

I'll update the changelogs and bump the version later before releasing the next version, you just need to update the docs with pros and cons.

[kartikey321]

@lamnhan066 Can you try check the documentation. Do you want me to add anything else it it

[lamnhan066]

The PR is completed and ready to merge, but this is a big implementation for the isolate_manager and it's a little bit out of my knowledge, so I need to spend more time understanding it before merging. I hope the PR will be merged this weekend. Thank you for your contribution!

[kartikey321]

Thank you for the update. I hope the pr helps. Also if you have any issues with this or anything else, I would be happy to contribute.

[lamnhan066]

Hello @kartikey321

After running the benchmark and reviewing the code, I’ve noticed a significant performance improvement, but I still have a few questions:

1. How did you implement the native_transferable_codec? Was it generated by AI, and how can we be confident it correctly handles edge cases in real-world scenarios?

2. Do the benchmarks include the time spent converting TransferableTypedData?

3. Should we automatically avoid using it in WASM (for example, by relying on the kIsWasm flag)?

[kartikey321]

Allow me to clarify the things one by one
Why did I need this implementation? In my company i am working on a performant file upload/download handler and while transfering the bytes to different isolates/ workers on a large file it felt the performance drop. In that impl i used TranferableTypedData for non web platforms but forced to use raw bytes in web version thats when I had raised the issue #56

While working on it I felt if we had it working for all platforms instead of just web not only will it be consistent along all platforms but also help in non web platforms.

Regarding the codec implementation the initial draft and working was done by me to parse the incoming message,
to first check if the message was sent by transferable or not use _envelopeMarkerKey and further check iteratively if any message value was uint8List or data. I used claude code to research on it and iteratively improve the code

Similar implementation existed in GraphQL Multipart Request (https://github.com/jaydenseric/graphql-upload/blob/master/processRequest.mjs#L140-L220)

So yeah ai was used to research , improve the code and write documentation for it

When i ran the tests i was not very happy with the results only the web was profiting from it , both dart vm and wasm were showing if not increased but similar or not much difference as i had quoted in the issue expectations

it was because it was converting it at that time so making the transferable use obsolete as a copy was regardlessly made. So i tried with prebuilt transferablebytes and arraybuffers and apart from wasm i was getting quite good results.

if you run the tests there are three types of tests, there are three types of results
without transferables | ByteBuffer transfer | pre-built TransferableTypedData

withouttranferables is how it is usually sent to sendport or workers
ByteBuffer transfer is my codec converting the data into transferables
third pre-built Transferables where the the tranferables are previously built
So yeah the second type ByteBuffer transfer includes that time

Should we automatically avoid using it in WASM (for example, by relying on the kIsWasm flag)?
we could do it but that would mean different api working on different platform, i was more thinking about a warning or something to not to use this on wasm

Lastly summarising the changes done by me were the initial draft code and changes, ai helped me in improving it adding additional test cases to come at par with the coverage and lastly write the readmes for the test results i shared with you

If you are skeptical of the codec, we could remove that and allow only prebuilt transferables, but in that case the user has to be vigilant about using it and not materialising if after sending it. I kept that to minimise the effort and even if low some performance improvements

[lamnhan066]

Thank you for your PR. I'll merge it and perform some additional testing and review before releasing it to production.