Skip to content

Conversation

@hyongtao-code
Copy link
Contributor

@hyongtao-code hyongtao-code commented Dec 28, 2025

On Windows, some overlapped I/O operations leak buffers when they fail to start, notably in WSASendTo(), WSARecvFrom(), and WSARecvFromInto().

This change ensures that buffers are released on failure by clearing the overlapped state, following the same pattern used in other overlapped error paths. This issue is similar in nature to commit 5485085, which fixed related overlapped cleanup problems.

Python script for tracking leaks

import sys
import _overlapped

def total():
    return sys.gettotalrefcount()

if not hasattr(sys, "gettotalrefcount"):
    raise SystemExit("need a debug build (Py_DEBUG) for gettotalrefcount")

before = total()

N = 20000
STEP = 1000

for i in range(1, N + 1):
    ov = _overlapped.Overlapped()
    buf = memoryview(bytearray(4096))
    try:
        ov.WSASendTo(0x1234, buf, 0, ("127.0.0.1", 1))
    except OSError:
        pass

    if i % STEP == 0:
        now = total()
        print(f"{i:6d}: totalrefcount delta = {now - before:+d}")

after = total()
print(f"done: N={N}, totalrefcount delta = {after - before:+d}")

Result without the patch

d:\MyCode\cpython\PCbuild\amd64>python_d.exe py_overlapped_leak.py
  1000: totalrefcount delta = +4007
  2000: totalrefcount delta = +8009
  3000: totalrefcount delta = +11906
  4000: totalrefcount delta = +15906
  5000: totalrefcount delta = +19906
......
 19000: totalrefcount delta = +75906
 20000: totalrefcount delta = +79906
done: N=20000, totalrefcount delta = +79905

Result with the patch

d:\MyCode\cpython\PCbuild\amd64>python_d.exe py_overlapped_leak.py
  1000: totalrefcount delta = +10
  2000: totalrefcount delta = +10
  3000: totalrefcount delta = +10
  4000: totalrefcount delta = +10
  5000: totalrefcount delta = +10
......
 19000: totalrefcount delta = +10
 20000: totalrefcount delta = +10
done: N=20000, totalrefcount delta = +9

picnixz
picnixz previously approved these changes Dec 28, 2025
Copy link
Member

@picnixz picnixz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll need a NEWS entry for that and some tests as well. Please add them so that we can run them under the refleak bot as well.

cc @zooba

Py_RETURN_NONE;
default:
self->type = TYPE_NOT_STARTED;
Overlapped_clear(self);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was very confused with this, I thought it was tp_clear but it's explicitly not tp_clear so this looks like the correct fix.

@picnixz picnixz dismissed their stale review December 28, 2025 14:41

wrong button

@picnixz picnixz requested a review from zooba December 28, 2025 14:44
@hyongtao-code
Copy link
Contributor Author

and some tests as well

Thanks for the review. I will add a suitable test case.

# If the exported buffer is still held, this will raise BufferError.
buf.append(1)

@support.refcount_test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need this? I think you can just run -R : from the CLI to check if three is a refleak. It's more robust as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the guidance — that makes sense now. I’ve updated the test case, and here are the results from my local run.

Without the patch

d:\MyCode\cpython>PCbuild\amd64\python_d.exe -m test -R 3:3 test_asyncio.test_windows_utils
Using random seed: 514712329
0:00:00 Run 1 test sequentially in a single process
0:00:00 [1/1] test_asyncio.test_windows_utils
beginning 6 repetitions. Showing number of leaks (. for 0 or less, X for 10 or more)
123:456
XX5 555
test_asyncio.test_windows_utils leaked [5, 5, 5] references, sum=15
0:00:01 [1/1/1] test_asyncio.test_windows_utils failed (reference leak)

== Tests result: FAILURE ==

1 test failed:
    test_asyncio.test_windows_utils

Total duration: 1.2 sec
Total tests: run=6
Total test files: run=1/1 failed=1
Result: FAILURE

with the patch

d:\MyCode\cpython>PCbuild\amd64\python_d.exe -m test -R 3:3 test_asyncio.test_windows_utils
Using random seed: 3834115587
0:00:00 Run 1 test sequentially in a single process
0:00:00 [1/1] test_asyncio.test_windows_utils
beginning 6 repetitions. Showing number of leaks (. for 0 or less, X for 10 or more)
123:456
XX. ...
0:00:01 [1/1] test_asyncio.test_windows_utils passed

== Tests result: SUCCESS ==

1 test OK.

Total duration: 1.2 sec
Total tests: run=6
Total test files: run=1/1
Result: SUCCESS

Comment on lines +149 to +150
ov.WSARecvFromInto(0x1234, buf, len(buf), 0)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ov.WSARecvFromInto(0x1234, buf, len(buf), 0)
ov.WSARecvFromInto(0x1234, buf, len(buf), 0)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants