Click 8.3.3 breaks `pytest` when stream `fd` is duplicated

[ryanking13]

Starting from Click 8.3.3, I've noticed that it breaks pytest when the standard stream is duplicated with os.dup2.

I guess the related change is: #3244

I am not sure if this is an expected behaviro of that change, but for me it was a bit surprising behavior, hence reporting.

Reproducer

"""
Passes on click <=8.3.2, crashes pytest on click 8.3.3.

    pip install click==8.3.3 pytest
    pytest test_click_fileno_regression.py  # OSError: Illegal seek

    pip install click==8.3.2
    pytest test_click_fileno_regression.py   # passes
"""

import io
import os
import sys

import click
from click.testing import CliRunner


@click.command()
def cmd():
    # Many libraries (build tools, logging tees, subprocess wrappers)
    # use os.dup2 on sys.stdout.fileno() to redirect output at the
    # fd level.
    #
    # On click <=8.3.2, sys.stdout.fileno() inside CliRunner raises
    # UnsupportedOperation.
    #
    # On click 8.3.3, sys.stdout.fileno() returns the original
    # stream's fd
    try:
        stdout_fd = sys.stdout.fileno()
    except io.UnsupportedOperation:
        click.echo("fileno() not available (click <=8.3.2)")
        return

    r, w = os.pipe()
    os.dup2(w, stdout_fd)
    os.close(w)
    os.close(r)
    click.echo("hello")


def test_invoke():
    runner = CliRunner()
    result = runner.invoke(cmd)
    assert result.exit_code == 0

Traceback (most recent call last):
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/bin/pytest", line 10, in <module>
    sys.exit(console_main())
             ^^^^^^^^^^^^^^
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 223, in console_main
    code = main()
           ^^^^^^
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 199, in main
    ret: ExitCode | int = config.hook.pytest_cmdline_main(config=config)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/_pytest/main.py", line 365, in pytest_cmdline_main
    return wrap_session(config, _main)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/_pytest/main.py", line 360, in wrap_session
    config._ensure_unconfigure()
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1171, in _ensure_unconfigure
    self._cleanup_stack.close()
  File "/Users/gyeongjae/.local/share/uv/python/cpython-3.12.12-macos-aarch64-none/lib/python3.12/contextlib.py", line 618, in close
    self.__exit__(None, None, None)
  File "/Users/gyeongjae/.local/share/uv/python/cpython-3.12.12-macos-aarch64-none/lib/python3.12/contextlib.py", line 610, in __exit__
    raise exc_details[1]
  File "/Users/gyeongjae/.local/share/uv/python/cpython-3.12.12-macos-aarch64-none/lib/python3.12/contextlib.py", line 595, in __exit__
    if cb(*exc_details):
       ^^^^^^^^^^^^^^^^
  File "/Users/gyeongjae/.local/share/uv/python/cpython-3.12.12-macos-aarch64-none/lib/python3.12/contextlib.py", line 478, in _exit_wrapper
    callback(*args, **kwds)
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/_pytest/capture.py", line 778, in stop_global_capturing
    self._global_capturing.pop_outerr_to_orig()
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/_pytest/capture.py", line 659, in pop_outerr_to_orig
    out, err = self.readouterr()
               ^^^^^^^^^^^^^^^^^
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/_pytest/capture.py", line 706, in readouterr
    out = self.out.snap() if self.out else ""
          ^^^^^^^^^^^^^^^
  File "/Users/gyeongjae/Desktop/github/pyodide/fork/pyodide-build/.venv/lib/python3.12/site-packages/_pytest/capture.py", line 591, in snap
    self.tmpfile.seek(0)
OSError: [Errno 29] Illegal seek

Environment:

• Python version: 3.12.3
• Click version: 8.3.3

---

[kdeldycke]

When working on #3244 I had no idea people were using os.dup2 in their own Click CLI but that's plausible. I guess this count as a legitimate limitation.

Sounds like I need to continue the work started at #3244. Let me look at pytest which serves me as an inspiration for capture management.

[ryanking13]

Thanks a lot!

[kdeldycke]

A fix is being attempted at: #3391

[kdeldycke]

@ryanking13 can you test it and provide me feedbacks?

[ryanking13]

@kdeldycke Hi, I can confirm that your fix works in our project!

[kdeldycke]

@kdeldycke Hi, I can confirm that your fix works in our project!

Thanks a lot for your feedback! Some rough edges to fix on my side but at least it validates the principle of the fix! :)