IndexError when upgrading to Python 3.11 with rlextra

[daveisfera]

I just tried upgrading from Python 3.9 because it's EOL and it runs into an exception when running pip-compile with rlextra on Python 3.11

Environment Versions

1. Debian Trixie
2. Python version: 3.11.14
3. pip version: 24.0
4. pip-tools version: 7.5.2

Steps to replicate

1. Put rlextra in a requirements.in file using Python 3.9
2. Build requirements.txt with pip-compile
3. Upgrade to Python 3.11
4. Run pip-compile
5. Observe traceback

Expected result

Would work on Python 3.11 like 3.9

Actual result

IndexError happens when processing rlextra. I believe this is because the versions available in pypi don't cover 3.11 and the private repo is needed to get the actual package, but pip-compile assumes that something will be available in pypi and that causes the IndexError

Traceback (most recent call last):
  File "/usr/src/venv/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/src/venv/lib/python3.11/site-packages/click/core.py", line 1485, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/venv/lib/python3.11/site-packages/click/core.py", line 1406, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/src/venv/lib/python3.11/site-packages/click/core.py", line 1269, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/venv/lib/python3.11/site-packages/click/core.py", line 824, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/venv/lib/python3.11/site-packages/click/decorators.py", line 34, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/venv/lib/python3.11/site-packages/piptools/scripts/compile.py", line 482, in cli
    hashes = resolver.resolve_hashes(results) if generate_hashes else None
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/venv/lib/python3.11/site-packages/piptools/resolver.py", line 171, in resolve_hashes
    return {ireq: self.repository.get_hashes(ireq) for ireq in ireqs}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/venv/lib/python3.11/site-packages/piptools/resolver.py", line 171, in <dictcomp>
    return {ireq: self.repository.get_hashes(ireq) for ireq in ireqs}
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/venv/lib/python3.11/site-packages/piptools/repositories/pypi.py", line 329, in get_hashes
    return self._get_req_hashes(ireq)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/venv/lib/python3.11/site-packages/piptools/repositories/pypi.py", line 336, in _get_req_hashes
    matching_candidates = self._get_matching_candidates(ireq)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/venv/lib/python3.11/site-packages/piptools/repositories/pypi.py", line 393, in _get_matching_candidates
    return candidates_by_version[matching_versions[0]]
                                 ~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

[sirosen]

@daveisfera, can you share what version of rlextra is currently pinned in your requirements.txt?

I don't have a full reproducer from this report. The issue I have is that the initial pip-compile run will fail, on Python 3.9 or, I believe, on any other Python version.

From my test run, we can see the error delivered by an attempt to build rlextra:

$ ./venv-39/bin/pip-compile                                   
    error: subprocess-exited-with-error
    
    × python setup.py egg_info did not run successfully.
    │ exit code: 1
    ╰─> [14 lines of output]
        
        !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        !                                                                   !
        ! This is a dummy version of rlextra intended to provide protection !
        ! against namesquatting in pypi.                                    !
        !                                                                   !
        ! To obtain the ReportLab version of rlextra you need to use        !
        !                                                                   !
        !   pip install -i"https://www.reportlab.com/pypi" rlextra          !
        !                                                                   !
        ! hope this helps                                                   !
        !                                                                   !
        !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
...

(more follows)

Using the reportlab.com index seems to require an account, which I do not have.

But presumably the issue can appear with other usages. So we'll need something which we can run -- ideally without special credentials -- to reproduce.

I also tried with rlextra==1.0 in requirements.txt, to make sure, and it didn't make a difference.

[daveisfera]

I'm using 3.6.12.1 of rlextra currently and I believe that the credential are required to reproduce the issue since the versions available in PyPI are just placeholders to prevent name squatting.

rlextra==3.6.12.1 \
    --hash=sha256:8d06328e9c176fb1584e8060ca176f64a128d6c777e25d077303810fd90e09b3
    # via -r requirements.in

I've reached out to their support and referenced this issue, but is there anything I can do to help debug or improve the handling of this because I think that there's something in the logic that's assuming that PyPI will have at least 1 entry for Python 3.11 and it not being there is causing pip-compile to run into the IndexError

[sirosen]

I was thinking that maybe having your exact pin would lead me somewhere interesting, but it didn't. As you note, the publicly available files for this package are just sdist stubs which intentionally fail at build time.

This is in a part of the project where I'm a bit less familiar. All I can tell from the trace and a quick look is that our expectations were violated, possibly by a file on the index which "should" exist having been deleted.

Right now, we don't have a reproducer for this. Creating one is the first step towards a bugfix.
Can you at least share the full pip-compile command you're using? Since you have hashes, clearly you are using some options.

[daveisfera]

pip-compile --allow-unsafe --generate-hashes --output-file=requirements.txt --strip-extras requirements.in

[moggers87]

I can produce the same error with the following command:

pip-compile --generate-hashes -U requirements.in

and requirements.in:

--extra-index-url https://codeberg.org/api/packages/Inboxen/pypi/simple/
inboxen
pip-tools
pylibmc
setuptools
wheel

The package "inboxen" also exists in PyPI (though not the same version) so I was able try without --extra-index-url and pip-compile works as expected.

Versions of pip-tools, pip, and Python used for completeness:

• pip-tools: 7.5.2
• pip: 25.3
• Python: 3.12

I'm wondering if this bug is related to #1536 - running the above command with -v shows that it gets through a fair number of packages before getting to that IndexError:

Generating hashes:                                                                
  psycopg2                                                                                                                                                           
  cachetools                                                                      
  django-otp                                                                      
  django-celery-results                                                                                                                                              
  django                                                                          
  click-plugins                                                                   
  click-didyoumean                                                                                                                                                   
  billiard                           
  django-sendfile2                   
  pyproject-hooks                                                                                                                                                    
  asgiref                                                                         
  faker                                                                           
  lockfile                                                                                                                                                           
  markdown                                                                        
  packaging                                                                       
  typing-extensions                
  amqp                                                                            
  click-repl
  exceptiongroup
  django-cursor-pagination
  dnspython
  wcwidth
  six
  inboxen
Traceback (most recent call last):

[moggers87]

Running the command through pdb:

(Pdb) print(matching_versions)
[]
(Pdb) print(all_candidates)
[InstallationCandidate(name='inboxen', version=<Version('0.0.1')>, link=<Link https://files.pythonhosted.org/packages/c7/c1/aa964a3104a8be4c0aa36fa21d508389c85a1cd8be6b0e805b12b33313f4/inboxen-0.0.1.tar.gz (from https://pypi.org/simple/inboxen/)>)]
(Pdb) print(ireq)
inboxen==2025.9.17.16.34.49+8.ge92f230a from https://codeberg.org/api/packages/Inboxen/pypi/files/inboxen/2025.9.17.16.34.49%2B8.ge92f230a/inboxen-2025.9.17.16.34.49%2B8.ge92f230a-py3-none-any.whl#sha256=1d121aca18001bf883133365873d511a72d5005bbbfea4017f5b6f231eb26fa3 (from -r requirements.in (line 2))

For whatever reason --extra-index-url gets igored when populating all_candidates and it tries PyPI instead. The version found there isn't the version found earlier via my package index on codeberg we get an empty matching_versions and an IndexError.

[webknjaz]

Does pip install behave different? I think it sticks to the first index where it finds project. And the lookup order is not guaranteed.

[webknjaz]

Does pip install behave different? I think it sticks to the first index where it finds project. And the lookup order is not guaranteed.

[moggers87]

pip install works fine with or without hashes.