Bulk copy fails on RHEL 8 / aarch64 with mssql-python 1.8.0: bundled `mssql_py_core` requires libssl.so.3 and GLIBC ≥ 2.34, but wheel is tagged `manylinux_2_28_aarch64`

[dlevy-msft-sql]

Summary

On a fresh RHEL 8.10 aarch64 host, mssql-python==1.8.0 installs cleanly from PyPI, but invoking bulk copy raises:

ImportError: Bulk copy requires the mssql_py_core library which is not available. This is an unexpected error.

This matches the report at https://www.reddit.com/r/SQLServer/comments/1trcsy7/comment/op95k4b/.

After investigation, the underlying cause appears to be that the bundled mssql_py_core extension shipped inside the manylinux_2_28_aarch64 wheel has runtime requirements (libssl.so.3, libcrypto.so.3, GLIBC_2.34) that exceed what the manylinux_2_28 tag promises (RHEL 8 / glibc 2.28 / OpenSSL 1.1). I may be wrong about the exact build-time root cause, but the loader-level symptoms are reproducible and shown below.

Environment

• OS: Red Hat Enterprise Linux release 8.10 (Ootpa)
• Arch: aarch64
• Python: 3.11.x (from python3.11 RHEL AppStream module)
• Install: pip install mssql-python==1.8.0 in a clean venv
• glibc on host: 2.28
• OpenSSL on host: 1.1.1k (no libssl.so.3 present)

Reproduction

sudo dnf install -y python3.11 python3.11-pip libtool-ltdl krb5-libs
python3.11 -m venv ~/mssqltest
source ~/mssqltest/bin/activate
pip install --upgrade pip
pip install mssql-python==1.8.0
python -c "import mssql_py_core"

Observations

1. mssql_py_core is in fact bundled (contrary to what the error message implies)

The mssql-python 1.8.0 wheel installs mssql_py_core as a sibling top-level package. From mssql_python-1.8.0.dist-info/RECORD:

mssql_py_core/__init__.py,sha256=dWpohDsyIDgFPp8wCLZEDeEsk_Kn4vL8Bp83Ptn-jNg,135
mssql_py_core/__pycache__/__init__.cpython-311.pyc,,
mssql_py_core/mssql_py_core.cpython-311-aarch64-linux-gnu.so,sha256=…,8225592

So this is not a missing-package problem — the file is on disk.

2. The real failure is at dynamic-link time

$ python -c "import mssql_py_core"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File ".../site-packages/mssql_py_core/__init__.py", line 1, in <module>
    from .mssql_py_core import *
ImportError: libssl.so.3: cannot open shared object file: No such file or directory

$ ldd .../mssql_py_core/mssql_py_core.cpython-311-aarch64-linux-gnu.so 2>&1 | grep -E 'ssl|crypto|not found'
        /lib64/libm.so.6: version `GLIBC_2.29' not found (required by …mssql_py_core…)
        /lib64/libc.so.6: version `GLIBC_2.32' not found (required by …mssql_py_core…)
        /lib64/libc.so.6: version `GLIBC_2.33' not found (required by …mssql_py_core…)
        /lib64/libc.so.6: version `GLIBC_2.34' not found (required by …mssql_py_core…)
        libssl.so.3 => not found
        libcrypto.so.3 => not found

Host versions for comparison:

$ ls /usr/lib64/libssl* /usr/lib64/libcrypto*
/usr/lib64/libcrypto.so.1.1     /usr/lib64/libssl.so.1.1
/usr/lib64/libcrypto.so.1.1.1k  /usr/lib64/libssl.so.1.1.1k
/usr/lib64/libssl3.so   # this is NSS, not OpenSSL 3 — unrelated despite the name

3. Wheel tag vs. actual requirements

The installed wheel is mssql_python-1.8.0-cp311-cp311-manylinux_2_28_aarch64.whl (from https://pypi.org/pypi/mssql-python/1.8.0/json). The manylinux_2_28 profile corresponds to glibc 2.28 / RHEL 8 / UBI 8 (PEP 600). RHEL 8 also ships OpenSSL 1.1, not OpenSSL 3. The bundled mssql_py_core .so requires symbols newer than that profile allows, so pip happily installs a wheel that cannot load on the platform it claims to support.

That said, I haven't run auditwheel show against the wheel myself, so it's possible the higher requirement comes from a different .so in the bundle and I've misattributed it. The loader error above is what users will hit either way.

4. Error-message UX (minor, separate)

In mssql_python/cursor.py (1.8.0), the bulk-copy path does:

try:
    import mssql_py_core
except ImportError as exc:
    logger.error("_bulkcopy: Failed to import mssql_py_core module")
    raise ImportError(
        "Bulk copy requires the mssql_py_core library which is not available. "
        "This is an unexpected error. "
    ) from exc

The from exc does preserve the real libssl.so.3: cannot open shared object file in the chained traceback, but the top-line message ("not available", "unexpected error") leads users to think the package is missing rather than failing to load. Surfacing str(exc) in the re-raised message would have saved both the Reddit user and me a fair amount of guessing.

Expected behavior

One of:

• The aarch64 wheel content is rebuilt against the manylinux_2_28 sysroot (glibc 2.28, OpenSSL 1.1) so it actually loads on the platforms the tag advertises, or
• The wheel is re-tagged (e.g. manylinux_2_34_aarch64) so pip declines to install it on RHEL 8 / UBI 8 / Amazon Linux 2 and falls back to a source install or an error users can act on.

It would also be worth checking the manylinux_2_28_x86_64 wheel against the same criteria — I only verified aarch64 on this host, so I can't say whether x86_64 has the same mismatch.

Suggestions (lower priority)

• Include str(exc) in the re-raised ImportError in cursor.py, so the missing .so dependency surfaces in the top-line message.
• Consider adding auditwheel show / repair to the release pipeline for the Linux wheels to catch tag/content mismatches before publish.

Sources

• Reddit report: https://www.reddit.com/r/SQLServer/comments/1trcsy7/comment/op95k4b/
• 1.8.0 wheel list & filenames: https://pypi.org/pypi/mssql-python/1.8.0/json
• Bulk-copy code path: mssql_python/cursor.py around the import mssql_py_core block in the 1.8.0 release
• manylinux profile / glibc mapping: PEP 600 (https://peps.python.org/pep-0600/)
• All ldd, ldconfig, and dist-info/RECORD output above was produced on the RHEL 8.10 aarch64 host described in the Environment section.

[github-actions]

Hi @dlevy-msft-sql, thank you for opening this issue!

Our team will review it shortly. We aim to triage all new issues within 24-48 hours and get back to you.

If you have additional information to share, please feel free to update the issue.

Thank you for your patience!

[bewithgaurav]

thanks for the detailed write-up @dlevy-msft-sql, repro confirmed.

what I found

• spun up an AlmaLinux 8.10 aarch64 container (glibc 2.28, OpenSSL 1.1.1k), hit the exact same libssl.so.3 / GLIBC_2.29..2.34 errors from ldd
• the mssql_py_core .so inside the wheel needs glibc 2.34 + OpenSSL 3, but the wheel is tagged manylinux_2_28
• ddbc_bindings (the main driver extension) is fine, only needs glibc 2.17. the mismatch is specific to mssql_py_core
• confirmed x86_64 wheels have the same mismatch, not just aarch64

context

this is related to #546 and #566. in v1.6.0 the wheels were tagged manylinux_2_34 (honest, but broke RHEL 8 installs entirely). the fix re-tagged to manylinux_2_28 which brought back the core driver on RHEL 8, but mssql_py_core was never rebuilt to match. so bulk copy on glibc < 2.34 has never actually worked, it went from "pip refuses to install" to "installs but bulk copy fails at runtime".

not a regression in the traditional sense (bulk copy on RHEL 8 was never functional), but it is an incomplete fix from the #546 work.

we're evaluating the right fix path here. will post updates as we go.

[saurabh500]

mssql-py-core is not designed to work on GLIBC 2.28 and OpenSSL 1.x

This is by design and is not merely a bug fix. The change is non-trivial.
This issue is by design.

[bewithgaurav]

@saurabh500 - thanks, understood that this is by design
I'm checking on an alternative approach for this and will come back on this thread once done