Add OpenVINO backend to pip wheel for Linux x86_64 (#18309)#18309
Open
shoumikhin wants to merge 1 commit intopytorch:mainfrom
Open
Add OpenVINO backend to pip wheel for Linux x86_64 (#18309)#18309shoumikhin wants to merge 1 commit intopytorch:mainfrom
shoumikhin wants to merge 1 commit intopytorch:mainfrom
Conversation
shoumikhin
requested review from
kirklandsign,
larryliu0820 and
mergennachin
as code owners
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18309
Note: Links to docs will display an error until the docs builds have been completed. ❌ 187 New Failures, 41 PendingAs of commit 727ca1d with merge base 569cf41 ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
Contributor
|
@shoumikhin has exported this pull request. If you are a Meta employee, you can view the originating Diff in D97202714. |
This PR needs a
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the OpenVINO backend to be included in the Linux x86_64 pip wheel by switching the runtime to dlopen/dlsym against the OpenVINO C API, removing any build-time dependency on the OpenVINO SDK.
Changes:
- Reworked the OpenVINO runtime backend to dynamically load
libopenvino_c.soand call into the C API via function pointers. - Enabled building and wheel packaging for the OpenVINO backend on Linux x86_64, plus CI wheel checks and an
executorch[openvino]optional dependency. - Added/updated documentation describing pip-wheel usage and runtime installation.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/cmake/preset/pybind.cmake | Adds an overridable preset option for EXECUTORCH_BUILD_OPENVINO. |
| setup.py | Turns on OpenVINO backend build for Linux x86_64 and builds the openvino_backend target when enabled. |
| pyproject.toml | Adds an openvino optional dependency extra with Linux x86_64 marker. |
| docs/source/build-run-openvino.md | Adds pip-wheel quick start instructions for Linux x86_64. |
| backends/openvino/runtime/OpenvinoBackend.h | Switches backend handle/types to C API and adds lazy-loader state. |
| backends/openvino/runtime/OpenvinoBackend.cpp | Implements runtime dlopen/dlsym loading and C API execution path. |
| backends/openvino/runtime/OpenvinoApi.h | Introduces OpenVINO C ABI forward decls + function pointer table. |
| backends/openvino/README.md | Adds pip-wheel quick start + env var guidance. |
| backends/openvino/CMakeLists.txt | Removes find_package(OpenVINO) and links only executorch_core + dl libs. |
| README-wheel.md | Documents that Linux x86_64 wheels include QNN + OpenVINO (runtime required). |
| .ci/scripts/wheel/test_linux.py | Asserts OpenvinoBackend is registered on Linux x86_64 wheels. |
| .ci/scripts/wheel/pre_build_script.sh | Installs OpenVINO runtime in Linux x86_64 wheel test environment. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
meta-codesync
bot
changed the title
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Mar 19, 2026
Summary: Rewrite the OpenVINO backend runtime to use dlopen/dlsym with the OpenVINO C API, eliminating the build-time dependency on the OpenVINO SDK. This follows the QNN backend's proven pattern: the backend links statically into _portable_lib.so with zero external library dependencies and loads libopenvino_c.so at runtime via dlopen when the user has the openvino pip package installed. Key design decisions: - Use OpenVINO C API (openvino/c/openvino.h) instead of C++ API to enable dlopen with dlsym function pointers. The C++ class-based API (ov::Core, ov::InferRequest) cannot be resolved via dlsym. - Forward-declare OpenVINO C types in OpenvinoApi.h instead of including headers, so there is no build-time SDK dependency at all. - Thread-safe lazy loading via std::call_once in ensure_loaded(). - OPENVINO_LIB_PATH env var for explicit library path override. - Backend always registers via static initializer (zero cost); actual OpenVINO loading deferred to first is_available()/init() call. - No changes to portable_lib.py needed — no RTLD_GLOBAL, no separate .so, no auditwheel workarounds. The backend is statically linked into _portable_lib.so like XNNPACK and has no NEEDED entries for any OpenVINO library. - Enable build on Linux x86_64 unconditionally in setup.py since there is no build-time dependency. User experience: pip install executorch # XNNPACK+QNN+OpenVINO registered pip install executorch[openvino] # + openvino runtime, backend activates Differential Revision: D97202714
shoumikhin
force-pushed
the
export-D97202714
branch
from
d6de3b1 to
e6675bc
Compare
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Mar 19, 2026
Summary: Rewrite the OpenVINO backend runtime to use dlopen/dlsym with the OpenVINO C API, eliminating the build-time dependency on the OpenVINO SDK. This follows the QNN backend's proven pattern: the backend links statically into _portable_lib.so with zero external library dependencies and loads libopenvino_c.so at runtime via dlopen when the user has the openvino pip package installed. Key design decisions: - Use OpenVINO C API (openvino/c/openvino.h) instead of C++ API to enable dlopen with dlsym function pointers. The C++ class-based API (ov::Core, ov::InferRequest) cannot be resolved via dlsym. - Forward-declare OpenVINO C types in OpenvinoApi.h instead of including headers, so there is no build-time SDK dependency at all. - Thread-safe lazy loading via std::call_once in ensure_loaded(). - OPENVINO_LIB_PATH env var for explicit library path override. - Backend always registers via static initializer (zero cost); actual OpenVINO loading deferred to first is_available()/init() call. - No changes to portable_lib.py needed — no RTLD_GLOBAL, no separate .so, no auditwheel workarounds. The backend is statically linked into _portable_lib.so like XNNPACK and has no NEEDED entries for any OpenVINO library. - Enable build on Linux x86_64 unconditionally in setup.py since there is no build-time dependency. User experience: pip install executorch # XNNPACK+QNN+OpenVINO registered pip install executorch[openvino] # + openvino runtime, backend activates Differential Revision: D97202714
shoumikhin
force-pushed
the
export-D97202714
branch
from
e6675bc to
48541fc
Compare
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Mar 19, 2026
Summary: Rewrite the OpenVINO backend runtime to use dlopen/dlsym with the OpenVINO C API, eliminating the build-time dependency on the OpenVINO SDK. This follows the QNN backend's proven pattern: the backend links statically into _portable_lib.so with zero external library dependencies and loads libopenvino_c.so at runtime via dlopen when the user has the openvino pip package installed. Key design decisions: - Use OpenVINO C API (openvino/c/openvino.h) instead of C++ API to enable dlopen with dlsym function pointers. The C++ class-based API (ov::Core, ov::InferRequest) cannot be resolved via dlsym. - Forward-declare OpenVINO C types in OpenvinoApi.h instead of including headers, so there is no build-time SDK dependency at all. - Thread-safe lazy loading via std::call_once in ensure_loaded(). - OPENVINO_LIB_PATH env var for explicit library path override. - Backend always registers via static initializer (zero cost); actual OpenVINO loading deferred to first is_available()/init() call. - No changes to portable_lib.py needed — no RTLD_GLOBAL, no separate .so, no auditwheel workarounds. The backend is statically linked into _portable_lib.so like XNNPACK and has no NEEDED entries for any OpenVINO library. - Enable build on Linux x86_64 unconditionally in setup.py since there is no build-time dependency. User experience: pip install executorch # XNNPACK+QNN+OpenVINO registered pip install executorch[openvino] # + openvino runtime, backend activates Differential Revision: D97202714
shoumikhin
force-pushed
the
export-D97202714
branch
from
48541fc to
4c53cb6
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Refactors the OpenVINO runtime backend to load the OpenVINO C API dynamically at runtime (dlopen/dlsym), enabling inclusion of the OpenVINO backend in the Linux x86_64 pip wheel without a build-time OpenVINO SDK dependency.
Changes:
- Reworked OpenVINO runtime to use OpenVINO C API function pointers with thread-safe lazy loading.
- Updated build/packaging to enable building the OpenVINO backend into the Linux x86_64 wheel and added an
executorch[openvino]optional dependency. - Updated docs and CI wheel checks to reflect/verify OpenVINO backend presence and runtime installation.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/cmake/preset/pybind.cmake | Adds an overridable preset option for building OpenVINO in pybind builds. |
| setup.py | Enables OpenVINO build for Linux x86_64 wheels and adds the build target when enabled. |
| pyproject.toml | Adds executorch[openvino] optional dependency with Linux x86_64 marker. |
| docs/source/build-run-openvino.md | Adds pip-wheel Quick Start instructions for Linux x86_64. |
| backends/openvino/runtime/OpenvinoBackend.h | Updates backend interface to use C API types and lazy-loading state. |
| backends/openvino/runtime/OpenvinoBackend.cpp | Implements dlopen/dlsym loading and rewrites init/execute to use the C API. |
| backends/openvino/runtime/OpenvinoApi.h | Introduces forward declarations + function pointer table for the OpenVINO C API. |
| backends/openvino/README.md | Documents pip-wheel usage and runtime library path configuration. |
| backends/openvino/CMakeLists.txt | Removes build-time OpenVINO SDK dependency; links against libdl instead. |
| README-wheel.md | Documents inclusion of QNN/OpenVINO backends in Linux x86_64 wheels. |
| .ci/scripts/wheel/test_linux.py | Verifies OpenvinoBackend is registered in Linux x86_64 wheel tests. |
| .ci/scripts/wheel/pre_build_script.sh | Installs OpenVINO runtime package on Linux x86_64 for wheel testing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
pyproject.toml
Outdated
|
|
||
| [project.optional-dependencies] | ||
| openvino = [ | ||
| "openvino>=2025.1.0,<2026.0.0; platform_system == 'Linux' and platform_machine == 'x86_64'", |
Comment on lines
+54
to
+57
| ### Building from Source | ||
|
|
||
| If you need a custom build (different platform, custom configuration, etc.), follow the instructions below. | ||
|
|
Summary: Rewrite the OpenVINO backend runtime to use dlopen/dlsym with the OpenVINO C API, eliminating the build-time dependency on the OpenVINO SDK. This follows the QNN backend's proven pattern: the backend links statically into _portable_lib.so with zero external library dependencies and loads libopenvino_c.so at runtime via dlopen when the user has the openvino pip package installed. Key design decisions: - Use OpenVINO C API (openvino/c/openvino.h) instead of C++ API to enable dlopen with dlsym function pointers. The C++ class-based API (ov::Core, ov::InferRequest) cannot be resolved via dlsym. - Forward-declare OpenVINO C types in OpenvinoApi.h instead of including headers, so there is no build-time SDK dependency at all. - Thread-safe lazy loading via std::call_once in ensure_loaded(). - OPENVINO_LIB_PATH env var for explicit library path override. - Backend always registers via static initializer (zero cost); actual OpenVINO loading deferred to first is_available()/init() call. - No changes to portable_lib.py needed — no RTLD_GLOBAL, no separate .so, no auditwheel workarounds. The backend is statically linked into _portable_lib.so like XNNPACK and has no NEEDED entries for any OpenVINO library. - Enabled on all Linux architectures (x86_64 and aarch64) since the backend has no build-time dependency and the openvino pip package supports both. ExecuTorch builds wheels for both platforms. Robustness: - LOAD_SYM macro zeros ov_ struct on partial failure to prevent dangling function pointers. - Bounds check on execute() args (ET_CHECK_OR_RETURN_ERROR). - Allocator null check with proper cleanup on failure. - Device string safely constructed from CompileSpec buffer with trailing null byte stripping. - Tensor rank validated against 1024 upper bound. - create_ov_tensor uses RAII (unique_ptr) for heap dims allocation. - Detailed comments on OpenVINO shared_ptr tensor ownership and destroy() lifecycle constraints. User experience: pip install executorch # XNNPACK+QNN+OpenVINO registered pip install executorch[openvino] # + openvino runtime, backend activates Differential Revision: D97202714
shoumikhin
force-pushed
the
export-D97202714
branch
from
4c53cb6 to
727ca1d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Rewrite the OpenVINO backend runtime to use dlopen/dlsym with the
OpenVINO C API, eliminating the build-time dependency on the OpenVINO
SDK. This follows the QNN backend's proven pattern: the backend links
statically into _portable_lib.so with zero external library dependencies
and loads libopenvino_c.so at runtime via dlopen when the user has the
openvino pip package installed.
Key design decisions:
enable dlopen with dlsym function pointers. The C++ class-based API
(ov::Core, ov::InferRequest) cannot be resolved via dlsym.
including headers, so there is no build-time SDK dependency at all.
OpenVINO loading deferred to first is_available()/init() call.
.so, no auditwheel workarounds. The backend is statically linked
into _portable_lib.so like XNNPACK and has no NEEDED entries for
any OpenVINO library.
there is no build-time dependency.
User experience:
pip install executorch # XNNPACK+QNN+OpenVINO registered
pip install executorch[openvino] # + openvino runtime, backend activates
Differential Revision: D97202714