WIP: Rename stdlib _queue init if gevent present#192
Closed
jayvdb wants to merge 21 commits intoindygreg:mainfrom
Closed
WIP: Rename stdlib _queue init if gevent present#192jayvdb wants to merge 21 commits intoindygreg:mainfrom
jayvdb wants to merge 21 commits intoindygreg:mainfrom
Conversation
Owner
|
@jayvdb could you please provide more context on why this is needed? I just want to understand why we need to do this... |
Contributor
Author
|
@indygreg this PR is irrelevant until #183 is merged. And the problem this PR fixes is documented fairly well on that PR, but it is only a workaround until rust crate |
jayvdb
changed the title
This causes the .obj files to be unparsable by editbin, dumpbin, nm, objcopy, goblin and object. Related to indygreg#169
Built extensions in packages often have common names like speedups, utils, _objects, cpython, etc. which reside inside the package namespace. The compiled extensions each have a PyInit_<module> which needs to be renamed to PyInit_<pkg>_<module> to avoid clashes when combined into a static binary. Fixes indygreg#169
Python import of simplejson fails with a UnicodeDecodeError on macOS-10.14 when its PyInit symbol is renamed.
cffi includes a near replica of the CPython 3.7's libffi on MSVC, which needs to be discard to avoid duplicate syms.
Replace pip install --target with venvs. As setuptools breakes out of the venv to load distutils build_ext command, first create a copy of the Python distribution and hack it. This also allows venv's to be populated manually outside of PyOxidizer's config, while retaining the ability for PyOxidizer to pick up and include any built object files. To facilitate that, the PyOxidizer state dir where objects are stored is a constant in the distutils compiler modules and in the Rust source. Also add venv_path to PipRequirementsFile, allowing the same venv to be incrementally populated in multiple rules, and the venv re-used across PyOxidizer build runs. Fixes indygreg#162 Fixes indygreg#170 Closes indygreg#194
Unsure why this is necessary; as it wasnt needed before
These libs are already in the static python3 binary so shouldnt be linked in shared libraries used during build dependencies.
The Windows standalone distribution doesnt include venv and presents other difficulties when running setup.py in the venv's due to various build dependencies fail.
cffi includes a near replica of the CPython 3.7's libffi on MSVC, which needs to be discard to avoid duplicate syms.
It isnt being used at the moment, as it doesnt align with a venv based solution, and pip.rs doesnt yet support the setup.py rule which is critical for working around various problems.
greenlet includes a pre-built .obj that is not re-built during the package build, and needs to be included in the static build process.
gevent._queue conflicts with stdlib _queue. One of their PyInit__queue symbols needs to be renamed. Due to missing features in the rust crate `object`, the gevent objects can not be processed. So the stdlib _queuemodule object is modified instead.
Need unreleased fix for COFF.
This is not needed for gevent, as we are rewriting the stdlib symbol, however it is a preventative measure for other symbol clashes which might require rewriting.
Specifially needed to workaround multiple __real@ caused by using rust crate object processing of stdlib queuemodule.obj. It also solves conflicts like stdlib ffi.obj vs cffi libffi.
Contributor
Author
|
This special case has been fixed upstream in gimli-rs/object#159 , so doesnt need a workaround here. Just need to wait for the next release of |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This depends on #183