-
Notifications
You must be signed in to change notification settings - Fork 633
TF2.16 support: hermetic python shim, TF 2.16.2 pin, legacy Keras path, setup/packaging fixes #913
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
TF2.16 support: hermetic python shim, TF 2.16.2 pin, legacy Keras path, setup/packaging fixes #913
Conversation
mhucka
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the work, and I'm sorry about the number of changes requested here …
Co-authored-by: Michael Hucka <mhucka@google.com>
Co-authored-by: Michael Hucka <mhucka@google.com>
Co-authored-by: Michael Hucka <mhucka@google.com>
Co-authored-by: Michael Hucka <mhucka@google.com>
Co-authored-by: Michael Hucka <mhucka@google.com>
Co-authored-by: Michael Hucka <mhucka@google.com>
Co-authored-by: Michael Hucka <mhucka@google.com>
|
About the version used here (0.7.5): I put 0.7.5 because in the master branch in releases/setup.py: Line 58 in c6a90ac
It was 0.7.4 As you mentioned the current version on PyPi is 0.7.3... I could patch releases/setup.py and tensorflow_quantum/init.py to match 0.7.4 (init has 0.7.2): quantum/tensorflow_quantum/__init__.py Line 67 in c6a90ac
And maybe this new version could be the tag 0.7.4 right? Or maybe the current version that is in master is the unofficial version 0.7.4 and I should update it to 0.7.5? (to avoid losing 0.7.4 that is in master). |
When `set -e` is used, the `|| exit` parts are unnecessary.
Inside Docker, the user is frequently root. Pip then complains that it's a bad idea to run pip as root. This is irrelevant inside a Docker container.
This adds a dry run option so that unit tests can be written for these scripts. Also, I realized belately that I've been using the wrong indentation settings for shell scripts for this project. TFQ and TF follow the Google style convention of indent = 2, not 4.
These tests can be executed by running ```shell bazel test //release:build_distribution_test bazel test //release:clean_distribution_test ``` for individual tests, or just ```shell bazel test //release:all ``` as part of testing everything. These tests were written with the help of Gemini CLI.
Changes: 1. Add the ability to use `and-tensorflow` as the name of the extra to install TensorFlow. This follows a similar TF's `and-cuda` option, and lets users do `pip install tensorflow-quantum[and-tensorflow]` to get everything. My hope is that this is more intuitive and clear. (Conversely, the previous `pip install tensorflow-quantum[extras]` left me wondering what all the extras were.) The value `extras` still exists and can still be used, to avoid breaking existing workflows. 2. Add tf-keras version 2.16 to the required packages list. TensorFlow 2.16 installs Keras 3 by default, and the user wasn't getting tf-keras unless they did it themselves. This way, installing TFQ will specify Keras 2. 3. Added a `python_requires` for version 3.10 or above, since that's our current cut-off due to constraints from the many dependencies needed.
Moving these scripts to a separate PR in order to avoid putting too many things into the current PR.
These flags are redundant but also not something that necessarily has to removed as part of this PR.
For things run manually, it should be okay to assume Bash and not limit ourselves to POSIX shell features.
This also should be done in a separate PR
Delay removing the `.bazelrc` files until configure.sh is actually ready to start writing a new one. That way, if the user ^C's in the middle of configure.sh running, it's less likely to overwrite what they currently have. (It happened to me: I realized I wanted to stop and check something at the point when it asked for cpu vs gpu, but by then it was too late: the .bazelrc file was gone.)
When output is redirected to logs and this script is invoked as part of a larger script, printing an initial welcome message helps reading and searchign the log.
Per Google guidance, executable scripts should use `/bin/bash`.
Summary
tf-keras) path viaTF_USE_LEGACY_KERAS=1to avoid Keras 3 breakagesrequirements.inand usepip-compileto produce `requirements.txtKey changes
@org_tensorflowto 2.16.2rules_pythonBazel rule setpip_parseandinstall_depsfromrules_pythonto readrequirements.txtand install dependencies@pythonlocal_repository shim to set Python interpreter paththird_party/python_legacy/defs.bzl:configure.sh:--python=flag.tf_configure.bazelrc+.bazelrc+ the Python shim.bazelrcalso import.bazelrc.userif it exists.bazelrc:TF_USE_LEGACY_KERAS=1for build/testrelease/build_pip_package.sh:${PYTHON_BIN_PATH:-python3}and ensure setuptools presentsetup.py:sympy==1.14)tensorflow>=2.16,<2.17requirements.in:requirements.txtis produced by runningpip-compile --allow-unsafescripts/:.bazelrc--cxxopt=-std=c++17--cxxopt=-D_GLIBCXX_USE_CXX11_ABI=1--experimental_repo_remote_execTesting
bazel build ... release:build_pip_packagesucceedsscripts/test_all.sh) passMiscellaneous
Notes