Add OSS-Fuzz Atheris fuzzers for core serialization

[skypher]

Summary

Adds an upstream-owned OSS-Fuzz fuzzer suite under ossfuzz/.

Fuzz targets (Atheris):

• DAG serialization/deserialization (serialized_dag_fuzz)
• Connection URI parsing (connection_uri_fuzz)

Each fuzzer includes:

• .options files with tuned input size limits
• .dict files for structured input fuzzing
• Small seed corpora under ossfuzz/seed_corpus/

Security Model Alignment

These fuzzers target code paths with clear security boundaries per Airflow's security model, avoiding the "DAG author trust zone" where DAG authors are expected to run arbitrary code.

Test plan

• Tested locally with atheris (-max_total_time=10)
• OSS-Fuzz integration build validation

[jscheffl]

As we have a large repo, we should maybe put this in an existing subfolder, not top-level. I would propose moving all below the ci/ folder

[potiuk]

Agreed: but we have scripts/ci folder :) - so maybe scripts/ossfuzz ?

[potiuk]

Nice ! Now - just rebase and resolving conflict :)

[skypher]

Nice ! Now - just rebase and resolving conflict :)

Awesome, I think it looks good now!

[potiuk]

Hmm.. the problem we see now - is that atheris does not seem to be well prepared for our environment:

Attempting to install atheris in our image - which is our "gold standard" repeatable "works for me" environment that we use in our CI and when we want to reproduce what happens there locally - fails:

1. Atheris seems like mostly C based library - that has thin python wrapper around - and it does not have Python 3.10 pre-compiled wheels - only 3.11- 3.13. And it fails when it's being build, because of clang compiler missing (and likely it would need some configuration of the image environment to fix it). We could likely overcome it by simply limiting the oss fuzzer to 3.11 - 3.13 though.

2. But more importantlky - there are no ARM wheels. Which means that most of our developers will not be able to run it locally on their M1 macs. This is a bigger issue, because it means that if someone would like to reproduce it locally on Mac, they won't be able to do so - or it will be generally much more brittle - also if they will not use image, this is more likely to fail because their environment is not properly configured for compiling atheris.

I believe you are somewhat connected to Atheris @skypher -> maybe they can simply update their build and release process and produce the binary wheels for all the common platforms - including ARM and MacOS ? Or at the very list make sure that manylinux ARM wheels are available.

See https://pypi.org/project/atheris/3.0.0/#files - there are just three binary wheels, only for AMD

[skypher]

Atheris PR: google/atheris#99

[AidenRHall]

Hey folks, thanks so much for putting this together. We have dropped support for old Python versions because now that the bytecode is changing to much between Python versions it's not realistic to maintain so many versions with the relatively limited engineering cycles we have to devote to this project. You can use older Atheris versions for that however. Adding ARM / Apple Silicon support is something we have on our roadmap and we are hoping to get that up and running this year.

I am curious what problem this PR is trying to solve? Having some CI tests sounds useful but why are we moving OSSFuzz tests into the fuzzing framework itself? There are a ton of atheris fuzzers in OSSFuzz, why is this one being moved specifically?

[skypher]

Hey folks, thanks so much for putting this together. We have dropped support for old Python versions because now that the bytecode is changing to much between Python versions it's not realistic to maintain so many versions with the relatively limited engineering cycles we have to devote to this project. You can use older Atheris versions for that however. Adding ARM / Apple Silicon support is something we have on our roadmap and we are hoping to get that up and running this year.

I am curious what problem this PR is trying to solve? Having some CI tests sounds useful but why are we moving OSSFuzz tests into the fuzzing framework itself? There are a ton of atheris fuzzers in OSSFuzz, why is this one being moved specifically?

Hi @AidenRHall, thanks for the comment!

I think there may be a small misunderstanding - we're not moving anything out of OSS-Fuzz. Airflow doesn't currently have OSS-Fuzz integration at all.

This PR adds new fuzzing harnesses to the Airflow repository itself, following the standard pattern where projects maintain their fuzz targets in-tree. These aren't CI tests - they're fuzz targets intended for continuous fuzzing via OSS-Fuzz. The eventual goal would be to set up a projects/airflow/ configuration in OSS-Fuzz that points to these harnesses.

Does that clarify things?

[potiuk]

And just to add @AidenRHall -> the idea here is that we would also like to experiment with more fuzzing ourselves in Airflow. We generally have approach that we do not add anything in our repo - even if it is going to be run externally by OSSFuzz - so that we can reproduce it locally easily.

We are starting small and we want to add more fuzzing in Airflow And @skypher was kind enough to propose the PR and adding PR that might be usable by OSSFuzz. However, if we are to make a good use of fuzzing and add it in various parts of Airflow, our contributors need to have an easy way of iterating on it - adding new fuzzing, modifying existing one - and this all should be locally runnable. Many of our contributors have Mac ARM devices they are developing Airflow on. Most PMC members and committers in fact. So if we are serious about fuzzing and about getting people involved in making good use of it - we need to make it easy for them to contribute to our fuzzing.

This is the main reason why we also try to use our CI to test it. While Python version is not a blocker (we can easily run it in CI only for Python 3.11+), lack of native ARM wheels is pretty much a blocker - taking into account the time it takes to build Atheris and the environment needed for build to succeed.

I hope that clarifies why ARM support is so important for us.

[potiuk]

BTW. @skypher -> you can get the Python 3.10 failure go away by adding ; python_version >= "3.11" to the dependencies in the added pyproject.toml

[skypher]

BTW. @skypher -> you can get the Python 3.10 failure go away by adding ; python_version >= "3.11" to the dependencies in the added pyproject.toml

Updated, thanks a lot!

[skypher]

Hey folks, thanks so much for putting this together. We have dropped support for old Python versions because now that the bytecode is changing to much between Python versions it's not realistic to maintain so many versions with the relatively limited engineering cycles we have to devote to this project. You can use older Atheris versions for that however. Adding ARM / Apple Silicon support is something we have on our roadmap and we are hoping to get that up and running this year.

Hey again Aiden, just wondering if there's anything we can do to unblock our Atheris PR for these binaries. For your convenience, here's a copy of the link: google/atheris#99

Let us know please :-)

[skypher]

@AidenRHall any update on this? Thanks!

[potiuk]

Would be great to get it in and try it :D

[skypher]

Would be great to get it in and try it :D

Glad to see your ping! What's needed to get it merged? Are we blocked on the Atheris issue or do you think we can proceed as-is? Doesn't seem like they're willing to get this in anytime soon.

[potiuk]

Would be great to get it in and try it :D

Glad to see your ping! What's needed to get it merged? Are we blocked on the Atheris issue or do you think we can proceed as-is? Doesn't seem like they're willing to get this in anytime soon.

I guess if ARM is not supported, we can try it without. That will limit local testing, but well, tough.

[jscheffl]

@skypher This PR has been converted to draft because it does not yet meet our Pull Request quality criteria.

Issues found:

• ❌ Pre-commit / static checks: Failing: CI image checks / Static checks. Run prek run --from-ref main locally to find and fix issues. See Pre-commit / static checks docs.
• ❌ mypy (type checking): Failing: CI image checks / MyPy checks (mypy-dev). Run prek --stage manual mypy-dev --all-files locally to reproduce. You need breeze ci-image build --python 3.10 for Docker-based mypy. See mypy (type checking) docs.

Note: Your branch is 846 commits behind main. Some check failures may be caused by changes in the base branch rather than by your PR. Please rebase your branch and push again to get up-to-date CI results.

What to do next:

• The comment informs you what you need to do.
• Fix each issue, then mark the PR as "Ready for review" in the GitHub UI - but only after making sure that all the issues are fixed.
• Maintainers will then proceed with a normal review.

Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. If you have questions, feel free to ask on the Airflow Slack.

[potiuk]

This draft pull request has had no activity from the author for over 3 weeks.

@skypher, you are welcome to reopen this PR when you are ready to continue working on it. Thank you for your contribution!