Skip to content

Fix DNS resolution performance regression during cloud-init local #6665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
drzee99 wants to merge 31 commits into from
Closed

Fix DNS resolution performance regression during cloud-init local #6665

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
aa4be53
Fix DNS resolution performance regression during cloud-init local
drzee99 Jan 10, 2026
44c23e7
Merge branch 'canonical:main' into fix-dns-resolution-performance
drzee99 Jan 19, 2026
5ec0eae
Fixed linting issues
Jan 19, 2026
470ef62
Merge branch 'canonical:main' into fix-dns-resolution-performance
drzee99 Jan 27, 2026
f860f87
Revert "Fixed linting issues"
drzee99 Jan 27, 2026
7d9192c
Fixed linting issue with correct user id for commit
drzee99 Jan 27, 2026
ef26ce0
Merge branch 'canonical:main' into fix-dns-resolution-performance
drzee99 Feb 2, 2026
2b42b2b
Fixed linting issues
Jan 19, 2026
780d869
Removed wrong commit
drzee99 Feb 2, 2026
79dff5c
docs: fix broken external documentation links (#6664)
ayushigithub12 Jan 12, 2026
11eae30
ci: add reviewdog workflow lint for github actions (#6662)
blackboxsw Jan 13, 2026
2c442e9
test: pytestified test_cc_growpart.py (#6625)
MoeSalah1999 Jan 13, 2026
6d89a5a
feat(lxd): add s390x virtio-ports detection for LXD (#6597)
blackboxsw Jan 15, 2026
98caebf
ci: export python path to GITHUB_ENV from tox venv
blackboxsw Jan 13, 2026
bd2c08a
ci: add actionlint.yml ignores
blackboxsw Jan 14, 2026
77bd696
ci: run and source pylint tox target used by TICS checkers
blackboxsw Jan 15, 2026
5f0e08e
ci: retain system packages for TICS workflow due to virtualenv versio…
blackboxsw Jan 15, 2026
619db91
doc: document socket protocol change
holmanb Jan 17, 2026
e2c5788
fix(cloudstack): Improve domain-name DHCP lease lookup (Cloudstack) (…
CodeBleu Jan 20, 2026
af5308d
fix: cloud-init clean --logs should not remove non-files (#6568)
blackboxsw Jan 23, 2026
4a9fc8a
fix: respect SSH key options for the root user (#6585)
sbraz Jan 23, 2026
55ff943
chore(stages): enable type checking (#6672)
holmanb Jan 23, 2026
a65c6b8
fix: migrate from ntp client package installed from ntp to ntpsec (#6…
blackboxsw Jan 23, 2026
b601c43
doc: clarify verbose language (#6688)
holmanb Jan 26, 2026
5a129a8
Fixed linting issue with correct user id for commit
drzee99 Jan 27, 2026
82e5373
feat(dhcp): enable --debug option for dhcpcd (#6693)
cjp256 Jan 29, 2026
76fa874
doc: clarify CLA check (#6692)
holmanb Jan 30, 2026
2d1f427
ci: add shared workflow for lxd_container integration tests
blackboxsw Jan 27, 2026
745e05a
docs: security company policy updates (#6677)
blackboxsw Jan 31, 2026
816dbeb
fix: datasource initialization order in stages (#6700)
holmanb Jan 31, 2026
c1024f2
Merge remote-tracking branch 'refs/remotes/origin/fix-dns-resolution-…
drzee99 Feb 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion cloudinit/sources/DataSourceEc2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,6 @@ class DataSourceEc2(sources.DataSource):
metadata_urls = [
"http://169.254.169.254",
"http://[fd00:ec2::254]",
"http://instance-data.:8773",
Copy link
Copy Markdown
Member

@holmanb holmanb Jan 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ec2 datasource is used by various other clouds besides just ec2 - and unfortunately not all clouds are known, so this change poses a risk.

Copy link
Copy Markdown
Contributor Author

@drzee99 drzee99 Jan 12, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is confusing. There should be a separate Data Source implementation for each provider. Even if some are same/similar to allow for future changes a cloud provider may implement.

That being said.

It is possible to override the metadata_urls ref: https://cloudinit.readthedocs.io/en/latest/reference/datasources/ec2.html

IMHO the default should be the ones that are provided by the named Data Source (in this case EC2). If this breaks other cloud providers that use the same path, then they should create an config setting for the metadata_url as pr. documentation to add the relevant metadata_urls.

Copy link
Copy Markdown
Member

@holmanb holmanb Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be sure that I understand: You are saying that breaking clouds is justified because the code is confusing and there is a workaround that involves manual modifications to the image, right?

Copy link
Copy Markdown
Contributor Author

@drzee99 drzee99 Jan 26, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The DataSource is named DataSourceEC2.py - EC2 is explicitly referring to Amazon Web Services EC2 service (in fact "Amazon EC2" is a registered trademark). Thus it should IMHO adhere to what ever is standard for Amazon EC2 at the current point in time.

It is fair that other clouds have implemented similar things, but they should either have their own data sources OR there should be a generic data source (DataSourceGeneric,py). They should not rely on that Amazon EC2 keeps doing things the same way. What if EC2 changes fundamentally "tomorrow"?

This change in DataSourceEC2.py is not important and I'm happy to back it out.

The important change to resolve the issue is that the check for IP addresses is made earlier in is_resolveable() so we do not unnecessarily go into the "Detect DNS Redirection check" when we don't even have a proper network and just try to query the metadata service (the result from this query is used to setup the network).

]

# The minimum supported metadata_version from the ec2 metadata apis
Expand Down
10 changes: 6 additions & 4 deletions cloudinit/util.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1295,6 +1295,12 @@ def is_resolvable(url) -> bool:
global _DNS_REDIRECT_IP
parsed_url = parse.urlparse(url)
name = parsed_url.hostname

# Early return for IP addresses - no DNS resolution needed
with suppress(ValueError):
if net.is_ip_address(parsed_url.netloc.strip("[]")):
return True

if _DNS_REDIRECT_IP is None:
badips = set()
badnames = (
Expand All @@ -1319,10 +1325,6 @@ def is_resolvable(url) -> bool:
LOG.debug("detected dns redirection: %s", badresults)

try:
# ip addresses need no resolution
with suppress(ValueError):
if net.is_ip_address(parsed_url.netloc.strip("[]")):
return True
result = socket.getaddrinfo(name, None)
# check first result's sockaddr field
addr = result[0][4][0]
Expand Down