Introduce stronger types for link-local addresses and unnumbered peers by jgallagher · Pull Request #10082 · oxidecomputer/omicron

jgallagher · 2026-03-18T01:27:29Z

This is a big chunk of #9832. Stealing from the doc comments in sled-agent/types/versions/src/stronger_bgp_unnumbered_types/early_networking.rs, the primary changes in this PR are:

Introduce SpecifiedIpNet, a newtype wrapper around IpNet that does not allow unspecified IP addresses.
Introduce SpecifiedIpAddr, a newtype wrapper around IpAddr that does not allow unspecified IP addresses.
Introduce UplinkAddress, a stronger type for specifying possibly-link-local IP nets. This is the new type of UplinkAddressConfig::address, which was previously an Option<IpNet> where both None and Some(UNSPECIFIED) were treated as link-local.
Introduce RouterPeerAddress, a stronger type for specifying possibly-unnumbered BGP peer addresses. This is the new type of BgpPeerConfig::addr, which was previously an IpAddr where an unspecified address was treated as unnumbered.

The rest of the changes are fallout from those: adding new types for any the that contains UplinkAddressConfig or BgpPeerConfig, since those changed, and updating all the places that create or consume any of those types. I'm hoping this PR is pretty straightforward to review despite its size, because much of the size is either tests or all the noise of redefining a bunch of big structs with a single field changed.

The two main things I'd consider part of #9832 that are NOT addressed in this PR:

Database representation; the columns where we store these values still allow NULL, 0.0.0.0, or ::. Fixing this will require a db migration, so I want to do that separately.
I didn't touch the external API. My sense is that it would be good to apply these stronger types there too, but I'll defer to @internet-diglett or @rcgoodfellow for that - I'm happy to do the work if it should be done.

A third thing we could consider is whether to push this stronger typing down to maghemite too.

For now, in all these cases we convert to or from the stronger types primarily through obnoxiously-long method names that should stick out like sore thumbs (RouterPeerAddress::from_optional_ip_treating_unspecified_as_unnumbered() and friends). This should make it obvious where we're switching from strong types to weaker or vice versa.

There are a couple of breaking changes in how we specify RSS configs. I'll leave a couple comments below with more details.

…red-types-1

jgallagher · 2026-03-18T01:31:22Z

wicket-common/src/rack_setup.rs

+impl UserSpecifiedUplinkAddressConfig {
+    /// String representation for [`UplinkAddress::LinkLocal`] when
+    /// serializing/deserializing [`UserSpecifiedUplinkAddressConfig`].
+    pub const LINK_LOCAL: &str = "link-local";


This is the first breaking change w.r.t. RSS. In the RSS config files passed to wicket, specifying a link-local uplink address now requires:

address = "link-local"

instead of passing "0.0.0.0" or "::"; both of those will now be rejected by wicket (with an error message suggesting "link-local" instead).

There's a similar change below for BGP peer addresses, where we're now required to say

addr = "unnumbered"

instead of "0.0.0.0" or "::" or omitting it entirely (all of which are similarly rejected with an error message suggesting "unnumbered").

(cc @taspelund who suggested "unnumbered" specifically)

One clarification on "link-local": do we support (or plan to support) explicit link-local uplink addresses? Or do/will we only support auto-derived link-local uplink addresses?

It seems like the current use of "link-local" is meant to represent an auto derived address, but the terminology seems like it would be slightly at odds with an explicit link-local address.
i.e.
"link-local" makes it seem like the only alternative is a routable address. But if we allow someone to configure a specific link-local address (e.g. address = fe80::beef) then the "link-local" string seems less meaningful since it doesn't quite capture the full nuance of it being both link-local and auto derived.

Hopefully I'm making sense and not just rambling

A good question but not for me 😅. A couple options here in terms of this representation:

UplinkAddress::Address { .. } could disallow link-local addresses in the inner IpNet just like it disallows 0.0.0.0/n and ::/n on this branch, although I'm not sure what to name the inner type if we do that; SpecifiedIpNet seems okay for "IpNet that isn't using an unspecified addr", but I don't know what to call "IpNet that isn't using an unspecified addr or a link-local address"

UplinkAddress::Address { .. } should allow explicit link-local addresses, and we rename UplinkAddress::LinkLocal to something like UplinkAddress::AutoLinkLocal

Is one of those correct in terms of what we want to support?

do we support (or plan to support) explicit link-local uplink addresses? Or do/will we only support auto-derived link-local uplink addresses?

Not sure I'm the correct person to answer this either, but looking at illumos today it would seem that we do not support explicit link local addressing, since the addrconf address type does not appear to allow an explicit address, and if you create a static ipv6 it automatically creates an addrconf link local address under the hood.

ipadm create-addr [-t] -T static [-d] -a [local|remote=]addr[/prefixlen]... addrobj Create an address on the specified IP interface using static configuration. The address will be enabled but can disabled using the ipadm disable-addr subcommand. Note that addrconf address configured on the interface is required to configure static IPv6 address on the same interface. This takes the following options: -a,--address Specify the address. The local or remote prefix can be used for a point-to-point interface. In this case, both addresses must be given. Otherwise, the equal sign ("=") should be omitted and the address should be provided by itself without second address. -d,--down The address is down. -t,--temporary Temporary, not persistent across reboots. ... ipadm create-addr [-t] -T addrconf [-i interface-id] [-p {stateful|stateless}={yes|no}]... addrobj Create an auto-configured address on the specified IP interface. This takes the following options: -i,--interface-id Specify the interface ID to be used. -p,--prop Specify which method of auto-configuration should be used. -t,--temporary Temporary, not persistent across reboots.

Something that might be worthwhile to distinguish is that this address configuration is creating a v6 link-local address, because technically there is a v4 link-local address space too (169.254.0.0/16).

jgallagher · 2026-03-18T01:35:53Z

smf/sled-agent/non-gimlet/config-rss.toml

 routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
 # Addresses associated with this port.
-addresses = [{address = "192.168.1.30/24"}]
+addresses = [{address = {type = "address", ip_net = "192.168.1.30/24"}}]


This is the second breaking change w.r.t. RSS configs - in the TOML files that specify the full config (that are only used in development, which does include a4x2), we now have to use the somewhat-more-annoying tagged representation for uplink addresses and BGP peer addresses, since these are now enums instead of Option<IpAddr> or Option<IpNet>.

We could make this use the more flexible parsing we're using now in wicket, but I didn't love that because that would affect our OpenAPI schema for real RSS config handoffs from sled-agent to Nexus, and eventually the external API if we reuse these types there. The wicket representation shows up in the OpenAPI spec as just "string", which happens to have a bunch of rules around it that can't be easily expressed in OpenAPI (e.g., "must be either unnumbered or an IP address other than 0.0.0.0 or ::"). The wicketd OpenAPI spec does have this problem, but it's only used by wicket, not the rest of the control plane or the external API.

We could potentially define different types for "reading config-rss.toml from disk" and "to use in OpenAPI", but that seemed like a bunch of duplication that is maybe even more confusing than the two different types of formatting. Feedback very welcome.

Are we tracking making this change to the lab configs, or customer versions and templates for them?

This change doesn't apply to any of the "real" RSS configs, for labs or customers; the only breaking change for those is #10082 (comment), which only applies to link-local addresses or unnumbered peers ("regular" IPs still parse the same way in the real path). I don't think any of the lab configs use those; I'll ask around about customer templates once this is ready to land.

jgallagher · 2026-03-18T15:22:56Z

sled-agent/types/versions/src/stronger_bgp_unnumbered_types/early_networking.rs

+    /// Associate a VLAN ID with a BGP peer session.
+    #[serde(default)]
+    pub vlan_id: Option<u16>,
+    /// Router lifetime in seconds for unnumbered BGP peers.


Huh, I just noticed this comment. Is router_lifetime only meaningful if addr is an unnumbered peer? If so, should we put it inside the RouterPeerAddress enum?

pub enum RouterPeerAddress { Unnumbered { lifetime: RouterLifetimeConfig }, Numbered { ip: SpecifiedIpAddr }, }

bnaecker

Looks great, thanks for doing all this! I have a few minor bits, but nothing blocking.

bnaecker · 2026-03-18T17:30:07Z

nexus/src/app/background/tasks/sync_switch_configuration.rs

 use sled_agent_types::early_networking::TxEqConfig;
 use sled_agent_types::early_networking::UplinkAddressConfig;
 use sled_agent_types::early_networking::WriteNetworkConfigRequest;
+use sled_agent_types::early_networking::{


Style nit: these are group-imported, but looks like everything else is on its own line. Could we do the same here for consistency?

bnaecker · 2026-03-18T17:40:28Z

sled-agent/types/versions/src/stronger_bgp_unnumbered_types/rack_init.rs

+    pub rack_network_config: RackNetworkConfig,
+
+    /// IPs or subnets allowed to make requests to user-facing services
+    #[serde(default = "default_allowed_source_ips")]


This is totally just idle curiosity, but do we need this default? This is deserialized by converting fallibly from the unvalidated version below, which does have and need that default.

bnaecker · 2026-03-18T17:42:29Z

smf/sled-agent/non-gimlet/config-rss.toml

 routes = [{nexthop = "192.168.1.199", destination = "0.0.0.0/0"}]
 # Addresses associated with this port.
-addresses = [{address = "192.168.1.30/24"}]
+addresses = [{address = {type = "address", ip_net = "192.168.1.30/24"}}]


Are we tracking making this change to the lab configs, or customer versions and templates for them?

jgallagher added 26 commits March 16, 2026 11:08

wip: introduce stronger unnumbered types

319605b

use new stronger types in sled-agent API

79ba415

add more briding helper methods

683343b

helper method for optional IP conversion

ecf4f66

clean up wicket interface

3ce05bb

unused imports

823ed21

fixups from rebase

97bd63e

openapi

ffc8114

address some TODOs

25ca92c

comment cleanup

64da05a

SpecifiedIpNet::addr(): return SpecifiedIpAddr

c696e2f

Merge branch 'main' into john/stronger-unnumbered-types-1

2e9fa2b

better wicket parsing + unit tests

87dcf69

make wicket example config more varied

b4ed2a2

Merge remote-tracking branch 'origin/main' into john/stronger-unnumbe…

40b7264

…red-types-1

fix display of uplink addresses (include subnet)

7aaf046

constants over string literals

7c92647

macro to reduce copy/paste in body -> envelope conversions

8e37371

add test for new early network blob format

4aca700

fix test RSS TOML files

1f0336b

add proptests for specified IP parsing and deserialization

902f7e4

Merge branch 'main' into john/stronger-unnumbered-types-1

cfc3611

add test for UplinkAddress::to_uplinkd_smf_property()

3a44f89

add tests for type conversion round trips

f5449ad

openapi

039ae86

typo

8c72b83

jgallagher requested review from bnaecker and internet-diglett March 18, 2026 01:27

jgallagher commented Mar 18, 2026

View reviewed changes

jgallagher mentioned this pull request Mar 18, 2026

Stronger types for BGP unnumbered peer addressing #9832

Open

bnaecker approved these changes Mar 18, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce stronger types for link-local addresses and unnumbered peers#10082

Introduce stronger types for link-local addresses and unnumbered peers#10082
jgallagher wants to merge 26 commits intomainfrom
john/stronger-unnumbered-types-1

jgallagher commented Mar 18, 2026

Uh oh!

jgallagher Mar 18, 2026

Uh oh!

taspelund Mar 18, 2026 •

edited

Loading

Uh oh!

jgallagher Mar 18, 2026

Uh oh!

internet-diglett Mar 18, 2026

Uh oh!

jgallagher Mar 18, 2026

Uh oh!

bnaecker Mar 18, 2026

Uh oh!

jgallagher Mar 18, 2026

Uh oh!

jgallagher Mar 18, 2026

Uh oh!

bnaecker left a comment

Uh oh!

bnaecker Mar 18, 2026

Uh oh!

bnaecker Mar 18, 2026

Uh oh!

bnaecker Mar 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

jgallagher commented Mar 18, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taspelund Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bnaecker left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

taspelund Mar 18, 2026 •

edited

Loading