Conversation
Collaborator
sk4zuzu
commented
Apr 27, 2026
sk4zuzu
commented
- Normalize lspci_devices's structure
- Add 'unlisted' option for hiding PCI devices from OpenNebula
- Add 'set_name' option for naming PCI devices
- Add 'match_address' test plugin to handle wildcards in PCI/MAC addresses
- Add ability to match NICs by their MAC address
- Add udev rules for net/pci/vfio device renaming/symlinking (set_name)
- Rename 99-vfio.rules to 99-mode.rules in both helper/pci and openvswitch roles
- Remove 'pci_passthrough_enabled'
- Rename 'pci_passthrough_default_filter' to 'pci_default_filter'
- Update precheck role
- Update README.md files
- Normalize lspci_devices's structure - Add 'unlisted' option for hiding PCI devices from OpenNebula - Add 'set_name' option for naming PCI devices - Add 'match_address' test plugin to handle wildcards in PCI/MAC addresses - Add ability to match NICs by their MAC address - Add udev rules for net/pci/vfio device renaming/symlinking (set_name) - Rename 99-vfio.rules to 99-mode.rules in both helper/pci and openvswitch roles - Remove 'pci_passthrough_enabled' - Rename 'pci_passthrough_default_filter' to 'pci_default_filter' - Update precheck role - Update README.md files Signed-off-by: Michal Opala <sk4zuzu@gmail.com>
Imperative top-to-bottom processing should be less confusing to users and allow mixing of address and vendor/device/class queries in predictable ways. Signed-off-by: Michal Opala <sk4zuzu@gmail.com>
Signed-off-by: Michal Opala <sk4zuzu@gmail.com>
Require users to define driver explicitly to prevent later confusion. Signed-off-by: Michal Opala <sk4zuzu@gmail.com>
Ignore devices for which helper/pci role does not manage driver changes. Signed-off-by: Michal Opala <sk4zuzu@gmail.com>
Allow opennebula-ovs.service to pick up renamed devices after reboot. This makes it possible to mix PCI/SR-IOV and OVS/DPDK devices. Signed-off-by: Michal Opala <sk4zuzu@gmail.com>
Signed-off-by: Michal Opala <sk4zuzu@gmail.com>
Collaborator
|
Hi @sk4zuzu. After the last batch of commits I'm no longer able to use the PCI roles. Here is the error That is the result of using the main playbook with this inventory snippet hosts:
sm15:
ansible_host: sm15
pci_devices:
# - address: "0000:01:00.0" TODO: active interface still no rename
# set_driver: omit
# set_name: vmnic0
# - address: "0000:01:00.1"
# set_driver: omit
# set_name: vmnic1
- vendor: "15b3"
device: "*"
class: "0200"
set_driver: omit
set_name: "vf{0[2]}{0[3]}"
- vendor: "15b3"
device: "1015"
class: "0200"
set_driver: omit
set_numvfs: max
set_name: "pf{0[3]}"
# - vendor: "15b3"
# device: "1016"
# class: "0200"
# set_driver: omit
# set_name: "vf{0[2]}{0[3]}"
ovs:
set:
- other_config:dpdk-init: 'false' # DPDK disabled
- other_config:dpdk-socket-mem: '6144'
- other_config:dpdk-hugepage-dir: '/dev/hugepages'
- other_config:pmd-cpu-mask: '0x3'
- other_config:hw-offload: 'true'
- other_config:tc-policy: 'skip_sw'
iface:
ovsbr0:
set:
- mtu_request: 1500
enp1s0f0np0: # TODO: infer name from PCI address
set:
# - type: dpdk
# - options:dpdk-devargs: '0000:01:00.0'
- mtu_request: 9126
enp1s0f1np1: # TODO: infer name from PCI address
set:
# - type: dpdk
# - options:dpdk-devargs: '0000:01:00.1'
- mtu_request: 9126
bond:
bond0:
ifaces: [enp1s0f0np0, enp1s0f1np1]
set:
- bond_mode: balance-slb
br:
ovsbr0:
ports: [bond0]
set:
- datapath_type: netdev
addrs:
- cidr: "{{ ansible_default_ipv4.address ~ '/' ~ ansible_default_ipv4.prefix }}"
metric: 400
gw: "{{ ansible_default_ipv4.gateway }}"
dns: [8.8.8.8, 1.1.1.1]Going back to commit cd607a4 works. Here is the output of said commit |
Collaborator
Author
|
@dann1 Hi, so the error doesn't make much sense, that means I need to debug the actual udev check (I'll come back to you 🤗). Apart from the error there is a logical inconsequence in your inventory:
those are exactly matched as |
Collaborator
Author
|
@dann1 Do you think we should keep the old name as an altname? 🤔 |
Signed-off-by: Michal Opala <sk4zuzu@gmail.com>
Collaborator
Author
Signed-off-by: Michal Opala <sk4zuzu@gmail.com>
Collaborator
|
Perfect, now it works.
Here is the PCI context of this server (sm15) where I'm testing all of the OVS and PCI related changes. These are the physical devices
It isn't a requirement of the feature, but I can see it being useful for reverts and debugging renaming issues. If it is not too problematic it is a good thing to do. |
Collaborator
|
Some more feedback. The timestamps are from last week but the issues happen with the latest change still. Cannot rename interface in useWhen declaring a PCI device by address node:
hosts:
sm15:
ansible_host: sm15
pci_devices:
- address: "0000:01:00.0"
set_driver: omit
set_name: vmnic0and said PCI device is the active network interface used to connect via SSH There is an ansible error Could it be possible to rename the management interface used for the ansible SSH connection ? For the context of the project, these interfaces become OVS interface bonds. In DPDK, we are thinking about setting the desired name directly in the OVS declaration since the interface loses its name. But for regular OVS we would need to rename it and set that name for the As of now the renaming + ovs interfacing works enp1s0f0np0:
set:
# - type: dpdk
# - options:dpdk-devargs: '0000:01:00.0'
- mtu_request: 9126
vmnic1:
set:
# - type: dpdk
# - options:dpdk-devargs: '0000:01:00.1'
- mtu_request: 9126But we would need the primary interface renamed VF renamingHow can I establish names for the virtual functions which don't exist yet ? For example, exposing vfs from these two physical functions yields The PFs have 1015 device and the VFs have 1016. The following inventory snippet fails fails, maybe because 1016 devices do not exist initially Using this pci_devices:
- address: "0000:01:00.1"
set_driver: omit
set_name: vmnic1
- vendor: "15b3"
device: "*"
class: "0200"
set_driver: omit
set_name: "vf{0[2]}{0[3]}"
- vendor: "15b3"
device: "1015"
class: "0200"
set_driver: omit
set_numvfs: max
set_name: "pf{0[3]}"works, since the udev rules are correct. Is this and appropriate way to deal with the use case (Renaming VFs and their PF) ? Just wanting to confirm. Since the PCI device ID of the Virtual Functions can be known beforehand, maybe the assertion of PCI devices existing could be relaxed somehow with a parameter ? - vendor: "15b3"
device: "1016" # TODO: How to deal with VFs that do not exist
exist: promise # <--
class: "0200"
set_driver: omit
set_name: "vf{0[2]}{0[3]}"or try to process this after VFs are enabled. Again, not really a problem since we can wildcard the device and rename the pfs with their naming No VFIO rules when
|
Collaborator
Author
|
@dann1 I came to conclusion coupling helper/pci and openvsiwtch roles has been a mistake all along. So I would have to do something like grep lspci_devices to find PCI address of renamed device defined in OVS config, then exclude by PCI address instead of interface name, but that's too spaghett for my taste (pun intended).. So let's try new approach, we'll allow users to decide and bypass the check via 'unguarded: true' attribute. I think this is less "magical" and actually more instructive. Please test it if you can, then if we agree that's the way let's proceed to other issues. 🙏 😇 |
Collaborator
|
Perfect, this worked pci_devices:
- address: "0000:01:00.0"
set_driver: omit
set_name: vmnic0
unguarded: true
- address: "0000:01:00.1"
set_driver: omit
set_name: vmnic1
.
.
iface:
ovsbr0:
set:
- mtu_request: 1500
vmnic0:
set:
# - type: dpdk
# - options:dpdk-devargs: '0000:01:00.0'
- mtu_request: 9126
vmnic1:
set:
# - type: dpdk
# - options:dpdk-devargs: '0000:01:00.1'
- mtu_request: 9126
bond:
bond0:
ifaces: [vmnic0, vmnic1]
set:
- bond_mode: balance-slbBoth interfaces got renamed and bonded |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.