TransWikia.com

Could multiple PCIe devices in one IOMMU group be added to different KVM VM separately?

Ask Ubuntu Asked by davyjang on December 4, 2021

I want to know could multiple PCIe devices in one IOMMU group be added to different KVM VM separately?

I have got failed message like "card1 is used by VM1, and card2 is in the card1’s group and added to VM2, then VM2 can’t boot"

I have tried vfio driver, but didn’t work, so what should I do? try SR-IOV? or update KVM-QEMU or update the kernel?

Motherboard: ASUS 390-A(intel supports VT-D)
Host OS: Ubuntu 18.04 with kernel 4.18.0-15
KVM-QEMU: version 2.11.1

One Answer

TL;DR: you shouldn't but can try to fake/force the system into it

Having the devices in the same group is a deficiency of the HW/FW that you use for the use-case that you want to achieve. Guests using the different cards can effectively read/write each others memory (more complex in reality but a nice simplification). That is very much not what you want from virtualization.

There are non upstream accepted patches around this which can help you to fake/force splitting some of these IOMMU groups. But really - please read what it does and understand the associated risks here by the maintainer.

If you really want you can consider compiling your own kernel with the patch and then use these boot parameters which can help then to (fake-)split the devices:

pcie_acs_override=downstream,multifunction

If you have multiple PCIe slots, moving around the cards is a much better (and safer) way to achieve the same.

Answered by Christian Ehrhardt on December 4, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP