MSI-X table with vfio (KVM PCIe Passthrough)

0

I am working on project that requires PCIe Passthrough (also referred as virtio-passthrough) with Intel x86 CPU and KVM.

Have an interesting situation where device that is passedthrough in Guest is configuring all entries in vector table correctly. However, if I see from host, that table is populated, but all vectors "Mask Interrupt" bit set. Note that capability register do set value to "Enable" and "Mask set to 0". And that happens in tandem with when Guest programs MSI-X control registers.

Interesting enough, PBA bits are also in sync with what Guest sees. Only MSI-X vector control in vector table is set to "1".

Target offset is 0x0, page size is 4096
mmap(0, 4096, 0x3, 0x1, 3, 0x0)
PCI Memory mapped to address 0x7f5f2d957000.
0x0000: 0x00000000
...
0x000C: 0x00000001
0x0010: 0x00000000
...
0x001C: 0x00000001
0x0020: 0x00000000

PBA:

Target offset is 0x1000, page size is 4096
mmap(0, 4096, 0x3, 0x1, 3, 0x1000)
PCI Memory mapped to address 0x7fc7e890d000.
0x1000: 0x00000003
0x1004: 0x00000000
0x1008: 0xFFFFFFFF

PS: Used pcimem utility to perform MMIO read.

In above case, guest saw device has MSI-X configured, performed DMA and eventually timedout waiting for request to finish (PCIe traces show that it waited for Interrupt).

My thesis is MSI-X interrupt didn't get routed to Guest (via IOMMU of course) because interrupt was masked. However, from guest perspective, it was free to perform DMA. And first request failed.

I need help debug this issue.

I am browsing through a code of [vfio_pci] (https://elixir.bootlin.com/linux/v5.4.61/source/drivers/vfio/pci) hoping to find where that MSI-X address mapping would have done. In particular, attempting to debug how host (KVM) come to know when that table gets programmed so that it finds appropriate address.

Any pointer to direction would be of great help!

pass-through
vfio
asked on Stack Overflow Sep 1, 2020 by reverse_waves • edited Sep 1, 2020 by Bob Arnson

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0