PROBLEM: pci_alloc_irq_vectors function request 32 MSI interrupts vectors, but return 1 in KVM virtual machine.

PROBLEM: pci_alloc_irq_vectors function request 32 MSI interrupts vectors, but return 1 in KVM virtual machine.

[Renjun Wang]

Hi all:
I have a question about PCI which troubled me for a few weeks.
I have a virtual machine with ubuntu 16.4.03  on KVM platform. There is a PCIe device(Xilinx PCIe IP) plugged in the host machine.
On the ubuntu operation system, I am developing the pcie driver. When I use pci_alloc_irq_vectors() function to allocate 32 msi vectors, but return 1.
The command  `lspci -vvv` output shows 
MSI: Enable+ Count=1/32 Maskable+ 64bit+


there is a similar case https://stackoverflow.com/questions/49821599/multiple-msi-vectors-linux-pci-alloc-irq-vectors-return-one-while-the-devi.
But not working for KVM virtual machine.


I do not known why the function  pci_alloc_irq_vectors() returns one ?


Best regards.

Re: PROBLEM: pci_alloc_irq_vectors function request 32 MSI interrupts vectors, but return 1 in KVM virtual machine.

[Keith Busch]

On Wed, Dec 25, 2019 at 01:34:26AM +0000, Renjun Wang wrote:
> I have a virtual machine with ubuntu 16.4.03  on KVM platform. There
> is a PCIe device(Xilinx PCIe IP) plugged in the host machine.
>
> On the ubuntu operation system, I am developing the pcie driver. When
> I use pci_alloc_irq_vectors() function to allocate 32 msi vectors, but
> return 1.
>
> The command  `lspci -vvv` output shows 
> MSI: Enable+ Count=1/32 Maskable+ 64bit+
> 
> 
> there is a similar case https://stackoverflow.com/questions/49821599/multiple-msi-vectors-linux-pci-alloc-irq-vectors-return-one-while-the-devi.
> But not working for KVM virtual machine.
> 
> I do not known why the function  pci_alloc_irq_vectors() returns one ?

Are you setting the "PCI_IRQ_AFFINITY" flag in your alloc call like
in your stackoverflow link? If so, how many CPUs does your virtual
machine have?

回复: PROBLEM: pci_alloc_irq_vectors function request 32 MSI interrupts vectors, but return 1 in KVM virtual machine.

[Renjun Wang]

Hi Keith, thanks for your response.
The flag for pci_alloc_irq_vectors() function is "PCI_IRQ_MSI | PCI_IRQ_AFFINITY", and not include "PCI_IRQ_MISX" which not supported for my PICe device currently. 
The vCPU setting for ubuntu virtual machine is 8 vCPU(1 socket, 4 cores, 2 threads). 
And the kernel configure option "CONFIG_IRQ_REMAP" are enabled for the ubuntu guest and KVM host machine. Is there any other constraint？

-----邮件原件-----
发件人: linux-pci-owner@vger.kernel.org <linux-pci-owner@vger.kernel.org> 代表 Keith Busch
发送时间: 2019年12月27日 0:13
收件人: Renjun Wang <rwang@panyi.ai>
抄送: linux-pci@vger.kernel.org
主题: Re: PROBLEM: pci_alloc_irq_vectors function request 32 MSI interrupts vectors, but return 1 in KVM virtual machine.

On Wed, Dec 25, 2019 at 01:34:26AM +0000, Renjun Wang wrote:
> I have a virtual machine with ubuntu 16.4.03  on KVM platform. There 
> is a PCIe device(Xilinx PCIe IP) plugged in the host machine.
>
> On the ubuntu operation system, I am developing the pcie driver. When 
> I use pci_alloc_irq_vectors() function to allocate 32 msi vectors, but 
> return 1.
>
> The command  `lspci -vvv` output shows
> MSI: Enable+ Count=1/32 Maskable+ 64bit+
> 
> 
> there is a similar case https://stackoverflow.com/questions/49821599/multiple-msi-vectors-linux-pci-alloc-irq-vectors-return-one-while-the-devi.
> But not working for KVM virtual machine.
> 
> I do not known why the function  pci_alloc_irq_vectors() returns one ?

Are you setting the "PCI_IRQ_AFFINITY" flag in your alloc call like in your stackoverflow link? If so, how many CPUs does your virtual machine have?