Does a Virtual PCI Device can have MSI's

Does a Virtual PCI Device can have MSI's

[manish jaggi]

Hi,

I think it should be possible, but confirming it that this feature is
enabled in xen. I don't know how to test it.

Does any virtual PCI device in DomU (I don't mean a virtual function)
have MSI interrupts ? If yes then how is that MSI handled in Xen

-Regards
Manish

Re: Does a Virtual PCI Device can have MSI's

[Stefano Stabellini]

On Wed, 13 Aug 2014, manish jaggi wrote:
> Hi,
> 
> I think it should be possible, but confirming it that this feature is
> enabled in xen. I don't know how to test it.
> 
> Does any virtual PCI device in DomU (I don't mean a virtual function)
> have MSI interrupts ?

Yes, they do.


> If yes then how is that MSI handled in Xen

PV guests on x86 don't receive MSIs or legacy interrupts as is.  They
map them into "pirqs" instead, that are a kind of event channels, Xen
specific software interrupts. For each MSI on the PCI device assigned to
the guest, the guest kernel would ask for a pirq, see:

arch/x86/pci/xen.c:xen_pcifront_enable_irq
arch/x86/pci/xen.c:xen_setup_msi_irqs

In the specific case of MSIs and MSI-X, pcifront issues an hypercall in
order to enable them, see:

drivers/pci/xen-pcifront.c:pci_frontend_enable_msi

and the backend returns the pirq number:

drivers/xen/xen-pciback/pciback_ops.c:xen_pcibk_enable_msi


On ARM I think it would be best if we delivered MSIs as MSIs to the
guest, rather than mapping them into pirqs, to take better advantage of
the hardware. But it would be up to you to change the pcifront/pciback
code to do it.
In first instance it would be fine if we end up using pirqs.

Re: Does a Virtual PCI Device can have MSI's

[manish jaggi]

On 13 August 2014 15:40, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> On Wed, 13 Aug 2014, manish jaggi wrote:
>> Hi,
>>
>> I think it should be possible, but confirming it that this feature is
>> enabled in xen. I don't know how to test it.
>>
>> Does any virtual PCI device in DomU (I don't mean a virtual function)
>> have MSI interrupts ?
>
> Yes, they do.
>
>
>> If yes then how is that MSI handled in Xen
>
> PV guests on x86 don't receive MSIs or legacy interrupts as is.  They
> map them into "pirqs" instead, that are a kind of event channels, Xen
> specific software interrupts. For each MSI on the PCI device assigned to
> the guest, the guest kernel would ask for a pirq, see:
>
> arch/x86/pci/xen.c:xen_pcifront_enable_irq
> arch/x86/pci/xen.c:xen_setup_msi_irqs
>
> In the specific case of MSIs and MSI-X, pcifront issues an hypercall in
> order to enable them, see:
>
> drivers/pci/xen-pcifront.c:pci_frontend_enable_msi
>
> and the backend returns the pirq number:
>
> drivers/xen/xen-pciback/pciback_ops.c:xen_pcibk_enable_msi
>
>
> On ARM I think it would be best if we delivered MSIs as MSIs to the
> guest, rather than mapping them into pirqs, to take better advantage of
> the hardware. But it would be up to you to change the pcifront/pciback
> code to do it.
> In first instance it would be fine if we end up using pirqs.

I am considering 2 cases here
a) physical PCI passthrough devices / functions assigned to domU
b) emulated (virtual) PCI devices assigned to domU

For (a) it is straight to configure and inject the MSI into guest
For (b) how does the configuring and injection should work,
- PCI Front driver using backops requests to enable msi
- At a later stage xen using dom0 (somehow) inject an virtual LPI into domU.

What are your thoughts on this?

Re: Does a Virtual PCI Device can have MSI's

[Stefano Stabellini]

On Wed, 13 Aug 2014, manish jaggi wrote:
> On 13 August 2014 15:40, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > On Wed, 13 Aug 2014, manish jaggi wrote:
> >> Hi,
> >>
> >> I think it should be possible, but confirming it that this feature is
> >> enabled in xen. I don't know how to test it.
> >>
> >> Does any virtual PCI device in DomU (I don't mean a virtual function)
> >> have MSI interrupts ?
> >
> > Yes, they do.
> >
> >
> >> If yes then how is that MSI handled in Xen
> >
> > PV guests on x86 don't receive MSIs or legacy interrupts as is.  They
> > map them into "pirqs" instead, that are a kind of event channels, Xen
> > specific software interrupts. For each MSI on the PCI device assigned to
> > the guest, the guest kernel would ask for a pirq, see:
> >
> > arch/x86/pci/xen.c:xen_pcifront_enable_irq
> > arch/x86/pci/xen.c:xen_setup_msi_irqs
> >
> > In the specific case of MSIs and MSI-X, pcifront issues an hypercall in
> > order to enable them, see:
> >
> > drivers/pci/xen-pcifront.c:pci_frontend_enable_msi
> >
> > and the backend returns the pirq number:
> >
> > drivers/xen/xen-pciback/pciback_ops.c:xen_pcibk_enable_msi
> >
> >
> > On ARM I think it would be best if we delivered MSIs as MSIs to the
> > guest, rather than mapping them into pirqs, to take better advantage of
> > the hardware. But it would be up to you to change the pcifront/pciback
> > code to do it.
> > In first instance it would be fine if we end up using pirqs.
> 
> I am considering 2 cases here
> a) physical PCI passthrough devices / functions assigned to domU

Are you sure you mean DomU here, or maybe you mean Dom0?


> b) emulated (virtual) PCI devices assigned to domU

We need to clarify the terminology here: what do you mean by (b)?
Emulating an entire PCI device and exposing it to domU? Why do you want
to do that? It is not a feature I am keen on having on Xen on ARM.
Otherwise if you are thinking of a virtual function of an SR-IOV card,
that is still (a) from the Xen point of view.


> For (a) it is straight to configure and inject the MSI into guest

Yep, that is what I was trying to say.


> For (b) how does the configuring and injection should work,
> - PCI Front driver using backops requests to enable msi
> - At a later stage xen using dom0 (somehow) inject an virtual LPI into domU.
> 
> What are your thoughts on this?

I am not sure I understand what you mean by (b) anymore. In fact
pcifront is used to deal with PCI passthrough to DomUs, that would be
(a) by your description.

Re: Does a Virtual PCI Device can have MSI's

[manish jaggi]

On 13 August 2014 16:20, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> On Wed, 13 Aug 2014, manish jaggi wrote:
>> On 13 August 2014 15:40, Stefano Stabellini
>> <stefano.stabellini@eu.citrix.com> wrote:
>> > On Wed, 13 Aug 2014, manish jaggi wrote:
>> >> Hi,
>> >>
>> >> I think it should be possible, but confirming it that this feature is
>> >> enabled in xen. I don't know how to test it.
>> >>
>> >> Does any virtual PCI device in DomU (I don't mean a virtual function)
>> >> have MSI interrupts ?
>> >
>> > Yes, they do.
>> >
>> >
>> >> If yes then how is that MSI handled in Xen
>> >
>> > PV guests on x86 don't receive MSIs or legacy interrupts as is.  They
>> > map them into "pirqs" instead, that are a kind of event channels, Xen
>> > specific software interrupts. For each MSI on the PCI device assigned to
>> > the guest, the guest kernel would ask for a pirq, see:
>> >
>> > arch/x86/pci/xen.c:xen_pcifront_enable_irq
>> > arch/x86/pci/xen.c:xen_setup_msi_irqs
>> >
>> > In the specific case of MSIs and MSI-X, pcifront issues an hypercall in
>> > order to enable them, see:
>> >
>> > drivers/pci/xen-pcifront.c:pci_frontend_enable_msi
>> >
>> > and the backend returns the pirq number:
>> >
>> > drivers/xen/xen-pciback/pciback_ops.c:xen_pcibk_enable_msi
>> >
>> >
>> > On ARM I think it would be best if we delivered MSIs as MSIs to the
>> > guest, rather than mapping them into pirqs, to take better advantage of
>> > the hardware. But it would be up to you to change the pcifront/pciback
>> > code to do it.
>> > In first instance it would be fine if we end up using pirqs.
>>
>> I am considering 2 cases here
>> a) physical PCI passthrough devices / functions assigned to domU
>
> Are you sure you mean DomU here, or maybe you mean Dom0?
>
Yes DomU only.
>
>> b) emulated (virtual) PCI devices assigned to domU
>
> We need to clarify the terminology here: what do you mean by (b)?
> Emulating an entire PCI device and exposing it to domU? Why do you want
> to do that? It is not a feature I am keen on having on Xen on ARM.
> Otherwise if you are thinking of a virtual function of an SR-IOV card,
> that is still (a) from the Xen point of view.

I was trying to understand that does Xen support some device like a
sata device which is a virtual one emulated using qemu on a PV domU
and its interrupts are MSIs
>
>
>> For (a) it is straight to configure and inject the MSI into guest
>
> Yep, that is what I was trying to say.
>
>
>> For (b) how does the configuring and injection should work,
>> - PCI Front driver using backops requests to enable msi
>> - At a later stage xen using dom0 (somehow) inject an virtual LPI into domU.
>>
>> What are your thoughts on this?
>
> I am not sure I understand what you mean by (b) anymore. In fact
> pcifront is used to deal with PCI passthrough to DomUs, that would be
> (a) by your description.

Re: Does a Virtual PCI Device can have MSI's

[Stefano Stabellini]

On Wed, 13 Aug 2014, manish jaggi wrote:
> On 13 August 2014 16:20, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > On Wed, 13 Aug 2014, manish jaggi wrote:
> >> On 13 August 2014 15:40, Stefano Stabellini
> >> <stefano.stabellini@eu.citrix.com> wrote:
> >> > On Wed, 13 Aug 2014, manish jaggi wrote:
> >> >> Hi,
> >> >>
> >> >> I think it should be possible, but confirming it that this feature is
> >> >> enabled in xen. I don't know how to test it.
> >> >>
> >> >> Does any virtual PCI device in DomU (I don't mean a virtual function)
> >> >> have MSI interrupts ?
> >> >
> >> > Yes, they do.
> >> >
> >> >
> >> >> If yes then how is that MSI handled in Xen
> >> >
> >> > PV guests on x86 don't receive MSIs or legacy interrupts as is.  They
> >> > map them into "pirqs" instead, that are a kind of event channels, Xen
> >> > specific software interrupts. For each MSI on the PCI device assigned to
> >> > the guest, the guest kernel would ask for a pirq, see:
> >> >
> >> > arch/x86/pci/xen.c:xen_pcifront_enable_irq
> >> > arch/x86/pci/xen.c:xen_setup_msi_irqs
> >> >
> >> > In the specific case of MSIs and MSI-X, pcifront issues an hypercall in
> >> > order to enable them, see:
> >> >
> >> > drivers/pci/xen-pcifront.c:pci_frontend_enable_msi
> >> >
> >> > and the backend returns the pirq number:
> >> >
> >> > drivers/xen/xen-pciback/pciback_ops.c:xen_pcibk_enable_msi
> >> >
> >> >
> >> > On ARM I think it would be best if we delivered MSIs as MSIs to the
> >> > guest, rather than mapping them into pirqs, to take better advantage of
> >> > the hardware. But it would be up to you to change the pcifront/pciback
> >> > code to do it.
> >> > In first instance it would be fine if we end up using pirqs.
> >>
> >> I am considering 2 cases here
> >> a) physical PCI passthrough devices / functions assigned to domU
> >
> > Are you sure you mean DomU here, or maybe you mean Dom0?
> >
> Yes DomU only.
> >
> >> b) emulated (virtual) PCI devices assigned to domU
> >
> > We need to clarify the terminology here: what do you mean by (b)?
> > Emulating an entire PCI device and exposing it to domU? Why do you want
> > to do that? It is not a feature I am keen on having on Xen on ARM.
> > Otherwise if you are thinking of a virtual function of an SR-IOV card,
> > that is still (a) from the Xen point of view.
> 
> I was trying to understand that does Xen support some device like a
> sata device which is a virtual one emulated using qemu on a PV domU
> and its interrupts are MSIs

I wouldn't want to support this use case at all, unless strictly
necessary: emulation is slower and less secure (larger surface of
attack) than PV interfaces.


> >> For (a) it is straight to configure and inject the MSI into guest
> >
> > Yep, that is what I was trying to say.
> >
> >
> >> For (b) how does the configuring and injection should work,
> >> - PCI Front driver using backops requests to enable msi
> >> - At a later stage xen using dom0 (somehow) inject an virtual LPI into domU.
> >>
> >> What are your thoughts on this?
> >
> > I am not sure I understand what you mean by (b) anymore. In fact
> > pcifront is used to deal with PCI passthrough to DomUs, that would be
> > (a) by your description.
> 

Re: Does a Virtual PCI Device can have MSI's

[manish jaggi]

On 13 August 2014 19:13, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> On Wed, 13 Aug 2014, manish jaggi wrote:
>> On 13 August 2014 16:20, Stefano Stabellini
>> <stefano.stabellini@eu.citrix.com> wrote:
>> > On Wed, 13 Aug 2014, manish jaggi wrote:
>> >> On 13 August 2014 15:40, Stefano Stabellini
>> >> <stefano.stabellini@eu.citrix.com> wrote:
>> >> > On Wed, 13 Aug 2014, manish jaggi wrote:
>> >> >> Hi,
>> >> >>
>> >> >> I think it should be possible, but confirming it that this feature is
>> >> >> enabled in xen. I don't know how to test it.
>> >> >>
>> >> >> Does any virtual PCI device in DomU (I don't mean a virtual function)
>> >> >> have MSI interrupts ?
>> >> >
>> >> > Yes, they do.
>> >> >
>> >> >
>> >> >> If yes then how is that MSI handled in Xen
>> >> >
>> >> > PV guests on x86 don't receive MSIs or legacy interrupts as is.  They
>> >> > map them into "pirqs" instead, that are a kind of event channels, Xen
>> >> > specific software interrupts. For each MSI on the PCI device assigned to
>> >> > the guest, the guest kernel would ask for a pirq, see:
>> >> >
>> >> > arch/x86/pci/xen.c:xen_pcifront_enable_irq
>> >> > arch/x86/pci/xen.c:xen_setup_msi_irqs
>> >> >
>> >> > In the specific case of MSIs and MSI-X, pcifront issues an hypercall in
>> >> > order to enable them, see:
>> >> >
>> >> > drivers/pci/xen-pcifront.c:pci_frontend_enable_msi
>> >> >
>> >> > and the backend returns the pirq number:
>> >> >
>> >> > drivers/xen/xen-pciback/pciback_ops.c:xen_pcibk_enable_msi
>> >> >
>> >> >
>> >> > On ARM I think it would be best if we delivered MSIs as MSIs to the
>> >> > guest, rather than mapping them into pirqs, to take better advantage of
>> >> > the hardware. But it would be up to you to change the pcifront/pciback
>> >> > code to do it.
>> >> > In first instance it would be fine if we end up using pirqs.
>> >>
>> >> I am considering 2 cases here
>> >> a) physical PCI passthrough devices / functions assigned to domU
>> >
>> > Are you sure you mean DomU here, or maybe you mean Dom0?
>> >
>> Yes DomU only.
>> >
>> >> b) emulated (virtual) PCI devices assigned to domU
>> >
>> > We need to clarify the terminology here: what do you mean by (b)?
>> > Emulating an entire PCI device and exposing it to domU? Why do you want
>> > to do that? It is not a feature I am keen on having on Xen on ARM.
>> > Otherwise if you are thinking of a virtual function of an SR-IOV card,
>> > that is still (a) from the Xen point of view.
>>
>> I was trying to understand that does Xen support some device like a
>> sata device which is a virtual one emulated using qemu on a PV domU
>> and its interrupts are MSIs
>
> I wouldn't want to support this use case at all, unless strictly
> necessary: emulation is slower and less secure (larger surface of
> attack) than PV interfaces.
>
Thats why I asked this question, if a virtual device is assigned to a
domU how would the MSIs be configured for it, using front-back
communication or using the Linux ITS driver which gets trapped into
Xens ITS driver.
>
>> >> For (a) it is straight to configure and inject the MSI into guest
>> >
>> > Yep, that is what I was trying to say.
>> >
>> >
>> >> For (b) how does the configuring and injection should work,
>> >> - PCI Front driver using backops requests to enable msi
>> >> - At a later stage xen using dom0 (somehow) inject an virtual LPI into domU.
>> >>
>> >> What are your thoughts on this?
>> >
>> > I am not sure I understand what you mean by (b) anymore. In fact
>> > pcifront is used to deal with PCI passthrough to DomUs, that would be
>> > (a) by your description.
>>

Re: Does a Virtual PCI Device can have MSI's

[Stefano Stabellini]

On Wed, 13 Aug 2014, manish jaggi wrote:
> On 13 August 2014 19:13, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > On Wed, 13 Aug 2014, manish jaggi wrote:
> >> On 13 August 2014 16:20, Stefano Stabellini
> >> <stefano.stabellini@eu.citrix.com> wrote:
> >> > On Wed, 13 Aug 2014, manish jaggi wrote:
> >> >> On 13 August 2014 15:40, Stefano Stabellini
> >> >> <stefano.stabellini@eu.citrix.com> wrote:
> >> >> > On Wed, 13 Aug 2014, manish jaggi wrote:
> >> >> >> Hi,
> >> >> >>
> >> >> >> I think it should be possible, but confirming it that this feature is
> >> >> >> enabled in xen. I don't know how to test it.
> >> >> >>
> >> >> >> Does any virtual PCI device in DomU (I don't mean a virtual function)
> >> >> >> have MSI interrupts ?
> >> >> >
> >> >> > Yes, they do.
> >> >> >
> >> >> >
> >> >> >> If yes then how is that MSI handled in Xen
> >> >> >
> >> >> > PV guests on x86 don't receive MSIs or legacy interrupts as is.  They
> >> >> > map them into "pirqs" instead, that are a kind of event channels, Xen
> >> >> > specific software interrupts. For each MSI on the PCI device assigned to
> >> >> > the guest, the guest kernel would ask for a pirq, see:
> >> >> >
> >> >> > arch/x86/pci/xen.c:xen_pcifront_enable_irq
> >> >> > arch/x86/pci/xen.c:xen_setup_msi_irqs
> >> >> >
> >> >> > In the specific case of MSIs and MSI-X, pcifront issues an hypercall in
> >> >> > order to enable them, see:
> >> >> >
> >> >> > drivers/pci/xen-pcifront.c:pci_frontend_enable_msi
> >> >> >
> >> >> > and the backend returns the pirq number:
> >> >> >
> >> >> > drivers/xen/xen-pciback/pciback_ops.c:xen_pcibk_enable_msi
> >> >> >
> >> >> >
> >> >> > On ARM I think it would be best if we delivered MSIs as MSIs to the
> >> >> > guest, rather than mapping them into pirqs, to take better advantage of
> >> >> > the hardware. But it would be up to you to change the pcifront/pciback
> >> >> > code to do it.
> >> >> > In first instance it would be fine if we end up using pirqs.
> >> >>
> >> >> I am considering 2 cases here
> >> >> a) physical PCI passthrough devices / functions assigned to domU
> >> >
> >> > Are you sure you mean DomU here, or maybe you mean Dom0?
> >> >
> >> Yes DomU only.
> >> >
> >> >> b) emulated (virtual) PCI devices assigned to domU
> >> >
> >> > We need to clarify the terminology here: what do you mean by (b)?
> >> > Emulating an entire PCI device and exposing it to domU? Why do you want
> >> > to do that? It is not a feature I am keen on having on Xen on ARM.
> >> > Otherwise if you are thinking of a virtual function of an SR-IOV card,
> >> > that is still (a) from the Xen point of view.
> >>
> >> I was trying to understand that does Xen support some device like a
> >> sata device which is a virtual one emulated using qemu on a PV domU
> >> and its interrupts are MSIs
> >
> > I wouldn't want to support this use case at all, unless strictly
> > necessary: emulation is slower and less secure (larger surface of
> > attack) than PV interfaces.
> >
> Thats why I asked this question, if a virtual device is assigned to a
> domU how would the MSIs be configured for it, using front-back
> communication or using the Linux ITS driver which gets trapped into
> Xens ITS driver.

If it is an emulated device, by definition there is no front-back
communication.


> >> >> For (a) it is straight to configure and inject the MSI into guest
> >> >
> >> > Yep, that is what I was trying to say.
> >> >
> >> >
> >> >> For (b) how does the configuring and injection should work,
> >> >> - PCI Front driver using backops requests to enable msi
> >> >> - At a later stage xen using dom0 (somehow) inject an virtual LPI into domU.
> >> >>
> >> >> What are your thoughts on this?
> >> >
> >> > I am not sure I understand what you mean by (b) anymore. In fact
> >> > pcifront is used to deal with PCI passthrough to DomUs, that would be
> >> > (a) by your description.
> >>
> 

Re: Does a Virtual PCI Device can have MSI's

[manish jaggi]

On 13 August 2014 23:21, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> On Wed, 13 Aug 2014, manish jaggi wrote:
>> On 13 August 2014 19:13, Stefano Stabellini
>> <stefano.stabellini@eu.citrix.com> wrote:
>> > On Wed, 13 Aug 2014, manish jaggi wrote:
>> >> On 13 August 2014 16:20, Stefano Stabellini
>> >> <stefano.stabellini@eu.citrix.com> wrote:
>> >> > On Wed, 13 Aug 2014, manish jaggi wrote:
>> >> >> On 13 August 2014 15:40, Stefano Stabellini
>> >> >> <stefano.stabellini@eu.citrix.com> wrote:
>> >> >> > On Wed, 13 Aug 2014, manish jaggi wrote:
>> >> >> >> Hi,
>> >> >> >>
>> >> >> >> I think it should be possible, but confirming it that this feature is
>> >> >> >> enabled in xen. I don't know how to test it.
>> >> >> >>
>> >> >> >> Does any virtual PCI device in DomU (I don't mean a virtual function)
>> >> >> >> have MSI interrupts ?
>> >> >> >
>> >> >> > Yes, they do.
>> >> >> >
>> >> >> >
>> >> >> >> If yes then how is that MSI handled in Xen
>> >> >> >
>> >> >> > PV guests on x86 don't receive MSIs or legacy interrupts as is.  They
>> >> >> > map them into "pirqs" instead, that are a kind of event channels, Xen
>> >> >> > specific software interrupts. For each MSI on the PCI device assigned to
>> >> >> > the guest, the guest kernel would ask for a pirq, see:
>> >> >> >
>> >> >> > arch/x86/pci/xen.c:xen_pcifront_enable_irq
>> >> >> > arch/x86/pci/xen.c:xen_setup_msi_irqs
>> >> >> >
>> >> >> > In the specific case of MSIs and MSI-X, pcifront issues an hypercall in
>> >> >> > order to enable them, see:
>> >> >> >
>> >> >> > drivers/pci/xen-pcifront.c:pci_frontend_enable_msi
>> >> >> >
>> >> >> > and the backend returns the pirq number:
>> >> >> >
>> >> >> > drivers/xen/xen-pciback/pciback_ops.c:xen_pcibk_enable_msi
>> >> >> >
>> >> >> >
>> >> >> > On ARM I think it would be best if we delivered MSIs as MSIs to the
>> >> >> > guest, rather than mapping them into pirqs, to take better advantage of
>> >> >> > the hardware. But it would be up to you to change the pcifront/pciback
>> >> >> > code to do it.
>> >> >> > In first instance it would be fine if we end up using pirqs.
>> >> >>
>> >> >> I am considering 2 cases here
>> >> >> a) physical PCI passthrough devices / functions assigned to domU
>> >> >
>> >> > Are you sure you mean DomU here, or maybe you mean Dom0?
>> >> >
>> >> Yes DomU only.
>> >> >
>> >> >> b) emulated (virtual) PCI devices assigned to domU
>> >> >
>> >> > We need to clarify the terminology here: what do you mean by (b)?
>> >> > Emulating an entire PCI device and exposing it to domU? Why do you want
>> >> > to do that? It is not a feature I am keen on having on Xen on ARM.
>> >> > Otherwise if you are thinking of a virtual function of an SR-IOV card,
>> >> > that is still (a) from the Xen point of view.
>> >>
>> >> I was trying to understand that does Xen support some device like a
>> >> sata device which is a virtual one emulated using qemu on a PV domU
>> >> and its interrupts are MSIs
>> >
>> > I wouldn't want to support this use case at all, unless strictly
>> > necessary: emulation is slower and less secure (larger surface of
>> > attack) than PV interfaces.
>> >
>> Thats why I asked this question, if a virtual device is assigned to a
>> domU how would the MSIs be configured for it, using front-back
>> communication or using the Linux ITS driver which gets trapped into
>> Xens ITS driver.
>
> If it is an emulated device, by definition there is no front-back
> communication.
>
So if it is not an emulated device and uses PV interfaces
a) Can MSIs be assigned to them
b) If they are,  How MSI are configured for them when their driver
calls (pci_enable_msi), Using front back communication ?
>From another mail thread in which we are discussing the possible
removal of front-back comm for configuring the MSI and rather using
platform ITS driver, In that case how would it work.

>
>> >> >> For (a) it is straight to configure and inject the MSI into guest
>> >> >
>> >> > Yep, that is what I was trying to say.
>> >> >
>> >> >
>> >> >> For (b) how does the configuring and injection should work,
>> >> >> - PCI Front driver using backops requests to enable msi
>> >> >> - At a later stage xen using dom0 (somehow) inject an virtual LPI into domU.
>> >> >>
>> >> >> What are your thoughts on this?
>> >> >
>> >> > I am not sure I understand what you mean by (b) anymore. In fact
>> >> > pcifront is used to deal with PCI passthrough to DomUs, that would be
>> >> > (a) by your description.
>> >>
>>

Re: Does a Virtual PCI Device can have MSI's

[Stefano Stabellini]

On Wed, 13 Aug 2014, manish jaggi wrote:
> On 13 August 2014 23:21, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > On Wed, 13 Aug 2014, manish jaggi wrote:
> >> On 13 August 2014 19:13, Stefano Stabellini
> >> <stefano.stabellini@eu.citrix.com> wrote:
> >> > On Wed, 13 Aug 2014, manish jaggi wrote:
> >> >> On 13 August 2014 16:20, Stefano Stabellini
> >> >> <stefano.stabellini@eu.citrix.com> wrote:
> >> >> > On Wed, 13 Aug 2014, manish jaggi wrote:
> >> >> >> On 13 August 2014 15:40, Stefano Stabellini
> >> >> >> <stefano.stabellini@eu.citrix.com> wrote:
> >> >> >> > On Wed, 13 Aug 2014, manish jaggi wrote:
> >> >> >> >> Hi,
> >> >> >> >>
> >> >> >> >> I think it should be possible, but confirming it that this feature is
> >> >> >> >> enabled in xen. I don't know how to test it.
> >> >> >> >>
> >> >> >> >> Does any virtual PCI device in DomU (I don't mean a virtual function)
> >> >> >> >> have MSI interrupts ?
> >> >> >> >
> >> >> >> > Yes, they do.
> >> >> >> >
> >> >> >> >
> >> >> >> >> If yes then how is that MSI handled in Xen
> >> >> >> >
> >> >> >> > PV guests on x86 don't receive MSIs or legacy interrupts as is.  They
> >> >> >> > map them into "pirqs" instead, that are a kind of event channels, Xen
> >> >> >> > specific software interrupts. For each MSI on the PCI device assigned to
> >> >> >> > the guest, the guest kernel would ask for a pirq, see:
> >> >> >> >
> >> >> >> > arch/x86/pci/xen.c:xen_pcifront_enable_irq
> >> >> >> > arch/x86/pci/xen.c:xen_setup_msi_irqs
> >> >> >> >
> >> >> >> > In the specific case of MSIs and MSI-X, pcifront issues an hypercall in
> >> >> >> > order to enable them, see:
> >> >> >> >
> >> >> >> > drivers/pci/xen-pcifront.c:pci_frontend_enable_msi
> >> >> >> >
> >> >> >> > and the backend returns the pirq number:
> >> >> >> >
> >> >> >> > drivers/xen/xen-pciback/pciback_ops.c:xen_pcibk_enable_msi
> >> >> >> >
> >> >> >> >
> >> >> >> > On ARM I think it would be best if we delivered MSIs as MSIs to the
> >> >> >> > guest, rather than mapping them into pirqs, to take better advantage of
> >> >> >> > the hardware. But it would be up to you to change the pcifront/pciback
> >> >> >> > code to do it.
> >> >> >> > In first instance it would be fine if we end up using pirqs.
> >> >> >>
> >> >> >> I am considering 2 cases here
> >> >> >> a) physical PCI passthrough devices / functions assigned to domU
> >> >> >
> >> >> > Are you sure you mean DomU here, or maybe you mean Dom0?
> >> >> >
> >> >> Yes DomU only.
> >> >> >
> >> >> >> b) emulated (virtual) PCI devices assigned to domU
> >> >> >
> >> >> > We need to clarify the terminology here: what do you mean by (b)?
> >> >> > Emulating an entire PCI device and exposing it to domU? Why do you want
> >> >> > to do that? It is not a feature I am keen on having on Xen on ARM.
> >> >> > Otherwise if you are thinking of a virtual function of an SR-IOV card,
> >> >> > that is still (a) from the Xen point of view.
> >> >>
> >> >> I was trying to understand that does Xen support some device like a
> >> >> sata device which is a virtual one emulated using qemu on a PV domU
> >> >> and its interrupts are MSIs
> >> >
> >> > I wouldn't want to support this use case at all, unless strictly
> >> > necessary: emulation is slower and less secure (larger surface of
> >> > attack) than PV interfaces.
> >> >
> >> Thats why I asked this question, if a virtual device is assigned to a
> >> domU how would the MSIs be configured for it, using front-back
> >> communication or using the Linux ITS driver which gets trapped into
> >> Xens ITS driver.
> >
> > If it is an emulated device, by definition there is no front-back
> > communication.
> >
> So if it is not an emulated device and uses PV interfaces
> a) Can MSIs be assigned to them

No. Only in case of PCI passthrough.


> b) If they are,  How MSI are configured for them when their driver
> calls (pci_enable_msi), Using front back communication ?
> >From another mail thread in which we are discussing the possible
> removal of front-back comm for configuring the MSI and rather using
> platform ITS driver, In that case how would it work.
> 
> >
> >> >> >> For (a) it is straight to configure and inject the MSI into guest
> >> >> >
> >> >> > Yep, that is what I was trying to say.
> >> >> >
> >> >> >
> >> >> >> For (b) how does the configuring and injection should work,
> >> >> >> - PCI Front driver using backops requests to enable msi
> >> >> >> - At a later stage xen using dom0 (somehow) inject an virtual LPI into domU.
> >> >> >>
> >> >> >> What are your thoughts on this?
> >> >> >
> >> >> > I am not sure I understand what you mean by (b) anymore. In fact
> >> >> > pcifront is used to deal with PCI passthrough to DomUs, that would be
> >> >> > (a) by your description.
> >> >>
> >>
>