PCIe passthrough support on ARM

PCIe passthrough support on ARM

[Edward Cragg]

Hi,

I'm involved in planning a project for which there is a requirement for PCIe
passthrough in KVM on ARMv8. We have no hardware to test on at the moment.

I understand that virtualisation support for ARM is quite young, and it seems
like support is trickling in at the moment for this sort of thing.

Is anyone working on PCIe passthrough on ARM platforms who might be able to
share some insight into what would be required to support this?

Thanks,

Ed

Re: PCIe passthrough support on ARM

[Eric Auger]

Hi Edward,
On 02/04/2016 05:53 PM, Edward Cragg wrote:
> Hi,
> 
> I'm involved in planning a project for which there is a requirement for PCIe
> passthrough in KVM on ARMv8. We have no hardware to test on at the moment.
> 
> I understand that virtualisation support for ARM is quite young, and it seems
> like support is trickling in at the moment for this sort of thing.
> 
> Is anyone working on PCIe passthrough on ARM platforms who might be able to
> share some insight into what would be required to support this?
Yes I am currently working on this topic at the moment and especially on
ARM PCIe passthrough with MSI enabled. I did not test PCIe passthrough
with legacy INTx (MSI disabled) but this should work already with
upstreamed QEMU and kernel.

With MSI enabled, the code is not upstreamed yet.

I have a QEMU series:

[RFC v2 0/8] KVM PCI/MSI passthrough with mach-virt
(https://www.mail-archive.com/qemu-devel@nongnu.org/msg349574.html)

and a kernel series:
[PATCH 00/10] KVM PCIe/MSI passthrough on ARM/ARM64"
  (https://lkml.org/lkml/2016/1/26/371)

This is functional, tested on v8 AMD HW with a GICv2M (single MSI frame).

This is in an early stage, especially on kernel side since lots of
issues still need to be addressed, especially handling proper IRQ isolation.

What kind of MSI controller do you plan to use?

Hope this helps

Best Regards

Eric

> 
> Thanks,
> 
> Ed
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
> 

Re: PCIe passthrough support on ARM

[Edward Cragg]

Hi Eric,

On Thu, Feb 04, 2016 at 06:30:34PM +0100, Eric Auger wrote:
> Hi Edward,
> On 02/04/2016 05:53 PM, Edward Cragg wrote:
> > Hi,
> > 
> > I'm involved in planning a project for which there is a requirement for PCIe
> > passthrough in KVM on ARMv8. We have no hardware to test on at the moment.
> > 
> > I understand that virtualisation support for ARM is quite young, and it seems
> > like support is trickling in at the moment for this sort of thing.
> > 
> > Is anyone working on PCIe passthrough on ARM platforms who might be able to
> > share some insight into what would be required to support this?
> Yes I am currently working on this topic at the moment and especially on
> ARM PCIe passthrough with MSI enabled. I did not test PCIe passthrough
> with legacy INTx (MSI disabled) but this should work already with
> upstreamed QEMU and kernel.
> 
> With MSI enabled, the code is not upstreamed yet.
> 
> I have a QEMU series:
> 
> [RFC v2 0/8] KVM PCI/MSI passthrough with mach-virt
> (https://www.mail-archive.com/qemu-devel@nongnu.org/msg349574.html)
> 
> and a kernel series:
> [PATCH 00/10] KVM PCIe/MSI passthrough on ARM/ARM64"
>   (https://lkml.org/lkml/2016/1/26/371)
> 
> This is functional, tested on v8 AMD HW with a GICv2M (single MSI frame).
> 
> This is in an early stage, especially on kernel side since lots of
> issues still need to be addressed, especially handling proper IRQ isolation.

Thank you, this is good to know.

> What kind of MSI controller do you plan to use?

Do you mean the GIC itself? From registers, it appears to be a standard ARM
GIC, though i'm not sure exactly which one yet. However, it's stated in the
processor's datasheet that legacy interrupts aren't supported. It's one of the
newer Samsung Exynos processors.

Thanks,

Ed

> Hope this helps
> 
> Best Regards
> 
> Eric
> 
> > 
> > Thanks,
> > 
> > Ed
> > _______________________________________________
> > kvmarm mailing list
> > kvmarm@lists.cs.columbia.edu
> > https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
> > 
> 
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Re: PCIe passthrough support on ARM

[Eric Auger]

Hi Edward,
On 02/08/2016 12:13 PM, Edward Cragg wrote:
> Hi Eric,
> 
> On Thu, Feb 04, 2016 at 06:30:34PM +0100, Eric Auger wrote:
>> Hi Edward,
>> On 02/04/2016 05:53 PM, Edward Cragg wrote:
>>> Hi,
>>>
>>> I'm involved in planning a project for which there is a requirement for PCIe
>>> passthrough in KVM on ARMv8. We have no hardware to test on at the moment.
>>>
>>> I understand that virtualisation support for ARM is quite young, and it seems
>>> like support is trickling in at the moment for this sort of thing.
>>>
>>> Is anyone working on PCIe passthrough on ARM platforms who might be able to
>>> share some insight into what would be required to support this?
>> Yes I am currently working on this topic at the moment and especially on
>> ARM PCIe passthrough with MSI enabled. I did not test PCIe passthrough
>> with legacy INTx (MSI disabled) but this should work already with
>> upstreamed QEMU and kernel.
>>
>> With MSI enabled, the code is not upstreamed yet.
>>
>> I have a QEMU series:
>>
>> [RFC v2 0/8] KVM PCI/MSI passthrough with mach-virt
>> (https://www.mail-archive.com/qemu-devel@nongnu.org/msg349574.html)
>>
>> and a kernel series:
>> [PATCH 00/10] KVM PCIe/MSI passthrough on ARM/ARM64"
>>   (https://lkml.org/lkml/2016/1/26/371)
>>
>> This is functional, tested on v8 AMD HW with a GICv2M (single MSI frame).
>>
>> This is in an early stage, especially on kernel side since lots of
>> issues still need to be addressed, especially handling proper IRQ isolation.
> 
> Thank you, this is good to know.
> 
>> What kind of MSI controller do you plan to use?
> 
> Do you mean the GIC itself? From registers, it appears to be a standard ARM
> GIC, though i'm not sure exactly which one yet. However, it's stated in the
> processor's datasheet that legacy interrupts aren't supported. It's one of the
> newer Samsung Exynos processors.
I meant GICv2m (featuring single or multiple MSI frames) or GICv3 ITS or
any other proprietary interrupt controller IP supporting MSIs.

Best Regards

Eric
> 
> Thanks,
> 
> Ed
> 
>> Hope this helps
>>
>> Best Regards
>>
>> Eric
>>
>>>
>>> Thanks,
>>>
>>> Ed
>>> _______________________________________________
>>> kvmarm mailing list
>>> kvmarm@lists.cs.columbia.edu
>>> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>>>
>>
>> _______________________________________________
>> kvmarm mailing list
>> kvmarm@lists.cs.columbia.edu
>> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Re: PCIe passthrough support on ARM

[Edward Cragg]

On Mon, Feb 08, 2016 at 01:56:34PM +0100, Eric Auger wrote:
> > Do you mean the GIC itself? From registers, it appears to be a standard ARM
> > GIC, though i'm not sure exactly which one yet. However, it's stated in the
> > processor's datasheet that legacy interrupts aren't supported. It's one of the
> > newer Samsung Exynos processors.
> I meant GICv2m (featuring single or multiple MSI frames) or GICv3 ITS or
> any other proprietary interrupt controller IP supporting MSIs.

>From the manual, it seems from the reset values for the identification
registers that it is most likely a GICv2.

If we continue with this project, and once we get access to hardware, we'd be
very interested in helping you with testing this.

Thanks,

Ed