kvm.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [Bug 202055] Failed to PCI passthrough SSD with SMI SM2262 controller.
       [not found] <bug-202055-28872@https.bugzilla.kernel.org/>
@ 2019-04-08  4:16 ` bugzilla-daemon
  2019-04-09 14:43 ` bugzilla-daemon
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 6+ messages in thread
From: bugzilla-daemon @ 2019-04-08  4:16 UTC (permalink / raw)
  To: kvm

https://bugzilla.kernel.org/show_bug.cgi?id=202055

--- Comment #42 from Alex Williamson (alex.williamson@redhat.com) ---
It seems there's a partial workaround available since QEMU v2.12 hiding under
our noses.  That version adds support for relocating the MSI-X vector table on
vfio-pci devices, which recreates the MSI-X MMIO space elsewhere on the device.
 A side-effect of this is that the vector table and PBA are properly sized so
as not to collide.  The size of the tables remains wrong, but this only becomes
a problem if the nvme code attempts to allocate >16 vectors, which requires >15
vCPU (or host CPUs if the device is returned to host drivers after being
assigned)(nvme appears to allocate 1 admin queue, plus a queue per CPU, each
making use of an IRQ vector).  The QEMU vfio-pci device option is
x-msix-relocation= which allows specifying the bar to use for the MSI-X tables,
ex. bar0...bar5.  Since this device uses a 64bit bar0, we can either extend
that BAR or choose another, excluding bar1, which is consumed by the upper half
of bar0.  For instance, I tested with:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
...
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <alias name='ua-sm2262'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00'
function='0x0'/>
    </hostdev>
...
  <qemu:commandline>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.ua-sm2262.x-msix-relocation=bar2'/>
  </qemu:commandline>


(NB: "ua-" is a required prefix when specifying an alias)

A new virtual BAR appears in the guest hosting the MSI-X table and QEMU starts
normally so long as the guest doesn't exceed 15 vCPUs.

The vCPU/pCPU count limitations are obviously not ideal, but hopefully this
provides some degree of workaround for typical configurations.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Bug 202055] Failed to PCI passthrough SSD with SMI SM2262 controller.
       [not found] <bug-202055-28872@https.bugzilla.kernel.org/>
  2019-04-08  4:16 ` [Bug 202055] Failed to PCI passthrough SSD with SMI SM2262 controller bugzilla-daemon
@ 2019-04-09 14:43 ` bugzilla-daemon
  2019-04-09 14:57 ` bugzilla-daemon
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 6+ messages in thread
From: bugzilla-daemon @ 2019-04-09 14:43 UTC (permalink / raw)
  To: kvm

https://bugzilla.kernel.org/show_bug.cgi?id=202055

--- Comment #43 from LimeTech (tomm@lime-technology.com) ---
(In reply to Alex Williamson from comment #42)
> It seems there's a partial workaround available since QEMU v2.12 hiding
> under our noses.  That version adds support for relocating the MSI-X vector
> table on vfio-pci devices, which recreates the MSI-X MMIO space elsewhere on
> the device.  A side-effect of this is that the vector table and PBA are
> properly sized so as not to collide.  The size of the tables remains wrong,
> but this only becomes a problem if the nvme code attempts to allocate >16
> vectors, which requires >15 vCPU (or host CPUs if the device is returned to
> host drivers after being assigned)(nvme appears to allocate 1 admin queue,
> plus a queue per CPU, each making use of an IRQ vector).  The QEMU vfio-pci
> device option is x-msix-relocation= which allows specifying the bar to use
> for the MSI-X tables, ex. bar0...bar5.  Since this device uses a 64bit bar0,
> we can either extend that BAR or choose another, excluding bar1, which is
> consumed by the upper half of bar0.  For instance, I tested with:
> 
> <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
> ...
>     <hostdev mode='subsystem' type='pci' managed='yes'>
>       <source>
>         <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
>       </source>
>       <alias name='ua-sm2262'/>
>       <address type='pci' domain='0x0000' bus='0x02' slot='0x00'
> function='0x0'/>
>     </hostdev>
> ...
>   <qemu:commandline>
>     <qemu:arg value='-set'/>
>     <qemu:arg value='device.ua-sm2262.x-msix-relocation=bar2'/>
>   </qemu:commandline>
> 
> 
> (NB: "ua-" is a required prefix when specifying an alias)
> 
> A new virtual BAR appears in the guest hosting the MSI-X table and QEMU
> starts normally so long as the guest doesn't exceed 15 vCPUs.
> 
> The vCPU/pCPU count limitations are obviously not ideal, but hopefully this
> provides some degree of workaround for typical configurations.

Hi Alex, are you saying the above coding in the VM xml is all that is necessary
(with noted vCPU/pCPU count limitations) to successfully pass-through
sm2262-based controllers, without above kernel patch?  Or is a kernel patch
also necessary (if so, which one)?

thx,
-tom

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Bug 202055] Failed to PCI passthrough SSD with SMI SM2262 controller.
       [not found] <bug-202055-28872@https.bugzilla.kernel.org/>
  2019-04-08  4:16 ` [Bug 202055] Failed to PCI passthrough SSD with SMI SM2262 controller bugzilla-daemon
  2019-04-09 14:43 ` bugzilla-daemon
@ 2019-04-09 14:57 ` bugzilla-daemon
  2020-07-28 15:41 ` bugzilla-daemon
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 6+ messages in thread
From: bugzilla-daemon @ 2019-04-09 14:57 UTC (permalink / raw)
  To: kvm

https://bugzilla.kernel.org/show_bug.cgi?id=202055

--- Comment #44 from Alex Williamson (alex.williamson@redhat.com) ---
(In reply to LimeTech from comment #43)
> 
> Hi Alex, are you saying the above coding in the VM xml is all that is
> necessary (with noted vCPU/pCPU count limitations) to successfully
> pass-through sm2262-based controllers, without above kernel patch?  Or is a
> kernel patch also necessary (if so, which one)?

Yes, without kernel patch.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Bug 202055] Failed to PCI passthrough SSD with SMI SM2262 controller.
       [not found] <bug-202055-28872@https.bugzilla.kernel.org/>
                   ` (2 preceding siblings ...)
  2019-04-09 14:57 ` bugzilla-daemon
@ 2020-07-28 15:41 ` bugzilla-daemon
  2020-07-29 14:34 ` bugzilla-daemon
  2020-12-29  1:00 ` bugzilla-daemon
  5 siblings, 0 replies; 6+ messages in thread
From: bugzilla-daemon @ 2020-07-28 15:41 UTC (permalink / raw)
  To: kvm

https://bugzilla.kernel.org/show_bug.cgi?id=202055

FCL (Felix.leclair123@hotmail.com) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |Felix.leclair123@hotmail.co
                   |                            |m

--- Comment #46 from FCL (Felix.leclair123@hotmail.com) ---
Hey gents! 

Trying to apply this to a proxmox VM, any advice on how to do so? I'm running
on top of the 5.4.x linux kernel. I'm experiencing the same bug as in topic
name, using the SMI 2263 controller. (Adata as above)

Anything I can provide in terms of debugging data? Otherwise, any advice on how
to use the above non kernel patch? also comfortable doing a kernel recompile if
thats the better solution. Will be going to a FreeBSD (FreeNAS 11.3) VM with 4
threads and 72GB of ram alongside an LSI SAS controller. 

Platform is a Dual Xeon 5670 on a HP ml350 G6 mother board. 

Also attached but on a different VM is a Radeon 5600XT, an intel USB controller
and an intel Nic to a windows VM

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Bug 202055] Failed to PCI passthrough SSD with SMI SM2262 controller.
       [not found] <bug-202055-28872@https.bugzilla.kernel.org/>
                   ` (3 preceding siblings ...)
  2020-07-28 15:41 ` bugzilla-daemon
@ 2020-07-29 14:34 ` bugzilla-daemon
  2020-12-29  1:00 ` bugzilla-daemon
  5 siblings, 0 replies; 6+ messages in thread
From: bugzilla-daemon @ 2020-07-29 14:34 UTC (permalink / raw)
  To: kvm

https://bugzilla.kernel.org/show_bug.cgi?id=202055

--- Comment #47 from FCL (Felix.leclair123@hotmail.com) ---
(In reply to FCL from comment #46)
> Hey gents! 
> 
> Trying to apply this to a proxmox VM, any advice on how to do so? I'm
> running on top of the 5.4.x linux kernel. I'm experiencing the same bug as
> in topic name, using the SMI 2263 controller. (Adata as above)
> 
> Anything I can provide in terms of debugging data? Otherwise, any advice on
> how to use the above non kernel patch? also comfortable doing a kernel
> recompile if thats the better solution. Will be going to a FreeBSD (FreeNAS
> 11.3) VM with 4 threads and 72GB of ram alongside an LSI SAS controller. 
> 
> Platform is a Dual Xeon 5670 on a HP ml350 G6 mother board. 
> 
> Also attached but on a different VM is a Radeon 5600XT, an intel USB
> controller and an intel Nic to a windows VM

Found the solution without the need for a kernel recompile. in my Qemu conf
file added:
args: -set device.hostpci1.x-msix-relocation=bar2

Where hostpci1 is 
hostpci1: 08:00.0

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Bug 202055] Failed to PCI passthrough SSD with SMI SM2262 controller.
       [not found] <bug-202055-28872@https.bugzilla.kernel.org/>
                   ` (4 preceding siblings ...)
  2020-07-29 14:34 ` bugzilla-daemon
@ 2020-12-29  1:00 ` bugzilla-daemon
  5 siblings, 0 replies; 6+ messages in thread
From: bugzilla-daemon @ 2020-12-29  1:00 UTC (permalink / raw)
  To: kvm

https://bugzilla.kernel.org/show_bug.cgi?id=202055

plantroon@plantroon.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |plantroon@plantroon.com

--- Comment #48 from plantroon@plantroon.com ---
For Intel SSD 660p series, the latest firmware is required for pass-through to
work without issues. The latest one should be from mid 2020 at the time of
writing this.

Not even any workarounds or special settings for qemu/libvirt are needed, just
a simple pass-through setup as with any other PCIe device.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2020-12-29  1:01 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <bug-202055-28872@https.bugzilla.kernel.org/>
2019-04-08  4:16 ` [Bug 202055] Failed to PCI passthrough SSD with SMI SM2262 controller bugzilla-daemon
2019-04-09 14:43 ` bugzilla-daemon
2019-04-09 14:57 ` bugzilla-daemon
2020-07-28 15:41 ` bugzilla-daemon
2020-07-29 14:34 ` bugzilla-daemon
2020-12-29  1:00 ` bugzilla-daemon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).