[SPDK] VFIO/UIO Helper Function?

[SPDK] VFIO/UIO Helper Function?

[Stephen Bates]

[-- Attachment #1: Type: text/plain, Size: 311 bytes --]

Hi All

Does anyone know if there is a helper function in SPDK/DPDK or the libs it ties into that can tell if a PCIe device is mapped into user-space via VFIO vs UIO? I need to ban CMB allocation on VFIO mapped devices and want to leverage existing helper code if at all possible.

Cheers
 
Stephen
 

Re: [SPDK] VFIO/UIO Helper Function?

[Harris, James R]

[-- Attachment #1: Type: text/plain, Size: 864 bytes --]



On 3/2/18, 3:27 PM, "SPDK on behalf of Stephen  Bates" <spdk-bounces(a)lists.01.org on behalf of sbates(a)raithlin.com> wrote:

    > There’s pci_vfio_is_enabled() 
    
    Jim, Thanks for the pointer. My concern here is that this function only checks if VFIO is running in the host kernel. That does not assure me that the NVMe device in question (the one with the CMB) was passed-through to SPDK using VFIO. Although setup.sh either does all UIO or all VFIO I assume some users may use their own script a mix UIO and VFIO.
    
  I would prefer to do a helper that takes the PCIe ID as an input and checks on a per-device basis. I will go off and do some digging...

Try looking at dev->kdrv in the rte_pci_device object.  That is set up by DPDK and you can get access to it through the lib/env_dpdk code (just needs a new env API).

-Jim
 

Re: [SPDK] VFIO/UIO Helper Function?

[Stephen Bates]

[-- Attachment #1: Type: text/plain, Size: 543 bytes --]

> There’s pci_vfio_is_enabled() 

Jim, Thanks for the pointer. My concern here is that this function only checks if VFIO is running in the host kernel. That does not assure me that the NVMe device in question (the one with the CMB) was passed-through to SPDK using VFIO. Although setup.sh either does all UIO or all VFIO I assume some users may use their own script a mix UIO and VFIO.

I would prefer to do a helper that takes the PCIe ID as an input and checks on a per-device basis. I will go off and do some digging...

Stephen

Re: [SPDK] VFIO/UIO Helper Function?

[Harris, James R]

[-- Attachment #1: Type: text/plain, Size: 1120 bytes --]

Hi Stephen,

Sort of.  There’s pci_vfio_is_enabled() in DPDK which technically isn’t a part of the public API.  The vhost code declares this function internally and uses it.  Probably we should have an spdk_pci_vfio_is_enable() env function and then both vhost and your CMB code could use that.  See include/spdk/env.h and lib/env_dpdk/pci.c.  Then someone should get a patch into upstream DPDK to have this vfio function (or a variant of it) in the DPDK public API.

-Jim


On 3/1/18, 6:53 PM, "SPDK on behalf of Stephen  Bates" <spdk-bounces(a)lists.01.org on behalf of sbates(a)raithlin.com> wrote:

    Hi All
    
    Does anyone know if there is a helper function in SPDK/DPDK or the libs it ties into that can tell if a PCIe device is mapped into user-space via VFIO vs UIO? I need to ban CMB allocation on VFIO mapped devices and want to leverage existing helper code if at all possible.
    
    Cheers
     
    Stephen
     
    
    _______________________________________________
    SPDK mailing list
    SPDK(a)lists.01.org
    https://lists.01.org/mailman/listinfo/spdk