Re: [PATCH 00/10] Infrastructure to detect iova mapping on the bus

[santosh <santosh.shukla@caviumnetworks.com>]

Hi,

On Thursday 08 June 2017 04:35 PM, Santosh Shukla wrote:

> Q) Why do we need such infrastructure?
>
> A) Some NPU hardware like OCTEONTX follows push model to get the packet
> from the pktio device. Where packet allocation and freeing done
> by the HW. Since HW can operate only on IOVA with help of SMMU/IOMMU,
> when packet receives from the Ethernet device, it is the IOVA address
> (which is PA in existing scheme).
>
> Mapping IOVA as PA is expensive on those HW, where every packet
> needs to be converted to VA from PA/IOVA.
>
> This patchset proposes the method to autodetect the preferred
> IOVA mode for a device. Summary of IOVA scheme:
> - If all the devices are iommu capable and support IOMMU
>   capable driver then selects IOVA_VA.
> - If any of the devices are non-iommu then use default IOVA
>   scheme ie. IOVA_PA.
> - If no device found then IOVA scheme would be
>   IOVA_DC (Don't care).
>
> To achieve that, two global APIs introduced:
> - rte_bus_get_iommu_class
> - rte_pci_get_iommu_class
>
> Return values for those APIs are:
> enum rte_iova_mod {
>         RTE_IOVA_DC, /* Don't care */
>         RTE_IOVA_PA,
>         RTE_IOVA_VA
> }
>
> Those are the bus policy for selecting IOVA mode. In case user
> want to override bus IOVA mapping then added an EAL option
> "--iova-mode=<string>". User to pass string format 'pa' --> IOVA_PA,
> 'va' --> IOVA_VA.
>
> To support new eal option, adding global API:
> - rte_eal_iova_mode
>
> Patch Summary:
> 2) 1st - 2th patch: Adds infrastructure in linuxapp and bsdapp
> layer.
> 1) 3rd patch: Introduces global bus api named rte_bus_get_iommu_class.
> 3) 4th patch: Add new eal option called --iova-mode=<mode-string>.
> 4) 5th - 6th patch: Logic to detect iova scheme.
> 5) 9th patch: Check IOVA mode before programing vfio dma_map.iova.
> Default scheme is IOVA_PA.
> 6) 10th-12th patch: Check for IOVA_VA mode in below APIs
>         - rte_mem_virt2phy
>         - rte_mempool_virt2phy
>         - rte_malloc_virt2phy
> If set then return paddr=vaddr, else return value from default
> implementation.
>
> Test History:
> - Tested for x86/XL710 40G NIC card for both modes (iova_va/pa).
> - Tested for arm64/thunderx vNIC Integrated NIC for both modes
> - Tested for arm64/Octeontx integrated NICs for only
>   Iova_va mode(It supports only one mode.)
> - Ran standalone tests like mempool_autotest, mbuf_autotest.
> - Verified for Doxygen.
>
> Work History:
> Refer prev RFC proposal[1].
>
> Noticed false positive checkpatch error:
> - WARNING: quoted string split across lines
> #60: FILE: lib/librte_eal/common/eal_common_bus.c:164:
> +				RTE_LOG(INFO, EAL, "Bus (%s) iommu class of"
> +					" devices not found.\n", bus->name);
>
> - WARNING: LINUX_VERSION_CODE should be avoided, code should be for the version to which it is merged
> #86: FILE: lib/librte_eal/linuxapp/eal/eal_vfio.c:822:
> +#if LINUX_VERSION_CODE < KERNEL_VERSION(4, 2, 0)
>
> Thanks.
>
> [1] http://dpdk.org/dev/patchwork/patch/24549/
>
Ping?

Thanks.