From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ferruh Yigit Subject: Re: [PATCH v8 0/5] add ifcvf vdpa driver Date: Mon, 16 Apr 2018 17:36:43 +0100 Message-ID: References: <20180415153349.62105-2-xiao.w.wang@intel.com> <20180416153438.79355-1-xiao.w.wang@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Cc: dev@dpdk.org, maxime.coquelin@redhat.com, zhihong.wang@intel.com, tiwei.bie@intel.com, jianfeng.tan@intel.com, cunming.liang@intel.com, dan.daly@intel.com, thomas@monjalon.net To: Xiao Wang , anatoly.burakov@intel.com Return-path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 833A8AAC9 for ; Mon, 16 Apr 2018 18:36:48 +0200 (CEST) In-Reply-To: <20180416153438.79355-1-xiao.w.wang@intel.com> Content-Language: en-US List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 4/16/2018 4:34 PM, Xiao Wang wrote: > IFCVF driver > ============ > The IFCVF vDPA (vhost data path acceleration) driver provides support for the > Intel FPGA 100G VF (IFCVF). IFCVF's datapath is virtio ring compatible, it > works as a HW vhost backend which can send/receive packets to/from virtio > directly by DMA. Besides, it supports dirty page logging and device state > report/restore. This driver enables its vDPA functionality with live migration > feature. > > vDPA mode > ========= > IFCVF's vendor ID and device ID are same as that of virtio net pci device, > with its specific subsystem vendor ID and device ID. To let the device be > probed by IFCVF driver, adding "vdpa=1" parameter helps to specify that this > device is to be used in vDPA mode, rather than polling mode, virtio pmd will > skip when it detects this message. > > Container per device > ==================== > vDPA needs to create different containers for different devices, thus this > patch set adds some APIs in eal/vfio to support multiple container, e.g. > - rte_vfio_create_container > - rte_vfio_destroy_container > - rte_vfio_bind_group > - rte_vfio_unbind_group > > By this extension, a device can be put into a new specific container, rather > than the previous default container. > > IFCVF vDPA details > ================== > Key vDPA driver ops implemented: > - ifcvf_dev_config: > Enable VF data path with virtio information provided by vhost lib, including > IOMMU programming to enable VF DMA to VM's memory, VFIO interrupt setup to > route HW interrupt to virtio driver, create notify relay thread to translate > virtio driver's kick to a MMIO write onto HW, HW queues configuration. > > This function gets called to set up HW data path backend when virtio driver > in VM gets ready. > > - ifcvf_dev_close: > Revoke all the setup in ifcvf_dev_config. > > This function gets called when virtio driver stops device in VM. > > Change log > ========== > v8: > - Rebase on HEAD. > - Move vfio_group definition back to eal_vfio.h. > - Return NULL when vfio group num/fd is not found, let caller handle that. > - Fix wrong API name in commit log. > - Rename bind/unbind function to rte_vfio_container_group_bind/unbind for > consistensy. > - Add note for rte_vfio_container_create and rte_vfio_dma_map and fix typo > in comment. > - Extract out the shared code snip of rte_vfio_dma_map and > rte_vfio_container_dma_map to avoid code duplication. So do for the unmap. > > v7: > - Rebase on HEAD. > - Split the vfio patch into 2 parts, one for data structure extension, one for > adding new API. > - Use static vfio_config array instead of dynamic alloating. > - Change rte_vfio_container_dma_map/unmap's parameters to use (va, iova, len). > > v6: > - Rebase on master branch. > - Document "vdpa" devarg in virtio documentation. > - Rename ifcvf config option to CONFIG_RTE_LIBRTE_IFCVF_VDPA_PMD for > consistensy, and add it into driver documentation. > - Add comments for ifcvf device ID. > - Minor code cleaning. > > v5: > - Fix compilation in BSD, remove the rte_vfio.h including in BSD. > > v4: > - Rebase on Zhihong's latest vDPA lib patch, with vDPA ops names change. > - Remove API "rte_vfio_get_group_fd", "rte_vfio_bind_group" will return the fd. > - Align the vfio_cfg search internal APIs naming. > > v3: > - Add doc and release note for the new driver. > - Remove the vdev concept, make the driver as a PCI driver, it will get probed > by PCI bus driver. > - Rebase on the v4 vDPA lib patch, register a vDPA device instead of a engine. > - Remove the PCI API exposure accordingly. > - Move the MAX_VFIO_CONTAINERS definition to config file. > - Let virtio pmd skips when a virtio device needs to work in vDPA mode. > > v2: > - Rename function pci_get_kernel_driver_by_path to rte_pci_device_kdriver_name > to make the API generic cross Linux and BSD, make it as EXPERIMENTAL. > - Rebase on Zhihong's vDPA v3 patch set. > - Minor code cleanup on vfio extension. > > > Xiao Wang (5): > vfio: extend data structure for multi container > vfio: add multi container support > net/virtio: skip device probe in vdpa mode > net/ifcvf: add ifcvf vdpa driver > doc: add ifcvf driver document and release note Hi Xiao, Getting following build error for 32bit [1], can you please check them? [1] .../dpdk/drivers/net/ifc/ifcvf_vdpa.c: In function ‘ifcvf_dma_map’: .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:24:3: error: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 6 has type ‘uint64_t {aka long long unsigned int}’ [-Werror=format=] "%s(): " fmt "\n", __func__, ##args) ^ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:178:22: "size 0x%lx.", i, reg->host_user_addr, ~~~~~~ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:3: note: in expansion of macro ‘DRV_LOG’ DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, " ^~~~~~~ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:37: note: format string is defined here DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, " ~~^ %llx .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:24:3: error: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 7 has type ‘uint64_t {aka long long unsigned int}’ [-Werror=format=] "%s(): " fmt "\n", __func__, ##args) ^ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:179:4: reg->guest_phys_addr, reg->size); ~~~~~~ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:3: note: in expansion of macro ‘DRV_LOG’ DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, " ^~~~~~~ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:48: note: format string is defined here DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, " ~~^ %llx .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:24:3: error: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 8 has type ‘uint64_t {aka long long unsigned int}’ [-Werror=format=] "%s(): " fmt "\n", __func__, ##args) ^ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:179:26: reg->guest_phys_addr, reg->size); ~~~~~~ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:3: note: in expansion of macro ‘DRV_LOG’ DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, " ^~~~~~~ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:178:14: note: format string is defined here "size 0x%lx.", i, reg->host_user_addr, ~~^ %llx .../dpdk/drivers/net/ifc/ifcvf_vdpa.c: In function ‘vdpa_ifcvf_start’: .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:266:39: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast] hw->vring[i].desc = qva_to_gpa(vid, (uint64_t)vq.desc); ^ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:267:40: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast] hw->vring[i].avail = qva_to_gpa(vid, (uint64_t)vq.avail); ^ .../dpdk/drivers/net/ifc/ifcvf_vdpa.c:268:39: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast] hw->vring[i].used = qva_to_gpa(vid, (uint64_t)vq.used); ^