From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ferruh Yigit Subject: Re: [PATCH v2] kni: add IOVA va support for kni Date: Thu, 4 Apr 2019 12:21:17 +0100 Message-ID: <305ee63d-4f7c-da9d-a1a2-e761a9fcedd2@intel.com> References: <20180927104846.16356-1-kkokkilagadda@caviumnetworks.com> <20190401095118.4176-1-kirankumark@marvell.com> <4c3ef828-652b-1498-b89b-7b7b58208f46@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: "dev@dpdk.org" , Jerin Jacob To: "Burakov, Anatoly" , Kiran Kumar Kokkilagadda Return-path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 6083B1B227 for ; Thu, 4 Apr 2019 13:21:20 +0200 (CEST) In-Reply-To: <4c3ef828-652b-1498-b89b-7b7b58208f46@intel.com> Content-Language: en-US List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 4/4/2019 10:57 AM, Burakov, Anatoly wrote: > On 03-Apr-19 5:29 PM, Ferruh Yigit wrote: >> On 4/1/2019 10:51 AM, Kiran Kumar Kokkilagadda wrote: >>> From: Kiran Kumar K >>> >>> With current KNI implementation kernel module will work only in >>> IOVA=PA mode. This patch will add support for kernel module to work >>> with IOVA=VA mode. >> >> Thanks Kiran for removing the limitation, I have a few questions, can you please >> help me understand. >> >> And when this patch is ready, the restriction in 'linux/eal/eal.c', in >> 'rte_eal_init' should be removed, perhaps with this patch. I assume you already >> doing it to be able to test this patch. >> >>> >>> The idea is to maintain a mapping in KNI module between user pages and >>> kernel pages and in fast path perform a lookup in this table and get >>> the kernel virtual address for corresponding user virtual address. >>> >>> In IOVA=VA mode, the memory allocated to the pool is physically >>> and virtually contiguous. We will take advantage of this and create a >>> mapping in the kernel.In kernel we need mapping for queues >>> (tx_q, rx_q,... slow path) and mbuf memory (fast path). >> >> Is it? >> As far as I know mempool can have multiple chunks and they can be both virtually >> and physically separated. >> >> And even for a single chunk, that will be virtually continuous, but will it be >> physically continuous? > > Just to clarify. > > Within DPDK, we do not make a distinction between physical address and > IOVA address - we never need the actual physical address, we just need > the DMA addresses, which can either match the physical address, or be > completely arbitrary (in our case, they will match VA addresses, but it > doesn't have to be the case - in fact, 17.11 will, under some > circumstances, populate IOVA addresses simply starting from address 0). > > However, one has to remember that IOVA address *is not a physical > address*. The pages backing a VA chunk may be *IOVA*-contiguous, but may > not necessarily be *physically* contiguous. Under normal circumstances > we really don't care, because the VFIO/IOMMU takes care of the mapping > between IOVA and PA transparently for the hardware. > > So, in IOVA as VA mode, the memory allocated to the mempool will be > (within a given chunk) both VA and IOVA contiguous - but not necessarily > *physically* contiguous! In fact, if you try calling rte_mem_virt2phy() > on the mempool pages, you'll likely find that they aren't (i've seen > cases where pages were mapped /backwards/!). > > Therefore, unless by "physically contiguous" you mean "IOVA-contiguous", > you *cannot* rely on memory in mempool being *physically* contiguous > merely based on the fact that it's IOVA-contiguous. Thanks for clarification.