From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69BC5C5DF60 for ; Tue, 5 Nov 2019 16:03:32 +0000 (UTC) Received: from dpdk.org (dpdk.org [92.243.14.124]) by mail.kernel.org (Postfix) with ESMTP id 0E66D214B2 for ; Tue, 5 Nov 2019 16:03:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0E66D214B2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=6wind.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6C8DD1C02B; Tue, 5 Nov 2019 17:03:31 +0100 (CET) Received: from proxy.6wind.com (host.76.145.23.62.rev.coltfrance.com [62.23.145.76]) by dpdk.org (Postfix) with ESMTP id 09C091C029 for ; Tue, 5 Nov 2019 17:03:30 +0100 (CET) Received: from glumotte.dev.6wind.com (unknown [10.16.0.195]) by proxy.6wind.com (Postfix) with ESMTP id CFDF533B11F; Tue, 5 Nov 2019 17:03:29 +0100 (CET) Date: Tue, 5 Nov 2019 17:03:29 +0100 From: Olivier Matz To: dev@dpdk.org Cc: Anatoly Burakov , Andrew Rybchenko , Ferruh Yigit , "Giridharan, Ganesan" , Jerin Jacob Kollanukkaran , Kiran Kumar Kokkilagadda , Stephen Hemminger , Thomas Monjalon , Vamsi Krishna Attunuru , Hemant Agrawal , Nipun Gupta , David Marchand Message-ID: <20191105160329.GC6671@glumotte.dev.6wind.com> References: <20190719133845.32432-1-olivier.matz@6wind.com> <20191105153707.14645-1-olivier.matz@6wind.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20191105153707.14645-1-olivier.matz@6wind.com> User-Agent: Mutt/1.10.1 (2018-07-13) Subject: Re: [dpdk-dev] [PATCH v4 0/7] mempool: avoid objects allocations across pages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Tue, Nov 05, 2019 at 04:36:59PM +0100, Olivier Matz wrote: > KNI supposes that mbufs are contiguous in kernel virtual memory. This > may not be true when using the IOVA=VA mode. To fix this, a possibility > is to ensure that objects do not cross page boundaries in mempool. This > patchset implements this in the last patch (5/5). > > The previous patches prepare the job: > - allow to populate with an unaligned virtual area (1/5). > - reduce spaced wasted in the mempool size calculation when not using > the iova-contiguous allocation (2/5). > - remove the iova-contiguous allocation when populating mempool (3/5): > a va-contiguous alloc does the job as well if we want to populate > without crossing page boundaries, so simplify the mempool populate > function. > - export a function to get the minimum page used in a mempool (4/5) > > Memory consumption impact when using hugepages: > - worst case: + ~0.1% for a mbuf pool (objsize ~= 2368) > - best case: -50% for if pool size is just above page size > > The memory consumption impact with 4K pages in IOVA=VA mode could > however consume up to 75% more memory for mbuf pool, because there will > be only 1 mbuf per page. Not sure how common this usecase is. > > Caveat: this changes the behavior of the mempool (calc_mem_size and > populate), and there is a small risk to break things, especially with > alternate mempool drivers. > > v4 > > * remove useless comments in Makefiles and meson.build (sugg by David) > * add EXPERIMENTAL banner on new functions in API comments (David) > * sort by version in rte_mempool_version.map (David) > * remove duplicated -DALLOW_EXPERIMENTAL_API flag in octeontx2 > mempool driver > * enhance API comments for new helpers I forgot: * move bucket mempool driver modifications from patch 7 to patch 6 (seen by Andrew) > > v3 > > * introduce new helpers to calculate required memory size and to > populate mempool, use them in drivers: the alignment constraint > of octeontx/octeontx2 is managed in this common code. > * fix octeontx mempool driver by taking alignment constraint in account > like in octeontx2 > * fix bucket mempool driver with 4K pages: limit bucket size in this > case to ensure that objects do not cross page boundaries. With larger > pages, it was already ok, because bucket size (64K) is smaller than > a page. > * fix some api comments in mempool header file > > v2 > > * update octeontx2 driver to keep alignment constraint (issue seen by > Vamsi) > * add a new patch to use RTE_MEMPOOL_ALIGN (Andrew) > * fix initialization of for loop in rte_mempool_populate_virt() (Andrew) > * use rte_mempool_populate_iova() if mz_flags has > RTE_MEMZONE_IOVA_CONTIG (Andrew) > * check rte_mempool_get_page_size() return value (Andrew) > * some other minor style improvements > > rfc -> v1 > > * remove first cleanup patch, it was pushed separately > a2b5a8722f20 ("mempool: clarify default populate function") > * add missing change in rte_mempool_op_calc_mem_size_default() > * allow unaligned addr/len in populate virt > * better split patches > * try to better explain the change > * use DPDK align macros when relevant > > Olivier Matz (7): > mempool: allow unaligned addr/len in populate virt > mempool: reduce wasted space on mempool populate > mempool: remove optimistic IOVA-contiguous allocation > mempool: introduce function to get mempool page size > mempool: introduce helpers for populate and calc mem size > mempool: prevent objects from being across pages > mempool: use the specific macro for object alignment > > drivers/mempool/bucket/Makefile | 1 + > drivers/mempool/bucket/meson.build | 2 + > drivers/mempool/bucket/rte_mempool_bucket.c | 10 +- > drivers/mempool/dpaa/dpaa_mempool.c | 4 +- > drivers/mempool/dpaa2/dpaa2_hw_mempool.c | 4 +- > drivers/mempool/octeontx/Makefile | 2 + > drivers/mempool/octeontx/meson.build | 2 + > .../mempool/octeontx/rte_mempool_octeontx.c | 21 +-- > drivers/mempool/octeontx2/Makefile | 2 + > drivers/mempool/octeontx2/meson.build | 2 + > drivers/mempool/octeontx2/otx2_mempool_ops.c | 21 ++- > lib/librte_mempool/rte_mempool.c | 147 +++++++----------- > lib/librte_mempool/rte_mempool.h | 114 ++++++++++++-- > lib/librte_mempool/rte_mempool_ops.c | 4 +- > lib/librte_mempool/rte_mempool_ops_default.c | 113 +++++++++++--- > lib/librte_mempool/rte_mempool_version.map | 6 + > 16 files changed, 312 insertions(+), 143 deletions(-) > > -- > 2.20.1 >