From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DE12C433B4 for ; Fri, 14 May 2021 09:18:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 80309613FB for ; Fri, 14 May 2021 09:17:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 80309613FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F14736B0036; Fri, 14 May 2021 05:17:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC3436B006E; Fri, 14 May 2021 05:17:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CEFF26B0070; Fri, 14 May 2021 05:17:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0174.hostedemail.com [216.40.44.174]) by kanga.kvack.org (Postfix) with ESMTP id 9A12F6B0036 for ; Fri, 14 May 2021 05:17:58 -0400 (EDT) Received: from smtpin40.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 374CD9895 for ; Fri, 14 May 2021 09:17:58 +0000 (UTC) X-FDA: 78139284636.40.73E7F90 Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) by imf09.hostedemail.com (Postfix) with ESMTP id E893F6000104 for ; Fri, 14 May 2021 09:17:56 +0000 (UTC) Received: by mail-ej1-f42.google.com with SMTP id l1so7724735ejb.6 for ; Fri, 14 May 2021 02:17:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=KpUWkUumoCfSvMpSBUUJEGaD2Gr/qdkOSMbFvxzdrIs=; b=wOVTnfZj32G+M9YPoPGwNSJl3x2VMEEDdWxCWvw3KCnTNmhxbmAr2LphAYMvK9rotD 0iqItIiOPagYPG2tNmBId2QaztuDBpgz+SyfPCCKBPUKwDH+uSN8Xhf3dmU/orKKupfP lHDei2hgUfFvCLXDvmB6XJQv6OvXTT6QMv1nPeYaVAOpRQA97jrLd448+nGbh/EOW8Ki fidWFFcC7TzGuU73oCdA8/xEoF9kecR8Bkkwj9bEWzyyH+F+ywO31BlqLSUvTqvSSBgp edFAYgN6oKtQB0xW4wuna7uGatVbp0Ai9v7v90MM0nWYCwAeNkkH2zPRWWDrlenHUNjT RUbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=KpUWkUumoCfSvMpSBUUJEGaD2Gr/qdkOSMbFvxzdrIs=; b=axfSnwZqOIUZ73pziZnJpHRnLuuuBH8R2YV8k+nkv37vwrcrc6FTgLnF5it7iizWj+ r4VHlVgOpGv3J6TjldaLTa+gZaiC6lLdF5B2sdkX7rXW2w0ebW9MK37LAn3eQjCIRpF6 kxfE0R75bddzbnBKi8vTOPMTW0QL3VtuELU/ksUA8ksjZ4PDBF1q+LAFfCi4Ae71dNIC ITPWUwjiv6pD9S6ppYU6PJHAtjr2VS5cV56RcHvlOO527Wic8k4V0OsXTW10QnP/r+5Z xyq0WudZ1k8o0dqoksboLt99517Dzjn92Sb2nCBH9ypzT22kkzHtwDC7CNuy9iVM/uWH a2gQ== X-Gm-Message-State: AOAM530kzZrTqd8f7QNTTKuVdXwH1PWkjZc/vAbvssgEVe13MO+A6eD7 S23kMsCuADy+69q4BXBsvyd1gQ== X-Google-Smtp-Source: ABdhPJy7u0vp54IwPd698tdL4ob3exH4UZJnsi9nd6zWe+32A7rELSgshx9as+boeH8h/FE7Srq6yw== X-Received: by 2002:a17:906:9bf3:: with SMTP id de51mr1246754ejc.394.1620983876468; Fri, 14 May 2021 02:17:56 -0700 (PDT) Received: from apalos.home ([94.69.77.156]) by smtp.gmail.com with ESMTPSA id rs8sm3298538ejb.17.2021.05.14.02.17.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 May 2021 02:17:56 -0700 (PDT) Date: Fri, 14 May 2021 12:17:50 +0300 From: Ilias Apalodimas To: Yunsheng Lin Cc: Matteo Croce , netdev@vger.kernel.org, linux-mm@kvack.org, Ayush Sawal , Vinay Kumar Yadav , Rohit Maheshwari , "David S. Miller" , Jakub Kicinski , Thomas Petazzoni , Marcin Wojtas , Russell King , Mirko Lindner , Stephen Hemminger , Tariq Toukan , Jesper Dangaard Brouer , Alexei Starovoitov , Daniel Borkmann , John Fastabend , Boris Pismenny , Arnd Bergmann , Andrew Morton , "Peter Zijlstra (Intel)" , Vlastimil Babka , Yu Zhao , Will Deacon , Fenghua Yu , Roman Gushchin , Hugh Dickins , Peter Xu , Jason Gunthorpe , Jonathan Lemon , Alexander Lobakin , Cong Wang , wenxu , Kevin Hao , Jakub Sitnicki , Marco Elver , Willem de Bruijn , Miaohe Lin , Guillaume Nault , linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, bpf@vger.kernel.org, Matthew Wilcox , Eric Dumazet , David Ahern , Lorenzo Bianconi , Saeed Mahameed , Andrew Lunn , Paolo Abeni , Sven Auhagen Subject: Re: [PATCH net-next v5 3/5] page_pool: Allow drivers to hint on SKB recycling Message-ID: References: <20210513165846.23722-1-mcroce@linux.microsoft.com> <20210513165846.23722-4-mcroce@linux.microsoft.com> <798d6dad-7950-91b2-46a5-3535f44df4e2@huawei.com> <212498cf-376b-2dac-e1cd-12c7cc7910c6@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <212498cf-376b-2dac-e1cd-12c7cc7910c6@huawei.com> Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b=wOVTnfZj; dmarc=pass (policy=none) header.from=linaro.org; spf=pass (imf09.hostedemail.com: domain of ilias.apalodimas@linaro.org designates 209.85.218.42 as permitted sender) smtp.mailfrom=ilias.apalodimas@linaro.org X-Stat-Signature: 6qm6zampu99eiu89zahmr9ojcutmjbnw X-Rspamd-Queue-Id: E893F6000104 X-Rspamd-Server: rspam02 X-HE-Tag: 1620983876-227501 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, May 14, 2021 at 04:31:50PM +0800, Yunsheng Lin wrote: > On 2021/5/14 15:36, Ilias Apalodimas wrote: > > [...] > >>> + return false; > >>> + > >>> + pp =3D (struct page_pool *)page->pp; > >>> + > >>> + /* Driver set this to memory recycling info. Reset it on recycle. > >>> + * This will *not* work for NIC using a split-page memory model. > >>> + * The page will be returned to the pool here regardless of the > >>> + * 'flipped' fragment being in use or not. > >>> + */ > >>> + page->pp =3D NULL; > >> > >> Why not only clear the page->pp when the page can not be recycled > >> by the page pool? so that we do not need to set and clear it every > >> time the page is recycled=E3=80=82 > >> > >=20 > > If the page cannot be recycled, page->pp will not probably be set to = begin > > with. Since we don't embed the feature in page_pool and we require th= e > > driver to explicitly enable it, as part of the 'skb flow', I'd rather= keep=20 > > it as is. When we set/clear the page->pp, the page is probably alrea= dy in=20 > > cache, so I doubt this will have any measurable impact. >=20 > The point is that we already have the skb->pp_recycle to let driver to > explicitly enable recycling, as part of the 'skb flow, if the page pool= keep > the page->pp while it owns the page, then the driver may only need to c= all > one skb_mark_for_recycle() for a skb, instead of call skb_mark_for_recy= cle() > for each page frag of a skb. >=20 The driver is meant to call skb_mark_for_recycle for the skb and page_pool_store_mem_info() for the fragments (in order to store page->pp)= . Nothing bad will happen if you call skb_mark_for_recycle on a frag though= , but in any case you need to store the page_pool pointer of each frag to struct page. > Maybe we can add a parameter in "struct page_pool_params" to let driver > to decide if the page pool ptr is stored in page->pp while the page poo= l > owns the page? Then you'd have to check the page pool config before saving the meta-data= , and you would have to make the skb path aware of that as well (I assume y= ou mean replace pp_recycle with this?). If not and you just want to add an extra flag on page_pool_params and be = able=20 to enable recycling depending on that flag, we just add a patch afterward= s. I am not sure we need an extra if for each packet though. >=20 > Another thing accured to me is that if the driver use page from the > page pool to form a skb, and it does not call skb_mark_for_recycle(), > then there will be resource leaking, right? if yes, it seems the > skb_mark_for_recycle() call does not seems to add any value? >=20 Not really, the driver has 2 choices: - call page_pool_release_page() once it receives the payload. That will clean up dma mappings (if page pool is responsible for them) and free t= he buffer - call skb_mark_for_recycle(). Which will end up recycling the buffer. If you call none of those, you'd leak a page, but that's a driver bug. patches [4/5, 5/5] do that for two marvell drivers. I really want to make drivers opt-in in the feature instead of always enabling it. Thanks /Ilias >=20 > >=20 > >>> + page_pool_put_full_page(pp, virt_to_head_page(data), false); > >>> + > >>> C(end); > >=20 > > [...] >=20 >=20