From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS, USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41671C282C4 for ; Tue, 12 Feb 2019 18:20:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0F549222BB for ; Tue, 12 Feb 2019 18:20:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="Bwg4eIV/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729700AbfBLSUi (ORCPT ); Tue, 12 Feb 2019 13:20:38 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:35203 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726685AbfBLSUh (ORCPT ); Tue, 12 Feb 2019 13:20:37 -0500 Received: by mail-wm1-f65.google.com with SMTP id t200so4129533wmt.0 for ; Tue, 12 Feb 2019 10:20:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=bUE+CRybLrXIaYOIjcJlj0+EDkY0lhfvzRxWOTDT9hU=; b=Bwg4eIV/uk5JnpxvEDZ7aYV3p09wJgMLUuo7zVplKIugygptVP3tmWzBexfR7U9AX2 m31g4Bt+jDYkbUTpfyG2UFQORs5AdHxempNNY349LCkrYWEUaj2rHhjytDhHWNtU+Apv DAVMon85Be1HZFCYvEIveRqJ0ixVf7QFQv9k7iUJEuhI3jwcMTGXQHDrDQxHqJfSjsT8 zb/GdOAcv9oWbqmTrJlu+UaJ4D0Rq4kU3P6XLZ7uXyWvk0P6DUCCoxarCgFPK04EK2D6 QdHVYrlCi9PARKmFkwXHqG2Zrp6ANfZIDG63bz/mjeCx+FUi88fm4AyKg3VTrLbjIooW 6ffQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=bUE+CRybLrXIaYOIjcJlj0+EDkY0lhfvzRxWOTDT9hU=; b=pcVMDQuI806wUpNZKPpZDRVKMqgMz86ghcmxkJ2XfleAUETZ3b0lfs7AXzWzoUBYlQ S/xp/97lb0LU6qscH+6/OitdcfmDpUuPv0EDP51zn0QEZKw0uhmh0oJLXUMf31wx/p8D CE63NJ1jOl8Kflj1yiynnf+3juUIr40k6rwqjSq1RrqSBH15RbAj4al8sbYk4YkOSatz s9OWJLxEAN2v9sFIhx7UORAV5+sFK1d2YBVTVMFY/1jsSRltPafbjJKrNozBBxY1HfS9 gp2H9CbtsmysLenlxB17LvBHiBIIEj5kCAvvW7YWTPHo/bhLM2G6ZAwRNY9RwIeciV6P mNHQ== X-Gm-Message-State: AHQUAuaW7og7tItn/OGYovclHETo9w3UnnQXOedhAHuSl8Tcfe+YS/57 jP7AT8BNWVdegGUkaG5ALb61JQ== X-Google-Smtp-Source: AHgI3IahLHucSGj2w0o3CQgw8Qtdao3WBGaSLsyOEh9sRpTljzdv+xhS90TJhKJxiQUKaRQxPdYMHw== X-Received: by 2002:a1c:f00a:: with SMTP id a10mr138351wmb.148.1549995635073; Tue, 12 Feb 2019 10:20:35 -0800 (PST) Received: from apalos (ppp-94-65-225-153.home.otenet.gr. [94.65.225.153]) by smtp.gmail.com with ESMTPSA id z1sm8274672wrw.28.2019.02.12.10.20.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Feb 2019 10:20:34 -0800 (PST) Date: Tue, 12 Feb 2019 20:20:31 +0200 From: Ilias Apalodimas To: Alexander Duyck Cc: Eric Dumazet , Tariq Toukan , Matthew Wilcox , "brouer@redhat.com" , David Miller , "toke@redhat.com" , "netdev@vger.kernel.org" , "mgorman@techsingularity.net" , "linux-mm@kvack.org" Subject: Re: [RFC, PATCH] net: page_pool: Don't use page->private to store dma_addr_t Message-ID: <20190212182031.GA23057@apalos> References: <20190207150745.GW21860@bombadil.infradead.org> <20190207152034.GA3295@apalos> <20190207.132519.1698007650891404763.davem@davemloft.net> <20190207213400.GA21860@bombadil.infradead.org> <20190207214237.GA10676@Iliass-MBP.lan> <64f7af75-e6df-7abc-c4ce-82e6ca51fafe@gmail.com> <27e97aac-f25b-d46c-3e70-7d0d44f784b5@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Hi Alexander, On Tue, Feb 12, 2019 at 10:13:30AM -0800, Alexander Duyck wrote: > On Tue, Feb 12, 2019 at 7:16 AM Eric Dumazet wrote: > > > > > > > > On 02/12/2019 04:39 AM, Tariq Toukan wrote: > > > > > > > > > On 2/11/2019 7:14 PM, Eric Dumazet wrote: > > >> > > >> > > >> On 02/11/2019 12:53 AM, Tariq Toukan wrote: > > >>> > > >> > > >>> Hi, > > >>> > > >>> It's great to use the struct page to store its dma mapping, but I am > > >>> worried about extensibility. > > >>> page_pool is evolving, and it would need several more per-page fields. > > >>> One of them would be pageref_bias, a planned optimization to reduce the > > >>> number of the costly atomic pageref operations (and replace existing > > >>> code in several drivers). > > >>> > > >> > > >> But the point about pageref_bias is to place it in a different cache line than "struct page" > > >> > > >> The major cost is having a cache line bouncing between producer and consumer. > > >> > > > > > > pageref_bias is meant to be dirtied only by the page requester, i.e. the > > > NIC driver / page_pool. > > > All other components (basically, SKB release flow / put_page) should > > > continue working with the atomic page_refcnt, and not dirty the > > > pageref_bias. > > > > This is exactly my point. > > > > You suggested to put pageref_bias in struct page, which breaks this completely. > > > > pageref_bias is better kept in a driver structure, with appropriate prefetching > > since most NIC use a ring buffer for their queues. > > > > The dma address _can_ be put in the struct page, since the driver does not dirty it > > and does not even read it when page can be recycled. > > Instead of maintaining the pageref_bias in the page itself it could be > maintained in some sort of separate structure. You could just maintain > a pointer to a slot in an array somewhere. Then you can still access > it if needed, the pointer would be static for as long as it is in the > page pool, and you could invalidate the pointer prior to removing the > bias from the page. I think that's what Tariq was suggesting in the first place. /Ilias