From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C286DC433C1 for ; Wed, 24 Mar 2021 20:53:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4FECA619C2 for ; Wed, 24 Mar 2021 20:53:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4FECA619C2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CF2736B031A; Wed, 24 Mar 2021 16:53:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CA2E16B031C; Wed, 24 Mar 2021 16:53:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B463F6B031D; Wed, 24 Mar 2021 16:53:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0139.hostedemail.com [216.40.44.139]) by kanga.kvack.org (Postfix) with ESMTP id 991746B031A for ; Wed, 24 Mar 2021 16:53:48 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 5ADF2824999B for ; Wed, 24 Mar 2021 20:53:48 +0000 (UTC) X-FDA: 77955969336.37.1925140 Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) by imf28.hostedemail.com (Postfix) with ESMTP id 363C3200024E for ; Wed, 24 Mar 2021 20:53:47 +0000 (UTC) Received: by mail-lf1-f49.google.com with SMTP id b83so34029143lfd.11 for ; Wed, 24 Mar 2021 13:53:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=9IAjIh7gf44rR4epDI0vFAr+lh0+8IoyQ48EbdutPC4=; b=X+i+XcHts81gTX2YQSoV3UFieH2kepMG7Wygr2n+8tt5NHgaQ+mstmTUI3J5ISoI7U SUl3tiJvsbwRVOLXnmugaf7VxQxxdhAWYfBnAG00WyqH9bu/e4yNA5k3oEymKHyTzkLh TTT+a8G71fKF6cc1A4Vkr/zD4HBlTVppNUf5AaI1iqxq87iXB7fbncGKeplOfgTH82ws hdeCH/aQeXDSvTbv1qPdy2Lh7zjA6lcDs1+f+DHw1q7iu3l5yphxkZpMqeDxCa8CTxyZ 5VTmYNZSiJct4WOQqCy68/ZoK5rI3L/qFEYbBPrXDsH1QW2R7QyRDf07jo8vRTObrEez ApCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=9IAjIh7gf44rR4epDI0vFAr+lh0+8IoyQ48EbdutPC4=; b=NBS3Q3x3iZT5/N5OsjQApnwO12mpoohGWqhZfOQ+LrOoDw0sWv351uE1d/ySaBxUSO ZDT8crAOmPw2mulafRxOtd0PxRKorKSFCBkdKTtotdBNZ8xcT3j689tSiW0DqxNU+TR/ Rx8NqRfoEKSkYM6wEVVRi83qrekvmRffQDo3ilEghIBICVuV7WgVX4pMRC/IztiI6427 4/yFEnXwn8upQ6ohnHl5sdlWNH5ytliRPANsVbck8J8xruXY1U3bOPsaVA3Faw4PsedH vZSRkePE/QK47VxjI8QrtFiJ9BgQ5Ttxb9UEgwFh/yIUMP1yFZ/YE3L/+Mtn6C/5NDWi ViLQ== X-Gm-Message-State: AOAM531eJfNUrfCkKlO8AYJDUJ4yLsIPTI9M4nsjVCEoIkhLnemVGVzP GZly5Rg5fLygZj+9/230ERvJIhHcYp78wD504KgWLg== X-Google-Smtp-Source: ABdhPJw0lQ/lHICaIcFDzikDJKy0JhvudtbKraeVHNse46qud1jmkyD20uZNubiTW6uoYiF8GjRukNZVynWF7Og7DJA= X-Received: by 2002:a19:c14a:: with SMTP id r71mr2932916lff.358.1616619225992; Wed, 24 Mar 2021 13:53:45 -0700 (PDT) MIME-Version: 1.0 References: <20210316041645.144249-1-arjunroy.kdev@gmail.com> In-Reply-To: From: Shakeel Butt Date: Wed, 24 Mar 2021 13:53:34 -0700 Message-ID: Subject: Re: [mm, net-next v2] mm: net: memcg accounting for TCP rx zerocopy To: Arjun Roy Cc: Michal Hocko , Johannes Weiner , Arjun Roy , Andrew Morton , David Miller , netdev , Linux Kernel Mailing List , Cgroups , Linux MM , Eric Dumazet , Soheil Hassas Yeganeh , Jakub Kicinski , Yang Shi , Roman Gushchin Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: aayzimifpntgczx4bahkgsnstd1jmth1 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 363C3200024E Received-SPF: none (google.com>: No applicable sender policy available) receiver=imf28; identity=mailfrom; envelope-from=""; helo=mail-lf1-f49.google.com; client-ip=209.85.167.49 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616619227-170188 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Mar 24, 2021 at 1:39 PM Arjun Roy wrote: > > On Wed, Mar 24, 2021 at 2:12 AM Michal Hocko wrote: > > > > On Tue 23-03-21 11:47:54, Arjun Roy wrote: > > > On Tue, Mar 23, 2021 at 7:34 AM Michal Hocko wrote: > > > > > > > > On Wed 17-03-21 18:12:55, Johannes Weiner wrote: > > > > [...] > > > > > Here is an idea of how it could work: > > > > > > > > > > struct page already has > > > > > > > > > > struct { /* page_pool used by netstack */ > > > > > /** > > > > > * @dma_addr: might require a 64-bit value even on > > > > > * 32-bit architectures. > > > > > */ > > > > > dma_addr_t dma_addr; > > > > > }; > > > > > > > > > > and as you can see from its union neighbors, there is quite a bit more > > > > > room to store private data necessary for the page pool. > > > > > > > > > > When a page's refcount hits zero and it's a networking page, we can > > > > > feed it back to the page pool instead of the page allocator. > > > > > > > > > > From a first look, we should be able to use the PG_owner_priv_1 page > > > > > flag for network pages (see how this flag is overloaded, we can add a > > > > > PG_network alias). With this, we can identify the page in __put_page() > > > > > and __release_page(). These functions are already aware of different > > > > > types of pages and do their respective cleanup handling. We can > > > > > similarly make network a first-class citizen and hand pages back to > > > > > the network allocator from in there. > > > > > > > > For compound pages we have a concept of destructors. Maybe we can extend > > > > that for order-0 pages as well. The struct page is heavily packed and > > > > compound_dtor shares the storage without other metadata > > > > int pages; /* 16 4 */ > > > > unsigned char compound_dtor; /* 16 1 */ > > > > atomic_t hpage_pinned_refcount; /* 16 4 */ > > > > pgtable_t pmd_huge_pte; /* 16 8 */ > > > > void * zone_device_data; /* 16 8 */ > > > > > > > > But none of those should really require to be valid when a page is freed > > > > unless I am missing something. It would really require to check their > > > > users whether they can leave the state behind. But if we can establish a > > > > contract that compound_dtor can be always valid when a page is freed > > > > this would be really a nice and useful abstraction because you wouldn't > > > > have to care about the specific type of page. > > > > > > > > But maybe I am just overlooking the real complexity there. > > > > -- > > > > > > For now probably the easiest way is to have network pages be first > > > class with a specific flag as previously discussed and have concrete > > > handling for it, rather than trying to establish the contract across > > > page types. > > > > If you are going to claim a page flag then it would be much better to > > have it more generic. Flags are really scarce and if all you care about > > is PageHasDestructor() and provide one via page->dtor then the similar > > mechanism can be reused by somebody else. Or does anything prevent that? > > The way I see it - the fundamental want here is, for some arbitrary > page that we are dropping a reference on, to be able to tell that the > provenance of the page is some network driver's page pool. If we added > an enum target to compound_dtor, if we examine that offset in the page > and look at that value, what guarantee do we have that the page isn't > instead some other kind of page, and the byte value there was just > coincidentally the one we were looking for (but it wasn't a network > driver pool page)? > > Existing users of compound_dtor seem to check first that a > PageCompound() or PageHead() return true - the specific scenario here, > of receiving network packets, those pages will tend to not be compound > (and more specifically, compound pages are explicitly disallowed for > TCP receive zerocopy). > > Given that's the case, the options seem to be: > 1) Use a page flag - with the downside that they are a severely > limited resource, > 2) Use some bits inside page->memcg_data - this I believe Johannes had > reasons against, and it isn't always the case that MEMCG support is > enabled. > 3) Use compound_dtor - but I think this would have problems for the > prior reasons. I don't think Michal is suggesting to use PageCompound() or PageHead(). He is suggesting to add a more general page flag (PageHasDestructor) and corresponding page->dtor, so other potential users can use it too.