From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ECBAC433F5 for ; Tue, 31 May 2022 17:15:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E85036B0072; Tue, 31 May 2022 13:15:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E081E6B0073; Tue, 31 May 2022 13:15:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA62C6B0074; Tue, 31 May 2022 13:15:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id BCAE06B0072 for ; Tue, 31 May 2022 13:15:20 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8E85F607D7 for ; Tue, 31 May 2022 17:15:20 +0000 (UTC) X-FDA: 79526689200.15.7270D06 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf24.hostedemail.com (Postfix) with ESMTP id 295D918004D for ; Tue, 31 May 2022 17:15:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654017319; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K99R58snJJwFfcIwa+YFRkkNuV7A1vgEdno4MZw9LsQ=; b=Y3785Ju7bfNv4lCXJUegY5OKMXvsHvGNJ6ux9RmOfO5+CQ0NBltSDp4/0eaKl7kRsI0t5k NZ96qfGZcxlLHo/tt2lKBaoAIBsNFb4XJ7/qyINp1hnxe9bgeRkDNJnwER7f6UR/l8J6cp VkBxxTTQdn2pmsvVxcIy1GV7MyoyvLA= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-648-uQ4S4uOkO4GCIvVUGA6u2A-1; Tue, 31 May 2022 13:15:18 -0400 X-MC-Unique: uQ4S4uOkO4GCIvVUGA6u2A-1 Received: by mail-wm1-f70.google.com with SMTP id o32-20020a05600c512000b0039c1c56e757so560849wms.1 for ; Tue, 31 May 2022 10:15:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:references:from:organization:in-reply-to :content-transfer-encoding; bh=K99R58snJJwFfcIwa+YFRkkNuV7A1vgEdno4MZw9LsQ=; b=5+JeJIPsDWeMKsX3Fu4iOsMrSDYjSpTlbnF7NQTAYYqH72OpEvbQ9HO6nuq7GJ06E7 lSCi7rZMfwhQ3ODDChLLVwhir1IS7U58h6hIjnkAl2LbDio3dlJgD9MfiIvteNwWsP3j zqclPee4Hi2o1YzR9XU+I71m67WLqIdF6b4bBEpQyKcx/UTqwKVgJTzomGrmkvbPPMR/ B29Nz3ixQLovsAzRMxqifTH/zMKdj/7o72awBZbsbaa+uqzDX0eXq57zyfdd1YJvpnaU oH6XqyakWK1bdkYxrhb1dC7DuXi8BHOL3nIBp+01cHTrgMVRgyU6+d4tW3nVxeIQh//K pALg== X-Gm-Message-State: AOAM530bDWrFqnT8XEZNtaXa1pI4s7bLC729hgFOstEEMYkP3uvKdHVU IV9SBDHsAGLKYun6MYIqacMi9GipMytcwDl5OuJeHcrJhLY0+Yh2vmzABMjYDtyhXsNzChG8hHc JGUPklTqWtLc= X-Received: by 2002:adf:fb03:0:b0:20a:e253:b8c7 with SMTP id c3-20020adffb03000000b0020ae253b8c7mr50441480wrr.119.1654017316967; Tue, 31 May 2022 10:15:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwdoUh5gXmEaJIGPD5mRGrHIXBSg4248MBxJRmtzlAPOPT10+rqdYEm1V88W+Qs5ngchw/D5w== X-Received: by 2002:adf:fb03:0:b0:20a:e253:b8c7 with SMTP id c3-20020adffb03000000b0020ae253b8c7mr50441465wrr.119.1654017316712; Tue, 31 May 2022 10:15:16 -0700 (PDT) Received: from ?IPV6:2003:cb:c708:f100:8096:9368:ba52:a341? (p200300cbc708f10080969368ba52a341.dip0.t-ipconnect.de. [2003:cb:c708:f100:8096:9368:ba52:a341]) by smtp.gmail.com with ESMTPSA id m126-20020a1c2684000000b003942a244f30sm3284044wmm.9.2022.05.31.10.15.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 31 May 2022 10:15:15 -0700 (PDT) Message-ID: <40d658da-6220-e05e-ba0b-d95c82f6bfb3@redhat.com> Date: Tue, 31 May 2022 19:15:14 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.0 Subject: Re: [PATCH 5/6] slab: Allocate frozen pages To: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org References: <20220531150611.1303156-1-willy@infradead.org> <20220531150611.1303156-6-willy@infradead.org> From: David Hildenbrand Organization: Red Hat In-Reply-To: <20220531150611.1303156-6-willy@infradead.org> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 295D918004D X-Stat-Signature: bfz4x7bwopgu3j3kifsb7zhjp86hr8dr X-Rspam-User: Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Y3785Ju7; spf=none (imf24.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-HE-Tag: 1654017305-995837 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 31.05.22 17:06, Matthew Wilcox (Oracle) wrote: > Since slab does not use the page refcount, it can allocate and > free frozen pages, saving one atomic operation per free. > > Signed-off-by: Matthew Wilcox (Oracle) > --- > mm/slab.c | 23 +++++++++++------------ > 1 file changed, 11 insertions(+), 12 deletions(-) > > diff --git a/mm/slab.c b/mm/slab.c > index f8cd00f4ba13..c5c53ed304d1 100644 > --- a/mm/slab.c > +++ b/mm/slab.c > @@ -1355,23 +1355,23 @@ slab_out_of_memory(struct kmem_cache *cachep, gfp_t gfpflags, int nodeid) > static struct slab *kmem_getpages(struct kmem_cache *cachep, gfp_t flags, > int nodeid) > { > - struct folio *folio; > + struct page *page; > struct slab *slab; > > flags |= cachep->allocflags; > > - folio = (struct folio *) __alloc_pages_node(nodeid, flags, cachep->gfporder); > - if (!folio) { > + page = __alloc_frozen_pages(flags, cachep->gfporder, nodeid, NULL); > + if (!page) { > slab_out_of_memory(cachep, flags, nodeid); > return NULL; > } > > - slab = folio_slab(folio); > + __SetPageSlab(page); > + slab = (struct slab *)page; > > account_slab(slab, cachep->gfporder, cachep, flags); > - __folio_set_slab(folio); > /* Record if ALLOC_NO_WATERMARKS was set when allocating the slab */ > - if (sk_memalloc_socks() && page_is_pfmemalloc(folio_page(folio, 0))) > + if (sk_memalloc_socks() && page_is_pfmemalloc(page)) > slab_set_pfmemalloc(slab); > > return slab; > @@ -1383,18 +1383,17 @@ static struct slab *kmem_getpages(struct kmem_cache *cachep, gfp_t flags, > static void kmem_freepages(struct kmem_cache *cachep, struct slab *slab) > { > int order = cachep->gfporder; > - struct folio *folio = slab_folio(slab); > + struct page *page = (struct page *)slab; > > - BUG_ON(!folio_test_slab(folio)); > __slab_clear_pfmemalloc(slab); > - __folio_clear_slab(folio); > - page_mapcount_reset(folio_page(folio, 0)); > - folio->mapping = NULL; > + __ClearPageSlab(page); > + page_mapcount_reset(page); > + page->mapping = NULL; > > if (current->reclaim_state) > current->reclaim_state->reclaimed_slab += 1 << order; > unaccount_slab(slab, order, cachep); > - __free_pages(folio_page(folio, 0), order); > + free_frozen_pages(page, order); > } > > static void kmem_rcu_free(struct rcu_head *head) I assume that implies that pages that are actually allocated *from* the buddy now have a refcount == 0. IIRC, page isolation code (e.g., !page_ref_count check in has_unmovable_pages()) assumes that any page with a refcount of 0 is essentially either already free (buddy) or is on its way of getting freed (!buddy). There might be other PFN walker code (like compaction) that makes similar assumptions that hold for now. -- Thanks, David / dhildenb