From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15A19C11F68 for ; Wed, 30 Jun 2021 04:10:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B1B176162C for ; Wed, 30 Jun 2021 04:10:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B1B176162C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 22CDC8D0173; Wed, 30 Jun 2021 00:10:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2035E8D0171; Wed, 30 Jun 2021 00:10:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0A4308D0173; Wed, 30 Jun 2021 00:10:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0235.hostedemail.com [216.40.44.235]) by kanga.kvack.org (Postfix) with ESMTP id DAF5D8D0171 for ; Wed, 30 Jun 2021 00:10:16 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B9A261822CC44 for ; Wed, 30 Jun 2021 04:10:16 +0000 (UTC) X-FDA: 78309062832.24.1A9A60F Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf05.hostedemail.com (Postfix) with ESMTP id 6836950003AB for ; Wed, 30 Jun 2021 04:10:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Xs+C8igKGtsfobOsWI7v7pFbJ5dQkVc5kSbNpGFbjuk=; b=hBR/N+GqwANlp/g5TmqeHCtGAg C8T+YV+0kVEgaWdbkCvDXfY+uM/hxn0n5HxcG2qh+E32tX2mXG1PsiDEjhEUqlyBn0mNfdnTFTHzJ w3QAfffEj0GP9aNtpsoNpz2I4xDedCYKqLw5O2vz+ylbF+cHS3aLrkIuR7xWAlbhwvLZvWMmz8mNN f//ECNgEu84NMivae2qsAk9WxY8jL4sXa4Sfck2wDsJnoWROIj2dn0Pco67o0qU3XBVvBquyABvAB NxXPIrEtwIB+b4bgt6jJ2j9zykeuE3kA6LJD5GsBUuAKhyw5hYXiOJDX7poFYT9l8ClaPQXlBDFlN AMi9b5MA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1lyRXF-004rJw-TY; Wed, 30 Jun 2021 04:09:24 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, cgroups@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , Johannes Weiner , Michal Hocko , Vladimir Davydov Subject: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock() Date: Wed, 30 Jun 2021 05:00:29 +0100 Message-Id: <20210630040034.1155892-14-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210630040034.1155892-1-willy@infradead.org> References: <20210630040034.1155892-1-willy@infradead.org> MIME-Version: 1.0 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="hBR/N+Gq"; dmarc=none; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspamd-Server: rspam02 X-Stat-Signature: o9gehynhq1bsxszn7ido4utabxy8hrwx X-Rspamd-Queue-Id: 6836950003AB X-HE-Tag: 1625026216-785765 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: These are the folio equivalents of lock_page_memcg() and unlock_page_memcg(). Reimplement them as wrappers. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/memcontrol.h | 10 +++++++++ mm/memcontrol.c | 45 ++++++++++++++++++++++++-------------- 2 files changed, 39 insertions(+), 16 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index ef79f9c0b296..279ea2640365 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -951,6 +951,8 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *me= mcg); extern bool cgroup_memory_noswap; #endif =20 +void folio_memcg_lock(struct folio *folio); +void folio_memcg_unlock(struct folio *folio); void lock_page_memcg(struct page *page); void unlock_page_memcg(struct page *page); =20 @@ -1363,6 +1365,14 @@ static inline void unlock_page_memcg(struct page *= page) { } =20 +static inline void folio_memcg_lock(struct folio *folio) +{ +} + +static inline void folio_memcg_unlock(struct folio *folio) +{ +} + static inline void mem_cgroup_handle_over_high(void) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b925bdce0c6e..b94a6122f27d 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1960,18 +1960,17 @@ void mem_cgroup_print_oom_group(struct mem_cgroup= *memcg) } =20 /** - * lock_page_memcg - lock a page and memcg binding - * @page: the page + * folio_memcg_lock - Bind a folio to its memcg. + * @folio: The folio. * - * This function protects unlocked LRU pages from being moved to + * This function prevents unlocked LRU folios from being moved to * another cgroup. * - * It ensures lifetime of the locked memcg. Caller is responsible - * for the lifetime of the page. + * It ensures lifetime of the bound memcg. The caller is responsible + * for the lifetime of the folio. */ -void lock_page_memcg(struct page *page) +void folio_memcg_lock(struct folio *folio) { - struct page *head =3D compound_head(page); /* rmap on tail pages */ struct mem_cgroup *memcg; unsigned long flags; =20 @@ -1985,7 +1984,7 @@ void lock_page_memcg(struct page *page) if (mem_cgroup_disabled()) return; again: - memcg =3D page_memcg(head); + memcg =3D folio_memcg(folio); if (unlikely(!memcg)) return; =20 @@ -1999,7 +1998,7 @@ void lock_page_memcg(struct page *page) return; =20 spin_lock_irqsave(&memcg->move_lock, flags); - if (memcg !=3D page_memcg(head)) { + if (memcg !=3D folio_memcg(folio)) { spin_unlock_irqrestore(&memcg->move_lock, flags); goto again; } @@ -2013,9 +2012,15 @@ void lock_page_memcg(struct page *page) memcg->move_lock_task =3D current; memcg->move_lock_flags =3D flags; } +EXPORT_SYMBOL(folio_memcg_lock); + +void lock_page_memcg(struct page *page) +{ + folio_memcg_lock(page_folio(page)); +} EXPORT_SYMBOL(lock_page_memcg); =20 -static void __unlock_page_memcg(struct mem_cgroup *memcg) +static void __memcg_unlock(struct mem_cgroup *memcg) { if (memcg && memcg->move_lock_task =3D=3D current) { unsigned long flags =3D memcg->move_lock_flags; @@ -2030,14 +2035,22 @@ static void __unlock_page_memcg(struct mem_cgroup= *memcg) } =20 /** - * unlock_page_memcg - unlock a page and memcg binding - * @page: the page + * folio_memcg_unlock - Release the binding between a folio and its memc= g. + * @folio: The folio. + * + * This releases the binding created by folio_memcg_lock(). This does + * not change the accounting of this folio to its memcg, but it does + * permit others to change it. */ -void unlock_page_memcg(struct page *page) +void folio_memcg_unlock(struct folio *folio) { - struct page *head =3D compound_head(page); + __memcg_unlock(folio_memcg(folio)); +} +EXPORT_SYMBOL(folio_memcg_unlock); =20 - __unlock_page_memcg(page_memcg(head)); +void unlock_page_memcg(struct page *page) +{ + folio_memcg_unlock(page_folio(page)); } EXPORT_SYMBOL(unlock_page_memcg); =20 @@ -5661,7 +5674,7 @@ static int mem_cgroup_move_account(struct page *pag= e, =20 page->memcg_data =3D (unsigned long)to; =20 - __unlock_page_memcg(from); + __memcg_unlock(from); =20 ret =3D 0; nid =3D page_to_nid(page); --=20 2.30.2 From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Matthew Wilcox (Oracle)" Subject: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock() Date: Wed, 30 Jun 2021 05:00:29 +0100 Message-ID: <20210630040034.1155892-14-willy@infradead.org> References: <20210630040034.1155892-1-willy@infradead.org> Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Return-path: DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Xs+C8igKGtsfobOsWI7v7pFbJ5dQkVc5kSbNpGFbjuk=; b=hBR/N+GqwANlp/g5TmqeHCtGAg C8T+YV+0kVEgaWdbkCvDXfY+uM/hxn0n5HxcG2qh+E32tX2mXG1PsiDEjhEUqlyBn0mNfdnTFTHzJ w3QAfffEj0GP9aNtpsoNpz2I4xDedCYKqLw5O2vz+ylbF+cHS3aLrkIuR7xWAlbhwvLZvWMmz8mNN f//ECNgEu84NMivae2qsAk9WxY8jL4sXa4Sfck2wDsJnoWROIj2dn0Pco67o0qU3XBVvBquyABvAB NxXPIrEtwIB+b4bgt6jJ2j9zykeuE3kA6LJD5GsBUuAKhyw5hYXiOJDX7poFYT9l8ClaPQXlBDFlN AMi9b5MA==; In-Reply-To: <20210630040034.1155892-1-willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> List-ID: Content-Type: text/plain; charset="us-ascii" To: linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org Cc: "Matthew Wilcox (Oracle)" , Johannes Weiner , Michal Hocko , Vladimir Davydov These are the folio equivalents of lock_page_memcg() and unlock_page_memcg(). Reimplement them as wrappers. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/memcontrol.h | 10 +++++++++ mm/memcontrol.c | 45 ++++++++++++++++++++++++-------------- 2 files changed, 39 insertions(+), 16 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index ef79f9c0b296..279ea2640365 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -951,6 +951,8 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg); extern bool cgroup_memory_noswap; #endif +void folio_memcg_lock(struct folio *folio); +void folio_memcg_unlock(struct folio *folio); void lock_page_memcg(struct page *page); void unlock_page_memcg(struct page *page); @@ -1363,6 +1365,14 @@ static inline void unlock_page_memcg(struct page *page) { } +static inline void folio_memcg_lock(struct folio *folio) +{ +} + +static inline void folio_memcg_unlock(struct folio *folio) +{ +} + static inline void mem_cgroup_handle_over_high(void) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b925bdce0c6e..b94a6122f27d 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1960,18 +1960,17 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg) } /** - * lock_page_memcg - lock a page and memcg binding - * @page: the page + * folio_memcg_lock - Bind a folio to its memcg. + * @folio: The folio. * - * This function protects unlocked LRU pages from being moved to + * This function prevents unlocked LRU folios from being moved to * another cgroup. * - * It ensures lifetime of the locked memcg. Caller is responsible - * for the lifetime of the page. + * It ensures lifetime of the bound memcg. The caller is responsible + * for the lifetime of the folio. */ -void lock_page_memcg(struct page *page) +void folio_memcg_lock(struct folio *folio) { - struct page *head = compound_head(page); /* rmap on tail pages */ struct mem_cgroup *memcg; unsigned long flags; @@ -1985,7 +1984,7 @@ void lock_page_memcg(struct page *page) if (mem_cgroup_disabled()) return; again: - memcg = page_memcg(head); + memcg = folio_memcg(folio); if (unlikely(!memcg)) return; @@ -1999,7 +1998,7 @@ void lock_page_memcg(struct page *page) return; spin_lock_irqsave(&memcg->move_lock, flags); - if (memcg != page_memcg(head)) { + if (memcg != folio_memcg(folio)) { spin_unlock_irqrestore(&memcg->move_lock, flags); goto again; } @@ -2013,9 +2012,15 @@ void lock_page_memcg(struct page *page) memcg->move_lock_task = current; memcg->move_lock_flags = flags; } +EXPORT_SYMBOL(folio_memcg_lock); + +void lock_page_memcg(struct page *page) +{ + folio_memcg_lock(page_folio(page)); +} EXPORT_SYMBOL(lock_page_memcg); -static void __unlock_page_memcg(struct mem_cgroup *memcg) +static void __memcg_unlock(struct mem_cgroup *memcg) { if (memcg && memcg->move_lock_task == current) { unsigned long flags = memcg->move_lock_flags; @@ -2030,14 +2035,22 @@ static void __unlock_page_memcg(struct mem_cgroup *memcg) } /** - * unlock_page_memcg - unlock a page and memcg binding - * @page: the page + * folio_memcg_unlock - Release the binding between a folio and its memcg. + * @folio: The folio. + * + * This releases the binding created by folio_memcg_lock(). This does + * not change the accounting of this folio to its memcg, but it does + * permit others to change it. */ -void unlock_page_memcg(struct page *page) +void folio_memcg_unlock(struct folio *folio) { - struct page *head = compound_head(page); + __memcg_unlock(folio_memcg(folio)); +} +EXPORT_SYMBOL(folio_memcg_unlock); - __unlock_page_memcg(page_memcg(head)); +void unlock_page_memcg(struct page *page) +{ + folio_memcg_unlock(page_folio(page)); } EXPORT_SYMBOL(unlock_page_memcg); @@ -5661,7 +5674,7 @@ static int mem_cgroup_move_account(struct page *page, page->memcg_data = (unsigned long)to; - __unlock_page_memcg(from); + __memcg_unlock(from); ret = 0; nid = page_to_nid(page); -- 2.30.2