From: "Vishal Moola (Oracle)" <vishal.moola@gmail.com>
To: Andrew Morton <akpm@linux-foundation.org>,
Matthew Wilcox <willy@infradead.org>
Cc: linux-arch@vger.kernel.org, linux-s390@vger.kernel.org,
Claudio Imbrenda <imbrenda@linux.ibm.com>,
kvm@vger.kernel.org, linux-openrisc@vger.kernel.org,
linux-hexagon@vger.kernel.org, linux-sh@vger.kernel.org,
linux-um@lists.infradead.org, linux-mips@vger.kernel.org,
linux-csky@vger.kernel.org,
"Vishal Moola \(Oracle\)" <vishal.moola@gmail.com>,
linux-mm@kvack.org, linux-m68k@lists.linux-m68k.org,
loongarch@lists.linux.dev, sparclinux@vger.kernel.org,
xen-devel@lists.xenproject.org, linux-riscv@lists.infradead.org,
David Hildenbrand <david@redhat.com>,
linuxppc-dev@lists.ozlabs.org,
linux-arm-kernel@lists.infradead.org
Subject: [PATCH v3 03/34] s390: Use pt_frag_refcount for pagetables
Date: Wed, 31 May 2023 14:30:01 -0700 [thread overview]
Message-ID: <20230531213032.25338-4-vishal.moola@gmail.com> (raw)
In-Reply-To: <20230531213032.25338-1-vishal.moola@gmail.com>
s390 currently uses _refcount to identify fragmented page tables.
The page table struct already has a member pt_frag_refcount used by
powerpc, so have s390 use that instead of the _refcount field as well.
This improves the safety for _refcount and the page table tracking.
This also allows us to simplify the tracking since we can once again use
the lower byte of pt_frag_refcount instead of the upper byte of _refcount.
Signed-off-by: Vishal Moola (Oracle) <vishal.moola@gmail.com>
---
arch/s390/mm/pgalloc.c | 38 +++++++++++++++-----------------------
1 file changed, 15 insertions(+), 23 deletions(-)
diff --git a/arch/s390/mm/pgalloc.c b/arch/s390/mm/pgalloc.c
index 66ab68db9842..6b99932abc66 100644
--- a/arch/s390/mm/pgalloc.c
+++ b/arch/s390/mm/pgalloc.c
@@ -182,20 +182,17 @@ void page_table_free_pgste(struct page *page)
* As follows from the above, no unallocated or fully allocated parent
* pages are contained in mm_context_t::pgtable_list.
*
- * The upper byte (bits 24-31) of the parent page _refcount is used
+ * The lower byte (bits 0-7) of the parent page pt_frag_refcount is used
* for tracking contained 2KB-pgtables and has the following format:
*
* PP AA
- * 01234567 upper byte (bits 24-31) of struct page::_refcount
+ * 01234567 upper byte (bits 0-7) of struct page::pt_frag_refcount
* || ||
* || |+--- upper 2KB-pgtable is allocated
* || +---- lower 2KB-pgtable is allocated
* |+------- upper 2KB-pgtable is pending for removal
* +-------- lower 2KB-pgtable is pending for removal
*
- * (See commit 620b4e903179 ("s390: use _refcount for pgtables") on why
- * using _refcount is possible).
- *
* When 2KB-pgtable is allocated the corresponding AA bit is set to 1.
* The parent page is either:
* - added to mm_context_t::pgtable_list in case the second half of the
@@ -243,11 +240,12 @@ unsigned long *page_table_alloc(struct mm_struct *mm)
if (!list_empty(&mm->context.pgtable_list)) {
page = list_first_entry(&mm->context.pgtable_list,
struct page, lru);
- mask = atomic_read(&page->_refcount) >> 24;
+ mask = atomic_read(&page->pt_frag_refcount);
/*
* The pending removal bits must also be checked.
* Failure to do so might lead to an impossible
- * value of (i.e 0x13 or 0x23) written to _refcount.
+ * value of (i.e 0x13 or 0x23) written to
+ * pt_frag_refcount.
* Such values violate the assumption that pending and
* allocation bits are mutually exclusive, and the rest
* of the code unrails as result. That could lead to
@@ -259,8 +257,8 @@ unsigned long *page_table_alloc(struct mm_struct *mm)
bit = mask & 1; /* =1 -> second 2K */
if (bit)
table += PTRS_PER_PTE;
- atomic_xor_bits(&page->_refcount,
- 0x01U << (bit + 24));
+ atomic_xor_bits(&page->pt_frag_refcount,
+ 0x01U << bit);
list_del(&page->lru);
}
}
@@ -281,12 +279,12 @@ unsigned long *page_table_alloc(struct mm_struct *mm)
table = (unsigned long *) page_to_virt(page);
if (mm_alloc_pgste(mm)) {
/* Return 4K page table with PGSTEs */
- atomic_xor_bits(&page->_refcount, 0x03U << 24);
+ atomic_xor_bits(&page->pt_frag_refcount, 0x03U);
memset64((u64 *)table, _PAGE_INVALID, PTRS_PER_PTE);
memset64((u64 *)table + PTRS_PER_PTE, 0, PTRS_PER_PTE);
} else {
/* Return the first 2K fragment of the page */
- atomic_xor_bits(&page->_refcount, 0x01U << 24);
+ atomic_xor_bits(&page->pt_frag_refcount, 0x01U);
memset64((u64 *)table, _PAGE_INVALID, 2 * PTRS_PER_PTE);
spin_lock_bh(&mm->context.lock);
list_add(&page->lru, &mm->context.pgtable_list);
@@ -323,22 +321,19 @@ void page_table_free(struct mm_struct *mm, unsigned long *table)
* will happen outside of the critical section from this
* function or from __tlb_remove_table()
*/
- mask = atomic_xor_bits(&page->_refcount, 0x11U << (bit + 24));
- mask >>= 24;
+ mask = atomic_xor_bits(&page->pt_frag_refcount, 0x11U << bit);
if (mask & 0x03U)
list_add(&page->lru, &mm->context.pgtable_list);
else
list_del(&page->lru);
spin_unlock_bh(&mm->context.lock);
- mask = atomic_xor_bits(&page->_refcount, 0x10U << (bit + 24));
- mask >>= 24;
+ mask = atomic_xor_bits(&page->pt_frag_refcount, 0x10U << bit);
if (mask != 0x00U)
return;
half = 0x01U << bit;
} else {
half = 0x03U;
- mask = atomic_xor_bits(&page->_refcount, 0x03U << 24);
- mask >>= 24;
+ mask = atomic_xor_bits(&page->pt_frag_refcount, 0x03U);
}
page_table_release_check(page, table, half, mask);
@@ -368,8 +363,7 @@ void page_table_free_rcu(struct mmu_gather *tlb, unsigned long *table,
* outside of the critical section from __tlb_remove_table() or from
* page_table_free()
*/
- mask = atomic_xor_bits(&page->_refcount, 0x11U << (bit + 24));
- mask >>= 24;
+ mask = atomic_xor_bits(&page->pt_frag_refcount, 0x11U << bit);
if (mask & 0x03U)
list_add_tail(&page->lru, &mm->context.pgtable_list);
else
@@ -391,14 +385,12 @@ void __tlb_remove_table(void *_table)
return;
case 0x01U: /* lower 2K of a 4K page table */
case 0x02U: /* higher 2K of a 4K page table */
- mask = atomic_xor_bits(&page->_refcount, mask << (4 + 24));
- mask >>= 24;
+ mask = atomic_xor_bits(&page->pt_frag_refcount, mask << 4);
if (mask != 0x00U)
return;
break;
case 0x03U: /* 4K page table with pgstes */
- mask = atomic_xor_bits(&page->_refcount, 0x03U << 24);
- mask >>= 24;
+ mask = atomic_xor_bits(&page->pt_frag_refcount, 0x03U);
break;
}
--
2.40.1
next prev parent reply other threads:[~2023-05-31 21:34 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-31 21:29 [PATCH v3 00/34] Split ptdesc from struct page Vishal Moola (Oracle)
2023-05-31 21:29 ` [PATCH v3 01/34] mm: Add PAGE_TYPE_OP folio functions Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 02/34] s390: Use _pt_s390_gaddr for gmap address tracking Vishal Moola (Oracle)
2023-05-31 21:30 ` Vishal Moola (Oracle) [this message]
2023-06-01 13:19 ` [PATCH v3 03/34] s390: Use pt_frag_refcount for pagetables Gerald Schaefer
2023-06-01 20:43 ` Vishal Moola
2023-05-31 21:30 ` [PATCH v3 04/34] pgtable: Create struct ptdesc Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 05/34] mm: add utility functions for ptdesc Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 06/34] mm: Convert pmd_pgtable_page() to pmd_ptdesc() Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 07/34] mm: Convert ptlock_alloc() to use ptdescs Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 08/34] mm: Convert ptlock_ptr() " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 09/34] mm: Convert pmd_ptlock_init() " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 10/34] mm: Convert ptlock_init() " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 11/34] mm: Convert pmd_ptlock_free() " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 12/34] mm: Convert ptlock_free() " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 13/34] mm: Create ptdesc equivalents for pgtable_{pte,pmd}_page_{ctor,dtor} Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 14/34] powerpc: Convert various functions to use ptdescs Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 15/34] x86: " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 16/34] s390: Convert various gmap " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 17/34] s390: Convert various pgalloc " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 18/34] mm: Remove page table members from struct page Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 19/34] pgalloc: Convert various functions to use ptdescs Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 20/34] arm: " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 21/34] arm64: " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 22/34] csky: Convert __pte_free_tlb() " Vishal Moola (Oracle)
2023-06-01 4:36 ` Guo Ren
2023-05-31 21:30 ` [PATCH v3 23/34] hexagon: " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 24/34] loongarch: Convert various functions " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 25/34] m68k: " Vishal Moola (Oracle)
2023-06-01 7:40 ` Geert Uytterhoeven
2023-06-01 8:20 ` Vishal Moola
2023-06-01 14:18 ` kernel test robot
2023-05-31 21:30 ` [PATCH v3 26/34] mips: " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 27/34] nios2: Convert __pte_free_tlb() " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 28/34] openrisc: " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 29/34] riscv: Convert alloc_{pmd, pte}_late() " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 30/34] sh: Convert pte_free_tlb() " Vishal Moola (Oracle)
2023-06-01 7:20 ` Geert Uytterhoeven
2023-06-01 7:28 ` John Paul Adrian Glaubitz
2023-06-01 7:42 ` Geert Uytterhoeven
2023-06-01 7:46 ` John Paul Adrian Glaubitz
2023-06-01 8:22 ` Vishal Moola
2023-06-08 10:12 ` John Paul Adrian Glaubitz
2023-05-31 21:30 ` [PATCH v3 31/34] sparc64: Convert various functions " Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 32/34] sparc: Convert pgtable_pte_page_{ctor, dtor}() to ptdesc equivalents Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 33/34] um: Convert {pmd, pte}_free_tlb() to use ptdescs Vishal Moola (Oracle)
2023-05-31 21:30 ` [PATCH v3 34/34] mm: Remove pgtable_{pmd, pte}_page_{ctor, dtor}() wrappers Vishal Moola (Oracle)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230531213032.25338-4-vishal.moola@gmail.com \
--to=vishal.moola@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=david@redhat.com \
--cc=imbrenda@linux.ibm.com \
--cc=kvm@vger.kernel.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-csky@vger.kernel.org \
--cc=linux-hexagon@vger.kernel.org \
--cc=linux-m68k@lists.linux-m68k.org \
--cc=linux-mips@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-openrisc@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=linux-s390@vger.kernel.org \
--cc=linux-sh@vger.kernel.org \
--cc=linux-um@lists.infradead.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=loongarch@lists.linux.dev \
--cc=sparclinux@vger.kernel.org \
--cc=willy@infradead.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).