[PATCH 0/8] Restructure struct page

[PATCH 0/8] Restructure struct page

[Matthew Wilcox]

From: Matthew Wilcox <mawilcox@microsoft.com>

This series does not attempt any grand restructuring as I proposed last
week.  Instead, it cures the worst of the indentitis, fixes the
documentation and reduces the ifdeffery.  The only layout change is
compound_dtor and compound_order are each reduced to one byte.  At
least, that's my intent.  

Here's a diff from pahole's output:

--- old-struct-page	2017-12-16 09:58:09.653936791 -0500
+++ new-struct-page	2017-12-16 09:58:32.009832964 -0500
@@ -11,17 +11,15 @@
 	};                                               /*    16     8 */
 	union {
 		long unsigned int  counters;             /*           8 */
+		unsigned int       active;               /*           4 */
 		struct {
-			union {
-				atomic_t _mapcount;      /*           4 */
-				unsigned int active;     /*           4 */
-				struct {
-					unsigned int inuse:16; /*    24:16  4 */
-					unsigned int objects:15; /*    24: 1  4 */
-					unsigned int frozen:1; /*    24: 0  4 */
-				};                       /*           4 */
-				int units;               /*           4 */
-			};                               /*    24     4 */
+			unsigned int inuse:16;           /*    24:16  4 */
+			unsigned int objects:15;         /*    24: 1  4 */
+			unsigned int frozen:1;           /*    24: 0  4 */
+		};                                       /*           4 */
+		int                units;                /*           4 */
+		struct {
+			atomic_t   _mapcount;            /*    24     4 */
 			atomic_t   _refcount;            /*    28     4 */
 		};                                       /*           8 */
 	};                                               /*    24     8 */
@@ -36,8 +34,8 @@
 		struct callback_head callback_head;      /*          16 */
 		struct {
 			long unsigned int compound_head; /*    32     8 */
-			unsigned int compound_dtor;      /*    40     4 */
-			unsigned int compound_order;     /*    44     4 */
+			unsigned char compound_dtor;     /*    40     1 */
+			unsigned char compound_order;    /*    41     1 */
 		};                                       /*          16 */
 		struct {
 			long unsigned int __pad;         /*    32     8 */

Matthew Wilcox (8):
  mm: Align struct page more aesthetically
  mm: De-indent struct page
  mm: Remove misleading alignment claims
  mm: Improve comment on page->mapping
  mm: Introduce _slub_counter_t
  mm: Store compound_dtor / compound_order as bytes
  mm: Document how to use struct page
  mm: Remove reference to PG_buddy

 include/linux/mm_types.h | 153 ++++++++++++++++++++++-------------------------
 1 file changed, 73 insertions(+), 80 deletions(-)

-- 
2.15.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 1/8] mm: Align struct page more aesthetically

[Matthew Wilcox]

From: Matthew Wilcox <mawilcox@microsoft.com>

instead of an ifdef block at the end of the struct, which needed
its own comment, define _struct_page_alignment up at the top where it
fits nicely with the existing comment.

Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
---
 include/linux/mm_types.h | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index cfd0ac4e5e0e..4509f0cfaf39 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -39,6 +39,12 @@ struct hmm;
  * allows the use of atomic double word operations on the flags/mapping
  * and lru list pointers also.
  */
+#ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
+#define _struct_page_alignment	__aligned(2 * sizeof(unsigned long))
+#else
+#define _struct_page_alignment
+#endif
+
 struct page {
 	/* First double word block */
 	unsigned long flags;		/* Atomic flags, some possibly
@@ -212,15 +218,7 @@ struct page {
 #ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS
 	int _last_cpupid;
 #endif
-}
-/*
- * The struct page can be forced to be double word aligned so that atomic ops
- * on double words work. The SLUB allocator can make use of such a feature.
- */
-#ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
-	__aligned(2 * sizeof(unsigned long))
-#endif
-;
+} _struct_page_alignment;
 
 #define PAGE_FRAG_CACHE_MAX_SIZE	__ALIGN_MASK(32768, ~PAGE_MASK)
 #define PAGE_FRAG_CACHE_MAX_ORDER	get_order(PAGE_FRAG_CACHE_MAX_SIZE)
-- 
2.15.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 2/8] mm: De-indent struct page

[Matthew Wilcox]

From: Matthew Wilcox <mawilcox@microsoft.com>

I found the struct { union { struct { union { struct { } } } } }
layout rather confusing.  Fortunately, there is an easier way to write
this.  The innermost union is of four things which are the size of an
int, so the ones which are used by slab/slob/slub can be pulled up
two levels to be in the outermost union with 'counters'.  That leaves
us with struct { union { struct { atomic_t; atomic_t; } } } which
has the same layout, but is easier to read.

Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
---
 include/linux/mm_types.h | 40 +++++++++++++++++++---------------------
 1 file changed, 19 insertions(+), 21 deletions(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 4509f0cfaf39..27973166af28 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -84,28 +84,26 @@ struct page {
 		 */
 		unsigned counters;
 #endif
-		struct {
+		unsigned int active;		/* SLAB */
+		struct {			/* SLUB */
+			unsigned inuse:16;
+			unsigned objects:15;
+			unsigned frozen:1;
+		};
+		int units;			/* SLOB */
+
+		struct {			/* Page cache */
+			/*
+			 * Count of ptes mapped in mms, to show when
+			 * page is mapped & limit reverse map searches.
+			 *
+			 * Extra information about page type may be
+			 * stored here for pages that are never mapped,
+			 * in which case the value MUST BE <= -2.
+			 * See page-flags.h for more details.
+			 */
+			atomic_t _mapcount;
 
-			union {
-				/*
-				 * Count of ptes mapped in mms, to show when
-				 * page is mapped & limit reverse map searches.
-				 *
-				 * Extra information about page type may be
-				 * stored here for pages that are never mapped,
-				 * in which case the value MUST BE <= -2.
-				 * See page-flags.h for more details.
-				 */
-				atomic_t _mapcount;
-
-				unsigned int active;		/* SLAB */
-				struct {			/* SLUB */
-					unsigned inuse:16;
-					unsigned objects:15;
-					unsigned frozen:1;
-				};
-				int units;			/* SLOB */
-			};
 			/*
 			 * Usage count, *USE WRAPPER FUNCTION* when manual
 			 * accounting. See page_ref.h
-- 
2.15.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 3/8] mm: Remove misleading alignment claims

[Matthew Wilcox]

From: Matthew Wilcox <mawilcox@microsoft.com>

The "third double word block" isn't on 32-bit systems.  The layout
looks like this:

	unsigned long flags;
	struct address_space *mapping
	pgoff_t index;
	atomic_t _mapcount;
	atomic_t _refcount;

which is 32 bytes on 64-bit, but 20 bytes on 32-bit.  Nobody is trying
to use the fact that it's double-word aligned today, so just remove the
misleading claims.

Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
---
 include/linux/mm_types.h | 13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 27973166af28..c2294e6204e8 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -33,11 +33,11 @@ struct hmm;
  * a page, though if it is a pagecache page, rmap structures can tell us
  * who is mapping it.
  *
- * The objects in struct page are organized in double word blocks in
- * order to allows us to use atomic double word operations on portions
- * of struct page. That is currently only used by slub but the arrangement
- * allows the use of atomic double word operations on the flags/mapping
- * and lru list pointers also.
+ * SLUB uses cmpxchg_double() to atomically update its freelist and
+ * counters.  That requires that freelist & counters be adjacent and
+ * double-word aligned.  We align all struct pages to double-word
+ * boundaries, and ensure that 'freelist' is aligned within the
+ * struct.
  */
 #ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
 #define _struct_page_alignment	__aligned(2 * sizeof(unsigned long))
@@ -113,8 +113,6 @@ struct page {
 	};
 
 	/*
-	 * Third double word block
-	 *
 	 * WARNING: bit 0 of the first word encode PageTail(). That means
 	 * the rest users of the storage space MUST NOT use the bit to
 	 * avoid collision and false-positive PageTail().
@@ -175,7 +173,6 @@ struct page {
 #endif
 	};
 
-	/* Remainder is not double word aligned */
 	union {
 		unsigned long private;		/* Mapping-private opaque data:
 					 	 * usually used for buffer_heads
-- 
2.15.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 4/8] mm: Improve comment on page->mapping

[Matthew Wilcox]

From: Matthew Wilcox <mawilcox@microsoft.com>

The comment on page->mapping is terse, and out of date (it does not
mention the possibility of PAGE_MAPPING_MOVABLE).  Instead, point
the interested reader to page-flags.h where there is a much better
comment.

Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
---
 include/linux/mm_types.h | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index c2294e6204e8..8c3b8cea22ee 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -50,15 +50,9 @@ struct page {
 	unsigned long flags;		/* Atomic flags, some possibly
 					 * updated asynchronously */
 	union {
-		struct address_space *mapping;	/* If low bit clear, points to
-						 * inode address_space, or NULL.
-						 * If page mapped as anonymous
-						 * memory, low bit is set, and
-						 * it points to anon_vma object
-						 * or KSM private structure. See
-						 * PAGE_MAPPING_ANON and
-						 * PAGE_MAPPING_KSM.
-						 */
+		/* See page-flags.h for the definition of PAGE_MAPPING_FLAGS */
+		struct address_space *mapping;
+
 		void *s_mem;			/* slab first object */
 		atomic_t compound_mapcount;	/* first tail page */
 		/* page_deferred_list().next	 -- second tail page */
-- 
2.15.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 5/8] mm: Introduce _slub_counter_t

[Matthew Wilcox]

From: Matthew Wilcox <mawilcox@microsoft.com>

Instead of putting the ifdef in the middle of the definition of struct
page, pull it forward to the rest of the ifdeffery around the SLUB
cmpxchg_double optimisation.

Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
---
 include/linux/mm_types.h | 21 ++++++++-------------
 1 file changed, 8 insertions(+), 13 deletions(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 8c3b8cea22ee..5521c9799c50 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -41,9 +41,15 @@ struct hmm;
  */
 #ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
 #define _struct_page_alignment	__aligned(2 * sizeof(unsigned long))
+#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE)
+#define _slub_counter_t		unsigned long
 #else
-#define _struct_page_alignment
+#define _slub_counter_t		unsigned int
 #endif
+#else /* !CONFIG_HAVE_ALIGNED_STRUCT_PAGE */
+#define _struct_page_alignment
+#define _slub_counter_t		unsigned int
+#endif /* !CONFIG_HAVE_ALIGNED_STRUCT_PAGE */
 
 struct page {
 	/* First double word block */
@@ -66,18 +72,7 @@ struct page {
 	};
 
 	union {
-#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE) && \
-	defined(CONFIG_HAVE_ALIGNED_STRUCT_PAGE)
-		/* Used for cmpxchg_double in slub */
-		unsigned long counters;
-#else
-		/*
-		 * Keep _refcount separate from slub cmpxchg_double data.
-		 * As the rest of the double word is protected by slab_lock
-		 * but _refcount is not.
-		 */
-		unsigned counters;
-#endif
+		_slub_counter_t counters;
 		unsigned int active;		/* SLAB */
 		struct {			/* SLUB */
 			unsigned inuse:16;
-- 
2.15.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 6/8] mm: Store compound_dtor / compound_order as bytes

[Matthew Wilcox]

From: Matthew Wilcox <mawilcox@microsoft.com>

Neither of these values get even close to 256; compound_dtor is
currently at a maximum of 3, and compound_order can't be over 64.
No machine has inefficient access to bytes since EV5, and while
those are still supported, we don't optimise for them any more.

Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
---
 include/linux/mm_types.h | 15 ++-------------
 1 file changed, 2 insertions(+), 13 deletions(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 5521c9799c50..1a3ba1f1605d 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -136,19 +136,8 @@ struct page {
 			unsigned long compound_head; /* If bit zero is set */
 
 			/* First tail page only */
-#ifdef CONFIG_64BIT
-			/*
-			 * On 64 bit system we have enough space in struct page
-			 * to encode compound_dtor and compound_order with
-			 * unsigned int. It can help compiler generate better or
-			 * smaller code on some archtectures.
-			 */
-			unsigned int compound_dtor;
-			unsigned int compound_order;
-#else
-			unsigned short int compound_dtor;
-			unsigned short int compound_order;
-#endif
+			unsigned char compound_dtor;
+			unsigned char compound_order;
 		};
 
 #if defined(CONFIG_TRANSPARENT_HUGEPAGE) && USE_SPLIT_PMD_PTLOCKS
-- 
2.15.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 7/8] mm: Document how to use struct page

[Matthew Wilcox]

From: Matthew Wilcox <mawilcox@microsoft.com>

Be really explicit about what bits / bytes are reserved for users that
want to store extra information about the pages they allocate.

Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
---
 include/linux/mm_types.h | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 1a3ba1f1605d..a517d210f177 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -31,7 +31,28 @@ struct hmm;
  * it to keep track of whatever it is we are using the page for at the
  * moment. Note that we have no way to track which tasks are using
  * a page, though if it is a pagecache page, rmap structures can tell us
- * who is mapping it.
+ * who is mapping it. If you allocate the page using alloc_pages(), you
+ * can use some of the space in struct page for your own purposes.
+ *
+ * Pages that were once in the page cache may be found under the RCU lock
+ * even after they have been recycled to a different purpose.  The page cache
+ * will read and writes some of the fields in struct page to lock the page,
+ * then check that it's still in the page cache.  It is vital that all users
+ * of struct page:
+ * 1. Use the first word as PageFlags.
+ * 2. Clear or preserve bit 0 of page->compound_head.  It is used as
+ *    PageTail for compound pages, and the page cache must not see false
+ *    positives.  Some users put a pointer here (guaranteed to be at least
+ *    4-byte aligned), other users avoid using the word altogether.
+ * 3. page->_refcount must either not be used, or must be used in such a
+ *    way that other CPUs temporarily incrementing and then decrementing the
+ *    refcount does not cause problems.  On receiving the page from
+ *    alloc_pages(), the refcount will be positive.
+ *
+ * If you allocate pages of order > 0, you can use the fields in the struct
+ * page associated with each page, but bear in mind that the pages may have
+ * been inserted individually into the page cache, so you must use the above
+ * three fields in a compatible way for each struct page.
  *
  * SLUB uses cmpxchg_double() to atomically update its freelist and
  * counters.  That requires that freelist & counters be adjacent and
-- 
2.15.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 8/8] mm: Remove reference to PG_buddy

[Matthew Wilcox]

From: Matthew Wilcox <mawilcox@microsoft.com>

PG_buddy doesn't exist any more.  It's called PageBuddy now.

Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
---
 include/linux/mm_types.h | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index a517d210f177..06f16a451a53 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -173,13 +173,14 @@ struct page {
 	};
 
 	union {
-		unsigned long private;		/* Mapping-private opaque data:
-					 	 * usually used for buffer_heads
-						 * if PagePrivate set; used for
-						 * swp_entry_t if PageSwapCache;
-						 * indicates order in the buddy
-						 * system if PG_buddy is set.
-						 */
+		/*
+		 * Mapping-private opaque data:
+		 * Usually used for buffer_heads if PagePrivate
+		 * Used for swp_entry_t if PageSwapCache
+		 * Indicates order in the buddy system if PageBuddy
+		 */
+		unsigned long private;
+
 #if USE_SPLIT_PTE_PTLOCKS
 #if ALLOC_SPLIT_PTLOCKS
 		spinlock_t *ptl;
-- 
2.15.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/8] Restructure struct page

[Matthew Wilcox]

On Wed, Dec 20, 2017 at 07:52:48AM -0800, Matthew Wilcox wrote:
> From: Matthew Wilcox <mawilcox@microsoft.com>
> 
> This series does not attempt any grand restructuring as I proposed last
> week.  Instead, it cures the worst of the indentitis, fixes the
> documentation and reduces the ifdeffery.  The only layout change is
> compound_dtor and compound_order are each reduced to one byte.  At
> least, that's my intent.  

My apologies.  I typod my git send-email and resent v1, instead of sending v2.

"A computer lets you make more mistakes faster than any other invention
with the possible exceptions of handguns and Tequila." a?? Mitch Ratcliffe

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 7/8] mm: Document how to use struct page

[Matthew Wilcox]

On Tue, Dec 19, 2017 at 09:53:16AM -0600, Christopher Lameter wrote:
> On Sat, 16 Dec 2017, Matthew Wilcox wrote:
> 
> > + * If you allocate pages of order > 0, you can use the fields in the struct
> > + * page associated with each page, but bear in mind that the pages may have
> > + * been inserted individually into the page cache, so you must use the above
> > + * three fields in a compatible way for each struct page.
> 
> If they are inserted into the page cache then also other fields are
> required like the lru one right?

The page cache won't touch the LRU field until it has at least pinned
the page.  This text is explaining what may happen to a page during
the process of the page cache trying (and failing) to pin the page,
which can happen after your driver has allocated the page.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 7/8] mm: Document how to use struct page

[Kirill A. Shutemov]

On Tue, Dec 19, 2017 at 09:53:16AM -0600, Christopher Lameter wrote:
> On Sat, 16 Dec 2017, Matthew Wilcox wrote:
> 
> > + * If you allocate pages of order > 0, you can use the fields in the struct
> > + * page associated with each page, but bear in mind that the pages may have
> > + * been inserted individually into the page cache, so you must use the above
> > + * three fields in a compatible way for each struct page.
> 
> If they are inserted into the page cache then also other fields are
> required like the lru one right?

No. For compound pages, only head is on lru.

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 7/8] mm: Document how to use struct page

[Christopher Lameter]

On Sat, 16 Dec 2017, Matthew Wilcox wrote:

> + * If you allocate pages of order > 0, you can use the fields in the struct
> + * page associated with each page, but bear in mind that the pages may have
> + * been inserted individually into the page cache, so you must use the above
> + * three fields in a compatible way for each struct page.

If they are inserted into the page cache then also other fields are
required like the lru one right?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 7/8] mm: Document how to use struct page

[Michal Hocko]

On Tue 19-12-17 16:43:00, Kirill A. Shutemov wrote:
> On Tue, Dec 19, 2017 at 05:07:03AM -0800, Matthew Wilcox wrote:
> > I'm also teaching myself more about ReStructuredText, and to that end I've
> > started to document all these pages side-by-side in a table.
> 
> Maybe we should get the table to the point where we can generate C description
> of the struct and stop writing in C manually? :)

Heh, autogenerated and compressed struct page sounds like a heaven
musing to me :p

But seriously, your original patch sounds like a good start. Let's add
more on top (I haven't checked your table throughly yet and it will
likely take some time). Document piece at a time would likely result
in having the reasonable outcome sooner.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 7/8] mm: Document how to use struct page

[Kirill A. Shutemov]

On Tue, Dec 19, 2017 at 05:07:03AM -0800, Matthew Wilcox wrote:
> I'm also teaching myself more about ReStructuredText, and to that end I've
> started to document all these pages side-by-side in a table.

Maybe we should get the table to the point where we can generate C description
of the struct and stop writing in C manually? :)

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 7/8] mm: Document how to use struct page

[Matthew Wilcox]

On Tue, Dec 19, 2017 at 10:59:27AM +0100, Michal Hocko wrote:
> On Sat 16-12-17 08:44:24, Matthew Wilcox wrote:
> > From: Matthew Wilcox <mawilcox@microsoft.com>
> > 
> > Be really explicit about what bits / bytes are reserved for users that
> > want to store extra information about the pages they allocate.
> > 
> > Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
> 
> I think that struct page would benefit from more documentation. But this
> looks good to me already. Hugetlb pages abuse some fields in page[1],
> page_is_pfmemalloc is abusing index and there are probably more. It
> would be great to have all those described at the single place. I will
> update hugetlb part along with my recent patches which are in RFC right
> now. Maybe a good project for somebody who wants to learn a lot about MM
> and interaction with other subsystems (or maybe not ;))
> 
> Acked-by: Michal Hocko <mhocko@suse.com>

Thanks!  Completely agree that what I have written here so far reflects
my own limited understanding of the MM.  Kirill's been really patient
with teaching me some of the things I didn't know, so it seems only fair
to write down what I do know so it doesn't have to be explained to the
next eager developer who isn't steeped in the mythos of the MM system.

I'm also teaching myself more about ReStructuredText, and to that end I've
started to document all these pages side-by-side in a table.  Here's what
I have so far (and I know it's incomplete):

+---+-----------+-----------+--------------+----------+--------+--------------+
| B | slab      | pagecache | tail 1       | anon     | tail 2 | hugetlb      |
+===+===========+===========+==============+==========+========+==============+
| 0 | flags                                                                   |
+---+                                                                         |
| 4 |                                                                         |
+---+-----------+-----------+--------------+----------+--------+--------------+
| 8 | s_mem     | mapping   | cmp_mapcount | anon_vma | defer  | mapping      |
+---+           |           +--------------+          | list   |              |
|12 |           |           |              |          |        |              |
+---+-----------+-----------+--------------+----------+        +--------------+
|16 | freelist  | index                               |        | index        |
+---+           |                                     |        | (shifted)    |
|20 |           |                                     |        |              |
+---+-----------+-------------------------------------+--------+--------------+
|24 | counters  | mapcount                                                    |
+---+           +-----------+--------------+----------+--------+--------------+
|28 |           | refcount  |              |          |        | refcount     |
+---+-----------+-----------+--------------+----------+--------+--------------+
|32 | next      | lru       | cmpd_head    |                                  |
+---+           |           |              +----------------------------------+
|36 |           |           |              |                                  |
+---+-----------+           +--------------+----------------------------------+
|40 | pages     |           | dtor / order |                                  |
+---+-----------+           +--------------+----------------------------------+
|44 | pobjects  |           |              |                                  |
+---+-----------+-----------+--------------+----------------------------------+
|48 | slb_cache | private   |              |                                  |
+---+           |           +--------------+----------------------------------+
|52 |           |           |              |                                  |
+---+-----------+-----------+--------------+----------------------------------+

Obviously it's simplified -- no mention of slub's use of rcu_head; no
column for page table allocations; I left off the mem_cgroup, virtual
and last_cpupid possibilities (intentionally); I don't know much about
anonymous pages yet; no mention of KSM pages; hugetlb is still mostly
a mystery to me.

I haven't even run it through an RST parser yet to see if this is a good
table ;-)

Once it is good, then I'll duplicate it for 32-bit.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 7/8] mm: Document how to use struct page

[Michal Hocko]

On Sat 16-12-17 08:44:24, Matthew Wilcox wrote:
> From: Matthew Wilcox <mawilcox@microsoft.com>
> 
> Be really explicit about what bits / bytes are reserved for users that
> want to store extra information about the pages they allocate.
> 
> Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>

I think that struct page would benefit from more documentation. But this
looks good to me already. Hugetlb pages abuse some fields in page[1],
page_is_pfmemalloc is abusing index and there are probably more. It
would be great to have all those described at the single place. I will
update hugetlb part along with my recent patches which are in RFC right
now. Maybe a good project for somebody who wants to learn a lot about MM
and interaction with other subsystems (or maybe not ;))

Acked-by: Michal Hocko <mhocko@suse.com>

> ---
>  include/linux/mm_types.h | 23 ++++++++++++++++++++++-
>  1 file changed, 22 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index 1a3ba1f1605d..a517d210f177 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -31,7 +31,28 @@ struct hmm;
>   * it to keep track of whatever it is we are using the page for at the
>   * moment. Note that we have no way to track which tasks are using
>   * a page, though if it is a pagecache page, rmap structures can tell us
> - * who is mapping it.
> + * who is mapping it. If you allocate the page using alloc_pages(), you
> + * can use some of the space in struct page for your own purposes.
> + *
> + * Pages that were once in the page cache may be found under the RCU lock
> + * even after they have been recycled to a different purpose.  The page cache
> + * will read and writes some of the fields in struct page to lock the page,
> + * then check that it's still in the page cache.  It is vital that all users
> + * of struct page:
> + * 1. Use the first word as PageFlags.
> + * 2. Clear or preserve bit 0 of page->compound_head.  It is used as
> + *    PageTail for compound pages, and the page cache must not see false
> + *    positives.  Some users put a pointer here (guaranteed to be at least
> + *    4-byte aligned), other users avoid using the word altogether.
> + * 3. page->_refcount must either not be used, or must be used in such a
> + *    way that other CPUs temporarily incrementing and then decrementing the
> + *    refcount does not cause problems.  On receiving the page from
> + *    alloc_pages(), the refcount will be positive.
> + *
> + * If you allocate pages of order > 0, you can use the fields in the struct
> + * page associated with each page, but bear in mind that the pages may have
> + * been inserted individually into the page cache, so you must use the above
> + * three fields in a compatible way for each struct page.
>   *
>   * SLUB uses cmpxchg_double() to atomically update its freelist and
>   * counters.  That requires that freelist & counters be adjacent and
> -- 
> 2.15.1
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 7/8] mm: Document how to use struct page

[Matthew Wilcox]

On Sat, Dec 16, 2017 at 09:47:16AM -0800, Randy Dunlap wrote:
> On 12/16/2017 08:44 AM, Matthew Wilcox wrote:
> > + * even after they have been recycled to a different purpose.  The page cache
> > + * will read and writes some of the fields in struct page to lock the page,
> 
> "will read and writes" seems awkward to me.
> Can that be:
>     * reads and writes

Sure!  I think I intended to write "will read and write", but fewer words is
usually better.

Thanks for reading.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 7/8] mm: Document how to use struct page

[Randy Dunlap]

On 12/16/2017 08:44 AM, Matthew Wilcox wrote:
> From: Matthew Wilcox <mawilcox@microsoft.com>
> 
> Be really explicit about what bits / bytes are reserved for users that
> want to store extra information about the pages they allocate.
> 
> Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
> ---
>  include/linux/mm_types.h | 23 ++++++++++++++++++++++-
>  1 file changed, 22 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index 1a3ba1f1605d..a517d210f177 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -31,7 +31,28 @@ struct hmm;
>   * it to keep track of whatever it is we are using the page for at the
>   * moment. Note that we have no way to track which tasks are using
>   * a page, though if it is a pagecache page, rmap structures can tell us
> - * who is mapping it.
> + * who is mapping it. If you allocate the page using alloc_pages(), you
> + * can use some of the space in struct page for your own purposes.
> + *
> + * Pages that were once in the page cache may be found under the RCU lock
> + * even after they have been recycled to a different purpose.  The page cache
> + * will read and writes some of the fields in struct page to lock the page,

"will read and writes" seems awkward to me.
Can that be:
    * reads and writes

> + * then check that it's still in the page cache.  It is vital that all users
> + * of struct page:
ta.
-- 
~Randy

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 7/8] mm: Document how to use struct page

[Matthew Wilcox]

From: Matthew Wilcox <mawilcox@microsoft.com>

Be really explicit about what bits / bytes are reserved for users that
want to store extra information about the pages they allocate.

Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
---
 include/linux/mm_types.h | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 1a3ba1f1605d..a517d210f177 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -31,7 +31,28 @@ struct hmm;
  * it to keep track of whatever it is we are using the page for at the
  * moment. Note that we have no way to track which tasks are using
  * a page, though if it is a pagecache page, rmap structures can tell us
- * who is mapping it.
+ * who is mapping it. If you allocate the page using alloc_pages(), you
+ * can use some of the space in struct page for your own purposes.
+ *
+ * Pages that were once in the page cache may be found under the RCU lock
+ * even after they have been recycled to a different purpose.  The page cache
+ * will read and writes some of the fields in struct page to lock the page,
+ * then check that it's still in the page cache.  It is vital that all users
+ * of struct page:
+ * 1. Use the first word as PageFlags.
+ * 2. Clear or preserve bit 0 of page->compound_head.  It is used as
+ *    PageTail for compound pages, and the page cache must not see false
+ *    positives.  Some users put a pointer here (guaranteed to be at least
+ *    4-byte aligned), other users avoid using the word altogether.
+ * 3. page->_refcount must either not be used, or must be used in such a
+ *    way that other CPUs temporarily incrementing and then decrementing the
+ *    refcount does not cause problems.  On receiving the page from
+ *    alloc_pages(), the refcount will be positive.
+ *
+ * If you allocate pages of order > 0, you can use the fields in the struct
+ * page associated with each page, but bear in mind that the pages may have
+ * been inserted individually into the page cache, so you must use the above
+ * three fields in a compatible way for each struct page.
  *
  * SLUB uses cmpxchg_double() to atomically update its freelist and
  * counters.  That requires that freelist & counters be adjacent and
-- 
2.15.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>