[PATCH v14 0/6] mm / virtio: Provide support for unused page reporting

[PATCH v14 0/6] mm / virtio: Provide support for unused page reporting

[Alexander Duyck]

This series provides an asynchronous means of reporting unused guest
pages to a hypervisor so that the memory associated with those pages can
be dropped and reused by other processes and/or guests on the host. Using
this it is possible to avoid unnecessary I/O to disk and greatly improve
performance in the case of memory overcommit on the host.

When enabled it will allocate a set of statistics to track the number of
reported pages. When the nr_free for a given free area is greater than
this by the high water mark we will schedule a worker to begin pulling the
non-reported memory and to provide it to the reporting interface via a
scatterlist.

Currently this is only in use by virtio-balloon however there is the hope
that at some point in the future other hypervisors might be able to make
use of it. In the virtio-balloon/QEMU implementation the hypervisor is
currently using MADV_DONTNEED to indicate to the host kernel that the page
is currently unused. It will be zeroed and faulted back into the guest the
next time the page is accessed.

To track if a page is reported or not the Uptodate flag was repurposed and
used as a Reported flag for Buddy pages. We walk though the free list
isolating pages and adding them to the scatterlist until we either
encounter the end of the list or have filled the scatterlist with pages to
be reported. If we fill the scatterlist before we reach the end of the
list we rotate the list so that the first unreported page we encounter is
moved to the head of the list as that is where we will resume after we
have freed the reported pages back into the tail of the list.

Below are the results from various benchmarks. I primarily focused on two
tests. The first is the will-it-scale/page_fault2 test, and the other is
a modified version of will-it-scale/page_fault1 that was enabled to use
THP. I did this as it allows for better visibility into different parts
of the memory subsystem. The guest is running with 32G for RAM on one
node of a E5-2630 v3. The host has had some power saving features disabled
by setting the /dev/cpu_dma_latency value to 10ms.

Test                page_fault1 (THP)     page_fault2
Name         tasks  Process Iter  STDEV  Process Iter  STDEV
Baseline         1    1203934.75  0.04%     379940.75  0.11%
                16    8828217.00  0.85%    3178653.00  1.28%

Patches applied  1    1207961.25  0.10%     380852.25  0.25%
                16    8862373.00  0.98%    3246397.25  0.68%

Patches enabled  1    1207758.75  0.17%     373079.25  0.60%
 MADV disabled  16    8870373.75  0.29%	   3204989.75  1.08%

Patches enabled  1    1261183.75  0.39%     373201.50  0.50%
                16    8371359.75  0.65%    3233665.50  0.84%

Patches enabled  1    1090201.50  0.25%     376967.25  0.29%
 page shuffle   16    8108719.75  0.58%    3218450.25  1.07%

The results above are for a baseline with a linux-next-20191115 kernel,
that kernel with this patch set applied but page reporting disabled in
virtio-balloon, patches applied but the madvise disabled by direct
assigning a device, the patches applied and page reporting fully
enabled, and the patches enabled with page shuffling enabled.  These
results include the deviation seen between the average value reported here
versus the high and/or low value. I observed that during the test memory
usage for the first three tests never dropped whereas with the patches
fully enabled the VM would drop to using only a few GB of the host's
memory when switching from memhog to page fault tests.

Most of the overhead seen with this patch set enabled seems due to page
faults caused by accessing the reported pages and the host zeroing the page
before giving it back to the guest. This overhead is much more visible when
using THP than with standard 4K pages. In addition page shuffling seemed to
increase the amount of faults generated due to an increase in memory churn.

The overall guest size is kept fairly small to only a few GB while the test
is running. If the host memory were oversubscribed this patch set should
result in a performance improvement as swapping memory in the host can be
avoided.

A brief history on the background of unused page reporting can be found at:
https://lore.kernel.org/lkml/29f43d5796feed0dec8e8bb98b187d9dac03b900.camel@linux.intel.com/

Changes from v12:
https://lore.kernel.org/lkml/20191022221223.17338.5860.stgit@localhost.localdomain/
Rebased on linux-next 20191031
Renamed page_is_reported to page_reported
Renamed add_page_to_reported_list to mark_page_reported
Dropped unused definition of add_page_to_reported_list for non-reporting case
Split free_area_reporting out from get_unreported_tail
Minor updates to cover page

Changes from v13:
https://lore.kernel.org/lkml/20191105215940.15144.65968.stgit@localhost.localdomain/
Rewrote core reporting functionality
  Merged patches 3 & 4
  Dropped boundary list and related code
  Folded get_reported_page into page_reporting_fill
  Folded page_reporting_fill into page_reporting_cycle
Pulled reporting functionality out of free_reported_page
  Renamed it to __free_isolated_page
  Moved page reporting specific bits to page_reporting_drain
Renamed phdev to prdev since we aren't "hinting" we are "reporting"
Added documentation to describe the usage of unused page reporting
Updated cover page and patch descriptions to avoid mention of boundary


---

Alexander Duyck (6):
      mm: Adjust shuffle code to allow for future coalescing
      mm: Use zone and order instead of free area in free_list manipulators
      mm: Introduce Reported pages
      mm: Add unused page reporting documentation
      virtio-balloon: Pull page poisoning config out of free page hinting
      virtio-balloon: Add support for providing unused page reports to host


 Documentation/vm/unused_page_reporting.rst |   44 ++++
 drivers/virtio/Kconfig                     |    1 
 drivers/virtio/virtio_balloon.c            |   88 +++++++
 include/linux/mmzone.h                     |   56 +----
 include/linux/page-flags.h                 |   11 +
 include/linux/page_reporting.h             |   31 +++
 include/uapi/linux/virtio_balloon.h        |    1 
 mm/Kconfig                                 |   11 +
 mm/Makefile                                |    1 
 mm/memory_hotplug.c                        |    2 
 mm/page_alloc.c                            |  181 +++++++++++----
 mm/page_reporting.c                        |  337 ++++++++++++++++++++++++++++
 mm/page_reporting.h                        |  125 ++++++++++
 mm/shuffle.c                               |   12 -
 mm/shuffle.h                               |    6 
 15 files changed, 805 insertions(+), 102 deletions(-)
 create mode 100644 Documentation/vm/unused_page_reporting.rst
 create mode 100644 include/linux/page_reporting.h
 create mode 100644 mm/page_reporting.c
 create mode 100644 mm/page_reporting.h

--

[PATCH v14 1/6] mm: Adjust shuffle code to allow for future coalescing

[Alexander Duyck]

From: Alexander Duyck <alexander.h.duyck@linux.intel.com>

Move the head/tail adding logic out of the shuffle code and into the
__free_one_page function since ultimately that is where it is really
needed anyway. By doing this we should be able to reduce the overhead
and can consolidate all of the list addition bits in one spot.

Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
---
 include/linux/mmzone.h |   12 --------
 mm/page_alloc.c        |   71 ++++++++++++++++++++++++++++--------------------
 mm/shuffle.c           |   12 ++++----
 mm/shuffle.h           |    6 ++++
 4 files changed, 54 insertions(+), 47 deletions(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 89d8ff06c9ce..245010b24747 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -116,18 +116,6 @@ static inline void add_to_free_area_tail(struct page *page, struct free_area *ar
 	area->nr_free++;
 }
 
-#ifdef CONFIG_SHUFFLE_PAGE_ALLOCATOR
-/* Used to preserve page allocation order entropy */
-void add_to_free_area_random(struct page *page, struct free_area *area,
-		int migratetype);
-#else
-static inline void add_to_free_area_random(struct page *page,
-		struct free_area *area, int migratetype)
-{
-	add_to_free_area(page, area, migratetype);
-}
-#endif
-
 /* Used for pages which are on another list */
 static inline void move_to_free_area(struct page *page, struct free_area *area,
 			     int migratetype)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 62dcd6b76c80..32e9cc092656 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -878,6 +878,36 @@ static inline struct capture_control *task_capc(struct zone *zone)
 #endif /* CONFIG_COMPACTION */
 
 /*
+ * If this is not the largest possible page, check if the buddy
+ * of the next-highest order is free. If it is, it's possible
+ * that pages are being freed that will coalesce soon. In case,
+ * that is happening, add the free page to the tail of the list
+ * so it's less likely to be used soon and more likely to be merged
+ * as a higher order page
+ */
+static inline bool
+buddy_merge_likely(unsigned long pfn, unsigned long buddy_pfn,
+		   struct page *page, unsigned int order)
+{
+	struct page *higher_page, *higher_buddy;
+	unsigned long combined_pfn;
+
+	if (order >= MAX_ORDER - 2)
+		return false;
+
+	if (!pfn_valid_within(buddy_pfn))
+		return false;
+
+	combined_pfn = buddy_pfn & pfn;
+	higher_page = page + (combined_pfn - pfn);
+	buddy_pfn = __find_buddy_pfn(combined_pfn, order + 1);
+	higher_buddy = higher_page + (buddy_pfn - combined_pfn);
+
+	return pfn_valid_within(buddy_pfn) &&
+	       page_is_buddy(higher_page, higher_buddy, order + 1);
+}
+
+/*
  * Freeing function for a buddy system allocator.
  *
  * The concept of a buddy system is to maintain direct-mapped table
@@ -906,11 +936,13 @@ static inline void __free_one_page(struct page *page,
 		struct zone *zone, unsigned int order,
 		int migratetype)
 {
-	unsigned long combined_pfn;
+	struct capture_control *capc = task_capc(zone);
 	unsigned long uninitialized_var(buddy_pfn);
-	struct page *buddy;
+	unsigned long combined_pfn;
+	struct free_area *area;
 	unsigned int max_order;
-	struct capture_control *capc = task_capc(zone);
+	struct page *buddy;
+	bool to_tail;
 
 	max_order = min_t(unsigned int, MAX_ORDER, pageblock_order + 1);
 
@@ -979,35 +1011,16 @@ static inline void __free_one_page(struct page *page,
 done_merging:
 	set_page_order(page, order);
 
-	/*
-	 * If this is not the largest possible page, check if the buddy
-	 * of the next-highest order is free. If it is, it's possible
-	 * that pages are being freed that will coalesce soon. In case,
-	 * that is happening, add the free page to the tail of the list
-	 * so it's less likely to be used soon and more likely to be merged
-	 * as a higher order page
-	 */
-	if ((order < MAX_ORDER-2) && pfn_valid_within(buddy_pfn)
-			&& !is_shuffle_order(order)) {
-		struct page *higher_page, *higher_buddy;
-		combined_pfn = buddy_pfn & pfn;
-		higher_page = page + (combined_pfn - pfn);
-		buddy_pfn = __find_buddy_pfn(combined_pfn, order + 1);
-		higher_buddy = higher_page + (buddy_pfn - combined_pfn);
-		if (pfn_valid_within(buddy_pfn) &&
-		    page_is_buddy(higher_page, higher_buddy, order + 1)) {
-			add_to_free_area_tail(page, &zone->free_area[order],
-					      migratetype);
-			return;
-		}
-	}
-
+	area = &zone->free_area[order];
 	if (is_shuffle_order(order))
-		add_to_free_area_random(page, &zone->free_area[order],
-				migratetype);
+		to_tail = shuffle_pick_tail();
 	else
-		add_to_free_area(page, &zone->free_area[order], migratetype);
+		to_tail = buddy_merge_likely(pfn, buddy_pfn, page, order);
 
+	if (to_tail)
+		add_to_free_area_tail(page, area, migratetype);
+	else
+		add_to_free_area(page, area, migratetype);
 }
 
 /*
diff --git a/mm/shuffle.c b/mm/shuffle.c
index b3fe97fd6654..e65d57f39486 100644
--- a/mm/shuffle.c
+++ b/mm/shuffle.c
@@ -183,11 +183,11 @@ void __meminit __shuffle_free_memory(pg_data_t *pgdat)
 		shuffle_zone(z);
 }
 
-void add_to_free_area_random(struct page *page, struct free_area *area,
-		int migratetype)
+bool shuffle_pick_tail(void)
 {
 	static u64 rand;
 	static u8 rand_bits;
+	bool ret;
 
 	/*
 	 * The lack of locking is deliberate. If 2 threads race to
@@ -198,10 +198,10 @@ void add_to_free_area_random(struct page *page, struct free_area *area,
 		rand = get_random_u64();
 	}
 
-	if (rand & 1)
-		add_to_free_area(page, area, migratetype);
-	else
-		add_to_free_area_tail(page, area, migratetype);
+	ret = rand & 1;
+
 	rand_bits--;
 	rand >>= 1;
+
+	return ret;
 }
diff --git a/mm/shuffle.h b/mm/shuffle.h
index 777a257a0d2f..4d79f03b6658 100644
--- a/mm/shuffle.h
+++ b/mm/shuffle.h
@@ -22,6 +22,7 @@ enum mm_shuffle_ctl {
 DECLARE_STATIC_KEY_FALSE(page_alloc_shuffle_key);
 extern void page_alloc_shuffle(enum mm_shuffle_ctl ctl);
 extern void __shuffle_free_memory(pg_data_t *pgdat);
+extern bool shuffle_pick_tail(void);
 static inline void shuffle_free_memory(pg_data_t *pgdat)
 {
 	if (!static_branch_unlikely(&page_alloc_shuffle_key))
@@ -44,6 +45,11 @@ static inline bool is_shuffle_order(int order)
 	return order >= SHUFFLE_ORDER;
 }
 #else
+static inline bool shuffle_pick_tail(void)
+{
+	return false;
+}
+
 static inline void shuffle_free_memory(pg_data_t *pgdat)
 {
 }

[PATCH v14 2/6] mm: Use zone and order instead of free area in free_list manipulators

[Alexander Duyck]

From: Alexander Duyck <alexander.h.duyck@linux.intel.com>

In order to enable the use of the zone from the list manipulator functions
I will need access to the zone pointer. As it turns out most of the
accessors were always just being directly passed &zone->free_area[order]
anyway so it would make sense to just fold that into the function itself
and pass the zone and order as arguments instead of the free area.

In order to be able to reference the zone we need to move the declaration
of the functions down so that we have the zone defined before we define the
list manipulation functions. Since the functions are only used in the file
mm/page_alloc.c we can just move them there to reduce noise in the header.

Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Pankaj Gupta <pagupta@redhat.com>
Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
---
 include/linux/mmzone.h |   32 -----------------------
 mm/page_alloc.c        |   67 +++++++++++++++++++++++++++++++++++-------------
 2 files changed, 49 insertions(+), 50 deletions(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 245010b24747..8d93106490f3 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -100,29 +100,6 @@ struct free_area {
 	unsigned long		nr_free;
 };
 
-/* Used for pages not on another list */
-static inline void add_to_free_area(struct page *page, struct free_area *area,
-			     int migratetype)
-{
-	list_add(&page->lru, &area->free_list[migratetype]);
-	area->nr_free++;
-}
-
-/* Used for pages not on another list */
-static inline void add_to_free_area_tail(struct page *page, struct free_area *area,
-				  int migratetype)
-{
-	list_add_tail(&page->lru, &area->free_list[migratetype]);
-	area->nr_free++;
-}
-
-/* Used for pages which are on another list */
-static inline void move_to_free_area(struct page *page, struct free_area *area,
-			     int migratetype)
-{
-	list_move(&page->lru, &area->free_list[migratetype]);
-}
-
 static inline struct page *get_page_from_free_area(struct free_area *area,
 					    int migratetype)
 {
@@ -130,15 +107,6 @@ static inline struct page *get_page_from_free_area(struct free_area *area,
 					struct page, lru);
 }
 
-static inline void del_page_from_free_area(struct page *page,
-		struct free_area *area)
-{
-	list_del(&page->lru);
-	__ClearPageBuddy(page);
-	set_page_private(page, 0);
-	area->nr_free--;
-}
-
 static inline bool free_area_empty(struct free_area *area, int migratetype)
 {
 	return list_empty(&area->free_list[migratetype]);
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 32e9cc092656..e0a7895300fb 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -877,6 +877,44 @@ static inline struct capture_control *task_capc(struct zone *zone)
 }
 #endif /* CONFIG_COMPACTION */
 
+/* Used for pages not on another list */
+static inline void add_to_free_list(struct page *page, struct zone *zone,
+				    unsigned int order, int migratetype)
+{
+	struct free_area *area = &zone->free_area[order];
+
+	list_add(&page->lru, &area->free_list[migratetype]);
+	area->nr_free++;
+}
+
+/* Used for pages not on another list */
+static inline void add_to_free_list_tail(struct page *page, struct zone *zone,
+					 unsigned int order, int migratetype)
+{
+	struct free_area *area = &zone->free_area[order];
+
+	list_add_tail(&page->lru, &area->free_list[migratetype]);
+	area->nr_free++;
+}
+
+/* Used for pages which are on another list */
+static inline void move_to_free_list(struct page *page, struct zone *zone,
+				     unsigned int order, int migratetype)
+{
+	struct free_area *area = &zone->free_area[order];
+
+	list_move(&page->lru, &area->free_list[migratetype]);
+}
+
+static inline void del_page_from_free_list(struct page *page, struct zone *zone,
+					   unsigned int order)
+{
+	list_del(&page->lru);
+	__ClearPageBuddy(page);
+	set_page_private(page, 0);
+	zone->free_area[order].nr_free--;
+}
+
 /*
  * If this is not the largest possible page, check if the buddy
  * of the next-highest order is free. If it is, it's possible
@@ -939,7 +977,6 @@ static inline void __free_one_page(struct page *page,
 	struct capture_control *capc = task_capc(zone);
 	unsigned long uninitialized_var(buddy_pfn);
 	unsigned long combined_pfn;
-	struct free_area *area;
 	unsigned int max_order;
 	struct page *buddy;
 	bool to_tail;
@@ -977,7 +1014,7 @@ static inline void __free_one_page(struct page *page,
 		if (page_is_guard(buddy))
 			clear_page_guard(zone, buddy, order, migratetype);
 		else
-			del_page_from_free_area(buddy, &zone->free_area[order]);
+			del_page_from_free_list(buddy, zone, order);
 		combined_pfn = buddy_pfn & pfn;
 		page = page + (combined_pfn - pfn);
 		pfn = combined_pfn;
@@ -1011,16 +1048,15 @@ static inline void __free_one_page(struct page *page,
 done_merging:
 	set_page_order(page, order);
 
-	area = &zone->free_area[order];
 	if (is_shuffle_order(order))
 		to_tail = shuffle_pick_tail();
 	else
 		to_tail = buddy_merge_likely(pfn, buddy_pfn, page, order);
 
 	if (to_tail)
-		add_to_free_area_tail(page, area, migratetype);
+		add_to_free_list_tail(page, zone, order, migratetype);
 	else
-		add_to_free_area(page, area, migratetype);
+		add_to_free_list(page, zone, order, migratetype);
 }
 
 /*
@@ -2038,13 +2074,11 @@ void __init init_cma_reserved_pageblock(struct page *page)
  * -- nyc
  */
 static inline void expand(struct zone *zone, struct page *page,
-	int low, int high, struct free_area *area,
-	int migratetype)
+	int low, int high, int migratetype)
 {
 	unsigned long size = 1 << high;
 
 	while (high > low) {
-		area--;
 		high--;
 		size >>= 1;
 		VM_BUG_ON_PAGE(bad_range(zone, &page[size]), &page[size]);
@@ -2058,7 +2092,7 @@ static inline void expand(struct zone *zone, struct page *page,
 		if (set_page_guard(zone, &page[size], high, migratetype))
 			continue;
 
-		add_to_free_area(&page[size], area, migratetype);
+		add_to_free_list(&page[size], zone, high, migratetype);
 		set_page_order(&page[size], high);
 	}
 }
@@ -2216,8 +2250,8 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
 		page = get_page_from_free_area(area, migratetype);
 		if (!page)
 			continue;
-		del_page_from_free_area(page, area);
-		expand(zone, page, order, current_order, area, migratetype);
+		del_page_from_free_list(page, zone, current_order);
+		expand(zone, page, order, current_order, migratetype);
 		set_pcppage_migratetype(page, migratetype);
 		return page;
 	}
@@ -2291,7 +2325,7 @@ static int move_freepages(struct zone *zone,
 		VM_BUG_ON_PAGE(page_zone(page) != zone, page);
 
 		order = page_order(page);
-		move_to_free_area(page, &zone->free_area[order], migratetype);
+		move_to_free_list(page, zone, order, migratetype);
 		page += 1 << order;
 		pages_moved += 1 << order;
 	}
@@ -2407,7 +2441,6 @@ static void steal_suitable_fallback(struct zone *zone, struct page *page,
 		unsigned int alloc_flags, int start_type, bool whole_block)
 {
 	unsigned int current_order = page_order(page);
-	struct free_area *area;
 	int free_pages, movable_pages, alike_pages;
 	int old_block_type;
 
@@ -2478,8 +2511,7 @@ static void steal_suitable_fallback(struct zone *zone, struct page *page,
 	return;
 
 single_page:
-	area = &zone->free_area[current_order];
-	move_to_free_area(page, area, start_type);
+	move_to_free_list(page, zone, current_order, start_type);
 }
 
 /*
@@ -3150,7 +3182,6 @@ void split_page(struct page *page, unsigned int order)
 
 int __isolate_free_page(struct page *page, unsigned int order)
 {
-	struct free_area *area = &page_zone(page)->free_area[order];
 	unsigned long watermark;
 	struct zone *zone;
 	int mt;
@@ -3176,7 +3207,7 @@ int __isolate_free_page(struct page *page, unsigned int order)
 
 	/* Remove page from free list */
 
-	del_page_from_free_area(page, area);
+	del_page_from_free_list(page, zone, order);
 
 	/*
 	 * Set the pageblock if the isolated page is at least half of a
@@ -8703,7 +8734,7 @@ void zone_pcp_reset(struct zone *zone)
 		pr_info("remove from free list %lx %d %lx\n",
 			pfn, 1 << order, end_pfn);
 #endif
-		del_page_from_free_area(page, &zone->free_area[order]);
+		del_page_from_free_list(page, zone, order);
 		pfn += (1 << order);
 	}
 	spin_unlock_irqrestore(&zone->lock, flags);

[PATCH v14 3/6] mm: Introduce Reported pages

[Alexander Duyck]

From: Alexander Duyck <alexander.h.duyck@linux.intel.com>

In order to pave the way for free page reporting in virtualized
environments we will need a way to get pages out of the free lists and
identify those pages after they have been returned. To accomplish this,
this patch adds the concept of a Reported Buddy, which is essentially
meant to just be the Uptodate flag used in conjunction with the Buddy
page type.

To prevent the reported pages from leaking outside of the buddy lists I
have added a call to clear_reported_page to the del_page_from_free_list
function. As a result any reported page that is split, merged, or
allocated will have the flag cleared prior to the PageBuddy value being
cleared.

The process for reporting pages is fairly simple. Once the nr_free for
a given free area has exceeded the number of pages reported for that area
plus a certain high watermark value we will flag the zone as needing
reporting and schedule the worker thread to start reporting. That worker
thread will begin working from the lowest supported page reporting order
up to MAX_ORDER - 1 pulling unreported pages from the free list and
storing them in the scatterlist.

When processing each individual free list it is necessary for the worker
thread to release the zone lock when it needs to stop and report the full
scatterlist of pages. To reduce the work of the next iteration the worker
thread will rotate the free list so that the first unreported page in the
free list becomes the first entry in the list. Doing this we should only
have to walk an entire free list no more than twice assuming large
numbers of pages are not being added to the tail of the list.

It will then call a reporting function providing information on how many
entries are in the scatterlist. Once the function completes it will return
the pages to the tail of the free area from which they were allocated and
start over pulling more pages from the free areas until there are no
longer enough pages to report on to keep the worker busy.

The worker thread will work in a round-robin fashion making its way
though each zone requesting reporting, and through each reportable free
list within that zone. Once all free areas within the zone is below the
high watermark level for free pages to report the flag indicating that the
zone has requested reporting will be cleared, and if no zones are
requesting reporting the worker thread will exit.

Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
---
 include/linux/mmzone.h         |   12 +
 include/linux/page-flags.h     |   11 +
 include/linux/page_reporting.h |   31 ++++
 mm/Kconfig                     |   11 +
 mm/Makefile                    |    1 
 mm/memory_hotplug.c            |    2 
 mm/page_alloc.c                |   53 ++++++
 mm/page_reporting.c            |  337 ++++++++++++++++++++++++++++++++++++++++
 mm/page_reporting.h            |  125 +++++++++++++++
 9 files changed, 579 insertions(+), 4 deletions(-)
 create mode 100644 include/linux/page_reporting.h
 create mode 100644 mm/page_reporting.c
 create mode 100644 mm/page_reporting.h

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 8d93106490f3..9647499983b1 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -478,6 +478,14 @@ struct zone {
 	seqlock_t		span_seqlock;
 #endif
 
+#ifdef CONFIG_PAGE_REPORTING
+	/*
+	 * Pointer to reported page tracking statistics array. The size of
+	 * the array is MAX_ORDER - PAGE_REPORTING_MIN_ORDER. NULL when
+	 * unused page reporting is not present.
+	 */
+	unsigned long		*reported_pages;
+#endif
 	int initialized;
 
 	/* Write-intensive fields used from the page allocator */
@@ -550,6 +558,10 @@ enum zone_flags {
 	ZONE_BOOSTED_WATERMARK,		/* zone recently boosted watermarks.
 					 * Cleared when kswapd is woken.
 					 */
+	ZONE_PAGE_REPORTING_REQUESTED,	/* zone enabled page reporting and has
+					 * requested flushing the data out of
+					 * higher order pages.
+					 */
 };
 
 static inline unsigned long zone_managed_pages(struct zone *zone)
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 1bf83c8fcaa7..a3a3b15b56a8 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -163,6 +163,9 @@ enum pageflags {
 
 	/* non-lru isolated movable page */
 	PG_isolated = PG_reclaim,
+
+	/* Buddy pages. Used to track which pages have been reported */
+	PG_reported = PG_uptodate,
 };
 
 #ifndef __GENERATING_BOUNDS_H
@@ -432,6 +435,14 @@ static inline bool set_hwpoison_free_buddy_page(struct page *page)
 #endif
 
 /*
+ * PageReported() is used to track reported free pages within the Buddy
+ * allocator. We can use the non-atomic version of the test and set
+ * operations as both should be shielded with the zone lock to prevent
+ * any possible races on the setting or clearing of the bit.
+ */
+__PAGEFLAG(Reported, reported, PF_NO_COMPOUND)
+
+/*
  * On an anonymous page mapped into a user virtual memory area,
  * page->mapping points to its anon_vma, not to a struct address_space;
  * with the PAGE_MAPPING_ANON bit set to distinguish it.  See rmap.h.
diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h
new file mode 100644
index 000000000000..925a16b1d14b
--- /dev/null
+++ b/include/linux/page_reporting.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_PAGE_REPORTING_H
+#define _LINUX_PAGE_REPORTING_H
+
+#include <linux/mmzone.h>
+
+struct page_reporting_dev_info {
+	/* function that alters pages to make them "reported" */
+	void (*report)(struct page_reporting_dev_info *prdev,
+		       unsigned int nents);
+
+	/* pointer to scatterlist containing pages to be processed */
+	struct scatterlist *sg;
+
+	/*
+	 * Upper limit on the number of pages that the report function
+	 * expects to be placed into the scatterlist to be processed.
+	 */
+	unsigned int capacity;
+
+	/* The number of zones requesting reporting */
+	atomic_t refcnt;
+
+	/* work struct for processing reports */
+	struct delayed_work work;
+};
+
+/* Tear-down and bring-up for page reporting devices */
+void page_reporting_unregister(struct page_reporting_dev_info *prdev);
+int page_reporting_register(struct page_reporting_dev_info *prdev);
+#endif /*_LINUX_PAGE_REPORTING_H */
diff --git a/mm/Kconfig b/mm/Kconfig
index e38ff1d5968d..bd68901bbfe2 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -237,6 +237,17 @@ config COMPACTION
           linux-mm@kvack.org.
 
 #
+# support for unused page reporting
+config PAGE_REPORTING
+	bool "Allow for reporting of unused pages"
+	def_bool n
+	help
+	  Unused page reporting allows for the incremental acquisition of
+	  unused pages from the buddy allocator for the purpose of reporting
+	  those pages to another entity, such as a hypervisor, so that the
+	  memory can be freed up for other uses.
+
+#
 # support for page migration
 #
 config MIGRATION
diff --git a/mm/Makefile b/mm/Makefile
index fa02f19f8e50..41e6aa432781 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -108,3 +108,4 @@ obj-$(CONFIG_PERCPU_STATS) += percpu-stats.o
 obj-$(CONFIG_ZONE_DEVICE) += memremap.o
 obj-$(CONFIG_HMM_MIRROR) += hmm.o
 obj-$(CONFIG_MEMFD_CREATE) += memfd.o
+obj-$(CONFIG_PAGE_REPORTING) += page_reporting.o
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 46b2e056a43f..bf19d94cb206 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -41,6 +41,7 @@
 
 #include "internal.h"
 #include "shuffle.h"
+#include "page_reporting.h"
 
 /*
  * online_page_callback contains pointer to current page onlining function.
@@ -1566,6 +1567,7 @@ static int __ref __offline_pages(unsigned long start_pfn,
 	if (!populated_zone(zone)) {
 		zone_pcp_reset(zone);
 		build_all_zonelists(NULL);
+		page_reporting_reset_zone(zone);
 	} else
 		zone_pcp_update(zone);
 
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index e0a7895300fb..5ef8f5abdd4d 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -74,6 +74,7 @@
 #include <asm/div64.h>
 #include "internal.h"
 #include "shuffle.h"
+#include "page_reporting.h"
 
 /* prevent >1 _updater_ of zone percpu pageset ->high and ->batch fields */
 static DEFINE_MUTEX(pcp_batch_high_lock);
@@ -909,6 +910,10 @@ static inline void move_to_free_list(struct page *page, struct zone *zone,
 static inline void del_page_from_free_list(struct page *page, struct zone *zone,
 					   unsigned int order)
 {
+	/* clear reported state and update reported page count */
+	if (page_reported(page))
+		clear_page_reported(page, zone, order);
+
 	list_del(&page->lru);
 	__ClearPageBuddy(page);
 	set_page_private(page, 0);
@@ -972,7 +977,7 @@ static inline void del_page_from_free_list(struct page *page, struct zone *zone,
 static inline void __free_one_page(struct page *page,
 		unsigned long pfn,
 		struct zone *zone, unsigned int order,
-		int migratetype)
+		int migratetype, bool reported)
 {
 	struct capture_control *capc = task_capc(zone);
 	unsigned long uninitialized_var(buddy_pfn);
@@ -1048,7 +1053,9 @@ static inline void __free_one_page(struct page *page,
 done_merging:
 	set_page_order(page, order);
 
-	if (is_shuffle_order(order))
+	if (reported)
+		to_tail = true;
+	else if (is_shuffle_order(order))
 		to_tail = shuffle_pick_tail();
 	else
 		to_tail = buddy_merge_likely(pfn, buddy_pfn, page, order);
@@ -1057,6 +1064,14 @@ static inline void __free_one_page(struct page *page,
 		add_to_free_list_tail(page, zone, order, migratetype);
 	else
 		add_to_free_list(page, zone, order, migratetype);
+
+	/*
+	 * No need to notify on a reported page as the total count of
+	 * unreported pages will not have increased since we have essentially
+	 * merged the reported page with one or more unreported pages.
+	 */
+	if (!reported)
+		page_reporting_notify_free(zone, order);
 }
 
 /*
@@ -1373,7 +1388,7 @@ static void free_pcppages_bulk(struct zone *zone, int count,
 		if (unlikely(isolated_pageblocks))
 			mt = get_pageblock_migratetype(page);
 
-		__free_one_page(page, page_to_pfn(page), zone, 0, mt);
+		__free_one_page(page, page_to_pfn(page), zone, 0, mt, false);
 		trace_mm_page_pcpu_drain(page, 0, mt);
 	}
 	spin_unlock(&zone->lock);
@@ -1389,7 +1404,7 @@ static void free_one_page(struct zone *zone,
 		is_migrate_isolate(migratetype))) {
 		migratetype = get_pfnblock_migratetype(page, pfn);
 	}
-	__free_one_page(page, pfn, zone, order, migratetype);
+	__free_one_page(page, pfn, zone, order, migratetype, false);
 	spin_unlock(&zone->lock);
 }
 
@@ -3228,6 +3243,36 @@ int __isolate_free_page(struct page *page, unsigned int order)
 	return 1UL << order;
 }
 
+#ifdef CONFIG_PAGE_REPORTING
+/**
+ * __free_isolated_page - Return a now-isolated page back where we got it
+ * @page: Page that was isolated
+ * @order: Order of the isolated page
+ *
+ * This function is meant to return a page pulled from the free lists via
+ * __isolate_free_page back to the free lists they were pulled from.
+ */
+void __free_isolated_page(struct page *page, unsigned int order)
+{
+	struct zone *zone = page_zone(page);
+	unsigned long pfn;
+	unsigned int mt;
+
+	/* zone lock should be held when this function is called */
+	lockdep_assert_held(&zone->lock);
+
+	pfn = page_to_pfn(page);
+	mt = get_pfnblock_migratetype(page, pfn);
+
+	/*
+	 * Return isolated page to tail of freelist and don't bother with
+	 * triggering the page reporting notifiers since this page was
+	 * previously on the freelist and has likely already been reported.
+	 */
+	__free_one_page(page, pfn, zone, order, mt, true);
+}
+#endif /* CONFIG_PAGE_REPORTING */
+
 /*
  * Update NUMA hit/miss statistics
  *
diff --git a/mm/page_reporting.c b/mm/page_reporting.c
new file mode 100644
index 000000000000..4844f0aa2904
--- /dev/null
+++ b/mm/page_reporting.c
@@ -0,0 +1,337 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/mm.h>
+#include <linux/mmzone.h>
+#include <linux/page_reporting.h>
+#include <linux/gfp.h>
+#include <linux/export.h>
+#include <linux/delay.h>
+#include <linux/scatterlist.h>
+
+#include "page_reporting.h"
+#include "internal.h"
+
+static struct page_reporting_dev_info __rcu *pr_dev_info __read_mostly;
+
+#define for_each_reporting_migratetype_order(_order, _type) \
+	for (_order = PAGE_REPORTING_MIN_ORDER; _order < MAX_ORDER; _order++) \
+		for (_type = 0; _type < MIGRATE_TYPES; _type++) \
+			if (!is_migrate_isolate(_type))
+
+static void page_reporting_populate_metadata(struct zone *zone)
+{
+	size_t size;
+	int node;
+
+	/*
+	 * We need to make sure we have somewhere to store the tracking
+	 * data for how many reported pages are in the zone. To do that
+	 * we need to make certain zone->reported_pages is populated.
+	 */
+	if (zone->reported_pages)
+		return;
+
+	node = zone_to_nid(zone);
+	size = (MAX_ORDER - PAGE_REPORTING_MIN_ORDER) * sizeof(unsigned long);
+	zone->reported_pages = kzalloc_node(size, GFP_KERNEL, node);
+}
+
+static void
+page_reporting_drain(struct page_reporting_dev_info *prdev, struct zone *zone)
+{
+	struct scatterlist *sg = prdev->sg;
+
+	/*
+	 * Drain the now reported pages back into their respective
+	 * free lists/areas. We assume at least one page is populated.
+	 */
+	do {
+		unsigned int order = get_order(sg->length);
+		struct page *page = sg_page(sg);
+
+		__free_isolated_page(page, order);
+
+		/*
+		 * If page was not comingled with another page we can
+		 * consider the result to be "reported" since the page
+		 * hasn't been modified, otherwise we will need to
+		 * report on the new larger page when we make our way
+		 * up to that higher order.
+		 */
+		if (PageBuddy(page) && page_order(page) == order)
+			mark_page_reported(page, zone, order);
+	} while (!sg_is_last(sg++));
+}
+
+/*
+ * The page reporting cycle consists of 4 stages, fill, report, drain, and
+ * idle. We will cycle through the first 3 stages until we cannot obtain a
+ * full scatterlist of pages, in that case we will switch to idle.
+ */
+static unsigned int
+page_reporting_cycle(struct page_reporting_dev_info *prdev, struct zone *zone,
+		     unsigned int order, unsigned int mt, unsigned int nents)
+{
+	struct list_head *list = &zone->free_area[order].free_list[mt];
+	unsigned int page_len = PAGE_SIZE << order;
+	struct scatterlist *sg = prdev->sg;
+	struct page *page, *next;
+
+	/*
+	 * Perform early check, if free area is empty there is
+	 * nothing to process so we can skip this free_list.
+	 */
+	if (list_empty(list))
+		return nents;
+
+	spin_lock_irq(&zone->lock);
+
+	/* loop through free list adding unreported pages to sg list */
+	list_for_each_entry_safe(page, next, list, lru) {
+		/* We are going to skip over the reported pages. */
+		if (PageReported(page))
+			continue;
+
+		/* Attempt to add page to sg list */
+		if (nents < prdev->capacity) {
+			if (!__isolate_free_page(page, order))
+				break;
+
+			sg_set_page(&sg[nents++], page, page_len, 0);
+			continue;
+		}
+
+		/*
+		 * Make the first non-reported entry in the free list
+		 * the new head of the free list before we exit.
+		 */
+		if (!list_is_first(&page->lru, list))
+			list_rotate_to_front(&page->lru, list);
+
+		/* release lock before waiting on report processing*/
+		spin_unlock_irq(&zone->lock);
+
+		/* begin processing pages in local list */
+		prdev->report(prdev, nents);
+
+		/* reset number of entries */
+		nents = 0;
+
+		/* reacquire zone lock and resume processing free lists */
+		spin_lock_irq(&zone->lock);
+
+		/* flush reported pages from the sg list */
+		page_reporting_drain(prdev, zone);
+
+		/*
+		 * Reset next to first entry, the old next isn't valid
+		 * since we dropped the lock to report the pages
+		 */
+		next = list_first_entry(list, struct page, lru);
+	}
+
+	spin_unlock_irq(&zone->lock);
+
+	return nents;
+}
+
+static int
+page_reporting_process_zone(struct page_reporting_dev_info *prdev,
+			    struct zone *zone)
+{
+	unsigned int order, mt, nents = 0;
+	unsigned long watermark;
+	int refcnt = -1;
+
+	page_reporting_populate_metadata(zone);
+
+	/* Generate minimum watermark to be able to guarantee progress */
+	watermark = low_wmark_pages(zone) +
+		    (prdev->capacity << PAGE_REPORTING_MIN_ORDER);
+
+	/*
+	 * Cancel request if insufficient free memory or if we failed
+	 * to allocate page reporting statistics for the zone.
+	 */
+	if (!zone_watermark_ok(zone, 0, watermark, 0, ALLOC_CMA) ||
+	    !zone->reported_pages) {
+		spin_lock_irq(&zone->lock);
+		goto zone_not_ready;
+	}
+
+	sg_init_table(prdev->sg, prdev->capacity);
+
+	/* Process each free list starting from lowest order/mt */
+	for_each_reporting_migratetype_order(order, mt)
+		nents = page_reporting_cycle(prdev, zone, order, mt, nents);
+
+	/* mark end of sg list and report the remainder */
+	if (nents) {
+		sg_mark_end(&prdev->sg[nents - 1]);
+		prdev->report(prdev, nents);
+	}
+
+	spin_lock_irq(&zone->lock);
+
+	/* flush any remaining pages out from the last report */
+	if (nents)
+		page_reporting_drain(prdev, zone);
+
+	/* check to see if values are low enough for us to stop for now */
+	for (order = PAGE_REPORTING_MIN_ORDER; order < MAX_ORDER; order++) {
+		if (pages_unreported(zone, order) < PAGE_REPORTING_HWM)
+			continue;
+#ifdef CONFIG_MEMORY_ISOLATION
+		/*
+		 * Do not allow a free_area with isolated pages to request
+		 * that we continue with page reporting. Keep the reporting
+		 * light until the isolated pages have been cleared.
+		 */
+		if (!free_area_empty(&zone->free_area[order], MIGRATE_ISOLATE))
+			continue;
+#endif
+		goto zone_not_complete;
+	}
+
+zone_not_ready:
+	/*
+	 * If there are no longer enough free pages to fully populate
+	 * the scatterlist, then we can just shut it down for this zone.
+	 */
+	__clear_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags);
+	refcnt = atomic_dec_return(&prdev->refcnt);
+zone_not_complete:
+	spin_unlock_irq(&zone->lock);
+
+	return refcnt;
+}
+
+static void page_reporting_process(struct work_struct *work)
+{
+	struct delayed_work *d_work = to_delayed_work(work);
+	struct page_reporting_dev_info *prdev =
+		container_of(d_work, struct page_reporting_dev_info, work);
+	struct zone *zone = first_online_pgdat()->node_zones;
+	int refcnt = -1;
+
+	do {
+		if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
+			refcnt = page_reporting_process_zone(prdev, zone);
+
+		/* Move to next zone, if at end of list start over */
+		zone = next_zone(zone) ? : first_online_pgdat()->node_zones;
+
+		/*
+		 * As long as refcnt has not reached zero there are still
+		 * zones to be processed.
+		 */
+	} while (refcnt);
+}
+
+/* request page reporting on this zone */
+void __page_reporting_request(struct zone *zone)
+{
+	struct page_reporting_dev_info *prdev;
+
+	rcu_read_lock();
+
+	/*
+	 * We use RCU to protect the pr_dev_info pointer. In almost all
+	 * cases this should be present, however in the unlikely case of
+	 * a shutdown this will be NULL and we should exit.
+	 */
+	prdev = rcu_dereference(pr_dev_info);
+	if (unlikely(!prdev))
+		goto out;
+
+	/*
+	 * We can use separate test and set operations here as there
+	 * is nothing else that can set or clear this bit while we are
+	 * holding the zone lock. The advantage to doing it this way is
+	 * that we don't have to dirty the cacheline unless we are
+	 * changing the value.
+	 */
+	__set_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags);
+
+	/*
+	 * Delay the start of work to allow a sizable queue to
+	 * build. For now we are limiting this to running no more
+	 * than 5 times per second.
+	 */
+	if (!atomic_fetch_inc(&prdev->refcnt))
+		schedule_delayed_work(&prdev->work, HZ / 5);
+out:
+	rcu_read_unlock();
+}
+
+static DEFINE_MUTEX(page_reporting_mutex);
+DEFINE_STATIC_KEY_FALSE(page_reporting_enabled);
+
+void page_reporting_unregister(struct page_reporting_dev_info *prdev)
+{
+	mutex_lock(&page_reporting_mutex);
+
+	if (rcu_access_pointer(pr_dev_info) == prdev) {
+		/* Disable page reporting notification */
+		RCU_INIT_POINTER(pr_dev_info, NULL);
+		synchronize_rcu();
+
+		/* Flush any existing work, and lock it out */
+		cancel_delayed_work_sync(&prdev->work);
+
+		/* Free scatterlist */
+		kfree(prdev->sg);
+		prdev->sg = NULL;
+	}
+
+	mutex_unlock(&page_reporting_mutex);
+}
+EXPORT_SYMBOL_GPL(page_reporting_unregister);
+
+int page_reporting_register(struct page_reporting_dev_info *prdev)
+{
+	struct zone *zone;
+	int err = 0;
+
+	/* No point in enabling this if it cannot handle any pages */
+	if (WARN_ON(!prdev->capacity || prdev->capacity > PAGE_REPORTING_HWM))
+		return -EINVAL;
+
+	mutex_lock(&page_reporting_mutex);
+
+	/* nothing to do if already in use */
+	if (rcu_access_pointer(pr_dev_info)) {
+		err = -EBUSY;
+		goto err_out;
+	}
+
+	/* allocate scatterlist to store pages being reported on */
+	prdev->sg = kcalloc(prdev->capacity, sizeof(*prdev->sg), GFP_KERNEL);
+	if (!prdev->sg) {
+		err = -ENOMEM;
+		goto err_out;
+	}
+
+
+	/* initialize refcnt and work structures */
+	atomic_set(&prdev->refcnt, 0);
+	INIT_DELAYED_WORK(&prdev->work, &page_reporting_process);
+
+	/* assign device, and begin initial flush of populated zones */
+	rcu_assign_pointer(pr_dev_info, prdev);
+	for_each_populated_zone(zone) {
+		spin_lock_irq(&zone->lock);
+		__page_reporting_request(zone);
+		spin_unlock_irq(&zone->lock);
+	}
+
+	/* enable page reporting notification */
+	if (!static_key_enabled(&page_reporting_enabled)) {
+		static_branch_enable(&page_reporting_enabled);
+		pr_info("Unused page reporting enabled\n");
+	}
+err_out:
+	mutex_unlock(&page_reporting_mutex);
+
+	return err;
+}
+EXPORT_SYMBOL_GPL(page_reporting_register);
diff --git a/mm/page_reporting.h b/mm/page_reporting.h
new file mode 100644
index 000000000000..2ad31bbb0036
--- /dev/null
+++ b/mm/page_reporting.h
@@ -0,0 +1,125 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _MM_PAGE_REPORTING_H
+#define _MM_PAGE_REPORTING_H
+
+#include <linux/mmzone.h>
+#include <linux/pageblock-flags.h>
+#include <linux/page-isolation.h>
+#include <linux/jump_label.h>
+#include <linux/slab.h>
+#include <asm/pgtable.h>
+
+#define PAGE_REPORTING_MIN_ORDER	pageblock_order
+#define PAGE_REPORTING_HWM		32
+
+#ifdef CONFIG_PAGE_REPORTING
+/* Reported page accessors, defined in page_alloc.c */
+void __free_isolated_page(struct page *page, unsigned int order);
+
+/* Free reported_pages and reset reported page tracking count to 0 */
+static inline void page_reporting_reset_zone(struct zone *zone)
+{
+	kfree(zone->reported_pages);
+	zone->reported_pages = NULL;
+}
+
+DECLARE_STATIC_KEY_FALSE(page_reporting_enabled);
+void __page_reporting_request(struct zone *zone);
+
+static inline bool page_reported(struct page *page)
+{
+	return static_branch_unlikely(&page_reporting_enabled) &&
+	       PageReported(page);
+}
+
+static inline unsigned long
+pages_unreported(struct zone *zone, int order)
+{
+	unsigned long nr_free;
+	int report_order;
+
+	/* Limit notifications only to higher order pages */
+	report_order = order - PAGE_REPORTING_MIN_ORDER;
+	if (report_order < 0)
+		return 0;
+
+	nr_free = zone->free_area[order].nr_free;
+
+	/* Only subtract reported_pages count if it is present */
+	if (!zone->reported_pages)
+		return nr_free;
+
+	return nr_free - zone->reported_pages[report_order];
+}
+
+/**
+ * page_reporting_notify_free - Free page notification to start page processing
+ * @zone: Pointer to current zone of last page processed
+ * @order: Order of last page added to zone
+ *
+ * This function is meant to act as a screener for __page_reporting_request
+ * which will determine if a give zone has crossed over the high-water mark
+ * that will justify us beginning page treatment. If we have crossed that
+ * threshold then it will start the process of pulling some pages and
+ * placing them in the batch list for treatment.
+ */
+static inline void page_reporting_notify_free(struct zone *zone, int order)
+{
+	/* Called from hot path in __free_one_page() */
+	if (!static_branch_unlikely(&page_reporting_enabled))
+		return;
+
+	/* Do not bother with tests if we have already requested reporting */
+	if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
+		return;
+
+	/* Determine if we have crossed reporting threshold */
+	if (pages_unreported(zone, order) < PAGE_REPORTING_HWM)
+		return;
+
+	/* This is slow, but should be called very rarely */
+	__page_reporting_request(zone);
+}
+
+/*
+ * Functions for marking/clearing reported pages from the freelist.
+ * All of them expect the zone lock to be held to maintain
+ * consistency of the reported_pages count versus nr_free.
+ */
+static inline void
+mark_page_reported(struct page *page, struct zone *zone, unsigned int order)
+{
+	/* flag page as reported */
+	__SetPageReported(page);
+
+	/* update areated page accounting */
+	zone->reported_pages[order - PAGE_REPORTING_MIN_ORDER]++;
+}
+
+static inline void
+clear_page_reported(struct page *page, struct zone *zone, unsigned int order)
+{
+	/* page_private will contain the page order, so just use it directly */
+	zone->reported_pages[order - PAGE_REPORTING_MIN_ORDER]--;
+
+	/* clear the flag so we can report on it when it returns */
+	__ClearPageReported(page);
+}
+
+#else /* CONFIG_PAGE_REPORTING */
+#define page_reported(_page)	false
+
+static inline void page_reporting_reset_zone(struct zone *zone)
+{
+}
+
+static inline void page_reporting_notify_free(struct zone *zone, int order)
+{
+}
+
+static inline void
+clear_page_reported(struct page *page, struct zone *zone, unsigned int order)
+{
+}
+#endif /* CONFIG_PAGE_REPORTING */
+#endif /*_MM_PAGE_REPORTING_H */

[PATCH v14 4/6] mm: Add unused page reporting documentation

[Alexander Duyck]

From: Alexander Duyck <alexander.h.duyck@linux.intel.com>

Add documentation for unused page reporting. Currently the only consumer is
virtio-balloon, however it is possible that other drivers might make use of
this so it is best to add a bit of documetation explaining at a high level
how to use the API.

Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
---
 Documentation/vm/unused_page_reporting.rst |   44 ++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)
 create mode 100644 Documentation/vm/unused_page_reporting.rst

diff --git a/Documentation/vm/unused_page_reporting.rst b/Documentation/vm/unused_page_reporting.rst
new file mode 100644
index 000000000000..932406f48842
--- /dev/null
+++ b/Documentation/vm/unused_page_reporting.rst
@@ -0,0 +1,44 @@
+.. _unused_page_reporting:
+
+=====================
+Unused Page Reporting
+=====================
+
+Unused page reporting is an API by which a device can register to receive
+lists of pages that are currently unused by the system. This is useful in
+the case of virtualization where a guest is then able to use this data to
+notify the hypervisor that it is no longer using certain pages in memory.
+
+For the driver, typically a balloon driver, to use of this functionality
+it will allocate and initialize a page_reporting_dev_info structure. The
+fields within the structure it will populate are the "report" function
+pointer used to process the scatterlist and "capacity" representing the
+number of entries that the device can support in a single request. Once
+those are populated a call to page_reporting_register will allocate the
+scatterlist and register the device with the reporting framework assuming
+no other page reporting devices are already registered.
+
+Once registered the page reporting API will begin reporting batches of
+pages to the driver. The API determines that it needs to start reporting by
+measuring the number of pages in a given free area versus the number of
+reported pages for that free area. If the value meets or exceeds the value
+defined by PAGE_REPORTING_HWM then the zone is flagged as requesting
+reporting and a worker is scheduled to process zone requesting reporting.
+
+Pages reported will be stored in the scatterlist pointed to in the
+page_reporting_dev_info with the final entry having the end bit set in
+entry nent - 1. While pages are being processed by the report function they
+will not be accessible to the allocator. Once the report function has been
+completed the pages will be returned to the free area from which they were
+obtained.
+
+Prior to removing a driver that is making use of unused page reporting it
+is necessary to call page_reporting_unregister to have the
+page_reporting_dev_info structure that is currently in use by unused page
+reporting removed. Doing this will prevent further reports from being
+issued via the interface. If another driver or the same driver is
+registered it is possible for it to resume where the previous driver had
+left off in terms of reporting unused pages.
+
+Alexander Duyck, Nov 15, 2019
+

[PATCH v14 5/6] virtio-balloon: Pull page poisoning config out of free page hinting

[Alexander Duyck]

From: Alexander Duyck <alexander.h.duyck@linux.intel.com>

Currently the page poisoning setting wasn't being enabled unless free page
hinting was enabled. However we will need the page poisoning tracking logic
as well for unused page reporting. As such pull it out and make it a
separate bit of config in the probe function.

In addition we need to add support for the more recent init_on_free feature
which expects a behavior similar to page poisoning in that we expect the
page to be pre-zeroed.

Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
---
 drivers/virtio/virtio_balloon.c |   23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 226fbb995fb0..92099298bc16 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -842,7 +842,6 @@ static int virtio_balloon_register_shrinker(struct virtio_balloon *vb)
 static int virtballoon_probe(struct virtio_device *vdev)
 {
 	struct virtio_balloon *vb;
-	__u32 poison_val;
 	int err;
 
 	if (!vdev->config->get) {
@@ -909,11 +908,20 @@ static int virtballoon_probe(struct virtio_device *vdev)
 						  VIRTIO_BALLOON_CMD_ID_STOP);
 		spin_lock_init(&vb->free_page_list_lock);
 		INIT_LIST_HEAD(&vb->free_page_list);
-		if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON)) {
+	}
+	if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON)) {
+		/* Start with poison val of 0 representing general init */
+		__u32 poison_val = 0;
+
+		/*
+		 * Let the hypervisor know that we are expecting a
+		 * specific value to be written back in unused pages.
+		 */
+		if (!want_init_on_free())
 			memset(&poison_val, PAGE_POISON, sizeof(poison_val));
-			virtio_cwrite(vb->vdev, struct virtio_balloon_config,
-				      poison_val, &poison_val);
-		}
+
+		virtio_cwrite(vb->vdev, struct virtio_balloon_config,
+			      poison_val, &poison_val);
 	}
 	/*
 	 * We continue to use VIRTIO_BALLOON_F_DEFLATE_ON_OOM to decide if a
@@ -1014,7 +1022,10 @@ static int virtballoon_restore(struct virtio_device *vdev)
 
 static int virtballoon_validate(struct virtio_device *vdev)
 {
-	if (!page_poisoning_enabled())
+	/* Tell the host whether we care about poisoned pages. */
+	if (!want_init_on_free() &&
+	    (IS_ENABLED(CONFIG_PAGE_POISONING_NO_SANITY) ||
+	     !page_poisoning_enabled()))
 		__virtio_clear_bit(vdev, VIRTIO_BALLOON_F_PAGE_POISON);
 
 	__virtio_clear_bit(vdev, VIRTIO_F_IOMMU_PLATFORM);

[PATCH v14 6/6] virtio-balloon: Add support for providing unused page reports to host

[Alexander Duyck]

From: Alexander Duyck <alexander.h.duyck@linux.intel.com>

Add support for the page reporting feature provided by virtio-balloon.
Reporting differs from the regular balloon functionality in that is is
much less durable than a standard memory balloon. Instead of creating a
list of pages that cannot be accessed the pages are only inaccessible
while they are being indicated to the virtio interface. Once the
interface has acknowledged them they are placed back into their respective
free lists and are once again accessible by the guest system.

Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
---
 drivers/virtio/Kconfig              |    1 +
 drivers/virtio/virtio_balloon.c     |   65 +++++++++++++++++++++++++++++++++++
 include/uapi/linux/virtio_balloon.h |    1 +
 3 files changed, 67 insertions(+)

diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig
index 078615cf2afc..4b2dd8259ff5 100644
--- a/drivers/virtio/Kconfig
+++ b/drivers/virtio/Kconfig
@@ -58,6 +58,7 @@ config VIRTIO_BALLOON
 	tristate "Virtio balloon driver"
 	depends on VIRTIO
 	select MEMORY_BALLOON
+	select PAGE_REPORTING
 	---help---
 	 This driver supports increasing and decreasing the amount
 	 of memory within a KVM guest.
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 92099298bc16..6f5c6555765a 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -19,6 +19,7 @@
 #include <linux/mount.h>
 #include <linux/magic.h>
 #include <linux/pseudo_fs.h>
+#include <linux/page_reporting.h>
 
 /*
  * Balloon device works in 4K page units.  So each page is pointed to by
@@ -37,6 +38,9 @@
 #define VIRTIO_BALLOON_FREE_PAGE_SIZE \
 	(1 << (VIRTIO_BALLOON_FREE_PAGE_ORDER + PAGE_SHIFT))
 
+/*  limit on the number of pages that can be on the reporting vq */
+#define VIRTIO_BALLOON_VRING_HINTS_MAX	16
+
 #ifdef CONFIG_BALLOON_COMPACTION
 static struct vfsmount *balloon_mnt;
 #endif
@@ -46,6 +50,7 @@ enum virtio_balloon_vq {
 	VIRTIO_BALLOON_VQ_DEFLATE,
 	VIRTIO_BALLOON_VQ_STATS,
 	VIRTIO_BALLOON_VQ_FREE_PAGE,
+	VIRTIO_BALLOON_VQ_REPORTING,
 	VIRTIO_BALLOON_VQ_MAX
 };
 
@@ -113,6 +118,10 @@ struct virtio_balloon {
 
 	/* To register a shrinker to shrink memory upon memory pressure */
 	struct shrinker shrinker;
+
+	/* Unused page reporting device */
+	struct virtqueue *reporting_vq;
+	struct page_reporting_dev_info pr_dev_info;
 };
 
 static struct virtio_device_id id_table[] = {
@@ -152,6 +161,32 @@ static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq)
 
 }
 
+void virtballoon_unused_page_report(struct page_reporting_dev_info *pr_dev_info,
+				    unsigned int nents)
+{
+	struct virtio_balloon *vb =
+		container_of(pr_dev_info, struct virtio_balloon, pr_dev_info);
+	struct virtqueue *vq = vb->reporting_vq;
+	unsigned int unused, err;
+
+	/* We should always be able to add these buffers to an empty queue. */
+	err = virtqueue_add_inbuf(vq, pr_dev_info->sg, nents, vb,
+				  GFP_NOWAIT | __GFP_NOWARN);
+
+	/*
+	 * In the extremely unlikely case that something has changed and we
+	 * are able to trigger an error we will simply display a warning
+	 * and exit without actually processing the pages.
+	 */
+	if (WARN_ON(err))
+		return;
+
+	virtqueue_kick(vq);
+
+	/* When host has read buffer, this completes via balloon_ack */
+	wait_event(vb->acked, virtqueue_get_buf(vq, &unused));
+}
+
 static void set_page_pfns(struct virtio_balloon *vb,
 			  __virtio32 pfns[], struct page *page)
 {
@@ -476,6 +511,7 @@ static int init_vqs(struct virtio_balloon *vb)
 	names[VIRTIO_BALLOON_VQ_DEFLATE] = "deflate";
 	names[VIRTIO_BALLOON_VQ_STATS] = NULL;
 	names[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL;
+	names[VIRTIO_BALLOON_VQ_REPORTING] = NULL;
 
 	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
 		names[VIRTIO_BALLOON_VQ_STATS] = "stats";
@@ -487,11 +523,19 @@ static int init_vqs(struct virtio_balloon *vb)
 		callbacks[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL;
 	}
 
+	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
+		names[VIRTIO_BALLOON_VQ_REPORTING] = "reporting_vq";
+		callbacks[VIRTIO_BALLOON_VQ_REPORTING] = balloon_ack;
+	}
+
 	err = vb->vdev->config->find_vqs(vb->vdev, VIRTIO_BALLOON_VQ_MAX,
 					 vqs, callbacks, names, NULL, NULL);
 	if (err)
 		return err;
 
+	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
+		vb->reporting_vq = vqs[VIRTIO_BALLOON_VQ_REPORTING];
+
 	vb->inflate_vq = vqs[VIRTIO_BALLOON_VQ_INFLATE];
 	vb->deflate_vq = vqs[VIRTIO_BALLOON_VQ_DEFLATE];
 	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
@@ -932,12 +976,30 @@ static int virtballoon_probe(struct virtio_device *vdev)
 		if (err)
 			goto out_del_balloon_wq;
 	}
+
+	vb->pr_dev_info.report = virtballoon_unused_page_report;
+	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
+		unsigned int capacity;
+
+		capacity = min_t(unsigned int,
+				 virtqueue_get_vring_size(vb->reporting_vq),
+				 VIRTIO_BALLOON_VRING_HINTS_MAX);
+		vb->pr_dev_info.capacity = capacity;
+
+		err = page_reporting_register(&vb->pr_dev_info);
+		if (err)
+			goto out_unregister_shrinker;
+	}
+
 	virtio_device_ready(vdev);
 
 	if (towards_target(vb))
 		virtballoon_changed(vdev);
 	return 0;
 
+out_unregister_shrinker:
+	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
+		virtio_balloon_unregister_shrinker(vb);
 out_del_balloon_wq:
 	if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_FREE_PAGE_HINT))
 		destroy_workqueue(vb->balloon_wq);
@@ -966,6 +1028,8 @@ static void virtballoon_remove(struct virtio_device *vdev)
 {
 	struct virtio_balloon *vb = vdev->priv;
 
+	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
+		page_reporting_unregister(&vb->pr_dev_info);
 	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
 		virtio_balloon_unregister_shrinker(vb);
 	spin_lock_irq(&vb->stop_update_lock);
@@ -1038,6 +1102,7 @@ static int virtballoon_validate(struct virtio_device *vdev)
 	VIRTIO_BALLOON_F_DEFLATE_ON_OOM,
 	VIRTIO_BALLOON_F_FREE_PAGE_HINT,
 	VIRTIO_BALLOON_F_PAGE_POISON,
+	VIRTIO_BALLOON_F_REPORTING,
 };
 
 static struct virtio_driver virtio_balloon_driver = {
diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virtio_balloon.h
index a1966cd7b677..19974392d324 100644
--- a/include/uapi/linux/virtio_balloon.h
+++ b/include/uapi/linux/virtio_balloon.h
@@ -36,6 +36,7 @@
 #define VIRTIO_BALLOON_F_DEFLATE_ON_OOM	2 /* Deflate balloon on OOM */
 #define VIRTIO_BALLOON_F_FREE_PAGE_HINT	3 /* VQ to report free pages */
 #define VIRTIO_BALLOON_F_PAGE_POISON	4 /* Guest is using page poisoning */
+#define VIRTIO_BALLOON_F_REPORTING	5 /* Page reporting virtqueue */
 
 /* Size of a PFN in the balloon interface. */
 #define VIRTIO_BALLOON_PFN_SHIFT 12

[PATCH v14 QEMU 1/3] virtio-ballon: Implement support for page poison tracking feature

[Alexander Duyck]

From: Alexander Duyck <alexander.h.duyck@linux.intel.com>

We need to make certain to advertise support for page poison tracking if
we want to actually get data on if the guest will be poisoning pages. So
if free page hinting is active we should add page poisoning support and
let the guest disable it if it isn't using it.

Page poisoning will result in a page being dirtied on free. As such we
cannot really avoid having to copy the page at least one more time since
we will need to write the poison value to the destination. As such we can
just ignore free page hinting if page poisoning is enabled as it will
actually reduce the work we have to do.

Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
---
 hw/virtio/virtio-balloon.c         |   25 +++++++++++++++++++++----
 include/hw/virtio/virtio-balloon.h |    1 +
 2 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 40b04f518028..6ecfec422309 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -531,6 +531,15 @@ static void virtio_balloon_free_page_start(VirtIOBalloon *s)
         return;
     }
 
+    /*
+     * If page poisoning is enabled then we probably shouldn't bother with
+     * the hinting since the poisoning will dirty the page and invalidate
+     * the work we are doing anyway.
+     */
+    if (virtio_vdev_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON)) {
+        return;
+    }
+
     if (s->free_page_report_cmd_id == UINT_MAX) {
         s->free_page_report_cmd_id =
                        VIRTIO_BALLOON_FREE_PAGE_REPORT_CMD_ID_MIN;
@@ -618,12 +627,10 @@ static size_t virtio_balloon_config_size(VirtIOBalloon *s)
     if (s->qemu_4_0_config_size) {
         return sizeof(struct virtio_balloon_config);
     }
-    if (virtio_has_feature(features, VIRTIO_BALLOON_F_PAGE_POISON)) {
+    if (virtio_has_feature(features, VIRTIO_BALLOON_F_PAGE_POISON) ||
+        virtio_has_feature(features, VIRTIO_BALLOON_F_FREE_PAGE_HINT)) {
         return sizeof(struct virtio_balloon_config);
     }
-    if (virtio_has_feature(features, VIRTIO_BALLOON_F_FREE_PAGE_HINT)) {
-        return offsetof(struct virtio_balloon_config, poison_val);
-    }
     return offsetof(struct virtio_balloon_config, free_page_report_cmd_id);
 }
 
@@ -634,6 +641,7 @@ static void virtio_balloon_get_config(VirtIODevice *vdev, uint8_t *config_data)
 
     config.num_pages = cpu_to_le32(dev->num_pages);
     config.actual = cpu_to_le32(dev->actual);
+    config.poison_val = cpu_to_le32(dev->poison_val);
 
     if (dev->free_page_report_status == FREE_PAGE_REPORT_S_REQUESTED) {
         config.free_page_report_cmd_id =
@@ -697,6 +705,8 @@ static void virtio_balloon_set_config(VirtIODevice *vdev,
         qapi_event_send_balloon_change(vm_ram_size -
                         ((ram_addr_t) dev->actual << VIRTIO_BALLOON_PFN_SHIFT));
     }
+    dev->poison_val = virtio_vdev_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON) ? 
+                      le32_to_cpu(config.poison_val) : 0;
     trace_virtio_balloon_set_config(dev->actual, oldactual);
 }
 
@@ -706,6 +716,9 @@ static uint64_t virtio_balloon_get_features(VirtIODevice *vdev, uint64_t f,
     VirtIOBalloon *dev = VIRTIO_BALLOON(vdev);
     f |= dev->host_features;
     virtio_add_feature(&f, VIRTIO_BALLOON_F_STATS_VQ);
+    if (virtio_has_feature(f, VIRTIO_BALLOON_F_FREE_PAGE_HINT)) {
+        virtio_add_feature(&f, VIRTIO_BALLOON_F_PAGE_POISON);
+    }
 
     return f;
 }
@@ -847,6 +860,8 @@ static void virtio_balloon_device_reset(VirtIODevice *vdev)
         g_free(s->stats_vq_elem);
         s->stats_vq_elem = NULL;
     }
+
+    s->poison_val = 0;
 }
 
 static void virtio_balloon_set_status(VirtIODevice *vdev, uint8_t status)
@@ -909,6 +924,8 @@ static Property virtio_balloon_properties[] = {
                     VIRTIO_BALLOON_F_DEFLATE_ON_OOM, false),
     DEFINE_PROP_BIT("free-page-hint", VirtIOBalloon, host_features,
                     VIRTIO_BALLOON_F_FREE_PAGE_HINT, false),
+    DEFINE_PROP_BIT("x-page-poison", VirtIOBalloon, host_features,
+                    VIRTIO_BALLOON_F_PAGE_POISON, false),
     /* QEMU 4.0 accidentally changed the config size even when free-page-hint
      * is disabled, resulting in QEMU 3.1 migration incompatibility.  This
      * property retains this quirk for QEMU 4.1 machine types.
diff --git a/include/hw/virtio/virtio-balloon.h b/include/hw/virtio/virtio-balloon.h
index d1c968d2376e..7fe78e5c14d7 100644
--- a/include/hw/virtio/virtio-balloon.h
+++ b/include/hw/virtio/virtio-balloon.h
@@ -70,6 +70,7 @@ typedef struct VirtIOBalloon {
     uint32_t host_features;
 
     bool qemu_4_0_config_size;
+    uint32_t poison_val;
 } VirtIOBalloon;
 
 #endif

[PATCH v14 QEMU 2/3] virtio-balloon: Add bit to notify guest of unused page reporting

[Alexander Duyck]

From: Alexander Duyck <alexander.h.duyck@linux.intel.com>

Add a bit for the page reporting feature provided by virtio-balloon.

This patch should be replaced once the feature is added to the Linux kernel
and the bit is backported into this exported kernel header.

Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
---
 include/standard-headers/linux/virtio_balloon.h |    1 +
 1 file changed, 1 insertion(+)

diff --git a/include/standard-headers/linux/virtio_balloon.h b/include/standard-headers/linux/virtio_balloon.h
index 9375ca2a70de..1c5f6d6f2de6 100644
--- a/include/standard-headers/linux/virtio_balloon.h
+++ b/include/standard-headers/linux/virtio_balloon.h
@@ -36,6 +36,7 @@
 #define VIRTIO_BALLOON_F_DEFLATE_ON_OOM	2 /* Deflate balloon on OOM */
 #define VIRTIO_BALLOON_F_FREE_PAGE_HINT	3 /* VQ to report free pages */
 #define VIRTIO_BALLOON_F_PAGE_POISON	4 /* Guest is using page poisoning */
+#define VIRTIO_BALLOON_F_REPORTING	5 /* Page reporting virtqueue */
 
 /* Size of a PFN in the balloon interface. */
 #define VIRTIO_BALLOON_PFN_SHIFT 12

[PATCH v14 QEMU 3/3] virtio-balloon: Provide a interface for unused page reporting

[Alexander Duyck]

From: Alexander Duyck <alexander.h.duyck@linux.intel.com>

Add support for what I am referring to as "unused page reporting".
Basically the idea is to function very similar to how the balloon works
in that we basically end up madvising the page as not being used. However
we don't really need to bother with any deflate type logic since the page
will be faulted back into the guest when it is read or written to.

This is meant to be a simplification of the existing balloon interface
to use for providing hints to what memory needs to be freed. I am assuming
this is safe to do as the deflate logic does not actually appear to do very
much other than tracking what subpages have been released and which ones
haven't.

Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
---
 hw/virtio/virtio-balloon.c         |   46 ++++++++++++++++++++++++++++++++++--
 include/hw/virtio/virtio-balloon.h |    2 +-
 2 files changed, 45 insertions(+), 3 deletions(-)

diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 6ecfec422309..47f253d016db 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -321,6 +321,40 @@ static void balloon_stats_set_poll_interval(Object *obj, Visitor *v,
     balloon_stats_change_timer(s, 0);
 }
 
+static void virtio_balloon_handle_report(VirtIODevice *vdev, VirtQueue *vq)
+{
+    VirtIOBalloon *dev = VIRTIO_BALLOON(vdev);
+    VirtQueueElement *elem;
+
+    while ((elem = virtqueue_pop(vq, sizeof(VirtQueueElement)))) {
+    	unsigned int i;
+
+        for (i = 0; i < elem->in_num; i++) {
+            void *addr = elem->in_sg[i].iov_base;
+            size_t size = elem->in_sg[i].iov_len;
+            ram_addr_t ram_offset;
+            size_t rb_page_size;
+            RAMBlock *rb;
+
+            if (qemu_balloon_is_inhibited() || dev->poison_val)
+                continue;
+
+            rb = qemu_ram_block_from_host(addr, false, &ram_offset);
+            rb_page_size = qemu_ram_pagesize(rb);
+
+            /* For now we will simply ignore unaligned memory regions */
+            if ((ram_offset | size) & (rb_page_size - 1))
+                continue;
+
+            ram_block_discard_range(rb, ram_offset, size);
+        }
+
+        virtqueue_push(vq, elem, 0);
+        virtio_notify(vdev, vq);
+        g_free(elem);
+    }
+}
+
 static void virtio_balloon_handle_output(VirtIODevice *vdev, VirtQueue *vq)
 {
     VirtIOBalloon *s = VIRTIO_BALLOON(vdev);
@@ -628,7 +662,8 @@ static size_t virtio_balloon_config_size(VirtIOBalloon *s)
         return sizeof(struct virtio_balloon_config);
     }
     if (virtio_has_feature(features, VIRTIO_BALLOON_F_PAGE_POISON) ||
-        virtio_has_feature(features, VIRTIO_BALLOON_F_FREE_PAGE_HINT)) {
+        virtio_has_feature(features, VIRTIO_BALLOON_F_FREE_PAGE_HINT) ||
+        virtio_has_feature(features, VIRTIO_BALLOON_F_REPORTING)) {
         return sizeof(struct virtio_balloon_config);
     }
     return offsetof(struct virtio_balloon_config, free_page_report_cmd_id);
@@ -716,7 +751,8 @@ static uint64_t virtio_balloon_get_features(VirtIODevice *vdev, uint64_t f,
     VirtIOBalloon *dev = VIRTIO_BALLOON(vdev);
     f |= dev->host_features;
     virtio_add_feature(&f, VIRTIO_BALLOON_F_STATS_VQ);
-    if (virtio_has_feature(f, VIRTIO_BALLOON_F_FREE_PAGE_HINT)) {
+    if (virtio_has_feature(f, VIRTIO_BALLOON_F_FREE_PAGE_HINT) ||
+        virtio_has_feature(f, VIRTIO_BALLOON_F_REPORTING)) {
         virtio_add_feature(&f, VIRTIO_BALLOON_F_PAGE_POISON);
     }
 
@@ -806,6 +842,10 @@ static void virtio_balloon_device_realize(DeviceState *dev, Error **errp)
     s->dvq = virtio_add_queue(vdev, 128, virtio_balloon_handle_output);
     s->svq = virtio_add_queue(vdev, 128, virtio_balloon_receive_stats);
 
+    if (virtio_has_feature(s->host_features, VIRTIO_BALLOON_F_REPORTING)) {
+        s->rvq = virtio_add_queue(vdev, 32, virtio_balloon_handle_report);
+    }
+
     if (virtio_has_feature(s->host_features,
                            VIRTIO_BALLOON_F_FREE_PAGE_HINT)) {
         s->free_page_vq = virtio_add_queue(vdev, VIRTQUEUE_MAX_SIZE,
@@ -932,6 +972,8 @@ static Property virtio_balloon_properties[] = {
      */
     DEFINE_PROP_BOOL("qemu-4-0-config-size", VirtIOBalloon,
                      qemu_4_0_config_size, false),
+    DEFINE_PROP_BIT("unused-page-reporting", VirtIOBalloon, host_features,
+                    VIRTIO_BALLOON_F_REPORTING, true),
     DEFINE_PROP_LINK("iothread", VirtIOBalloon, iothread, TYPE_IOTHREAD,
                      IOThread *),
     DEFINE_PROP_END_OF_LIST(),
diff --git a/include/hw/virtio/virtio-balloon.h b/include/hw/virtio/virtio-balloon.h
index 7fe78e5c14d7..db5bf7127112 100644
--- a/include/hw/virtio/virtio-balloon.h
+++ b/include/hw/virtio/virtio-balloon.h
@@ -42,7 +42,7 @@ enum virtio_balloon_free_page_report_status {
 
 typedef struct VirtIOBalloon {
     VirtIODevice parent_obj;
-    VirtQueue *ivq, *dvq, *svq, *free_page_vq;
+    VirtQueue *ivq, *dvq, *svq, *free_page_vq, *rvq;
     uint32_t free_page_report_status;
     uint32_t num_pages;
     uint32_t actual;

Re: [PATCH v14 0/6] mm / virtio: Provide support for unused page reporting

[David Hildenbrand]

On 19.11.19 22:46, Alexander Duyck wrote:
> This series provides an asynchronous means of reporting unused guest
> pages to a hypervisor so that the memory associated with those pages can
> be dropped and reused by other processes and/or guests on the host. Using
> this it is possible to avoid unnecessary I/O to disk and greatly improve
> performance in the case of memory overcommit on the host.
> 
> When enabled it will allocate a set of statistics to track the number of
> reported pages. When the nr_free for a given free area is greater than
> this by the high water mark we will schedule a worker to begin pulling the
> non-reported memory and to provide it to the reporting interface via a
> scatterlist.
> 
> Currently this is only in use by virtio-balloon however there is the hope
> that at some point in the future other hypervisors might be able to make
> use of it. In the virtio-balloon/QEMU implementation the hypervisor is
> currently using MADV_DONTNEED to indicate to the host kernel that the page
> is currently unused. It will be zeroed and faulted back into the guest the
> next time the page is accessed.

Remind me why we are using MADV_DONTNEED? Mostly for debugging purposes
right now, right? Did you do any measurements with MADV_FREE? I guess
there should be quite a performance increase in some scenarios.

> 
> To track if a page is reported or not the Uptodate flag was repurposed and
> used as a Reported flag for Buddy pages. We walk though the free list
> isolating pages and adding them to the scatterlist until we either
> encounter the end of the list or have filled the scatterlist with pages to
> be reported. If we fill the scatterlist before we reach the end of the
> list we rotate the list so that the first unreported page we encounter is
> moved to the head of the list as that is where we will resume after we
> have freed the reported pages back into the tail of the list.

So the boundary pointer didn't actually provide that big of a benefit I
assume (IOW, worst thing is you have to re-scan the whole list)?

> 
> Below are the results from various benchmarks. I primarily focused on two
> tests. The first is the will-it-scale/page_fault2 test, and the other is
> a modified version of will-it-scale/page_fault1 that was enabled to use
> THP. I did this as it allows for better visibility into different parts
> of the memory subsystem. The guest is running with 32G for RAM on one
> node of a E5-2630 v3. The host has had some power saving features disabled
> by setting the /dev/cpu_dma_latency value to 10ms.
> 
> Test                page_fault1 (THP)     page_fault2
> Name         tasks  Process Iter  STDEV  Process Iter  STDEV
> Baseline         1    1203934.75  0.04%     379940.75  0.11%
>                 16    8828217.00  0.85%    3178653.00  1.28%
> 
> Patches applied  1    1207961.25  0.10%     380852.25  0.25%
>                 16    8862373.00  0.98%    3246397.25  0.68%
> 
> Patches enabled  1    1207758.75  0.17%     373079.25  0.60%
>  MADV disabled  16    8870373.75  0.29%	   3204989.75  1.08%
> 
> Patches enabled  1    1261183.75  0.39%     373201.50  0.50%
>                 16    8371359.75  0.65%    3233665.50  0.84%
> 
> Patches enabled  1    1090201.50  0.25%     376967.25  0.29%
>  page shuffle   16    8108719.75  0.58%    3218450.25  1.07%
> 
> The results above are for a baseline with a linux-next-20191115 kernel,
> that kernel with this patch set applied but page reporting disabled in
> virtio-balloon, patches applied but the madvise disabled by direct
> assigning a device, the patches applied and page reporting fully
> enabled, and the patches enabled with page shuffling enabled.  These
> results include the deviation seen between the average value reported here
> versus the high and/or low value. I observed that during the test memory
> usage for the first three tests never dropped whereas with the patches
> fully enabled the VM would drop to using only a few GB of the host's
> memory when switching from memhog to page fault tests.
> 
> Most of the overhead seen with this patch set enabled seems due to page
> faults caused by accessing the reported pages and the host zeroing the page
> before giving it back to the guest. This overhead is much more visible when
> using THP than with standard 4K pages. In addition page shuffling seemed to
> increase the amount of faults generated due to an increase in memory churn.

MADV_FREE would be interesting.

> 
> The overall guest size is kept fairly small to only a few GB while the test
> is running. If the host memory were oversubscribed this patch set should
> result in a performance improvement as swapping memory in the host can be
> avoided.
> 
> A brief history on the background of unused page reporting can be found at:
> https://lore.kernel.org/lkml/29f43d5796feed0dec8e8bb98b187d9dac03b900.camel@linux.intel.com/
> 
> Changes from v12:
> https://lore.kernel.org/lkml/20191022221223.17338.5860.stgit@localhost.localdomain/
> Rebased on linux-next 20191031
> Renamed page_is_reported to page_reported
> Renamed add_page_to_reported_list to mark_page_reported
> Dropped unused definition of add_page_to_reported_list for non-reporting case
> Split free_area_reporting out from get_unreported_tail
> Minor updates to cover page
> 
> Changes from v13:
> https://lore.kernel.org/lkml/20191105215940.15144.65968.stgit@localhost.localdomain/
> Rewrote core reporting functionality
>   Merged patches 3 & 4
>   Dropped boundary list and related code
>   Folded get_reported_page into page_reporting_fill
>   Folded page_reporting_fill into page_reporting_cycle
> Pulled reporting functionality out of free_reported_page
>   Renamed it to __free_isolated_page
>   Moved page reporting specific bits to page_reporting_drain
> Renamed phdev to prdev since we aren't "hinting" we are "reporting"
> Added documentation to describe the usage of unused page reporting
> Updated cover page and patch descriptions to avoid mention of boundary
> 
> 
> ---
> 
> Alexander Duyck (6):
>       mm: Adjust shuffle code to allow for future coalescing
>       mm: Use zone and order instead of free area in free_list manipulators
>       mm: Introduce Reported pages
>       mm: Add unused page reporting documentation
>       virtio-balloon: Pull page poisoning config out of free page hinting
>       virtio-balloon: Add support for providing unused page reports to host
> 
> 
>  Documentation/vm/unused_page_reporting.rst |   44 ++++
>  drivers/virtio/Kconfig                     |    1 
>  drivers/virtio/virtio_balloon.c            |   88 +++++++
>  include/linux/mmzone.h                     |   56 +----
>  include/linux/page-flags.h                 |   11 +
>  include/linux/page_reporting.h             |   31 +++
>  include/uapi/linux/virtio_balloon.h        |    1 
>  mm/Kconfig                                 |   11 +
>  mm/Makefile                                |    1 
>  mm/memory_hotplug.c                        |    2 
>  mm/page_alloc.c                            |  181 +++++++++++----
>  mm/page_reporting.c                        |  337 ++++++++++++++++++++++++++++
>  mm/page_reporting.h                        |  125 ++++++++++
>  mm/shuffle.c                               |   12 -
>  mm/shuffle.h                               |    6 
>  15 files changed, 805 insertions(+), 102 deletions(-)

So roughly 100 LOC less added, that's nice to see :)

I'm planning to look into the details soon, just fairly busy lately. I
hope Mel Et al. can also comment.

-- 
Thanks,

David / dhildenb

Re: [PATCH v14 0/6] mm / virtio: Provide support for unused page reporting

[Alexander Duyck]

On Tue, 2019-11-26 at 13:20 +0100, David Hildenbrand wrote:
> On 19.11.19 22:46, Alexander Duyck wrote:
> > This series provides an asynchronous means of reporting unused guest
> > pages to a hypervisor so that the memory associated with those pages can
> > be dropped and reused by other processes and/or guests on the host. Using
> > this it is possible to avoid unnecessary I/O to disk and greatly improve
> > performance in the case of memory overcommit on the host.
> > 
> > When enabled it will allocate a set of statistics to track the number of
> > reported pages. When the nr_free for a given free area is greater than
> > this by the high water mark we will schedule a worker to begin pulling the
> > non-reported memory and to provide it to the reporting interface via a
> > scatterlist.
> > 
> > Currently this is only in use by virtio-balloon however there is the hope
> > that at some point in the future other hypervisors might be able to make
> > use of it. In the virtio-balloon/QEMU implementation the hypervisor is
> > currently using MADV_DONTNEED to indicate to the host kernel that the page
> > is currently unused. It will be zeroed and faulted back into the guest the
> > next time the page is accessed.
> 
> Remind me why we are using MADV_DONTNEED? Mostly for debugging purposes
> right now, right? Did you do any measurements with MADV_FREE? I guess
> there should be quite a performance increase in some scenarios.

There are actually a few reasons for not using MADV_FREE.

The first one was debugging as I could visibly see how much memory had
been freed by just checking the memory consumption by the guest. I didn't
have to wait for memory pressure to trigger the memory freeing. In
addition it would force the pages out of the guest so it was much easier
to see if I was freeing the wrong pages.

The second reason is because it is much more portable. The MADV_FREE has
only been a part of the Linux kernel since about 4.5. So if you are
running on an older kernel the option might not be available.

The third reason is simply effort involved. If I used MADV_DONTNEED then I
can just use ram_block_discard_range which is the same function used by
other parts of the balloon driver.

Finally it is my understanding is that MADV_FREE only works on anonymous
memory (https://elixir.bootlin.com/linux/v5.4/source/mm/madvise.c#L700). I
was concerned that using MADV_FREE wouldn't work if used on file backed
memory such as hugetlbfs which is an option for QEMU if I am not mistaken.

> > To track if a page is reported or not the Uptodate flag was repurposed and
> > used as a Reported flag for Buddy pages. We walk though the free list
> > isolating pages and adding them to the scatterlist until we either
> > encounter the end of the list or have filled the scatterlist with pages to
> > be reported. If we fill the scatterlist before we reach the end of the
> > list we rotate the list so that the first unreported page we encounter is
> > moved to the head of the list as that is where we will resume after we
> > have freed the reported pages back into the tail of the list.
> 
> So the boundary pointer didn't actually provide that big of a benefit I
> assume (IOW, worst thing is you have to re-scan the whole list)?

I rewrote the code quite a bit to get rid of the disadvantages.
Specifically what the boundary pointer was doing was saving our place in
the list when we left. Even without that we still had to re-scan the
entire list with each zone processed anyway. With these changes we end up
potentially having to perform one additional rescan per free list.

Where things differ now is that the fetching function doesn't bail out of
the list and start over per page. Instead it fills the entire scatterlist
before it exits, and before doing so it will advance the head to the next
non-reported page in the list. In addition instead of walking all of the
orders and migrate types looking for each page the code is now more
methodical and will only work one free list at a time and do not revisit
it until we have processed the entire zone.

Even with all that we still take a pretty significant performance hit in
the page shuffing case, however I am willing to give that up for the sake
of being less intrusive.

> > Below are the results from various benchmarks. I primarily focused on two
> > tests. The first is the will-it-scale/page_fault2 test, and the other is
> > a modified version of will-it-scale/page_fault1 that was enabled to use
> > THP. I did this as it allows for better visibility into different parts
> > of the memory subsystem. The guest is running with 32G for RAM on one
> > node of a E5-2630 v3. The host has had some power saving features disabled
> > by setting the /dev/cpu_dma_latency value to 10ms.
> > 
> > Test                page_fault1 (THP)     page_fault2
> > Name         tasks  Process Iter  STDEV  Process Iter  STDEV
> > Baseline         1    1203934.75  0.04%     379940.75  0.11%
> >                 16    8828217.00  0.85%    3178653.00  1.28%
> > 
> > Patches applied  1    1207961.25  0.10%     380852.25  0.25%
> >                 16    8862373.00  0.98%    3246397.25  0.68%
> > 
> > Patches enabled  1    1207758.75  0.17%     373079.25  0.60%
> >  MADV disabled  16    8870373.75  0.29%    3204989.75  1.08%
> > 
> > Patches enabled  1    1261183.75  0.39%     373201.50  0.50%
> >                 16    8371359.75  0.65%    3233665.50  0.84%
> > 
> > Patches enabled  1    1090201.50  0.25%     376967.25  0.29%
> >  page shuffle   16    8108719.75  0.58%    3218450.25  1.07%
> > 
> > The results above are for a baseline with a linux-next-20191115 kernel,
> > that kernel with this patch set applied but page reporting disabled in
> > virtio-balloon, patches applied but the madvise disabled by direct
> > assigning a device, the patches applied and page reporting fully
> > enabled, and the patches enabled with page shuffling enabled.  These
> > results include the deviation seen between the average value reported here
> > versus the high and/or low value. I observed that during the test memory
> > usage for the first three tests never dropped whereas with the patches
> > fully enabled the VM would drop to using only a few GB of the host's
> > memory when switching from memhog to page fault tests.
> > 
> > Most of the overhead seen with this patch set enabled seems due to page
> > faults caused by accessing the reported pages and the host zeroing the page
> > before giving it back to the guest. This overhead is much more visible when
> > using THP than with standard 4K pages. In addition page shuffling seemed to
> > increase the amount of faults generated due to an increase in memory churn.
> 
> MADV_FREE would be interesting.

I can probably code something up. However that is going to push a bunch of
complexity into the QEMU code and doesn't really mean much to the kernel
code. I can probably add it as another QEMU patch to the set since it is
just a matter of having a function similar to ram_block_discard_range that
uses MADV_FREE instead of MADV_DONTNEED.

> > The overall guest size is kept fairly small to only a few GB while the test
> > is running. If the host memory were oversubscribed this patch set should
> > result in a performance improvement as swapping memory in the host can be
> > avoided.
> > 
> > A brief history on the background of unused page reporting can be found at:
> > https://lore.kernel.org/lkml/29f43d5796feed0dec8e8bb98b187d9dac03b900.camel@linux.intel.com/
> > 
> > Changes from v12:
> > https://lore.kernel.org/lkml/20191022221223.17338.5860.stgit@localhost.localdomain/
> > Rebased on linux-next 20191031
> > Renamed page_is_reported to page_reported
> > Renamed add_page_to_reported_list to mark_page_reported
> > Dropped unused definition of add_page_to_reported_list for non-reporting case
> > Split free_area_reporting out from get_unreported_tail
> > Minor updates to cover page
> > 
> > Changes from v13:
> > https://lore.kernel.org/lkml/20191105215940.15144.65968.stgit@localhost.localdomain/
> > Rewrote core reporting functionality
> >   Merged patches 3 & 4
> >   Dropped boundary list and related code
> >   Folded get_reported_page into page_reporting_fill
> >   Folded page_reporting_fill into page_reporting_cycle
> > Pulled reporting functionality out of free_reported_page
> >   Renamed it to __free_isolated_page
> >   Moved page reporting specific bits to page_reporting_drain
> > Renamed phdev to prdev since we aren't "hinting" we are "reporting"
> > Added documentation to describe the usage of unused page reporting
> > Updated cover page and patch descriptions to avoid mention of boundary
> > 
> > 
> > ---
> > 
> > Alexander Duyck (6):
> >       mm: Adjust shuffle code to allow for future coalescing
> >       mm: Use zone and order instead of free area in free_list manipulators
> >       mm: Introduce Reported pages
> >       mm: Add unused page reporting documentation
> >       virtio-balloon: Pull page poisoning config out of free page hinting
> >       virtio-balloon: Add support for providing unused page reports to host
> > 
> > 
> >  Documentation/vm/unused_page_reporting.rst |   44 ++++
> >  drivers/virtio/Kconfig                     |    1 
> >  drivers/virtio/virtio_balloon.c            |   88 +++++++
> >  include/linux/mmzone.h                     |   56 +----
> >  include/linux/page-flags.h                 |   11 +
> >  include/linux/page_reporting.h             |   31 +++
> >  include/uapi/linux/virtio_balloon.h        |    1 
> >  mm/Kconfig                                 |   11 +
> >  mm/Makefile                                |    1 
> >  mm/memory_hotplug.c                        |    2 
> >  mm/page_alloc.c                            |  181 +++++++++++----
> >  mm/page_reporting.c                        |  337 ++++++++++++++++++++++++++++
> >  mm/page_reporting.h                        |  125 ++++++++++
> >  mm/shuffle.c                               |   12 -
> >  mm/shuffle.h                               |    6 
> >  15 files changed, 805 insertions(+), 102 deletions(-)
> 
> So roughly 100 LOC less added, that's nice to see :)
> 
> I'm planning to look into the details soon, just fairly busy lately. I
> hope Mel Et al. can also comment.

Agreed. I can see if I can generate something to get the MADV_FREE
numbers. I suspect they were probably be somewhere between the MADV
disabled and fully enabled case, since we will still be taking the page
faults but not doing the zeroing.

Re: [PATCH v14 0/6] mm / virtio: Provide support for unused page reporting

[David Hildenbrand]

On 26.11.19 17:45, Alexander Duyck wrote:
> On Tue, 2019-11-26 at 13:20 +0100, David Hildenbrand wrote:
>> On 19.11.19 22:46, Alexander Duyck wrote:
>>> This series provides an asynchronous means of reporting unused guest
>>> pages to a hypervisor so that the memory associated with those pages can
>>> be dropped and reused by other processes and/or guests on the host. Using
>>> this it is possible to avoid unnecessary I/O to disk and greatly improve
>>> performance in the case of memory overcommit on the host.
>>>
>>> When enabled it will allocate a set of statistics to track the number of
>>> reported pages. When the nr_free for a given free area is greater than
>>> this by the high water mark we will schedule a worker to begin pulling the
>>> non-reported memory and to provide it to the reporting interface via a
>>> scatterlist.
>>>
>>> Currently this is only in use by virtio-balloon however there is the hope
>>> that at some point in the future other hypervisors might be able to make
>>> use of it. In the virtio-balloon/QEMU implementation the hypervisor is
>>> currently using MADV_DONTNEED to indicate to the host kernel that the page
>>> is currently unused. It will be zeroed and faulted back into the guest the
>>> next time the page is accessed.
>>
>> Remind me why we are using MADV_DONTNEED? Mostly for debugging purposes
>> right now, right? Did you do any measurements with MADV_FREE? I guess
>> there should be quite a performance increase in some scenarios.
> 
> There are actually a few reasons for not using MADV_FREE.
> 
> The first one was debugging as I could visibly see how much memory had
> been freed by just checking the memory consumption by the guest. I didn't
> have to wait for memory pressure to trigger the memory freeing. In
> addition it would force the pages out of the guest so it was much easier
> to see if I was freeing the wrong pages.
> 
> The second reason is because it is much more portable. The MADV_FREE has
> only been a part of the Linux kernel since about 4.5. So if you are
> running on an older kernel the option might not be available.

I guess optionally enabling it (for !filebacked and !huge pages) in QEMU
after sensing would be possible. Fallback to ram_discard_range().

> 
> The third reason is simply effort involved. If I used MADV_DONTNEED then I
> can just use ram_block_discard_range which is the same function used by
> other parts of the balloon driver.

Yes, that makes perfect sense.

> 
> Finally it is my understanding is that MADV_FREE only works on anonymous
> memory (https://elixir.bootlin.com/linux/v5.4/source/mm/madvise.c#L700). I
> was concerned that using MADV_FREE wouldn't work if used on file backed
> memory such as hugetlbfs which is an option for QEMU if I am not mistaken.

Yes, MADV_FREE works just like MADV_DONTNEED only on anonymous memory.
In case of files/hugetlbfs you have to use

fallocate(rb->fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, ...).

E.g., see qemu/exec.c:ram_block_discard_range. You can do something
similar to this:


static bool madv_free_sensed, madv_free_available;
int ret = -EINVAL;

/*
 * MADV_FREE only works on anonymous memory, and especially not on
 * hugetlbfs. Older kernels don't support it.
 */
if (rb->page_size == qemu_host_page_size && rb->fb != -1 &&
    (!madv_free_sensed || madv_free_available)) {
    ret = madvise(start, length, MADV_FREE);
    if (ret) {
	madv_free_sensed = true;
        madv_free_available = false;
    } else if (!madv_free_sensed) {
        madv_free_sensed = true;
	madv_free_available = true;
    }
}

/* fallback to MADV_DONTNEED / FALLOC_FL_PUNCH_HOLE */
if (ret) {
    ram_block_discard_range(rb, start, length);
}


I agree that something like should be an addon to the current patch set.

> 
>>> To track if a page is reported or not the Uptodate flag was repurposed and
>>> used as a Reported flag for Buddy pages. We walk though the free list
>>> isolating pages and adding them to the scatterlist until we either
>>> encounter the end of the list or have filled the scatterlist with pages to
>>> be reported. If we fill the scatterlist before we reach the end of the
>>> list we rotate the list so that the first unreported page we encounter is
>>> moved to the head of the list as that is where we will resume after we
>>> have freed the reported pages back into the tail of the list.
>>
>> So the boundary pointer didn't actually provide that big of a benefit I
>> assume (IOW, worst thing is you have to re-scan the whole list)?
> 
> I rewrote the code quite a bit to get rid of the disadvantages.
> Specifically what the boundary pointer was doing was saving our place in
> the list when we left. Even without that we still had to re-scan the
> entire list with each zone processed anyway. With these changes we end up
> potentially having to perform one additional rescan per free list.
> 
> Where things differ now is that the fetching function doesn't bail out of
> the list and start over per page. Instead it fills the entire scatterlist
> before it exits, and before doing so it will advance the head to the next
> non-reported page in the list. In addition instead of walking all of the
> orders and migrate types looking for each page the code is now more
> methodical and will only work one free list at a time and do not revisit
> it until we have processed the entire zone.

Makes perfect sense to me.

> 
> Even with all that we still take a pretty significant performance hit in
> the page shuffing case, however I am willing to give that up for the sake
> of being less intrusive.

Makes sense as well, especially for a first version.

> 
>>> Below are the results from various benchmarks. I primarily focused on two
>>> tests. The first is the will-it-scale/page_fault2 test, and the other is
>>> a modified version of will-it-scale/page_fault1 that was enabled to use
>>> THP. I did this as it allows for better visibility into different parts
>>> of the memory subsystem. The guest is running with 32G for RAM on one
>>> node of a E5-2630 v3. The host has had some power saving features disabled
>>> by setting the /dev/cpu_dma_latency value to 10ms.
>>>
>>> Test                page_fault1 (THP)     page_fault2
>>> Name         tasks  Process Iter  STDEV  Process Iter  STDEV
>>> Baseline         1    1203934.75  0.04%     379940.75  0.11%
>>>                 16    8828217.00  0.85%    3178653.00  1.28%
>>>
>>> Patches applied  1    1207961.25  0.10%     380852.25  0.25%
>>>                 16    8862373.00  0.98%    3246397.25  0.68%
>>>
>>> Patches enabled  1    1207758.75  0.17%     373079.25  0.60%
>>>  MADV disabled  16    8870373.75  0.29%    3204989.75  1.08%
>>>
>>> Patches enabled  1    1261183.75  0.39%     373201.50  0.50%
>>>                 16    8371359.75  0.65%    3233665.50  0.84%
>>>
>>> Patches enabled  1    1090201.50  0.25%     376967.25  0.29%
>>>  page shuffle   16    8108719.75  0.58%    3218450.25  1.07%
>>>
>>> The results above are for a baseline with a linux-next-20191115 kernel,
>>> that kernel with this patch set applied but page reporting disabled in
>>> virtio-balloon, patches applied but the madvise disabled by direct
>>> assigning a device, the patches applied and page reporting fully
>>> enabled, and the patches enabled with page shuffling enabled.  These
>>> results include the deviation seen between the average value reported here
>>> versus the high and/or low value. I observed that during the test memory
>>> usage for the first three tests never dropped whereas with the patches
>>> fully enabled the VM would drop to using only a few GB of the host's
>>> memory when switching from memhog to page fault tests.
>>>
>>> Most of the overhead seen with this patch set enabled seems due to page
>>> faults caused by accessing the reported pages and the host zeroing the page
>>> before giving it back to the guest. This overhead is much more visible when
>>> using THP than with standard 4K pages. In addition page shuffling seemed to
>>> increase the amount of faults generated due to an increase in memory churn.
>>
>> MADV_FREE would be interesting.
> 
> I can probably code something up. However that is going to push a bunch of
> complexity into the QEMU code and doesn't really mean much to the kernel
> code. I can probably add it as another QEMU patch to the set since it is
> just a matter of having a function similar to ram_block_discard_range that
> uses MADV_FREE instead of MADV_DONTNEED.

Yes, addon patch makes perfect sense. The nice thing about MADV_FREE is
that you only take back pages from a process when really under memory
pressure (before going to SWAP). You will still get a pagefault on the
next access (to identify that the page is still in use after all), but
don't have to fault in a fresh page.

> 
>>> The overall guest size is kept fairly small to only a few GB while the test
>>> is running. If the host memory were oversubscribed this patch set should
>>> result in a performance improvement as swapping memory in the host can be
>>> avoided.
>>>
>>> A brief history on the background of unused page reporting can be found at:
>>> https://lore.kernel.org/lkml/29f43d5796feed0dec8e8bb98b187d9dac03b900.camel@linux.intel.com/
>>>
>>> Changes from v12:
>>> https://lore.kernel.org/lkml/20191022221223.17338.5860.stgit@localhost.localdomain/
>>> Rebased on linux-next 20191031
>>> Renamed page_is_reported to page_reported
>>> Renamed add_page_to_reported_list to mark_page_reported
>>> Dropped unused definition of add_page_to_reported_list for non-reporting case
>>> Split free_area_reporting out from get_unreported_tail
>>> Minor updates to cover page
>>>
>>> Changes from v13:
>>> https://lore.kernel.org/lkml/20191105215940.15144.65968.stgit@localhost.localdomain/
>>> Rewrote core reporting functionality
>>>   Merged patches 3 & 4
>>>   Dropped boundary list and related code
>>>   Folded get_reported_page into page_reporting_fill
>>>   Folded page_reporting_fill into page_reporting_cycle
>>> Pulled reporting functionality out of free_reported_page
>>>   Renamed it to __free_isolated_page
>>>   Moved page reporting specific bits to page_reporting_drain
>>> Renamed phdev to prdev since we aren't "hinting" we are "reporting"
>>> Added documentation to describe the usage of unused page reporting
>>> Updated cover page and patch descriptions to avoid mention of boundary
>>>
>>>
>>> ---
>>>
>>> Alexander Duyck (6):
>>>       mm: Adjust shuffle code to allow for future coalescing
>>>       mm: Use zone and order instead of free area in free_list manipulators
>>>       mm: Introduce Reported pages
>>>       mm: Add unused page reporting documentation
>>>       virtio-balloon: Pull page poisoning config out of free page hinting
>>>       virtio-balloon: Add support for providing unused page reports to host
>>>
>>>
>>>  Documentation/vm/unused_page_reporting.rst |   44 ++++
>>>  drivers/virtio/Kconfig                     |    1 
>>>  drivers/virtio/virtio_balloon.c            |   88 +++++++
>>>  include/linux/mmzone.h                     |   56 +----
>>>  include/linux/page-flags.h                 |   11 +
>>>  include/linux/page_reporting.h             |   31 +++
>>>  include/uapi/linux/virtio_balloon.h        |    1 
>>>  mm/Kconfig                                 |   11 +
>>>  mm/Makefile                                |    1 
>>>  mm/memory_hotplug.c                        |    2 
>>>  mm/page_alloc.c                            |  181 +++++++++++----
>>>  mm/page_reporting.c                        |  337 ++++++++++++++++++++++++++++
>>>  mm/page_reporting.h                        |  125 ++++++++++
>>>  mm/shuffle.c                               |   12 -
>>>  mm/shuffle.h                               |    6 
>>>  15 files changed, 805 insertions(+), 102 deletions(-)
>>
>> So roughly 100 LOC less added, that's nice to see :)
>>
>> I'm planning to look into the details soon, just fairly busy lately. I
>> hope Mel Et al. can also comment.
> 
> Agreed. I can see if I can generate something to get the MADV_FREE
> numbers. I suspect they were probably be somewhere between the MADV
> disabled and fully enabled case, since we will still be taking the page
> faults but not doing the zeroing.

Exactly.

-- 
Thanks,

David / dhildenb

Re: [PATCH v14 1/6] mm: Adjust shuffle code to allow for future coalescing

[Mel Gorman]

On Tue, Nov 19, 2019 at 01:46:18PM -0800, Alexander Duyck wrote:
> From: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> 
> Move the head/tail adding logic out of the shuffle code and into the
> __free_one_page function since ultimately that is where it is really
> needed anyway. By doing this we should be able to reduce the overhead
> and can consolidate all of the list addition bits in one spot.
> 
> Acked-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>

Acked-by: Mel Gorman <mgorman@techsingularity.net>

-- 
Mel Gorman
SUSE Labs

Re: [PATCH v14 2/6] mm: Use zone and order instead of free area in free_list manipulators

[Mel Gorman]

On Tue, Nov 19, 2019 at 01:46:26PM -0800, Alexander Duyck wrote:
> From: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> 
> In order to enable the use of the zone from the list manipulator functions
> I will need access to the zone pointer. As it turns out most of the
> accessors were always just being directly passed &zone->free_area[order]
> anyway so it would make sense to just fold that into the function itself
> and pass the zone and order as arguments instead of the free area.
> 
> In order to be able to reference the zone we need to move the declaration
> of the functions down so that we have the zone defined before we define the
> list manipulation functions. Since the functions are only used in the file
> mm/page_alloc.c we can just move them there to reduce noise in the header.
> 
> Reviewed-by: Dan Williams <dan.j.williams@intel.com>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Pankaj Gupta <pagupta@redhat.com>
> Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>

Acked-by: Mel Gorman <mgorman@techsingularity.net>

-- 
Mel Gorman
SUSE Labs

Re: [PATCH v14 3/6] mm: Introduce Reported pages

[Mel Gorman]

On Tue, Nov 19, 2019 at 01:46:33PM -0800, Alexander Duyck wrote:
> From: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> 
> In order to pave the way for free page reporting in virtualized
> environments we will need a way to get pages out of the free lists and
> identify those pages after they have been returned. To accomplish this,
> this patch adds the concept of a Reported Buddy, which is essentially
> meant to just be the Uptodate flag used in conjunction with the Buddy
> page type.
> 
> To prevent the reported pages from leaking outside of the buddy lists I
> have added a call to clear_reported_page to the del_page_from_free_list
> function. As a result any reported page that is split, merged, or
> allocated will have the flag cleared prior to the PageBuddy value being
> cleared.
> 
> The process for reporting pages is fairly simple. Once the nr_free for
> a given free area has exceeded the number of pages reported for that area
> plus a certain high watermark value we will flag the zone as needing
> reporting and schedule the worker thread to start reporting. That worker
> thread will begin working from the lowest supported page reporting order
> up to MAX_ORDER - 1 pulling unreported pages from the free list and
> storing them in the scatterlist.
> 
> When processing each individual free list it is necessary for the worker
> thread to release the zone lock when it needs to stop and report the full
> scatterlist of pages. To reduce the work of the next iteration the worker
> thread will rotate the free list so that the first unreported page in the
> free list becomes the first entry in the list. Doing this we should only
> have to walk an entire free list no more than twice assuming large
> numbers of pages are not being added to the tail of the list.
> 
> It will then call a reporting function providing information on how many
> entries are in the scatterlist. Once the function completes it will return
> the pages to the tail of the free area from which they were allocated and
> start over pulling more pages from the free areas until there are no
> longer enough pages to report on to keep the worker busy.
> 
> The worker thread will work in a round-robin fashion making its way
> though each zone requesting reporting, and through each reportable free
> list within that zone. Once all free areas within the zone is below the
> high watermark level for free pages to report the flag indicating that the
> zone has requested reporting will be cleared, and if no zones are
> requesting reporting the worker thread will exit.
> 
> Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>

Ok, I'm ok with how this hooks into the allocator as the overhead is
minimal. However, the patch itself still includes a number of
optimisations instead of being a bare-boned implementation of the
feature with optimisations layered on top. I think some of the
optimisations are also broken but it's harder to be sure because both
the feature and the optimisations are lumped together.

> ---
>  include/linux/mmzone.h         |   12 +
>  include/linux/page-flags.h     |   11 +
>  include/linux/page_reporting.h |   31 ++++
>  mm/Kconfig                     |   11 +
>  mm/Makefile                    |    1 
>  mm/memory_hotplug.c            |    2 
>  mm/page_alloc.c                |   53 ++++++
>  mm/page_reporting.c            |  337 ++++++++++++++++++++++++++++++++++++++++
>  mm/page_reporting.h            |  125 +++++++++++++++
>  9 files changed, 579 insertions(+), 4 deletions(-)
>  create mode 100644 include/linux/page_reporting.h
>  create mode 100644 mm/page_reporting.c
>  create mode 100644 mm/page_reporting.h
> 
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index 8d93106490f3..9647499983b1 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -478,6 +478,14 @@ struct zone {
>  	seqlock_t		span_seqlock;
>  #endif
>  
> +#ifdef CONFIG_PAGE_REPORTING
> +	/*
> +	 * Pointer to reported page tracking statistics array. The size of
> +	 * the array is MAX_ORDER - PAGE_REPORTING_MIN_ORDER. NULL when
> +	 * unused page reporting is not present.
> +	 */
> +	unsigned long		*reported_pages;
> +#endif
>  	int initialized;
>  
>  	/* Write-intensive fields used from the page allocator */

Comment could be better. We know it's a pointer.

/*
 * When a device is registered to track free pages, this array tracks
 * the number of pages already reported at each order from
 * PAGE_REPORTING_MIN_ORDER up to MAX_ORDER.
 */

> @@ -550,6 +558,10 @@ enum zone_flags {
>  	ZONE_BOOSTED_WATERMARK,		/* zone recently boosted watermarks.
>  					 * Cleared when kswapd is woken.
>  					 */
> +	ZONE_PAGE_REPORTING_REQUESTED,	/* zone enabled page reporting and has
> +					 * requested flushing the data out of
> +					 * higher order pages.
> +					 */
>  };

Set if a *driver* enabled page reporting and has requested that unused
pages are reported?

"flushing the data" would normally indicate that it's something to do
with writeback.

>  
>  static inline unsigned long zone_managed_pages(struct zone *zone)
> diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
> index 1bf83c8fcaa7..a3a3b15b56a8 100644
> --- a/include/linux/page-flags.h
> +++ b/include/linux/page-flags.h
> @@ -163,6 +163,9 @@ enum pageflags {
>  
>  	/* non-lru isolated movable page */
>  	PG_isolated = PG_reclaim,
> +
> +	/* Buddy pages. Used to track which pages have been reported */
> +	PG_reported = PG_uptodate,
>  };
>  
>  #ifndef __GENERATING_BOUNDS_H

Only valid for buddy pages.....

> diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h
> new file mode 100644
> index 000000000000..925a16b1d14b
> --- /dev/null
> +++ b/include/linux/page_reporting.h
> @@ -0,0 +1,31 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _LINUX_PAGE_REPORTING_H
> +#define _LINUX_PAGE_REPORTING_H
> +
> +#include <linux/mmzone.h>
> +
> +struct page_reporting_dev_info {
> +	/* function that alters pages to make them "reported" */
> +	void (*report)(struct page_reporting_dev_info *prdev,
> +		       unsigned int nents);
> +
> +	/* pointer to scatterlist containing pages to be processed */
> +	struct scatterlist *sg;
> +

The choice of scatterlist is curious because it adds unnecessary complexity
without needing the benefits scatterlist and sg_table provides. All you
really need is an array of pages that is NULL terminated because you can
get the order of the struct page when it's a buddy page.

> +	/*
> +	 * Upper limit on the number of pages that the report function
> +	 * expects to be placed into the scatterlist to be processed.
> +	 */
> +	unsigned int capacity;
> +

Instead of requiring the driver to specify capacity, why did you not
simply make it PAGE_REPORTING_HWM and statically declare the array?

> +	/* The number of zones requesting reporting */
> +	atomic_t refcnt;
> +

The refcnt is overkill. Simply loop over all zones until there is a
clear pass. Despite being atomic, it actually requires the zone lock to
avoid races in places like this

		if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
			refcnt = page_reporting_process_zone(prdev, zone);

The optimisation is small and it's relatively subtle.

>  static inline void __free_one_page(struct page *page,
>  		unsigned long pfn,
>  		struct zone *zone, unsigned int order,
> -		int migratetype)
> +		int migratetype, bool reported)
>  {
>  	struct capture_control *capc = task_capc(zone);
>  	unsigned long uninitialized_var(buddy_pfn);
> @@ -1048,7 +1053,9 @@ static inline void __free_one_page(struct page *page,
>  done_merging:
>  	set_page_order(page, order);
>  
> -	if (is_shuffle_order(order))
> +	if (reported)
> +		to_tail = true;
> +	else if (is_shuffle_order(order))
>  		to_tail = shuffle_pick_tail();
>  	else
>  		to_tail = buddy_merge_likely(pfn, buddy_pfn, page, order);
> @@ -1057,6 +1064,14 @@ static inline void __free_one_page(struct page *page,
>  		add_to_free_list_tail(page, zone, order, migratetype);
>  	else
>  		add_to_free_list(page, zone, order, migratetype);
> +
> +	/*
> +	 * No need to notify on a reported page as the total count of
> +	 * unreported pages will not have increased since we have essentially
> +	 * merged the reported page with one or more unreported pages.
> +	 */
> +	if (!reported)
> +		page_reporting_notify_free(zone, order);
>  }
>  
>  /*

If a reported page is merged with a larger buddy then the counts for the
lower order needs to be updated or the counter gets out of sync. Then the
reported status of the page needs to be cleared as the larger block may
only be partially reported and the count is wrong. I know why you want
the reported pages but it really should be in its own patch. This patch
really should be just the bare essentials to support reporting of free
pages even if it's sub-optimal. I know you also use reported_pages as
part of a stop condition but a basic stop condition is simply to do a
single pass.

> @@ -3228,6 +3243,36 @@ int __isolate_free_page(struct page *page, unsigned int order)
>  	return 1UL << order;
>  }
>  
> +#ifdef CONFIG_PAGE_REPORTING
> +/**
> + * __free_isolated_page - Return a now-isolated page back where we got it
> + * @page: Page that was isolated
> + * @order: Order of the isolated page
> + *
> + * This function is meant to return a page pulled from the free lists via
> + * __isolate_free_page back to the free lists they were pulled from.
> + */

Isolated within mm has special meaning already. __putback_reported_page?

> +void __free_isolated_page(struct page *page, unsigned int order)
> +{
> +	struct zone *zone = page_zone(page);
> +	unsigned long pfn;
> +	unsigned int mt;
> +
> +	/* zone lock should be held when this function is called */
> +	lockdep_assert_held(&zone->lock);
> +
> +	pfn = page_to_pfn(page);
> +	mt = get_pfnblock_migratetype(page, pfn);
> +
> +	/*
> +	 * Return isolated page to tail of freelist and don't bother with
> +	 * triggering the page reporting notifiers since this page was
> +	 * previously on the freelist and has likely already been reported.
> +	 */
> +	__free_one_page(page, pfn, zone, order, mt, true);
> +}
> +#endif /* CONFIG_PAGE_REPORTING */
> +
>  /*
>   * Update NUMA hit/miss statistics
>   *
> diff --git a/mm/page_reporting.c b/mm/page_reporting.c
> new file mode 100644
> index 000000000000..4844f0aa2904
> --- /dev/null
> +++ b/mm/page_reporting.c
> @@ -0,0 +1,337 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <linux/mm.h>
> +#include <linux/mmzone.h>
> +#include <linux/page_reporting.h>
> +#include <linux/gfp.h>
> +#include <linux/export.h>
> +#include <linux/delay.h>
> +#include <linux/scatterlist.h>
> +
> +#include "page_reporting.h"
> +#include "internal.h"
> +
> +static struct page_reporting_dev_info __rcu *pr_dev_info __read_mostly;
> +
> +#define for_each_reporting_migratetype_order(_order, _type) \
> +	for (_order = PAGE_REPORTING_MIN_ORDER; _order < MAX_ORDER; _order++) \
> +		for (_type = 0; _type < MIGRATE_TYPES; _type++) \
> +			if (!is_migrate_isolate(_type))
> +

This only has one user so simply put it in place. As this stands, _order
and _type can be evaluated multiple times which potentially leads to
chaos

> +static void page_reporting_populate_metadata(struct zone *zone)
> +{
> +	size_t size;
> +	int node;
> +
> +	/*
> +	 * We need to make sure we have somewhere to store the tracking
> +	 * data for how many reported pages are in the zone. To do that
> +	 * we need to make certain zone->reported_pages is populated.
> +	 */
> +	if (zone->reported_pages)
> +		return;
> +
> +	node = zone_to_nid(zone);
> +	size = (MAX_ORDER - PAGE_REPORTING_MIN_ORDER) * sizeof(unsigned long);
> +	zone->reported_pages = kzalloc_node(size, GFP_KERNEL, node);
> +}
> +

reported_pages is not necessarily updated by a CPU local to the zone.
There is not much point worrying about its locality.

> +static void
> +page_reporting_drain(struct page_reporting_dev_info *prdev, struct zone *zone)
> +{
> +	struct scatterlist *sg = prdev->sg;
> +
> +	/*
> +	 * Drain the now reported pages back into their respective
> +	 * free lists/areas. We assume at least one page is populated.
> +	 */
> +	do {
> +		unsigned int order = get_order(sg->length);
> +		struct page *page = sg_page(sg);
> +
> +		__free_isolated_page(page, order);
> +
> +		/*
> +		 * If page was not comingled with another page we can
> +		 * consider the result to be "reported" since the page
> +		 * hasn't been modified, otherwise we will need to
> +		 * report on the new larger page when we make our way
> +		 * up to that higher order.
> +		 */
> +		if (PageBuddy(page) && page_order(page) == order)
> +			mark_page_reported(page, zone, order);
> +	} while (!sg_is_last(sg++));
> +}
> +
> +/*
> + * The page reporting cycle consists of 4 stages, fill, report, drain, and
> + * idle. We will cycle through the first 3 stages until we cannot obtain a
> + * full scatterlist of pages, in that case we will switch to idle.
> + */

Document the return value because it's unclear why the number of populated
elements in sg is not tracked in page_reporting_dev_info. It seems
unnecessarily fragile.

> +static unsigned int
> +page_reporting_cycle(struct page_reporting_dev_info *prdev, struct zone *zone,
> +		     unsigned int order, unsigned int mt, unsigned int nents)
> +{
> +	struct list_head *list = &zone->free_area[order].free_list[mt];
> +	unsigned int page_len = PAGE_SIZE << order;
> +	struct scatterlist *sg = prdev->sg;
> +	struct page *page, *next;
> +
> +	/*
> +	 * Perform early check, if free area is empty there is
> +	 * nothing to process so we can skip this free_list.
> +	 */
> +	if (list_empty(list))
> +		return nents;
> +
> +	spin_lock_irq(&zone->lock);
> +
> +	/* loop through free list adding unreported pages to sg list */
> +	list_for_each_entry_safe(page, next, list, lru) {
> +		/* We are going to skip over the reported pages. */
> +		if (PageReported(page))
> +			continue;
> +
> +		/* Attempt to add page to sg list */
> +		if (nents < prdev->capacity) {
> +			if (!__isolate_free_page(page, order))
> +				break;
> +
> +			sg_set_page(&sg[nents++], page, page_len, 0);
> +			continue;
> +		}
> +
> +		/*
> +		 * Make the first non-reported entry in the free list
> +		 * the new head of the free list before we exit.
> +		 */
> +		if (!list_is_first(&page->lru, list))
> +			list_rotate_to_front(&page->lru, list);
> +
> +		/* release lock before waiting on report processing*/
> +		spin_unlock_irq(&zone->lock);
> +
> +		/* begin processing pages in local list */
> +		prdev->report(prdev, nents);
> +
> +		/* reset number of entries */
> +		nents = 0;
> +
> +		/* reacquire zone lock and resume processing free lists */
> +		spin_lock_irq(&zone->lock);
> +
> +		/* flush reported pages from the sg list */
> +		page_reporting_drain(prdev, zone);
> +
> +		/*
> +		 * Reset next to first entry, the old next isn't valid
> +		 * since we dropped the lock to report the pages
> +		 */
> +		next = list_first_entry(list, struct page, lru);
> +	}
> +
> +	spin_unlock_irq(&zone->lock);
> +
> +	return nents;
> +}
> +
> +static int
> +page_reporting_process_zone(struct page_reporting_dev_info *prdev,
> +			    struct zone *zone)
> +{
> +	unsigned int order, mt, nents = 0;
> +	unsigned long watermark;
> +	int refcnt = -1;
> +
> +	page_reporting_populate_metadata(zone);
> +
> +	/* Generate minimum watermark to be able to guarantee progress */
> +	watermark = low_wmark_pages(zone) +
> +		    (prdev->capacity << PAGE_REPORTING_MIN_ORDER);
> +
> +	/*
> +	 * Cancel request if insufficient free memory or if we failed
> +	 * to allocate page reporting statistics for the zone.
> +	 */
> +	if (!zone_watermark_ok(zone, 0, watermark, 0, ALLOC_CMA) ||
> +	    !zone->reported_pages) {
> +		spin_lock_irq(&zone->lock);
> +		goto zone_not_ready;
> +	}

The zone lock is acquired just for the zone bit and the refcnt? That
seems drastic overkill because the bit handling is already racy and the
refcnt is overkill.

> +
> +	sg_init_table(prdev->sg, prdev->capacity);
> +
> +	/* Process each free list starting from lowest order/mt */
> +	for_each_reporting_migratetype_order(order, mt)
> +		nents = page_reporting_cycle(prdev, zone, order, mt, nents);
> +


> +	/* mark end of sg list and report the remainder */
> +	if (nents) {
> +		sg_mark_end(&prdev->sg[nents - 1]);
> +		prdev->report(prdev, nents);
> +	}
> +
> +	spin_lock_irq(&zone->lock);
> +
> +	/* flush any remaining pages out from the last report */
> +	if (nents)
> +		page_reporting_drain(prdev, zone);
> +
> +	/* check to see if values are low enough for us to stop for now */
> +	for (order = PAGE_REPORTING_MIN_ORDER; order < MAX_ORDER; order++) {
> +		if (pages_unreported(zone, order) < PAGE_REPORTING_HWM)
> +			continue;
> +#ifdef CONFIG_MEMORY_ISOLATION
> +		/*
> +		 * Do not allow a free_area with isolated pages to request
> +		 * that we continue with page reporting. Keep the reporting
> +		 * light until the isolated pages have been cleared.
> +		 */
> +		if (!free_area_empty(&zone->free_area[order], MIGRATE_ISOLATE))
> +			continue;
> +#endif
> +		goto zone_not_complete;
> +	}
> +
> +zone_not_ready:
> +	/*
> +	 * If there are no longer enough free pages to fully populate
> +	 * the scatterlist, then we can just shut it down for this zone.
> +	 */
> +	__clear_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags);
> +	refcnt = atomic_dec_return(&prdev->refcnt);

If a request comes in while processing is already happening then the
subsequent request is lost.

> +zone_not_complete:
> +	spin_unlock_irq(&zone->lock);
> +
> +	return refcnt;
> +}
> +
> +static void page_reporting_process(struct work_struct *work)
> +{
> +	struct delayed_work *d_work = to_delayed_work(work);
> +	struct page_reporting_dev_info *prdev =
> +		container_of(d_work, struct page_reporting_dev_info, work);
> +	struct zone *zone = first_online_pgdat()->node_zones;
> +	int refcnt = -1;
> +
> +	do {
> +		if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
> +			refcnt = page_reporting_process_zone(prdev, zone);
> +
> +		/* Move to next zone, if at end of list start over */
> +		zone = next_zone(zone) ? : first_online_pgdat()->node_zones;
> +
> +		/*
> +		 * As long as refcnt has not reached zero there are still
> +		 * zones to be processed.
> +		 */
> +	} while (refcnt);
> +}
> +

This is being done from a workqueue context and potentially this runs
forever if the refcnt remains elevated. It's dangerous and should be in
it's own patch. In the basic implementation, just do a single pass of
all zones.

> +/* request page reporting on this zone */
> +void __page_reporting_request(struct zone *zone)
> +{
> +	struct page_reporting_dev_info *prdev;
> +
> +	rcu_read_lock();
> +
> +	/*
> +	 * We use RCU to protect the pr_dev_info pointer. In almost all
> +	 * cases this should be present, however in the unlikely case of
> +	 * a shutdown this will be NULL and we should exit.
> +	 */
> +	prdev = rcu_dereference(pr_dev_info);
> +	if (unlikely(!prdev))
> +		goto out;
> +
> +	/*
> +	 * We can use separate test and set operations here as there
> +	 * is nothing else that can set or clear this bit while we are
> +	 * holding the zone lock. The advantage to doing it this way is
> +	 * that we don't have to dirty the cacheline unless we are
> +	 * changing the value.
> +	 */
> +	__set_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags);
> +

The comment implies that the bit should have been tested first.

> +	/*
> +	 * Delay the start of work to allow a sizable queue to
> +	 * build. For now we are limiting this to running no more
> +	 * than 5 times per second.
> +	 */
> +	if (!atomic_fetch_inc(&prdev->refcnt))
> +		schedule_delayed_work(&prdev->work, HZ / 5);

The refcnt is incremented whether this zone has already been requested or
not but only decremented when the bit is cleared so the refcnt potentially
remains elevated forever.

> +out:
> +	rcu_read_unlock();
> +}
> +
> +static DEFINE_MUTEX(page_reporting_mutex);
> +DEFINE_STATIC_KEY_FALSE(page_reporting_enabled);
> +
> +void page_reporting_unregister(struct page_reporting_dev_info *prdev)
> +{
> +	mutex_lock(&page_reporting_mutex);
> +
> +	if (rcu_access_pointer(pr_dev_info) == prdev) {
> +		/* Disable page reporting notification */
> +		RCU_INIT_POINTER(pr_dev_info, NULL);
> +		synchronize_rcu();
> +
> +		/* Flush any existing work, and lock it out */
> +		cancel_delayed_work_sync(&prdev->work);
> +
> +		/* Free scatterlist */
> +		kfree(prdev->sg);
> +		prdev->sg = NULL;
> +	}
> +
> +	mutex_unlock(&page_reporting_mutex);
> +}
> +EXPORT_SYMBOL_GPL(page_reporting_unregister);
> +
> +int page_reporting_register(struct page_reporting_dev_info *prdev)
> +{
> +	struct zone *zone;
> +	int err = 0;
> +
> +	/* No point in enabling this if it cannot handle any pages */
> +	if (WARN_ON(!prdev->capacity || prdev->capacity > PAGE_REPORTING_HWM))
> +		return -EINVAL;
> +
> +	mutex_lock(&page_reporting_mutex);
> +
> +	/* nothing to do if already in use */
> +	if (rcu_access_pointer(pr_dev_info)) {
> +		err = -EBUSY;
> +		goto err_out;
> +	}
> +
> +	/* allocate scatterlist to store pages being reported on */
> +	prdev->sg = kcalloc(prdev->capacity, sizeof(*prdev->sg), GFP_KERNEL);
> +	if (!prdev->sg) {
> +		err = -ENOMEM;
> +		goto err_out;
> +	}
> +
> +
> +	/* initialize refcnt and work structures */
> +	atomic_set(&prdev->refcnt, 0);
> +	INIT_DELAYED_WORK(&prdev->work, &page_reporting_process);
> +
> +	/* assign device, and begin initial flush of populated zones */
> +	rcu_assign_pointer(pr_dev_info, prdev);
> +	for_each_populated_zone(zone) {
> +		spin_lock_irq(&zone->lock);
> +		__page_reporting_request(zone);
> +		spin_unlock_irq(&zone->lock);
> +	}
> +
> +	/* enable page reporting notification */
> +	if (!static_key_enabled(&page_reporting_enabled)) {
> +		static_branch_enable(&page_reporting_enabled);
> +		pr_info("Unused page reporting enabled\n");
> +	}
> +err_out:
> +	mutex_unlock(&page_reporting_mutex);
> +
> +	return err;
> +}
> +EXPORT_SYMBOL_GPL(page_reporting_register);
> diff --git a/mm/page_reporting.h b/mm/page_reporting.h
> new file mode 100644
> index 000000000000..2ad31bbb0036
> --- /dev/null
> +++ b/mm/page_reporting.h
> @@ -0,0 +1,125 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _MM_PAGE_REPORTING_H
> +#define _MM_PAGE_REPORTING_H
> +
> +#include <linux/mmzone.h>
> +#include <linux/pageblock-flags.h>
> +#include <linux/page-isolation.h>
> +#include <linux/jump_label.h>
> +#include <linux/slab.h>
> +#include <asm/pgtable.h>
> +
> +#define PAGE_REPORTING_MIN_ORDER	pageblock_order

This is potentially the same as MAX_ORDER-1. Not sure if it matters or
not.

> +#define PAGE_REPORTING_HWM		32
> +
> +#ifdef CONFIG_PAGE_REPORTING
> +/* Reported page accessors, defined in page_alloc.c */
> +void __free_isolated_page(struct page *page, unsigned int order);
> +
> +/* Free reported_pages and reset reported page tracking count to 0 */
> +static inline void page_reporting_reset_zone(struct zone *zone)
> +{
> +	kfree(zone->reported_pages);
> +	zone->reported_pages = NULL;
> +}
> +
> +DECLARE_STATIC_KEY_FALSE(page_reporting_enabled);
> +void __page_reporting_request(struct zone *zone);
> +
> +static inline bool page_reported(struct page *page)
> +{
> +	return static_branch_unlikely(&page_reporting_enabled) &&
> +	       PageReported(page);
> +}
> +
> +static inline unsigned long
> +pages_unreported(struct zone *zone, int order)
> +{
> +	unsigned long nr_free;
> +	int report_order;
> +
> +	/* Limit notifications only to higher order pages */
> +	report_order = order - PAGE_REPORTING_MIN_ORDER;
> +	if (report_order < 0)
> +		return 0;
> +
> +	nr_free = zone->free_area[order].nr_free;
> +
> +	/* Only subtract reported_pages count if it is present */
> +	if (!zone->reported_pages)
> +		return nr_free;
> +
> +	return nr_free - zone->reported_pages[report_order];
> +}

Initially, this should be a full list traversal because I'm not
convinced the reported_pages count is kept perfectly in sync. The
optimisation confuses the review of the basic feature itself.

> +
> +/**
> + * page_reporting_notify_free - Free page notification to start page processing
> + * @zone: Pointer to current zone of last page processed
> + * @order: Order of last page added to zone
> + *
> + * This function is meant to act as a screener for __page_reporting_request
> + * which will determine if a give zone has crossed over the high-water mark
> + * that will justify us beginning page treatment. If we have crossed that
> + * threshold then it will start the process of pulling some pages and
> + * placing them in the batch list for treatment.
> + */
> +static inline void page_reporting_notify_free(struct zone *zone, int order)
> +{
> +	/* Called from hot path in __free_one_page() */
> +	if (!static_branch_unlikely(&page_reporting_enabled))
> +		return;
> +
> +	/* Do not bother with tests if we have already requested reporting */
> +	if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
> +		return;
> +
> +	/* Determine if we have crossed reporting threshold */
> +	if (pages_unreported(zone, order) < PAGE_REPORTING_HWM)
> +		return;
> +
> +	/* This is slow, but should be called very rarely */
> +	__page_reporting_request(zone);
> +}
> +
> +/*
> + * Functions for marking/clearing reported pages from the freelist.
> + * All of them expect the zone lock to be held to maintain
> + * consistency of the reported_pages count versus nr_free.
> + */
> +static inline void
> +mark_page_reported(struct page *page, struct zone *zone, unsigned int order)
> +{
> +	/* flag page as reported */
> +	__SetPageReported(page);
> +
> +	/* update areated page accounting */
> +	zone->reported_pages[order - PAGE_REPORTING_MIN_ORDER]++;
> +}
> +
> +static inline void
> +clear_page_reported(struct page *page, struct zone *zone, unsigned int order)
> +{
> +	/* page_private will contain the page order, so just use it directly */
> +	zone->reported_pages[order - PAGE_REPORTING_MIN_ORDER]--;
> +
> +	/* clear the flag so we can report on it when it returns */
> +	__ClearPageReported(page);
> +}
> +
> +#else /* CONFIG_PAGE_REPORTING */
> +#define page_reported(_page)	false
> +
> +static inline void page_reporting_reset_zone(struct zone *zone)
> +{
> +}
> +
> +static inline void page_reporting_notify_free(struct zone *zone, int order)
> +{
> +}
> +
> +static inline void
> +clear_page_reported(struct page *page, struct zone *zone, unsigned int order)
> +{
> +}
> +#endif /* CONFIG_PAGE_REPORTING */
> +#endif /*_MM_PAGE_REPORTING_H */
> 

-- 
Mel Gorman
SUSE Labs

Re: [PATCH v14 3/6] mm: Introduce Reported pages

[Alexander Duyck]

On Wed, 2019-11-27 at 15:24 +0000, Mel Gorman wrote:
> On Tue, Nov 19, 2019 at 01:46:33PM -0800, Alexander Duyck wrote:
> > From: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> > 
> > In order to pave the way for free page reporting in virtualized
> > environments we will need a way to get pages out of the free lists and
> > identify those pages after they have been returned. To accomplish this,
> > this patch adds the concept of a Reported Buddy, which is essentially
> > meant to just be the Uptodate flag used in conjunction with the Buddy
> > page type.
> > 
> > To prevent the reported pages from leaking outside of the buddy lists I
> > have added a call to clear_reported_page to the del_page_from_free_list
> > function. As a result any reported page that is split, merged, or
> > allocated will have the flag cleared prior to the PageBuddy value being
> > cleared.
> > 
> > The process for reporting pages is fairly simple. Once the nr_free for
> > a given free area has exceeded the number of pages reported for that area
> > plus a certain high watermark value we will flag the zone as needing
> > reporting and schedule the worker thread to start reporting. That worker
> > thread will begin working from the lowest supported page reporting order
> > up to MAX_ORDER - 1 pulling unreported pages from the free list and
> > storing them in the scatterlist.
> > 
> > When processing each individual free list it is necessary for the worker
> > thread to release the zone lock when it needs to stop and report the full
> > scatterlist of pages. To reduce the work of the next iteration the worker
> > thread will rotate the free list so that the first unreported page in the
> > free list becomes the first entry in the list. Doing this we should only
> > have to walk an entire free list no more than twice assuming large
> > numbers of pages are not being added to the tail of the list.
> > 
> > It will then call a reporting function providing information on how many
> > entries are in the scatterlist. Once the function completes it will return
> > the pages to the tail of the free area from which they were allocated and
> > start over pulling more pages from the free areas until there are no
> > longer enough pages to report on to keep the worker busy.
> > 
> > The worker thread will work in a round-robin fashion making its way
> > though each zone requesting reporting, and through each reportable free
> > list within that zone. Once all free areas within the zone is below the
> > high watermark level for free pages to report the flag indicating that the
> > zone has requested reporting will be cleared, and if no zones are
> > requesting reporting the worker thread will exit.
> > 
> > Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> 
> Ok, I'm ok with how this hooks into the allocator as the overhead is
> minimal. However, the patch itself still includes a number of
> optimisations instead of being a bare-boned implementation of the
> feature with optimisations layered on top. I think some of the
> optimisations are also broken but it's harder to be sure because both
> the feature and the optimisations are lumped together.

Well I can work on splitting them out if need be. 

> > ---
> >  include/linux/mmzone.h         |   12 +
> >  include/linux/page-flags.h     |   11 +
> >  include/linux/page_reporting.h |   31 ++++
> >  mm/Kconfig                     |   11 +
> >  mm/Makefile                    |    1 
> >  mm/memory_hotplug.c            |    2 
> >  mm/page_alloc.c                |   53 ++++++
> >  mm/page_reporting.c            |  337 ++++++++++++++++++++++++++++++++++++++++
> >  mm/page_reporting.h            |  125 +++++++++++++++
> >  9 files changed, 579 insertions(+), 4 deletions(-)
> >  create mode 100644 include/linux/page_reporting.h
> >  create mode 100644 mm/page_reporting.c
> >  create mode 100644 mm/page_reporting.h
> > 
> > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> > index 8d93106490f3..9647499983b1 100644
> > --- a/include/linux/mmzone.h
> > +++ b/include/linux/mmzone.h
> > @@ -478,6 +478,14 @@ struct zone {
> >  	seqlock_t		span_seqlock;
> >  #endif
> >  
> > +#ifdef CONFIG_PAGE_REPORTING
> > +	/*
> > +	 * Pointer to reported page tracking statistics array. The size of
> > +	 * the array is MAX_ORDER - PAGE_REPORTING_MIN_ORDER. NULL when
> > +	 * unused page reporting is not present.
> > +	 */
> > +	unsigned long		*reported_pages;
> > +#endif
> >  	int initialized;
> >  
> >  	/* Write-intensive fields used from the page allocator */
> 
> Comment could be better. We know it's a pointer.

Okay, I can rework the comment. I'll call out specifically that I am
leaving it as a pointer because PAGE_REPORTING_MIN_ORDER might be variable
on some systems so I cannot allocate it statically.

> /*
>  * When a device is registered to track free pages, this array tracks
>  * the number of pages already reported at each order from
>  * PAGE_REPORTING_MIN_ORDER up to MAX_ORDER.
>  */
> 
> > @@ -550,6 +558,10 @@ enum zone_flags {
> >  	ZONE_BOOSTED_WATERMARK,		/* zone recently boosted watermarks.
> >  					 * Cleared when kswapd is woken.
> >  					 */
> > +	ZONE_PAGE_REPORTING_REQUESTED,	/* zone enabled page reporting and has
> > +					 * requested flushing the data out of
> > +					 * higher order pages.
> > +					 */
> >  };
> 
> Set if a *driver* enabled page reporting and has requested that unused
> pages are reported?
> 
> "flushing the data" would normally indicate that it's something to do
> with writeback.

Okay, I can work on the wording.

> >  
> >  static inline unsigned long zone_managed_pages(struct zone *zone)
> > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
> > index 1bf83c8fcaa7..a3a3b15b56a8 100644
> > --- a/include/linux/page-flags.h
> > +++ b/include/linux/page-flags.h
> > @@ -163,6 +163,9 @@ enum pageflags {
> >  
> >  	/* non-lru isolated movable page */
> >  	PG_isolated = PG_reclaim,
> > +
> > +	/* Buddy pages. Used to track which pages have been reported */
> > +	PG_reported = PG_uptodate,
> >  };
> >  
> >  #ifndef __GENERATING_BOUNDS_H
> 
> Only valid for buddy pages.....

Okay I can tweak the wording.

> > diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h
> > new file mode 100644
> > index 000000000000..925a16b1d14b
> > --- /dev/null
> > +++ b/include/linux/page_reporting.h
> > @@ -0,0 +1,31 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef _LINUX_PAGE_REPORTING_H
> > +#define _LINUX_PAGE_REPORTING_H
> > +
> > +#include <linux/mmzone.h>
> > +
> > +struct page_reporting_dev_info {
> > +	/* function that alters pages to make them "reported" */
> > +	void (*report)(struct page_reporting_dev_info *prdev,
> > +		       unsigned int nents);
> > +
> > +	/* pointer to scatterlist containing pages to be processed */
> > +	struct scatterlist *sg;
> > +
> 
> The choice of scatterlist is curious because it adds unnecessary complexity
> without needing the benefits scatterlist and sg_table provides. All you
> really need is an array of pages that is NULL terminated because you can
> get the order of the struct page when it's a buddy page.

The scatterlist is mostly about dealing with DMA mapping of the pages.
Most devices will need a way to push the pages out to the device doing the
reporting so I thought a scatterlist was the best way to gather the pages
together so that they could be mapped with one call.

> > +	/*
> > +	 * Upper limit on the number of pages that the report function
> > +	 * expects to be placed into the scatterlist to be processed.
> > +	 */
> > +	unsigned int capacity;
> > +
> 
> Instead of requiring the driver to specify capacity, why did you not
> simply make it PAGE_REPORTING_HWM and statically declare the array?

The problem is the device might not be able to support PAGE_REPORTING_HWM.
So I needed some way for it to communicate that. I could allocate the
array statically if that is what you prefer.

> > +	/* The number of zones requesting reporting */
> > +	atomic_t refcnt;
> > +
> 
> The refcnt is overkill. Simply loop over all zones until there is a
> clear pass. Despite being atomic, it actually requires the zone lock to
> avoid races in places like this
> 
> 		if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
> 			refcnt = page_reporting_process_zone(prdev, zone);
> 
> The optimisation is small and it's relatively subtle.

I disagree. Without that we can easily get into states where a zone
requests reporting and we never get around to servicing it. I am using the
refcount to track that so that we don't shutdown the worker thread when
there is at least one zone requesting service.

> >  static inline void __free_one_page(struct page *page,
> >  		unsigned long pfn,
> >  		struct zone *zone, unsigned int order,
> > -		int migratetype)
> > +		int migratetype, bool reported)
> >  {
> >  	struct capture_control *capc = task_capc(zone);
> >  	unsigned long uninitialized_var(buddy_pfn);
> > @@ -1048,7 +1053,9 @@ static inline void __free_one_page(struct page *page,
> >  done_merging:
> >  	set_page_order(page, order);
> >  
> > -	if (is_shuffle_order(order))
> > +	if (reported)
> > +		to_tail = true;
> > +	else if (is_shuffle_order(order))
> >  		to_tail = shuffle_pick_tail();
> >  	else
> >  		to_tail = buddy_merge_likely(pfn, buddy_pfn, page, order);
> > @@ -1057,6 +1064,14 @@ static inline void __free_one_page(struct page *page,
> >  		add_to_free_list_tail(page, zone, order, migratetype);
> >  	else
> >  		add_to_free_list(page, zone, order, migratetype);
> > +
> > +	/*
> > +	 * No need to notify on a reported page as the total count of
> > +	 * unreported pages will not have increased since we have essentially
> > +	 * merged the reported page with one or more unreported pages.
> > +	 */
> > +	if (!reported)
> > +		page_reporting_notify_free(zone, order);
> >  }
> >  
> >  /*
> 
> If a reported page is merged with a larger buddy then the counts for the
> lower order needs to be updated or the counter gets out of sync. Then the
> reported status of the page needs to be cleared as the larger block may
> only be partially reported and the count is wrong. I know why you want
> the reported pages but it really should be in its own patch. This patch
> really should be just the bare essentials to support reporting of free
> pages even if it's sub-optimal. I know you also use reported_pages as
> part of a stop condition but a basic stop condition is simply to do a
> single pass.

Okay so there are actually a few things here to unpack.

First the comment is a bit out of date. The reason I don't need to bother
with reporting is because I process the lower orders before the higher
orders. As such we are still processing this zone if we are returning a
reported page and it is merged. We will reprocess the larger page in the
next pass.

Second, at the point where we call this function with the reported flag
set we have not yet set the page as reported. There is logic that will
check for that later and set the bits and increment the count if the page
becomes a buddy page and the order is still the same as the order we
originally freed it under.

Lastly, if you are concerned about us merging a reported page already on
the list the count is updated and the flag is cleared from the page when
it is deleted from the free list to be merged with the new page.

> > @@ -3228,6 +3243,36 @@ int __isolate_free_page(struct page *page, unsigned int order)
> >  	return 1UL << order;
> >  }
> >  
> > +#ifdef CONFIG_PAGE_REPORTING
> > +/**
> > + * __free_isolated_page - Return a now-isolated page back where we got it
> > + * @page: Page that was isolated
> > + * @order: Order of the isolated page
> > + *
> > + * This function is meant to return a page pulled from the free lists via
> > + * __isolate_free_page back to the free lists they were pulled from.
> > + */
> 
> Isolated within mm has special meaning already. __putback_reported_page?

I wanted to get away from calling it a reported page because it isn't at
the point where we are calling this function. We don't set the reported
bit on the page until we have placed it back into the free list and
verified it wasn't merged.

I'm open to suggestions on how to name a function that is meant to be the
inverse of __isolate_free_page. All I did is swap the isolate and free
verb/adjective.

> > +void __free_isolated_page(struct page *page, unsigned int order)
> > +{
> > +	struct zone *zone = page_zone(page);
> > +	unsigned long pfn;
> > +	unsigned int mt;
> > +
> > +	/* zone lock should be held when this function is called */
> > +	lockdep_assert_held(&zone->lock);
> > +
> > +	pfn = page_to_pfn(page);
> > +	mt = get_pfnblock_migratetype(page, pfn);
> > +
> > +	/*
> > +	 * Return isolated page to tail of freelist and don't bother with
> > +	 * triggering the page reporting notifiers since this page was
> > +	 * previously on the freelist and has likely already been reported.
> > +	 */
> > +	__free_one_page(page, pfn, zone, order, mt, true);
> > +}
> > +#endif /* CONFIG_PAGE_REPORTING */
> > +
> >  /*
> >   * Update NUMA hit/miss statistics
> >   *
> > diff --git a/mm/page_reporting.c b/mm/page_reporting.c
> > new file mode 100644
> > index 000000000000..4844f0aa2904
> > --- /dev/null
> > +++ b/mm/page_reporting.c
> > @@ -0,0 +1,337 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +#include <linux/mm.h>
> > +#include <linux/mmzone.h>
> > +#include <linux/page_reporting.h>
> > +#include <linux/gfp.h>
> > +#include <linux/export.h>
> > +#include <linux/delay.h>
> > +#include <linux/scatterlist.h>
> > +
> > +#include "page_reporting.h"
> > +#include "internal.h"
> > +
> > +static struct page_reporting_dev_info __rcu *pr_dev_info __read_mostly;
> > +
> > +#define for_each_reporting_migratetype_order(_order, _type) \
> > +	for (_order = PAGE_REPORTING_MIN_ORDER; _order < MAX_ORDER; _order++) \
> > +		for (_type = 0; _type < MIGRATE_TYPES; _type++) \
> > +			if (!is_migrate_isolate(_type))
> > +
> 
> This only has one user so simply put it in place. As this stands, _order
> and _type can be evaluated multiple times which potentially leads to
> chaos

Good point. Back when I started I had several spots using it. I guess I
have consolidated all of them now.

> > +static void page_reporting_populate_metadata(struct zone *zone)
> > +{
> > +	size_t size;
> > +	int node;
> > +
> > +	/*
> > +	 * We need to make sure we have somewhere to store the tracking
> > +	 * data for how many reported pages are in the zone. To do that
> > +	 * we need to make certain zone->reported_pages is populated.
> > +	 */
> > +	if (zone->reported_pages)
> > +		return;
> > +
> > +	node = zone_to_nid(zone);
> > +	size = (MAX_ORDER - PAGE_REPORTING_MIN_ORDER) * sizeof(unsigned long);
> > +	zone->reported_pages = kzalloc_node(size, GFP_KERNEL, node);
> > +}
> > +
> 
> reported_pages is not necessarily updated by a CPU local to the zone.
> There is not much point worrying about its locality.

Okay, I can drop that.

> > +static void
> > +page_reporting_drain(struct page_reporting_dev_info *prdev, struct zone *zone)
> > +{
> > +	struct scatterlist *sg = prdev->sg;
> > +
> > +	/*
> > +	 * Drain the now reported pages back into their respective
> > +	 * free lists/areas. We assume at least one page is populated.
> > +	 */
> > +	do {
> > +		unsigned int order = get_order(sg->length);
> > +		struct page *page = sg_page(sg);
> > +
> > +		__free_isolated_page(page, order);
> > +
> > +		/*
> > +		 * If page was not comingled with another page we can
> > +		 * consider the result to be "reported" since the page
> > +		 * hasn't been modified, otherwise we will need to
> > +		 * report on the new larger page when we make our way
> > +		 * up to that higher order.
> > +		 */
> > +		if (PageBuddy(page) && page_order(page) == order)
> > +			mark_page_reported(page, zone, order);
> > +	} while (!sg_is_last(sg++));
> > +}
> > +
> > +/*
> > + * The page reporting cycle consists of 4 stages, fill, report, drain, and
> > + * idle. We will cycle through the first 3 stages until we cannot obtain a
> > + * full scatterlist of pages, in that case we will switch to idle.
> > + */
> 
> Document the return value because it's unclear why the number of populated
> elements in sg is not tracked in page_reporting_dev_info. It seems
> unnecessarily fragile.

Will do in terms of documenting the return value.

As far as tracking the number of elements it mostly just has to do with
how this code has evolved. Originally this was just going straight to the
report function. However that leads to partial requests and I wanted to
avoid those since the cost for each call into the hypervisor can be high.

> > +static unsigned int
> > +page_reporting_cycle(struct page_reporting_dev_info *prdev, struct zone *zone,
> > +		     unsigned int order, unsigned int mt, unsigned int nents)
> > +{
> > +	struct list_head *list = &zone->free_area[order].free_list[mt];
> > +	unsigned int page_len = PAGE_SIZE << order;
> > +	struct scatterlist *sg = prdev->sg;
> > +	struct page *page, *next;
> > +
> > +	/*
> > +	 * Perform early check, if free area is empty there is
> > +	 * nothing to process so we can skip this free_list.
> > +	 */
> > +	if (list_empty(list))
> > +		return nents;
> > +
> > +	spin_lock_irq(&zone->lock);
> > +
> > +	/* loop through free list adding unreported pages to sg list */
> > +	list_for_each_entry_safe(page, next, list, lru) {
> > +		/* We are going to skip over the reported pages. */
> > +		if (PageReported(page))
> > +			continue;
> > +
> > +		/* Attempt to add page to sg list */
> > +		if (nents < prdev->capacity) {
> > +			if (!__isolate_free_page(page, order))
> > +				break;
> > +
> > +			sg_set_page(&sg[nents++], page, page_len, 0);
> > +			continue;
> > +		}
> > +
> > +		/*
> > +		 * Make the first non-reported entry in the free list
> > +		 * the new head of the free list before we exit.
> > +		 */
> > +		if (!list_is_first(&page->lru, list))
> > +			list_rotate_to_front(&page->lru, list);
> > +
> > +		/* release lock before waiting on report processing*/
> > +		spin_unlock_irq(&zone->lock);
> > +
> > +		/* begin processing pages in local list */
> > +		prdev->report(prdev, nents);
> > +
> > +		/* reset number of entries */
> > +		nents = 0;
> > +
> > +		/* reacquire zone lock and resume processing free lists */
> > +		spin_lock_irq(&zone->lock);
> > +
> > +		/* flush reported pages from the sg list */
> > +		page_reporting_drain(prdev, zone);
> > +
> > +		/*
> > +		 * Reset next to first entry, the old next isn't valid
> > +		 * since we dropped the lock to report the pages
> > +		 */
> > +		next = list_first_entry(list, struct page, lru);
> > +	}
> > +
> > +	spin_unlock_irq(&zone->lock);
> > +
> > +	return nents;
> > +}
> > +
> > +static int
> > +page_reporting_process_zone(struct page_reporting_dev_info *prdev,
> > +			    struct zone *zone)
> > +{
> > +	unsigned int order, mt, nents = 0;
> > +	unsigned long watermark;
> > +	int refcnt = -1;
> > +
> > +	page_reporting_populate_metadata(zone);
> > +
> > +	/* Generate minimum watermark to be able to guarantee progress */
> > +	watermark = low_wmark_pages(zone) +
> > +		    (prdev->capacity << PAGE_REPORTING_MIN_ORDER);
> > +
> > +	/*
> > +	 * Cancel request if insufficient free memory or if we failed
> > +	 * to allocate page reporting statistics for the zone.
> > +	 */
> > +	if (!zone_watermark_ok(zone, 0, watermark, 0, ALLOC_CMA) ||
> > +	    !zone->reported_pages) {
> > +		spin_lock_irq(&zone->lock);
> > +		goto zone_not_ready;
> > +	}
> 
> The zone lock is acquired just for the zone bit and the refcnt? That
> seems drastic overkill because the bit handling is already racy and the
> refcnt is overkill.

I'm not sure I follow.

> > +
> > +	sg_init_table(prdev->sg, prdev->capacity);
> > +
> > +	/* Process each free list starting from lowest order/mt */
> > +	for_each_reporting_migratetype_order(order, mt)
> > +		nents = page_reporting_cycle(prdev, zone, order, mt, nents);
> > +
> > +	/* mark end of sg list and report the remainder */
> > +	if (nents) {
> > +		sg_mark_end(&prdev->sg[nents - 1]);
> > +		prdev->report(prdev, nents);
> > +	}
> > +
> > +	spin_lock_irq(&zone->lock);
> > +
> > +	/* flush any remaining pages out from the last report */
> > +	if (nents)
> > +		page_reporting_drain(prdev, zone);
> > +
> > +	/* check to see if values are low enough for us to stop for now */
> > +	for (order = PAGE_REPORTING_MIN_ORDER; order < MAX_ORDER; order++) {
> > +		if (pages_unreported(zone, order) < PAGE_REPORTING_HWM)
> > +			continue;
> > +#ifdef CONFIG_MEMORY_ISOLATION
> > +		/*
> > +		 * Do not allow a free_area with isolated pages to request
> > +		 * that we continue with page reporting. Keep the reporting
> > +		 * light until the isolated pages have been cleared.
> > +		 */
> > +		if (!free_area_empty(&zone->free_area[order], MIGRATE_ISOLATE))
> > +			continue;
> > +#endif
> > +		goto zone_not_complete;
> > +	}
> > +
> > +zone_not_ready:
> > +	/*
> > +	 * If there are no longer enough free pages to fully populate
> > +	 * the scatterlist, then we can just shut it down for this zone.
> > +	 */
> > +	__clear_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags);
> > +	refcnt = atomic_dec_return(&prdev->refcnt);
> 
> If a request comes in while processing is already happening then the
> subsequent request is lost.

How can a request come in when we are holding the zone lock? Basically the
notify call can only occur while holding the zone lock. That is one of the
reasons for holding the lock is to protect the flag and keep it consistent
with the page counts as we clear it.

The only case where we are not checking the counts is for the case above
where we clear it due to us being too close to the watermark. In that case
we want to just ignore the request anyway.

> > +zone_not_complete:
> > +	spin_unlock_irq(&zone->lock);
> > +
> > +	return refcnt;
> > +}
> > +
> > +static void page_reporting_process(struct work_struct *work)
> > +{
> > +	struct delayed_work *d_work = to_delayed_work(work);
> > +	struct page_reporting_dev_info *prdev =
> > +		container_of(d_work, struct page_reporting_dev_info, work);
> > +	struct zone *zone = first_online_pgdat()->node_zones;
> > +	int refcnt = -1;
> > +
> > +	do {
> > +		if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
> > +			refcnt = page_reporting_process_zone(prdev, zone);
> > +
> > +		/* Move to next zone, if at end of list start over */
> > +		zone = next_zone(zone) ? : first_online_pgdat()->node_zones;
> > +
> > +		/*
> > +		 * As long as refcnt has not reached zero there are still
> > +		 * zones to be processed.
> > +		 */
> > +	} while (refcnt);
> > +}
> > +
> 
> This is being done from a workqueue context and potentially this runs
> forever if the refcnt remains elevated. It's dangerous and should be in
> it's own patch. In the basic implementation, just do a single pass of
> all zones.

So I already have a modification that I have for the next version of the
patch set where instead of doing a do/while I simply reschedule the
delayed work to run again in 200ms if we still have work outstanding.

What I can do is split that out so that it is in the second patch with
optimizations.

> > +/* request page reporting on this zone */
> > +void __page_reporting_request(struct zone *zone)
> > +{
> > +	struct page_reporting_dev_info *prdev;
> > +
> > +	rcu_read_lock();
> > +
> > +	/*
> > +	 * We use RCU to protect the pr_dev_info pointer. In almost all
> > +	 * cases this should be present, however in the unlikely case of
> > +	 * a shutdown this will be NULL and we should exit.
> > +	 */
> > +	prdev = rcu_dereference(pr_dev_info);
> > +	if (unlikely(!prdev))
> > +		goto out;
> > +
> > +	/*
> > +	 * We can use separate test and set operations here as there
> > +	 * is nothing else that can set or clear this bit while we are
> > +	 * holding the zone lock. The advantage to doing it this way is
> > +	 * that we don't have to dirty the cacheline unless we are
> > +	 * changing the value.
> > +	 */
> > +	__set_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags);
> > +
> 
> The comment implies that the bit should have been tested first.

It was in the page_reporting_notify_free() call. I can update the comment.

> > +	/*
> > +	 * Delay the start of work to allow a sizable queue to
> > +	 * build. For now we are limiting this to running no more
> > +	 * than 5 times per second.
> > +	 */
> > +	if (!atomic_fetch_inc(&prdev->refcnt))
> > +		schedule_delayed_work(&prdev->work, HZ / 5);
> 
> The refcnt is incremented whether this zone has already been requested or
> not but only decremented when the bit is cleared so the refcnt potentially
> remains elevated forever.

So this is one of the reasons why all accesses to the flag are protected
by the zone lock. Basically page_reporting_notify_free() and this function
are both called with the zone lock held.

The only other caller of this function is page_reporting_register which
calls it for each populated zone. I suppose I could make it for every zone
since it is possible that someone unloaded the driver, unpopulated a zone,
and then reloaded the driver, but anyway the idea is that the zone lock is
keeping the flag and refcount consistent.

> > +out:
> > +	rcu_read_unlock();
> > +}
> > +
> > +static DEFINE_MUTEX(page_reporting_mutex);
> > +DEFINE_STATIC_KEY_FALSE(page_reporting_enabled);
> > +
> > +void page_reporting_unregister(struct page_reporting_dev_info *prdev)
> > +{
> > +	mutex_lock(&page_reporting_mutex);
> > +
> > +	if (rcu_access_pointer(pr_dev_info) == prdev) {
> > +		/* Disable page reporting notification */
> > +		RCU_INIT_POINTER(pr_dev_info, NULL);
> > +		synchronize_rcu();
> > +
> > +		/* Flush any existing work, and lock it out */
> > +		cancel_delayed_work_sync(&prdev->work);
> > +
> > +		/* Free scatterlist */
> > +		kfree(prdev->sg);
> > +		prdev->sg = NULL;
> > +	}
> > +
> > +	mutex_unlock(&page_reporting_mutex);
> > +}
> > +EXPORT_SYMBOL_GPL(page_reporting_unregister);
> > +
> > +int page_reporting_register(struct page_reporting_dev_info *prdev)
> > +{
> > +	struct zone *zone;
> > +	int err = 0;
> > +
> > +	/* No point in enabling this if it cannot handle any pages */
> > +	if (WARN_ON(!prdev->capacity || prdev->capacity > PAGE_REPORTING_HWM))
> > +		return -EINVAL;
> > +
> > +	mutex_lock(&page_reporting_mutex);
> > +
> > +	/* nothing to do if already in use */
> > +	if (rcu_access_pointer(pr_dev_info)) {
> > +		err = -EBUSY;
> > +		goto err_out;
> > +	}
> > +
> > +	/* allocate scatterlist to store pages being reported on */
> > +	prdev->sg = kcalloc(prdev->capacity, sizeof(*prdev->sg), GFP_KERNEL);
> > +	if (!prdev->sg) {
> > +		err = -ENOMEM;
> > +		goto err_out;
> > +	}
> > +
> > +
> > +	/* initialize refcnt and work structures */
> > +	atomic_set(&prdev->refcnt, 0);
> > +	INIT_DELAYED_WORK(&prdev->work, &page_reporting_process);
> > +
> > +	/* assign device, and begin initial flush of populated zones */
> > +	rcu_assign_pointer(pr_dev_info, prdev);
> > +	for_each_populated_zone(zone) {
> > +		spin_lock_irq(&zone->lock);
> > +		__page_reporting_request(zone);
> > +		spin_unlock_irq(&zone->lock);
> > +	}
> > +
> > +	/* enable page reporting notification */
> > +	if (!static_key_enabled(&page_reporting_enabled)) {
> > +		static_branch_enable(&page_reporting_enabled);
> > +		pr_info("Unused page reporting enabled\n");
> > +	}
> > +err_out:
> > +	mutex_unlock(&page_reporting_mutex);
> > +
> > +	return err;
> > +}
> > +EXPORT_SYMBOL_GPL(page_reporting_register);
> > diff --git a/mm/page_reporting.h b/mm/page_reporting.h
> > new file mode 100644
> > index 000000000000..2ad31bbb0036
> > --- /dev/null
> > +++ b/mm/page_reporting.h
> > @@ -0,0 +1,125 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef _MM_PAGE_REPORTING_H
> > +#define _MM_PAGE_REPORTING_H
> > +
> > +#include <linux/mmzone.h>
> > +#include <linux/pageblock-flags.h>
> > +#include <linux/page-isolation.h>
> > +#include <linux/jump_label.h>
> > +#include <linux/slab.h>
> > +#include <asm/pgtable.h>
> > +
> > +#define PAGE_REPORTING_MIN_ORDER	pageblock_order
> 
> This is potentially the same as MAX_ORDER-1. Not sure if it matters or
> not.

It doesn't. Basically what I am going for is the largest order that it is
safe to report on that will not break THP. If THP is not enabled then this
value should be MAX_ORDER - 1 if I am not mistaken.

> > +#define PAGE_REPORTING_HWM		32
> > +
> > +#ifdef CONFIG_PAGE_REPORTING
> > +/* Reported page accessors, defined in page_alloc.c */
> > +void __free_isolated_page(struct page *page, unsigned int order);
> > +
> > +/* Free reported_pages and reset reported page tracking count to 0 */
> > +static inline void page_reporting_reset_zone(struct zone *zone)
> > +{
> > +	kfree(zone->reported_pages);
> > +	zone->reported_pages = NULL;
> > +}
> > +
> > +DECLARE_STATIC_KEY_FALSE(page_reporting_enabled);
> > +void __page_reporting_request(struct zone *zone);
> > +
> > +static inline bool page_reported(struct page *page)
> > +{
> > +	return static_branch_unlikely(&page_reporting_enabled) &&
> > +	       PageReported(page);
> > +}
> > +
> > +static inline unsigned long
> > +pages_unreported(struct zone *zone, int order)
> > +{
> > +	unsigned long nr_free;
> > +	int report_order;
> > +
> > +	/* Limit notifications only to higher order pages */
> > +	report_order = order - PAGE_REPORTING_MIN_ORDER;
> > +	if (report_order < 0)
> > +		return 0;
> > +
> > +	nr_free = zone->free_area[order].nr_free;
> > +
> > +	/* Only subtract reported_pages count if it is present */
> > +	if (!zone->reported_pages)
> > +		return nr_free;
> > +
> > +	return nr_free - zone->reported_pages[report_order];
> > +}
> 
> Initially, this should be a full list traversal because I'm not
> convinced the reported_pages count is kept perfectly in sync. The
> optimisation confuses the review of the basic feature itself.

This is called per freed page in page_reporting_notify_free below. Are you
saying I have to traverse the entire free list per freed page?

If that is what you are wanting I would still probably want to limit it
such that we only walk the first PAGE_REPORTING_HWM worth of pages and if
those are not reported then just report that many.

> > +
> > +/**
> > + * page_reporting_notify_free - Free page notification to start page processing
> > + * @zone: Pointer to current zone of last page processed
> > + * @order: Order of last page added to zone
> > + *
> > + * This function is meant to act as a screener for __page_reporting_request
> > + * which will determine if a give zone has crossed over the high-water mark
> > + * that will justify us beginning page treatment. If we have crossed that
> > + * threshold then it will start the process of pulling some pages and
> > + * placing them in the batch list for treatment.
> > + */
> > +static inline void page_reporting_notify_free(struct zone *zone, int order)
> > +{
> > +	/* Called from hot path in __free_one_page() */
> > +	if (!static_branch_unlikely(&page_reporting_enabled))
> > +		return;
> > +
> > +	/* Do not bother with tests if we have already requested reporting */
> > +	if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
> > +		return;
> > +
> > +	/* Determine if we have crossed reporting threshold */
> > +	if (pages_unreported(zone, order) < PAGE_REPORTING_HWM)
> > +		return;
> > +
> > +	/* This is slow, but should be called very rarely */
> > +	__page_reporting_request(zone);
> > +}
> > +
> > +/*
> > + * Functions for marking/clearing reported pages from the freelist.
> > + * All of them expect the zone lock to be held to maintain
> > + * consistency of the reported_pages count versus nr_free.
> > + */
> > +static inline void
> > +mark_page_reported(struct page *page, struct zone *zone, unsigned int order)
> > +{
> > +	/* flag page as reported */
> > +	__SetPageReported(page);
> > +
> > +	/* update areated page accounting */
> > +	zone->reported_pages[order - PAGE_REPORTING_MIN_ORDER]++;
> > +}
> > +
> > +static inline void
> > +clear_page_reported(struct page *page, struct zone *zone, unsigned int order)
> > +{
> > +	/* page_private will contain the page order, so just use it directly */
> > +	zone->reported_pages[order - PAGE_REPORTING_MIN_ORDER]--;
> > +
> > +	/* clear the flag so we can report on it when it returns */
> > +	__ClearPageReported(page);
> > +}
> > +
> > +#else /* CONFIG_PAGE_REPORTING */
> > +#define page_reported(_page)	false
> > +
> > +static inline void page_reporting_reset_zone(struct zone *zone)
> > +{
> > +}
> > +
> > +static inline void page_reporting_notify_free(struct zone *zone, int order)
> > +{
> > +}
> > +
> > +static inline void
> > +clear_page_reported(struct page *page, struct zone *zone, unsigned int order)
> > +{
> > +}
> > +#endif /* CONFIG_PAGE_REPORTING */
> > +#endif /*_MM_PAGE_REPORTING_H */
> > 

Re: [PATCH v14 0/6] mm / virtio: Provide support for unused page reporting

[Alexander Duyck]

On Wed, 2019-11-27 at 11:01 +0100, David Hildenbrand wrote:
> On 26.11.19 17:45, Alexander Duyck wrote:
> > On Tue, 2019-11-26 at 13:20 +0100, David Hildenbrand wrote:
> > > On 19.11.19 22:46, Alexander Duyck wrote:

<snip>

> > > > Below are the results from various benchmarks. I primarily focused on two
> > > > tests. The first is the will-it-scale/page_fault2 test, and the other is
> > > > a modified version of will-it-scale/page_fault1 that was enabled to use
> > > > THP. I did this as it allows for better visibility into different parts
> > > > of the memory subsystem. The guest is running with 32G for RAM on one
> > > > node of a E5-2630 v3. The host has had some power saving features disabled
> > > > by setting the /dev/cpu_dma_latency value to 10ms.
> > > > 
> > > > Test                page_fault1 (THP)     page_fault2
> > > > Name         tasks  Process Iter  STDEV  Process Iter  STDEV
> > > > Baseline         1    1203934.75  0.04%     379940.75  0.11%
> > > >                 16    8828217.00  0.85%    3178653.00  1.28%
> > > > 
> > > > Patches applied  1    1207961.25  0.10%     380852.25  0.25%
> > > >                 16    8862373.00  0.98%    3246397.25  0.68%
> > > > 
> > > > Patches enabled  1    1207758.75  0.17%     373079.25  0.60%
> > > >  MADV disabled  16    8870373.75  0.29%    3204989.75  1.08%
> > > > 
> > > > Patches enabled  1    1261183.75  0.39%     373201.50  0.50%
> > > >                 16    8371359.75  0.65%    3233665.50  0.84%
> > > > 
> > > > Patches enabled  1    1090201.50  0.25%     376967.25  0.29%
> > > >  page shuffle   16    8108719.75  0.58%    3218450.25  1.07%
> > > > 
> > > > The results above are for a baseline with a linux-next-20191115 kernel,
> > > > that kernel with this patch set applied but page reporting disabled in
> > > > virtio-balloon, patches applied but the madvise disabled by direct
> > > > assigning a device, the patches applied and page reporting fully
> > > > enabled, and the patches enabled with page shuffling enabled.  These
> > > > results include the deviation seen between the average value reported here
> > > > versus the high and/or low value. I observed that during the test memory
> > > > usage for the first three tests never dropped whereas with the patches
> > > > fully enabled the VM would drop to using only a few GB of the host's
> > > > memory when switching from memhog to page fault tests.
> > > > 
> > > > Most of the overhead seen with this patch set enabled seems due to page
> > > > faults caused by accessing the reported pages and the host zeroing the page
> > > > before giving it back to the guest. This overhead is much more visible when
> > > > using THP than with standard 4K pages. In addition page shuffling seemed to
> > > > increase the amount of faults generated due to an increase in memory churn.
> > > 
> > > MADV_FREE would be interesting.
> > 
> > I can probably code something up. However that is going to push a bunch of
> > complexity into the QEMU code and doesn't really mean much to the kernel
> > code. I can probably add it as another QEMU patch to the set since it is
> > just a matter of having a function similar to ram_block_discard_range that
> > uses MADV_FREE instead of MADV_DONTNEED.
> 
> Yes, addon patch makes perfect sense. The nice thing about MADV_FREE is
> that you only take back pages from a process when really under memory
> pressure (before going to SWAP). You will still get a pagefault on the
> next access (to identify that the page is still in use after all), but
> don't have to fault in a fresh page.

So I got things running with a proof of concept using MADV_FREE.
Apparently another roadblock I hadn't realized is that you have to have
the right version of glibc for MADV_FREE to be present.

Anyway with MADV_FREE the numbers actually look pretty close to the
numbers with the madvise disabled. Apparently the page fault overhead
isn't all that significant. When I push the next patch set I will include
the actual numbers, but even with shuffling enabled the results were in
the 8.7 to 8.8 million iteration range.

Re: [PATCH v14 0/6] mm / virtio: Provide support for unused page reporting

[David Hildenbrand]

On 27.11.19 18:36, Alexander Duyck wrote:
> On Wed, 2019-11-27 at 11:01 +0100, David Hildenbrand wrote:
>> On 26.11.19 17:45, Alexander Duyck wrote:
>>> On Tue, 2019-11-26 at 13:20 +0100, David Hildenbrand wrote:
>>>> On 19.11.19 22:46, Alexander Duyck wrote:
> 
> <snip>
> 
>>>>> Below are the results from various benchmarks. I primarily focused on two
>>>>> tests. The first is the will-it-scale/page_fault2 test, and the other is
>>>>> a modified version of will-it-scale/page_fault1 that was enabled to use
>>>>> THP. I did this as it allows for better visibility into different parts
>>>>> of the memory subsystem. The guest is running with 32G for RAM on one
>>>>> node of a E5-2630 v3. The host has had some power saving features disabled
>>>>> by setting the /dev/cpu_dma_latency value to 10ms.
>>>>>
>>>>> Test                page_fault1 (THP)     page_fault2
>>>>> Name         tasks  Process Iter  STDEV  Process Iter  STDEV
>>>>> Baseline         1    1203934.75  0.04%     379940.75  0.11%
>>>>>                 16    8828217.00  0.85%    3178653.00  1.28%
>>>>>
>>>>> Patches applied  1    1207961.25  0.10%     380852.25  0.25%
>>>>>                 16    8862373.00  0.98%    3246397.25  0.68%
>>>>>
>>>>> Patches enabled  1    1207758.75  0.17%     373079.25  0.60%
>>>>>  MADV disabled  16    8870373.75  0.29%    3204989.75  1.08%
>>>>>
>>>>> Patches enabled  1    1261183.75  0.39%     373201.50  0.50%
>>>>>                 16    8371359.75  0.65%    3233665.50  0.84%
>>>>>
>>>>> Patches enabled  1    1090201.50  0.25%     376967.25  0.29%
>>>>>  page shuffle   16    8108719.75  0.58%    3218450.25  1.07%
>>>>>
>>>>> The results above are for a baseline with a linux-next-20191115 kernel,
>>>>> that kernel with this patch set applied but page reporting disabled in
>>>>> virtio-balloon, patches applied but the madvise disabled by direct
>>>>> assigning a device, the patches applied and page reporting fully
>>>>> enabled, and the patches enabled with page shuffling enabled.  These
>>>>> results include the deviation seen between the average value reported here
>>>>> versus the high and/or low value. I observed that during the test memory
>>>>> usage for the first three tests never dropped whereas with the patches
>>>>> fully enabled the VM would drop to using only a few GB of the host's
>>>>> memory when switching from memhog to page fault tests.
>>>>>
>>>>> Most of the overhead seen with this patch set enabled seems due to page
>>>>> faults caused by accessing the reported pages and the host zeroing the page
>>>>> before giving it back to the guest. This overhead is much more visible when
>>>>> using THP than with standard 4K pages. In addition page shuffling seemed to
>>>>> increase the amount of faults generated due to an increase in memory churn.
>>>>
>>>> MADV_FREE would be interesting.
>>>
>>> I can probably code something up. However that is going to push a bunch of
>>> complexity into the QEMU code and doesn't really mean much to the kernel
>>> code. I can probably add it as another QEMU patch to the set since it is
>>> just a matter of having a function similar to ram_block_discard_range that
>>> uses MADV_FREE instead of MADV_DONTNEED.
>>
>> Yes, addon patch makes perfect sense. The nice thing about MADV_FREE is
>> that you only take back pages from a process when really under memory
>> pressure (before going to SWAP). You will still get a pagefault on the
>> next access (to identify that the page is still in use after all), but
>> don't have to fault in a fresh page.
> 
> So I got things running with a proof of concept using MADV_FREE.
> Apparently another roadblock I hadn't realized is that you have to have
> the right version of glibc for MADV_FREE to be present.
> 
> Anyway with MADV_FREE the numbers actually look pretty close to the
> numbers with the madvise disabled. Apparently the page fault overhead
> isn't all that significant. When I push the next patch set I will include
> the actual numbers, but even with shuffling enabled the results were in
> the 8.7 to 8.8 million iteration range.
> 

Cool, thanks for evaluating!

-- 
Thanks,

David / dhildenb

Re: [PATCH v14 3/6] mm: Introduce Reported pages

[Mel Gorman]

On Wed, Nov 27, 2019 at 09:22:09AM -0800, Alexander Duyck wrote:
> > Ok, I'm ok with how this hooks into the allocator as the overhead is
> > minimal. However, the patch itself still includes a number of
> > optimisations instead of being a bare-boned implementation of the
> > feature with optimisations layered on top. I think some of the
> > optimisations are also broken but it's harder to be sure because both
> > the feature and the optimisations are lumped together.
> 
> Well I can work on splitting them out if need be. 
> 

It would be preferred because some of the optimisations are fairly
subtle and it distracts from having a basic implementation of the
feature as a foundation to build upon.

> > > diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h
> > > new file mode 100644
> > > index 000000000000..925a16b1d14b
> > > --- /dev/null
> > > +++ b/include/linux/page_reporting.h
> > > @@ -0,0 +1,31 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +#ifndef _LINUX_PAGE_REPORTING_H
> > > +#define _LINUX_PAGE_REPORTING_H
> > > +
> > > +#include <linux/mmzone.h>
> > > +
> > > +struct page_reporting_dev_info {
> > > +	/* function that alters pages to make them "reported" */
> > > +	void (*report)(struct page_reporting_dev_info *prdev,
> > > +		       unsigned int nents);
> > > +
> > > +	/* pointer to scatterlist containing pages to be processed */
> > > +	struct scatterlist *sg;
> > > +
> > 
> > The choice of scatterlist is curious because it adds unnecessary complexity
> > without needing the benefits scatterlist and sg_table provides. All you
> > really need is an array of pages that is NULL terminated because you can
> > get the order of the struct page when it's a buddy page.
> 
> The scatterlist is mostly about dealing with DMA mapping of the pages.
> Most devices will need a way to push the pages out to the device doing the
> reporting so I thought a scatterlist was the best way to gather the pages
> together so that they could be mapped with one call.
> 

Ok, might be worth adding a comment about that noting that it may appear
excessive but it's to avoid a situation where drivers have to convert
page arrays to scatterlists.

> > > +	/*
> > > +	 * Upper limit on the number of pages that the report function
> > > +	 * expects to be placed into the scatterlist to be processed.
> > > +	 */
> > > +	unsigned int capacity;
> > > +
> > 
> > Instead of requiring the driver to specify capacity, why did you not
> > simply make it PAGE_REPORTING_HWM and statically declare the array?
> 
> The problem is the device might not be able to support PAGE_REPORTING_HWM.

I don't see why given that a driver that cannot handle PAGE_REPORTING_HWM
could process subsets of the pages reported. It seems unnecessary.

> So I needed some way for it to communicate that. I could allocate the
> array statically if that is what you prefer.
> 

It would make sense if it's a fixed size and let drivers figure out whether
the full array can be processed at once or has to be split into parts.

> > > +	/* The number of zones requesting reporting */
> > > +	atomic_t refcnt;
> > > +
> > 
> > The refcnt is overkill. Simply loop over all zones until there is a
> > clear pass. Despite being atomic, it actually requires the zone lock to
> > avoid races in places like this
> > 
> > 		if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
> > 			refcnt = page_reporting_process_zone(prdev, zone);
> > 
> > The optimisation is small and it's relatively subtle.
> 
> I disagree. Without that we can easily get into states where a zone
> requests reporting and we never get around to servicing it.

Why would that happen if all zones were iterated during a single pass?

> I am using the
> refcount to track that so that we don't shutdown the worker thread when
> there is at least one zone requesting service.
> 

Then either single pass or loop through all zones until a loop completes
with no zones to be processed. It's far simplier than a refcount.

> > >  static inline void __free_one_page(struct page *page,
> > >  		unsigned long pfn,
> > >  		struct zone *zone, unsigned int order,
> > > -		int migratetype)
> > > +		int migratetype, bool reported)
> > >  {
> > >  	struct capture_control *capc = task_capc(zone);
> > >  	unsigned long uninitialized_var(buddy_pfn);
> > > @@ -1048,7 +1053,9 @@ static inline void __free_one_page(struct page *page,
> > >  done_merging:
> > >  	set_page_order(page, order);
> > >  
> > > -	if (is_shuffle_order(order))
> > > +	if (reported)
> > > +		to_tail = true;
> > > +	else if (is_shuffle_order(order))
> > >  		to_tail = shuffle_pick_tail();
> > >  	else
> > >  		to_tail = buddy_merge_likely(pfn, buddy_pfn, page, order);
> > > @@ -1057,6 +1064,14 @@ static inline void __free_one_page(struct page *page,
> > >  		add_to_free_list_tail(page, zone, order, migratetype);
> > >  	else
> > >  		add_to_free_list(page, zone, order, migratetype);
> > > +
> > > +	/*
> > > +	 * No need to notify on a reported page as the total count of
> > > +	 * unreported pages will not have increased since we have essentially
> > > +	 * merged the reported page with one or more unreported pages.
> > > +	 */
> > > +	if (!reported)
> > > +		page_reporting_notify_free(zone, order);
> > >  }
> > >  
> > >  /*
> > 
> > If a reported page is merged with a larger buddy then the counts for the
> > lower order needs to be updated or the counter gets out of sync. Then the
> > reported status of the page needs to be cleared as the larger block may
> > only be partially reported and the count is wrong. I know why you want
> > the reported pages but it really should be in its own patch. This patch
> > really should be just the bare essentials to support reporting of free
> > pages even if it's sub-optimal. I know you also use reported_pages as
> > part of a stop condition but a basic stop condition is simply to do a
> > single pass.
> 
> Okay so there are actually a few things here to unpack.
> 
> First the comment is a bit out of date. The reason I don't need to bother
> with reporting is because I process the lower orders before the higher
> orders.
> As such we are still processing this zone if we are returning a
> reported page and it is merged. We will reprocess the larger page in the
> next pass.
> 

I don't see what guarantees the lower order buddy was definitely
reported. The lower order could have been in an isolated pageblock --
bit insane but not impossible. It also could have failed to be isolated
due to a watermark check. Either way, the accounting gets screwy.

> Second, at the point where we call this function with the reported flag
> set we have not yet set the page as reported. There is logic that will
> check for that later and set the bits and increment the count if the page
> becomes a buddy page and the order is still the same as the order we
> originally freed it under.
> 

That is sufficiently complex that it really should be in its own patch
because right now, any bug there would either revert the whole feature
or do a partial revert if it couldn't be fixed properly. At least having
the option of reverting the optimisation would be useful for testing.
By not splitting it out, we're also prevented from being able to ack the
basic infrastructure itself. Even if the optimisations could not be agreed
upon, the feature would still exist -- just slower than is ideal.

> Lastly, if you are concerned about us merging a reported page already on
> the list the count is updated and the flag is cleared from the page when
> it is deleted from the free list to be merged with the new page.
> 
> > > @@ -3228,6 +3243,36 @@ int __isolate_free_page(struct page *page, unsigned int order)
> > >  	return 1UL << order;
> > >  }
> > >  
> > > +#ifdef CONFIG_PAGE_REPORTING
> > > +/**
> > > + * __free_isolated_page - Return a now-isolated page back where we got it
> > > + * @page: Page that was isolated
> > > + * @order: Order of the isolated page
> > > + *
> > > + * This function is meant to return a page pulled from the free lists via
> > > + * __isolate_free_page back to the free lists they were pulled from.
> > > + */
> > 
> > Isolated within mm has special meaning already. __putback_reported_page?
> 
> I wanted to get away from calling it a reported page because it isn't at
> the point where we are calling this function. We don't set the reported
> bit on the page until we have placed it back into the free list and
> verified it wasn't merged.
> 
> I'm open to suggestions on how to name a function that is meant to be the
> inverse of __isolate_free_page. All I did is swap the isolate and free
> verb/adjective.
> 

__putback_isolated_page would at least have a similar naming pattern to
how LRU pages are isolated and putback.

> > > +static void
> > > +page_reporting_drain(struct page_reporting_dev_info *prdev, struct zone *zone)
> > > +{
> > > +	struct scatterlist *sg = prdev->sg;
> > > +
> > > +	/*
> > > +	 * Drain the now reported pages back into their respective
> > > +	 * free lists/areas. We assume at least one page is populated.
> > > +	 */
> > > +	do {
> > > +		unsigned int order = get_order(sg->length);
> > > +		struct page *page = sg_page(sg);
> > > +
> > > +		__free_isolated_page(page, order);
> > > +
> > > +		/*
> > > +		 * If page was not comingled with another page we can
> > > +		 * consider the result to be "reported" since the page
> > > +		 * hasn't been modified, otherwise we will need to
> > > +		 * report on the new larger page when we make our way
> > > +		 * up to that higher order.
> > > +		 */
> > > +		if (PageBuddy(page) && page_order(page) == order)
> > > +			mark_page_reported(page, zone, order);
> > > +	} while (!sg_is_last(sg++));
> > > +}
> > > +
> > > +/*
> > > + * The page reporting cycle consists of 4 stages, fill, report, drain, and
> > > + * idle. We will cycle through the first 3 stages until we cannot obtain a
> > > + * full scatterlist of pages, in that case we will switch to idle.
> > > + */
> > 
> > Document the return value because it's unclear why the number of populated
> > elements in sg is not tracked in page_reporting_dev_info. It seems
> > unnecessarily fragile.
> 
> Will do in terms of documenting the return value.
> 
> As far as tracking the number of elements it mostly just has to do with
> how this code has evolved. Originally this was just going straight to the
> report function. However that leads to partial requests and I wanted to
> avoid those since the cost for each call into the hypervisor can be high.
> 

It might be worth putting the count into the struct then as it's more
straight-forward than tracking struct state outside of the struct.

> > > +static int
> > > +page_reporting_process_zone(struct page_reporting_dev_info *prdev,
> > > +			    struct zone *zone)
> > > +{
> > > +	unsigned int order, mt, nents = 0;
> > > +	unsigned long watermark;
> > > +	int refcnt = -1;
> > > +
> > > +	page_reporting_populate_metadata(zone);
> > > +
> > > +	/* Generate minimum watermark to be able to guarantee progress */
> > > +	watermark = low_wmark_pages(zone) +
> > > +		    (prdev->capacity << PAGE_REPORTING_MIN_ORDER);
> > > +
> > > +	/*
> > > +	 * Cancel request if insufficient free memory or if we failed
> > > +	 * to allocate page reporting statistics for the zone.
> > > +	 */
> > > +	if (!zone_watermark_ok(zone, 0, watermark, 0, ALLOC_CMA) ||
> > > +	    !zone->reported_pages) {
> > > +		spin_lock_irq(&zone->lock);
> > > +		goto zone_not_ready;
> > > +	}
> > 
> > The zone lock is acquired just for the zone bit and the refcnt? That
> > seems drastic overkill because the bit handling is already racy and the
> > refcnt is overkill.
> 
> I'm not sure I follow.
> 

The lock is acquired before branching to zone_not_ready for the purpose
of clearing a zone bit and updating the ref count. That is hinting at
the possibility that zone lock is protecting much more than it should.

> > > +
> > > +	sg_init_table(prdev->sg, prdev->capacity);
> > > +
> > > +	/* Process each free list starting from lowest order/mt */
> > > +	for_each_reporting_migratetype_order(order, mt)
> > > +		nents = page_reporting_cycle(prdev, zone, order, mt, nents);
> > > +
> > > +	/* mark end of sg list and report the remainder */
> > > +	if (nents) {
> > > +		sg_mark_end(&prdev->sg[nents - 1]);
> > > +		prdev->report(prdev, nents);
> > > +	}
> > > +
> > > +	spin_lock_irq(&zone->lock);
> > > +
> > > +	/* flush any remaining pages out from the last report */
> > > +	if (nents)
> > > +		page_reporting_drain(prdev, zone);
> > > +
> > > +	/* check to see if values are low enough for us to stop for now */
> > > +	for (order = PAGE_REPORTING_MIN_ORDER; order < MAX_ORDER; order++) {
> > > +		if (pages_unreported(zone, order) < PAGE_REPORTING_HWM)
> > > +			continue;
> > > +#ifdef CONFIG_MEMORY_ISOLATION
> > > +		/*
> > > +		 * Do not allow a free_area with isolated pages to request
> > > +		 * that we continue with page reporting. Keep the reporting
> > > +		 * light until the isolated pages have been cleared.
> > > +		 */
> > > +		if (!free_area_empty(&zone->free_area[order], MIGRATE_ISOLATE))
> > > +			continue;
> > > +#endif
> > > +		goto zone_not_complete;
> > > +	}
> > > +
> > > +zone_not_ready:
> > > +	/*
> > > +	 * If there are no longer enough free pages to fully populate
> > > +	 * the scatterlist, then we can just shut it down for this zone.
> > > +	 */
> > > +	__clear_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags);
> > > +	refcnt = atomic_dec_return(&prdev->refcnt);
> > 
> > If a request comes in while processing is already happening then the
> > subsequent request is lost.
> 
> How can a request come in when we are holding the zone lock?
 
page_reporting_cycle drops the lock so there is no obvious guarantee
that requests are not lost.

> Basically the
> notify call can only occur while holding the zone lock. That is one of the
> reasons for holding the lock is to protect the flag and keep it consistent
> with the page counts as we clear it.
> 

Ok, you really should not be relying on the zone lock for protecting
state like this. It's fragile and it abuses what zone lock is for. It's
fragile because zone flags is not intended to protected by the zone
lock. Historically there would have been more flags but even currently,
the flag is tested and set outside of the zone lock. As you are using
non-atomic ops, it gets even more fragile because flag updates are
potentially lost.

The zone lock is primarily to protect the free lists and the watermarks
which are strongly related to the free lists and expanding that scope to
protect an atomic and a zone flag is very questionable. At some point,
zone lock is going to have to be broken up somehow because it's a very
heavy lock on the allocator side. If zone lock is protecting multiple
things like this, that would get even harder.

If necessary, add a separate lock and use it to protect state related to
the page reporting but as part of that, you almost certainly will have
to move to atomic versions of bit operations for zone flags because it's
not protected as such.

Again, it really should be the case that ZONE_PAGE_REPORTING_REQUESTED
is split out into a separate patch and introduced standalone. It would
probably be on the justification of avoiding unnecessary zone free list
searches but still, the optimisation should be reviewed separately and
the locking of state should be clear.

> The only case where we are not checking the counts is for the case above
> where we clear it due to us being too close to the watermark. In that case
> we want to just ignore the request anyway.
> 
> > > +zone_not_complete:
> > > +	spin_unlock_irq(&zone->lock);
> > > +
> > > +	return refcnt;
> > > +}
> > > +
> > > +static void page_reporting_process(struct work_struct *work)
> > > +{
> > > +	struct delayed_work *d_work = to_delayed_work(work);
> > > +	struct page_reporting_dev_info *prdev =
> > > +		container_of(d_work, struct page_reporting_dev_info, work);
> > > +	struct zone *zone = first_online_pgdat()->node_zones;
> > > +	int refcnt = -1;
> > > +
> > > +	do {
> > > +		if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
> > > +			refcnt = page_reporting_process_zone(prdev, zone);
> > > +
> > > +		/* Move to next zone, if at end of list start over */
> > > +		zone = next_zone(zone) ? : first_online_pgdat()->node_zones;
> > > +
> > > +		/*
> > > +		 * As long as refcnt has not reached zero there are still
> > > +		 * zones to be processed.
> > > +		 */
> > > +	} while (refcnt);
> > > +}
> > > +
> > 
> > This is being done from a workqueue context and potentially this runs
> > forever if the refcnt remains elevated. It's dangerous and should be in
> > it's own patch. In the basic implementation, just do a single pass of
> > all zones.
> 
> So I already have a modification that I have for the next version of the
> patch set where instead of doing a do/while I simply reschedule the
> delayed work to run again in 200ms if we still have work outstanding.
> 

That would be a bit more reasonable. It would probably remove the need
for the refcount entirely but either way -- split out the optimisations.
Otherwise the entirity of the feature has to be re-reviewed every time
which is time consuming.

> What I can do is split that out so that it is in the second patch with
> optimizations.
> 

One patch per optimisation please with data backing it up because it should
be possible to show that pages are reported faster or that overhead is
much lower. It would give a hint on whether the complexity is justified,
particularly if the locking is subtle.

> > > +/* request page reporting on this zone */
> > > +void __page_reporting_request(struct zone *zone)
> > > +{
> > > +	struct page_reporting_dev_info *prdev;
> > > +
> > > +	rcu_read_lock();
> > > +
> > > +	/*
> > > +	 * We use RCU to protect the pr_dev_info pointer. In almost all
> > > +	 * cases this should be present, however in the unlikely case of
> > > +	 * a shutdown this will be NULL and we should exit.
> > > +	 */
> > > +	prdev = rcu_dereference(pr_dev_info);
> > > +	if (unlikely(!prdev))
> > > +		goto out;
> > > +
> > > +	/*
> > > +	 * We can use separate test and set operations here as there
> > > +	 * is nothing else that can set or clear this bit while we are
> > > +	 * holding the zone lock. The advantage to doing it this way is
> > > +	 * that we don't have to dirty the cacheline unless we are
> > > +	 * changing the value.
> > > +	 */
> > > +	__set_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags);
> > > +
> > 
> > The comment implies that the bit should have been tested first.
> 
> It was in the page_reporting_notify_free() call. I can update the comment.
> 

But not the registration path (which probably doesn't matter). What is
much more critical is that again you are relying on the zone lock for
protection and correctness of a tiny micro-optimisation. Page reporting
really must not rely on the zone lock for anything outside of the free
list. We've already had a massive headache with mmap_sem protecting way
too much unnecessarily, we don't need a new one with zone that might
make it very difficult to break up that lock in the future.

> > > +	/*
> > > +	 * Delay the start of work to allow a sizable queue to
> > > +	 * build. For now we are limiting this to running no more
> > > +	 * than 5 times per second.
> > > +	 */
> > > +	if (!atomic_fetch_inc(&prdev->refcnt))
> > > +		schedule_delayed_work(&prdev->work, HZ / 5);
> > 
> > The refcnt is incremented whether this zone has already been requested or
> > not but only decremented when the bit is cleared so the refcnt potentially
> > remains elevated forever.
> 
> So this is one of the reasons why all accesses to the flag are protected
> by the zone lock.  Basically page_reporting_notify_free() and this function
> are both called with the zone lock held.
> 

This is extremely subtle and again, it's really not what zone lock is
for. It's too fragile but maybe refcnt is going away anyway. If not,
make a separate lock and document what it's protecting in the patch
introducing the optimisation.

> The only other caller of this function is page_reporting_register which
> calls it for each populated zone. I suppose I could make it for every zone
> since it is possible that someone unloaded the driver, unpopulated a zone,
> and then reloaded the driver, but anyway the idea is that the zone lock is
> keeping the flag and refcount consistent.
> 

I think you can guess at this point what I'll say about zone lock.

> > > +#define PAGE_REPORTING_MIN_ORDER	pageblock_order
> > 
> > This is potentially the same as MAX_ORDER-1. Not sure if it matters or
> > not.
> 
> It doesn't. Basically what I am going for is the largest order that it is
> safe to report on that will not break THP. If THP is not enabled then this
> value should be MAX_ORDER - 1 if I am not mistaken.
> 

Well, it's more about huge pages but yes, if you're ok with the
possibility that reporting only happens for the largest possible page
then that's fine.

> > > +	/* Limit notifications only to higher order pages */
> > > +	report_order = order - PAGE_REPORTING_MIN_ORDER;
> > > +	if (report_order < 0)
> > > +		return 0;
> > > +
> > > +	nr_free = zone->free_area[order].nr_free;
> > > +
> > > +	/* Only subtract reported_pages count if it is present */
> > > +	if (!zone->reported_pages)
> > > +		return nr_free;
> > > +
> > > +	return nr_free - zone->reported_pages[report_order];
> > > +}
> > 
> > Initially, this should be a full list traversal because I'm not
> > convinced the reported_pages count is kept perfectly in sync. The
> > optimisation confuses the review of the basic feature itself.
> 
> This is called per freed page in page_reporting_notify_free below. Are you
> saying I have to traverse the entire free list per freed page?
> 

In a basic unoptimised version -- yes. Put optimisations in separate
patches because then the basic feature itself can be reviewed and
commented upon. It'll also make it easier to understand any special
locking requirements for state.

> If that is what you are wanting I would still probably want to limit it
> such that we only walk the first PAGE_REPORTING_HWM worth of pages and if
> those are not reported then just report that many.
> 

Which in itself could be an optimisation patch. Maybe it'll be enough that
keeping track of the count is not worthwhile. Either way, the separate
patch could have supporting data on how much it improves the speed of
reporting pages so it can be compared to any other optimisation that
may be proposed. Supporting data would also help make the case that any
complexity introduced by the optimisation is worthwhile.

-- 
Mel Gorman
SUSE Labs

Re: [PATCH v14 3/6] mm: Introduce Reported pages

[Alexander Duyck]

On Wed, 2019-11-27 at 18:35 +0000, Mel Gorman wrote:
> On Wed, Nov 27, 2019 at 09:22:09AM -0800, Alexander Duyck wrote:
> > > Ok, I'm ok with how this hooks into the allocator as the overhead is
> > > minimal. However, the patch itself still includes a number of
> > > optimisations instead of being a bare-boned implementation of the
> > > feature with optimisations layered on top. I think some of the
> > > optimisations are also broken but it's harder to be sure because both
> > > the feature and the optimisations are lumped together.
> > 
> > Well I can work on splitting them out if need be. 
> > 
> 
> It would be preferred because some of the optimisations are fairly
> subtle and it distracts from having a basic implementation of the
> feature as a foundation to build upon.

Yeah, I will work on cleaning this up and then splitting out the
optimizations.

> > > > diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h
> > > > new file mode 100644
> > > > index 000000000000..925a16b1d14b
> > > > --- /dev/null
> > > > +++ b/include/linux/page_reporting.h
> > > > @@ -0,0 +1,31 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > +#ifndef _LINUX_PAGE_REPORTING_H
> > > > +#define _LINUX_PAGE_REPORTING_H
> > > > +
> > > > +#include <linux/mmzone.h>
> > > > +
> > > > +struct page_reporting_dev_info {
> > > > +	/* function that alters pages to make them "reported" */
> > > > +	void (*report)(struct page_reporting_dev_info *prdev,
> > > > +		       unsigned int nents);
> > > > +
> > > > +	/* pointer to scatterlist containing pages to be processed */
> > > > +	struct scatterlist *sg;
> > > > +
> > > 
> > > The choice of scatterlist is curious because it adds unnecessary complexity
> > > without needing the benefits scatterlist and sg_table provides. All you
> > > really need is an array of pages that is NULL terminated because you can
> > > get the order of the struct page when it's a buddy page.
> > 
> > The scatterlist is mostly about dealing with DMA mapping of the pages.
> > Most devices will need a way to push the pages out to the device doing the
> > reporting so I thought a scatterlist was the best way to gather the pages
> > together so that they could be mapped with one call.
> > 
> 
> Ok, might be worth adding a comment about that noting that it may appear
> excessive but it's to avoid a situation where drivers have to convert
> page arrays to scatterlists.

I'll probably do that. I might also be able to split this out of the
structure. I will still pass around the scatterlist but I can explain the
purpose.

> > > > +	/*
> > > > +	 * Upper limit on the number of pages that the report function
> > > > +	 * expects to be placed into the scatterlist to be processed.
> > > > +	 */
> > > > +	unsigned int capacity;
> > > > +
> > > 
> > > Instead of requiring the driver to specify capacity, why did you not
> > > simply make it PAGE_REPORTING_HWM and statically declare the array?
> > 
> > The problem is the device might not be able to support PAGE_REPORTING_HWM.
> 
> I don't see why given that a driver that cannot handle PAGE_REPORTING_HWM
> could process subsets of the pages reported. It seems unnecessary.

I suppose it might be considered an optimization. Basically what I was
wanting to avoid is pulling out more pages then the device can process in
a single call. That is what the capacity represents. Basically the virtio
device can have a limited ring size which limits how many pages we can
have outstanding at a time. So I wanted to avoid pulling pages that I
couldn't immediately report.

> > So I needed some way for it to communicate that. I could allocate the
> > array statically if that is what you prefer.
> > 
> 
> It would make sense if it's a fixed size and let drivers figure out whether
> the full array can be processed at once or has to be split into parts.

In terms of allocation the fixed size makes sense to me. I still think it
would be good to communicate how many the device can process in a single
request so we don't have pages having to wait as the device has to segment
the scatterlist to process pieces of it.

> > > > +	/* The number of zones requesting reporting */
> > > > +	atomic_t refcnt;
> > > > +
> > > 
> > > The refcnt is overkill. Simply loop over all zones until there is a
> > > clear pass. Despite being atomic, it actually requires the zone lock to
> > > avoid races in places like this
> > > 
> > > 		if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
> > > 			refcnt = page_reporting_process_zone(prdev, zone);
> > > 
> > > The optimisation is small and it's relatively subtle.
> > 
> > I disagree. Without that we can easily get into states where a zone
> > requests reporting and we never get around to servicing it.
> 
> Why would that happen if all zones were iterated during a single pass?

It is because I am not rescheduling the work while it is ongoing. My
notifier is using the reference count to only schedule on the first
request and not stop processing until we reach 0. Without that I have to
schedule the worker ever time a zone requests processing.

> > I am using the
> > refcount to track that so that we don't shutdown the worker thread when
> > there is at least one zone requesting service.
> > 
> 
> Then either single pass or loop through all zones until a loop completes
> with no zones to be processed. It's far simplier than a refcount.

I could do that if we only had one zone to process. However the problem is
that we could be processing multiple zones.

> > > >  static inline void __free_one_page(struct page *page,
> > > >  		unsigned long pfn,
> > > >  		struct zone *zone, unsigned int order,
> > > > -		int migratetype)
> > > > +		int migratetype, bool reported)
> > > >  {
> > > >  	struct capture_control *capc = task_capc(zone);
> > > >  	unsigned long uninitialized_var(buddy_pfn);
> > > > @@ -1048,7 +1053,9 @@ static inline void __free_one_page(struct page *page,
> > > >  done_merging:
> > > >  	set_page_order(page, order);
> > > >  
> > > > -	if (is_shuffle_order(order))
> > > > +	if (reported)
> > > > +		to_tail = true;
> > > > +	else if (is_shuffle_order(order))
> > > >  		to_tail = shuffle_pick_tail();
> > > >  	else
> > > >  		to_tail = buddy_merge_likely(pfn, buddy_pfn, page, order);
> > > > @@ -1057,6 +1064,14 @@ static inline void __free_one_page(struct page *page,
> > > >  		add_to_free_list_tail(page, zone, order, migratetype);
> > > >  	else
> > > >  		add_to_free_list(page, zone, order, migratetype);
> > > > +
> > > > +	/*
> > > > +	 * No need to notify on a reported page as the total count of
> > > > +	 * unreported pages will not have increased since we have essentially
> > > > +	 * merged the reported page with one or more unreported pages.
> > > > +	 */
> > > > +	if (!reported)
> > > > +		page_reporting_notify_free(zone, order);
> > > >  }
> > > >  
> > > >  /*
> > > 
> > > If a reported page is merged with a larger buddy then the counts for the
> > > lower order needs to be updated or the counter gets out of sync. Then the
> > > reported status of the page needs to be cleared as the larger block may
> > > only be partially reported and the count is wrong. I know why you want
> > > the reported pages but it really should be in its own patch. This patch
> > > really should be just the bare essentials to support reporting of free
> > > pages even if it's sub-optimal. I know you also use reported_pages as
> > > part of a stop condition but a basic stop condition is simply to do a
> > > single pass.
> > 
> > Okay so there are actually a few things here to unpack.
> > 
> > First the comment is a bit out of date. The reason I don't need to bother
> > with reporting is because I process the lower orders before the higher
> > orders.
> > As such we are still processing this zone if we are returning a
> > reported page and it is merged. We will reprocess the larger page in the
> > next pass.
> > 
> 
> I don't see what guarantees the lower order buddy was definitely
> reported. The lower order could have been in an isolated pageblock --
> bit insane but not impossible. It also could have failed to be isolated
> due to a watermark check. Either way, the accounting gets screwy.

So if we merge a page it is not considered reported. I think you might be
confusing this with the bits in page_reporting_drain which check to verify
that the page we freed became a buddy and that buddy is the same order as
what we originally released. If a reported page is merged it is no longer
considered to be reported.

Basically if we pull a reported page from the free list to merge it then
we clear the reported status and decrement the reported page count for
that order.

> > Second, at the point where we call this function with the reported flag
> > set we have not yet set the page as reported. There is logic that will
> > check for that later and set the bits and increment the count if the page
> > becomes a buddy page and the order is still the same as the order we
> > originally freed it under.
> > 
> 
> That is sufficiently complex that it really should be in its own patch
> because right now, any bug there would either revert the whole feature
> or do a partial revert if it couldn't be fixed properly. At least having
> the option of reverting the optimisation would be useful for testing.
> By not splitting it out, we're also prevented from being able to ack the
> basic infrastructure itself. Even if the optimisations could not be agreed
> upon, the feature would still exist -- just slower than is ideal.

The only optimization here is the use of the reported variable. That can
actually go away since I hadn't realized I had changed things in the page
reporting cycle so that we don't clear the REPORTING_REQUESTED flag until
after the last drain has completed.

I'll work on splitting out the reported_pages bit into a separate patch,
and I will see what I can do about refcount since that seems to be the
other pain point.

> > Lastly, if you are concerned about us merging a reported page already on
> > the list the count is updated and the flag is cleared from the page when
> > it is deleted from the free list to be merged with the new page.
> > 
> > > > @@ -3228,6 +3243,36 @@ int __isolate_free_page(struct page *page, unsigned int order)
> > > >  	return 1UL << order;
> > > >  }
> > > >  
> > > > +#ifdef CONFIG_PAGE_REPORTING
> > > > +/**
> > > > + * __free_isolated_page - Return a now-isolated page back where we got it
> > > > + * @page: Page that was isolated
> > > > + * @order: Order of the isolated page
> > > > + *
> > > > + * This function is meant to return a page pulled from the free lists via
> > > > + * __isolate_free_page back to the free lists they were pulled from.
> > > > + */
> > > 
> > > Isolated within mm has special meaning already. __putback_reported_page?
> > 
> > I wanted to get away from calling it a reported page because it isn't at
> > the point where we are calling this function. We don't set the reported
> > bit on the page until we have placed it back into the free list and
> > verified it wasn't merged.
> > 
> > I'm open to suggestions on how to name a function that is meant to be the
> > inverse of __isolate_free_page. All I did is swap the isolate and free
> > verb/adjective.
> > 
> 
> __putback_isolated_page would at least have a similar naming pattern to
> how LRU pages are isolated and putback.

I can work with that then.

> > > > +static void
> > > > +page_reporting_drain(struct page_reporting_dev_info *prdev, struct zone *zone)
> > > > +{
> > > > +	struct scatterlist *sg = prdev->sg;
> > > > +
> > > > +	/*
> > > > +	 * Drain the now reported pages back into their respective
> > > > +	 * free lists/areas. We assume at least one page is populated.
> > > > +	 */
> > > > +	do {
> > > > +		unsigned int order = get_order(sg->length);
> > > > +		struct page *page = sg_page(sg);
> > > > +
> > > > +		__free_isolated_page(page, order);
> > > > +
> > > > +		/*
> > > > +		 * If page was not comingled with another page we can
> > > > +		 * consider the result to be "reported" since the page
> > > > +		 * hasn't been modified, otherwise we will need to
> > > > +		 * report on the new larger page when we make our way
> > > > +		 * up to that higher order.
> > > > +		 */
> > > > +		if (PageBuddy(page) && page_order(page) == order)
> > > > +			mark_page_reported(page, zone, order);
> > > > +	} while (!sg_is_last(sg++));
> > > > +}
> > > > +
> > > > +/*
> > > > + * The page reporting cycle consists of 4 stages, fill, report, drain, and
> > > > + * idle. We will cycle through the first 3 stages until we cannot obtain a
> > > > + * full scatterlist of pages, in that case we will switch to idle.
> > > > + */
> > > 
> > > Document the return value because it's unclear why the number of populated
> > > elements in sg is not tracked in page_reporting_dev_info. It seems
> > > unnecessarily fragile.
> > 
> > Will do in terms of documenting the return value.
> > 
> > As far as tracking the number of elements it mostly just has to do with
> > how this code has evolved. Originally this was just going straight to the
> > report function. However that leads to partial requests and I wanted to
> > avoid those since the cost for each call into the hypervisor can be high.
> > 
> 
> It might be worth putting the count into the struct then as it's more
> straight-forward than tracking struct state outside of the struct.

Yeah, that is kind of what I was thinking. If nothing else I might look at
using something like an sg_table structure to pass the entries around.

> > > > +static int
> > > > +page_reporting_process_zone(struct page_reporting_dev_info *prdev,
> > > > +			    struct zone *zone)
> > > > +{
> > > > +	unsigned int order, mt, nents = 0;
> > > > +	unsigned long watermark;
> > > > +	int refcnt = -1;
> > > > +
> > > > +	page_reporting_populate_metadata(zone);
> > > > +
> > > > +	/* Generate minimum watermark to be able to guarantee progress */
> > > > +	watermark = low_wmark_pages(zone) +
> > > > +		    (prdev->capacity << PAGE_REPORTING_MIN_ORDER);
> > > > +
> > > > +	/*
> > > > +	 * Cancel request if insufficient free memory or if we failed
> > > > +	 * to allocate page reporting statistics for the zone.
> > > > +	 */
> > > > +	if (!zone_watermark_ok(zone, 0, watermark, 0, ALLOC_CMA) ||
> > > > +	    !zone->reported_pages) {
> > > > +		spin_lock_irq(&zone->lock);
> > > > +		goto zone_not_ready;
> > > > +	}
> > > 
> > > The zone lock is acquired just for the zone bit and the refcnt? That
> > > seems drastic overkill because the bit handling is already racy and the
> > > refcnt is overkill.
> > 
> > I'm not sure I follow.
> > 
> 
> The lock is acquired before branching to zone_not_ready for the purpose
> of clearing a zone bit and updating the ref count. That is hinting at
> the possibility that zone lock is protecting much more than it should.

I'm using it to protect the bit and the page counts. Basically I cannot
rely on the page counts being valid if I am not holding the lock. So in
order to avoid races between the bit and the page counts I hold the lock
when I clear the bit.

> > > > +
> > > > +	sg_init_table(prdev->sg, prdev->capacity);
> > > > +
> > > > +	/* Process each free list starting from lowest order/mt */
> > > > +	for_each_reporting_migratetype_order(order, mt)
> > > > +		nents = page_reporting_cycle(prdev, zone, order, mt, nents);
> > > > +
> > > > +	/* mark end of sg list and report the remainder */
> > > > +	if (nents) {
> > > > +		sg_mark_end(&prdev->sg[nents - 1]);
> > > > +		prdev->report(prdev, nents);
> > > > +	}
> > > > +
> > > > +	spin_lock_irq(&zone->lock);
> > > > +
> > > > +	/* flush any remaining pages out from the last report */
> > > > +	if (nents)
> > > > +		page_reporting_drain(prdev, zone);
> > > > +
> > > > +	/* check to see if values are low enough for us to stop for now */
> > > > +	for (order = PAGE_REPORTING_MIN_ORDER; order < MAX_ORDER; order++) {
> > > > +		if (pages_unreported(zone, order) < PAGE_REPORTING_HWM)
> > > > +			continue;
> > > > +#ifdef CONFIG_MEMORY_ISOLATION
> > > > +		/*
> > > > +		 * Do not allow a free_area with isolated pages to request
> > > > +		 * that we continue with page reporting. Keep the reporting
> > > > +		 * light until the isolated pages have been cleared.
> > > > +		 */
> > > > +		if (!free_area_empty(&zone->free_area[order], MIGRATE_ISOLATE))
> > > > +			continue;
> > > > +#endif
> > > > +		goto zone_not_complete;
> > > > +	}
> > > > +
> > > > +zone_not_ready:
> > > > +	/*
> > > > +	 * If there are no longer enough free pages to fully populate
> > > > +	 * the scatterlist, then we can just shut it down for this zone.
> > > > +	 */
> > > > +	__clear_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags);
> > > > +	refcnt = atomic_dec_return(&prdev->refcnt);
> > > 
> > > If a request comes in while processing is already happening then the
> > > subsequent request is lost.
> > 
> > How can a request come in when we are holding the zone lock?
>  
> page_reporting_cycle drops the lock so there is no obvious guarantee
> that requests are not lost.

Yes, but page_reporting_cycle will not clear the bit. It is only cleared
in this last bleock where we go through and verify the page counts are low
enough that we can justify clearing the bit.

This is what I am talking about in terms of the page counts and the bit
being synchronized via the lock. Both the notifier and this piece of code
will not set nor clear the bit without first having to check
pages_unreported and then either setting or clearing the bit depending on
if we are the notifier or the reporting process.

> > Basically the
> > notify call can only occur while holding the zone lock. That is one of the
> > reasons for holding the lock is to protect the flag and keep it consistent
> > with the page counts as we clear it.
> > 
> 
> Ok, you really should not be relying on the zone lock for protecting
> state like this. It's fragile and it abuses what zone lock is for. It's
> fragile because zone flags is not intended to protected by the zone
> lock. Historically there would have been more flags but even currently,
> the flag is tested and set outside of the zone lock. As you are using
> non-atomic ops, it gets even more fragile because flag updates are
> potentially lost.
> 
> The zone lock is primarily to protect the free lists and the watermarks
> which are strongly related to the free lists and expanding that scope to
> protect an atomic and a zone flag is very questionable. At some point,
> zone lock is going to have to be broken up somehow because it's a very
> heavy lock on the allocator side. If zone lock is protecting multiple
> things like this, that would get even harder.

So I am not necessarily using it to protect the atomic refcount. The
placement of the atomic refcount as more to do with the setting and
clearing of the flag bit.

What I am using the lock for is mostly to keep the farea[x].nr_free
coherent versus the reported_pages counts.

> If necessary, add a separate lock and use it to protect state related to
> the page reporting but as part of that, you almost certainly will have
> to move to atomic versions of bit operations for zone flags because it's
> not protected as such.
> 
> Again, it really should be the case that ZONE_PAGE_REPORTING_REQUESTED
> is split out into a separate patch and introduced standalone. It would
> probably be on the justification of avoiding unnecessary zone free list
> searches but still, the optimisation should be reviewed separately and
> the locking of state should be clear.

I can look into it. However between this, the removal of the
reported_pages counters, and the desire to remove the reference count it
sounds like what we have is essentially just a blind scanner that will
never really know when it is done and would likely be very lossy.

> > The only case where we are not checking the counts is for the case above
> > where we clear it due to us being too close to the watermark. In that case
> > we want to just ignore the request anyway.
> > 
> > > > +zone_not_complete:
> > > > +	spin_unlock_irq(&zone->lock);
> > > > +
> > > > +	return refcnt;
> > > > +}
> > > > +
> > > > +static void page_reporting_process(struct work_struct *work)
> > > > +{
> > > > +	struct delayed_work *d_work = to_delayed_work(work);
> > > > +	struct page_reporting_dev_info *prdev =
> > > > +		container_of(d_work, struct page_reporting_dev_info, work);
> > > > +	struct zone *zone = first_online_pgdat()->node_zones;
> > > > +	int refcnt = -1;
> > > > +
> > > > +	do {
> > > > +		if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
> > > > +			refcnt = page_reporting_process_zone(prdev, zone);
> > > > +
> > > > +		/* Move to next zone, if at end of list start over */
> > > > +		zone = next_zone(zone) ? : first_online_pgdat()->node_zones;
> > > > +
> > > > +		/*
> > > > +		 * As long as refcnt has not reached zero there are still
> > > > +		 * zones to be processed.
> > > > +		 */
> > > > +	} while (refcnt);
> > > > +}
> > > > +
> > > 
> > > This is being done from a workqueue context and potentially this runs
> > > forever if the refcnt remains elevated. It's dangerous and should be in
> > > it's own patch. In the basic implementation, just do a single pass of
> > > all zones.
> > 
> > So I already have a modification that I have for the next version of the
> > patch set where instead of doing a do/while I simply reschedule the
> > delayed work to run again in 200ms if we still have work outstanding.
> > 
> 
> That would be a bit more reasonable. It would probably remove the need
> for the refcount entirely but either way -- split out the optimisations.
> Otherwise the entirity of the feature has to be re-reviewed every time
> which is time consuming.

Okay.

> > What I can do is split that out so that it is in the second patch with
> > optimizations.
> > 
> 
> One patch per optimisation please with data backing it up because it should
> be possible to show that pages are reported faster or that overhead is
> much lower. It would give a hint on whether the complexity is justified,
> particularly if the locking is subtle.

Okay, got it.

> > > > +/* request page reporting on this zone */
> > > > +void __page_reporting_request(struct zone *zone)
> > > > +{
> > > > +	struct page_reporting_dev_info *prdev;
> > > > +
> > > > +	rcu_read_lock();
> > > > +
> > > > +	/*
> > > > +	 * We use RCU to protect the pr_dev_info pointer. In almost all
> > > > +	 * cases this should be present, however in the unlikely case of
> > > > +	 * a shutdown this will be NULL and we should exit.
> > > > +	 */
> > > > +	prdev = rcu_dereference(pr_dev_info);
> > > > +	if (unlikely(!prdev))
> > > > +		goto out;
> > > > +
> > > > +	/*
> > > > +	 * We can use separate test and set operations here as there
> > > > +	 * is nothing else that can set or clear this bit while we are
> > > > +	 * holding the zone lock. The advantage to doing it this way is
> > > > +	 * that we don't have to dirty the cacheline unless we are
> > > > +	 * changing the value.
> > > > +	 */
> > > > +	__set_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags);
> > > > +
> > > 
> > > The comment implies that the bit should have been tested first.
> > 
> > It was in the page_reporting_notify_free() call. I can update the comment.
> > 
> 
> But not the registration path (which probably doesn't matter). What is
> much more critical is that again you are relying on the zone lock for
> protection and correctness of a tiny micro-optimisation. Page reporting
> really must not rely on the zone lock for anything outside of the free
> list. We've already had a massive headache with mmap_sem protecting way
> too much unnecessarily, we don't need a new one with zone that might
> make it very difficult to break up that lock in the future.

So the main use of the zone lock is for the nr_free values,
reported_pages, and the REPORTING_REQUESTED flag. I had lumped them
together under the zone lock as they were already all being used together
anyway.

Adding another lock would likely just add more complexity since then we
have to worry about lock inversions and such. That was one motivation for
not taking that approach.

> > > > +	/*
> > > > +	 * Delay the start of work to allow a sizable queue to
> > > > +	 * build. For now we are limiting this to running no more
> > > > +	 * than 5 times per second.
> > > > +	 */
> > > > +	if (!atomic_fetch_inc(&prdev->refcnt))
> > > > +		schedule_delayed_work(&prdev->work, HZ / 5);
> > > 
> > > The refcnt is incremented whether this zone has already been requested or
> > > not but only decremented when the bit is cleared so the refcnt potentially
> > > remains elevated forever.
> > 
> > So this is one of the reasons why all accesses to the flag are protected
> > by the zone lock.  Basically page_reporting_notify_free() and this function
> > are both called with the zone lock held.
> > 
> 
> This is extremely subtle and again, it's really not what zone lock is
> for. It's too fragile but maybe refcnt is going away anyway. If not,
> make a separate lock and document what it's protecting in the patch
> introducing the optimisation.

Again the problem is I have 3 elements that I have to keep in sync. In the
case of the page_reporting_notify_free I wouldn't be able to drop the zone
lock without either adding additional complexity to the page freeing path.

> > The only other caller of this function is page_reporting_register which
> > calls it for each populated zone. I suppose I could make it for every zone
> > since it is possible that someone unloaded the driver, unpopulated a zone,
> > and then reloaded the driver, but anyway the idea is that the zone lock is
> > keeping the flag and refcount consistent.
> > 
> 
> I think you can guess at this point what I'll say about zone lock.

I get that you don't like the use of the zone lock but the problem is I am
not sure there is a good way for the reported_pages and flag to stay in
sync without it. I could do away with the flag if that is what we really
want but then I am still stuck using the zone lock as I have to either
walk the lists or read the nr_free from the free area to determine if a
zone has to be processed.

> > > > +#define PAGE_REPORTING_MIN_ORDER	pageblock_order
> > > 
> > > This is potentially the same as MAX_ORDER-1. Not sure if it matters or
> > > not.
> > 
> > It doesn't. Basically what I am going for is the largest order that it is
> > safe to report on that will not break THP. If THP is not enabled then this
> > value should be MAX_ORDER - 1 if I am not mistaken.
> > 
> 
> Well, it's more about huge pages but yes, if you're ok with the
> possibility that reporting only happens for the largest possible page
> then that's fine.
> 
> > > > +	/* Limit notifications only to higher order pages */
> > > > +	report_order = order - PAGE_REPORTING_MIN_ORDER;
> > > > +	if (report_order < 0)
> > > > +		return 0;
> > > > +
> > > > +	nr_free = zone->free_area[order].nr_free;
> > > > +
> > > > +	/* Only subtract reported_pages count if it is present */
> > > > +	if (!zone->reported_pages)
> > > > +		return nr_free;
> > > > +
> > > > +	return nr_free - zone->reported_pages[report_order];
> > > > +}
> > > 
> > > Initially, this should be a full list traversal because I'm not
> > > convinced the reported_pages count is kept perfectly in sync. The
> > > optimisation confuses the review of the basic feature itself.
> > 
> > This is called per freed page in page_reporting_notify_free below. Are you
> > saying I have to traverse the entire free list per freed page?
> > 
> 
> In a basic unoptimised version -- yes. Put optimisations in separate
> patches because then the basic feature itself can be reviewed and
> commented upon. It'll also make it easier to understand any special
> locking requirements for state.
> 
> > If that is what you are wanting I would still probably want to limit it
> > such that we only walk the first PAGE_REPORTING_HWM worth of pages and if
> > those are not reported then just report that many.
> > 
> 
> Which in itself could be an optimisation patch. Maybe it'll be enough that
> keeping track of the count is not worthwhile. Either way, the separate
> patch could have supporting data on how much it improves the speed of
> reporting pages so it can be compared to any other optimisation that
> may be proposed. Supporting data would also help make the case that any
> complexity introduced by the optimisation is worthwhile.

I'll see what I can do to break this apart. I'm just not a fan of redoing
the work multiple times so that I can have a non-optimized version versus
an optimized one.

I'll also go take a look over the work scheduler. It is possible that
maybe I can get away from some of the refcount stuff if the scheduling of
delayed work takes care of it for me.

Re: [PATCH v14 3/6] mm: Introduce Reported pages

[Mel Gorman]

On Wed, Nov 27, 2019 at 01:55:02PM -0800, Alexander Duyck wrote:
> > > > > +	/*
> > > > > +	 * Upper limit on the number of pages that the report function
> > > > > +	 * expects to be placed into the scatterlist to be processed.
> > > > > +	 */
> > > > > +	unsigned int capacity;
> > > > > +
> > > > 
> > > > Instead of requiring the driver to specify capacity, why did you not
> > > > simply make it PAGE_REPORTING_HWM and statically declare the array?
> > > 
> > > The problem is the device might not be able to support PAGE_REPORTING_HWM.
> > 
> > I don't see why given that a driver that cannot handle PAGE_REPORTING_HWM
> > could process subsets of the pages reported. It seems unnecessary.
> 
> I suppose it might be considered an optimization. Basically what I was
> wanting to avoid is pulling out more pages then the device can process in
> a single call. That is what the capacity represents. Basically the virtio
> device can have a limited ring size which limits how many pages we can
> have outstanding at a time. So I wanted to avoid pulling pages that I
> couldn't immediately report.
> 

Given that the first user of this is part of the same series, I think
it's reasonable to only consider its limitations and keep it relatively
simple. We don't know what additional users, if any, may exist. Even if
they do, notifiers will need to be chained and there will be additional
work necessary no matter what way we jump now.

> > > So I needed some way for it to communicate that. I could allocate the
> > > array statically if that is what you prefer.
> > > 
> > 
> > It would make sense if it's a fixed size and let drivers figure out whether
> > the full array can be processed at once or has to be split into parts.
> 
> In terms of allocation the fixed size makes sense to me. I still think it
> would be good to communicate how many the device can process in a single
> request so we don't have pages having to wait as the device has to segment
> the scatterlist to process pieces of it.
> 

If that limitation should arise in the future, it can be considered
then.

> > > > > +	/* The number of zones requesting reporting */
> > > > > +	atomic_t refcnt;
> > > > > +
> > > > 
> > > > The refcnt is overkill. Simply loop over all zones until there is a
> > > > clear pass. Despite being atomic, it actually requires the zone lock to
> > > > avoid races in places like this
> > > > 
> > > > 		if (test_bit(ZONE_PAGE_REPORTING_REQUESTED, &zone->flags))
> > > > 			refcnt = page_reporting_process_zone(prdev, zone);
> > > > 
> > > > The optimisation is small and it's relatively subtle.
> > > 
> > > I disagree. Without that we can easily get into states where a zone
> > > requests reporting and we never get around to servicing it.
> > 
> > Why would that happen if all zones were iterated during a single pass?
> 
> It is because I am not rescheduling the work while it is ongoing. My
> notifier is using the reference count to only schedule on the first
> request and not stop processing until we reach 0. Without that I have to
> schedule the worker ever time a zone requests processing.
> 

Which in itself is simpler than trying to maintain a coherent refcount
with separate locking. When it gets down to it, a reschedule is not that
bad and it's a lot less subtle.

> > > I am using the
> > > refcount to track that so that we don't shutdown the worker thread when
> > > there is at least one zone requesting service.
> > > 
> > 
> > Then either single pass or loop through all zones until a loop completes
> > with no zones to be processed. It's far simplier than a refcount.
> 
> I could do that if we only had one zone to process. However the problem is
> that we could be processing multiple zones.
> 

Again, not necessarily a problem. There is no data on how much of a
problem, if any, there is with delays in zone processing.

> > > Second, at the point where we call this function with the reported flag
> > > set we have not yet set the page as reported. There is logic that will
> > > check for that later and set the bits and increment the count if the page
> > > becomes a buddy page and the order is still the same as the order we
> > > originally freed it under.
> > > 
> > 
> > That is sufficiently complex that it really should be in its own patch
> > because right now, any bug there would either revert the whole feature
> > or do a partial revert if it couldn't be fixed properly. At least having
> > the option of reverting the optimisation would be useful for testing.
> > By not splitting it out, we're also prevented from being able to ack the
> > basic infrastructure itself. Even if the optimisations could not be agreed
> > upon, the feature would still exist -- just slower than is ideal.
> 
> The only optimization here is the use of the reported variable. That can
> actually go away since I hadn't realized I had changed things in the page
> reporting cycle so that we don't clear the REPORTING_REQUESTED flag until
> after the last drain has completed.
> 
> I'll work on splitting out the reported_pages bit into a separate patch,
> and I will see what I can do about refcount since that seems to be the
> other pain point.
> 

Thanks

> > > > > +static int
> > > > > +page_reporting_process_zone(struct page_reporting_dev_info *prdev,
> > > > > +			    struct zone *zone)
> > > > > +{
> > > > > +	unsigned int order, mt, nents = 0;
> > > > > +	unsigned long watermark;
> > > > > +	int refcnt = -1;
> > > > > +
> > > > > +	page_reporting_populate_metadata(zone);
> > > > > +
> > > > > +	/* Generate minimum watermark to be able to guarantee progress */
> > > > > +	watermark = low_wmark_pages(zone) +
> > > > > +		    (prdev->capacity << PAGE_REPORTING_MIN_ORDER);
> > > > > +
> > > > > +	/*
> > > > > +	 * Cancel request if insufficient free memory or if we failed
> > > > > +	 * to allocate page reporting statistics for the zone.
> > > > > +	 */
> > > > > +	if (!zone_watermark_ok(zone, 0, watermark, 0, ALLOC_CMA) ||
> > > > > +	    !zone->reported_pages) {
> > > > > +		spin_lock_irq(&zone->lock);
> > > > > +		goto zone_not_ready;
> > > > > +	}
> > > > 
> > > > The zone lock is acquired just for the zone bit and the refcnt? That
> > > > seems drastic overkill because the bit handling is already racy and the
> > > > refcnt is overkill.
> > > 
> > > I'm not sure I follow.
> > > 
> > 
> > The lock is acquired before branching to zone_not_ready for the purpose
> > of clearing a zone bit and updating the ref count. That is hinting at
> > the possibility that zone lock is protecting much more than it should.
> 
> I'm using it to protect the bit and the page counts. Basically I cannot
> rely on the page counts being valid if I am not holding the lock. So in
> order to avoid races between the bit and the page counts I hold the lock
> when I clear the bit.
> 

In the initial step, avoid the page counts and take the hit. I might be
willing to give ground on the zone lock protecting the reported counts
as it is tightly related to the free page count. The zone lock should
definitely not protect the zone flags or refcount because it would make
the zone lock difficult, if not impossible, to split in the future. For
example, the most obvious approach to splitting the zone lock would be
to split it by PFN range within the zone or hash it based on some low
bits in the PFN number. That would be relatively trivial to implement
but impossible if the lock is also protecting zone-wide state such as a
refcount or zone flag. Sure, someone dealing with the zone lock could
try to fix page reporting at the same time but it'll be harder for them
to test and puts an unfortunate obstacle in their way.

> > > > If a request comes in while processing is already happening then the
> > > > subsequent request is lost.
> > > 
> > > How can a request come in when we are holding the zone lock?
> >  
> > page_reporting_cycle drops the lock so there is no obvious guarantee
> > that requests are not lost.
> 
> Yes, but page_reporting_cycle will not clear the bit. It is only cleared
> in this last bleock where we go through and verify the page counts are low
> enough that we can justify clearing the bit.
> 

Ok, that makes some sense although it could do with comments on the
lifecycle of the bit.

> This is what I am talking about in terms of the page counts and the bit
> being synchronized via the lock. Both the notifier and this piece of code
> will not set nor clear the bit without first having to check
> pages_unreported and then either setting or clearing the bit depending on
> if we are the notifier or the reporting process.
> 

But using the zone lock to protect that bit is a hard no because that zone
lock is way too heavy as it is, particularly as zone sizes are growing
ever larger and we now have machine where multiple L3 caches share the
same zone lock which is very expensive to bounce around.

Someone else reviewing might override me on this one as being an
obstacle based on patches that don't exist but I think they would find
it very hard to argue that the zone lock is fine as it is.

> > If necessary, add a separate lock and use it to protect state related to
> > the page reporting but as part of that, you almost certainly will have
> > to move to atomic versions of bit operations for zone flags because it's
> > not protected as such.
> > 
> > Again, it really should be the case that ZONE_PAGE_REPORTING_REQUESTED
> > is split out into a separate patch and introduced standalone. It would
> > probably be on the justification of avoiding unnecessary zone free list
> > searches but still, the optimisation should be reviewed separately and
> > the locking of state should be clear.
> 
> I can look into it. However between this, the removal of the
> reported_pages counters, and the desire to remove the reference count it
> sounds like what we have is essentially just a blind scanner that will
> never really know when it is done and would likely be very lossy.
> 

Well yes, initially that's exactly what it would be. Free page reporting
is not a critical path and the initial implementation should be simple.
That allows it to be determined how much complexity and maintenance
burden is needed to make it a bit faster. Prematurely expanding the
scope of the zone lock is such a maintenance burden.

> > But not the registration path (which probably doesn't matter). What is
> > much more critical is that again you are relying on the zone lock for
> > protection and correctness of a tiny micro-optimisation. Page reporting
> > really must not rely on the zone lock for anything outside of the free
> > list. We've already had a massive headache with mmap_sem protecting way
> > too much unnecessarily, we don't need a new one with zone that might
> > make it very difficult to break up that lock in the future.
> 
> So the main use of the zone lock is for the nr_free values,
> reported_pages, and the REPORTING_REQUESTED flag. I had lumped them
> together under the zone lock as they were already all being used together
> anyway.
> 

reported_pages may not be too bad because it's tightly related to the
free page count and would be updated at the same time that list operations
take place but not other zone-wide state.

> Adding another lock would likely just add more complexity since then we
> have to worry about lock inversions and such. That was one motivation for
> not taking that approach.
> 

Which is a fair reason but it gets hung up on the fact that it makes
splitting the zone lock and free list handling much harder.

> > > > > +	/*
> > > > > +	 * Delay the start of work to allow a sizable queue to
> > > > > +	 * build. For now we are limiting this to running no more
> > > > > +	 * than 5 times per second.
> > > > > +	 */
> > > > > +	if (!atomic_fetch_inc(&prdev->refcnt))
> > > > > +		schedule_delayed_work(&prdev->work, HZ / 5);
> > > > 
> > > > The refcnt is incremented whether this zone has already been requested or
> > > > not but only decremented when the bit is cleared so the refcnt potentially
> > > > remains elevated forever.
> > > 
> > > So this is one of the reasons why all accesses to the flag are protected
> > > by the zone lock.  Basically page_reporting_notify_free() and this function
> > > are both called with the zone lock held.
> > > 
> > 
> > This is extremely subtle and again, it's really not what zone lock is
> > for. It's too fragile but maybe refcnt is going away anyway. If not,
> > make a separate lock and document what it's protecting in the patch
> > introducing the optimisation.
> 
> Again the problem is I have 3 elements that I have to keep in sync. In the
> case of the page_reporting_notify_free I wouldn't be able to drop the zone
> lock without either adding additional complexity to the page freeing path.
> 

It leaks an unnecessary amount of maintenance burden into page_alloc,
which is performance critical, to optimise page reporting which is not
performance critical. Minimally, if zone lock is protecting an entire page
reporting cycle, it's blocking other allocations taking place at the same
time as reporting for an unnecessary length of time and incurring stalls.

> > > The only other caller of this function is page_reporting_register which
> > > calls it for each populated zone. I suppose I could make it for every zone
> > > since it is possible that someone unloaded the driver, unpopulated a zone,
> > > and then reloaded the driver, but anyway the idea is that the zone lock is
> > > keeping the flag and refcount consistent.
> > > 
> > 
> > I think you can guess at this point what I'll say about zone lock.
> 
> I get that you don't like the use of the zone lock but the problem is I am
> not sure there is a good way for the reported_pages and flag to stay in
> sync without it. I could do away with the flag if that is what we really
> want but then I am still stuck using the zone lock as I have to either
> walk the lists or read the nr_free from the free area to determine if a
> zone has to be processed.
> 

I'm not saying the flag needs to go away entirely, just that it should
not share the same lock as the page allocators free list unnecessarily.
Free page reporting can afford to stall or even go to sleep if protected
by a mutex when it's not manipulating free lists even if that means some
duplicate processing may be necessary.

> > 
> > Which in itself could be an optimisation patch. Maybe it'll be enough that
> > keeping track of the count is not worthwhile. Either way, the separate
> > patch could have supporting data on how much it improves the speed of
> > reporting pages so it can be compared to any other optimisation that
> > may be proposed. Supporting data would also help make the case that any
> > complexity introduced by the optimisation is worthwhile.
> 
> I'll see what I can do to break this apart. I'm just not a fan of redoing
> the work multiple times so that I can have a non-optimized version versus
> an optimized one.
> 

While I understand that, right now the optimisations are blocking the
feature itself which is also not a situation you want to be in.

-- 
Mel Gorman
SUSE Labs

Re: [PATCH v14 6/6] virtio-balloon: Add support for providing unused page reports to host

[David Hildenbrand]

On 19.11.19 22:46, Alexander Duyck wrote:
> From: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> 
> Add support for the page reporting feature provided by virtio-balloon.
> Reporting differs from the regular balloon functionality in that is is
> much less durable than a standard memory balloon. Instead of creating a
> list of pages that cannot be accessed the pages are only inaccessible
> while they are being indicated to the virtio interface. Once the
> interface has acknowledged them they are placed back into their respective
> free lists and are once again accessible by the guest system.

Maybe add something like "In contrast to ordinary balloon
inflation/deflation, the guest can reuse all reported pages immediately
after reporting has finished, without having to notify the hypervisor
about it (e.g., VIRTIO_BALLOON_F_MUST_TELL_HOST does not apply)."

[...]

>  /*
>   * Balloon device works in 4K page units.  So each page is pointed to by
> @@ -37,6 +38,9 @@
>  #define VIRTIO_BALLOON_FREE_PAGE_SIZE \
>  	(1 << (VIRTIO_BALLOON_FREE_PAGE_ORDER + PAGE_SHIFT))
>  
> +/*  limit on the number of pages that can be on the reporting vq */
> +#define VIRTIO_BALLOON_VRING_HINTS_MAX	16

Maybe rename that from HINTS to REPORTS

> +
>  #ifdef CONFIG_BALLOON_COMPACTION
>  static struct vfsmount *balloon_mnt;
>  #endif
> @@ -46,6 +50,7 @@ enum virtio_balloon_vq {
>  	VIRTIO_BALLOON_VQ_DEFLATE,
>  	VIRTIO_BALLOON_VQ_STATS,
>  	VIRTIO_BALLOON_VQ_FREE_PAGE,
> +	VIRTIO_BALLOON_VQ_REPORTING,
>  	VIRTIO_BALLOON_VQ_MAX
>  };
>  
> @@ -113,6 +118,10 @@ struct virtio_balloon {
>  
>  	/* To register a shrinker to shrink memory upon memory pressure */
>  	struct shrinker shrinker;
> +
> +	/* Unused page reporting device */

Sounds like the device is unused :D

"Device info for reporting unused pages" ?

I am in general wondering, should we rename "unused" to "free". I.e.,
"free page reporting" instead of "unused page reporting"? Or what was
the motivation behind using "unused" ?

> +	struct virtqueue *reporting_vq;
> +	struct page_reporting_dev_info pr_dev_info;
>  };
>  
>  static struct virtio_device_id id_table[] = {
> @@ -152,6 +161,32 @@ static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq)
>  
>  }
>  
> +void virtballoon_unused_page_report(struct page_reporting_dev_info *pr_dev_info,
> +				    unsigned int nents)
> +{
> +	struct virtio_balloon *vb =
> +		container_of(pr_dev_info, struct virtio_balloon, pr_dev_info);
> +	struct virtqueue *vq = vb->reporting_vq;
> +	unsigned int unused, err;
> +
> +	/* We should always be able to add these buffers to an empty queue. */

This comment somewhat contradicts the error handling (and comment)
below. Maybe just drop it?

> +	err = virtqueue_add_inbuf(vq, pr_dev_info->sg, nents, vb,
> +				  GFP_NOWAIT | __GFP_NOWARN);
> +
> +	/*
> +	 * In the extremely unlikely case that something has changed and we
> +	 * are able to trigger an error we will simply display a warning
> +	 * and exit without actually processing the pages.
> +	 */
> +	if (WARN_ON(err))
> +		return;

Maybe WARN_ON_ONCE? (to not flood the log on recurring errors)

> +
> +	virtqueue_kick(vq);
> +
> +	/* When host has read buffer, this completes via balloon_ack */
> +	wait_event(vb->acked, virtqueue_get_buf(vq, &unused));

Is it safe to rely on the same ack-ing mechanism as the inflate/deflate
queue? What if both mechanisms are used concurrently and race/both wait
for the hypervisor?

Maybe we need a separate vb->acked + callback function.

> +}
> +
>  static void set_page_pfns(struct virtio_balloon *vb,
>  			  __virtio32 pfns[], struct page *page)
>  {
> @@ -476,6 +511,7 @@ static int init_vqs(struct virtio_balloon *vb)
>  	names[VIRTIO_BALLOON_VQ_DEFLATE] = "deflate";
>  	names[VIRTIO_BALLOON_VQ_STATS] = NULL;
>  	names[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL;
> +	names[VIRTIO_BALLOON_VQ_REPORTING] = NULL;
>  
>  	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
>  		names[VIRTIO_BALLOON_VQ_STATS] = "stats";
> @@ -487,11 +523,19 @@ static int init_vqs(struct virtio_balloon *vb)
>  		callbacks[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL;
>  	}
>  
> +	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
> +		names[VIRTIO_BALLOON_VQ_REPORTING] = "reporting_vq";
> +		callbacks[VIRTIO_BALLOON_VQ_REPORTING] = balloon_ack;
> +	}
> +
>  	err = vb->vdev->config->find_vqs(vb->vdev, VIRTIO_BALLOON_VQ_MAX,
>  					 vqs, callbacks, names, NULL, NULL);
>  	if (err)
>  		return err;
>  
> +	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
> +		vb->reporting_vq = vqs[VIRTIO_BALLOON_VQ_REPORTING];
> +

I'd register these in the same order they are defined (IOW, move this
further down)

>  	vb->inflate_vq = vqs[VIRTIO_BALLOON_VQ_INFLATE];
>  	vb->deflate_vq = vqs[VIRTIO_BALLOON_VQ_DEFLATE];
>  	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
> @@ -932,12 +976,30 @@ static int virtballoon_probe(struct virtio_device *vdev)
>  		if (err)
>  			goto out_del_balloon_wq;
>  	}
> +
> +	vb->pr_dev_info.report = virtballoon_unused_page_report;
> +	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
> +		unsigned int capacity;
> +
> +		capacity = min_t(unsigned int,
> +				 virtqueue_get_vring_size(vb->reporting_vq),
> +				 VIRTIO_BALLOON_VRING_HINTS_MAX);
> +		vb->pr_dev_info.capacity = capacity;
> +
> +		err = page_reporting_register(&vb->pr_dev_info);
> +		if (err)
> +			goto out_unregister_shrinker;
> +	}

It can happen here that we start reporting before marking the device
ready. Can that be problematic?

Maybe we have to ignore any reports in virtballoon_unused_page_report()
until ready...

> +
>  	virtio_device_ready(vdev);
>  
>  	if (towards_target(vb))
>  		virtballoon_changed(vdev);
>  	return 0;
>  
> +out_unregister_shrinker:
> +	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
> +		virtio_balloon_unregister_shrinker(vb);

A sync is done implicitly, right? So after this call, we won't get any
new callbacks/are stuck in a callback.

>  out_del_balloon_wq:
>  	if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_FREE_PAGE_HINT))
>  		destroy_workqueue(vb->balloon_wq);
> @@ -966,6 +1028,8 @@ static void virtballoon_remove(struct virtio_device *vdev)
>  {
>  	struct virtio_balloon *vb = vdev->priv;
>  
> +	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
> +		page_reporting_unregister(&vb->pr_dev_info);

Dito, same question regarding syncs.

>  	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
>  		virtio_balloon_unregister_shrinker(vb);
>  	spin_lock_irq(&vb->stop_update_lock);
> @@ -1038,6 +1102,7 @@ static int virtballoon_validate(struct virtio_device *vdev)
>  	VIRTIO_BALLOON_F_DEFLATE_ON_OOM,
>  	VIRTIO_BALLOON_F_FREE_PAGE_HINT,
>  	VIRTIO_BALLOON_F_PAGE_POISON,
> +	VIRTIO_BALLOON_F_REPORTING,
>  };
>  
>  static struct virtio_driver virtio_balloon_driver = {
> diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virtio_balloon.h
> index a1966cd7b677..19974392d324 100644
> --- a/include/uapi/linux/virtio_balloon.h
> +++ b/include/uapi/linux/virtio_balloon.h
> @@ -36,6 +36,7 @@
>  #define VIRTIO_BALLOON_F_DEFLATE_ON_OOM	2 /* Deflate balloon on OOM */
>  #define VIRTIO_BALLOON_F_FREE_PAGE_HINT	3 /* VQ to report free pages */
>  #define VIRTIO_BALLOON_F_PAGE_POISON	4 /* Guest is using page poisoning */
> +#define VIRTIO_BALLOON_F_REPORTING	5 /* Page reporting virtqueue */
>  
>  /* Size of a PFN in the balloon interface. */
>  #define VIRTIO_BALLOON_PFN_SHIFT 12
> 
> 

Small and powerful patch :)

-- 
Thanks,

David / dhildenb

Re: [PATCH v14 6/6] virtio-balloon: Add support for providing unused page reports to host

[Michael S. Tsirkin]

On Thu, Nov 28, 2019 at 04:25:54PM +0100, David Hildenbrand wrote:
> On 19.11.19 22:46, Alexander Duyck wrote:
> > From: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> > 
> > Add support for the page reporting feature provided by virtio-balloon.
> > Reporting differs from the regular balloon functionality in that is is
> > much less durable than a standard memory balloon. Instead of creating a
> > list of pages that cannot be accessed the pages are only inaccessible
> > while they are being indicated to the virtio interface. Once the
> > interface has acknowledged them they are placed back into their respective
> > free lists and are once again accessible by the guest system.
> 
> Maybe add something like "In contrast to ordinary balloon
> inflation/deflation, the guest can reuse all reported pages immediately
> after reporting has finished, without having to notify the hypervisor
> about it (e.g., VIRTIO_BALLOON_F_MUST_TELL_HOST does not apply)."

Maybe we can make apply. The effect of reporting a page is effectively
putting it in a balloon then immediately taking it out. Maybe without
VIRTIO_BALLOON_F_MUST_TELL_HOST the pages can be reused before host
marked buffers used?

We didn't teach existing page hinting to behave like this, but maybe we
should, and maybe it's not too late, not a long time passed
since it was merged, and the whole shrinker based thing
seems to have been broken ...


BTW generally UAPI patches will have to be sent to virtio-dev
mailing list before they are merged.

> [...]
> 
> >  /*
> >   * Balloon device works in 4K page units.  So each page is pointed to by
> > @@ -37,6 +38,9 @@
> >  #define VIRTIO_BALLOON_FREE_PAGE_SIZE \
> >  	(1 << (VIRTIO_BALLOON_FREE_PAGE_ORDER + PAGE_SHIFT))
> >  
> > +/*  limit on the number of pages that can be on the reporting vq */
> > +#define VIRTIO_BALLOON_VRING_HINTS_MAX	16
> 
> Maybe rename that from HINTS to REPORTS
> 
> > +
> >  #ifdef CONFIG_BALLOON_COMPACTION
> >  static struct vfsmount *balloon_mnt;
> >  #endif
> > @@ -46,6 +50,7 @@ enum virtio_balloon_vq {
> >  	VIRTIO_BALLOON_VQ_DEFLATE,
> >  	VIRTIO_BALLOON_VQ_STATS,
> >  	VIRTIO_BALLOON_VQ_FREE_PAGE,
> > +	VIRTIO_BALLOON_VQ_REPORTING,
> >  	VIRTIO_BALLOON_VQ_MAX
> >  };
> >  
> > @@ -113,6 +118,10 @@ struct virtio_balloon {
> >  
> >  	/* To register a shrinker to shrink memory upon memory pressure */
> >  	struct shrinker shrinker;
> > +
> > +	/* Unused page reporting device */
> 
> Sounds like the device is unused :D
> 
> "Device info for reporting unused pages" ?
> 
> I am in general wondering, should we rename "unused" to "free". I.e.,
> "free page reporting" instead of "unused page reporting"? Or what was
> the motivation behind using "unused" ?
> 
> > +	struct virtqueue *reporting_vq;
> > +	struct page_reporting_dev_info pr_dev_info;
> >  };
> >  
> >  static struct virtio_device_id id_table[] = {
> > @@ -152,6 +161,32 @@ static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq)
> >  
> >  }
> >  
> > +void virtballoon_unused_page_report(struct page_reporting_dev_info *pr_dev_info,
> > +				    unsigned int nents)
> > +{
> > +	struct virtio_balloon *vb =
> > +		container_of(pr_dev_info, struct virtio_balloon, pr_dev_info);
> > +	struct virtqueue *vq = vb->reporting_vq;
> > +	unsigned int unused, err;
> > +
> > +	/* We should always be able to add these buffers to an empty queue. */
> 
> This comment somewhat contradicts the error handling (and comment)
> below. Maybe just drop it?
> 
> > +	err = virtqueue_add_inbuf(vq, pr_dev_info->sg, nents, vb,
> > +				  GFP_NOWAIT | __GFP_NOWARN);
> > +
> > +	/*
> > +	 * In the extremely unlikely case that something has changed and we
> > +	 * are able to trigger an error we will simply display a warning
> > +	 * and exit without actually processing the pages.
> > +	 */
> > +	if (WARN_ON(err))
> > +		return;
> 
> Maybe WARN_ON_ONCE? (to not flood the log on recurring errors)
> 
> > +
> > +	virtqueue_kick(vq);
> > +
> > +	/* When host has read buffer, this completes via balloon_ack */
> > +	wait_event(vb->acked, virtqueue_get_buf(vq, &unused));
> 
> Is it safe to rely on the same ack-ing mechanism as the inflate/deflate
> queue? What if both mechanisms are used concurrently and race/both wait
> for the hypervisor?
> 
> Maybe we need a separate vb->acked + callback function.
> 
> > +}
> > +
> >  static void set_page_pfns(struct virtio_balloon *vb,
> >  			  __virtio32 pfns[], struct page *page)
> >  {
> > @@ -476,6 +511,7 @@ static int init_vqs(struct virtio_balloon *vb)
> >  	names[VIRTIO_BALLOON_VQ_DEFLATE] = "deflate";
> >  	names[VIRTIO_BALLOON_VQ_STATS] = NULL;
> >  	names[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL;
> > +	names[VIRTIO_BALLOON_VQ_REPORTING] = NULL;
> >  
> >  	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
> >  		names[VIRTIO_BALLOON_VQ_STATS] = "stats";
> > @@ -487,11 +523,19 @@ static int init_vqs(struct virtio_balloon *vb)
> >  		callbacks[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL;
> >  	}
> >  
> > +	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
> > +		names[VIRTIO_BALLOON_VQ_REPORTING] = "reporting_vq";
> > +		callbacks[VIRTIO_BALLOON_VQ_REPORTING] = balloon_ack;
> > +	}
> > +
> >  	err = vb->vdev->config->find_vqs(vb->vdev, VIRTIO_BALLOON_VQ_MAX,
> >  					 vqs, callbacks, names, NULL, NULL);
> >  	if (err)
> >  		return err;
> >  
> > +	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
> > +		vb->reporting_vq = vqs[VIRTIO_BALLOON_VQ_REPORTING];
> > +
> 
> I'd register these in the same order they are defined (IOW, move this
> further down)
> 
> >  	vb->inflate_vq = vqs[VIRTIO_BALLOON_VQ_INFLATE];
> >  	vb->deflate_vq = vqs[VIRTIO_BALLOON_VQ_DEFLATE];
> >  	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
> > @@ -932,12 +976,30 @@ static int virtballoon_probe(struct virtio_device *vdev)
> >  		if (err)
> >  			goto out_del_balloon_wq;
> >  	}
> > +
> > +	vb->pr_dev_info.report = virtballoon_unused_page_report;
> > +	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
> > +		unsigned int capacity;
> > +
> > +		capacity = min_t(unsigned int,
> > +				 virtqueue_get_vring_size(vb->reporting_vq),
> > +				 VIRTIO_BALLOON_VRING_HINTS_MAX);
> > +		vb->pr_dev_info.capacity = capacity;
> > +
> > +		err = page_reporting_register(&vb->pr_dev_info);
> > +		if (err)
> > +			goto out_unregister_shrinker;
> > +	}
> 
> It can happen here that we start reporting before marking the device
> ready. Can that be problematic?
> 
> Maybe we have to ignore any reports in virtballoon_unused_page_report()
> until ready...
> 
> > +
> >  	virtio_device_ready(vdev);
> >  
> >  	if (towards_target(vb))
> >  		virtballoon_changed(vdev);
> >  	return 0;
> >  
> > +out_unregister_shrinker:
> > +	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
> > +		virtio_balloon_unregister_shrinker(vb);
> 
> A sync is done implicitly, right? So after this call, we won't get any
> new callbacks/are stuck in a callback.
> 
> >  out_del_balloon_wq:
> >  	if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_FREE_PAGE_HINT))
> >  		destroy_workqueue(vb->balloon_wq);
> > @@ -966,6 +1028,8 @@ static void virtballoon_remove(struct virtio_device *vdev)
> >  {
> >  	struct virtio_balloon *vb = vdev->priv;
> >  
> > +	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
> > +		page_reporting_unregister(&vb->pr_dev_info);
> 
> Dito, same question regarding syncs.
> 
> >  	if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
> >  		virtio_balloon_unregister_shrinker(vb);
> >  	spin_lock_irq(&vb->stop_update_lock);
> > @@ -1038,6 +1102,7 @@ static int virtballoon_validate(struct virtio_device *vdev)
> >  	VIRTIO_BALLOON_F_DEFLATE_ON_OOM,
> >  	VIRTIO_BALLOON_F_FREE_PAGE_HINT,
> >  	VIRTIO_BALLOON_F_PAGE_POISON,
> > +	VIRTIO_BALLOON_F_REPORTING,
> >  };
> >  
> >  static struct virtio_driver virtio_balloon_driver = {
> > diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virtio_balloon.h
> > index a1966cd7b677..19974392d324 100644
> > --- a/include/uapi/linux/virtio_balloon.h
> > +++ b/include/uapi/linux/virtio_balloon.h
> > @@ -36,6 +36,7 @@
> >  #define VIRTIO_BALLOON_F_DEFLATE_ON_OOM	2 /* Deflate balloon on OOM */
> >  #define VIRTIO_BALLOON_F_FREE_PAGE_HINT	3 /* VQ to report free pages */
> >  #define VIRTIO_BALLOON_F_PAGE_POISON	4 /* Guest is using page poisoning */
> > +#define VIRTIO_BALLOON_F_REPORTING	5 /* Page reporting virtqueue */
> >  
> >  /* Size of a PFN in the balloon interface. */
> >  #define VIRTIO_BALLOON_PFN_SHIFT 12
> > 
> > 
> 
> Small and powerful patch :)
> 
> -- 
> Thanks,
> 
> David / dhildenb

Re: [PATCH v14 3/6] mm: Introduce Reported pages

[Alexander Duyck]

On Thu, Nov 28, 2019 at 1:22 AM Mel Gorman <mgorman@techsingularity.net> wrote:
>
> On Wed, Nov 27, 2019 at 01:55:02PM -0800, Alexander Duyck wrote:

< snip >

> > >
> > > Which in itself could be an optimisation patch. Maybe it'll be enough that
> > > keeping track of the count is not worthwhile. Either way, the separate
> > > patch could have supporting data on how much it improves the speed of
> > > reporting pages so it can be compared to any other optimisation that
> > > may be proposed. Supporting data would also help make the case that any
> > > complexity introduced by the optimisation is worthwhile.
> >
> > I'll see what I can do to break this apart. I'm just not a fan of redoing
> > the work multiple times so that I can have a non-optimized version versus
> > an optimized one.
> >
>
> While I understand that, right now the optimisations are blocking the
> feature itself which is also not a situation you want to be in.

I'll see what I can do. I can probably replace the reference count and
zone flags with an atomic state in the prdev that cycles between
inactive, requested, and active. With that I can at least guarantee
that we shouldn't have any races that result in us missing freed
pages. The only downside is that I have to keep a boolean in the
__free_one_page call so that when we free back the reported pages we
don't retrigger the reporting and cause an infinite loop.

Re: [PATCH v14 6/6] virtio-balloon: Add support for providing unused page reports to host

[Alexander Duyck]

On Thu, Nov 28, 2019 at 7:26 AM David Hildenbrand <david@redhat.com> wrote:
>
> On 19.11.19 22:46, Alexander Duyck wrote:
> > From: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> >
> > Add support for the page reporting feature provided by virtio-balloon.
> > Reporting differs from the regular balloon functionality in that is is
> > much less durable than a standard memory balloon. Instead of creating a
> > list of pages that cannot be accessed the pages are only inaccessible
> > while they are being indicated to the virtio interface. Once the
> > interface has acknowledged them they are placed back into their respective
> > free lists and are once again accessible by the guest system.
>
> Maybe add something like "In contrast to ordinary balloon
> inflation/deflation, the guest can reuse all reported pages immediately
> after reporting has finished, without having to notify the hypervisor
> about it (e.g., VIRTIO_BALLOON_F_MUST_TELL_HOST does not apply)."

Okay. I'll make a note of it for next version.

> [...]
>
> >  /*
> >   * Balloon device works in 4K page units.  So each page is pointed to by
> > @@ -37,6 +38,9 @@
> >  #define VIRTIO_BALLOON_FREE_PAGE_SIZE \
> >       (1 << (VIRTIO_BALLOON_FREE_PAGE_ORDER + PAGE_SHIFT))
> >
> > +/*  limit on the number of pages that can be on the reporting vq */
> > +#define VIRTIO_BALLOON_VRING_HINTS_MAX       16
>
> Maybe rename that from HINTS to REPORTS

I'll fix it for the next version.

> > +
> >  #ifdef CONFIG_BALLOON_COMPACTION
> >  static struct vfsmount *balloon_mnt;
> >  #endif
> > @@ -46,6 +50,7 @@ enum virtio_balloon_vq {
> >       VIRTIO_BALLOON_VQ_DEFLATE,
> >       VIRTIO_BALLOON_VQ_STATS,
> >       VIRTIO_BALLOON_VQ_FREE_PAGE,
> > +     VIRTIO_BALLOON_VQ_REPORTING,
> >       VIRTIO_BALLOON_VQ_MAX
> >  };
> >
> > @@ -113,6 +118,10 @@ struct virtio_balloon {
> >
> >       /* To register a shrinker to shrink memory upon memory pressure */
> >       struct shrinker shrinker;
> > +
> > +     /* Unused page reporting device */
>
> Sounds like the device is unused :D
>
> "Device info for reporting unused pages" ?
>
> I am in general wondering, should we rename "unused" to "free". I.e.,
> "free page reporting" instead of "unused page reporting"? Or what was
> the motivation behind using "unused" ?

I honestly don't remember why I chose "unused" at this point. I can
switch over to "free" if that is what is preferred.

Looking over the code a bit more I suspect the reason for avoiding it
is because free page hinting also mentioned reporting in a few spots.

> > +     struct virtqueue *reporting_vq;
> > +     struct page_reporting_dev_info pr_dev_info;
> >  };
> >
> >  static struct virtio_device_id id_table[] = {
> > @@ -152,6 +161,32 @@ static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq)
> >
> >  }
> >
> > +void virtballoon_unused_page_report(struct page_reporting_dev_info *pr_dev_info,
> > +                                 unsigned int nents)
> > +{
> > +     struct virtio_balloon *vb =
> > +             container_of(pr_dev_info, struct virtio_balloon, pr_dev_info);
> > +     struct virtqueue *vq = vb->reporting_vq;
> > +     unsigned int unused, err;
> > +
> > +     /* We should always be able to add these buffers to an empty queue. */
>
> This comment somewhat contradicts the error handling (and comment)
> below. Maybe just drop it?
>
> > +     err = virtqueue_add_inbuf(vq, pr_dev_info->sg, nents, vb,
> > +                               GFP_NOWAIT | __GFP_NOWARN);
> > +
> > +     /*
> > +      * In the extremely unlikely case that something has changed and we
> > +      * are able to trigger an error we will simply display a warning
> > +      * and exit without actually processing the pages.
> > +      */
> > +     if (WARN_ON(err))
> > +             return;
>
> Maybe WARN_ON_ONCE? (to not flood the log on recurring errors)

Actually I might need to tweak things here a bit. It occurs to me that
this can fail for more than just there not being space in the ring. I
forgot that DMA mapping needs to also occur so in the case of a DMA
mapping failure we would also see an error.

I probably will switch it to a WARN_ON_ONCE. I may also need to add a
return value to the function so that we can indicate that an entire
batch has failed and that we need to abort.

> > +
> > +     virtqueue_kick(vq);
> > +
> > +     /* When host has read buffer, this completes via balloon_ack */
> > +     wait_event(vb->acked, virtqueue_get_buf(vq, &unused));
>
> Is it safe to rely on the same ack-ing mechanism as the inflate/deflate
> queue? What if both mechanisms are used concurrently and race/both wait
> for the hypervisor?
>
> Maybe we need a separate vb->acked + callback function.

So if I understand correctly what is actually happening is that the
wait event is simply a trigger that will wake us up, and at that point
we check to see if the buffer we submitted is done. If not we go back
to sleep. As such all we are really waiting on is the notification
that the buffers we submitted have been processed. So it is using the
same function but on a different virtual queue.

> > +}
> > +
> >  static void set_page_pfns(struct virtio_balloon *vb,
> >                         __virtio32 pfns[], struct page *page)
> >  {
> > @@ -476,6 +511,7 @@ static int init_vqs(struct virtio_balloon *vb)
> >       names[VIRTIO_BALLOON_VQ_DEFLATE] = "deflate";
> >       names[VIRTIO_BALLOON_VQ_STATS] = NULL;
> >       names[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL;
> > +     names[VIRTIO_BALLOON_VQ_REPORTING] = NULL;
> >
> >       if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
> >               names[VIRTIO_BALLOON_VQ_STATS] = "stats";
> > @@ -487,11 +523,19 @@ static int init_vqs(struct virtio_balloon *vb)
> >               callbacks[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL;
> >       }
> >
> > +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
> > +             names[VIRTIO_BALLOON_VQ_REPORTING] = "reporting_vq";
> > +             callbacks[VIRTIO_BALLOON_VQ_REPORTING] = balloon_ack;
> > +     }
> > +
> >       err = vb->vdev->config->find_vqs(vb->vdev, VIRTIO_BALLOON_VQ_MAX,
> >                                        vqs, callbacks, names, NULL, NULL);
> >       if (err)
> >               return err;
> >
> > +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
> > +             vb->reporting_vq = vqs[VIRTIO_BALLOON_VQ_REPORTING];
> > +
>
> I'd register these in the same order they are defined (IOW, move this
> further down)

done.

> >       vb->inflate_vq = vqs[VIRTIO_BALLOON_VQ_INFLATE];
> >       vb->deflate_vq = vqs[VIRTIO_BALLOON_VQ_DEFLATE];
> >       if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
> > @@ -932,12 +976,30 @@ static int virtballoon_probe(struct virtio_device *vdev)
> >               if (err)
> >                       goto out_del_balloon_wq;
> >       }
> > +
> > +     vb->pr_dev_info.report = virtballoon_unused_page_report;
> > +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
> > +             unsigned int capacity;
> > +
> > +             capacity = min_t(unsigned int,
> > +                              virtqueue_get_vring_size(vb->reporting_vq),
> > +                              VIRTIO_BALLOON_VRING_HINTS_MAX);
> > +             vb->pr_dev_info.capacity = capacity;
> > +
> > +             err = page_reporting_register(&vb->pr_dev_info);
> > +             if (err)
> > +                     goto out_unregister_shrinker;
> > +     }
>
> It can happen here that we start reporting before marking the device
> ready. Can that be problematic?
>
> Maybe we have to ignore any reports in virtballoon_unused_page_report()
> until ready...

I don't think there is an issue with us putting buffers on the ring
before it is ready. I think it will just cause our function to sleep.

I'm guessing that is the case since init_vqs will add a buffer to the
stats vq and that happens even earlier in virtballoon_probe.

> > +
> >       virtio_device_ready(vdev);
> >
> >       if (towards_target(vb))
> >               virtballoon_changed(vdev);
> >       return 0;
> >
> > +out_unregister_shrinker:
> > +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
> > +             virtio_balloon_unregister_shrinker(vb);
>
> A sync is done implicitly, right? So after this call, we won't get any
> new callbacks/are stuck in a callback.

From what I can tell a read/write semaphore is used in
unregister_shrinker when we delete it from the list so it shouldn't be
an issue.

> >  out_del_balloon_wq:
> >       if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_FREE_PAGE_HINT))
> >               destroy_workqueue(vb->balloon_wq);
> > @@ -966,6 +1028,8 @@ static void virtballoon_remove(struct virtio_device *vdev)
> >  {
> >       struct virtio_balloon *vb = vdev->priv;
> >
> > +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
> > +             page_reporting_unregister(&vb->pr_dev_info);
>
> Dito, same question regarding syncs.

Yes, although for that one I was using pointer deletion, a barrier,
and a cancel_work_sync since I didn't support a list.

> >       if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
> >               virtio_balloon_unregister_shrinker(vb);
> >       spin_lock_irq(&vb->stop_update_lock);
> > @@ -1038,6 +1102,7 @@ static int virtballoon_validate(struct virtio_device *vdev)
> >       VIRTIO_BALLOON_F_DEFLATE_ON_OOM,
> >       VIRTIO_BALLOON_F_FREE_PAGE_HINT,
> >       VIRTIO_BALLOON_F_PAGE_POISON,
> > +     VIRTIO_BALLOON_F_REPORTING,
> >  };
> >
> >  static struct virtio_driver virtio_balloon_driver = {
> > diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virtio_balloon.h
> > index a1966cd7b677..19974392d324 100644
> > --- a/include/uapi/linux/virtio_balloon.h
> > +++ b/include/uapi/linux/virtio_balloon.h
> > @@ -36,6 +36,7 @@
> >  #define VIRTIO_BALLOON_F_DEFLATE_ON_OOM      2 /* Deflate balloon on OOM */
> >  #define VIRTIO_BALLOON_F_FREE_PAGE_HINT      3 /* VQ to report free pages */
> >  #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */
> > +#define VIRTIO_BALLOON_F_REPORTING   5 /* Page reporting virtqueue */
> >
> >  /* Size of a PFN in the balloon interface. */
> >  #define VIRTIO_BALLOON_PFN_SHIFT 12
> >
> >
>
> Small and powerful patch :)

Agreed. Although we will have to see if we can keep it that way.
Ideally I want to leave this with the ability so specify what size
scatterlist we receive. However if we have to flip it around then it
will force us to add logic for chopping up the scatterlist for
processing in chunks.

Thanks for the review.

- Alex

Re: [PATCH v14 6/6] virtio-balloon: Add support for providing unused page reports to host

[Michael S. Tsirkin]

On Fri, Nov 29, 2019 at 01:13:32PM -0800, Alexander Duyck wrote:
> On Thu, Nov 28, 2019 at 7:26 AM David Hildenbrand <david@redhat.com> wrote:
> >
> > On 19.11.19 22:46, Alexander Duyck wrote:
> > > From: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> > >
> > > Add support for the page reporting feature provided by virtio-balloon.
> > > Reporting differs from the regular balloon functionality in that is is
> > > much less durable than a standard memory balloon. Instead of creating a
> > > list of pages that cannot be accessed the pages are only inaccessible
> > > while they are being indicated to the virtio interface. Once the
> > > interface has acknowledged them they are placed back into their respective
> > > free lists and are once again accessible by the guest system.
> >
> > Maybe add something like "In contrast to ordinary balloon
> > inflation/deflation, the guest can reuse all reported pages immediately
> > after reporting has finished, without having to notify the hypervisor
> > about it (e.g., VIRTIO_BALLOON_F_MUST_TELL_HOST does not apply)."
> 
> Okay. I'll make a note of it for next version.


VIRTIO_BALLOON_F_MUST_TELL_HOST is IMHO misdocumented.
It states:
	VIRTIO_BALLOON_F_MUST_TELL_HOST (0) Host has to be told before pages from the balloon are
	used.
but really balloon always told host. The difference is in timing,
historically balloon gave up pages before sending the
message and before waiting for the buffer to be used by host.

I think this feature can be the same if we want.


> > [...]
> >
> > >  /*
> > >   * Balloon device works in 4K page units.  So each page is pointed to by
> > > @@ -37,6 +38,9 @@
> > >  #define VIRTIO_BALLOON_FREE_PAGE_SIZE \
> > >       (1 << (VIRTIO_BALLOON_FREE_PAGE_ORDER + PAGE_SHIFT))
> > >
> > > +/*  limit on the number of pages that can be on the reporting vq */
> > > +#define VIRTIO_BALLOON_VRING_HINTS_MAX       16
> >
> > Maybe rename that from HINTS to REPORTS
> 
> I'll fix it for the next version.
> 
> > > +
> > >  #ifdef CONFIG_BALLOON_COMPACTION
> > >  static struct vfsmount *balloon_mnt;
> > >  #endif
> > > @@ -46,6 +50,7 @@ enum virtio_balloon_vq {
> > >       VIRTIO_BALLOON_VQ_DEFLATE,
> > >       VIRTIO_BALLOON_VQ_STATS,
> > >       VIRTIO_BALLOON_VQ_FREE_PAGE,
> > > +     VIRTIO_BALLOON_VQ_REPORTING,
> > >       VIRTIO_BALLOON_VQ_MAX
> > >  };
> > >
> > > @@ -113,6 +118,10 @@ struct virtio_balloon {
> > >
> > >       /* To register a shrinker to shrink memory upon memory pressure */
> > >       struct shrinker shrinker;
> > > +
> > > +     /* Unused page reporting device */
> >
> > Sounds like the device is unused :D
> >
> > "Device info for reporting unused pages" ?
> >
> > I am in general wondering, should we rename "unused" to "free". I.e.,
> > "free page reporting" instead of "unused page reporting"? Or what was
> > the motivation behind using "unused" ?
> 
> I honestly don't remember why I chose "unused" at this point. I can
> switch over to "free" if that is what is preferred.
> 
> Looking over the code a bit more I suspect the reason for avoiding it
> is because free page hinting also mentioned reporting in a few spots.
> 
> > > +     struct virtqueue *reporting_vq;
> > > +     struct page_reporting_dev_info pr_dev_info;
> > >  };
> > >
> > >  static struct virtio_device_id id_table[] = {
> > > @@ -152,6 +161,32 @@ static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq)
> > >
> > >  }
> > >
> > > +void virtballoon_unused_page_report(struct page_reporting_dev_info *pr_dev_info,
> > > +                                 unsigned int nents)
> > > +{
> > > +     struct virtio_balloon *vb =
> > > +             container_of(pr_dev_info, struct virtio_balloon, pr_dev_info);
> > > +     struct virtqueue *vq = vb->reporting_vq;
> > > +     unsigned int unused, err;
> > > +
> > > +     /* We should always be able to add these buffers to an empty queue. */
> >
> > This comment somewhat contradicts the error handling (and comment)
> > below. Maybe just drop it?
> >
> > > +     err = virtqueue_add_inbuf(vq, pr_dev_info->sg, nents, vb,
> > > +                               GFP_NOWAIT | __GFP_NOWARN);
> > > +
> > > +     /*
> > > +      * In the extremely unlikely case that something has changed and we
> > > +      * are able to trigger an error we will simply display a warning
> > > +      * and exit without actually processing the pages.
> > > +      */
> > > +     if (WARN_ON(err))
> > > +             return;
> >
> > Maybe WARN_ON_ONCE? (to not flood the log on recurring errors)
> 
> Actually I might need to tweak things here a bit. It occurs to me that
> this can fail for more than just there not being space in the ring. I
> forgot that DMA mapping needs to also occur so in the case of a DMA
> mapping failure we would also see an error.

Balloon assumes DMA mapping is bypassed right now:

static int virtballoon_validate(struct virtio_device *vdev)
{
        if (!page_poisoning_enabled())
                __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_PAGE_POISON);

        __virtio_clear_bit(vdev, VIRTIO_F_IOMMU_PLATFORM);

^^^^^^^^


        return 0;
}

I don't think it can work with things like a bounce buffer.

> I probably will switch it to a WARN_ON_ONCE. I may also need to add a
> return value to the function so that we can indicate that an entire
> batch has failed and that we need to abort.
> 
> > > +
> > > +     virtqueue_kick(vq);
> > > +
> > > +     /* When host has read buffer, this completes via balloon_ack */
> > > +     wait_event(vb->acked, virtqueue_get_buf(vq, &unused));
> >
> > Is it safe to rely on the same ack-ing mechanism as the inflate/deflate
> > queue? What if both mechanisms are used concurrently and race/both wait
> > for the hypervisor?
> >
> > Maybe we need a separate vb->acked + callback function.
> 
> So if I understand correctly what is actually happening is that the
> wait event is simply a trigger that will wake us up, and at that point
> we check to see if the buffer we submitted is done. If not we go back
> to sleep. As such all we are really waiting on is the notification
> that the buffers we submitted have been processed. So it is using the
> same function but on a different virtual queue.
> 
> > > +}
> > > +
> > >  static void set_page_pfns(struct virtio_balloon *vb,
> > >                         __virtio32 pfns[], struct page *page)
> > >  {
> > > @@ -476,6 +511,7 @@ static int init_vqs(struct virtio_balloon *vb)
> > >       names[VIRTIO_BALLOON_VQ_DEFLATE] = "deflate";
> > >       names[VIRTIO_BALLOON_VQ_STATS] = NULL;
> > >       names[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL;
> > > +     names[VIRTIO_BALLOON_VQ_REPORTING] = NULL;
> > >
> > >       if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
> > >               names[VIRTIO_BALLOON_VQ_STATS] = "stats";
> > > @@ -487,11 +523,19 @@ static int init_vqs(struct virtio_balloon *vb)
> > >               callbacks[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL;
> > >       }
> > >
> > > +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
> > > +             names[VIRTIO_BALLOON_VQ_REPORTING] = "reporting_vq";
> > > +             callbacks[VIRTIO_BALLOON_VQ_REPORTING] = balloon_ack;
> > > +     }
> > > +
> > >       err = vb->vdev->config->find_vqs(vb->vdev, VIRTIO_BALLOON_VQ_MAX,
> > >                                        vqs, callbacks, names, NULL, NULL);
> > >       if (err)
> > >               return err;
> > >
> > > +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
> > > +             vb->reporting_vq = vqs[VIRTIO_BALLOON_VQ_REPORTING];
> > > +
> >
> > I'd register these in the same order they are defined (IOW, move this
> > further down)
> 
> done.
> 
> > >       vb->inflate_vq = vqs[VIRTIO_BALLOON_VQ_INFLATE];
> > >       vb->deflate_vq = vqs[VIRTIO_BALLOON_VQ_DEFLATE];
> > >       if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
> > > @@ -932,12 +976,30 @@ static int virtballoon_probe(struct virtio_device *vdev)
> > >               if (err)
> > >                       goto out_del_balloon_wq;
> > >       }
> > > +
> > > +     vb->pr_dev_info.report = virtballoon_unused_page_report;
> > > +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
> > > +             unsigned int capacity;
> > > +
> > > +             capacity = min_t(unsigned int,
> > > +                              virtqueue_get_vring_size(vb->reporting_vq),
> > > +                              VIRTIO_BALLOON_VRING_HINTS_MAX);
> > > +             vb->pr_dev_info.capacity = capacity;
> > > +
> > > +             err = page_reporting_register(&vb->pr_dev_info);
> > > +             if (err)
> > > +                     goto out_unregister_shrinker;
> > > +     }
> >
> > It can happen here that we start reporting before marking the device
> > ready. Can that be problematic?
> >
> > Maybe we have to ignore any reports in virtballoon_unused_page_report()
> > until ready...
> 
> I don't think there is an issue with us putting buffers on the ring
> before it is ready. I think it will just cause our function to sleep.
> 
> I'm guessing that is the case since init_vqs will add a buffer to the
> stats vq and that happens even earlier in virtballoon_probe.
> 
> > > +
> > >       virtio_device_ready(vdev);
> > >
> > >       if (towards_target(vb))
> > >               virtballoon_changed(vdev);
> > >       return 0;
> > >
> > > +out_unregister_shrinker:
> > > +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
> > > +             virtio_balloon_unregister_shrinker(vb);
> >
> > A sync is done implicitly, right? So after this call, we won't get any
> > new callbacks/are stuck in a callback.
> 
> >From what I can tell a read/write semaphore is used in
> unregister_shrinker when we delete it from the list so it shouldn't be
> an issue.
> 
> > >  out_del_balloon_wq:
> > >       if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_FREE_PAGE_HINT))
> > >               destroy_workqueue(vb->balloon_wq);
> > > @@ -966,6 +1028,8 @@ static void virtballoon_remove(struct virtio_device *vdev)
> > >  {
> > >       struct virtio_balloon *vb = vdev->priv;
> > >
> > > +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
> > > +             page_reporting_unregister(&vb->pr_dev_info);
> >
> > Dito, same question regarding syncs.
> 
> Yes, although for that one I was using pointer deletion, a barrier,
> and a cancel_work_sync since I didn't support a list.
> 
> > >       if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
> > >               virtio_balloon_unregister_shrinker(vb);
> > >       spin_lock_irq(&vb->stop_update_lock);
> > > @@ -1038,6 +1102,7 @@ static int virtballoon_validate(struct virtio_device *vdev)
> > >       VIRTIO_BALLOON_F_DEFLATE_ON_OOM,
> > >       VIRTIO_BALLOON_F_FREE_PAGE_HINT,
> > >       VIRTIO_BALLOON_F_PAGE_POISON,
> > > +     VIRTIO_BALLOON_F_REPORTING,
> > >  };
> > >
> > >  static struct virtio_driver virtio_balloon_driver = {
> > > diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virtio_balloon.h
> > > index a1966cd7b677..19974392d324 100644
> > > --- a/include/uapi/linux/virtio_balloon.h
> > > +++ b/include/uapi/linux/virtio_balloon.h
> > > @@ -36,6 +36,7 @@
> > >  #define VIRTIO_BALLOON_F_DEFLATE_ON_OOM      2 /* Deflate balloon on OOM */
> > >  #define VIRTIO_BALLOON_F_FREE_PAGE_HINT      3 /* VQ to report free pages */
> > >  #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */
> > > +#define VIRTIO_BALLOON_F_REPORTING   5 /* Page reporting virtqueue */
> > >
> > >  /* Size of a PFN in the balloon interface. */
> > >  #define VIRTIO_BALLOON_PFN_SHIFT 12
> > >
> > >
> >
> > Small and powerful patch :)
> 
> Agreed. Although we will have to see if we can keep it that way.
> Ideally I want to leave this with the ability so specify what size
> scatterlist we receive. However if we have to flip it around then it
> will force us to add logic for chopping up the scatterlist for
> processing in chunks.
> 
> Thanks for the review.
> 
> - Alex

Re: [PATCH v14 6/6] virtio-balloon: Add support for providing unused page reports to host

[Alexander Duyck]

On Sun, Dec 1, 2019 at 3:46 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Fri, Nov 29, 2019 at 01:13:32PM -0800, Alexander Duyck wrote:
> > On Thu, Nov 28, 2019 at 7:26 AM David Hildenbrand <david@redhat.com> wrote:
> > >
> > > On 19.11.19 22:46, Alexander Duyck wrote:
> > > > From: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> > > >
> > > > Add support for the page reporting feature provided by virtio-balloon.
> > > > Reporting differs from the regular balloon functionality in that is is
> > > > much less durable than a standard memory balloon. Instead of creating a
> > > > list of pages that cannot be accessed the pages are only inaccessible
> > > > while they are being indicated to the virtio interface. Once the
> > > > interface has acknowledged them they are placed back into their respective
> > > > free lists and are once again accessible by the guest system.
> > >
> > > Maybe add something like "In contrast to ordinary balloon
> > > inflation/deflation, the guest can reuse all reported pages immediately
> > > after reporting has finished, without having to notify the hypervisor
> > > about it (e.g., VIRTIO_BALLOON_F_MUST_TELL_HOST does not apply)."
> >
> > Okay. I'll make a note of it for next version.
>
>
> VIRTIO_BALLOON_F_MUST_TELL_HOST is IMHO misdocumented.
> It states:
>         VIRTIO_BALLOON_F_MUST_TELL_HOST (0) Host has to be told before pages from the balloon are
>         used.
> but really balloon always told host. The difference is in timing,
> historically balloon gave up pages before sending the
> message and before waiting for the buffer to be used by host.
>
> I think this feature can be the same if we want.

Okay. I'll still probably try to document the behavior a bit better though.

> > > [...]
> > >
> > > >  /*
> > > >   * Balloon device works in 4K page units.  So each page is pointed to by
> > > > @@ -37,6 +38,9 @@
> > > >  #define VIRTIO_BALLOON_FREE_PAGE_SIZE \
> > > >       (1 << (VIRTIO_BALLOON_FREE_PAGE_ORDER + PAGE_SHIFT))
> > > >
> > > > +/*  limit on the number of pages that can be on the reporting vq */
> > > > +#define VIRTIO_BALLOON_VRING_HINTS_MAX       16
> > >
> > > Maybe rename that from HINTS to REPORTS
> >
> > I'll fix it for the next version.
> >
> > > > +
> > > >  #ifdef CONFIG_BALLOON_COMPACTION
> > > >  static struct vfsmount *balloon_mnt;
> > > >  #endif
> > > > @@ -46,6 +50,7 @@ enum virtio_balloon_vq {
> > > >       VIRTIO_BALLOON_VQ_DEFLATE,
> > > >       VIRTIO_BALLOON_VQ_STATS,
> > > >       VIRTIO_BALLOON_VQ_FREE_PAGE,
> > > > +     VIRTIO_BALLOON_VQ_REPORTING,
> > > >       VIRTIO_BALLOON_VQ_MAX
> > > >  };
> > > >
> > > > @@ -113,6 +118,10 @@ struct virtio_balloon {
> > > >
> > > >       /* To register a shrinker to shrink memory upon memory pressure */
> > > >       struct shrinker shrinker;
> > > > +
> > > > +     /* Unused page reporting device */
> > >
> > > Sounds like the device is unused :D
> > >
> > > "Device info for reporting unused pages" ?
> > >
> > > I am in general wondering, should we rename "unused" to "free". I.e.,
> > > "free page reporting" instead of "unused page reporting"? Or what was
> > > the motivation behind using "unused" ?
> >
> > I honestly don't remember why I chose "unused" at this point. I can
> > switch over to "free" if that is what is preferred.
> >
> > Looking over the code a bit more I suspect the reason for avoiding it
> > is because free page hinting also mentioned reporting in a few spots.
> >
> > > > +     struct virtqueue *reporting_vq;
> > > > +     struct page_reporting_dev_info pr_dev_info;
> > > >  };
> > > >
> > > >  static struct virtio_device_id id_table[] = {
> > > > @@ -152,6 +161,32 @@ static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq)
> > > >
> > > >  }
> > > >
> > > > +void virtballoon_unused_page_report(struct page_reporting_dev_info *pr_dev_info,
> > > > +                                 unsigned int nents)
> > > > +{
> > > > +     struct virtio_balloon *vb =
> > > > +             container_of(pr_dev_info, struct virtio_balloon, pr_dev_info);
> > > > +     struct virtqueue *vq = vb->reporting_vq;
> > > > +     unsigned int unused, err;
> > > > +
> > > > +     /* We should always be able to add these buffers to an empty queue. */
> > >
> > > This comment somewhat contradicts the error handling (and comment)
> > > below. Maybe just drop it?
> > >
> > > > +     err = virtqueue_add_inbuf(vq, pr_dev_info->sg, nents, vb,
> > > > +                               GFP_NOWAIT | __GFP_NOWARN);
> > > > +
> > > > +     /*
> > > > +      * In the extremely unlikely case that something has changed and we
> > > > +      * are able to trigger an error we will simply display a warning
> > > > +      * and exit without actually processing the pages.
> > > > +      */
> > > > +     if (WARN_ON(err))
> > > > +             return;
> > >
> > > Maybe WARN_ON_ONCE? (to not flood the log on recurring errors)
> >
> > Actually I might need to tweak things here a bit. It occurs to me that
> > this can fail for more than just there not being space in the ring. I
> > forgot that DMA mapping needs to also occur so in the case of a DMA
> > mapping failure we would also see an error.
>
> Balloon assumes DMA mapping is bypassed right now:
>
> static int virtballoon_validate(struct virtio_device *vdev)
> {
>         if (!page_poisoning_enabled())
>                 __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_PAGE_POISON);
>
>         __virtio_clear_bit(vdev, VIRTIO_F_IOMMU_PLATFORM);
>
> ^^^^^^^^
>
>
>         return 0;
> }
>
> I don't think it can work with things like a bounce buffer.

Right. It wouldn't work with a bounce buffer. I was thinking more of
something like an IOMMU. So it sounds like the device is doing direct
map always anyway.

In any case I will add some logic so that if we encounter an error we
will just abort the reporting. That way if another user has some issue
like that it can be dealt with sooner and we can avoid flagging pages
as reported that are not.

- Alex

Re: [PATCH v14 6/6] virtio-balloon: Add support for providing unused page reports to host

[David Hildenbrand]

[...]

>> Sounds like the device is unused :D
>>
>> "Device info for reporting unused pages" ?
>>
>> I am in general wondering, should we rename "unused" to "free". I.e.,
>> "free page reporting" instead of "unused page reporting"? Or what was
>> the motivation behind using "unused" ?
> 
> I honestly don't remember why I chose "unused" at this point. I can
> switch over to "free" if that is what is preferred.
> 
> Looking over the code a bit more I suspect the reason for avoiding it
> is because free page hinting also mentioned reporting in a few spots.

Maybe we should fix these cases. FWIW, I'd prefer "free page reporting".
(e.g., pairs nicely with "free page hinting").

>>> +
>>> +     virtqueue_kick(vq);
>>> +
>>> +     /* When host has read buffer, this completes via balloon_ack */
>>> +     wait_event(vb->acked, virtqueue_get_buf(vq, &unused));
>>
>> Is it safe to rely on the same ack-ing mechanism as the inflate/deflate
>> queue? What if both mechanisms are used concurrently and race/both wait
>> for the hypervisor?
>>
>> Maybe we need a separate vb->acked + callback function.
> 
> So if I understand correctly what is actually happening is that the
> wait event is simply a trigger that will wake us up, and at that point
> we check to see if the buffer we submitted is done. If not we go back
> to sleep. As such all we are really waiting on is the notification
> that the buffers we submitted have been processed. So it is using the
> same function but on a different virtual queue.

Very right, this is just a waitqueue (was only looking at this patch,
not the full code). This should indeed be fine.

>>>       vb->inflate_vq = vqs[VIRTIO_BALLOON_VQ_INFLATE];
>>>       vb->deflate_vq = vqs[VIRTIO_BALLOON_VQ_DEFLATE];
>>>       if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
>>> @@ -932,12 +976,30 @@ static int virtballoon_probe(struct virtio_device *vdev)
>>>               if (err)
>>>                       goto out_del_balloon_wq;
>>>       }
>>> +
>>> +     vb->pr_dev_info.report = virtballoon_unused_page_report;
>>> +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
>>> +             unsigned int capacity;
>>> +
>>> +             capacity = min_t(unsigned int,
>>> +                              virtqueue_get_vring_size(vb->reporting_vq),
>>> +                              VIRTIO_BALLOON_VRING_HINTS_MAX);
>>> +             vb->pr_dev_info.capacity = capacity;
>>> +
>>> +             err = page_reporting_register(&vb->pr_dev_info);
>>> +             if (err)
>>> +                     goto out_unregister_shrinker;
>>> +     }
>>
>> It can happen here that we start reporting before marking the device
>> ready. Can that be problematic?
>>
>> Maybe we have to ignore any reports in virtballoon_unused_page_report()
>> until ready...
> 
> I don't think there is an issue with us putting buffers on the ring
> before it is ready. I think it will just cause our function to sleep.
> 
> I'm guessing that is the case since init_vqs will add a buffer to the
> stats vq and that happens even earlier in virtballoon_probe.
> 

Interesting: "Note: vqs are enabled automatically after probe returns.".
Learned something new.

The virtballoon_changed(vdev) *after* virtio_device_ready(vdev) made me
wonder, because that could also fill the queues.

Maybe Michael can clarify.

>>> +
>>>       virtio_device_ready(vdev);
>>>
>>>       if (towards_target(vb))
>>>               virtballoon_changed(vdev);
>>>       return 0;
>>>
>>> +out_unregister_shrinker:
>>> +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
>>> +             virtio_balloon_unregister_shrinker(vb);
>>
>> A sync is done implicitly, right? So after this call, we won't get any
>> new callbacks/are stuck in a callback.
> 
> From what I can tell a read/write semaphore is used in
> unregister_shrinker when we delete it from the list so it shouldn't be
> an issue.

Yes, makes sense.

> 
>>>  out_del_balloon_wq:
>>>       if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_FREE_PAGE_HINT))
>>>               destroy_workqueue(vb->balloon_wq);
>>> @@ -966,6 +1028,8 @@ static void virtballoon_remove(struct virtio_device *vdev)
>>>  {
>>>       struct virtio_balloon *vb = vdev->priv;
>>>
>>> +     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
>>> +             page_reporting_unregister(&vb->pr_dev_info);
>>
>> Dito, same question regarding syncs.
> 
> Yes, although for that one I was using pointer deletion, a barrier,
> and a cancel_work_sync since I didn't support a list.

Okay, perfect.

[...]
>>
>> Small and powerful patch :)
> 
> Agreed. Although we will have to see if we can keep it that way.
> Ideally I want to leave this with the ability so specify what size
> scatterlist we receive. However if we have to flip it around then it
> will force us to add logic for chopping up the scatterlist for
> processing in chunks.

I hope we can keep it like that. Otherwise each and every driver has to
implement this chopping-up (e.g., a hypervisor that can only send one
hint at a time - e.g., via  a simple hypercall - would have to implement
that).


-- 
Thanks,

David / dhildenb

Re: [PATCH v14 6/6] virtio-balloon: Add support for providing unused page reports to host

[Alexander Duyck]

On Thu, Nov 28, 2019 at 9:00 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Nov 28, 2019 at 04:25:54PM +0100, David Hildenbrand wrote:
> > On 19.11.19 22:46, Alexander Duyck wrote:
> > > From: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> > >
> > > Add support for the page reporting feature provided by virtio-balloon.
> > > Reporting differs from the regular balloon functionality in that is is
> > > much less durable than a standard memory balloon. Instead of creating a
> > > list of pages that cannot be accessed the pages are only inaccessible
> > > while they are being indicated to the virtio interface. Once the
> > > interface has acknowledged them they are placed back into their respective
> > > free lists and are once again accessible by the guest system.
> >
> > Maybe add something like "In contrast to ordinary balloon
> > inflation/deflation, the guest can reuse all reported pages immediately
> > after reporting has finished, without having to notify the hypervisor
> > about it (e.g., VIRTIO_BALLOON_F_MUST_TELL_HOST does not apply)."
>
> Maybe we can make apply. The effect of reporting a page is effectively
> putting it in a balloon then immediately taking it out. Maybe without
> VIRTIO_BALLOON_F_MUST_TELL_HOST the pages can be reused before host
> marked buffers used?
>
> We didn't teach existing page hinting to behave like this, but maybe we
> should, and maybe it's not too late, not a long time passed
> since it was merged, and the whole shrinker based thing
> seems to have been broken ...
>
>
> BTW generally UAPI patches will have to be sent to virtio-dev
> mailing list before they are merged.
>
> > [...]
> >
> > >  /*
> > >   * Balloon device works in 4K page units.  So each page is pointed to by
> > > @@ -37,6 +38,9 @@
> > >  #define VIRTIO_BALLOON_FREE_PAGE_SIZE \
> > >     (1 << (VIRTIO_BALLOON_FREE_PAGE_ORDER + PAGE_SHIFT))
> > >
> > > +/*  limit on the number of pages that can be on the reporting vq */
> > > +#define VIRTIO_BALLOON_VRING_HINTS_MAX     16
> >
> > Maybe rename that from HINTS to REPORTS
> >
> > > +
> > >  #ifdef CONFIG_BALLOON_COMPACTION
> > >  static struct vfsmount *balloon_mnt;
> > >  #endif
> > > @@ -46,6 +50,7 @@ enum virtio_balloon_vq {
> > >     VIRTIO_BALLOON_VQ_DEFLATE,
> > >     VIRTIO_BALLOON_VQ_STATS,
> > >     VIRTIO_BALLOON_VQ_FREE_PAGE,
> > > +   VIRTIO_BALLOON_VQ_REPORTING,
> > >     VIRTIO_BALLOON_VQ_MAX
> > >  };
> > >
> > > @@ -113,6 +118,10 @@ struct virtio_balloon {
> > >
> > >     /* To register a shrinker to shrink memory upon memory pressure */
> > >     struct shrinker shrinker;
> > > +
> > > +   /* Unused page reporting device */
> >
> > Sounds like the device is unused :D
> >
> > "Device info for reporting unused pages" ?
> >
> > I am in general wondering, should we rename "unused" to "free". I.e.,
> > "free page reporting" instead of "unused page reporting"? Or what was
> > the motivation behind using "unused" ?
> >
> > > +   struct virtqueue *reporting_vq;
> > > +   struct page_reporting_dev_info pr_dev_info;
> > >  };
> > >
> > >  static struct virtio_device_id id_table[] = {
> > > @@ -152,6 +161,32 @@ static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq)
> > >
> > >  }
> > >
> > > +void virtballoon_unused_page_report(struct page_reporting_dev_info *pr_dev_info,
> > > +                               unsigned int nents)
> > > +{
> > > +   struct virtio_balloon *vb =
> > > +           container_of(pr_dev_info, struct virtio_balloon, pr_dev_info);
> > > +   struct virtqueue *vq = vb->reporting_vq;
> > > +   unsigned int unused, err;
> > > +
> > > +   /* We should always be able to add these buffers to an empty queue. */
> >
> > This comment somewhat contradicts the error handling (and comment)
> > below. Maybe just drop it?
> >
> > > +   err = virtqueue_add_inbuf(vq, pr_dev_info->sg, nents, vb,
> > > +                             GFP_NOWAIT | __GFP_NOWARN);
> > > +
> > > +   /*
> > > +    * In the extremely unlikely case that something has changed and we
> > > +    * are able to trigger an error we will simply display a warning
> > > +    * and exit without actually processing the pages.
> > > +    */
> > > +   if (WARN_ON(err))
> > > +           return;
> >
> > Maybe WARN_ON_ONCE? (to not flood the log on recurring errors)
> >
> > > +
> > > +   virtqueue_kick(vq);
> > > +
> > > +   /* When host has read buffer, this completes via balloon_ack */
> > > +   wait_event(vb->acked, virtqueue_get_buf(vq, &unused));
> >
> > Is it safe to rely on the same ack-ing mechanism as the inflate/deflate
> > queue? What if both mechanisms are used concurrently and race/both wait
> > for the hypervisor?
> >
> > Maybe we need a separate vb->acked + callback function.
> >
> > > +}
> > > +
> > >  static void set_page_pfns(struct virtio_balloon *vb,
> > >                       __virtio32 pfns[], struct page *page)
> > >  {
> > > @@ -476,6 +511,7 @@ static int init_vqs(struct virtio_balloon *vb)
> > >     names[VIRTIO_BALLOON_VQ_DEFLATE] = "deflate";
> > >     names[VIRTIO_BALLOON_VQ_STATS] = NULL;
> > >     names[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL;
> > > +   names[VIRTIO_BALLOON_VQ_REPORTING] = NULL;
> > >
> > >     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
> > >             names[VIRTIO_BALLOON_VQ_STATS] = "stats";
> > > @@ -487,11 +523,19 @@ static int init_vqs(struct virtio_balloon *vb)
> > >             callbacks[VIRTIO_BALLOON_VQ_FREE_PAGE] = NULL;
> > >     }
> > >
> > > +   if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
> > > +           names[VIRTIO_BALLOON_VQ_REPORTING] = "reporting_vq";
> > > +           callbacks[VIRTIO_BALLOON_VQ_REPORTING] = balloon_ack;
> > > +   }
> > > +
> > >     err = vb->vdev->config->find_vqs(vb->vdev, VIRTIO_BALLOON_VQ_MAX,
> > >                                      vqs, callbacks, names, NULL, NULL);
> > >     if (err)
> > >             return err;
> > >
> > > +   if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
> > > +           vb->reporting_vq = vqs[VIRTIO_BALLOON_VQ_REPORTING];
> > > +
> >
> > I'd register these in the same order they are defined (IOW, move this
> > further down)
> >
> > >     vb->inflate_vq = vqs[VIRTIO_BALLOON_VQ_INFLATE];
> > >     vb->deflate_vq = vqs[VIRTIO_BALLOON_VQ_DEFLATE];
> > >     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
> > > @@ -932,12 +976,30 @@ static int virtballoon_probe(struct virtio_device *vdev)
> > >             if (err)
> > >                     goto out_del_balloon_wq;
> > >     }
> > > +
> > > +   vb->pr_dev_info.report = virtballoon_unused_page_report;
> > > +   if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) {
> > > +           unsigned int capacity;
> > > +
> > > +           capacity = min_t(unsigned int,
> > > +                            virtqueue_get_vring_size(vb->reporting_vq),
> > > +                            VIRTIO_BALLOON_VRING_HINTS_MAX);
> > > +           vb->pr_dev_info.capacity = capacity;
> > > +
> > > +           err = page_reporting_register(&vb->pr_dev_info);
> > > +           if (err)
> > > +                   goto out_unregister_shrinker;
> > > +   }
> >
> > It can happen here that we start reporting before marking the device
> > ready. Can that be problematic?
> >
> > Maybe we have to ignore any reports in virtballoon_unused_page_report()
> > until ready...
> >
> > > +
> > >     virtio_device_ready(vdev);
> > >
> > >     if (towards_target(vb))
> > >             virtballoon_changed(vdev);
> > >     return 0;
> > >
> > > +out_unregister_shrinker:
> > > +   if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
> > > +           virtio_balloon_unregister_shrinker(vb);
> >
> > A sync is done implicitly, right? So after this call, we won't get any
> > new callbacks/are stuck in a callback.
> >
> > >  out_del_balloon_wq:
> > >     if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_FREE_PAGE_HINT))
> > >             destroy_workqueue(vb->balloon_wq);
> > > @@ -966,6 +1028,8 @@ static void virtballoon_remove(struct virtio_device *vdev)
> > >  {
> > >     struct virtio_balloon *vb = vdev->priv;
> > >
> > > +   if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING))
> > > +           page_reporting_unregister(&vb->pr_dev_info);
> >
> > Dito, same question regarding syncs.
> >
> > >     if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
> > >             virtio_balloon_unregister_shrinker(vb);
> > >     spin_lock_irq(&vb->stop_update_lock);
> > > @@ -1038,6 +1102,7 @@ static int virtballoon_validate(struct virtio_device *vdev)
> > >     VIRTIO_BALLOON_F_DEFLATE_ON_OOM,
> > >     VIRTIO_BALLOON_F_FREE_PAGE_HINT,
> > >     VIRTIO_BALLOON_F_PAGE_POISON,
> > > +   VIRTIO_BALLOON_F_REPORTING,
> > >  };
> > >
> > >  static struct virtio_driver virtio_balloon_driver = {
> > > diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virtio_balloon.h
> > > index a1966cd7b677..19974392d324 100644
> > > --- a/include/uapi/linux/virtio_balloon.h
> > > +++ b/include/uapi/linux/virtio_balloon.h
> > > @@ -36,6 +36,7 @@
> > >  #define VIRTIO_BALLOON_F_DEFLATE_ON_OOM    2 /* Deflate balloon on OOM */
> > >  #define VIRTIO_BALLOON_F_FREE_PAGE_HINT    3 /* VQ to report free pages */
> > >  #define VIRTIO_BALLOON_F_PAGE_POISON       4 /* Guest is using page poisoning */
> > > +#define VIRTIO_BALLOON_F_REPORTING 5 /* Page reporting virtqueue */
> > >
> > >  /* Size of a PFN in the balloon interface. */
> > >  #define VIRTIO_BALLOON_PFN_SHIFT 12
> > >
> > >
> >
> > Small and powerful patch :)
> >
> > --
> > Thanks,
> >
> > David / dhildenb
>
>

Re: [PATCH v14 6/6] virtio-balloon: Add support for providing unused page reports to host

[Alexander Duyck]

On Thu, Nov 28, 2019 at 9:00 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Nov 28, 2019 at 04:25:54PM +0100, David Hildenbrand wrote:
> > On 19.11.19 22:46, Alexander Duyck wrote:
> > > From: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> > >
> > > Add support for the page reporting feature provided by virtio-balloon.
> > > Reporting differs from the regular balloon functionality in that is is
> > > much less durable than a standard memory balloon. Instead of creating a
> > > list of pages that cannot be accessed the pages are only inaccessible
> > > while they are being indicated to the virtio interface. Once the
> > > interface has acknowledged them they are placed back into their respective
> > > free lists and are once again accessible by the guest system.
> >
> > Maybe add something like "In contrast to ordinary balloon
> > inflation/deflation, the guest can reuse all reported pages immediately
> > after reporting has finished, without having to notify the hypervisor
> > about it (e.g., VIRTIO_BALLOON_F_MUST_TELL_HOST does not apply)."
>
> Maybe we can make apply. The effect of reporting a page is effectively
> putting it in a balloon then immediately taking it out. Maybe without
> VIRTIO_BALLOON_F_MUST_TELL_HOST the pages can be reused before host
> marked buffers used?
>
> We didn't teach existing page hinting to behave like this, but maybe we
> should, and maybe it's not too late, not a long time passed
> since it was merged, and the whole shrinker based thing
> seems to have been broken ...

The problem is the existing hinting implementation relies on pushing
the memory to the point of OOM in order to avoid having to re-hint on
pages. What it is looking for is a snapshot rather than a running
tally. The page reporting bit approach would only work for the first
migration. The problem is the bit is persistent and would leave unused
pages flagged as reported if another migration starts so it wouldn't
re-report those pages.

> BTW generally UAPI patches will have to be sent to virtio-dev
> mailing list before they are merged.

Do you need just the QEMU patches submitted to virtio-dev or both the
virtio kernel patches and the QEMU patches?

One piece of feedback I got was that it was annoying that I was
including virtio-dev since it requires a subscription to send to it.
If you would like I could apply it on the QEMU patches which would
make the changes more visible at least.

Thanks.

- Alex