linux-arm-kernel.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
From: Alexander Duyck <alexander.duyck@gmail.com>
To: virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org,
	mst@redhat.com, catalin.marinas@arm.com, david@redhat.com,
	dave.hansen@intel.com, linux-kernel@vger.kernel.org,
	willy@infradead.org, mhocko@kernel.org, linux-mm@kvack.org,
	akpm@linux-foundation.org, will@kernel.org,
	linux-arm-kernel@lists.infradead.org, osalvador@suse.de
Cc: yang.zhang.wz@gmail.com, pagupta@redhat.com, riel@surriel.com,
	konrad.wilk@oracle.com, ying.huang@intel.com,
	lcapitulino@redhat.com, wei.w.wang@intel.com,
	aarcange@redhat.com, nitesh@redhat.com, pbonzini@redhat.com,
	dan.j.williams@intel.com, fengguang.wu@intel.com,
	alexander.h.duyck@linux.intel.com,
	kirill.shutemov@linux.intel.com
Subject: [PATCH v9 1/8] mm: Add per-cpu logic to page shuffling
Date: Sat, 07 Sep 2019 10:25:12 -0700	[thread overview]
Message-ID: <20190907172512.10910.74435.stgit@localhost.localdomain> (raw)
In-Reply-To: <20190907172225.10910.34302.stgit@localhost.localdomain>

From: Alexander Duyck <alexander.h.duyck@linux.intel.com>

Change the logic used to generate randomness in the suffle path so that we
can avoid cache line bouncing. The previous logic was sharing the offset
and entropy word between all CPUs. As such this can result in cache line
bouncing and will ultimately hurt performance when enabled.

To resolve this I have moved to a per-cpu logic for maintaining a unsigned
long containing some amount of bits, and an offset value for which bit we
can use for entropy with each call.

Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
---
 mm/shuffle.c |   33 +++++++++++++++++++++++----------
 1 file changed, 23 insertions(+), 10 deletions(-)

diff --git a/mm/shuffle.c b/mm/shuffle.c
index 3ce12481b1dc..9ba542ecf335 100644
--- a/mm/shuffle.c
+++ b/mm/shuffle.c
@@ -183,25 +183,38 @@ void __meminit __shuffle_free_memory(pg_data_t *pgdat)
 		shuffle_zone(z);
 }
 
+struct batched_bit_entropy {
+	unsigned long entropy_bool;
+	int position;
+};
+
+static DEFINE_PER_CPU(struct batched_bit_entropy, batched_entropy_bool);
+
 void add_to_free_area_random(struct page *page, struct free_area *area,
 		int migratetype)
 {
-	static u64 rand;
-	static u8 rand_bits;
+	struct batched_bit_entropy *batch;
+	unsigned long entropy;
+	int position;
 
 	/*
-	 * The lack of locking is deliberate. If 2 threads race to
-	 * update the rand state it just adds to the entropy.
+	 * We shouldn't need to disable IRQs as the only caller is
+	 * __free_one_page and it should only be called with the zone lock
+	 * held and either from IRQ context or with local IRQs disabled.
 	 */
-	if (rand_bits == 0) {
-		rand_bits = 64;
-		rand = get_random_u64();
+	batch = raw_cpu_ptr(&batched_entropy_bool);
+	position = batch->position;
+
+	if (--position < 0) {
+		batch->entropy_bool = get_random_long();
+		position = BITS_PER_LONG - 1;
 	}
 
-	if (rand & 1)
+	batch->position = position;
+	entropy = batch->entropy_bool;
+
+	if (1ul & (entropy >> position))
 		add_to_free_area(page, area, migratetype);
 	else
 		add_to_free_area_tail(page, area, migratetype);
-	rand_bits--;
-	rand >>= 1;
 }


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

  reply	other threads:[~2019-09-07 17:25 UTC|newest]

Thread overview: 82+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-07 17:25 [PATCH v9 0/8] stg mail -e --version=v9 \ Alexander Duyck
2019-09-07 17:25 ` Alexander Duyck [this message]
2019-09-09  8:14   ` [PATCH v9 1/8] mm: Add per-cpu logic to page shuffling David Hildenbrand
2019-09-09 15:11     ` Alexander Duyck
2019-09-10 12:11       ` Michal Hocko
2019-09-10 22:14         ` Alexander Duyck
2019-09-10 22:11     ` Alexander Duyck
2019-09-09  9:07   ` Kirill A. Shutemov
2019-09-09 15:12     ` Alexander Duyck
2019-09-07 17:25 ` [PATCH v9 2/8] mm: Adjust shuffle code to allow for future coalescing Alexander Duyck
2019-09-09  8:19   ` David Hildenbrand
2019-09-09  9:47   ` Kirill A. Shutemov
2019-09-09 15:22     ` Alexander Duyck
2019-09-09 15:35       ` Kirill A. Shutemov
2019-09-09 15:37         ` Alexander Duyck
2019-09-09 16:43     ` Alexander Duyck
2019-09-09 17:00       ` Kirill A. Shutemov
2019-09-10 12:20   ` Michal Hocko
2019-09-10 14:48     ` Alexander Duyck
2019-09-07 17:25 ` [PATCH v9 3/8] mm: Move set/get_pcppage_migratetype to mmzone.h Alexander Duyck
2019-09-09  8:22   ` David Hildenbrand
2019-09-09  9:56   ` Kirill A. Shutemov
2019-09-09 18:01     ` Alexander Duyck
2019-09-09 18:12       ` Alexander Duyck
2019-09-10 12:23   ` Michal Hocko
2019-09-10 14:46     ` Alexander Duyck
2019-09-10 17:45       ` Michal Hocko
2019-09-10 20:26         ` Alexander Duyck
2019-09-07 17:25 ` [PATCH v9 4/8] mm: Use zone and order instead of free area in free_list manipulators Alexander Duyck
2019-09-10 12:27   ` Michal Hocko
2019-09-07 17:25 ` [PATCH v9 5/8] arm64: Move hugetlb related definitions out of pgtable.h to page-defs.h Alexander Duyck
2019-09-09  8:52   ` David Hildenbrand
2019-09-09 15:27     ` Alexander Duyck
2019-09-17 17:48   ` Will Deacon
2019-09-17 20:07     ` Alexander Duyck
2019-09-07 17:25 ` [PATCH v9 6/8] mm: Introduce Reported pages Alexander Duyck
2019-09-09 14:42   ` Kirill A. Shutemov
2019-09-09 16:25     ` Alexander Duyck
2019-09-09 16:33       ` Kirill A. Shutemov
2019-09-07 17:26 ` [PATCH v9 7/8] virtio-balloon: Pull page poisoning config out of free page hinting Alexander Duyck
2019-09-09  8:59   ` David Hildenbrand
2019-09-09 15:31     ` Alexander Duyck
2019-09-07 17:26 ` [PATCH v9 8/8] virtio-balloon: Add support for providing unused page reports to host Alexander Duyck
2019-09-07 17:34 ` [PATCH v9 0/8] mm / virtio: Provide support for unused page reporting Alexander Duyck
2019-09-10 12:42 ` [PATCH v9 0/8] stg mail -e --version=v9 \ Michal Hocko
2019-09-10 14:42   ` Alexander Duyck
2019-09-10 14:47     ` Michal Hocko
2019-09-10 16:05       ` Alexander Duyck
2019-09-10 16:18         ` [virtio-dev] " Dr. David Alan Gilbert
2019-09-10 16:22           ` David Hildenbrand
2019-09-11  9:23             ` Michael S. Tsirkin
2019-09-11  9:50               ` David Hildenbrand
2019-09-10 17:52         ` Michal Hocko
2019-09-10 18:00           ` Michal Hocko
2019-09-10 20:37             ` Alexander Duyck
2019-09-10 21:23           ` Alexander Duyck
2019-09-11 11:36             ` Michal Hocko
2019-09-11 11:47               ` David Hildenbrand
2019-09-11 12:08               ` Michael S. Tsirkin
2019-09-11 12:19                 ` Michal Hocko
2019-09-11 12:25                   ` Michal Hocko
2019-09-11 12:42                     ` David Hildenbrand
2019-09-11 12:54                       ` Michal Hocko
2019-09-11 13:03                         ` David Hildenbrand
2019-09-11 13:20                           ` Michal Hocko
2019-09-11 13:51                             ` Michal Hocko
2019-09-11 16:09                               ` David Hildenbrand
2019-09-12  7:16                                 ` Michal Hocko
2019-09-12  7:47                                   ` David Hildenbrand
2019-09-12  9:26                                     ` Michal Hocko
2019-09-12 12:00                                     ` Nitesh Narayan Lal
2019-09-11 14:03                             ` Nitesh Narayan Lal
2019-09-11 16:02                             ` David Hildenbrand
2019-09-11 13:19                         ` Nitesh Narayan Lal
2019-09-11 12:55                       ` Nitesh Narayan Lal
2019-09-11 15:12               ` Alexander Duyck
2019-09-12  9:19                 ` Michal Hocko
2019-09-12 10:24                   ` Kirill A. Shutemov
2019-09-12 11:11                     ` Michal Hocko
2019-09-12 15:42                   ` Alexander Duyck
2019-09-12 16:35                   ` Mel Gorman
2019-09-12 17:48                     ` Alexander Duyck

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190907172512.10910.74435.stgit@localhost.localdomain \
    --to=alexander.duyck@gmail.com \
    --cc=aarcange@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=alexander.h.duyck@linux.intel.com \
    --cc=catalin.marinas@arm.com \
    --cc=dan.j.williams@intel.com \
    --cc=dave.hansen@intel.com \
    --cc=david@redhat.com \
    --cc=fengguang.wu@intel.com \
    --cc=kirill.shutemov@linux.intel.com \
    --cc=konrad.wilk@oracle.com \
    --cc=kvm@vger.kernel.org \
    --cc=lcapitulino@redhat.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@kernel.org \
    --cc=mst@redhat.com \
    --cc=nitesh@redhat.com \
    --cc=osalvador@suse.de \
    --cc=pagupta@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=riel@surriel.com \
    --cc=virtio-dev@lists.oasis-open.org \
    --cc=wei.w.wang@intel.com \
    --cc=will@kernel.org \
    --cc=willy@infradead.org \
    --cc=yang.zhang.wz@gmail.com \
    --cc=ying.huang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).