Re: [PATCH v2 13/19] mm/migrate: Use xchg instead of spinlock

From: Peter Zijlstra <peterz@infradead.org>
To: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Mel Gorman <mgorman@techsingularity.net>,
	Rik van Riel <riel@surriel.com>,
	Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [PATCH v2 13/19] mm/migrate: Use xchg instead of spinlock
Date: Mon, 23 Jul 2018 12:54:38 +0200	[thread overview]
Message-ID: <20180723105438.GD2494@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <1529514181-9842-14-git-send-email-srikar@linux.vnet.ibm.com>

On Wed, Jun 20, 2018 at 10:32:54PM +0530, Srikar Dronamraju wrote:
> Currently resetting the migrate rate limit is under a spinlock.
> The spinlock will only serialize the migrate rate limiting and something
> similar can actually be achieved by a simpler xchg.

You're again not explaining things right. The xchg isn't simpler in any
way. It just happens to be faster, esp. so on PPC.

> diff --git a/mm/migrate.c b/mm/migrate.c
> index 8c0af0f..c774990 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -1868,17 +1868,25 @@ static struct page *alloc_misplaced_dst_page(struct page *page,
>  static bool numamigrate_update_ratelimit(pg_data_t *pgdat,
>  					unsigned long nr_pages)
>  {
> +	unsigned long next_window, interval;
> +
> +	next_window = READ_ONCE(pgdat->numabalancing_migrate_next_window);
> +	interval = msecs_to_jiffies(migrate_interval_millisecs);
> +
>  	/*
>  	 * Rate-limit the amount of data that is being migrated to a node.
>  	 * Optimal placement is no good if the memory bus is saturated and
>  	 * all the time is being spent migrating!
>  	 */
> +	if (time_after(jiffies, next_window)) {
> +		if (xchg(&pgdat->numabalancing_migrate_nr_pages, 0)) {
> +			do {
> +				next_window += interval;
> +			} while (unlikely(time_after(jiffies, next_window)));
> +
> +			WRITE_ONCE(pgdat->numabalancing_migrate_next_window,
> +							       next_window);
> +		}
>  	}
>  	if (pgdat->numabalancing_migrate_nr_pages > ratelimit_pages) {
>  		trace_mm_numa_migrate_ratelimit(current, pgdat->node_id,

If you maybe write that like:

	if (time_after(jiffies, next_window) &&
	    xchg(&pgdat->numabalancing_migrate_nr_pages, 0UL)) {

		do {
			next_window += interval;
		} while (unlikely(time_after(jiffies, next_window)));

		WRITE_ONCE(pgdat->numabalancing_migrate_next_window, next_window);
	}

Then you avoid an indent level and line-wrap, resulting imo easier to
read code.