linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Michal Hocko <mhocko@suse.cz>
To: Tejun Heo <tj@kernel.org>
Cc: axboe@kernel.dk, linux-kernel@vger.kernel.org, jack@suse.cz,
	hch@infradead.org, hannes@cmpxchg.org,
	linux-fsdevel@vger.kernel.org, vgoyal@redhat.com,
	lizefan@huawei.com, cgroups@vger.kernel.org, linux-mm@kvack.org,
	clm@fb.com, fengguang.wu@intel.com, david@fromorbit.com,
	gthelen@google.com, khlebnikov@yandex-team.ru
Subject: Re: [PATCH 01/19] memcg: make mem_cgroup_read_{stat|event}() iterate possible cpus instead of online
Date: Wed, 17 Jun 2015 16:41:34 +0200	[thread overview]
Message-ID: <20150617144134.GH25056@dhcp22.suse.cz> (raw)
In-Reply-To: <1432333416-6221-2-git-send-email-tj@kernel.org>

On Fri 22-05-15 18:23:18, Tejun Heo wrote:
> cpu_possible_mask represents the CPUs which are actually possible
> during that boot instance.  For systems which don't support CPU
> hotplug, this will match cpu_online_mask exactly in most cases.  Even
> for systems which support CPU hotplug, the number of possible CPU
> slots is highly unlikely to diverge greatly from the number of online
> CPUs.  The only cases where the difference between possible and online
> caused problems were when the boot code failed to initialize the
> possible mask and left it fully set at NR_CPUS - 1.
> 
> As such, most per-cpu constructs allocate for all possible CPUs and
> often iterate over the possibles, which also has the benefit of
> avoiding the blocking CPU hotplug synchronization.
> 
> memcg open codes per-cpu stat counting for mem_cgroup_read_stat() and
> mem_cgroup_read_events(), which iterates over online CPUs and handles
> CPU hotplug operations explicitly.  This complexity doesn't actually
> buy anything.  Switch to iterating over the possibles and drop the
> explicit CPU hotplug handling.
> 
> Eventually, we want to convert memcg to use percpu_counter instead of
> its own custom implementation which also benefits from quick access
> w/o summing for cases where larger error margin is acceptable.
> 
> This will allow mem_cgroup_read_stat() to be called from non-sleepable
> contexts which will be used by cgroup writeback.
> 
> Signed-off-by: Tejun Heo <tj@kernel.org>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Michal Hocko <mhocko@suse.cz>

I am sorry for being late in this thread. 

I have seen systems where the number of possible CPUs was really high
wrt. online ones but I wouldn't worry about them. The change is an
overal improvement for usual configurations though.

Acked-by: Michal Hocko <mhocko@suse.cz>
> ---
>  mm/memcontrol.c | 51 ++-------------------------------------------------
>  1 file changed, 2 insertions(+), 49 deletions(-)
> 
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 6732c2c..d7d270a 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -324,11 +324,6 @@ struct mem_cgroup {
>  	 * percpu counter.
>  	 */
>  	struct mem_cgroup_stat_cpu __percpu *stat;
> -	/*
> -	 * used when a cpu is offlined or other synchronizations
> -	 * See mem_cgroup_read_stat().
> -	 */
> -	struct mem_cgroup_stat_cpu nocpu_base;
>  	spinlock_t pcp_counter_lock;
>  
>  #if defined(CONFIG_MEMCG_KMEM) && defined(CONFIG_INET)
> @@ -815,15 +810,8 @@ static long mem_cgroup_read_stat(struct mem_cgroup *memcg,
>  	long val = 0;
>  	int cpu;
>  
> -	get_online_cpus();
> -	for_each_online_cpu(cpu)
> +	for_each_possible_cpu(cpu)
>  		val += per_cpu(memcg->stat->count[idx], cpu);
> -#ifdef CONFIG_HOTPLUG_CPU
> -	spin_lock(&memcg->pcp_counter_lock);
> -	val += memcg->nocpu_base.count[idx];
> -	spin_unlock(&memcg->pcp_counter_lock);
> -#endif
> -	put_online_cpus();
>  	return val;
>  }
>  
> @@ -833,15 +821,8 @@ static unsigned long mem_cgroup_read_events(struct mem_cgroup *memcg,
>  	unsigned long val = 0;
>  	int cpu;
>  
> -	get_online_cpus();
> -	for_each_online_cpu(cpu)
> +	for_each_possible_cpu(cpu)
>  		val += per_cpu(memcg->stat->events[idx], cpu);
> -#ifdef CONFIG_HOTPLUG_CPU
> -	spin_lock(&memcg->pcp_counter_lock);
> -	val += memcg->nocpu_base.events[idx];
> -	spin_unlock(&memcg->pcp_counter_lock);
> -#endif
> -	put_online_cpus();
>  	return val;
>  }
>  
> @@ -2191,37 +2172,12 @@ static void drain_all_stock(struct mem_cgroup *root_memcg)
>  	mutex_unlock(&percpu_charge_mutex);
>  }
>  
> -/*
> - * This function drains percpu counter value from DEAD cpu and
> - * move it to local cpu. Note that this function can be preempted.
> - */
> -static void mem_cgroup_drain_pcp_counter(struct mem_cgroup *memcg, int cpu)
> -{
> -	int i;
> -
> -	spin_lock(&memcg->pcp_counter_lock);
> -	for (i = 0; i < MEM_CGROUP_STAT_NSTATS; i++) {
> -		long x = per_cpu(memcg->stat->count[i], cpu);
> -
> -		per_cpu(memcg->stat->count[i], cpu) = 0;
> -		memcg->nocpu_base.count[i] += x;
> -	}
> -	for (i = 0; i < MEM_CGROUP_EVENTS_NSTATS; i++) {
> -		unsigned long x = per_cpu(memcg->stat->events[i], cpu);
> -
> -		per_cpu(memcg->stat->events[i], cpu) = 0;
> -		memcg->nocpu_base.events[i] += x;
> -	}
> -	spin_unlock(&memcg->pcp_counter_lock);
> -}
> -
>  static int memcg_cpu_hotplug_callback(struct notifier_block *nb,
>  					unsigned long action,
>  					void *hcpu)
>  {
>  	int cpu = (unsigned long)hcpu;
>  	struct memcg_stock_pcp *stock;
> -	struct mem_cgroup *iter;
>  
>  	if (action == CPU_ONLINE)
>  		return NOTIFY_OK;
> @@ -2229,9 +2185,6 @@ static int memcg_cpu_hotplug_callback(struct notifier_block *nb,
>  	if (action != CPU_DEAD && action != CPU_DEAD_FROZEN)
>  		return NOTIFY_OK;
>  
> -	for_each_mem_cgroup(iter)
> -		mem_cgroup_drain_pcp_counter(iter, cpu);
> -
>  	stock = &per_cpu(memcg_stock, cpu);
>  	drain_stock(stock);
>  	return NOTIFY_OK;
> -- 
> 2.4.0
> 

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2015-06-17 14:41 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-05-22 22:23 [PATCHSET 2/3 v3 block/for-4.2/core] writeback: cgroup writeback backpressure propagation Tejun Heo
2015-05-22 22:23 ` [PATCH 01/19] memcg: make mem_cgroup_read_{stat|event}() iterate possible cpus instead of online Tejun Heo
     [not found]   ` <1432333416-6221-2-git-send-email-tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2015-05-22 23:12     ` Johannes Weiner
2015-06-17 14:41   ` Michal Hocko [this message]
2015-05-22 22:23 ` [PATCH 02/19] writeback: clean up wb_dirty_limit() Tejun Heo
2015-05-22 22:23 ` [PATCH 03/19] writeback: reorganize [__]wb_update_bandwidth() Tejun Heo
2015-05-22 22:23 ` [PATCH 04/19] writeback: implement wb_domain Tejun Heo
2015-05-22 22:23 ` [PATCH 05/19] writeback: move global_dirty_limit into wb_domain Tejun Heo
2015-05-22 22:23 ` [PATCH 06/19] writeback: consolidate dirty throttle parameters into dirty_throttle_control Tejun Heo
2015-05-22 22:23 ` [PATCH 07/19] writeback: add dirty_throttle_control->wb_bg_thresh Tejun Heo
2015-05-22 22:23 ` [PATCH 08/19] writeback: make __wb_calc_thresh() take dirty_throttle_control Tejun Heo
2015-05-22 22:23 ` [PATCH 09/19] writeback: add dirty_throttle_control->pos_ratio Tejun Heo
2015-05-22 22:23 ` [PATCH 10/19] writeback: add dirty_throttle_control->wb_completions Tejun Heo
2015-05-22 22:23 ` [PATCH 11/19] writeback: add dirty_throttle_control->dom Tejun Heo
2015-05-22 22:23 ` [PATCH 12/19] writeback: make __wb_writeout_inc() and hard_dirty_limit() take wb_domaas a parameter Tejun Heo
2015-05-22 22:23 ` [PATCH 13/19] writeback: separate out domain_dirty_limits() Tejun Heo
2015-05-22 22:23 ` [PATCH 14/19] writeback: move over_bground_thresh() to mm/page-writeback.c Tejun Heo
2015-05-22 22:23 ` [PATCH 15/19] writeback: update wb_over_bg_thresh() to use wb_domain aware operations Tejun Heo
2015-05-22 22:23 ` [PATCH 16/19] writeback: implement memcg wb_domain Tejun Heo
2015-05-22 22:23 ` [PATCH 17/19] writeback: reset wb_domain->dirty_limit[_tstmp] when memcg domain size changes Tejun Heo
2015-05-22 22:23 ` [PATCH 18/19] writeback: implement memcg writeback domain based throttling Tejun Heo
2015-05-22 22:23 ` [PATCH 19/19] mm: vmscan: disable memcg direct reclaim stalling if cgroup writeback support is in use Tejun Heo
  -- strict thread matches above, loose matches on Subject: below --
2015-04-06 20:04 [PATCHSET 2/3 v2 block/for-4.1/core] writeback: cgroup writeback backpressure propagation Tejun Heo
2015-04-06 20:04 ` [PATCH 01/19] memcg: make mem_cgroup_read_{stat|event}() iterate possible cpus instead of online Tejun Heo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150617144134.GH25056@dhcp22.suse.cz \
    --to=mhocko@suse.cz \
    --cc=axboe@kernel.dk \
    --cc=cgroups@vger.kernel.org \
    --cc=clm@fb.com \
    --cc=david@fromorbit.com \
    --cc=fengguang.wu@intel.com \
    --cc=gthelen@google.com \
    --cc=hannes@cmpxchg.org \
    --cc=hch@infradead.org \
    --cc=jack@suse.cz \
    --cc=khlebnikov@yandex-team.ru \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=lizefan@huawei.com \
    --cc=tj@kernel.org \
    --cc=vgoyal@redhat.com \
    --subject='Re: [PATCH 01/19] memcg: make mem_cgroup_read_{stat|event}() iterate possible cpus instead of online' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).