From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 370A0C33FA2 for ; Mon, 17 Feb 2020 13:24:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1810A2070B for ; Mon, 17 Feb 2020 13:24:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728448AbgBQNYO (ORCPT ); Mon, 17 Feb 2020 08:24:14 -0500 Received: from out30-57.freemail.mail.aliyun.com ([115.124.30.57]:35205 "EHLO out30-57.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728124AbgBQNYN (ORCPT ); Mon, 17 Feb 2020 08:24:13 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01f04452;MF=yun.wang@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0TqAk0uJ_1581945832; Received: from testdeMacBook-Pro.local(mailfrom:yun.wang@linux.alibaba.com fp:SMTPD_---0TqAk0uJ_1581945832) by smtp.aliyun-inc.com(127.0.0.1); Mon, 17 Feb 2020 21:23:53 +0800 Subject: Re: [PATCH RESEND v8 1/2] sched/numa: introduce per-cgroup NUMA locality info To: Mel Gorman , Peter Zijlstra Cc: Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Luis Chamberlain , Kees Cook , Iurii Zaikin , Michal Koutn? , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, "Paul E. McKenney" , Randy Dunlap , Jonathan Corbet References: <20200214151048.GL14914@hirez.programming.kicks-ass.net> <20200217115810.GA3420@suse.de> From: =?UTF-8?B?546L6LSH?= Message-ID: <881deb50-163e-442a-41ec-b375cc445e4d@linux.alibaba.com> Date: Mon, 17 Feb 2020 21:23:52 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: <20200217115810.GA3420@suse.de> Content-Type: text/plain; charset=UTF-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org On 2020/2/17 下午7:58, Mel Gorman wrote: [snip] >> Mel, I suspect you still feel that way, right? >> > > Yes, I still think it would be a struggle to interpret the data > meaningfully without very specific knowledge of the implementation. If > the scan rate was constant, it would be easier but that would make NUMA > balancing worse overall. Similarly, the stat might get very difficult to > interpret when NUMA balancing is failing because of a load imbalance, > pages are shared and being interleaved or NUMA groups span multiple > active nodes. Hi, Mel, appreciated to have you back on the table :-) IMHO the scan period changing should not be a problem now, since the maximum period is defined by user, so monitoring at maximum period on the accumulated page accessing counters is always meaningful, correct? FYI, by monitoring locality, we found that the kvm vcpu thread is not covered by NUMA Balancing, whatever how many maximum period passed, the counters are not increasing, or very slowly, although inside guest we are copying memory. Later we found such task rarely exit to user space to trigger task work callbacks, and NUMA Balancing scan depends on that, which help us realize the importance to enable NUMA Balancing inside guest, with the correct NUMA topo, a big performance risk I'll say :-P Maybe not a good example, but we just try to highlight that NUMA Balancing could have issue in some cases, and we want them to be exposed, somehow, maybe by the locality. Regards, Michael Wang > > For example, the series that reconciles NUMA and CPU balancers may look > worse in these stats even though the overall performance may be better. > >> In the document (patch 2/2) you write: >> >>> +However, there are no hardware counters for per-task local/remote accessing >>> +info, we don't know how many remote page accesses have occurred for a >>> +particular task. >> >> We can of course 'fix' that by adding a tracepoint. >> >> Mel, would you feel better by having a tracepoint in task_numa_fault() ? >> > > A bit, although interpreting the data would still be difficult and the > tracepoint would have to include information about the cgroup. While > I've never tried, this seems like the type of thing that would be suited > to a BPF script that probes task_numa_fault and extract the information > it needs. >