RE: perf tools:Is there any tools to found out the max latency by irq or cpu idle

From: Linhaifeng <haifeng.lin@huawei.com>
To: Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com>,
	Paul Clarke <pc@us.ibm.com>
Cc: "linux-perf-users@vger.kernel.org"
	<linux-perf-users@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: perf tools:Is there any tools to found out the max latency by irq or cpu idle
Date: Tue, 16 Apr 2019 01:09:33 +0000	[thread overview]
Message-ID: <4099DE2E54AFAD489356C6C9161D53339592850D@dggeml522-mbs.china.huawei.com> (raw)
In-Reply-To: <20190415175907.GC11294@kernel.org>

Hi,

Thank you every much!
I found MSI in my system too and there are some errors found in bios log. Replace the memorys can fix it.

L2GW_2680:/home/fsp/ycb # perf stat -C 1-11,14-25,29-39,42-52 --smi-cost
^C
Performance counter stats for 'CPU(s) 1-11,14-25,29-39,42-52':

SMI cycles% SMI# 
-0.2% 90 

4472.483413712 seconds time elapsed

-----Original Message-----
From: Arnaldo Carvalho de Melo [mailto:arnaldo.melo@gmail.com] 
Sent: 2019年4月16日 1:59
To: Paul Clarke <pc@us.ibm.com>
Cc: Linhaifeng <haifeng.lin@huawei.com>; linux-perf-users@vger.kernel.org; linux-kernel@vger.kernel.org
Subject: Re: perf tools:Is there any tools to found out the max latency by irq or cpu idle

Em Mon, Apr 15, 2019 at 07:52:59AM -0500, Paul Clarke escreveu:
> On 4/13/19 1:01 AM, Linhaifeng wrote:
> > Sorry, the value 131081408 is just for example. Actually the result is like this:
> >  sqrt 2019-04-10 23:53:50: 43968
> >  sqrt 2019-04-10 23:53:51: 44060
> >  sqrt 2019-04-10 23:53:52: 49012
> >  sqrt 2019-04-10 23:53:53: 38172
> >  sqrt 2019-04-10 23:53:54: 131081408  sqrt 2019-04-10 23:53:55: 
> > 43600  sqrt 2019-04-10 23:53:56: 46704  sqrt 2019-04-10 23:53:57: 
> > 46880  sqrt 2019-04-10 23:53:58: 44332  ……  sqrt 2019-04-10 
> > 02:17:15: 136345876  ……  sqrt 2019-04-10 04:40:35: 136335644  ……
> 
> Given the periodicity, I'm wondering if it could be System Management Interrupts (SMIs)?
> 
> If so, some BIOSes have settings to mitigate, otherwise, there's not much that can be done, per my understanding.

[root@quaco perf]# perf list smi

List of pre-defined events (to be used in -e):

  msr/smi/                                           [Kernel PMU event]

  sunrpc:xprt_transmit                               [Tracepoint event]
  tcp:tcp_retransmit_skb                             [Tracepoint event]
  tcp:tcp_retransmit_synack                          [Tracepoint event]

  sdt_qemu:qmp_block_job_dismiss                     [SDT event]
  sdt_qemu:qmp_job_dismiss                           [SDT event]

Metric Groups:

[root@quaco perf]#

Here, using it:

[root@quaco perf]# perf stat -I 1000 -e msr/smi/
#           time             counts unit events
     1.000919819                  0      msr/smi/                 
     2.001689950                  0      msr/smi/                 
     3.002266988                  0      msr/smi/                 
     4.003049439                 16      msr/smi/                 
     5.003676064                  0      msr/smi/                 
     6.004420613                 16      msr/smi/                 
^C     6.609405666                  0      msr/smi/                 

[root@quaco perf]# 

In this Lenovo T480s I generate SMIs by pressing the mute button :-)

- Arnaldo