All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] arm64: defconfig: enable fine-grained task level IRQ time accounting
@ 2017-07-31 20:22 ` Marcin Wojtas
  0 siblings, 0 replies; 8+ messages in thread
From: Marcin Wojtas @ 2017-07-31 20:22 UTC (permalink / raw)
  To: linux-kernel, linux-arm-kernel
  Cc: catalin.marinas, will.deacon, andrew, thomas.petazzoni,
	gregory.clement, nadavh, neta, mw, jaz, tn

Tests showed, that under certain conditions, the summary number of jiffies
spent on softirq/idle, which are counted by system statistics can be even
below 10% of expected value, resulting in false load presentation.

The issue was observed on the quad-core Marvell Armada 8k SoC, whose two
10G ports were bound into L2 bridge. Load was controlled by bidirectional
UDP traffic, produced by a packet generator. Under such condition,
the dominant load is softirq. With 100% single CPU occupation or without
any activity (all CPUs 100% idle), total number of jiffies is 10000 (2500
per each core) in 10s interval. Also with other kind of load this was
true.

However below a saturation threshold it was observed, that with CPU which
was occupied almost by softirqs only, the statistic were awkward. See
the mpstat output:

CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
all 0.00  0.00 0.13    0.00 0.00  0.55   0.00   0.00   0.00 99.32
  0 0.00  0.00 0.00    0.00 0.00 23.08   0.00   0.00   0.00 76.92
  1 0.00  0.00 0.40    0.00 0.00  0.00   0.00   0.00   0.00 99.60
  2 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
  3 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00

Above would mean basically no total load, debug CPU0 occupied in 25%.
Raw statistics, printed every 10s from /proc/stat unveiled a root
cause - summary idle/softirq jiffies on loaded CPU were below 200,
i.e. over 90% samples lost. All problems were gone after enabling
fine granulity IRQ time accounting.

This patch fixes possible wrong statistics processing by enabling
CONFIG_IRQ_TIME_ACCOUNTING for arm64 platfroms, which is by
default done on other architectures, e.g. x86 and arm. Tests
showed no noticeable performance penalty, nor stability impact.

Signed-off-by: Marcin Wojtas <mw@semihalf.com>
---
 arch/arm64/configs/defconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index 44423e6..ed51ac6 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -3,6 +3,7 @@ CONFIG_POSIX_MQUEUE=y
 CONFIG_AUDIT=y
 CONFIG_NO_HZ_IDLE=y
 CONFIG_HIGH_RES_TIMERS=y
+CONFIG_IRQ_TIME_ACCOUNTING=y
 CONFIG_BSD_PROCESS_ACCT=y
 CONFIG_BSD_PROCESS_ACCT_V3=y
 CONFIG_TASKSTATS=y
-- 
1.8.3.1

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH] arm64: defconfig: enable fine-grained task level IRQ time accounting
@ 2017-07-31 20:22 ` Marcin Wojtas
  0 siblings, 0 replies; 8+ messages in thread
From: Marcin Wojtas @ 2017-07-31 20:22 UTC (permalink / raw)
  To: linux-arm-kernel

Tests showed, that under certain conditions, the summary number of jiffies
spent on softirq/idle, which are counted by system statistics can be even
below 10% of expected value, resulting in false load presentation.

The issue was observed on the quad-core Marvell Armada 8k SoC, whose two
10G ports were bound into L2 bridge. Load was controlled by bidirectional
UDP traffic, produced by a packet generator. Under such condition,
the dominant load is softirq. With 100% single CPU occupation or without
any activity (all CPUs 100% idle), total number of jiffies is 10000 (2500
per each core) in 10s interval. Also with other kind of load this was
true.

However below a saturation threshold it was observed, that with CPU which
was occupied almost by softirqs only, the statistic were awkward. See
the mpstat output:

CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
all 0.00  0.00 0.13    0.00 0.00  0.55   0.00   0.00   0.00 99.32
  0 0.00  0.00 0.00    0.00 0.00 23.08   0.00   0.00   0.00 76.92
  1 0.00  0.00 0.40    0.00 0.00  0.00   0.00   0.00   0.00 99.60
  2 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
  3 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00

Above would mean basically no total load, debug CPU0 occupied in 25%.
Raw statistics, printed every 10s from /proc/stat unveiled a root
cause - summary idle/softirq jiffies on loaded CPU were below 200,
i.e. over 90% samples lost. All problems were gone after enabling
fine granulity IRQ time accounting.

This patch fixes possible wrong statistics processing by enabling
CONFIG_IRQ_TIME_ACCOUNTING for arm64 platfroms, which is by
default done on other architectures, e.g. x86 and arm. Tests
showed no noticeable performance penalty, nor stability impact.

Signed-off-by: Marcin Wojtas <mw@semihalf.com>
---
 arch/arm64/configs/defconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index 44423e6..ed51ac6 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -3,6 +3,7 @@ CONFIG_POSIX_MQUEUE=y
 CONFIG_AUDIT=y
 CONFIG_NO_HZ_IDLE=y
 CONFIG_HIGH_RES_TIMERS=y
+CONFIG_IRQ_TIME_ACCOUNTING=y
 CONFIG_BSD_PROCESS_ACCT=y
 CONFIG_BSD_PROCESS_ACCT_V3=y
 CONFIG_TASKSTATS=y
-- 
1.8.3.1

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH] arm64: defconfig: enable fine-grained task level IRQ time accounting
  2017-07-31 20:22 ` Marcin Wojtas
@ 2017-08-02 13:11   ` Gregory CLEMENT
  -1 siblings, 0 replies; 8+ messages in thread
From: Gregory CLEMENT @ 2017-08-02 13:11 UTC (permalink / raw)
  To: catalin.marinas, will.deacon, Arnd Bergmann, Olof Johansson
  Cc: Marcin Wojtas, linux-kernel, linux-arm-kernel, andrew,
	thomas.petazzoni, nadavh, neta, jaz, tn

Hi,

(Adding Arnd and Olof)
 
 On lun., juil. 31 2017, Marcin Wojtas <mw@semihalf.com> wrote:

> Tests showed, that under certain conditions, the summary number of jiffies
> spent on softirq/idle, which are counted by system statistics can be even
> below 10% of expected value, resulting in false load presentation.
>
> The issue was observed on the quad-core Marvell Armada 8k SoC, whose two
> 10G ports were bound into L2 bridge. Load was controlled by bidirectional
> UDP traffic, produced by a packet generator. Under such condition,
> the dominant load is softirq. With 100% single CPU occupation or without
> any activity (all CPUs 100% idle), total number of jiffies is 10000 (2500
> per each core) in 10s interval. Also with other kind of load this was
> true.
>
> However below a saturation threshold it was observed, that with CPU which
> was occupied almost by softirqs only, the statistic were awkward. See
> the mpstat output:
>
> CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
> all 0.00  0.00 0.13    0.00 0.00  0.55   0.00   0.00   0.00 99.32
>   0 0.00  0.00 0.00    0.00 0.00 23.08   0.00   0.00   0.00 76.92
>   1 0.00  0.00 0.40    0.00 0.00  0.00   0.00   0.00   0.00 99.60
>   2 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
>   3 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
>
> Above would mean basically no total load, debug CPU0 occupied in 25%.
> Raw statistics, printed every 10s from /proc/stat unveiled a root
> cause - summary idle/softirq jiffies on loaded CPU were below 200,
> i.e. over 90% samples lost. All problems were gone after enabling
> fine granulity IRQ time accounting.
>
> This patch fixes possible wrong statistics processing by enabling
> CONFIG_IRQ_TIME_ACCOUNTING for arm64 platfroms, which is by
> default done on other architectures, e.g. x86 and arm. Tests
> showed no noticeable performance penalty, nor stability impact.

Who should take this patch?

I think that all the defconfig under arm64 are merged through the
arm-soc subsystem, but this one is not really specific to a
SoC. However, as it was experimented on an mvebu SoC, if you agree I can
take it.

Thanks,

Gregory

>
> Signed-off-by: Marcin Wojtas <mw@semihalf.com>
> ---
>  arch/arm64/configs/defconfig | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
> index 44423e6..ed51ac6 100644
> --- a/arch/arm64/configs/defconfig
> +++ b/arch/arm64/configs/defconfig
> @@ -3,6 +3,7 @@ CONFIG_POSIX_MQUEUE=y
>  CONFIG_AUDIT=y
>  CONFIG_NO_HZ_IDLE=y
>  CONFIG_HIGH_RES_TIMERS=y
> +CONFIG_IRQ_TIME_ACCOUNTING=y
>  CONFIG_BSD_PROCESS_ACCT=y
>  CONFIG_BSD_PROCESS_ACCT_V3=y
>  CONFIG_TASKSTATS=y
> -- 
> 1.8.3.1
>

-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH] arm64: defconfig: enable fine-grained task level IRQ time accounting
@ 2017-08-02 13:11   ` Gregory CLEMENT
  0 siblings, 0 replies; 8+ messages in thread
From: Gregory CLEMENT @ 2017-08-02 13:11 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

(Adding Arnd and Olof)
 
 On lun., juil. 31 2017, Marcin Wojtas <mw@semihalf.com> wrote:

> Tests showed, that under certain conditions, the summary number of jiffies
> spent on softirq/idle, which are counted by system statistics can be even
> below 10% of expected value, resulting in false load presentation.
>
> The issue was observed on the quad-core Marvell Armada 8k SoC, whose two
> 10G ports were bound into L2 bridge. Load was controlled by bidirectional
> UDP traffic, produced by a packet generator. Under such condition,
> the dominant load is softirq. With 100% single CPU occupation or without
> any activity (all CPUs 100% idle), total number of jiffies is 10000 (2500
> per each core) in 10s interval. Also with other kind of load this was
> true.
>
> However below a saturation threshold it was observed, that with CPU which
> was occupied almost by softirqs only, the statistic were awkward. See
> the mpstat output:
>
> CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
> all 0.00  0.00 0.13    0.00 0.00  0.55   0.00   0.00   0.00 99.32
>   0 0.00  0.00 0.00    0.00 0.00 23.08   0.00   0.00   0.00 76.92
>   1 0.00  0.00 0.40    0.00 0.00  0.00   0.00   0.00   0.00 99.60
>   2 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
>   3 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
>
> Above would mean basically no total load, debug CPU0 occupied in 25%.
> Raw statistics, printed every 10s from /proc/stat unveiled a root
> cause - summary idle/softirq jiffies on loaded CPU were below 200,
> i.e. over 90% samples lost. All problems were gone after enabling
> fine granulity IRQ time accounting.
>
> This patch fixes possible wrong statistics processing by enabling
> CONFIG_IRQ_TIME_ACCOUNTING for arm64 platfroms, which is by
> default done on other architectures, e.g. x86 and arm. Tests
> showed no noticeable performance penalty, nor stability impact.

Who should take this patch?

I think that all the defconfig under arm64 are merged through the
arm-soc subsystem, but this one is not really specific to a
SoC. However, as it was experimented on an mvebu SoC, if you agree I can
take it.

Thanks,

Gregory

>
> Signed-off-by: Marcin Wojtas <mw@semihalf.com>
> ---
>  arch/arm64/configs/defconfig | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
> index 44423e6..ed51ac6 100644
> --- a/arch/arm64/configs/defconfig
> +++ b/arch/arm64/configs/defconfig
> @@ -3,6 +3,7 @@ CONFIG_POSIX_MQUEUE=y
>  CONFIG_AUDIT=y
>  CONFIG_NO_HZ_IDLE=y
>  CONFIG_HIGH_RES_TIMERS=y
> +CONFIG_IRQ_TIME_ACCOUNTING=y
>  CONFIG_BSD_PROCESS_ACCT=y
>  CONFIG_BSD_PROCESS_ACCT_V3=y
>  CONFIG_TASKSTATS=y
> -- 
> 1.8.3.1
>

-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] arm64: defconfig: enable fine-grained task level IRQ time accounting
  2017-08-02 13:11   ` Gregory CLEMENT
@ 2017-08-02 14:33     ` Catalin Marinas
  -1 siblings, 0 replies; 8+ messages in thread
From: Catalin Marinas @ 2017-08-02 14:33 UTC (permalink / raw)
  To: Gregory CLEMENT
  Cc: will.deacon, Arnd Bergmann, Olof Johansson, thomas.petazzoni,
	andrew, jaz, linux-kernel, nadavh, neta, tn, Marcin Wojtas,
	linux-arm-kernel

On Wed, Aug 02, 2017 at 03:11:43PM +0200, Gregory CLEMENT wrote:
>  On lun., juil. 31 2017, Marcin Wojtas <mw@semihalf.com> wrote:
> > Tests showed, that under certain conditions, the summary number of jiffies
> > spent on softirq/idle, which are counted by system statistics can be even
> > below 10% of expected value, resulting in false load presentation.
> >
> > The issue was observed on the quad-core Marvell Armada 8k SoC, whose two
> > 10G ports were bound into L2 bridge. Load was controlled by bidirectional
> > UDP traffic, produced by a packet generator. Under such condition,
> > the dominant load is softirq. With 100% single CPU occupation or without
> > any activity (all CPUs 100% idle), total number of jiffies is 10000 (2500
> > per each core) in 10s interval. Also with other kind of load this was
> > true.
> >
> > However below a saturation threshold it was observed, that with CPU which
> > was occupied almost by softirqs only, the statistic were awkward. See
> > the mpstat output:
> >
> > CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
> > all 0.00  0.00 0.13    0.00 0.00  0.55   0.00   0.00   0.00 99.32
> >   0 0.00  0.00 0.00    0.00 0.00 23.08   0.00   0.00   0.00 76.92
> >   1 0.00  0.00 0.40    0.00 0.00  0.00   0.00   0.00   0.00 99.60
> >   2 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
> >   3 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
> >
> > Above would mean basically no total load, debug CPU0 occupied in 25%.
> > Raw statistics, printed every 10s from /proc/stat unveiled a root
> > cause - summary idle/softirq jiffies on loaded CPU were below 200,
> > i.e. over 90% samples lost. All problems were gone after enabling
> > fine granulity IRQ time accounting.
> >
> > This patch fixes possible wrong statistics processing by enabling
> > CONFIG_IRQ_TIME_ACCOUNTING for arm64 platfroms, which is by
> > default done on other architectures, e.g. x86 and arm. Tests
> > showed no noticeable performance penalty, nor stability impact.
> 
> Who should take this patch?
> 
> I think that all the defconfig under arm64 are merged through the
> arm-soc subsystem, but this one is not really specific to a
> SoC. However, as it was experimented on an mvebu SoC, if you agree I
> can take it.

It's fine by me to go via arm-soc.

-- 
Catalin

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH] arm64: defconfig: enable fine-grained task level IRQ time accounting
@ 2017-08-02 14:33     ` Catalin Marinas
  0 siblings, 0 replies; 8+ messages in thread
From: Catalin Marinas @ 2017-08-02 14:33 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Aug 02, 2017 at 03:11:43PM +0200, Gregory CLEMENT wrote:
>  On lun., juil. 31 2017, Marcin Wojtas <mw@semihalf.com> wrote:
> > Tests showed, that under certain conditions, the summary number of jiffies
> > spent on softirq/idle, which are counted by system statistics can be even
> > below 10% of expected value, resulting in false load presentation.
> >
> > The issue was observed on the quad-core Marvell Armada 8k SoC, whose two
> > 10G ports were bound into L2 bridge. Load was controlled by bidirectional
> > UDP traffic, produced by a packet generator. Under such condition,
> > the dominant load is softirq. With 100% single CPU occupation or without
> > any activity (all CPUs 100% idle), total number of jiffies is 10000 (2500
> > per each core) in 10s interval. Also with other kind of load this was
> > true.
> >
> > However below a saturation threshold it was observed, that with CPU which
> > was occupied almost by softirqs only, the statistic were awkward. See
> > the mpstat output:
> >
> > CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
> > all 0.00  0.00 0.13    0.00 0.00  0.55   0.00   0.00   0.00 99.32
> >   0 0.00  0.00 0.00    0.00 0.00 23.08   0.00   0.00   0.00 76.92
> >   1 0.00  0.00 0.40    0.00 0.00  0.00   0.00   0.00   0.00 99.60
> >   2 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
> >   3 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
> >
> > Above would mean basically no total load, debug CPU0 occupied in 25%.
> > Raw statistics, printed every 10s from /proc/stat unveiled a root
> > cause - summary idle/softirq jiffies on loaded CPU were below 200,
> > i.e. over 90% samples lost. All problems were gone after enabling
> > fine granulity IRQ time accounting.
> >
> > This patch fixes possible wrong statistics processing by enabling
> > CONFIG_IRQ_TIME_ACCOUNTING for arm64 platfroms, which is by
> > default done on other architectures, e.g. x86 and arm. Tests
> > showed no noticeable performance penalty, nor stability impact.
> 
> Who should take this patch?
> 
> I think that all the defconfig under arm64 are merged through the
> arm-soc subsystem, but this one is not really specific to a
> SoC. However, as it was experimented on an mvebu SoC, if you agree I
> can take it.

It's fine by me to go via arm-soc.

-- 
Catalin

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] arm64: defconfig: enable fine-grained task level IRQ time accounting
  2017-07-31 20:22 ` Marcin Wojtas
@ 2017-08-03 12:26   ` Gregory CLEMENT
  -1 siblings, 0 replies; 8+ messages in thread
From: Gregory CLEMENT @ 2017-08-03 12:26 UTC (permalink / raw)
  To: Marcin Wojtas
  Cc: linux-kernel, linux-arm-kernel, catalin.marinas, will.deacon,
	andrew, thomas.petazzoni, nadavh, neta, jaz, tn

Hi Marcin,
 
 On lun., juil. 31 2017, Marcin Wojtas <mw@semihalf.com> wrote:

> Tests showed, that under certain conditions, the summary number of jiffies
> spent on softirq/idle, which are counted by system statistics can be even
> below 10% of expected value, resulting in false load presentation.
>
> The issue was observed on the quad-core Marvell Armada 8k SoC, whose two
> 10G ports were bound into L2 bridge. Load was controlled by bidirectional
> UDP traffic, produced by a packet generator. Under such condition,
> the dominant load is softirq. With 100% single CPU occupation or without
> any activity (all CPUs 100% idle), total number of jiffies is 10000 (2500
> per each core) in 10s interval. Also with other kind of load this was
> true.
>
> However below a saturation threshold it was observed, that with CPU which
> was occupied almost by softirqs only, the statistic were awkward. See
> the mpstat output:
>
> CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
> all 0.00  0.00 0.13    0.00 0.00  0.55   0.00   0.00   0.00 99.32
>   0 0.00  0.00 0.00    0.00 0.00 23.08   0.00   0.00   0.00 76.92
>   1 0.00  0.00 0.40    0.00 0.00  0.00   0.00   0.00   0.00 99.60
>   2 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
>   3 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
>
> Above would mean basically no total load, debug CPU0 occupied in 25%.
> Raw statistics, printed every 10s from /proc/stat unveiled a root
> cause - summary idle/softirq jiffies on loaded CPU were below 200,
> i.e. over 90% samples lost. All problems were gone after enabling
> fine granulity IRQ time accounting.
>
> This patch fixes possible wrong statistics processing by enabling
> CONFIG_IRQ_TIME_ACCOUNTING for arm64 platfroms, which is by
> default done on other architectures, e.g. x86 and arm. Tests
> showed no noticeable performance penalty, nor stability impact.
>
> Signed-off-by: Marcin Wojtas <mw@semihalf.com>

Applied on mvebu/arm64

Thanks,

Gregory

> ---
>  arch/arm64/configs/defconfig | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
> index 44423e6..ed51ac6 100644
> --- a/arch/arm64/configs/defconfig
> +++ b/arch/arm64/configs/defconfig
> @@ -3,6 +3,7 @@ CONFIG_POSIX_MQUEUE=y
>  CONFIG_AUDIT=y
>  CONFIG_NO_HZ_IDLE=y
>  CONFIG_HIGH_RES_TIMERS=y
> +CONFIG_IRQ_TIME_ACCOUNTING=y
>  CONFIG_BSD_PROCESS_ACCT=y
>  CONFIG_BSD_PROCESS_ACCT_V3=y
>  CONFIG_TASKSTATS=y
> -- 
> 1.8.3.1
>

-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH] arm64: defconfig: enable fine-grained task level IRQ time accounting
@ 2017-08-03 12:26   ` Gregory CLEMENT
  0 siblings, 0 replies; 8+ messages in thread
From: Gregory CLEMENT @ 2017-08-03 12:26 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marcin,
 
 On lun., juil. 31 2017, Marcin Wojtas <mw@semihalf.com> wrote:

> Tests showed, that under certain conditions, the summary number of jiffies
> spent on softirq/idle, which are counted by system statistics can be even
> below 10% of expected value, resulting in false load presentation.
>
> The issue was observed on the quad-core Marvell Armada 8k SoC, whose two
> 10G ports were bound into L2 bridge. Load was controlled by bidirectional
> UDP traffic, produced by a packet generator. Under such condition,
> the dominant load is softirq. With 100% single CPU occupation or without
> any activity (all CPUs 100% idle), total number of jiffies is 10000 (2500
> per each core) in 10s interval. Also with other kind of load this was
> true.
>
> However below a saturation threshold it was observed, that with CPU which
> was occupied almost by softirqs only, the statistic were awkward. See
> the mpstat output:
>
> CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
> all 0.00  0.00 0.13    0.00 0.00  0.55   0.00   0.00   0.00 99.32
>   0 0.00  0.00 0.00    0.00 0.00 23.08   0.00   0.00   0.00 76.92
>   1 0.00  0.00 0.40    0.00 0.00  0.00   0.00   0.00   0.00 99.60
>   2 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
>   3 0.00  0.00 0.00    0.00 0.00  0.00   0.00   0.00   0.00 100.00
>
> Above would mean basically no total load, debug CPU0 occupied in 25%.
> Raw statistics, printed every 10s from /proc/stat unveiled a root
> cause - summary idle/softirq jiffies on loaded CPU were below 200,
> i.e. over 90% samples lost. All problems were gone after enabling
> fine granulity IRQ time accounting.
>
> This patch fixes possible wrong statistics processing by enabling
> CONFIG_IRQ_TIME_ACCOUNTING for arm64 platfroms, which is by
> default done on other architectures, e.g. x86 and arm. Tests
> showed no noticeable performance penalty, nor stability impact.
>
> Signed-off-by: Marcin Wojtas <mw@semihalf.com>

Applied on mvebu/arm64

Thanks,

Gregory

> ---
>  arch/arm64/configs/defconfig | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
> index 44423e6..ed51ac6 100644
> --- a/arch/arm64/configs/defconfig
> +++ b/arch/arm64/configs/defconfig
> @@ -3,6 +3,7 @@ CONFIG_POSIX_MQUEUE=y
>  CONFIG_AUDIT=y
>  CONFIG_NO_HZ_IDLE=y
>  CONFIG_HIGH_RES_TIMERS=y
> +CONFIG_IRQ_TIME_ACCOUNTING=y
>  CONFIG_BSD_PROCESS_ACCT=y
>  CONFIG_BSD_PROCESS_ACCT_V3=y
>  CONFIG_TASKSTATS=y
> -- 
> 1.8.3.1
>

-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2017-08-03 12:27 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-07-31 20:22 [PATCH] arm64: defconfig: enable fine-grained task level IRQ time accounting Marcin Wojtas
2017-07-31 20:22 ` Marcin Wojtas
2017-08-02 13:11 ` Gregory CLEMENT
2017-08-02 13:11   ` Gregory CLEMENT
2017-08-02 14:33   ` Catalin Marinas
2017-08-02 14:33     ` Catalin Marinas
2017-08-03 12:26 ` Gregory CLEMENT
2017-08-03 12:26   ` Gregory CLEMENT

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.