All of lore.kernel.org
 help / color / mirror / Atom feed
From: Mark Rutland <mark.rutland@arm.com>
To: Ganapatrao Prabhakerrao Kulkarni <gkulkarni@marvell.com>
Cc: "linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-arm-kernel@lists.infradead.org" 
	<linux-arm-kernel@lists.infradead.org>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"will@kernel.org" <will@kernel.org>,
	"corbet@lwn.net" <corbet@lwn.net>,
	"gklkml16@gmail.com" <gklkml16@gmail.com>
Subject: Re: [PATCH 2/2] Thunderx2, uncore: Add workaround for ThunderX2 erratum 221
Date: Wed, 6 Nov 2019 11:37:30 +0000	[thread overview]
Message-ID: <20191106113730.GB50610@lakrids.cambridge.arm.com> (raw)
In-Reply-To: <1573002091-9744-3-git-send-email-gkulkarni@marvell.com>

On Wed, Nov 06, 2019 at 01:01:41AM +0000, Ganapatrao Prabhakerrao Kulkarni wrote:
> When perf tried with more events than the PMU supported counters, the perf
> core uses event multiplexing to accommodate all events. This results in
> burst of PMU register read and writes and causes the system hang, when
> executed along with the CPU intensive applications.

Can you please elaborate on how a burst of PMU reads/writes leads to a
hang?

I see the PMU counts DMC/L3C events -- does this occur under heavy /cpu/
load, or heavy /memory/ load?

Does this only happen with a specific timing of reads/writes, or is it
always possible that accessing the PMU can trigger a lockup, and it's
just more likely when the PMU is accessed more often?

Thanks,
Mark.

> 
> Adding software workaround by disabling event multiplexing.
> 
> Signed-off-by: Ganapatrao Prabhakerrao Kulkarni <gkulkarni@marvell.com>
> ---
>  Documentation/admin-guide/perf/thunderx2-pmu.rst | 9 +++++++++
>  drivers/perf/thunderx2_pmu.c                     | 3 ++-
>  2 files changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/admin-guide/perf/thunderx2-pmu.rst b/Documentation/admin-guide/perf/thunderx2-pmu.rst
> index 08e33675853a..fff65382c887 100644
> --- a/Documentation/admin-guide/perf/thunderx2-pmu.rst
> +++ b/Documentation/admin-guide/perf/thunderx2-pmu.rst
> @@ -40,3 +40,12 @@ Examples::
>    uncore_l3c_0/read_hit/,\
>    uncore_l3c_0/inv_request/,\
>    uncore_l3c_0/inv_hit/ sleep 1
> +
> +ThunderX2 erratum 221:
> +When perf tried with more events than the PMU supported counters, the perf core
> +uses event multiplexing to accommodate all events. This results in burst of PMU
> +registers read and write and leading to system hang when executed along with
> +CPU intensive applications.
> +
> +
> +Disabling PMUs event multiplexing capability.
> diff --git a/drivers/perf/thunderx2_pmu.c b/drivers/perf/thunderx2_pmu.c
> index 43d76c85da56..c443be8bd449 100644
> --- a/drivers/perf/thunderx2_pmu.c
> +++ b/drivers/perf/thunderx2_pmu.c
> @@ -563,7 +563,8 @@ static int tx2_uncore_pmu_register(
>  		.start		= tx2_uncore_event_start,
>  		.stop		= tx2_uncore_event_stop,
>  		.read		= tx2_uncore_event_read,
> -		.capabilities	= PERF_PMU_CAP_NO_EXCLUDE,
> +		.capabilities	= PERF_PMU_CAP_NO_EXCLUDE |
> +					PERF_PMU_CAP_NO_MUX_EVENTS,
>  	};
>  
>  	tx2_pmu->pmu.name = devm_kasprintf(dev, GFP_KERNEL,
> -- 
> 2.17.1
> 

WARNING: multiple messages have this Message-ID (diff)
From: Mark Rutland <mark.rutland@arm.com>
To: Ganapatrao Prabhakerrao Kulkarni <gkulkarni@marvell.com>
Cc: "linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"corbet@lwn.net" <corbet@lwn.net>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"gklkml16@gmail.com" <gklkml16@gmail.com>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"will@kernel.org" <will@kernel.org>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH 2/2] Thunderx2, uncore: Add workaround for ThunderX2 erratum 221
Date: Wed, 6 Nov 2019 11:37:30 +0000	[thread overview]
Message-ID: <20191106113730.GB50610@lakrids.cambridge.arm.com> (raw)
In-Reply-To: <1573002091-9744-3-git-send-email-gkulkarni@marvell.com>

On Wed, Nov 06, 2019 at 01:01:41AM +0000, Ganapatrao Prabhakerrao Kulkarni wrote:
> When perf tried with more events than the PMU supported counters, the perf
> core uses event multiplexing to accommodate all events. This results in
> burst of PMU register read and writes and causes the system hang, when
> executed along with the CPU intensive applications.

Can you please elaborate on how a burst of PMU reads/writes leads to a
hang?

I see the PMU counts DMC/L3C events -- does this occur under heavy /cpu/
load, or heavy /memory/ load?

Does this only happen with a specific timing of reads/writes, or is it
always possible that accessing the PMU can trigger a lockup, and it's
just more likely when the PMU is accessed more often?

Thanks,
Mark.

> 
> Adding software workaround by disabling event multiplexing.
> 
> Signed-off-by: Ganapatrao Prabhakerrao Kulkarni <gkulkarni@marvell.com>
> ---
>  Documentation/admin-guide/perf/thunderx2-pmu.rst | 9 +++++++++
>  drivers/perf/thunderx2_pmu.c                     | 3 ++-
>  2 files changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/admin-guide/perf/thunderx2-pmu.rst b/Documentation/admin-guide/perf/thunderx2-pmu.rst
> index 08e33675853a..fff65382c887 100644
> --- a/Documentation/admin-guide/perf/thunderx2-pmu.rst
> +++ b/Documentation/admin-guide/perf/thunderx2-pmu.rst
> @@ -40,3 +40,12 @@ Examples::
>    uncore_l3c_0/read_hit/,\
>    uncore_l3c_0/inv_request/,\
>    uncore_l3c_0/inv_hit/ sleep 1
> +
> +ThunderX2 erratum 221:
> +When perf tried with more events than the PMU supported counters, the perf core
> +uses event multiplexing to accommodate all events. This results in burst of PMU
> +registers read and write and leading to system hang when executed along with
> +CPU intensive applications.
> +
> +
> +Disabling PMUs event multiplexing capability.
> diff --git a/drivers/perf/thunderx2_pmu.c b/drivers/perf/thunderx2_pmu.c
> index 43d76c85da56..c443be8bd449 100644
> --- a/drivers/perf/thunderx2_pmu.c
> +++ b/drivers/perf/thunderx2_pmu.c
> @@ -563,7 +563,8 @@ static int tx2_uncore_pmu_register(
>  		.start		= tx2_uncore_event_start,
>  		.stop		= tx2_uncore_event_stop,
>  		.read		= tx2_uncore_event_read,
> -		.capabilities	= PERF_PMU_CAP_NO_EXCLUDE,
> +		.capabilities	= PERF_PMU_CAP_NO_EXCLUDE |
> +					PERF_PMU_CAP_NO_MUX_EVENTS,
>  	};
>  
>  	tx2_pmu->pmu.name = devm_kasprintf(dev, GFP_KERNEL,
> -- 
> 2.17.1
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

  reply	other threads:[~2019-11-06 11:37 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-06  1:01 [PATCH 0/2] Workaround for ThunderX2 erratum 221 Ganapatrao Prabhakerrao Kulkarni
2019-11-06  1:01 ` Ganapatrao Prabhakerrao Kulkarni
2019-11-06  1:01 ` [PATCH 1/2] perf/core: Adding capability to disable PMUs event multiplexing Ganapatrao Prabhakerrao Kulkarni
2019-11-06  1:01   ` Ganapatrao Prabhakerrao Kulkarni
2019-11-06  9:40   ` Peter Zijlstra
2019-11-06  9:40     ` Peter Zijlstra
2019-11-06  9:58     ` Peter Zijlstra
2019-11-06  9:58       ` Peter Zijlstra
2019-11-06 11:28   ` Mark Rutland
2019-11-06 11:28     ` Mark Rutland
2019-11-06 23:28     ` Ganapatrao Kulkarni
2019-11-06 23:28       ` Ganapatrao Kulkarni
2019-11-07 14:35       ` Ganapatrao Kulkarni
2019-11-07 14:35         ` Ganapatrao Kulkarni
2019-11-07 14:52       ` Mark Rutland
2019-11-07 14:52         ` Mark Rutland
2019-11-07 15:45         ` Ganapatrao Kulkarni
2019-11-07 15:45           ` Ganapatrao Kulkarni
2019-11-07 15:54           ` Mark Rutland
2019-11-07 15:54             ` Mark Rutland
2019-11-07 15:04       ` Peter Zijlstra
2019-11-07 15:04         ` Peter Zijlstra
2019-11-07 23:17   ` kbuild test robot
2019-11-07 23:17     ` kbuild test robot
2019-11-07 23:17     ` kbuild test robot
2019-11-06  1:01 ` [PATCH 2/2] Thunderx2, uncore: Add workaround for ThunderX2 erratum 221 Ganapatrao Prabhakerrao Kulkarni
2019-11-06  1:01   ` Ganapatrao Prabhakerrao Kulkarni
2019-11-06 11:37   ` Mark Rutland [this message]
2019-11-06 11:37     ` Mark Rutland

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191106113730.GB50610@lakrids.cambridge.arm.com \
    --to=mark.rutland@arm.com \
    --cc=corbet@lwn.net \
    --cc=gklkml16@gmail.com \
    --cc=gkulkarni@marvell.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.