KVM ARM Archive on lore.kernel.org
 help / color / Atom feed
From: Marc Zyngier <maz@kernel.org>
To: Robert Richter <rrichter@marvell.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
	Jason Cooper <jason@lakedaemon.net>,
	linux-kernel@vger.kernel.org,
	Andrew Murray <Andrew.Murray@arm.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	kvmarm@lists.cs.columbia.edu
Subject: Re: [PATCH v3 03/32] irqchip/gic-v3: Workaround Cavium TX1 erratum when reading GICD_TYPER2
Date: Tue, 10 Mar 2020 11:41:09 +0000
Message-ID: <b1b7db1f0e1c47b7d9e2dfbbe3409b77@kernel.org> (raw)
In-Reply-To: <20200309221137.5pjh4vkc62ft3h2a@rric.localdomain>

Hi Robert,

On 2020-03-09 22:11, Robert Richter wrote:
> On 24.12.19 11:10:26, Marc Zyngier wrote:
>> Despite the architecture spec being extremely clear about the fact
>> that reserved registers in the GIC distributor memory map are RES0
>> (and thus are not allowed to generate an exception), the Cavium
>> ThunderX (aka TX1) SoC explodes as such:
>> 
>> [    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
>> [    0.000000] GICv3: 128 SPIs implemented
>> [    0.000000] GICv3: 0 Extended SPIs implemented
>> [    0.000000] Internal error: synchronous external abort: 96000210 
>> [#1] SMP
>> [    0.000000] Modules linked in:
>> [    0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 
>> 5.4.0-rc4-00035-g3cf6a3d5725f #7956
>> [    0.000000] Hardware name: cavium,thunder-88xx (DT)
>> [    0.000000] pstate: 60000085 (nZCv daIf -PAN -UAO)
>> [    0.000000] pc : __raw_readl+0x0/0x8
>> [    0.000000] lr : gic_init_bases+0x110/0x560
>> [    0.000000] sp : ffff800011243d90
>> [    0.000000] x29: ffff800011243d90 x28: 0000000000000000
>> [    0.000000] x27: 0000000000000018 x26: 0000000000000002
>> [    0.000000] x25: ffff8000116f0000 x24: ffff000fbe6a2c80
>> [    0.000000] x23: 0000000000000000 x22: ffff010fdc322b68
>> [    0.000000] x21: ffff800010a7a208 x20: 00000000009b0404
>> [    0.000000] x19: ffff80001124dad0 x18: 0000000000000010
>> [    0.000000] x17: 000000004d8d492b x16: 00000000f67eb9af
>> [    0.000000] x15: ffffffffffffffff x14: ffff800011249908
>> [    0.000000] x13: ffff800091243ae7 x12: ffff800011243af4
>> [    0.000000] x11: ffff80001126e000 x10: ffff800011243a70
>> [    0.000000] x9 : 00000000ffffffd0 x8 : ffff80001069c828
>> [    0.000000] x7 : 0000000000000059 x6 : ffff8000113fb4d1
>> [    0.000000] x5 : 0000000000000001 x4 : 0000000000000000
>> [    0.000000] x3 : 0000000000000000 x2 : 0000000000000000
>> [    0.000000] x1 : 0000000000000000 x0 : ffff8000116f000c
>> [    0.000000] Call trace:
>> [    0.000000]  __raw_readl+0x0/0x8
>> [    0.000000]  gic_of_init+0x188/0x224
>> [    0.000000]  of_irq_init+0x200/0x3cc
>> [    0.000000]  irqchip_init+0x1c/0x40
>> [    0.000000]  init_IRQ+0x160/0x1d0
>> [    0.000000]  start_kernel+0x2ec/0x4b8
>> [    0.000000] Code: a8c47bfd d65f03c0 d538d080 d65f03c0 (b9400000)
>> 
>> when reading the GICv4.1 GICD_TYPER2 register, which is unexpected...
>> 
>> Work around it by adding a new quirk flagging all the A1 revisions
>> of the distributor, but it remains unknown whether this could affect
>> other revisions of this SoC (or even other SoCs from the same silicon
>> vendor).
>> 
>> Signed-off-by: Marc Zyngier <maz@kernel.org>
>> ---
>>  drivers/irqchip/irq-gic-v3.c | 23 ++++++++++++++++++++++-
>>  1 file changed, 22 insertions(+), 1 deletion(-)
>> 
>> diff --git a/drivers/irqchip/irq-gic-v3.c 
>> b/drivers/irqchip/irq-gic-v3.c
>> index 286f98222878..640d4db65b78 100644
>> --- a/drivers/irqchip/irq-gic-v3.c
>> +++ b/drivers/irqchip/irq-gic-v3.c
>> @@ -34,6 +34,7 @@
>>  #define GICD_INT_NMI_PRI	(GICD_INT_DEF_PRI & ~0x80)
>> 
>>  #define FLAGS_WORKAROUND_GICR_WAKER_MSM8996	(1ULL << 0)
>> +#define FLAGS_WORKAROUND_GICD_TYPER2_TX1	(1ULL << 1)
>> 
>>  struct redist_region {
>>  	void __iomem		*redist_base;
>> @@ -1464,6 +1465,15 @@ static bool gic_enable_quirk_msm8996(void 
>> *data)
>>  	return true;
>>  }
>> 
>> +static bool gic_enable_quirk_tx1(void *data)
>> +{
>> +	struct gic_chip_data *d = data;
>> +
>> +	d->flags |= FLAGS_WORKAROUND_GICD_TYPER2_TX1;
>> +
>> +	return true;
>> +}
>> +
>>  static bool gic_enable_quirk_hip06_07(void *data)
>>  {
>>  	struct gic_chip_data *d = data;
>> @@ -1502,6 +1512,12 @@ static const struct gic_quirk gic_quirks[] = {
>>  		.mask	= 0xffffffff,
>>  		.init	= gic_enable_quirk_hip06_07,
>>  	},
>> +	{
>> +		.desc	= "GICv3: Cavium TX1 GICD_TYPER2 erratum",
> 
> There is no errata number yet.

Please let me know when/if you obtain one.

> 
>> +		.iidr	= 0xa100034c,
>> +		.mask	= 0xfff00fff,
>> +		.init	= gic_enable_quirk_tx1,
> 
> All TX1 and OcteonTX parts are affected, which is a0-a7 and b0-b7. So
> the iidr/mask should be:
> 
> 		.iidr   = 0xa000034c,
> 		.mask   = 0xe8f00fff,

Thanks, that's pretty helpful. I'll update the patch with these values
and the corresponding description.

>> +	},
>>  	{
>>  	}
>>  };
>> @@ -1577,7 +1593,12 @@ static int __init gic_init_bases(void __iomem 
>> *dist_base,
>>  	pr_info("%d SPIs implemented\n", GIC_LINE_NR - 32);
>>  	pr_info("%d Extended SPIs implemented\n", GIC_ESPI_NR);
>> 
>> -	gic_data.rdists.gicd_typer2 = readl_relaxed(gic_data.dist_base + 
>> GICD_TYPER2);
>> +	/*
>> +	 * ThunderX1 explodes on reading GICD_TYPER2, in total violation
>> +	 * of the spec (which says that reserved addresses are RES0).
>> +	 */
>> +	if (!(gic_data.flags & FLAGS_WORKAROUND_GICD_TYPER2_TX1))
>> +		gic_data.rdists.gicd_typer2 = readl_relaxed(gic_data.dist_base + 
>> GICD_TYPER2);
> 
> You already said that checking for ArchRev of GICD_PIDR2 isn't an
> option here. Though, it could...

Once GICv3.2 starts using this register as well (because GICD_TYPER is
already completely full), we'd have to fix it again. There is also the 
thing
you hinted at in the other thread: TX1 will generate a SEA on every 
reserved
GICD registers, so we may need to protect more than just this one over 
time,
and maybe more than just in the distributor.

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

  reply index

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-24 11:10 [PATCH v3 00/32] irqchip/gic-v4: GICv4.1 architecture support Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 01/32] irqchip/gic-v3: Detect GICv4.1 supporting RVPEID Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 02/32] irqchip/gic-v3: Add GICv4.1 VPEID size discovery Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 03/32] irqchip/gic-v3: Workaround Cavium TX1 erratum when reading GICD_TYPER2 Marc Zyngier
2020-03-09 22:11   ` Robert Richter
2020-03-10 11:41     ` Marc Zyngier [this message]
2020-03-10 12:34       ` Robert Richter
2020-03-11  8:45       ` Robert Richter
2020-03-11  9:03         ` Marc Zyngier
2020-03-11  9:18           ` Robert Richter
2019-12-24 11:10 ` [PATCH v3 04/32] irqchip/gic-v3: Use SGIs without active state if offered Marc Zyngier
2019-12-28  8:56   ` Zenghui Yu
2019-12-28 10:36     ` Marc Zyngier
2019-12-30  3:50       ` Zenghui Yu
2019-12-24 11:10 ` [PATCH v3 05/32] irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation Marc Zyngier
2020-01-20 14:03   ` Zenghui Yu
2020-01-20 15:11     ` Marc Zyngier
2020-01-22  2:59   ` Zenghui Yu
2019-12-24 11:10 ` [PATCH v3 06/32] irqchip/gic-v4.1: Implement the v4.1 flavour of VMAPP Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 07/32] irqchip/gic-v4.1: Don't use the VPE proxy if RVPEID is set Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 08/32] irqchip/gic-v4.1: Implement the v4.1 flavour of VMOVP Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 09/32] irqchip/gic-v4.1: Plumb skeletal VPE irqchip Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 10/32] irqchip/gic-v4.1: Add mask/unmask doorbell callbacks Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 11/32] irqchip/gic-v4.1: Add VPE residency callback Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 12/32] irqchip/gic-v4.1: Add VPE eviction callback Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 13/32] irqchip/gic-v4.1: Add VPE INVALL callback Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 14/32] irqchip/gic-v4.1: Suppress per-VLPI doorbell Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 15/32] irqchip/gic-v4.1: Allow direct invalidation of VLPIs Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 16/32] irqchip/gic-v4.1: Advertise support v4.1 to KVM Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 17/32] irqchip/gic-v4.1: Map the ITS SGIR register page Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 18/32] irqchip/gic-v4.1: Plumb skeletal VSGI irqchip Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 19/32] irqchip/gic-v4.1: Add initial SGI configuration Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 20/32] irqchip/gic-v4.1: Plumb mask/unmask SGI callbacks Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 21/32] irqchip/gic-v4.1: Plumb get/set_irqchip_state " Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 22/32] irqchip/gic-v4.1: Plumb set_vcpu_affinity " Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 23/32] irqchip/gic-v4.1: Move doorbell management to the GICv4 abstraction layer Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 24/32] irqchip/gic-v4.1: Add VSGI allocation/teardown Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 25/32] irqchip/gic-v4.1: Add VSGI property setup Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 26/32] irqchip/gic-v4.1: Eagerly vmap vPEs Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 27/32] KVM: arm64: GICv4.1: Let doorbells be auto-enabled Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 28/32] KVM: arm64: GICv4.1: Add direct injection capability to SGI registers Marc Zyngier
2019-12-28  9:19   ` Zenghui Yu
2019-12-28 10:41     ` Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 29/32] KVM: arm64: GICv4.1: Allow SGIs to switch between HW and SW interrupts Marc Zyngier
2020-01-15  2:49   ` Shaokun Zhang
2020-01-15  3:49     ` Zenghui Yu
2020-01-15 13:32       ` Marc Zyngier
2020-01-15 13:49         ` Zenghui Yu
2020-01-16  6:13         ` Shaokun Zhang
2020-01-15 13:17     ` Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 30/32] KVM: arm64: GICv4.1: Plumb SGI implementation selection in the distributor Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 31/32] KVM: arm64: GICv4.1: Reload VLPI configuration on distributor enable/disable Marc Zyngier
2019-12-24 11:10 ` [PATCH v3 32/32] KVM: arm64: GICv4.1: Expose HW-based SGIs in debugfs Marc Zyngier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=b1b7db1f0e1c47b7d9e2dfbbe3409b77@kernel.org \
    --to=maz@kernel.org \
    --cc=Andrew.Murray@arm.com \
    --cc=jason@lakedaemon.net \
    --cc=kvmarm@lists.cs.columbia.edu \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lorenzo.pieralisi@arm.com \
    --cc=rrichter@marvell.com \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM ARM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvmarm/0 kvmarm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvmarm kvmarm/ https://lore.kernel.org/kvmarm \
		kvmarm@lists.cs.columbia.edu
	public-inbox-index kvmarm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/edu.columbia.cs.lists.kvmarm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git