KVM ARM Archive on lore.kernel.org
 help / color / Atom feed
From: Zenghui Yu <yuzenghui@huawei.com>
To: Marc Zyngier <maz@kernel.org>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
	Jason Cooper <jason@lakedaemon.net>,
	kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
	Robert Richter <rrichter@marvell.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	kvmarm@lists.cs.columbia.edu,
	linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH v4 08/20] irqchip/gic-v4.1: Plumb get/set_irqchip_state SGI callbacks
Date: Wed, 19 Feb 2020 19:50:45 +0800
Message-ID: <8db95a86-0981-710b-6c82-be7f7f844151@huawei.com> (raw)
In-Reply-To: <19a7c193f0e4b97343e822a35f0911ed@kernel.org>

Hi Marc,

On 2020/2/18 23:31, Marc Zyngier wrote:
> Hi Zenghui,
> 
> On 2020-02-18 09:27, Marc Zyngier wrote:
>> Hi Zenghui,
>>
>> On 2020-02-18 07:00, Zenghui Yu wrote:
>>> Hi Marc,
> 
> [...]
> 
>>> There might be a race on reading the 'vpe->col_idx' against a concurrent
>>> vPE schedule (col_idx will be modified in its_vpe_set_affinity)? Will we
>>> end up accessing the GICR_VSGI* registers of the old redistributor,
>>> while the vPE is now resident on the new one? Or is it harmful?
>>
>> Very well spotted. There is a potential problem if old and new RDs are 
>> not part
>> of the same CommonLPIAff group.
>>
>>> The same question for direct_lpi_inv(), where 'vpe->col_idx' will be
>>> used in irq_to_cpuid().
>>
>> Same problem indeed. We need to ensure that no VMOVP operation can 
>> occur whilst
>> we use col_idx to access a redistributor. This means a vPE lock of 
>> some sort
>> that will protect the affinity.

Yeah, I had the same view here, a vPE level lock might help.

>> But I think there is a slightly more general problem here, which we 
>> failed to
>> see initially: the same issue exists for physical LPIs, as col_map[] 
>> can be
>> updated (its_set_affinity()) in parallel with a direct invalidate.
>>
>> The good old invalidation through the ITS does guarantee that the two 
>> operation
>> don't overlap, but direct invalidation breaks it.

Agreed!

>> Let me have a think about it.
> 
> So I've thought about it, wrote a patch, and I don't really like the 
> look of it.
> This is pretty invasive, and we end-up serializing a lot more than we 
> used to
> (the repurposing of vlpi_lock to a general "lpi mapping lock" is 
> probably too
> coarse).
> 
> It of course needs splitting over at least three patches, but it'd be 
> good if
> you could have a look (applies on top of the whole series).

So the first thing is that

1. there're races on choosing the RD against a concurrent LPI/vPE
affinity changing.

And sure, I will have a look on the following patch! But I'd first
talk about some other issues I've seen today...

2. Another potential race is on accessing the same RD by different
CPUs, which gets more obvious after introducing the GICv4.1.
We can as least take two registers for example:

  - GICR_VSGIR:
    Let's assume that vPE0 is just descheduled from CPU0 and then vPE1
    is scheduled on. CPU0 is writing its GICR_VSGIR with vpeid1 to serve
    vPE1's GICR_ISPENDR0 read trap, whilst userspace is getting vSGI's
    pending state of vPE0 (i.e., by a debugfs read) thus another CPU
    will try to write the same GICR_VSGIR with vpeid0... without waiting
    GICR_VSGIPENDR.Busy reads as 0.
    This is a CONSTRAINED UNPREDICTABLE behavior from the spec and at
    least one of the queries will fail.
  - GICR_INV{LPI,ALL}R:
    Multiple LPIs can be targeted to the same RD, thus multiple writes to
    the same GICR_INVLPIR (with different INITID, even with different V)
    can happen concurrently...

Above comes from the fact that the same redistributor can be accessed
(concurrently) by multiple CPUs but we don't have a mechanism to ensure
some extent of serialization. I also had a look at how KVM will handle
this kind of access, but

3. it looks like KVM makes the assumption that the per-RD MMIO region
will only be accessed by the associated VCPU?  But I think this's not
restricted by the architecture, we can do it better.  Or I've just
missed some important points here.


I will look at the following patch asap but may need some time to
think about all above, and do some fix if possible :-)

> diff --git a/drivers/irqchip/irq-gic-v3-its.c 
> b/drivers/irqchip/irq-gic-v3-its.c
> index 7656b353a95f..0ed286dba827 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c

[...]


Thanks,
Zenghui

_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

  reply index

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-14 14:57 [PATCH v4 00/20] irqchip/gic-v4: GICv4.1 architecture support Marc Zyngier
2020-02-14 14:57 ` [PATCH v4 01/20] irqchip/gic-v4.1: Skip absent CPUs while iterating over redistributors Marc Zyngier
2020-02-17  9:11   ` Zenghui Yu
2020-02-14 14:57 ` [PATCH v4 02/20] irqchip/gic-v3: Use SGIs without active state if offered Marc Zyngier
2020-02-17  9:18   ` Zenghui Yu
2020-02-14 14:57 ` [PATCH v4 03/20] irqchip/gic-v4.1: Advertise support v4.1 to KVM Marc Zyngier
2020-02-17  9:09   ` Zenghui Yu
2020-02-14 14:57 ` [PATCH v4 04/20] irqchip/gic-v4.1: Map the ITS SGIR register page Marc Zyngier
2020-02-20  3:17   ` Zenghui Yu
2020-02-14 14:57 ` [PATCH v4 05/20] irqchip/gic-v4.1: Plumb skeletal VSGI irqchip Marc Zyngier
2020-02-20  3:21   ` Zenghui Yu
2020-02-14 14:57 ` [PATCH v4 06/20] irqchip/gic-v4.1: Add initial SGI configuration Marc Zyngier
2020-02-18  7:25   ` Zenghui Yu
2020-02-18  9:46     ` Marc Zyngier
2020-02-20  3:25       ` Zenghui Yu
2020-02-14 14:57 ` [PATCH v4 07/20] irqchip/gic-v4.1: Plumb mask/unmask SGI callbacks Marc Zyngier
2020-02-20  3:32   ` Zenghui Yu
2020-02-14 14:57 ` [PATCH v4 08/20] irqchip/gic-v4.1: Plumb get/set_irqchip_state " Marc Zyngier
2020-02-18  7:00   ` Zenghui Yu
2020-02-18  9:27     ` Marc Zyngier
2020-02-18 15:31       ` Marc Zyngier
2020-02-19 11:50         ` Zenghui Yu [this message]
2020-02-19 15:18           ` Zenghui Yu
2020-02-20  3:11         ` Zenghui Yu
2020-02-28 19:37           ` Marc Zyngier
2020-03-01 19:00             ` Marc Zyngier
2020-03-02  8:18               ` Zenghui Yu
2020-03-02 12:09                 ` Marc Zyngier
2020-03-02 14:21                   ` Bill Barrow
2020-02-14 14:57 ` [PATCH v4 09/20] irqchip/gic-v4.1: Plumb set_vcpu_affinity " Marc Zyngier
2020-02-20  3:37   ` Zenghui Yu
2020-02-28 19:00     ` Marc Zyngier
2020-02-14 14:57 ` [PATCH v4 10/20] irqchip/gic-v4.1: Move doorbell management to the GICv4 abstraction layer Marc Zyngier
2020-02-14 14:57 ` [PATCH v4 11/20] irqchip/gic-v4.1: Add VSGI allocation/teardown Marc Zyngier
2020-02-14 14:57 ` [PATCH v4 12/20] irqchip/gic-v4.1: Add VSGI property setup Marc Zyngier
2020-02-14 14:57 ` [PATCH v4 13/20] irqchip/gic-v4.1: Eagerly vmap vPEs Marc Zyngier
2020-02-14 14:57 ` [PATCH v4 14/20] KVM: arm64: GICv4.1: Let doorbells be auto-enabled Marc Zyngier
2020-02-14 14:57 ` [PATCH v4 15/20] KVM: arm64: GICv4.1: Add direct injection capability to SGI registers Marc Zyngier
2020-02-18  8:46   ` Zenghui Yu
2020-02-18  9:41     ` Marc Zyngier
2020-02-14 14:57 ` [PATCH v4 16/20] KVM: arm64: GICv4.1: Allow SGIs to switch between HW and SW interrupts Marc Zyngier
2020-02-20  3:55   ` Zenghui Yu
2020-02-28 19:16     ` Marc Zyngier
2020-03-02  2:40       ` Zenghui Yu
2020-02-14 14:57 ` [PATCH v4 17/20] KVM: arm64: GICv4.1: Plumb SGI implementation selection in the distributor Marc Zyngier
2020-02-14 14:57 ` [PATCH v4 18/20] KVM: arm64: GICv4.1: Reload VLPI configuration on distributor enable/disable Marc Zyngier
2020-02-14 14:57 ` [PATCH v4 19/20] KVM: arm64: GICv4.1: Allow non-trapping WFI when using HW SGIs Marc Zyngier
2020-02-14 14:57 ` [PATCH v4 20/20] KVM: arm64: GICv4.1: Expose HW-based SGIs in debugfs Marc Zyngier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8db95a86-0981-710b-6c82-be7f7f844151@huawei.com \
    --to=yuzenghui@huawei.com \
    --cc=jason@lakedaemon.net \
    --cc=kvm@vger.kernel.org \
    --cc=kvmarm@lists.cs.columbia.edu \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lorenzo.pieralisi@arm.com \
    --cc=maz@kernel.org \
    --cc=rrichter@marvell.com \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM ARM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvmarm/0 kvmarm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvmarm kvmarm/ https://lore.kernel.org/kvmarm \
		kvmarm@lists.cs.columbia.edu
	public-inbox-index kvmarm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/edu.columbia.cs.lists.kvmarm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git