From: Marc Zyngier <maz@kernel.org>
To: Shenming Lu <lushenming@huawei.com>
Cc: James Morse <james.morse@arm.com>,
Julien Thierry <julien.thierry.kdev@gmail.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
Eric Auger <eric.auger@redhat.com>,
linux-arm-kernel@lists.infradead.org,
kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org,
linux-kernel@vger.kernel.org,
Christoffer Dall <christoffer.dall@arm.com>,
Alex Williamson <alex.williamson@redhat.com>,
Kirti Wankhede <kwankhede@nvidia.com>,
Cornelia Huck <cohuck@redhat.com>, Neo Jia <cjia@nvidia.com>,
wanghaibin.wang@huawei.com, yuzenghui@huawei.com
Subject: Re: [RFC PATCH v1 3/4] KVM: arm64: GICv4.1: Restore VLPI's pending state to physical side
Date: Tue, 24 Nov 2020 08:44:59 +0000 [thread overview]
Message-ID: <2d2bcae4f871d239a1af50362f5c11a4@kernel.org> (raw)
In-Reply-To: <b03edcf2-2950-572f-fd31-601d8d766c80@huawei.com>
On 2020-11-24 08:10, Shenming Lu wrote:
> On 2020/11/23 17:27, Marc Zyngier wrote:
>> On 2020-11-23 06:54, Shenming Lu wrote:
>>> From: Zenghui Yu <yuzenghui@huawei.com>
>>>
>>> When setting the forwarding path of a VLPI, it is more consistent to
>>
>> I'm not sure it is more consistent. It is a *new* behaviour, because
>> it only
>> matters for migration, which has been so far unsupported.
>
> Alright, consistent may not be accurate...
> But I have doubt that whether there is really no need to transfer the
> pending states
> from kvm'vgic to VPT in set_forwarding regardless of migration, and the
> similar
> for unset_forwarding.
If you have to transfer that state outside of the a save/restore, it
means that
you have missed the programming of the PCI endpoint. This is an
established
restriction that the MSI programming must occur *after* the translation
has
been established using MAPI/MAPTI (see the large comment at the
beginning of
vgic-v4.c).
If you want to revisit this, fair enough. But you will need a lot more
than
just opportunistically transfer the pending state.
>
>>
>>> also transfer the pending state from irq->pending_latch to VPT
>>> (especially
>>> in migration, the pending states of VLPIs are restored into kvm’s
>>> vgic
>>> first). And we currently send "INT+VSYNC" to trigger a VLPI to
>>> pending.
>>>
>>> Signed-off-by: Zenghui Yu <yuzenghui@huawei.com>
>>> Signed-off-by: Shenming Lu <lushenming@huawei.com>
>>> ---
>>> arch/arm64/kvm/vgic/vgic-v4.c | 12 ++++++++++++
>>> 1 file changed, 12 insertions(+)
>>>
>>> diff --git a/arch/arm64/kvm/vgic/vgic-v4.c
>>> b/arch/arm64/kvm/vgic/vgic-v4.c
>>> index b5fa73c9fd35..cc3ab9cea182 100644
>>> --- a/arch/arm64/kvm/vgic/vgic-v4.c
>>> +++ b/arch/arm64/kvm/vgic/vgic-v4.c
>>> @@ -418,6 +418,18 @@ int kvm_vgic_v4_set_forwarding(struct kvm *kvm,
>>> int virq,
>>> irq->host_irq = virq;
>>> atomic_inc(&map.vpe->vlpi_count);
>>>
>>> + /* Transfer pending state */
>>> + ret = irq_set_irqchip_state(irq->host_irq,
>>> + IRQCHIP_STATE_PENDING,
>>> + irq->pending_latch);
>>> + WARN_RATELIMIT(ret, "IRQ %d", irq->host_irq);
>>> +
>>> + /*
>>> + * Let it be pruned from ap_list later and don't bother
>>> + * the List Register.
>>> + */
>>> + irq->pending_latch = false;
>>
>> It occurs to me that calling into irq_set_irqchip_state() for a large
>> number of interrupts can take a significant amount of time. It is also
>> odd that you dump the VPT with the VPE unmapped, but rely on the VPE
>> being mapped for the opposite operation.
>>
>> Shouldn't these be symmetric, all performed while the VPE is unmapped?
>> It would also save a lot of ITS traffic.
>>
>
> My thought was to use the existing interface directly without
> unmapping...
>
> If you want to unmap the vPE and poke the VPT here, as I said in the
> cover
> letter, set/unset_forwarding might also be called when all devices are
> running
> at normal run time, in which case the unmapping of the vPE is not
> allowed...
No, I'm suggesting that you don't do anything here, but instead as a
by-product
of restoring the ITS tables. What goes wrong if you use the
KVM_DEV_ARM_ITS_RESTORE_TABLE backend instead?
> Another possible solution is to add a new dedicated interface to QEMU
> to transfer
> these pending states to HW in GIC VM state change handler corresponding
> to
> save_pending_tables?
Userspace has no way to know we use GICv4, and I intend to keep it
completely out of the loop. The API is already pretty tortuous, and
I really don't want to add any extra complexity to it.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
next prev parent reply other threads:[~2020-11-24 8:45 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-23 6:54 [RFC PATCH v1 0/4] KVM: arm64: Add VLPI migration support on GICv4.1 Shenming Lu
2020-11-23 6:54 ` [RFC PATCH v1 1/4] irqchip/gic-v4.1: Plumb get_irqchip_state VLPI callback Shenming Lu
2020-11-23 9:01 ` Marc Zyngier
2020-11-24 7:38 ` Shenming Lu
2020-11-24 8:08 ` Marc Zyngier
2020-11-28 7:19 ` luojiaxing
2020-11-28 10:18 ` Marc Zyngier
2020-12-01 9:38 ` luojiaxing
2020-12-01 10:58 ` Marc Zyngier
2020-11-23 6:54 ` [RFC PATCH v1 2/4] KVM: arm64: GICv4.1: Try to save hw pending state in save_pending_tables Shenming Lu
2020-11-23 9:18 ` Marc Zyngier
2020-11-24 7:40 ` Shenming Lu
2020-11-24 8:26 ` Marc Zyngier
2020-11-24 13:10 ` Shenming Lu
2020-11-23 6:54 ` [RFC PATCH v1 3/4] KVM: arm64: GICv4.1: Restore VLPI's pending state to physical side Shenming Lu
2020-11-23 9:27 ` Marc Zyngier
2020-11-24 8:10 ` Shenming Lu
2020-11-24 8:44 ` Marc Zyngier [this message]
2020-11-24 13:12 ` Shenming Lu
2020-11-30 7:23 ` Shenming Lu
2020-12-01 10:55 ` Marc Zyngier
2020-12-01 11:40 ` Shenming Lu
2020-12-01 11:50 ` Marc Zyngier
2020-12-01 12:15 ` Shenming Lu
2020-12-08 8:25 ` Shenming Lu
2020-12-16 10:35 ` Auger Eric
2020-12-17 4:19 ` Shenming Lu
2020-11-23 6:54 ` [RFC PATCH v1 4/4] KVM: arm64: GICv4.1: Give a chance to save VLPI's pending state Shenming Lu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2d2bcae4f871d239a1af50362f5c11a4@kernel.org \
--to=maz@kernel.org \
--cc=alex.williamson@redhat.com \
--cc=christoffer.dall@arm.com \
--cc=cjia@nvidia.com \
--cc=cohuck@redhat.com \
--cc=eric.auger@redhat.com \
--cc=james.morse@arm.com \
--cc=julien.thierry.kdev@gmail.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=kwankhede@nvidia.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lushenming@huawei.com \
--cc=suzuki.poulose@arm.com \
--cc=wanghaibin.wang@huawei.com \
--cc=yuzenghui@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).