From: Marc Zyngier <maz@kernel.org>
To: Zenghui Yu <yuzenghui@huawei.com>
Cc: <kvmarm@lists.cs.columbia.edu>, <linux-kernel@vger.kernel.org>,
Eric Auger <eric.auger@redhat.com>,
James Morse <james.morse@arm.com>,
Julien Thierry <julien.thierry.kdev@gmail.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
Thomas Gleixner <tglx@linutronix.de>,
Jason Cooper <jason@lakedaemon.net>,
Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
Andrew Murray <andrew.murray@arm.com>,
Jayachandran C <jnair@marvell.com>,
Robert Richter <rrichter@marvell.com>
Subject: Re: [PATCH v2 03/36] irqchip/gic-v3-its: Allow LPI invalidation via the DirectLPI interface
Date: Tue, 05 Nov 2019 13:21:34 +0109 [thread overview]
Message-ID: <796315fdd5a06cdce9e1546ff2c34433@www.loen.fr> (raw)
In-Reply-To: <6d96dfc1-723a-be1e-d4ae-39c79e7fe080@huawei.com>
On 2019-11-05 11:39, Zenghui Yu wrote:
> Hi Marc,
>
> On 2019/11/1 21:26, Marc Zyngier wrote:
>> On Thu, 31 Oct 2019 08:49:32 +0000,
>> Zenghui Yu <yuzenghui@huawei.com> wrote:
>>>
>>> But this patch really drives me to look through all callsites of
>>> dev_event_to_col(), the abstraction which can be used _only_ with
>>> physical LPI mappings.
>>>
>>> I find that when building the INV command, we use
>>> dev_event_to_col()
>>> to find the "sync_obj" and then pass it to the following SYNC
>>> command.
>>> But the "INV+SYNC" will be performed both on physical LPI and
>>> *VLPI*
>>> (lpi_update_config/its_send_inv).
>>> So I have two questions about the way we sending INV on VLPI:
>>>
>>> 1) Which "sync" command should be followed? SYNC or VSYNC?
>>> (we currently use SYNC, while the spec says, SYNC "ensures all
>>> outstanding ITS operations associated with *physical* interrupts
>>> for the Redistributor are globally observed ...")
>>>
>>> 2) The "sync_obj" we are currently using seems to be wrong.
>> I think you're right on both counts (where were you when I wrote the
>> initial GICv4 support? ;-). I think the confusion stems from the
>> fact
>
> (I'm a bit late here :-).
>
>> that there is no 'VINV' command, and we simply overlooked the sync
>> object issue. It is quite likely that existing implementations don't
>> care much about the difference (otherwise we'd have seen the problem
>> before), but it doesn't hurt to do the right thing.
>> I have the following patch as part of a set of fixes that I'm about
>> to
>> post (once I get a chance to test them), let me know what you think.
>>
>> M.
>> diff --git a/drivers/irqchip/irq-gic-v3-its.c
>> b/drivers/irqchip/irq-gic-v3-its.c
>> index a47ed2ba2907..75ab3716a870 100644
>> --- a/drivers/irqchip/irq-gic-v3-its.c
>> +++ b/drivers/irqchip/irq-gic-v3-its.c
>> @@ -702,6 +702,24 @@ static struct its_vpe
>> *its_build_vmovp_cmd(struct its_node *its,
>> return valid_vpe(its, desc->its_vmovp_cmd.vpe);
>> }
>> +static struct its_vpe *its_build_vinv_cmd(struct its_node *its,
>> + struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + struct its_vlpi_map *map;
>> +
>> + map = dev_event_to_vlpi_map(desc->its_inv_cmd.dev,
>> + desc->its_inv_cmd.event_id);
>
> Indeed! I think we need this kind of abstraction for VLPI.
Yeah, I finally realised we'd needed something like that, and made
it part of the get_vlpi_map() patch.
>
>> +
>> + its_encode_cmd(cmd, GITS_CMD_INV);
>> + its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> +
>> + return valid_vpe(its, map->vpe);
>> +}
>> +
>> static u64 its_cmd_ptr_to_offset(struct its_node *its,
>> struct its_cmd_block *ptr)
>> {
>> @@ -1068,6 +1086,20 @@ static void its_send_vinvall(struct its_node
>> *its, struct its_vpe *vpe)
>> its_send_single_vcommand(its, its_build_vinvall_cmd, &desc);
>> }
>> +static void its_send_vinv(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + /*
>> + * There is no real VINV command. This is just a normal INV,
>> + * with a VSYNC instead of a SYNC.
>> + */
>> + desc.its_inv_cmd.dev = dev;
>> + desc.its_inv_cmd.event_id = event_id;
>> +
>> + its_send_single_vcommand(dev->its, its_build_vinv_cmd, &desc);
>> +}
>> +
>> /*
>> * irqchip functions - assumes MSI, mostly.
>> */
>> @@ -1142,8 +1174,10 @@ static void lpi_update_config(struct irq_data
>> *d, u8 clr, u8 set)
>> lpi_write_config(d, clr, set);
>> if (gic_rdists->has_direct_lpi && !irqd_is_forwarded_to_vcpu(d))
>> direct_lpi_inv(d);
>> - else
>> + else if (!irqd_is_forwarded_to_vcpu(d))
>> its_send_inv(its_dev, its_get_event_id(d));
>> + else
>> + its_send_vinv(its_dev, its_get_event_id(d));
>
> Yeah, this is exactly what I was having in the mind when reporting
> this
> problem - "maybe we should have a SW emulated VINV+VSYNC for VLPI".
> So I think this patch has done the right thing.
>
> And what about the INT and CLEAR? In response to guest's INT/CLEAR
> commands, hypervisor (I mean KVM) will bother the ITS driver to send
> INT/CLEAR for VLPIs. They're both followed by SYNC and might need
> the
> same fixes?
Yup. Please see this series[1], which has grown quite a few fixups,
including some pretty old ones (I've just pushed an update, and should
post it shortly).
Thanks,
M.
[1]
https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=irq/gic-5.5-wip
--
Jazz is not dead. It just smells funny...
next prev parent reply other threads:[~2019-11-05 12:12 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-27 14:41 [PATCH v2 00/36] irqchip/gic-v4: GICv4.1 architecture support Marc Zyngier
2019-10-27 14:41 ` [PATCH v2 01/36] KVM: arm64: vgic-v4: Move the GICv4 residency flow to be driven by vcpu_load/put Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 02/36] irqchip/gic-v3-its: Factor out wait_for_syncr primitive Marc Zyngier
2019-10-28 9:20 ` Zenghui Yu
2019-11-20 13:21 ` [tip: irq/core] " tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 03/36] irqchip/gic-v3-its: Allow LPI invalidation via the DirectLPI interface Marc Zyngier
2019-10-31 8:49 ` Zenghui Yu
2019-11-01 13:26 ` Marc Zyngier
2019-11-05 10:30 ` Zenghui Yu
2019-11-05 12:12 ` Marc Zyngier [this message]
2019-11-20 13:21 ` [tip: irq/core] " tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 04/36] irqchip/gic-v3-its: Make is_v4 use a TYPER copy Marc Zyngier
2019-10-28 9:34 ` Zenghui Yu
2019-10-28 10:52 ` Marc Zyngier
2019-11-20 13:21 ` [tip: irq/core] " tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 05/36] irqchip/gic-v3-its: Kill its->ite_size and use TYPER copy instead Marc Zyngier
2019-10-28 9:40 ` Zenghui Yu
2019-11-20 13:21 ` [tip: irq/core] " tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 06/36] irqchip/gic-v3-its: Kill its->device_ids " Marc Zyngier
2019-10-31 6:33 ` Zenghui Yu
2019-10-31 8:30 ` Marc Zyngier
2019-10-31 9:08 ` Zenghui Yu
2019-11-20 13:21 ` [tip: irq/core] " tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 07/36] irqchip/gic-v3-its: Add get_vlpi_map() helper Marc Zyngier
2019-10-31 3:54 ` Zenghui Yu
2019-11-20 13:21 ` [tip: irq/core] irqchip/gic-v3-its: Add its_vlpi_map helpers tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 08/36] irqchip/gic-v3: Detect GICv4.1 supporting RVPEID Marc Zyngier
2019-10-31 11:34 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 09/36] irqchip/gic-v3: Add GICv4.1 VPEID size discovery Marc Zyngier
2019-10-31 12:02 ` Zenghui Yu
2019-11-01 15:13 ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 10/36] irqchip/gic-v3: Workaround Cavium TX1 erratum when reading GICD_TYPER2 Marc Zyngier
2020-03-15 9:55 ` [tip: irq/urgent] irqchip/gic-v3: Workaround Cavium erratum 38539 " tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 11/36] irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation Marc Zyngier
2019-12-24 7:10 ` Zenghui Yu
2019-12-24 9:19 ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 12/36] irqchip/gic-v4.1: Implement the v4.1 flavour of VMAPP Marc Zyngier
2019-11-01 10:58 ` Zenghui Yu
2019-11-13 8:02 ` Zenghui Yu
2019-11-13 9:47 ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 13/36] irqchip/gic-v4.1: Don't use the VPE proxy if RVPEID is set Marc Zyngier
2019-11-01 11:05 ` Zenghui Yu
2019-12-18 14:39 ` Marc Zyngier
2019-12-19 3:05 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 14/36] irqchip/gic-v4.1: Implement the v4.1 flavour of VMOVP Marc Zyngier
2019-11-01 11:10 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 15/36] irqchip/gic-v4.1: Plumb skeletal VPE irqchip Marc Zyngier
2019-11-01 11:13 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 16/36] irqchip/gic-v4.1: Add mask/unmask doorbell callbacks Marc Zyngier
2019-11-01 11:23 ` Zenghui Yu
2019-12-18 15:06 ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 17/36] irqchip/gic-v4.1: Add VPE residency callback Marc Zyngier
2019-11-01 11:34 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 18/36] irqchip/gic-v4.1: Add VPE eviction callback Marc Zyngier
2019-11-01 11:39 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 19/36] irqchip/gic-v4.1: Add VPE INVALL callback Marc Zyngier
2019-11-01 11:51 ` Zenghui Yu
2019-12-18 14:18 ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 20/36] irqchip/gic-v4.1: Suppress per-VLPI doorbell Marc Zyngier
2019-11-01 12:17 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 21/36] irqchip/gic-v4.1: Allow direct invalidation of VLPIs Marc Zyngier
2019-11-01 12:30 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 22/36] irqchip/gic-v4.1: Advertise support v4.1 to KVM Marc Zyngier
2019-11-01 12:55 ` Zenghui Yu
2019-12-18 14:48 ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 23/36] irqchip/gic-v4.1: Map the ITS SGIR register page Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 24/36] irqchip/gic-v4.1: Plumb skeletal VSGI irqchip Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 25/36] irqchip/gic-v4.1: Add initial SGI configuration Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 26/36] irqchip/gic-v4.1: Plumb mask/unmask SGI callbacks Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 27/36] irqchip/gic-v4.1: Plumb get/set_irqchip_state " Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 28/36] irqchip/gic-v4.1: Plumb set_vcpu_affinity " Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 29/36] irqchip/gic-v4.1: Move doorbell management to the GICv4 abstraction layer Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 30/36] irqchip/gic-v4.1: Add VSGI allocation/teardown Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 31/36] irqchip/gic-v4.1: Add VSGI property setup Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 32/36] irqchip/gic-v4.1: Eagerly vmap vPEs Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 33/36] KVM: arm64: GICv4.1: Let doorbells be auto-enabled Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 34/36] KVM: arm64: GICv4.1: Add direct injection capability to SGI registers Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 35/36] KVM: arm64: GICv4.1: Configure SGIs as HW interrupts Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 36/36] KVM: arm64: GICv4.1: Expose HW-based SGIs in debugfs Marc Zyngier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=796315fdd5a06cdce9e1546ff2c34433@www.loen.fr \
--to=maz@kernel.org \
--cc=andrew.murray@arm.com \
--cc=eric.auger@redhat.com \
--cc=james.morse@arm.com \
--cc=jason@lakedaemon.net \
--cc=jnair@marvell.com \
--cc=julien.thierry.kdev@gmail.com \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-kernel@vger.kernel.org \
--cc=lorenzo.pieralisi@arm.com \
--cc=rrichter@marvell.com \
--cc=suzuki.poulose@arm.com \
--cc=tglx@linutronix.de \
--cc=yuzenghui@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).