From: Zenghui Yu <yuzenghui@huawei.com>
To: Marc Zyngier <maz@kernel.org>
Cc: <kvmarm@lists.cs.columbia.edu>, <linux-kernel@vger.kernel.org>,
Eric Auger <eric.auger@redhat.com>,
James Morse <james.morse@arm.com>,
Julien Thierry <julien.thierry.kdev@gmail.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
Thomas Gleixner <tglx@linutronix.de>,
Jason Cooper <jason@lakedaemon.net>,
Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
Andrew Murray <Andrew.Murray@arm.com>,
Jayachandran C <jnair@marvell.com>,
Robert Richter <rrichter@marvell.com>
Subject: Re: [PATCH v2 03/36] irqchip/gic-v3-its: Allow LPI invalidation via the DirectLPI interface
Date: Tue, 5 Nov 2019 18:30:13 +0800 [thread overview]
Message-ID: <6d96dfc1-723a-be1e-d4ae-39c79e7fe080@huawei.com> (raw)
In-Reply-To: <86ftj7ybg2.wl-maz@kernel.org>
Hi Marc,
On 2019/11/1 21:26, Marc Zyngier wrote:
> On Thu, 31 Oct 2019 08:49:32 +0000,
> Zenghui Yu <yuzenghui@huawei.com> wrote:
>>
>> But this patch really drives me to look through all callsites of
>> dev_event_to_col(), the abstraction which can be used _only_ with
>> physical LPI mappings.
>>
>> I find that when building the INV command, we use dev_event_to_col()
>> to find the "sync_obj" and then pass it to the following SYNC command.
>> But the "INV+SYNC" will be performed both on physical LPI and *VLPI*
>> (lpi_update_config/its_send_inv).
>> So I have two questions about the way we sending INV on VLPI:
>>
>> 1) Which "sync" command should be followed? SYNC or VSYNC?
>> (we currently use SYNC, while the spec says, SYNC "ensures all
>> outstanding ITS operations associated with *physical* interrupts
>> for the Redistributor are globally observed ...")
>>
>> 2) The "sync_obj" we are currently using seems to be wrong.
>
> I think you're right on both counts (where were you when I wrote the
> initial GICv4 support? ;-). I think the confusion stems from the fact
(I'm a bit late here :-).
> that there is no 'VINV' command, and we simply overlooked the sync
> object issue. It is quite likely that existing implementations don't
> care much about the difference (otherwise we'd have seen the problem
> before), but it doesn't hurt to do the right thing.
>
> I have the following patch as part of a set of fixes that I'm about to
> post (once I get a chance to test them), let me know what you think.
>
> M.
>
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index a47ed2ba2907..75ab3716a870 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -702,6 +702,24 @@ static struct its_vpe *its_build_vmovp_cmd(struct its_node *its,
> return valid_vpe(its, desc->its_vmovp_cmd.vpe);
> }
>
> +static struct its_vpe *its_build_vinv_cmd(struct its_node *its,
> + struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + struct its_vlpi_map *map;
> +
> + map = dev_event_to_vlpi_map(desc->its_inv_cmd.dev,
> + desc->its_inv_cmd.event_id);
Indeed! I think we need this kind of abstraction for VLPI.
> +
> + its_encode_cmd(cmd, GITS_CMD_INV);
> + its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> +
> + return valid_vpe(its, map->vpe);
> +}
> +
> static u64 its_cmd_ptr_to_offset(struct its_node *its,
> struct its_cmd_block *ptr)
> {
> @@ -1068,6 +1086,20 @@ static void its_send_vinvall(struct its_node *its, struct its_vpe *vpe)
> its_send_single_vcommand(its, its_build_vinvall_cmd, &desc);
> }
>
> +static void its_send_vinv(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + /*
> + * There is no real VINV command. This is just a normal INV,
> + * with a VSYNC instead of a SYNC.
> + */
> + desc.its_inv_cmd.dev = dev;
> + desc.its_inv_cmd.event_id = event_id;
> +
> + its_send_single_vcommand(dev->its, its_build_vinv_cmd, &desc);
> +}
> +
> /*
> * irqchip functions - assumes MSI, mostly.
> */
> @@ -1142,8 +1174,10 @@ static void lpi_update_config(struct irq_data *d, u8 clr, u8 set)
> lpi_write_config(d, clr, set);
> if (gic_rdists->has_direct_lpi && !irqd_is_forwarded_to_vcpu(d))
> direct_lpi_inv(d);
> - else
> + else if (!irqd_is_forwarded_to_vcpu(d))
> its_send_inv(its_dev, its_get_event_id(d));
> + else
> + its_send_vinv(its_dev, its_get_event_id(d));
Yeah, this is exactly what I was having in the mind when reporting this
problem - "maybe we should have a SW emulated VINV+VSYNC for VLPI".
So I think this patch has done the right thing.
And what about the INT and CLEAR? In response to guest's INT/CLEAR
commands, hypervisor (I mean KVM) will bother the ITS driver to send
INT/CLEAR for VLPIs. They're both followed by SYNC and might need the
same fixes?
Thanks,
Zenghui
> }
>
> static void its_vlpi_set_doorbell(struct irq_data *d, bool enable)
>
next prev parent reply other threads:[~2019-11-05 10:30 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-27 14:41 [PATCH v2 00/36] irqchip/gic-v4: GICv4.1 architecture support Marc Zyngier
2019-10-27 14:41 ` [PATCH v2 01/36] KVM: arm64: vgic-v4: Move the GICv4 residency flow to be driven by vcpu_load/put Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 02/36] irqchip/gic-v3-its: Factor out wait_for_syncr primitive Marc Zyngier
2019-10-28 9:20 ` Zenghui Yu
2019-11-20 13:21 ` [tip: irq/core] " tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 03/36] irqchip/gic-v3-its: Allow LPI invalidation via the DirectLPI interface Marc Zyngier
2019-10-31 8:49 ` Zenghui Yu
2019-11-01 13:26 ` Marc Zyngier
2019-11-05 10:30 ` Zenghui Yu [this message]
2019-11-05 12:12 ` Marc Zyngier
2019-11-20 13:21 ` [tip: irq/core] " tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 04/36] irqchip/gic-v3-its: Make is_v4 use a TYPER copy Marc Zyngier
2019-10-28 9:34 ` Zenghui Yu
2019-10-28 10:52 ` Marc Zyngier
2019-11-20 13:21 ` [tip: irq/core] " tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 05/36] irqchip/gic-v3-its: Kill its->ite_size and use TYPER copy instead Marc Zyngier
2019-10-28 9:40 ` Zenghui Yu
2019-11-20 13:21 ` [tip: irq/core] " tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 06/36] irqchip/gic-v3-its: Kill its->device_ids " Marc Zyngier
2019-10-31 6:33 ` Zenghui Yu
2019-10-31 8:30 ` Marc Zyngier
2019-10-31 9:08 ` Zenghui Yu
2019-11-20 13:21 ` [tip: irq/core] " tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 07/36] irqchip/gic-v3-its: Add get_vlpi_map() helper Marc Zyngier
2019-10-31 3:54 ` Zenghui Yu
2019-11-20 13:21 ` [tip: irq/core] irqchip/gic-v3-its: Add its_vlpi_map helpers tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 08/36] irqchip/gic-v3: Detect GICv4.1 supporting RVPEID Marc Zyngier
2019-10-31 11:34 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 09/36] irqchip/gic-v3: Add GICv4.1 VPEID size discovery Marc Zyngier
2019-10-31 12:02 ` Zenghui Yu
2019-11-01 15:13 ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 10/36] irqchip/gic-v3: Workaround Cavium TX1 erratum when reading GICD_TYPER2 Marc Zyngier
2020-03-15 9:55 ` [tip: irq/urgent] irqchip/gic-v3: Workaround Cavium erratum 38539 " tip-bot2 for Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 11/36] irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation Marc Zyngier
2019-12-24 7:10 ` Zenghui Yu
2019-12-24 9:19 ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 12/36] irqchip/gic-v4.1: Implement the v4.1 flavour of VMAPP Marc Zyngier
2019-11-01 10:58 ` Zenghui Yu
2019-11-13 8:02 ` Zenghui Yu
2019-11-13 9:47 ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 13/36] irqchip/gic-v4.1: Don't use the VPE proxy if RVPEID is set Marc Zyngier
2019-11-01 11:05 ` Zenghui Yu
2019-12-18 14:39 ` Marc Zyngier
2019-12-19 3:05 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 14/36] irqchip/gic-v4.1: Implement the v4.1 flavour of VMOVP Marc Zyngier
2019-11-01 11:10 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 15/36] irqchip/gic-v4.1: Plumb skeletal VPE irqchip Marc Zyngier
2019-11-01 11:13 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 16/36] irqchip/gic-v4.1: Add mask/unmask doorbell callbacks Marc Zyngier
2019-11-01 11:23 ` Zenghui Yu
2019-12-18 15:06 ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 17/36] irqchip/gic-v4.1: Add VPE residency callback Marc Zyngier
2019-11-01 11:34 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 18/36] irqchip/gic-v4.1: Add VPE eviction callback Marc Zyngier
2019-11-01 11:39 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 19/36] irqchip/gic-v4.1: Add VPE INVALL callback Marc Zyngier
2019-11-01 11:51 ` Zenghui Yu
2019-12-18 14:18 ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 20/36] irqchip/gic-v4.1: Suppress per-VLPI doorbell Marc Zyngier
2019-11-01 12:17 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 21/36] irqchip/gic-v4.1: Allow direct invalidation of VLPIs Marc Zyngier
2019-11-01 12:30 ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 22/36] irqchip/gic-v4.1: Advertise support v4.1 to KVM Marc Zyngier
2019-11-01 12:55 ` Zenghui Yu
2019-12-18 14:48 ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 23/36] irqchip/gic-v4.1: Map the ITS SGIR register page Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 24/36] irqchip/gic-v4.1: Plumb skeletal VSGI irqchip Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 25/36] irqchip/gic-v4.1: Add initial SGI configuration Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 26/36] irqchip/gic-v4.1: Plumb mask/unmask SGI callbacks Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 27/36] irqchip/gic-v4.1: Plumb get/set_irqchip_state " Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 28/36] irqchip/gic-v4.1: Plumb set_vcpu_affinity " Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 29/36] irqchip/gic-v4.1: Move doorbell management to the GICv4 abstraction layer Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 30/36] irqchip/gic-v4.1: Add VSGI allocation/teardown Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 31/36] irqchip/gic-v4.1: Add VSGI property setup Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 32/36] irqchip/gic-v4.1: Eagerly vmap vPEs Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 33/36] KVM: arm64: GICv4.1: Let doorbells be auto-enabled Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 34/36] KVM: arm64: GICv4.1: Add direct injection capability to SGI registers Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 35/36] KVM: arm64: GICv4.1: Configure SGIs as HW interrupts Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 36/36] KVM: arm64: GICv4.1: Expose HW-based SGIs in debugfs Marc Zyngier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6d96dfc1-723a-be1e-d4ae-39c79e7fe080@huawei.com \
--to=yuzenghui@huawei.com \
--cc=Andrew.Murray@arm.com \
--cc=eric.auger@redhat.com \
--cc=james.morse@arm.com \
--cc=jason@lakedaemon.net \
--cc=jnair@marvell.com \
--cc=julien.thierry.kdev@gmail.com \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-kernel@vger.kernel.org \
--cc=lorenzo.pieralisi@arm.com \
--cc=maz@kernel.org \
--cc=rrichter@marvell.com \
--cc=suzuki.poulose@arm.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).