KVM ARM Archive on lore.kernel.org
 help / color / Atom feed
From: Zenghui Yu <yuzenghui@huawei.com>
To: Marc Zyngier <maz@kernel.org>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
	Jason Cooper <jason@lakedaemon.net>,
	linux-kernel@vger.kernel.org,
	Robert Richter <rrichter@marvell.com>,
	Jayachandran C <jnair@marvell.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	kvmarm@lists.cs.columbia.edu
Subject: Re: [PATCH v2 03/36] irqchip/gic-v3-its: Allow LPI invalidation via the DirectLPI interface
Date: Tue, 5 Nov 2019 18:30:13 +0800
Message-ID: <6d96dfc1-723a-be1e-d4ae-39c79e7fe080@huawei.com> (raw)
In-Reply-To: <86ftj7ybg2.wl-maz@kernel.org>

Hi Marc,

On 2019/11/1 21:26, Marc Zyngier wrote:
> On Thu, 31 Oct 2019 08:49:32 +0000,
> Zenghui Yu <yuzenghui@huawei.com> wrote:
>>
>> But this patch really drives me to look through all callsites of
>> dev_event_to_col(), the abstraction which can be used _only_ with
>> physical LPI mappings.
>>
>> I find that when building the INV command, we use dev_event_to_col()
>> to find the "sync_obj" and then pass it to the following SYNC command.
>> But the "INV+SYNC" will be performed both on physical LPI and *VLPI*
>> (lpi_update_config/its_send_inv).
>> So I have two questions about the way we sending INV on VLPI:
>>
>> 1) Which "sync" command should be followed?  SYNC or VSYNC?
>> (we currently use SYNC, while the spec says, SYNC "ensures all
>> outstanding ITS operations associated with *physical* interrupts
>> for the Redistributor are globally observed ...")
>>
>> 2) The "sync_obj" we are currently using seems to be wrong.
> 
> I think you're right on both counts (where were you when I wrote the
> initial GICv4 support? ;-). I think the confusion stems from the fact

(I'm a bit late here :-).

> that there is no 'VINV' command, and we simply overlooked the sync
> object issue. It is quite likely that existing implementations don't
> care much about the difference (otherwise we'd have seen the problem
> before), but it doesn't hurt to do the right thing.
> 
> I have the following patch as part of a set of fixes that I'm about to
> post (once I get a chance to test them), let me know what you think.
> 
> 	M.
> 
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index a47ed2ba2907..75ab3716a870 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -702,6 +702,24 @@ static struct its_vpe *its_build_vmovp_cmd(struct its_node *its,
>   	return valid_vpe(its, desc->its_vmovp_cmd.vpe);
>   }
>   
> +static struct its_vpe *its_build_vinv_cmd(struct its_node *its,
> +					  struct its_cmd_block *cmd,
> +					  struct its_cmd_desc *desc)
> +{
> +	struct its_vlpi_map *map;
> +
> +	map = dev_event_to_vlpi_map(desc->its_inv_cmd.dev,
> +				    desc->its_inv_cmd.event_id);

Indeed!  I think we need this kind of abstraction for VLPI.

> +
> +	its_encode_cmd(cmd, GITS_CMD_INV);
> +	its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
> +	its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
> +
> +	its_fixup_cmd(cmd);
> +
> +	return valid_vpe(its, map->vpe);
> +}
> +
>   static u64 its_cmd_ptr_to_offset(struct its_node *its,
>   				 struct its_cmd_block *ptr)
>   {
> @@ -1068,6 +1086,20 @@ static void its_send_vinvall(struct its_node *its, struct its_vpe *vpe)
>   	its_send_single_vcommand(its, its_build_vinvall_cmd, &desc);
>   }
>   
> +static void its_send_vinv(struct its_device *dev, u32 event_id)
> +{
> +	struct its_cmd_desc desc;
> +
> +	/*
> +	 * There is no real VINV command. This is just a normal INV,
> +	 * with a VSYNC instead of a SYNC.
> +	 */
> +	desc.its_inv_cmd.dev = dev;
> +	desc.its_inv_cmd.event_id = event_id;
> +
> +	its_send_single_vcommand(dev->its, its_build_vinv_cmd, &desc);
> +}
> +
>   /*
>    * irqchip functions - assumes MSI, mostly.
>    */
> @@ -1142,8 +1174,10 @@ static void lpi_update_config(struct irq_data *d, u8 clr, u8 set)
>   	lpi_write_config(d, clr, set);
>   	if (gic_rdists->has_direct_lpi && !irqd_is_forwarded_to_vcpu(d))
>   		direct_lpi_inv(d);
> -	else
> +	else if (!irqd_is_forwarded_to_vcpu(d))
>   		its_send_inv(its_dev, its_get_event_id(d));
> +	else
> +		its_send_vinv(its_dev, its_get_event_id(d));

Yeah, this is exactly what I was having in the mind when reporting this
problem - "maybe we should have a SW emulated VINV+VSYNC for VLPI".
So I think this patch has done the right thing.

And what about the INT and CLEAR?  In response to guest's INT/CLEAR
commands, hypervisor (I mean KVM) will bother the ITS driver to send
INT/CLEAR for VLPIs.  They're both followed by SYNC and might need the
same fixes?


Thanks,
Zenghui

>   }
>   
>   static void its_vlpi_set_doorbell(struct irq_data *d, bool enable)
> 

_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

  reply index

Thread overview: 65+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-27 14:41 [PATCH v2 00/36] irqchip/gic-v4: GICv4.1 architecture support Marc Zyngier
2019-10-27 14:41 ` [PATCH v2 01/36] KVM: arm64: vgic-v4: Move the GICv4 residency flow to be driven by vcpu_load/put Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 02/36] irqchip/gic-v3-its: Factor out wait_for_syncr primitive Marc Zyngier
2019-10-28  9:20   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 03/36] irqchip/gic-v3-its: Allow LPI invalidation via the DirectLPI interface Marc Zyngier
2019-10-31  8:49   ` Zenghui Yu
2019-11-01 13:26     ` Marc Zyngier
2019-11-05 10:30       ` Zenghui Yu [this message]
2019-11-05 12:12         ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 04/36] irqchip/gic-v3-its: Make is_v4 use a TYPER copy Marc Zyngier
2019-10-28  9:34   ` Zenghui Yu
2019-10-28 10:52     ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 05/36] irqchip/gic-v3-its: Kill its->ite_size and use TYPER copy instead Marc Zyngier
2019-10-28  9:40   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 06/36] irqchip/gic-v3-its: Kill its->device_ids " Marc Zyngier
2019-10-31  6:33   ` Zenghui Yu
2019-10-31  8:30     ` Marc Zyngier
2019-10-31  9:08       ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 07/36] irqchip/gic-v3-its: Add get_vlpi_map() helper Marc Zyngier
2019-10-31  3:54   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 08/36] irqchip/gic-v3: Detect GICv4.1 supporting RVPEID Marc Zyngier
2019-10-31 11:34   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 09/36] irqchip/gic-v3: Add GICv4.1 VPEID size discovery Marc Zyngier
2019-10-31 12:02   ` Zenghui Yu
2019-11-01 15:13     ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 10/36] irqchip/gic-v3: Workaround Cavium TX1 erratum when reading GICD_TYPER2 Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 11/36] irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 12/36] irqchip/gic-v4.1: Implement the v4.1 flavour of VMAPP Marc Zyngier
2019-11-01 10:58   ` Zenghui Yu
2019-11-13  8:02   ` Zenghui Yu
2019-11-13  9:47     ` Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 13/36] irqchip/gic-v4.1: Don't use the VPE proxy if RVPEID is set Marc Zyngier
2019-11-01 11:05   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 14/36] irqchip/gic-v4.1: Implement the v4.1 flavour of VMOVP Marc Zyngier
2019-11-01 11:10   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 15/36] irqchip/gic-v4.1: Plumb skeletal VPE irqchip Marc Zyngier
2019-11-01 11:13   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 16/36] irqchip/gic-v4.1: Add mask/unmask doorbell callbacks Marc Zyngier
2019-11-01 11:23   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 17/36] irqchip/gic-v4.1: Add VPE residency callback Marc Zyngier
2019-11-01 11:34   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 18/36] irqchip/gic-v4.1: Add VPE eviction callback Marc Zyngier
2019-11-01 11:39   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 19/36] irqchip/gic-v4.1: Add VPE INVALL callback Marc Zyngier
2019-11-01 11:51   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 20/36] irqchip/gic-v4.1: Suppress per-VLPI doorbell Marc Zyngier
2019-11-01 12:17   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 21/36] irqchip/gic-v4.1: Allow direct invalidation of VLPIs Marc Zyngier
2019-11-01 12:30   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 22/36] irqchip/gic-v4.1: Advertise support v4.1 to KVM Marc Zyngier
2019-11-01 12:55   ` Zenghui Yu
2019-10-27 14:42 ` [PATCH v2 23/36] irqchip/gic-v4.1: Map the ITS SGIR register page Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 24/36] irqchip/gic-v4.1: Plumb skeletal VSGI irqchip Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 25/36] irqchip/gic-v4.1: Add initial SGI configuration Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 26/36] irqchip/gic-v4.1: Plumb mask/unmask SGI callbacks Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 27/36] irqchip/gic-v4.1: Plumb get/set_irqchip_state " Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 28/36] irqchip/gic-v4.1: Plumb set_vcpu_affinity " Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 29/36] irqchip/gic-v4.1: Move doorbell management to the GICv4 abstraction layer Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 30/36] irqchip/gic-v4.1: Add VSGI allocation/teardown Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 31/36] irqchip/gic-v4.1: Add VSGI property setup Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 32/36] irqchip/gic-v4.1: Eagerly vmap vPEs Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 33/36] KVM: arm64: GICv4.1: Let doorbells be auto-enabled Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 34/36] KVM: arm64: GICv4.1: Add direct injection capability to SGI registers Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 35/36] KVM: arm64: GICv4.1: Configure SGIs as HW interrupts Marc Zyngier
2019-10-27 14:42 ` [PATCH v2 36/36] KVM: arm64: GICv4.1: Expose HW-based SGIs in debugfs Marc Zyngier

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6d96dfc1-723a-be1e-d4ae-39c79e7fe080@huawei.com \
    --to=yuzenghui@huawei.com \
    --cc=jason@lakedaemon.net \
    --cc=jnair@marvell.com \
    --cc=kvmarm@lists.cs.columbia.edu \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lorenzo.pieralisi@arm.com \
    --cc=maz@kernel.org \
    --cc=rrichter@marvell.com \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM ARM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvmarm/0 kvmarm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvmarm kvmarm/ https://lore.kernel.org/kvmarm \
		kvmarm@lists.cs.columbia.edu
	public-inbox-index kvmarm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/edu.columbia.cs.lists.kvmarm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git