From: Auger Eric <eric.auger@redhat.com>
To: "Raslan, KarimAllah" <karahmed@amazon.de>,
"marc.zyngier@arm.com" <marc.zyngier@arm.com>,
"yuzenghui@huawei.com" <yuzenghui@huawei.com>,
"christoffer.dall@arm.com" <christoffer.dall@arm.com>,
"andre.przywara@arm.com" <andre.przywara@arm.com>
Cc: "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"kvmarm@lists.cs.columbia.edu" <kvmarm@lists.cs.columbia.edu>,
"james.morse@arm.com" <james.morse@arm.com>,
"mst@redhat.com" <mst@redhat.com>,
"suzuki.poulose@arm.com" <suzuki.poulose@arm.com>,
"pbonzini@redhat.com" <pbonzini@redhat.com>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
"julien.thierry@arm.com" <julien.thierry@arm.com>,
"rkrcmar@redhat.com" <rkrcmar@redhat.com>,
"wanghaibin.wang@huawei.com" <wanghaibin.wang@huawei.com>
Subject: Re: [RFC PATCH] KVM: arm/arm64: Enable direct irqfd MSI injection
Date: Sun, 17 Mar 2019 19:05:10 +0100 [thread overview]
Message-ID: <f53253f1-aa5e-3731-a1cd-bd14fb0509a8@redhat.com> (raw)
In-Reply-To: <1552834226.19343.1.camel@amazon.de>
Hi,
On 3/17/19 3:50 PM, Raslan, KarimAllah wrote:
> On Sun, 2019-03-17 at 14:36 +0000, Zenghui Yu wrote:
>> Currently, IRQFD on arm still uses the deferred workqueue mechanism
>> to inject interrupts into guest, which will likely lead to a busy
>> context-switching from/to the kworker thread. This overhead is for
>> no purpose (only in my view ...) and will result in an interrupt
>> performance degradation.
>>
>> Implement kvm_arch_set_irq_inatomic() for arm/arm64 to support direct
>> irqfd MSI injection, by which we can get rid of the annoying latency.
>> As a result, irqfd MSI intensive scenarios (e.g., DPDK with high packet
>> processing workloads) will benefit from it.
>>
>> Signed-off-by: Zenghui Yu <yuzenghui@huawei.com>
>> ---
>>
>> It seems that only MSI will follow the IRQFD path, did I miss something?
>>
>> This patch is still under test and sent out for early feedback. If I have
>> any mis-understanding, please fix me up and let me know. Thanks!
>>
>> ---
>> virt/kvm/arm/vgic/trace.h | 22 ++++++++++++++++++++++
>> virt/kvm/arm/vgic/vgic-irqfd.c | 21 +++++++++++++++++++++
>> 2 files changed, 43 insertions(+)
>>
>> diff --git a/virt/kvm/arm/vgic/trace.h b/virt/kvm/arm/vgic/trace.h
>> index 55fed77..bc1f4db 100644
>> --- a/virt/kvm/arm/vgic/trace.h
>> +++ b/virt/kvm/arm/vgic/trace.h
>> @@ -27,6 +27,28 @@
>> __entry->vcpu_id, __entry->irq, __entry->level)
>> );
>>
>> +TRACE_EVENT(kvm_arch_set_irq_inatomic,
>> + TP_PROTO(u32 gsi, u32 type, int level, int irq_source_id),
>> + TP_ARGS(gsi, type, level, irq_source_id),
>> +
>> + TP_STRUCT__entry(
>> + __field( u32, gsi )
>> + __field( u32, type )
>> + __field( int, level )
>> + __field( int, irq_source_id )
>> + ),
>> +
>> + TP_fast_assign(
>> + __entry->gsi = gsi;
>> + __entry->type = type;
>> + __entry->level = level;
>> + __entry->irq_source_id = irq_source_id;
>> + ),
>> +
>> + TP_printk("gsi %u type %u level %d source %d", __entry->gsi,
>> + __entry->type, __entry->level, __entry->irq_source_id)
>> +);
>> +
>> #endif /* _TRACE_VGIC_H */
>>
>> #undef TRACE_INCLUDE_PATH
>> diff --git a/virt/kvm/arm/vgic/vgic-irqfd.c b/virt/kvm/arm/vgic/vgic-irqfd.c
>> index 99e026d..4cfc3f4 100644
>> --- a/virt/kvm/arm/vgic/vgic-irqfd.c
>> +++ b/virt/kvm/arm/vgic/vgic-irqfd.c
>> @@ -19,6 +19,7 @@
>> #include <trace/events/kvm.h>
>> #include <kvm/arm_vgic.h>
>> #include "vgic.h"
>> +#include "trace.h"
>>
>> /**
>> * vgic_irqfd_set_irq: inject the IRQ corresponding to the
>> @@ -105,6 +106,26 @@ int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e,
>> return vgic_its_inject_msi(kvm, &msi);
>> }
>>
>> +/**
>> + * kvm_arch_set_irq_inatomic: fast-path for irqfd injection
>> + *
>> + * Currently only direct MSI injecton is supported.
>> + */
>> +int kvm_arch_set_irq_inatomic(struct kvm_kernel_irq_routing_entry *e,
>> + struct kvm *kvm, int irq_source_id, int level,
>> + bool line_status)
>> +{
>> + int ret;
>> +
>> + trace_kvm_arch_set_irq_inatomic(e->gsi, e->type, level, irq_source_id);
>> +
>> + if (unlikely(e->type != KVM_IRQ_ROUTING_MSI))
>> + return -EWOULDBLOCK;
>> +
>> + ret = kvm_set_msi(e, kvm, irq_source_id, level, line_status);
>
> The implementation of kvm_set_msi is not atomic. There is a mutex held in one
> of the execution paths. That is why it can not be used directly in this atomic
> context.
Yep, vgic_its_trigger_msi call is protected by the its->its_lock.
Thanks
Eric
>
>> + return ret;
>> +}
>> +
>> int kvm_vgic_setup_default_irq_routing(struct kvm *kvm)
>> {
>> struct kvm_irq_routing_entry *entries;
>
>
>
> Amazon Development Center Germany GmbH
> Krausenstr. 38
> 10117 Berlin
> Geschaeftsfuehrer: Christian Schlaeger, Ralf Herbrich
> Ust-ID: DE 289 237 879
> Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
>
next prev parent reply other threads:[~2019-03-17 18:05 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-03-17 14:36 [RFC PATCH] KVM: arm/arm64: Enable direct irqfd MSI injection Zenghui Yu
2019-03-17 14:50 ` Raslan, KarimAllah
2019-03-17 18:05 ` Auger Eric [this message]
2019-03-17 19:35 ` Marc Zyngier
2019-03-18 13:30 ` Marc Zyngier
2019-05-15 16:38 ` Andre Przywara
2019-05-16 7:21 ` Marc Zyngier
2019-05-20 15:31 ` Zenghui Yu
2019-05-20 18:00 ` Raslan, KarimAllah
2019-03-19 1:09 ` Zenghui Yu
2019-03-19 10:01 ` Marc Zyngier
2019-03-19 15:59 ` Zenghui Yu
2019-03-19 16:57 ` Marc Zyngier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f53253f1-aa5e-3731-a1cd-bd14fb0509a8@redhat.com \
--to=eric.auger@redhat.com \
--cc=andre.przywara@arm.com \
--cc=christoffer.dall@arm.com \
--cc=james.morse@arm.com \
--cc=julien.thierry@arm.com \
--cc=karahmed@amazon.de \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=marc.zyngier@arm.com \
--cc=mst@redhat.com \
--cc=pbonzini@redhat.com \
--cc=rkrcmar@redhat.com \
--cc=suzuki.poulose@arm.com \
--cc=wanghaibin.wang@huawei.com \
--cc=yuzenghui@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).