From: Jason Wang <jasowang@redhat.com>
To: Eli Cohen <eli@mellanox.com>
Cc: Zhu Lingshan <lingshan.zhu@intel.com>,
alex.williamson@redhat.com, mst@redhat.com, pbonzini@redhat.com,
sean.j.christopherson@intel.com, wanpengli@tencent.com,
virtualization@lists.linux-foundation.org,
netdev@vger.kernel.org, kvm@vger.kernel.org,
shahafs@mellanox.com, parav@mellanox.com
Subject: Re: [PATCH V4 4/6] vhost_vdpa: implement IRQ offloading in vhost_vdpa
Date: Wed, 29 Jul 2020 18:19:52 +0800 [thread overview]
Message-ID: <45b7e8aa-47a9-06f6-6b72-762d504adb00@redhat.com> (raw)
In-Reply-To: <20200729095503.GD35280@mtl-vdi-166.wap.labs.mlnx>
On 2020/7/29 下午5:55, Eli Cohen wrote:
> On Wed, Jul 29, 2020 at 05:21:53PM +0800, Jason Wang wrote:
>> On 2020/7/28 下午5:04, Eli Cohen wrote:
>>> On Tue, Jul 28, 2020 at 12:24:03PM +0800, Zhu Lingshan wrote:
>>>> +static void vhost_vdpa_setup_vq_irq(struct vhost_vdpa *v, int qid)
>>>> +{
>>>> + struct vhost_virtqueue *vq = &v->vqs[qid];
>>>> + const struct vdpa_config_ops *ops = v->vdpa->config;
>>>> + struct vdpa_device *vdpa = v->vdpa;
>>>> + int ret, irq;
>>>> +
>>>> + spin_lock(&vq->call_ctx.ctx_lock);
>>>> + irq = ops->get_vq_irq(vdpa, qid);
>>>> + if (!vq->call_ctx.ctx || irq == -EINVAL) {
>>>> + spin_unlock(&vq->call_ctx.ctx_lock);
>>>> + return;
>>>> + }
>>>> +
>>> If I understand correctly, this will cause these IRQs to be forwarded
>>> directly to the VCPU, e.g. will be handled by the guest/qemu.
>>
>> Yes, if it can bypassed, the interrupt will be delivered to vCPU directly.
>>
> So, usually the network driver knows how to handle interrups for its
> devices. I assume the virtio_net driver at the guest has some default
> processing but what if the underlying hardware device (such as the case
> of vdpa) needs to take some actions?
Virtio splits the bus operations out of device operations. So did the
driver.
The virtio-net driver depends on a transport driver to talk to the real
device. Usually PCI is used as the transport for the device. In this
case virtio-pci driver is in charge of dealing with irq
allocation/free/configuration and it needs to co-operate with platform
specific irqchip (virtualized by KVM) to finish the work like irq
acknowledge etc. E.g for x86, the irq offloading can only work when
there's a hardware support of virtual irqchip (APICv) then all stuffs
could be done without vmexits.
So no vendor specific part since the device and transport are all standard.
> Is there an option to do bounce the
> interrupt back to the vendor specific driver in the host so it can take
> these actions?
Currently not, but even if we can do this, I'm afraid we will lose the
performance advantage of irq bypassing.
>
>>> Does this mean that the host will not handle this interrupt? How does it
>>> work in case on level triggered interrupts?
>>
>> There's no guarantee that the KVM arch code can make sure the irq
>> bypass work for any type of irq. So if they the irq will still need
>> to be handled by host first. This means we should keep the host
>> interrupt handler as a slowpath (fallback).
>>
>>> In the case of ConnectX, I need to execute some code to acknowledge the
>>> interrupt.
>>
>> This turns out to be hard for irq bypassing to work. Is it because
>> the irq is shared or what kind of ack you need to do?
> I have an EQ which is a queue for events comming from the hardware. This
> EQ can created so it reports only completion events but I still need to
> execute code that roughly tells the device that I saw these event
> records and then arm it again so it can report more interrupts (e.g if
> more packets are received or sent). This is device specific code.
Any chance that the hardware can use MSI (which is not the case here)?
Thanks
>> Thanks
>>
>>
>>> Can you explain how this should be done?
>>>
next prev parent reply other threads:[~2020-07-29 10:20 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-28 4:23 [PATCH V4 0/6] IRQ offloading for vDPA Zhu Lingshan
2020-07-28 4:24 ` [PATCH V4 1/6] vhost: introduce vhost_vring_call Zhu Lingshan
2020-07-28 4:24 ` [PATCH V4 2/6] kvm: detect assigned device via irqbypass manager Zhu Lingshan
2020-07-28 4:24 ` [PATCH V4 3/6] vDPA: add get_vq_irq() in vdpa_config_ops Zhu Lingshan
2020-07-28 7:42 ` Jason Wang
2020-07-28 4:24 ` [PATCH V4 4/6] vhost_vdpa: implement IRQ offloading in vhost_vdpa Zhu Lingshan
2020-07-28 7:53 ` Jason Wang
[not found] ` <f3e375da-3aa8-7a0c-237c-25943667a535@intel.com>
2020-07-28 10:29 ` Jason Wang
2020-07-28 9:04 ` Eli Cohen
2020-07-29 9:21 ` Jason Wang
2020-07-29 9:55 ` Eli Cohen
2020-07-29 10:19 ` Jason Wang [this message]
2020-07-29 11:13 ` Eli Cohen
2020-07-29 14:15 ` Eli Cohen
2020-07-31 3:11 ` Jason Wang
2020-07-28 4:24 ` [PATCH V4 5/6] ifcvf: implement vdpa_config_ops.get_vq_irq() Zhu Lingshan
2020-07-28 4:24 ` [PATCH V4 6/6] irqbypass: do not start cons/prod when failed connect Zhu Lingshan
2020-07-31 3:18 ` [PATCH V4 0/6] IRQ offloading for vDPA Jason Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=45b7e8aa-47a9-06f6-6b72-762d504adb00@redhat.com \
--to=jasowang@redhat.com \
--cc=alex.williamson@redhat.com \
--cc=eli@mellanox.com \
--cc=kvm@vger.kernel.org \
--cc=lingshan.zhu@intel.com \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=parav@mellanox.com \
--cc=pbonzini@redhat.com \
--cc=sean.j.christopherson@intel.com \
--cc=shahafs@mellanox.com \
--cc=virtualization@lists.linux-foundation.org \
--cc=wanpengli@tencent.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).