All of lore.kernel.org
 help / color / mirror / Atom feed
From: Peter Xu <peterx@redhat.com>
To: Alex Williamson <alex.williamson@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-devel@nongnu.org, Eric Auger <eric.auger@redhat.com>
Subject: Re: [PATCH v2 4/5] KVM: Kick resamplefd for split kernel irqchip
Date: Mon, 9 Mar 2020 20:38:08 -0400	[thread overview]
Message-ID: <20200310003808.GC326977@xz-x1> (raw)
In-Reply-To: <20200309163359.45ac9c3e@w520.home>

On Mon, Mar 09, 2020 at 04:33:59PM -0600, Alex Williamson wrote:

[...]

> > > diff --git a/hw/intc/ioapic.c b/hw/intc/ioapic.c
> > > index 15747fe2c2..13921b333d 100644
> > > --- a/hw/intc/ioapic.c
> > > +++ b/hw/intc/ioapic.c
> > > @@ -236,8 +236,27 @@ void ioapic_eoi_broadcast(int vector)
> > >          for (n = 0; n < IOAPIC_NUM_PINS; n++) {
> > >              entry = s->ioredtbl[n];
> > >  
> > > -            if ((entry & IOAPIC_VECTOR_MASK) != vector ||
> > > -                ((entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1) != IOAPIC_TRIGGER_LEVEL) {
> > > +            if ((entry & IOAPIC_VECTOR_MASK) != vector) {
> > > +                continue;
> > > +            }
> > > +
> > > +            /*
> > > +             * When IOAPIC is in the userspace while APIC is still in
> > > +             * the kernel (i.e., split irqchip), we have a trick to
> > > +             * kick the resamplefd logic for registered irqfds from
> > > +             * userspace to deactivate the IRQ.  When that happens, it
> > > +             * means the irq bypassed userspace IOAPIC (so the irr and
> > > +             * remote-irr of the table entry should be bypassed too
> > > +             * even if interrupt come), then we don't need to clear
> > > +             * the remote-IRR and check irr again because they'll
> > > +             * always be zeros.
> > > +             */
> > > +            if (kvm_resample_fd_notify(n)) {
> > > +                continue;
> > > +            }  
> > 
> > It seems the problem I reported is here.  In my configuration virtio-blk
> > and an assigned e1000e share an interrupt.  virtio-blk is initializing
> > and apparently triggers an interrupt.  The vfio-pci device is
> > configured for INTx though not active yet, but kvm_resample_fd_notify()
> > kicks the fd here, so we continue.  If I remove the continue here both
> > devices seem to work, but I don't claim to understand the condition
> > we're trying to continue for here yet.  This series needs more testing
> > with shared interrupts.  Thanks,
> 
> I'm also curious how this ended up between testing whether the vector
> is masked and testing that it's level triggered.  We shouldn't have any
> edge triggered resamplers.

We had a similar discussion in V1 (with Paolo):

https://patchwork.kernel.org/patch/11407441/#23190891

So my understanding is that VFIO will unmask the intx IRQ when it
comes, and register the resamplefd too, no matter whether it's level
triggered (at least from what the code does).  Am I right?

> I find however that if I move the resampler
> notify to after the remote IRR test, my NIC gets starved of interrupts.
> So empirically, it seems kvm_resample_fd_notify() should be a void
> function called unconditionally between the original mask+level check
> removed above and the IRR check below.  Thanks,

Yes IMHO we can't move that notify() to be after the remote IRR check
because the IRR and remote IRR will be completely bypassed for the
assigned device.  In other words, even if the interrupt has arrived
for the assigned device, both IRR and remote IRR should always be zero
(assuming the virtio-blk device doesn't generate any IRQ).  So we
probably still need to do the notify even if remote-irr is not set.

Thanks,

-- 
Peter Xu



  reply	other threads:[~2020-03-10  0:39 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-28 16:14 [PATCH v2 0/5] vfio/pci: Fix up breakage against split irqchip and INTx Peter Xu
2020-02-28 16:14 ` [PATCH v2 1/5] vfio/pci: Disable INTx fast path if using split irqchip Peter Xu
2020-02-28 16:15 ` [PATCH v2 2/5] vfio/pci: Use kvm_irqchip_add_irqfd_notifier_gsi() for irqfds Peter Xu
2020-02-28 16:15 ` [PATCH v2 3/5] KVM: Pass EventNotifier into kvm_irqchip_assign_irqfd Peter Xu
2020-02-28 16:15 ` [PATCH v2 4/5] KVM: Kick resamplefd for split kernel irqchip Peter Xu
2020-03-02 15:07   ` Auger Eric
2020-03-05 23:58   ` Alex Williamson
2020-03-06  0:43     ` Peter Xu
2020-03-09 21:04       ` Alex Williamson
2020-03-09 22:10   ` Alex Williamson
2020-03-09 22:33     ` Alex Williamson
2020-03-10  0:38       ` Peter Xu [this message]
2020-03-10  1:54         ` Alex Williamson
2020-03-09 23:28     ` Peter Xu
2020-02-28 16:15 ` [PATCH v2 5/5] Revert "vfio/pci: Disable INTx fast path if using split irqchip" Peter Xu
2020-03-02 15:10   ` Auger Eric
2020-03-02 15:09 ` [PATCH v2 0/5] vfio/pci: Fix up breakage against split irqchip and INTx Auger Eric

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200310003808.GC326977@xz-x1 \
    --to=peterx@redhat.com \
    --cc=alex.williamson@redhat.com \
    --cc=cohuck@redhat.com \
    --cc=eric.auger@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.