From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756646AbbJ2DLH (ORCPT ); Wed, 28 Oct 2015 23:11:07 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35535 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756187AbbJ2DLE (ORCPT ); Wed, 28 Oct 2015 23:11:04 -0400 Message-ID: <1446088263.8018.435.camel@redhat.com> Subject: Re: [RFC PATCH] VFIO: Add a parameter to force nonthread IRQ From: Alex Williamson To: Paolo Bonzini Cc: Yunhong Jiang , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Marcelo Tosatti Date: Wed, 28 Oct 2015 21:11:03 -0600 In-Reply-To: <5631003C.1050508@redhat.com> References: <1445908801-14732-1-git-send-email-yunhong.jiang@linux.intel.com> <1445917034.8018.220.camel@redhat.com> <20151027063501.GA22054@jnakajim-build> <562F43F8.1040101@redhat.com> <20151027212648.GA22916@jnakajim-build> <56301A87.9030907@redhat.com> <1446048028.8018.387.camel@redhat.com> <5631003C.1050508@redhat.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 2015-10-28 at 18:05 +0100, Paolo Bonzini wrote: > > On 28/10/2015 17:00, Alex Williamson wrote: > > > Alex, would it make sense to use the IRQ bypass infrastructure always, > > > not just for VT-d, to do the MSI injection directly from the VFIO > > > interrupt handler and bypass the eventfd? Basically this would add an > > > RCU-protected list of consumers matching the token to struct > > > irq_bypass_producer, and a > > > > > > int (*inject)(struct irq_bypass_consumer *); > > > > > > callback to struct irq_bypass_consumer. If any callback returns true, > > > the eventfd is not signaled. > > > > Yeah, that might be a good idea, it's probably more plausible than > > making the eventfd_signal() code friendly to call from hard interrupt > > context. On the vfio side can we use request_threaded_irq() directly > > for this? > > I don't know if that gives you a non-threaded IRQ with the real-time > kernel... CCing Marcelo to get some insight. > > > Making the hard irq handler return IRQ_HANDLED if we can use > > the irq bypass manager or IRQ_WAKE_THREAD if we need to use the eventfd. > > I think we need some way to get back to irq thread context to use > > eventfd_signal(). > > The irqfd is already able to schedule a work item, because it runs with > interrupts disabled, so I think we can always return IRQ_HANDLED. I'm confused by this. The problem with adding IRQF_NO_THREAD to our current handler is that it hits the spinlock that can sleep in eventfd_signal() and the waitqueue further down the stack before we get to the irqfd. So if we split to a non-threaded handler vs a threaded handler, where the non-threaded handler either returns IRQ_HANDLED or IRQ_WAKE_THREAD to queue the threaded handler, there's only so much that the non-threaded handler can do before we start running into the same problem. I think that means that the non-threaded handler needs to return IRQ_WAKE_THREAD if we need to use the current eventfd_signal() path, such as if the bypass path is not available. If we can get through the bypass path and the KVM irqfd side is safe for the non-threaded handler, inject succeeds and we return IRQ_HANDLED, right? Thanks, Alex