From mboxrd@z Thu Jan 1 00:00:00 1970 From: Boris Ostrovsky Subject: Re: [patch 1/4] hotplug: Prevent alloc/free of irq descriptors during cpu up/down Date: Tue, 14 Jul 2015 16:04:24 -0400 Message-ID: <55A56B48.4060605__46574.3659799502$1436904421$gmane$org@oracle.com> References: <20150705170530.849428850@linutronix.de> <20150705171102.063519515@linutronix.de> <55A51F10.7010407@oracle.com> <55A532C2.4080306@oracle.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1ZF6SE-0005xX-Bl for xen-devel@lists.xenproject.org; Tue, 14 Jul 2015 20:05:26 +0000 In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Thomas Gleixner Cc: Yanmin Zhang , Joerg Roedel , Peter Zijlstra , LKML , xiao jin , Peter Anvin , xen-devel , Borislav Petkov , Ingo Molnar List-Id: xen-devel@lists.xenproject.org On 07/14/2015 01:32 PM, Thomas Gleixner wrote: > On Tue, 14 Jul 2015, Boris Ostrovsky wrote: >> On 07/14/2015 11:44 AM, Thomas Gleixner wrote: >>> On Tue, 14 Jul 2015, Boris Ostrovsky wrote: >>>>> Prevent allocation and freeing of interrupt descriptors accross cpu >>>>> hotplug. >>>> This breaks Xen guests that allocate interrupt descriptors in .cpu_up(). >>> And where exactly does XEN allocate those descriptors? >> xen_cpu_up() >> xen_setup_timer() >> bind_virq_to_irqhandler() >> bind_virq_to_irq() >> xen_allocate_irq_dynamic() >> xen_allocate_irqs_dynamic() >> irq_alloc_descs() >> >> >> There is also a similar pass via xen_cpu_up() -> xen_smp_intr_init() > Sigh. > >>> >>>> Any chance this locking can be moved into arch code? >>> No. > The issue here is that all architectures need that protection and just > Xen does irq allocations in cpu_up. > > So moving that protection into architecture code is not really an > option. > >>>> Otherwise we will need to have something like arch_post_cpu_up() >>>> after the lock is released. > I'm not sure, that this will work. You probably want to do this in the > cpu prepare stage, i.e. before calling __cpu_up(). For PV guests (the ones that use xen_cpu_up()) it will work either before or after __cpu_up(). At least my (somewhat limited) testing didn't show any problems so far. However, HVM CPUs use xen_hvm_cpu_up() and if you read comments there you will see that xen_smp_intr_init() needs to be called before native_cpu_up() but xen_init_lock_cpu() (which eventually calls irq_alloc_descs()) needs to be called after. I think I can split xen_init_lock_cpu() so that the part that needs to be called after will avoid going into irq core code. And then the rest will go into arch_cpu_prepare(). -boris