From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751355AbbBLQjE (ORCPT ); Thu, 12 Feb 2015 11:39:04 -0500 Received: from mail-qg0-f51.google.com ([209.85.192.51]:63696 "EHLO mail-qg0-f51.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751272AbbBLQjA (ORCPT ); Thu, 12 Feb 2015 11:39:00 -0500 MIME-Version: 1.0 X-Originating-IP: [191.180.238.226] In-Reply-To: References: Date: Thu, 12 Feb 2015 14:38:58 -0200 Message-ID: Subject: Re: smp_call_function_single lockups From: Rafael David Tinoco To: Linus Torvalds Cc: LKML , Thomas Gleixner , Jens Axboe , Frederic Weisbecker , chris.j.arges@canonical.com, gema.gomez-solano@canonical.com Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Meanwhile we'll take the opportunity to run same tests with the "smp_load_acquire/smp_store_release + outside sync/async" approach made by your latest patch on top of 3.19. If anything comes up I'll provide full back traces (2 vcpus). Here I can only reproduce this inside nested kvm on top of Proliant DL360 Gen8 machines with: - no opt out from x2apic (gen8 firmware asks for opting out but HP says x2apic should be used for >= gen8) - no intel_idle since proliant firmware is causing NMIs during MWAIT instructions As observed before, reducing performance made the problem to be triggered only after some days so, if nothing goes wrong with performance this time, I expect to have results in between 10 to 30 hours. Thank you Tinoco On Wed, Feb 11, 2015 at 6:42 PM, Linus Torvalds wrote: > > [ Added Frederic to the cc, since he's touched this file/area most ] > > On Wed, Feb 11, 2015 at 11:59 AM, Linus Torvalds > wrote: > > > > So the caller has a really hard time guaranteeing that CSD_LOCK isn't > > set. And if the call is done in interrupt context, for all we know it > > is interrupting the code that is going to clear CSD_LOCK, so CSD_LOCK > > will never be cleared at all, and csd_lock() will wait forever. > > > > So I actually think that for the async case, we really *should* unlock > > before doing the callback (which is what Thomas' old patch did). > > > > And we migth well be better off doing something like > > > > WARN_ON_ONCE(csd->flags & CSD_LOCK); > > > > in smp_call_function_single_async(), because that really is a hard requirement. > > > > And it strikes me that hrtick_csd is one of these cases that do this > > with interrupts disabled, and use the callback for serialization. So I > > really wonder if this is part of the problem.. > > > > Thomas? Am I missing something? > > Ok, this is a more involved patch than I'd like, but making the > *caller* do all the CSD maintenance actually cleans things up. > > And this is still completely untested, and may be entirely buggy. What > do you guys think? > > Linus