From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752894AbbBSQ7T (ORCPT ); Thu, 19 Feb 2015 11:59:19 -0500 Received: from mail-ie0-f172.google.com ([209.85.223.172]:40117 "EHLO mail-ie0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751977AbbBSQ7S (ORCPT ); Thu, 19 Feb 2015 11:59:18 -0500 MIME-Version: 1.0 In-Reply-To: References: <20150218222544.GA17717@twins.programming.kicks-ass.net> Date: Thu, 19 Feb 2015 08:59:17 -0800 X-Google-Sender-Auth: dNINSgOjoQDJFRUbV8nW-N8i18w Message-ID: Subject: Re: smp_call_function_single lockups From: Linus Torvalds To: Rafael David Tinoco Cc: Peter Zijlstra , LKML , Jens Axboe , Frederic Weisbecker , Gema Gomez , chris.j.arges@canonical.com, "the arch/x86 maintainers" Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Feb 19, 2015 at 8:32 AM, Rafael David Tinoco wrote: > Feb 19 08:21:28 derain kernel: [ 3.637682] Switched APIC routing to > cluster x2apic. Ok. That "cluster x2apic" mode is just about the nastiest mode when it comes to sending a single ipi. We do that insane dance where we - turn single cpu number into cpumask - copy the cpumask to a percpu temporary storage - walk each cpu in the cpumask - for each cpu, look up the cluster siblings - for each cluster sibling that is also in the cpumask, look up the logical apic mask and add it to the actual ipi destination mask - send an ipi to that final mask. which is just insane. It's complicated, it's fragile, and it's unnecessary. If we had a simple "send_IPI()" function, we could do this all with something much saner, and it would look sopmething like static void x2apic_send_IPI(int cpu, int vector) { u32 dest = per_cpu(x86_cpu_to_logical_apicid, cpu); x2apic_wrmsr_fence(); __x2apic_send_IPI_dest(dest, vector, APIC_DEST_LOGICAL); } and then 'void native_send_call_func_single_ipi()' would just look like void native_send_call_func_single_ipi(int cpu) { apic->send_IPI(cpu, CALL_FUNCTION_SINGLE_VECTOR); } but I might have missed something (and we might want to have a wrapper that says "if the apic doesn't have a 'send_IPI' function, use "send_IPI_mask(cpumask_of(cpu, vector) instead" The fact that you need that no_x2apic_optout (which in turn means that your ACPI tables seem to say "don't use x2apic") also makes me worry. Are there known errata for the x2apic? Linus