From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752534Ab3B0TyC (ORCPT ); Wed, 27 Feb 2013 14:54:02 -0500 Received: from mx1.redhat.com ([209.132.183.28]:58318 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751370Ab3B0TyA (ORCPT ); Wed, 27 Feb 2013 14:54:00 -0500 Message-ID: <512E6443.9050603@redhat.com> Date: Wed, 27 Feb 2013 14:53:39 -0500 From: Rik van Riel User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130110 Thunderbird/17.0.2 MIME-Version: 1.0 To: Linus Torvalds CC: Ingo Molnar , "H. Peter Anvin" , Linux Kernel Mailing List , Peter Zijlstra , aquini@redhat.com, Andrew Morton , Thomas Gleixner , Michel Lespinasse , linux-tip-commits@vger.kernel.org, Steven Rostedt , "Vinod, Chegu" , "Low, Jason" Subject: Re: [tip:core/locking] x86/smp: Move waiting on contended ticket lock out of line References: <20130206150403.006e5294@cuia.bos.redhat.com> <511BE4A3.8050607@redhat.com> <511C1204.9040608@redhat.com> <511C24A6.8020409@redhat.com> <512E376D.70105@redhat.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 02/27/2013 12:10 PM, Linus Torvalds wrote: > Ugh. That really is rather random. "short" and fserver seems to > improve a lot (including the "new" version), the others look like they > are either unchanged or huge regressions. > > Is there any way to get profiles for the improved versions vs the > regressed ones? It might well be that we have two different classes of > spinlocks. Maybe we could make the back-off version be *explicit* (ie > not part of the normal "spin_lock()", but you'd use a special > "spin_lock_backoff()" function for it) because it works well for some > cases but not for others? If we have two classes of spinlocks, I suspect we would be better off making those high-demand spinlocks MCS or LCH locks, which have the property that having N+1 CPUs contend on the lock will never result in slower aggregate throughput than having N CPUs contend. Both the current spinlock code and the spinlock code with backoff has the annoying property that adding more waiters to the lock can slow down aggregate throughput. The backoff code seems to push that point a little further out. It appears that that may result in "lesser bottlenecks" disappearing, which in turn triggers amplification of the worst bottlenecks. > Hmm? At the very least, it would give us an idea of *which* spinlock > it is that causes the most pain. I think your earlier indications was > that it's the mutex->wait_lock or something? I can certainly take profiles of various workloads, but there is absolutely no guarantee that I will see the same bottlenecks that eg. the people at HP have seen. The largest test system I currently have access to has 40 cores, vs. the 80 cores in the (much more interesting) HP results I pasted. Would you also be interested in performance numbers (and profiles) of a kernel that has bottleneck spinlocks replaced with MCS locks? That could make for an interesting comparison... -- All rights reversed