Linux-csky Archive on lore.kernel.org
 help / color / Atom feed
From: Arnd Bergmann <arnd@arndb.de>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Guo Ren <guoren@kernel.org>,
	linux-riscv <linux-riscv@lists.infradead.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-csky@vger.kernel.org,
	linux-arch <linux-arch@vger.kernel.org>,
	Guo Ren <guoren@linux.alibaba.com>, Will Deacon <will@kernel.org>,
	Ingo Molnar <mingo@redhat.com>, Waiman Long <longman@redhat.com>,
	Anup Patel <anup@brainfault.org>,
	Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Subject: Re: [PATCH v4 3/4] locking/qspinlock: Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32
Date: Mon, 29 Mar 2021 11:41:19 +0200
Message-ID: <CAK8P3a2bNH-1VjsZmZJkvGzzZY=ckaaOK9ZGL-oD0DH4jW-+kQ@mail.gmail.com> (raw)
In-Reply-To: <YGGGqftfr872/4CU@hirez.programming.kicks-ass.net>

On Mon, Mar 29, 2021 at 9:52 AM Peter Zijlstra <peterz@infradead.org> wrote:
> On Sat, Mar 27, 2021 at 06:06:38PM +0000, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Some architectures don't have sub-word swap atomic instruction,
> > they only have the full word's one.
> >
> > The sub-word swap only improve the performance when:
> > NR_CPUS < 16K
> >  *  0- 7: locked byte
> >  *     8: pending
> >  *  9-15: not used
> >  * 16-17: tail index
> >  * 18-31: tail cpu (+1)
> >
> > The 9-15 bits are wasted to use xchg16 in xchg_tail.
> >
> > Please let architecture select xchg16/xchg32 to implement
> > xchg_tail.
>
> So I really don't like this, this pushes complexity into the generic
> code for something that's really not needed.
>
> Lots of RISC already implement sub-word atomics using word ll/sc.
> Obviously they're not sharing code like they should be :/ See for
> example arch/mips/kernel/cmpxchg.c.

That is what the previous version of the patch set did, right?

I think this v4 is nicer because the code is already there in
qspinlock.c and just gets moved around, and the implementation
is likely more efficient this way. The mips version could be made
more generic, but it is also less efficient than a simple xchg
since it requires an indirect function call plus nesting a pair of
loops instead in place of the single single ll/sc loop in the 32-bit
xchg.

I think the weakly typed xchg/cmpxchg implementation causes
more problems than it solves, and we'd be better off using
a stronger version in general, with the 8-bit and 16-bit exchanges
using separate helpers in the same way that the fixed-length
cmpxchg64 is separate already, there are only a couple of instances
for each of these in the kernel.

Unfortunately, there is roughly a 50:50 split between fixed 32-bit
and long/pointer-sized xchg/cmpxchg users in the kernel, so
making the interface completely fixed-type would add a ton of
churn. I created an experimental patch for this, but it's probably
not worth it.

> Also, I really do think doing ticket locks first is a far more sensible
> step.

I think this is the important point to look at first. From what I can
tell, most users of ticket spinlocks have moved to qspinlock over
time, but I'm not sure about the exact tradeoff, in particular on
large machines without a 16-bit xchg operation.

FWIW, the implementations in the other SMP capable
architectures are

alpha    simple
arc      simple+custom
arm      ticket
arm64    qspinlock (formerly ticket)
csky     ticket/qrwlock (formerly simple+ticket/qrwlock)
hexagon  simple
ia64     ticket
mips     qspinlock (formerly ticket)
openrisc qspinlock
parisc   custom
powerpc  simple+qspinlock (formerly simple)
riscv    simple
s390     custom-queue
sh       simple
sparc    qspinlock
x86      qspinlock (formerly ticket+qspinlock)
xtensa   qspinlock

32-bit Arm is the only relevant user of ticket locks these days, but its
hardware has a practical limit of 16 CPUs and four nodes, with most
implementations having a single CPU cluster of at most four cores.

We had the same discussion about xchg16() when Sebastian submitted
the arm32 qspinlock implementation:
https://lore.kernel.org/linux-arm-kernel/20191007214439.27891-1-sebastian@breakpoint.cc/

        Arnd

  reply index

Thread overview: 63+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-27 18:06 [PATCH v4 0/4] riscv: Add qspinlock/qrwlock guoren
2021-03-27 18:06 ` [PATCH v4 1/4] riscv: cmpxchg.h: Cleanup unused code guoren
2021-03-27 18:06 ` [PATCH v4 2/4] riscv: cmpxchg.h: Merge macros guoren
2021-03-27 21:25   ` Arnd Bergmann
2021-03-28  1:50     ` Guo Ren
2021-03-27 18:06 ` [PATCH v4 3/4] locking/qspinlock: Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 guoren
2021-03-27 18:43   ` Waiman Long
2021-03-28  1:48     ` Guo Ren
2021-03-29  7:50   ` Peter Zijlstra
2021-03-29  9:41     ` Arnd Bergmann [this message]
2021-03-29 11:16       ` Peter Zijlstra
2021-03-29 11:29         ` Peter Zijlstra
2021-03-29 12:52           ` Guo Ren
2021-03-29 13:56             ` Arnd Bergmann
2021-03-30  2:26               ` Guo Ren
2021-03-30  5:51                 ` Anup Patel
2021-03-30  6:26                   ` Guo Ren
2021-03-30  7:11                 ` Arnd Bergmann
2021-03-31  4:18                   ` Guo Ren
2021-03-31  5:33                     ` Paul Campbell
2021-04-05 16:12                       ` Guo Ren
2021-03-31  6:44                     ` Guo Ren
2021-03-31  7:12                       ` Arnd Bergmann
2021-03-29 11:19     ` Guo Ren
2021-03-29 11:26       ` Peter Zijlstra
2021-03-29 12:01         ` Guo Ren
2021-03-29 12:49           ` Peter Zijlstra
2021-03-30  3:13             ` Guo Ren
2021-03-30  4:54               ` Anup Patel
2021-03-30  6:27                 ` Guo Ren
2021-03-30  8:31               ` David Laight
2021-03-30 14:09               ` Waiman Long
2021-03-31 14:47                 ` Guo Ren
2021-04-05 16:45                 ` Guo Ren
2021-03-30 16:08               ` Peter Zijlstra
2021-03-30 22:35                 ` Stafford Horne
2021-03-31  7:23                   ` Arnd Bergmann
2021-03-31 12:31                     ` Stafford Horne
2021-03-31 15:10                       ` Guo Ren
2021-04-06  8:51                         ` Stafford Horne
2021-04-06  3:50                     ` Guo Ren
2021-04-06  8:56                       ` Stafford Horne
2021-04-07  8:42                         ` Arnd Bergmann
2021-04-07 11:36                           ` Peter Zijlstra
2021-04-07 11:57                             ` Arnd Bergmann
2021-04-07 12:02                             ` Peter Zijlstra
2021-04-05 16:40                 ` Guo Ren
2021-03-31 15:22             ` Guo Ren
2021-04-06  7:15               ` Peter Zijlstra
2021-04-07  9:42                 ` Christoph Hellwig
2021-04-07 14:29                   ` Christoph Müllner
2021-04-07 14:34                     ` Christoph Hellwig
2021-04-07 15:51                     ` Peter Zijlstra
2021-04-07 16:44                       ` Peter Zijlstra
2021-04-07 15:52                     ` Peter Zijlstra
2021-04-07 16:54                       ` Peter Zijlstra
2021-04-07 16:00                     ` Peter Zijlstra
2021-04-07 19:50                       ` Christoph Müllner
2021-04-06 17:24               ` Boqun Feng
2021-04-07  9:26                 ` Peter Zijlstra
2021-03-29 12:13         ` Anup Patel
2021-03-29 12:54           ` Peter Zijlstra
2021-03-27 18:06 ` [PATCH v4 4/4] riscv: Convert custom spinlock/rwlock to generic qspinlock/qrwlock guoren

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAK8P3a2bNH-1VjsZmZJkvGzzZY=ckaaOK9ZGL-oD0DH4jW-+kQ@mail.gmail.com' \
    --to=arnd@arndb.de \
    --cc=anup@brainfault.org \
    --cc=guoren@kernel.org \
    --cc=guoren@linux.alibaba.com \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-csky@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=longman@redhat.com \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=sebastian@breakpoint.cc \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-csky Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-csky/0 linux-csky/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-csky linux-csky/ https://lore.kernel.org/linux-csky \
		linux-csky@vger.kernel.org
	public-inbox-index linux-csky

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-csky


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git