linux-arm-kernel.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
From: Jan Glauber <jglauber@marvell.com>
To: "catalin.marinas@arm.com" <catalin.marinas@arm.com>,
	"will.deacon@arm.com" <will.deacon@arm.com>
Cc: Jayachandran Chandrasekharan Nair <jnair@marvell.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>
Subject: [RFC] Disable lockref on arm64
Date: Mon, 29 Apr 2019 14:52:11 +0000	[thread overview]
Message-ID: <20190429145159.GA29076@hc> (raw)

Hi Catalin & Will,

I've been looking into performance issues that were reported for several
test-cases, for instance an nginx benchmark.

It turned out the issue we have on ThunderX2 is the file open-close sequence
with small read sizes. If the used files are opened read-only the
lockref code (enabled by ARCH_USE_CMPXCHG_LOCKREF) is used.

The lockref CMPXCHG_LOOP uses an unbound (as long as the associated
spinlock isn't taken) while loop to change the lock count. This behaves
badly under heavy contention (~25x retries for one cmpxchg to succeed
with 28 threads operating on the same file). In case of a NUMA system
it also behaves badly as the access from the other socket is much slower.

The fact that on ThunderX2 cpu_relax() turns only into one NOP
instruction doesn't help either. On Intel pause seems to block the thread
much longer, avoiding the heavy contention thereby.

With the queued spinlocks implementation I can see a major improvement
when I disable lockref. A trivial open-close test-case improves by
factor 2 while system time is decreasing also 2x. Looking at kernel compile
and dbench numbers didn't show any regression with lockref disabled.

Can we simply disable lockref? Is anyone else seeing this issue? Is there
an arm64 platform that actually implements yield?

Thanks,
Jan

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

             reply	other threads:[~2019-04-29 14:52 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-29 14:52 Jan Glauber [this message]
2019-05-01 16:01 ` [RFC] Disable lockref on arm64 Will Deacon
2019-05-02  8:38   ` Jan Glauber
2019-05-01 16:41 ` Linus Torvalds
2019-05-02  8:27   ` Jan Glauber
2019-05-02 16:12     ` Linus Torvalds
2019-05-02 23:19       ` Jayachandran Chandrasekharan Nair
2019-05-03 19:40         ` Linus Torvalds
2019-05-06  6:13           ` [EXT] " Jayachandran Chandrasekharan Nair
2019-05-06 17:13             ` Linus Torvalds
2019-05-06 18:10             ` Will Deacon
2019-05-18  4:24               ` Jayachandran Chandrasekharan Nair
2019-05-18 10:00                 ` Ard Biesheuvel
2019-05-22 16:04                   ` Will Deacon
2019-06-12  4:10                     ` Jayachandran Chandrasekharan Nair
2019-06-12  9:31                       ` Will Deacon
2019-06-14  7:09                         ` Jayachandran Chandrasekharan Nair
2019-06-14  9:58                           ` Will Deacon
2019-06-14 10:24                             ` Ard Biesheuvel
2019-06-14 10:38                               ` Will Deacon
2019-06-15  4:21                                 ` Kees Cook
2019-06-15  8:47                                   ` Ard Biesheuvel
2019-06-15 13:59                                     ` Kees Cook
2019-06-15 14:18                                       ` Ard Biesheuvel
2019-06-16 21:31                                         ` Kees Cook
2019-06-17 11:33                                           ` Ard Biesheuvel
2019-06-17 17:26                                             ` Will Deacon
2019-06-17 20:07                                               ` Jayachandran Chandrasekharan Nair
2019-06-18  5:41                                               ` Kees Cook
2019-06-13  9:53                       ` Hanjun Guo
2019-06-05 13:48   ` [PATCH] lockref: Limit number of cmpxchg loop retries Jan Glauber
2019-06-05 20:16     ` Linus Torvalds
2019-06-06  8:03       ` Jan Glauber
2019-06-06  9:41         ` Will Deacon
2019-06-06 10:28           ` Jan Glauber
2019-06-07  7:27             ` Jan Glauber
2019-06-07 20:14               ` Linus Torvalds

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190429145159.GA29076@hc \
    --to=jglauber@marvell.com \
    --cc=catalin.marinas@arm.com \
    --cc=jnair@marvell.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=will.deacon@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).