From: Andrew Morton <akpm@linux-foundation.org>
To: Davidlohr Bueso <dave@stgolabs.net>
Cc: jbaron@akamai.com, viro@zeniv.linux.org.uk,
linux-kernel@vger.kernel.org,
Peter Zijlstra <a.p.zijlstra@chello.nl>
Subject: Re: [PATCH -next 0/2] fs/epoll: loosen irq safety when possible
Date: Fri, 20 Jul 2018 13:44:29 -0700 [thread overview]
Message-ID: <20180720134429.1ba61018934b084bb2e17bdb@linux-foundation.org> (raw)
In-Reply-To: <20180720200559.27nc7j2rrxpy5p3n@linux-r8p5>
On Fri, 20 Jul 2018 13:05:59 -0700 Davidlohr Bueso <dave@stgolabs.net> wrote:
> On Fri, 20 Jul 2018, Andrew Morton wrote:
>
> >On Fri, 20 Jul 2018 10:29:54 -0700 Davidlohr Bueso <dave@stgolabs.net> wrote:
> >
> >> Hi,
> >>
> >> Both patches replace saving+restoring interrupts when taking the
> >> ep->lock (now the waitqueue lock), with just disabling local irqs.
> >> This shows immediate performance benefits in patch 1 for an epoll
> >> workload running on Xen.
> >
> >I'm surprised. Is spin_lock_irqsave() significantly more expensive
> >than spin_lock_irq()? Relative to all the other stuff those functions
> >are doing? If so, how come? Some architectural thing makes
> >local_irq_save() much more costly than local_irq_disable()?
>
> For example, if you compare x86 native_restore_fl() to xen_restore_fl(),
> the cost of Xen is much higher.
>
> And at least considering ep_scan_ready_list(), the lock is taken/released
> twice, to deal with the ovflist when the ep->wq.lock is not held. To the
> point that it yields measurable results (see patch 1) across incremental
> thread counts.
Did you try measuring it on bare hardware?
> >
> >> The main concern we need to have with this
> >> sort of changes in epoll is the ep_poll_callback() which is passed
> >> to the wait queue wakeup and is done very often under irq context,
> >> this patch does not touch this call.
> >
> >Yeah, these changes are scary. For the code as it stands now, and for
> >the code as it evolves.
>
> Yes which is why I've been throwing lots of epoll workloads at it.
I'm sure. It's the "as it evolves" that is worrisome, and has caught
us in the past.
> >
> >I'd have more confidence if we had some warning mechanism if we run
> >spin_lock_irq() when IRQs are disabled, which is probably-a-bug. But
> >afaict we don't have that. Probably for good reasons - I wonder what
> >they are?
Well ignored ;)
We could open-code it locally. Add a couple of
WARN_ON_ONCE(irqs_disabled())? That might need re-benchmarking with
Xen but surely just reading the thing isn't too expensive?
next prev parent reply other threads:[~2018-07-20 20:44 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-20 17:29 [PATCH -next 0/2] fs/epoll: loosen irq safety when possible Davidlohr Bueso
2018-07-20 17:29 ` [PATCH 1/2] fs/epoll: loosen irq safety in ep_scan_ready_list() Davidlohr Bueso
2018-07-20 17:29 ` [PATCH 2/2] fs/epoll: loosen irq safety in epoll_insert() and epoll_remove() Davidlohr Bueso
2018-07-20 19:42 ` [PATCH -next 0/2] fs/epoll: loosen irq safety when possible Andrew Morton
2018-07-20 20:05 ` Davidlohr Bueso
2018-07-20 20:44 ` Andrew Morton [this message]
2018-07-21 0:22 ` Davidlohr Bueso
2018-07-21 17:21 ` Davidlohr Bueso
2018-07-21 17:39 ` Peter Zijlstra
2018-07-21 18:31 ` Davidlohr Bueso
2018-07-24 23:43 ` Andrew Morton
2018-09-06 19:11 ` Peter Zijlstra
2018-09-06 20:55 ` Davidlohr Bueso
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180720134429.1ba61018934b084bb2e17bdb@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=a.p.zijlstra@chello.nl \
--cc=dave@stgolabs.net \
--cc=jbaron@akamai.com \
--cc=linux-kernel@vger.kernel.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).