All of lore.kernel.org
 help / color / mirror / Atom feed
* [RFC PATCH 0/2] ipc: do not hold ipc lock more than necessary
@ 2013-03-02  0:16 Davidlohr Bueso
  2013-03-02  1:32 ` Linus Torvalds
  2013-03-02  4:43 ` Emmanuel Benisty
  0 siblings, 2 replies; 8+ messages in thread
From: Davidlohr Bueso @ 2013-03-02  0:16 UTC (permalink / raw)
  To: Rik van Riel, Linus Torvalds
  Cc: Thomas Gleixner, Steven Rostedt, Vinod, Chegu, Low, Jason,
	linux-tip-commits, Peter Zijlstra, H. Peter Anvin, Andrew Morton,
	aquini, Michel Lespinasse, Ingo Molnar, Larry Woodman,
	Linux Kernel Mailing List

The following set of not-thoroughly-tested patches are based on the
discussion of holding the ipc lock unnecessarily, such as for permissions
and security checks:

https://lkml.org/lkml/2013/2/28/540

Patch 0/1: Introduces new functions, analogous to ipc_lock and ipc_lock_check
in the ipc utility code, allowing to obtain the ipc object without holding the lock.

Patch 0/2: Use the new functions and only acquire the ipc lock when needed.

With Rik's semop-multi.c microbenchmark we can see the following
results:

256 sems without patches:
+  59.40%            a.out  [kernel.kallsyms]  [k] _raw_spin_lock
+   6.14%            a.out  [kernel.kallsyms]  [k] sys_semtimedop
+   3.84%            a.out  [kernel.kallsyms]  [k] avc_has_perm_flags
+   3.64%            a.out  [kernel.kallsyms]  [k] __audit_syscall_exit
+   2.06%            a.out  [kernel.kallsyms]  [k] copy_user_enhanced_fast_string
+   1.86%            a.out  [kernel.kallsyms]  [k] ipc_lock
+   1.75%            a.out  [kernel.kallsyms]  [k] __audit_syscall_entry
+   1.69%            a.out  [kernel.kallsyms]  [k] ipc_has_perm.isra.21
+   1.47%            a.out  [kernel.kallsyms]  [k] do_smart_update
+   1.43%            a.out  [kernel.kallsyms]  [k] pid_vnr
+   1.39%            a.out  [kernel.kallsyms]  [k] try_atomic_semop.isra.5

total operations: 151452270, ops/sec 5048409

256 sems with patches:
+  17.47%            a.out  [kernel.kallsyms]  [k] _raw_spin_lock
+  11.08%            a.out  [kernel.kallsyms]  [k] sys_semtimedop
+   8.81%            a.out  [kernel.kallsyms]  [k] avc_has_perm_flags
+   7.96%            a.out  [kernel.kallsyms]  [k] ipc_has_perm.isra.21
+   6.50%            a.out  [kernel.kallsyms]  [k] __audit_syscall_exit
+   4.67%            a.out  [kernel.kallsyms]  [k] ipc_obtain_object_check
+   4.19%            a.out  [kernel.kallsyms]  [k] ipcperms
+   3.75%            a.out  [kernel.kallsyms]  [k] copy_user_enhanced_fast_string
+   3.38%            a.out  [kernel.kallsyms]  [k] system_call
+   3.05%            a.out  [kernel.kallsyms]  [k] try_atomic_semop.isra.5
+   2.70%            a.out  [kernel.kallsyms]  [k] do_smart_update
+   2.60%            a.out  [kernel.kallsyms]  [k] __audit_syscall_entry

total operations: 266502912, ops/sec 8883430

While the _raw_spin_lock time is drastically reduced, others do increase.
This results in an overall speedup of ~1.7x regarding ops/sec.



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH 0/2] ipc: do not hold ipc lock more than necessary
  2013-03-02  0:16 [RFC PATCH 0/2] ipc: do not hold ipc lock more than necessary Davidlohr Bueso
@ 2013-03-02  1:32 ` Linus Torvalds
  2013-03-02  1:35   ` Linus Torvalds
  2013-03-02 21:23   ` Davidlohr Bueso
  2013-03-02  4:43 ` Emmanuel Benisty
  1 sibling, 2 replies; 8+ messages in thread
From: Linus Torvalds @ 2013-03-02  1:32 UTC (permalink / raw)
  To: Davidlohr Bueso
  Cc: Rik van Riel, Thomas Gleixner, Steven Rostedt, Vinod, Chegu, Low,
	Jason, linux-tip-commits, Peter Zijlstra, H. Peter Anvin,
	Andrew Morton, aquini, Michel Lespinasse, Ingo Molnar,
	Larry Woodman, Linux Kernel Mailing List

On Fri, Mar 1, 2013 at 4:16 PM, Davidlohr Bueso <davidlohr.bueso@hp.com> wrote:
>
> With Rik's semop-multi.c microbenchmark we can see the following
> results:

Ok, that certainly looks very good.

> +  59.40%            a.out  [kernel.kallsyms]  [k] _raw_spin_lock
> +  17.47%            a.out  [kernel.kallsyms]  [k] _raw_spin_lock

I had somewhat high expectations, but that's just better than I really
hoped for. Not only is the percentage down, it's down for the case of
a much smaller number of overall cycle cost, so it's a really big
reduction in contention spinning.

Of course, contention will come back and overwhelm you at *some*
point, but it seems the patches certainly moved the really bad
contention point out some way..

> +   6.14%            a.out  [kernel.kallsyms]  [k] sys_semtimedop
> +  11.08%            a.out  [kernel.kallsyms]  [k] sys_semtimedop
> While the _raw_spin_lock time is drastically reduced, others do increase.
> This results in an overall speedup of ~1.7x regarding ops/sec.

Actually, the others don't really increase. Sure, the *percentages* go
up, but that's just because it has to add up to 100% in the end. So
it's not that you're moving costs from one place to another - the 1.7x
speedup is the real reduction in costs, and then that 6.14% -> 11.08%
"growth" is really nothing but that (and yes, 1.7 x 6.14 really does
get pretty close).

So nothing really got slower, despite the percentages going up.

Looks good to me. Of course, the *real* issue is if this is a win on
real code too. And I bet it is, it just won't be quite as noticeable.
But if anything, real code is likely to have less contention to begin
with, because it has more things going on outside of the spinlocks. So
it should see an improvement, but not nearly the kind of improvement
you quote here.

Although your 800-user swingbench numbers were pretty horrible, so
maybe that case can improve by comparable amounts in the bad cases.

                Linus

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH 0/2] ipc: do not hold ipc lock more than necessary
  2013-03-02  1:32 ` Linus Torvalds
@ 2013-03-02  1:35   ` Linus Torvalds
  2013-03-02 21:23   ` Davidlohr Bueso
  1 sibling, 0 replies; 8+ messages in thread
From: Linus Torvalds @ 2013-03-02  1:35 UTC (permalink / raw)
  To: Davidlohr Bueso
  Cc: Rik van Riel, Thomas Gleixner, Steven Rostedt, Vinod, Chegu, Low,
	Jason, linux-tip-commits, Peter Zijlstra, H. Peter Anvin,
	Andrew Morton, aquini, Michel Lespinasse, Ingo Molnar,
	Larry Woodman, Linux Kernel Mailing List

On Fri, Mar 1, 2013 at 5:32 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Fri, Mar 1, 2013 at 4:16 PM, Davidlohr Bueso <davidlohr.bueso@hp.com> wrote:
>>
>> With Rik's semop-multi.c microbenchmark we can see the following
>> results:
>
> Ok, that certainly looks very good.

Side note: it's fairly late in the merge window, and I don't feel
comfy merging this without more testing and more people looking at it,
so I think this is a 3.10 thing. But it would be great to see ipc
people really look at the patches, and perhaps run a few real loads to
see if it's even noticeable in practice..

              Linus

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH 0/2] ipc: do not hold ipc lock more than necessary
  2013-03-02  0:16 [RFC PATCH 0/2] ipc: do not hold ipc lock more than necessary Davidlohr Bueso
  2013-03-02  1:32 ` Linus Torvalds
@ 2013-03-02  4:43 ` Emmanuel Benisty
  2013-03-02  7:08   ` Michel Lespinasse
  1 sibling, 1 reply; 8+ messages in thread
From: Emmanuel Benisty @ 2013-03-02  4:43 UTC (permalink / raw)
  To: Davidlohr Bueso
  Cc: Rik van Riel, Linus Torvalds, Thomas Gleixner, Steven Rostedt,
	Vinod, Chegu, Low, Jason, linux-tip-commits, Peter Zijlstra,
	H. Peter Anvin, Andrew Morton, aquini, Michel Lespinasse,
	Ingo Molnar, Larry Woodman, Linux Kernel Mailing List

Hi,

On Sat, Mar 2, 2013 at 7:16 AM, Davidlohr Bueso <davidlohr.bueso@hp.com> wrote:
> The following set of not-thoroughly-tested patches are based on the
> discussion of holding the ipc lock unnecessarily, such as for permissions
> and security checks:
>
> https://lkml.org/lkml/2013/2/28/540
>
> Patch 0/1: Introduces new functions, analogous to ipc_lock and ipc_lock_check
> in the ipc utility code, allowing to obtain the ipc object without holding the lock.
>
> Patch 0/2: Use the new functions and only acquire the ipc lock when needed.

Not sure how much a work in progress this is but my machine dies
immediately when I start chromium, crappy mobile phone picture here:
http://i.imgur.com/S0hfPz3.jpg

Thanks.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH 0/2] ipc: do not hold ipc lock more than necessary
  2013-03-02  4:43 ` Emmanuel Benisty
@ 2013-03-02  7:08   ` Michel Lespinasse
  2013-03-02  8:35     ` Emmanuel Benisty
  0 siblings, 1 reply; 8+ messages in thread
From: Michel Lespinasse @ 2013-03-02  7:08 UTC (permalink / raw)
  To: Emmanuel Benisty
  Cc: Davidlohr Bueso, Rik van Riel, Linus Torvalds, Thomas Gleixner,
	Steven Rostedt, Vinod, Chegu, Low, Jason, linux-tip-commits,
	Peter Zijlstra, H. Peter Anvin, Andrew Morton, aquini,
	Ingo Molnar, Larry Woodman, Linux Kernel Mailing List

On Sat, Mar 2, 2013 at 12:43 PM, Emmanuel Benisty <benisty.e@gmail.com> wrote:
> Hi,
>
> On Sat, Mar 2, 2013 at 7:16 AM, Davidlohr Bueso <davidlohr.bueso@hp.com> wrote:
>> The following set of not-thoroughly-tested patches are based on the
>> discussion of holding the ipc lock unnecessarily, such as for permissions
>> and security checks:
>>
>> https://lkml.org/lkml/2013/2/28/540
>>
>> Patch 0/1: Introduces new functions, analogous to ipc_lock and ipc_lock_check
>> in the ipc utility code, allowing to obtain the ipc object without holding the lock.
>>
>> Patch 0/2: Use the new functions and only acquire the ipc lock when needed.
>
> Not sure how much a work in progress this is but my machine dies
> immediately when I start chromium, crappy mobile phone picture here:
> http://i.imgur.com/S0hfPz3.jpg

We are missing the top of the trace there, so it's hard to be sure -
however, this could well be caused by the if (!out) check (instead of
if (IS_ERR(out)) that I noticed in patch 1/2.

-- 
Michel "Walken" Lespinasse
A program is never fully debugged until the last user dies.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH 0/2] ipc: do not hold ipc lock more than necessary
  2013-03-02  7:08   ` Michel Lespinasse
@ 2013-03-02  8:35     ` Emmanuel Benisty
  2013-03-02 21:20       ` Davidlohr Bueso
  0 siblings, 1 reply; 8+ messages in thread
From: Emmanuel Benisty @ 2013-03-02  8:35 UTC (permalink / raw)
  To: Michel Lespinasse
  Cc: Davidlohr Bueso, Rik van Riel, Linus Torvalds, Thomas Gleixner,
	Steven Rostedt, Vinod, Chegu, Low, Jason, linux-tip-commits,
	Peter Zijlstra, H. Peter Anvin, Andrew Morton, aquini,
	Ingo Molnar, Larry Woodman, Linux Kernel Mailing List

On Sat, Mar 2, 2013 at 2:08 PM, Michel Lespinasse <walken@google.com> wrote:
> On Sat, Mar 2, 2013 at 12:43 PM, Emmanuel Benisty <benisty.e@gmail.com> wrote:
>> Hi,
>>
>> On Sat, Mar 2, 2013 at 7:16 AM, Davidlohr Bueso <davidlohr.bueso@hp.com> wrote:
>>> The following set of not-thoroughly-tested patches are based on the
>>> discussion of holding the ipc lock unnecessarily, such as for permissions
>>> and security checks:
>>>
>>> https://lkml.org/lkml/2013/2/28/540
>>>
>>> Patch 0/1: Introduces new functions, analogous to ipc_lock and ipc_lock_check
>>> in the ipc utility code, allowing to obtain the ipc object without holding the lock.
>>>
>>> Patch 0/2: Use the new functions and only acquire the ipc lock when needed.
>>
>> Not sure how much a work in progress this is but my machine dies
>> immediately when I start chromium, crappy mobile phone picture here:
>> http://i.imgur.com/S0hfPz3.jpg
>
> We are missing the top of the trace there, so it's hard to be sure -
> however, this could well be caused by the if (!out) check (instead of
> if (IS_ERR(out)) that I noticed in patch 1/2.

Merci Michel but unfortunately, I'm still getting the same issue.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH 0/2] ipc: do not hold ipc lock more than necessary
  2013-03-02  8:35     ` Emmanuel Benisty
@ 2013-03-02 21:20       ` Davidlohr Bueso
  0 siblings, 0 replies; 8+ messages in thread
From: Davidlohr Bueso @ 2013-03-02 21:20 UTC (permalink / raw)
  To: Emmanuel Benisty
  Cc: Michel Lespinasse, Rik van Riel, Linus Torvalds, Thomas Gleixner,
	Steven Rostedt, Vinod, Chegu, Low, Jason, linux-tip-commits,
	Peter Zijlstra, H. Peter Anvin, Andrew Morton, aquini,
	Ingo Molnar, Larry Woodman, Linux Kernel Mailing List

On Sat, 2013-03-02 at 15:35 +0700, Emmanuel Benisty wrote:
> On Sat, Mar 2, 2013 at 2:08 PM, Michel Lespinasse <walken@google.com> wrote:
> > On Sat, Mar 2, 2013 at 12:43 PM, Emmanuel Benisty <benisty.e@gmail.com> wrote:
> >> Hi,
> >>
> >> On Sat, Mar 2, 2013 at 7:16 AM, Davidlohr Bueso <davidlohr.bueso@hp.com> wrote:
> >>> The following set of not-thoroughly-tested patches are based on the
> >>> discussion of holding the ipc lock unnecessarily, such as for permissions
> >>> and security checks:
> >>>
> >>> https://lkml.org/lkml/2013/2/28/540
> >>>
> >>> Patch 0/1: Introduces new functions, analogous to ipc_lock and ipc_lock_check
> >>> in the ipc utility code, allowing to obtain the ipc object without holding the lock.
> >>>
> >>> Patch 0/2: Use the new functions and only acquire the ipc lock when needed.
> >>
> >> Not sure how much a work in progress this is but my machine dies
> >> immediately when I start chromium, crappy mobile phone picture here:
> >> http://i.imgur.com/S0hfPz3.jpg
> >
> > We are missing the top of the trace there, so it's hard to be sure -
> > however, this could well be caused by the if (!out) check (instead of
> > if (IS_ERR(out)) that I noticed in patch 1/2.
> 
> Merci Michel but unfortunately, I'm still getting the same issue.

Will try to reproduce (and further testing on other machines) and debug
later today.

Thanks for testing,
Davidlohr


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH 0/2] ipc: do not hold ipc lock more than necessary
  2013-03-02  1:32 ` Linus Torvalds
  2013-03-02  1:35   ` Linus Torvalds
@ 2013-03-02 21:23   ` Davidlohr Bueso
  1 sibling, 0 replies; 8+ messages in thread
From: Davidlohr Bueso @ 2013-03-02 21:23 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Rik van Riel, Thomas Gleixner, Steven Rostedt, Vinod, Chegu, Low,
	Jason, linux-tip-commits, Peter Zijlstra, H. Peter Anvin,
	Andrew Morton, aquini, Michel Lespinasse, Ingo Molnar,
	Larry Woodman, Linux Kernel Mailing List

On Fri, 2013-03-01 at 17:32 -0800, Linus Torvalds wrote:
> On Fri, Mar 1, 2013 at 4:16 PM, Davidlohr Bueso <davidlohr.bueso@hp.com> wrote:
> >
> > With Rik's semop-multi.c microbenchmark we can see the following
> > results:
> 
> Ok, that certainly looks very good.
> 
> > +  59.40%            a.out  [kernel.kallsyms]  [k] _raw_spin_lock
> > +  17.47%            a.out  [kernel.kallsyms]  [k] _raw_spin_lock
> 
> I had somewhat high expectations, but that's just better than I really
> hoped for. Not only is the percentage down, it's down for the case of
> a much smaller number of overall cycle cost, so it's a really big
> reduction in contention spinning.
> 
> Of course, contention will come back and overwhelm you at *some*
> point, but it seems the patches certainly moved the really bad
> contention point out some way..
> 
> > +   6.14%            a.out  [kernel.kallsyms]  [k] sys_semtimedop
> > +  11.08%            a.out  [kernel.kallsyms]  [k] sys_semtimedop
> > While the _raw_spin_lock time is drastically reduced, others do increase.
> > This results in an overall speedup of ~1.7x regarding ops/sec.
> 
> Actually, the others don't really increase. Sure, the *percentages* go
> up, but that's just because it has to add up to 100% in the end. So
> it's not that you're moving costs from one place to another - the 1.7x
> speedup is the real reduction in costs, and then that 6.14% -> 11.08%
> "growth" is really nothing but that (and yes, 1.7 x 6.14 really does
> get pretty close).
> 
> So nothing really got slower, despite the percentages going up.
> 
> Looks good to me. Of course, the *real* issue is if this is a win on
> real code too. And I bet it is, it just won't be quite as noticeable.
> But if anything, real code is likely to have less contention to begin
> with, because it has more things going on outside of the spinlocks. So
> it should see an improvement, but not nearly the kind of improvement
> you quote here.
> 
> Although your 800-user swingbench numbers were pretty horrible, so
> maybe that case can improve by comparable amounts in the bad cases.
> 

Absolutely, I'll be sure to try these changes with my Oracle workloads
and report with some numbers. This obviously still needs a lot of
testing.

Thanks,
Davidlohr


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2013-03-02 21:23 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-03-02  0:16 [RFC PATCH 0/2] ipc: do not hold ipc lock more than necessary Davidlohr Bueso
2013-03-02  1:32 ` Linus Torvalds
2013-03-02  1:35   ` Linus Torvalds
2013-03-02 21:23   ` Davidlohr Bueso
2013-03-02  4:43 ` Emmanuel Benisty
2013-03-02  7:08   ` Michel Lespinasse
2013-03-02  8:35     ` Emmanuel Benisty
2013-03-02 21:20       ` Davidlohr Bueso

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.