linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Very slow SYS_io_destroy()
@ 2018-07-20 19:39 Alex Richman
  2018-07-23 15:13 ` Jeff Moyer
  0 siblings, 1 reply; 4+ messages in thread
From: Alex Richman @ 2018-07-20 19:39 UTC (permalink / raw)
  To: linux-kernel, linux-fsdevel

Hi,

I'm seeing some weirdness with AIO, specifically SYS_io_destroy() is 
taking upwards of ~100000 microseconds (~100 miliseconds) per call.
How long is that call expected to take?  I can see from the source that 
it's waiting e.g. for requests to finish but at the time of my call all 
requests are already finished (and SYS_io_getevent()'d)...

Here's some logs for example (<annotations for clarity>):
1532115053.520016 > hook_1(1073741824, 94766015790162, 14, 100, 
139919450371008, 15) -> 0x7ffe9e505770
<3 IOCBs prepared, CMD_PWRITEV IOVs, 1 IOV per write, 14 bytes per write>
1532115053.520035 = {prep(0 - fd: 4, o: 0, b: 0x7ffe9e504658, n: 1, p: 
0, fl: 0)}
1532115053.520037 = {prep(1 - fd: 5, o: 0, b: 0x7ffe9e504658, n: 1, p: 
0, fl: 0)}
1532115053.520038 = {prep(2 - fd: 6, o: 0, b: 0x7ffe9e504658, n: 1, p: 
0, fl: 0)}
<IOCBs submitted>
1532115053.520415 = {submit(-1978707968, 3) -> 3}
<io_events retreived>
1532115053.520417 = {done(0 - r: 14)}
1532115053.520418 = {done(1 - r: 14)}
1532115053.520419 = {done(2 - r: 14)}
1532115053.520420 = {complete(3) -> 14}
<io_cxt destroyed>
1532115053.639594 = {destroy(-1978707968) -> 0}
1532115053.639601 = (0x7ffe9e505770 = 0)

Between the complete(3) line and destroy(-etc) line only 
SYS_io_destroy() is being called.  It's the only possible source of that 
lost ~100 miliseconds...

Any ideas?

Many thanks,
- Alex.

-- 
Alex Richman
alex.r@gblabs.co.uk
Engineering
Tel:+44 (0)118 455 5000
www.gblabs.com



^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Very slow SYS_io_destroy()
  2018-07-20 19:39 Very slow SYS_io_destroy() Alex Richman
@ 2018-07-23 15:13 ` Jeff Moyer
  2018-07-23 15:16   ` Alex Richman
  0 siblings, 1 reply; 4+ messages in thread
From: Jeff Moyer @ 2018-07-23 15:13 UTC (permalink / raw)
  To: Alex Richman; +Cc: linux-kernel, linux-fsdevel

Alex Richman <alex.r@gblabs.co.uk> writes:

> Hi,
>
> I'm seeing some weirdness with AIO, specifically SYS_io_destroy() is
> taking upwards of ~100000 microseconds (~100 miliseconds) per call.
> How long is that call expected to take?  I can see from the source

Well, it waits for an RCU grace period.  I would not expect that to take
100ms, though.  What kernel version is this?

-Jeff

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Very slow SYS_io_destroy()
  2018-07-23 15:13 ` Jeff Moyer
@ 2018-07-23 15:16   ` Alex Richman
  2018-07-26 16:21     ` Pavel Machek
  0 siblings, 1 reply; 4+ messages in thread
From: Alex Richman @ 2018-07-23 15:16 UTC (permalink / raw)
  To: Jeff Moyer; +Cc: linux-kernel, linux-fsdevel

Thanks for the response.  I'm running 4.4.123-1 from Gentoo.  It also 
happens on 4.4.87-r1.

Cheers,
- Alex.

On 23/07/18 16:13, Jeff Moyer wrote:
> Alex Richman <alex.r@gblabs.co.uk> writes:
>
>> Hi,
>>
>> I'm seeing some weirdness with AIO, specifically SYS_io_destroy() is
>> taking upwards of ~100000 microseconds (~100 miliseconds) per call.
>> How long is that call expected to take?  I can see from the source
> Well, it waits for an RCU grace period.  I would not expect that to take
> 100ms, though.  What kernel version is this?
>
> -Jeff

-- 
Alex Richman
alex.r@gblabs.co.uk
Engineering
Tel:+44 (0)118 455 5000
www.gblabs.com



^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Very slow SYS_io_destroy()
  2018-07-23 15:16   ` Alex Richman
@ 2018-07-26 16:21     ` Pavel Machek
  0 siblings, 0 replies; 4+ messages in thread
From: Pavel Machek @ 2018-07-26 16:21 UTC (permalink / raw)
  To: Alex Richman; +Cc: Jeff Moyer, linux-kernel, linux-fsdevel

On Mon 2018-07-23 16:16:34, Alex Richman wrote:
> Thanks for the response.?? I'm running 4.4.123-1 from Gentoo.?? It
> also happens on 4.4.87-r1.

You may want to try 4.18 if you are out of other ideas...
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2018-07-26 16:22 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-07-20 19:39 Very slow SYS_io_destroy() Alex Richman
2018-07-23 15:13 ` Jeff Moyer
2018-07-23 15:16   ` Alex Richman
2018-07-26 16:21     ` Pavel Machek

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).