question on freeze and aio

question on freeze and aio

[Oliver Neukum]

Hi,

can there be running aio requests if all tasks are frozen?

	Regards
		Oliver

Re: question on freeze and aio

[Rafael J. Wysocki]

On Friday, 9 February 2007 17:04, Oliver Neukum wrote:
> Hi,
> 
> can there be running aio requests if all tasks are frozen?

I think so.  We have to be able to save the image after all. :-)

Greetings,
Rafael

Re: question on freeze and aio

[Nigel Cunningham]

Hi.

On Fri, 2007-02-09 at 17:04 +0100, Oliver Neukum wrote:
> Hi,
> 
> can there be running aio requests if all tasks are frozen?

Since tasks are frozen, no new aio requests would be submitted. We do a
sync to ssek to complete existing requests; I haven't looked to see
whether that would need extending to flush aio as well, but don't think
that would be the case. Given everyone else's silence, so far, you might
be better to ask the author(s) of the aio code.

(As I type Rafael's reply has arrived, but I think my answer might still
be helpful, so I'll send it too).

Regards,

Nigel

Re: question on freeze and aio

[Alan Stern]

On Sat, 10 Feb 2007, Nigel Cunningham wrote:

> Hi.
> 
> On Fri, 2007-02-09 at 17:04 +0100, Oliver Neukum wrote:
> > Hi,
> > 
> > can there be running aio requests if all tasks are frozen?
> 
> Since tasks are frozen, no new aio requests would be submitted. We do a
> sync to ssek to complete existing requests; I haven't looked to see
> whether that would need extending to flush aio as well, but don't think
> that would be the case. Given everyone else's silence, so far, you might
> be better to ask the author(s) of the aio code.

I believe Nigel is basically correct.  However sync will only flush aio 
for block devices.  Other types of aio will remain active.

aio callbacks are interrupt-driven and execute in a workqueue (see
fs/aio.c), and as far as I can tell that workqueue doesn't freeze.  
Unless some general facility freezes all workqueues...

Alan Stern

Re: question on freeze and aio

[Nigel Cunningham]

Hi.

On Fri, 2007-02-09 at 17:03 -0500, Alan Stern wrote:
> On Sat, 10 Feb 2007, Nigel Cunningham wrote:
> 
> > Hi.
> > 
> > On Fri, 2007-02-09 at 17:04 +0100, Oliver Neukum wrote:
> > > Hi,
> > > 
> > > can there be running aio requests if all tasks are frozen?
> > 
> > Since tasks are frozen, no new aio requests would be submitted. We do a
> > sync to ssek to complete existing requests; I haven't looked to see
> > whether that would need extending to flush aio as well, but don't think
> > that would be the case. Given everyone else's silence, so far, you might
> > be better to ask the author(s) of the aio code.
> 
> I believe Nigel is basically correct.  However sync will only flush aio 
> for block devices.  Other types of aio will remain active.
> 
> aio callbacks are interrupt-driven and execute in a workqueue (see
> fs/aio.c), and as far as I can tell that workqueue doesn't freeze.  
> Unless some general facility freezes all workqueues...

Hmm. That applies to Raid, too? I've had some reports of raid i/o being
caught in the BUG_ON() I have to catch I/O that shouldn't be happening,
but haven't gotten around to looking at them yet.

Regards,

Nigel

Re: question on freeze and aio

[Rafael J. Wysocki]

On Friday, 9 February 2007 23:03, Alan Stern wrote:
> On Sat, 10 Feb 2007, Nigel Cunningham wrote:
> 
> > Hi.
> > 
> > On Fri, 2007-02-09 at 17:04 +0100, Oliver Neukum wrote:
> > > Hi,
> > > 
> > > can there be running aio requests if all tasks are frozen?
> > 
> > Since tasks are frozen, no new aio requests would be submitted. We do a
> > sync to ssek to complete existing requests; I haven't looked to see
> > whether that would need extending to flush aio as well, but don't think
> > that would be the case. Given everyone else's silence, so far, you might
> > be better to ask the author(s) of the aio code.
> 
> I believe Nigel is basically correct.  However sync will only flush aio 
> for block devices.  Other types of aio will remain active.
> 
> aio callbacks are interrupt-driven and execute in a workqueue (see
> fs/aio.c), and as far as I can tell that workqueue doesn't freeze.  
> Unless some general facility freezes all workqueues...

No, workqueues are not frozen unless they are explicitly declared as
freezeable.  Which currently is only done by XFS, AFAIKS.

Greetings,
Rafael


-- 
If you don't have the time to read,
you don't have the time or the tools to write.
		- Stephen King

Re: question on freeze and aio

[Rafael J. Wysocki]

On Friday, 9 February 2007 23:08, Nigel Cunningham wrote:
> Hi.
> 
> On Fri, 2007-02-09 at 17:03 -0500, Alan Stern wrote:
> > On Sat, 10 Feb 2007, Nigel Cunningham wrote:
> > 
> > > Hi.
> > > 
> > > On Fri, 2007-02-09 at 17:04 +0100, Oliver Neukum wrote:
> > > > Hi,
> > > > 
> > > > can there be running aio requests if all tasks are frozen?
> > > 
> > > Since tasks are frozen, no new aio requests would be submitted. We do a
> > > sync to ssek to complete existing requests; I haven't looked to see
> > > whether that would need extending to flush aio as well, but don't think
> > > that would be the case. Given everyone else's silence, so far, you might
> > > be better to ask the author(s) of the aio code.
> > 
> > I believe Nigel is basically correct.  However sync will only flush aio 
> > for block devices.  Other types of aio will remain active.
> > 
> > aio callbacks are interrupt-driven and execute in a workqueue (see
> > fs/aio.c), and as far as I can tell that workqueue doesn't freeze.  
> > Unless some general facility freezes all workqueues...
> 
> Hmm. That applies to Raid, too? I've had some reports of raid i/o being
> caught in the BUG_ON() I have to catch I/O that shouldn't be happening,
> but haven't gotten around to looking at them yet.

Yes, I think it applies to RAID, but that's because md_thread has been made
PF_NOFREEZE explicitly, since we want to be able to save the image to an
md-based swap.

Pretty much everything that uses workqueues can do that, too.

Greetings,
Rafael


-- 
If you don't have the time to read,
you don't have the time or the tools to write.
		- Stephen King

Re: question on freeze and aio

[Alan Stern]

On Sat, 10 Feb 2007, Nigel Cunningham wrote:

> > aio callbacks are interrupt-driven and execute in a workqueue (see
> > fs/aio.c), and as far as I can tell that workqueue doesn't freeze.  
> > Unless some general facility freezes all workqueues...
> 
> Hmm. That applies to Raid, too? I've had some reports of raid i/o being
> caught in the BUG_ON() I have to catch I/O that shouldn't be happening,
> but haven't gotten around to looking at them yet.

Sorry, I don't know how the RAID drivers work.

Alan Stern

Re: question on freeze and aio

[Pavel Machek]

Hi!

> can there be running aio requests if all tasks are frozen?

If you see them running, you've found a kernel bug.

No, really, I do not think so. AFAIK, even aio does its i/o requests
from process context, no?
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: question on freeze and aio

[Pavel Machek]

Hi!

> > > can there be running aio requests if all tasks are frozen?
> > 
> > Since tasks are frozen, no new aio requests would be submitted. We do a
> > sync to ssek to complete existing requests; I haven't looked to see
> > whether that would need extending to flush aio as well, but don't think
> > that would be the case. Given everyone else's silence, so far, you might
> > be better to ask the author(s) of the aio code.
> 
> I believe Nigel is basically correct.  However sync will only flush aio 
> for block devices.  Other types of aio will remain active.
> 
> aio callbacks are interrupt-driven and execute in a workqueue (see
> fs/aio.c), and as far as I can tell that workqueue doesn't freeze.  
> Unless some general facility freezes all workqueues...

If this is so... we need to make that particular workqueue
freezeable. (And we should really make most workqueues freezeable).

									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: question on freeze and aio

[Pavel Machek]

Hi!

> > > > can there be running aio requests if all tasks are frozen?
> > > 
> > > Since tasks are frozen, no new aio requests would be submitted. We do a
> > > sync to ssek to complete existing requests; I haven't looked to see
> > > whether that would need extending to flush aio as well, but don't think
> > > that would be the case. Given everyone else's silence, so far, you might
> > > be better to ask the author(s) of the aio code.
> > 
> > I believe Nigel is basically correct.  However sync will only flush aio 
> > for block devices.  Other types of aio will remain active.
> > 
> > aio callbacks are interrupt-driven and execute in a workqueue (see
> > fs/aio.c), and as far as I can tell that workqueue doesn't freeze.  
> > Unless some general facility freezes all workqueues...
> 
> No, workqueues are not frozen unless they are explicitly declared as
> freezeable.  Which currently is only done by XFS, AFAIKS.

Ok, then we have a bug in swsusp. Can someone locate _which_ workqueue
is used for aio... and make it freezeable?
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: question on freeze and aio

[Rafael J. Wysocki]

On Tuesday, 13 February 2007 11:18, Pavel Machek wrote:
> Hi!
> 
> > > > > can there be running aio requests if all tasks are frozen?
> > > > 
> > > > Since tasks are frozen, no new aio requests would be submitted. We do a
> > > > sync to ssek to complete existing requests; I haven't looked to see
> > > > whether that would need extending to flush aio as well, but don't think
> > > > that would be the case. Given everyone else's silence, so far, you might
> > > > be better to ask the author(s) of the aio code.
> > > 
> > > I believe Nigel is basically correct.  However sync will only flush aio 
> > > for block devices.  Other types of aio will remain active.
> > > 
> > > aio callbacks are interrupt-driven and execute in a workqueue (see
> > > fs/aio.c), and as far as I can tell that workqueue doesn't freeze.  
> > > Unless some general facility freezes all workqueues...
> > 
> > No, workqueues are not frozen unless they are explicitly declared as
> > freezeable.  Which currently is only done by XFS, AFAIKS.
> 
> Ok, then we have a bug in swsusp. Can someone locate _which_ workqueue
> is used for aio... and make it freezeable?

Hm, so AIO is not needed for saving the image regardless of the system
configuration?

Rafael

Re: question on freeze and aio

[Rafael J. Wysocki]

On Tuesday, 13 February 2007 11:17, Pavel Machek wrote:
> Hi!
> 
> > > > can there be running aio requests if all tasks are frozen?
> > > 
> > > Since tasks are frozen, no new aio requests would be submitted. We do a
> > > sync to ssek to complete existing requests; I haven't looked to see
> > > whether that would need extending to flush aio as well, but don't think
> > > that would be the case. Given everyone else's silence, so far, you might
> > > be better to ask the author(s) of the aio code.
> > 
> > I believe Nigel is basically correct.  However sync will only flush aio 
> > for block devices.  Other types of aio will remain active.
> > 
> > aio callbacks are interrupt-driven and execute in a workqueue (see
> > fs/aio.c), and as far as I can tell that workqueue doesn't freeze.  
> > Unless some general facility freezes all workqueues...
> 
> If this is so... we need to make that particular workqueue
> freezeable. (And we should really make most workqueues freezeable).

Except for the ones that are needed to save the image.  Do we know which
ones they are?

Rafael

Re: question on freeze and aio

[Alan Stern]

On Tue, 13 Feb 2007, Pavel Machek wrote:

> > No, workqueues are not frozen unless they are explicitly declared as
> > freezeable.  Which currently is only done by XFS, AFAIKS.
> 
> Ok, then we have a bug in swsusp. Can someone locate _which_ workqueue
> is used for aio... and make it freezeable?

Here is an untested patch.  Does anybody have a way to test it?

(BTW, should it be spelled "freezeable" or "freezable"?)

Alan Stern



Index: usb-2.6/fs/aio.c
===================================================================
--- usb-2.6.orig/fs/aio.c
+++ usb-2.6/fs/aio.c
@@ -73,7 +73,7 @@ static int __init aio_setup(void)
 	kioctx_cachep = kmem_cache_create("kioctx", sizeof(struct kioctx),
 				0, SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL, NULL);
 
-	aio_wq = create_workqueue("aio");
+	aio_wq = create_freezeable_workqueue("aio");
 
 	pr_debug("aio_setup: sizeof(struct page) = %d\n", (int)sizeof(struct page));
 

Re: question on freeze and aio

[Randy Dunlap]

On Tue, 13 Feb 2007 16:01:30 -0500 (EST) Alan Stern wrote:

> On Tue, 13 Feb 2007, Pavel Machek wrote:
> 
> > > No, workqueues are not frozen unless they are explicitly declared as
> > > freezeable.  Which currently is only done by XFS, AFAIKS.
> > 
> > Ok, then we have a bug in swsusp. Can someone locate _which_ workqueue
> > is used for aio... and make it freezeable?
> 
> Here is an untested patch.  Does anybody have a way to test it?
> 
> (BTW, should it be spelled "freezeable" or "freezable"?)

www.dict.org says freezable.


> Index: usb-2.6/fs/aio.c
> ===================================================================
> --- usb-2.6.orig/fs/aio.c
> +++ usb-2.6/fs/aio.c
> @@ -73,7 +73,7 @@ static int __init aio_setup(void)
>  	kioctx_cachep = kmem_cache_create("kioctx", sizeof(struct kioctx),
>  				0, SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL, NULL);
>  
> -	aio_wq = create_workqueue("aio");
> +	aio_wq = create_freezeable_workqueue("aio");
>  
>  	pr_debug("aio_setup: sizeof(struct page) = %d\n", (int)sizeof(struct page));


---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

Re: question on freeze and aio

[Rafael J. Wysocki]

On Tuesday, 13 February 2007 22:01, Alan Stern wrote:
> On Tue, 13 Feb 2007, Pavel Machek wrote:
> 
> > > No, workqueues are not frozen unless they are explicitly declared as
> > > freezeable.  Which currently is only done by XFS, AFAIKS.
> > 
> > Ok, then we have a bug in swsusp. Can someone locate _which_ workqueue
> > is used for aio... and make it freezeable?
> 
> Here is an untested patch.  Does anybody have a way to test it?

I'll give it a run on my test system.

Greetings,
Rafael

Re: question on freeze and aio

[Rafael J. Wysocki]

On Wednesday, 14 February 2007 18:25, Randy Dunlap wrote:
> On Tue, 13 Feb 2007 16:01:30 -0500 (EST) Alan Stern wrote:
> 
> > On Tue, 13 Feb 2007, Pavel Machek wrote:
> > 
> > > > No, workqueues are not frozen unless they are explicitly declared as
> > > > freezeable.  Which currently is only done by XFS, AFAIKS.
> > > 
> > > Ok, then we have a bug in swsusp. Can someone locate _which_ workqueue
> > > is used for aio... and make it freezeable?
> > 
> > Here is an untested patch.  Does anybody have a way to test it?
> > 
> > (BTW, should it be spelled "freezeable" or "freezable"?)
> 
> www.dict.org says freezable.

Yeah, but we have always spelled "freezeable".  Google search shows both are
being used.

Greetings,
Rafael


-- 
If you don't have the time to read,
you don't have the time or the tools to write.
		- Stephen King

Re: question on freeze and aio

[Pavel Machek]

Hi!

> > Ok, then we have a bug in swsusp. Can someone locate _which_ workqueue
> > is used for aio... and make it freezeable?
> 
> Hm, so AIO is not needed for saving the image regardless of the system
> configuration?

No, definitely not.
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: question on freeze and aio

[Rafael J. Wysocki]

On Wednesday, 14 February 2007 22:28, Rafael J. Wysocki wrote:
> On Tuesday, 13 February 2007 22:01, Alan Stern wrote:
> > On Tue, 13 Feb 2007, Pavel Machek wrote:
> > 
> > > > No, workqueues are not frozen unless they are explicitly declared as
> > > > freezeable.  Which currently is only done by XFS, AFAIKS.
> > > 
> > > Ok, then we have a bug in swsusp. Can someone locate _which_ workqueue
> > > is used for aio... and make it freezeable?
> > 
> > Here is an untested patch.  Does anybody have a way to test it?
> 
> I'll give it a run on my test system.

Unfortunately, it seems to break the suspend.

The system doesn't suspend, but enters an endless loop right after the freezing
of consoles.

I'll have to debug it a bit more on a machine with a serial console, but not
today.

Well, it looks like there's something we didn't know about. :-)

Greetings,
Rafael