request of blkfront may disorder after migration without waiting in suspend phase of migration

request of blkfront may disorder after migration without waiting in suspend phase of migration

[alice wan]

[-- Attachment #1.1: Type: text/plain, Size: 996 bytes --]

Hi konrad,

I found the mainline linux 3.0-rc6 xen-blkfront driver doesn't implement
suspend func. In my opinion, the blkfront should wait till responses of all
the pending requests come back, then domu can be suspended,
otherwise,  after the migration, vm of target host will request from target
host, meanwhile, the blkback of source host maybe do the pending requests
until they's done.

if migrate the vm with high io pressure,  the requests of this vm may
disorder.

And i saw the gpl windows pv driver, blkfront will change its state and wait
for blkback closing, after blkback closing, the pending requests definitely
are done.

More, i did some test with migration,  while vm was running dd task,  xm
migrate -l .  before it suspend, gdb tapdisk2.  after migration, new vm
started. then quit gdb, the pending request were written back to disk after
new requests.

above mentioned is just my opinion, i need your advice and confirmation or
deny

any help is appreciated.


Regards

wanjia

[-- Attachment #1.2: Type: text/html, Size: 1236 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: request of blkfront may disorder after migration without waiting in suspend phase of migration

[Daniel Stodden]

On Sun, 2011-07-10 at 00:36 -0400, alice wan wrote:
> Hi konrad,
> 
> I found the mainline linux 3.0-rc6 xen-blkfront driver doesn't implement suspend func. In my opinion, the blkfront should wait till responses of all the pending requests come back, then domu can be suspended, otherwise,  after the migration, vm of target host will request from target host, meanwhile, the blkback of source host maybe do the pending requests until they's done.

It's not done that way because the suspension alone is rather
transparent to the guest. There's normally not a problem with that. As
long as I/O completed on the source host before resuming the new machine
image (which will reissue the requests not observed complete).

This is a relatively safe procedure as long as the toolstack makes sure
the backend shut down before handing over.

> if migrate the vm with high io pressure,  the requests of this vm may disorder.
> 
> And i saw the gpl windows pv driver, blkfront will change its state and wait for blkback closing, after blkback closing, the pending requests definitely are done.

Huh? I don't know much about the GPL PV drivers. I do believe however
shouldn't be able observe blkback transitioning to Closing. The guest
should have been halted and destroyed before that even happens.

> More, i did some test with migration,  while vm was running dd task,  xm migrate -l .  before it suspend, gdb tapdisk2.  after migration, new vm started. then quit gdb, the pending request were written back to disk after new requests.
> 
> above mentioned is just my opinion, i need your advice and confirmation or deny
> 
> any help is appreciated.

Woah. Could someone with xm know how confirm or deny that? The former
will break disk images. 

It's quite fundamental that blkback was completely shut down (rm
backend/vbd doesn't count, it state hit Closed, i.e. xenblkd gone and
bdev release()d) before resuming.

Daniel

Re: request of blkfront may disorder after migration without waiting in suspend phase of migration

[Konrad Rzeszutek Wilk]

On Sun, Jul 10, 2011 at 12:36:06PM +0800, alice wan wrote:
> Hi konrad,
> 
> I found the mainline linux 3.0-rc6 xen-blkfront driver doesn't implement
> suspend func. In my opinion, the blkfront should wait till responses of all
> the pending requests come back, then domu can be suspended,

I think you mean blkback. It should be the job of blkback to account for
all I/Os and make sure they have completed before migrating?

I did have a patch for the 2.6.32 to deal with this but I never was able
to reproduce this. Are you able to reproduce this? Can you give me
step by step instructions on how to do it?

> otherwise,  after the migration, vm of target host will request from target
> host, meanwhile, the blkback of source host maybe do the pending requests
> until they's done.
> 
> if migrate the vm with high io pressure,  the requests of this vm may
> disorder.
> 
> And i saw the gpl windows pv driver, blkfront will change its state and wait
> for blkback closing, after blkback closing, the pending requests definitely
> are done.
> 
> More, i did some test with migration,  while vm was running dd task,  xm
> migrate -l .  before it suspend, gdb tapdisk2.  after migration, new vm
> started. then quit gdb, the pending request were written back to disk after
> new requests.
> 
> above mentioned is just my opinion, i need your advice and confirmation or
> deny
> 
> any help is appreciated.
> 
> 
> Regards
> 
> wanjia

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: request of blkfront may disorder after migration without waiting in suspend phase of migration

[Ian Campbell]

On Tue, 2011-07-12 at 19:23 +0100, Konrad Rzeszutek Wilk wrote:
> On Sun, Jul 10, 2011 at 12:36:06PM +0800, alice wan wrote:
> > Hi konrad,
> > 
> > I found the mainline linux 3.0-rc6 xen-blkfront driver doesn't implement
> > suspend func. In my opinion, the blkfront should wait till responses of all
> > the pending requests come back, then domu can be suspended,
> 
> I think you mean blkback. It should be the job of blkback to account for
> all I/Os and make sure they have completed before migrating?

blkfront has a resume hook which (AIUI) is intended to recover things
after a migration and this seemingly does not require any work on the
suspend path (or at least was presumably thought not to when it was
written...).

Ian.

> 
> I did have a patch for the 2.6.32 to deal with this but I never was able
> to reproduce this. Are you able to reproduce this? Can you give me
> step by step instructions on how to do it?
> 
> > otherwise,  after the migration, vm of target host will request from target
> > host, meanwhile, the blkback of source host maybe do the pending requests
> > until they's done.
> > 
> > if migrate the vm with high io pressure,  the requests of this vm may
> > disorder.
> > 
> > And i saw the gpl windows pv driver, blkfront will change its state and wait
> > for blkback closing, after blkback closing, the pending requests definitely
> > are done.
> > 
> > More, i did some test with migration,  while vm was running dd task,  xm
> > migrate -l .  before it suspend, gdb tapdisk2.  after migration, new vm
> > started. then quit gdb, the pending request were written back to disk after
> > new requests.
> > 
> > above mentioned is just my opinion, i need your advice and confirmation or
> > deny
> > 
> > any help is appreciated.
> > 
> > 
> > Regards
> > 
> > wanjia
> 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: request of blkfront may disorder after migration without waiting in suspend phase of migration

[alice wan]

[-- Attachment #1.1: Type: text/plain, Size: 2653 bytes --]

2011/7/13 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

> On Sun, Jul 10, 2011 at 12:36:06PM +0800, alice wan wrote:
> > Hi konrad,
> >
> > I found the mainline linux 3.0-rc6 xen-blkfront driver doesn't implement
> > suspend func. In my opinion, the blkfront should wait till responses of
> all
> > the pending requests come back, then domu can be suspended,
>
> I think you mean blkback. It should be the job of blkback to account for
> all I/Os and make sure they have completed before migrating?
>
> I did have a patch for the 2.6.32 to deal with this but I never was able
> to reproduce this. Are you able to reproduce this? Can you give me
> step by step instructions on how to do it?
>

yeah,  just because i didn't see any code in blkback to waiting for IO
before migrating,
blkfront's the rest one to do this job that it actively switches to Closing
state, then blkback can observe it and call blkif_disconnect to account for
all IO.

my test procedure is like this
> More, i did some test with migration,  while vm was running dd task,  xm
> migrate -l .  before it suspend, gdb tapdisk2.  after migration, new vm
> started. then quit gdb, the pending request were written back to disk
after
> new requests.

gdb tapdisk2 , the IO of src host hanged, when xc_save did suspend, after
100s, socket was shutted down, Vm of target host resumed and continued to
dd, then quit gdb.

this test is quite extreme. normal migration depends on whether all IO can
be completed in 100s.

well, i need the IO have 100% safety, especially in the distributed fs.

of course, blkback is the best choice to ensure this.

>
> > otherwise,  after the migration, vm of target host will request from
> target
> > host, meanwhile, the blkback of source host maybe do the pending requests
> > until they's done.
> >
> > if migrate the vm with high io pressure,  the requests of this vm may
> > disorder.
> >
> > And i saw the gpl windows pv driver, blkfront will change its state and
> wait
> > for blkback closing, after blkback closing, the pending requests
> definitely
> > are done.
> >
> > More, i did some test with migration,  while vm was running dd task,  xm
> > migrate -l .  before it suspend, gdb tapdisk2.  after migration, new vm
> > started. then quit gdb, the pending request were written back to disk
> after
> > new requests.
> >
> > above mentioned is just my opinion, i need your advice and confirmation
> or
> > deny
> >
> > any help is appreciated.
> >
> >
> > Regards
> >
> > wanjia
>
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
>
>

[-- Attachment #1.2: Type: text/html, Size: 3562 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: request of blkfront may disorder after migration without waiting in suspend phase of migration

[Daniel Stodden]

On Wed, 2011-07-13 at 05:09 -0400, Ian Campbell wrote: 
> On Tue, 2011-07-12 at 19:23 +0100, Konrad Rzeszutek Wilk wrote:
> > On Sun, Jul 10, 2011 at 12:36:06PM +0800, alice wan wrote:
> > > Hi konrad,
> > > 
> > > I found the mainline linux 3.0-rc6 xen-blkfront driver doesn't implement
> > > suspend func. In my opinion, the blkfront should wait till responses of all
> > > the pending requests come back, then domu can be suspended,
> > 
> > I think you mean blkback. It should be the job of blkback to account for
> > all I/Os and make sure they have completed before migrating?
> 
> blkfront has a resume hook which (AIUI) is intended to recover things
> after a migration and this seemingly does not require any work on the
> suspend path (or at least was presumably thought not to when it was
> written...).

>From a pre-copy migration perspective, adding a suspend hook to synch on
pending I/O is not ideal. It's just going to add unsolicited memory
footprint, with fairly arbitrary amounts of additional memory getting
dirtied ('arbitrary' depending on what else is allowed to keep running
in the guest in that state).

I guess that's why post-migration recovery was preferred, and I still
don't see why that's to blame.

Daniel

> Ian.
> 
> > 
> > I did have a patch for the 2.6.32 to deal with this but I never was able
> > to reproduce this. Are you able to reproduce this? Can you give me
> > step by step instructions on how to do it?
> > 
> > > otherwise,  after the migration, vm of target host will request from target
> > > host, meanwhile, the blkback of source host maybe do the pending requests
> > > until they's done.
> > > 
> > > if migrate the vm with high io pressure,  the requests of this vm may
> > > disorder.
> > > 
> > > And i saw the gpl windows pv driver, blkfront will change its state and wait
> > > for blkback closing, after blkback closing, the pending requests definitely
> > > are done.
> > > 
> > > More, i did some test with migration,  while vm was running dd task,  xm
> > > migrate -l .  before it suspend, gdb tapdisk2.  after migration, new vm
> > > started. then quit gdb, the pending request were written back to disk after
> > > new requests.
> > > 
> > > above mentioned is just my opinion, i need your advice and confirmation or
> > > deny
> > > 
> > > any help is appreciated.
> > > 
> > > 
> > > Regards
> > > 
> > > wanjia
> > 
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xensource.com
> > > http://lists.xensource.com/xen-devel
> > 
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel