Re: [Qemu-devel] 'qemu-nbd' explicite flush to disk

Re: [Qemu-devel] 'qemu-nbd' explicite flush to disk

[Mark Trumpold]

Paolo,

Please recall you had sent me the link below to a kernel patch to allow
'qemu-nbd' to flush to disk.  The patch was applied with good success.

However, recent testing indicates that filesystem corruption issue I am
seeing seems to be tied to large (>500mbyte) filesystem writes.  I also
observe the kernel 'flush' thread approaching 100% cpu at times, and i/o
wait % as reported by 'top' increasing sharply during the copy/write.

Was wondering if perhaps there are any special kernel config options that
I may be missing for 'nbd', or any other information that might help.

Best regards,
Mark Trumpold


>-----Original Message-----
>From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo
>Bonzini
>Sent: Wednesday, September 12, 2012 11:30 PM
>To: Mark Trumpold
>Cc: qemu-devel@nongnu.org
>Subject: Re: 'qemu-nbd' explicite flush to disk
>Il 12/09/2012 23:28, Mark Trumpold ha scritto:
>> So, I've been experimenting with 'qemu-nbd --cache=writeback ..'
>> This nicely eliminates the 'checkpoint' issue; however, I have as
>> yet been unable to explicitely flush things to disk -- which I would
>>like to
>> do just before a 'nilfs' snapshot.

>The Linux kernel driver for NBD does not support flushes.  Patches were
>sent to the maintainer, but he never applied them.
>You can get them at
>http://thread.gmane.org/gmane.linux.drivers.nbd.general/1108 and try
>them.
>> Subsequently I've been trying to call 'bdrv_co_flush(bs)' directly, but
>>I can't
>figure out how to dereference 'bs' for the call.
>>
>> I'm probably out in the weeds on this one.
>> Any guidance would be greatly appreciated.
>>
>> I am running:
>>     qemu-1.2.0
>>     linux kernel 3.3.1
>>
>> Thank you,
>> Mark Trumpold

[Qemu-devel] FW: 'qemu-nbd' explicite flush to disk

[Mark Trumpold]

Hello Paolo,

Before anyone expends any cycles on my last message,  I have new
information.

Today I was able to eliminate the file corruption issue by doing a
'blockdev --flushbufs' prior to taking a snapshot.  I haven't done
extensive testing yet, but it seems the issue if gone at this point.

So sorry for the disruption.

Best regards,
Mark Trumpold


On 10/22/12 8:10 AM, "Mark Trumpold" <markt@netqa.com> wrote:

>Paolo,
>
>Please recall you had sent me the link below to a kernel patch to allow
>'qemu-nbd' to flush to disk.  The patch was applied with good success.
>
>However, recent testing indicates that filesystem corruption issue I am
>seeing seems to be tied to large (>500mbyte) filesystem writes.  I also
>observe the kernel 'flush' thread approaching 100% cpu at times, and i/o
>wait % as reported by 'top' increasing sharply during the copy/write.
>
>Was wondering if perhaps there are any special kernel config options that
>I may be missing for 'nbd', or any other information that might help.
>
>Best regards,
>Mark Trumpold
>
>
>>-----Original Message-----
>>From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo
>>Bonzini
>>Sent: Wednesday, September 12, 2012 11:30 PM
>>To: Mark Trumpold
>>Cc: qemu-devel@nongnu.org
>>Subject: Re: 'qemu-nbd' explicite flush to disk
>>Il 12/09/2012 23:28, Mark Trumpold ha scritto:
>>> So, I've been experimenting with 'qemu-nbd --cache=writeback ..'
>>> This nicely eliminates the 'checkpoint' issue; however, I have as
>>> yet been unable to explicitely flush things to disk -- which I would
>>>like to
>>> do just before a 'nilfs' snapshot.
>
>>The Linux kernel driver for NBD does not support flushes.  Patches were
>>sent to the maintainer, but he never applied them.
>>You can get them at
>>http://thread.gmane.org/gmane.linux.drivers.nbd.general/1108 and try
>>them.
>>> Subsequently I've been trying to call 'bdrv_co_flush(bs)' directly,
>>>but 
>>>I can't
>>figure out how to dereference 'bs' for the call.
>>>
>>> I'm probably out in the weeds on this one.
>>> Any guidance would be greatly appreciated.
>>>
>>> I am running:
>>>     qemu-1.2.0
>>>     linux kernel 3.3.1
>>>
>>> Thank you,
>>> Mark Trumpold

Re: [Qemu-devel] FW: 'qemu-nbd' explicite flush to disk

[Paolo Bonzini]

Il 23/10/2012 04:50, Mark Trumpold ha scritto:
> Hello Paolo,
> 
> Before anyone expends any cycles on my last message,  I have new
> information.
> 
> Today I was able to eliminate the file corruption issue by doing a
> 'blockdev --flushbufs' prior to taking a snapshot.

Makes sense.  Even better, you could do fsfreeze -f before and fsfreeze
-u afterwards.

Paolo

Re: [Qemu-devel] 'qemu-nbd' explicite flush to disk

[Mark Trumpold]

Paolo,

Awesome!!!  I just tried your patch, and it works beautifully.
I've been struggling with the lack of '--cache=writeback' for
months now with many work-arounds/kludges.

Thank you again for responding.
Qemu rocks!

Best regards,
Mark Trumpold



> -----Original Message-----
> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo
> Bonzini
> Sent: Wednesday, September 12, 2012 11:30 PM
> To: Mark Trumpold
> Cc: qemu-devel@nongnu.org
> Subject: Re: 'qemu-nbd' explicite flush to disk
>
> Il 12/09/2012 23:28, Mark Trumpold ha scritto:
> > So, I've been experimenting with 'qemu-nbd --cache=writeback ..'
> > This nicely eliminates the 'checkpoint' issue; however, I have as
> > yet been unable to explicitely flush things to disk -- which I would like to
> > do just before a 'nilfs' snapshot.
>
> The Linux kernel driver for NBD does not support flushes.  Patches were
> sent to the maintainer, but he never applied them.
>
> You can get them at
> http://thread.gmane.org/gmane.linux.drivers.nbd.general/1108 and try
> them.
>
> > Subsequently I've been trying to call 'bdrv_co_flush(bs)' directly, but I can't
> figure out how to dereference 'bs' for the call.
> >
> > I'm probably out in the weeds on this one.
> > Any guidance would be greatly appreciated.
> >
> > I am running:
> >     qemu-1.2.0
> >     linux kernel 3.3.1
> >
> > Thank you,
> > Mark Trumpold
> >
> >
> > Confidentiality Notice: The information contained in this electronic
> > e-mail and any accompanying attachment(s) is intended only for the use
> > of the intended recipient and is confidential and/or privileged.
>
> I doubt so, since this is a public mailing list.
>
> Paolo


Confidentiality Notice:  The information contained in this electronic e-mail and any accompanying attachment(s) is intended only for the use of the intended recipient and is confidential and/or privileged. If you and we have a confidentiality agreement or other non-disclosure obligations between us, this Notice shall be deemed to mark and identify the content of this email and any attachments as confidential and proprietary.   If any reader of this communication is not the intended recipient, unauthorized use, disclosure or copying is strictly prohibited, and may be unlawful.  If you have received this communication in error, please immediately notify the sender by return e-mail, and delete the original message and all copies from your system.  Thank you.

IRS Circular 230 Disclosure: To ensure compliance with requirements imposed by the IRS, please be advised that any U.S. federal tax advice contained in this communication (including any attachments) is not intended or written to be used or relied upon, and cannot be used or relied upon, for the purpose of (i) avoiding penalties under the Internal Revenue Code, or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein.

E-mail is susceptible to data corruption, interception, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof.

Re: [Qemu-devel] 'qemu-nbd' explicite flush to disk

[Paolo Bonzini]

Il 12/09/2012 23:28, Mark Trumpold ha scritto:
> So, I've been experimenting with 'qemu-nbd --cache=writeback ..'
> This nicely eliminates the 'checkpoint' issue; however, I have as
> yet been unable to explicitely flush things to disk -- which I would like to
> do just before a 'nilfs' snapshot.

The Linux kernel driver for NBD does not support flushes.  Patches were
sent to the maintainer, but he never applied them.

You can get them at
http://thread.gmane.org/gmane.linux.drivers.nbd.general/1108 and try them.

> Subsequently I've been trying to call 'bdrv_co_flush(bs)' directly, but I can't figure out how to dereference 'bs' for the call.
> 
> I'm probably out in the weeds on this one.
> Any guidance would be greatly appreciated.
> 
> I am running:
>     qemu-1.2.0
>     linux kernel 3.3.1
> 
> Thank you,
> Mark Trumpold
> 
> 
> Confidentiality Notice: The information contained in this electronic
> e-mail and any accompanying attachment(s) is intended only for the use
> of the intended recipient and is confidential and/or privileged.

I doubt so, since this is a public mailing list.

Paolo

[Qemu-devel] 'qemu-nbd' explicite flush to disk

[Mark Trumpold]

Hello,

I posted a similar message to 'qemu-discuss', and was thinking my question may require a developer level of expertise.  I will try to be as brief as possible.

My context is using 'qemu-nbd' and 'qemu-img' to manage 'qcow2' loop filesystem images being hosted on a 'nilfs' filesystem.  This way I can 'nilfs snapshot' the images.

The problem is 'nilfs' creates a 'checkpoint' for every synchronous write, and 'qemu-nbd's default cache mode is 'writethrough'.   Disk space is consumed quickly with the 'checkpoint' overhead in this case.

So, I've been experimenting with 'qemu-nbd --cache=writeback ..'
This nicely eliminates the  'checkpoint' issue; however, I have as yet been unable to explicitely flush things to disk -- which I would like to do just before a 'nilfs' snapshot.

I've tried the 'bdrv_drain_all() / bdrv_flush_all()'.   I instrumented and found the 'bdrv_co_flush(bs)' doesn't get called as expected in the loop.  I had one image connected as follows for this test:
    qemu-nbd -v --cache=writeback -c /dev/nbd5 /images/root/snap.img

Subsequently I've been trying to call 'bdrv_co_flush(bs)' directly, but I can't figure out how to dereference 'bs' for the call.

I'm probably out in the weeds on this one.
Any guidance would be greatly appreciated.

I am running:
    qemu-1.2.0
    linux kernel 3.3.1

Thank you,
Mark Trumpold


Confidentiality Notice:  The information contained in this electronic e-mail and any accompanying attachment(s) is intended only for the use of the intended recipient and is confidential and/or privileged. If you and we have a confidentiality agreement or other non-disclosure obligations between us, this Notice shall be deemed to mark and identify the content of this email and any attachments as confidential and proprietary.   If any reader of this communication is not the intended recipient, unauthorized use, disclosure or copying is strictly prohibited, and may be unlawful.  If you have received this communication in error, please immediately notify the sender by return e-mail, and delete the original message and all copies from your system.  Thank you.

IRS Circular 230 Disclosure: To ensure compliance with requirements imposed by the IRS, please be advised that any U.S. federal tax advice contained in this communication (including any attachments) is not intended or written to be used or relied upon, and cannot be used or relied upon, for the purpose of (i) avoiding penalties under the Internal Revenue Code, or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein.

E-mail is susceptible to data corruption, interception, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof.