From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from eggs.gnu.org ([208.118.235.92]:37643)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <markt@netqa.com>) id 1UgHXp-0007hm-TV
	for qemu-devel@nongnu.org; Sat, 25 May 2013 12:42:16 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <markt@netqa.com>) id 1UgHXo-000285-KO
	for qemu-devel@nongnu.org; Sat, 25 May 2013 12:42:13 -0400
Received: from atl4mhob07.myregisteredsite.com ([209.17.115.45]:50403)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <markt@netqa.com>) id 1UgHXo-00027x-F5
	for qemu-devel@nongnu.org; Sat, 25 May 2013 12:42:12 -0400
Received: from mail.hostingplatform.com ([10.30.71.206])
	by atl4mhob07.myregisteredsite.com (8.14.4/8.14.4) with ESMTP id
	r4PGgApK001191
	for <qemu-devel@nongnu.org>; Sat, 25 May 2013 12:42:10 -0400
Date: Sat, 25 May 2013 09:42:08 -0800
From: Mark Trumpold <markt@netqa.com>
Message-ID: <CDC63015.85FF%markt@netqa.com>
In-Reply-To: <20130524090516.GE21639@stefanha-thinkpad.redhat.com>
Mime-version: 1.0
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit
Subject: Re: [Qemu-devel] 'qemu-nbd' explicit flush
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Stefan Hajnoczi <stefanha@gmail.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>, "qemu-devel@nongnu.org" <qemu-devel@nongnu.org>, "markt@tachyon.net" <markt@tachyon.net>

On 5/24/13 1:05 AM, "Stefan Hajnoczi" <stefanha@gmail.com> wrote:

>On Thu, May 23, 2013 at 09:58:31PM +0000, Mark Trumpold wrote:
>> I have a working configuration using the signal approach suggested by
>>Stefan.
>> 
>> 'qemu-nbd.c' is patched as follows:
>> 
>>     do {
>>         main_loop_wait(false);
>> +       if (sighup_reported) {
>> +           sighup_reported = false;
>> +           bdrv_drain_all();
>> +           bdrv_flush_all();
>>         }
>>     } while (!sigterm_reported && (persistent || !nbd_started || nb_fds
>>> 0));
>> 
>> The driving script was patched as follows:
>> 
>>      mount -o remount,ro /dev/nbd0
>>      blockdev --flushbufs /dev/nbd0
>> +    kill -HUP <qemu-nbd process id>
>> 
>> I needed to retain 'blockdev --flushbufs' for things to work.  Seems
>>the 'bdrv_flush_all' is flushing what is being missed by the blockdev
>>flush.  I did not go back an retest with 'fsync' or other approaches I
>>had tried before.
>
>Okay, that makes sense:
>
>'blockdev --flushbufs' is writing dirty pages to the NBD device.
>
>bdrv_drain_all() + bdrv_flush_all() ensures that image file writes reach
>the physical disk.
>
>One thing to be careful of is whether these operations are asynchronous.
>The signal is asynchronous, you have no way of knowing when qemu-nbd is
>finished flushing to the physical disk.

Right, of course.  I missed the obvious.

>
>I didn't check blockdev(8) but it could be the same there.
>
>So watch out, otherwise your script is timing-dependent and may not
>actually have finished flushing when you take the snapshot.
>
>Stefan
>

The race condition would not be acceptable.  You had mentioned another
approach using the socket interface:

>2. Instantiate a block/nbd.c client that connects to the running
>   qemu-nbd server (make sure format=raw).  Then call bdrv_flush() on
>   the NBD client.  You must use the qemu-nbd --shared=2 option.
>

In my case I only have a 'qemu-nbd' process per loop device.  Would the
'qemu-nbd' process act as the socket server, and I would then write a
simple socket client to instruct him to do the flush?  And, would the
client block until the flush is complete?

Thank you,
Mark T.