From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from eggs.gnu.org ([208.118.235.92]:41703)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <stefanha@gmail.com>) id 1UfnwC-0001L6-II
	for qemu-devel@nongnu.org; Fri, 24 May 2013 05:05:29 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <stefanha@gmail.com>) id 1Ufnw8-0003xv-1o
	for qemu-devel@nongnu.org; Fri, 24 May 2013 05:05:24 -0400
Received: from mail-wi0-x234.google.com ([2a00:1450:400c:c05::234]:42511)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <stefanha@gmail.com>) id 1Ufnw7-0003xo-S8
	for qemu-devel@nongnu.org; Fri, 24 May 2013 05:05:19 -0400
Received: by mail-wi0-f180.google.com with SMTP id hn14so2576872wib.7
	for <qemu-devel@nongnu.org>; Fri, 24 May 2013 02:05:19 -0700 (PDT)
Date: Fri, 24 May 2013 11:05:16 +0200
From: Stefan Hajnoczi <stefanha@gmail.com>
Message-ID: <20130524090516.GE21639@stefanha-thinkpad.redhat.com>
References: <W4504910164140841369346311@atl4webmail30>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <W4504910164140841369346311@atl4webmail30>
Subject: Re: [Qemu-devel] 'qemu-nbd' explicit flush
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Mark Trumpold <markt@netqa.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>, qemu-devel@nongnu.org, markt@tachyon.net

On Thu, May 23, 2013 at 09:58:31PM +0000, Mark Trumpold wrote:
> I have a working configuration using the signal approach suggested by Stefan.
> 
> 'qemu-nbd.c' is patched as follows:
> 
>     do {
>         main_loop_wait(false);
> +       if (sighup_reported) {
> +           sighup_reported = false;
> +           bdrv_drain_all();
> +           bdrv_flush_all();
>         }
>     } while (!sigterm_reported && (persistent || !nbd_started || nb_fds > 0));
> 
> The driving script was patched as follows:
> 
>      mount -o remount,ro /dev/nbd0
>      blockdev --flushbufs /dev/nbd0
> +    kill -HUP <qemu-nbd process id>
> 
> I needed to retain 'blockdev --flushbufs' for things to work.  Seems the 'bdrv_flush_all' is flushing what is being missed by the blockdev flush.  I did not go back an retest with 'fsync' or other approaches I had tried before.

Okay, that makes sense:

'blockdev --flushbufs' is writing dirty pages to the NBD device.

bdrv_drain_all() + bdrv_flush_all() ensures that image file writes reach
the physical disk.

One thing to be careful of is whether these operations are asynchronous.
The signal is asynchronous, you have no way of knowing when qemu-nbd is
finished flushing to the physical disk.

I didn't check blockdev(8) but it could be the same there.

So watch out, otherwise your script is timing-dependent and may not
actually have finished flushing when you take the snapshot.

Stefan