Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

From: Alex Bligh <alex@alex.org.uk>
To: Josef Bacik <jbacik@fb.com>
Cc: Alex Bligh <alex@alex.org.uk>, Wouter Verhelst <w@uter.be>,
	Christoph Hellwig <hch@infradead.org>,
	"nbd-general@lists.sourceforge.net"
	<nbd-general@lists.sourceforge.net>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	linux-block@vger.kernel.org, Markus Pargmann <mpa@pengutronix.de>,
	kernel-team@fb.com
Subject: Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements
Date: Thu, 15 Sep 2016 16:17:58 +0100	[thread overview]
Message-ID: <61FA9037-0D6C-4902-968C-11BCDEED3DC8@alex.org.uk> (raw)
In-Reply-To: <93b5c669-fbcf-e972-fc4d-3b08f1c76299@fb.com>

Josef,

> On 15 Sep 2016, at 14:57, Josef Bacik <jbacik@fb.com> wrote:
>=20
> This isn't an NBD problem, this is an application problem.  The =
application must wait for all writes it cares about _before_ issuing a =
flush.  This is the same as for normal storage as it is for NBD.  It is =
not NBD's responsibility to maintain coherency between multiple requests =
across connections, just simply to act on and respond to requests.
>=20
> I think changing the specification to indicate that this is the case =
for multiple connections is a good thing, to keep NBD servers from doing =
weird things like sending different connections to the same export to =
different backing stores without some sort of synchronization.  It =
should definitely be explicitly stated somewhere that NBD does not =
provide any ordering guarantees and that is up to the application.  =
Thanks,

I don't think that's correct.

The block stack issues a flush to mean (precisely) "do not reply to this =
until all preceding writes THAT HAVE BEEN REPLIED TO have been persisted =
to non-volatile storage".

The danger is with multiple connections (where apparently only one flush =
is sent - let's say down connection 1) that not al the writes that have =
been replied to on connection 2 have been persisted to non-volatile =
storage. Only the ones on connection 1 have been persisted (this is =
assuming the nbd server doesn't 'link' in some way the connections).

There's nothing the 'application' (here meaning the kernel or higher =
level) can do to mitigate this. Sure it can wait for all the replies, =
but this doesn't guarantee the writes have been persisted to =
non-volatile storage, precisely because writes may return prior to this.

--=20
Alex Bligh