From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46915) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cBPoC-0007oI-0V for qemu-devel@nongnu.org; Mon, 28 Nov 2016 12:33:41 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cBPo6-0001bI-UN for qemu-devel@nongnu.org; Mon, 28 Nov 2016 12:33:39 -0500 Received: from [2a01:4f8:140:52e5::2] (port=39521 helo=latin.grep.be) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cBPo6-0001b2-NU for qemu-devel@nongnu.org; Mon, 28 Nov 2016 12:33:34 -0500 Date: Mon, 28 Nov 2016 18:33:24 +0100 From: Wouter Verhelst Message-ID: <20161128173324.tjg64dxcthqeeusn@grep.be> References: <1480073296-6931-1-git-send-email-vsementsov@virtuozzo.com> <20161127191714.7wrvzbx7zpotad22@grep.be> <20161128111944.GB6411@stefanha-x1.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161128111944.GB6411@stefanha-x1.localdomain> Subject: Re: [Qemu-devel] [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: nbd-general@lists.sourceforge.net, kwolf@redhat.com, Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, pborzenkov@virtuozzo.com, pbonzini@redhat.com, mpa@pengutronix.de, den@openvz.org Hi Stefan, On Mon, Nov 28, 2016 at 11:19:44AM +0000, Stefan Hajnoczi wrote: > On Sun, Nov 27, 2016 at 08:17:14PM +0100, Wouter Verhelst wrote: > > Quickly: the reason I haven't merged this yes is twofold: > > - I wasn't thrilled with the proposal at the time. It felt a bit > > hackish, and bolted onto NBD so you could use it, but without defining > > everything in the NBD protocol. "We're reading some data, but it's not > > about you". That didn't feel right > > > > - There were a number of questions still unanswered (you're answering a > > few below, so that's good). > > > > For clarity, I have no objection whatsoever to adding more commands if > > they're useful, but I would prefer that they're also useful with NBD on > > its own, i.e., without requiring an initiation or correlation of some > > state through another protocol or network connection or whatever. If > > that's needed, that feels like I didn't do my job properly, if you get > > my point. > > The out-of-band operations you are referring to are for dirty bitmap > management. (The goal is to read out blocks that changed since the last > backup.) > > The client does not access the live disk, instead it accesses a > read-only snapshot and the dirty information (so that it can copy out > only blocks that were written). The client is allowed to read blocks > that are not dirty too. I understood as much, yes. > If you want to implement the whole incremental backup workflow in NBD > then the client would first have to connect to the live disk, set up > dirty tracking, create a snapshot export, and then connect to that > snapshot. > > That sounds like a big feature set and I'd argue it's for the control > plane (storage API) and not the data plane (NBD). There were > discussions about transferring the dirty information via the control > plane but it seems more appropriate to it in the data plane since it is > block-level information. I agree that creating and managing snapshots is out of scope for NBD. The protocol is not set up for that. However, I'm arguing that if we're going to provide information about snapshots, we should be able to properly refer to these snapshots from within an NBD context. My previous mail suggested adding a negotiation message that would essentially ask the server "tell me about the snapshots you know about", giving them an NBD identifier in the process (accompanied by a "foreign" identifier that is decidedly *not* an NBD identifier and that could be used to match the NBD identifier to something implementation-defined). This would be read-only information; the client cannot ask the server to create new snapshots. We can then later in the protocol refer to these snapshots by way of that NBD identifier. My proposal also makes it impossible to get updates of newly created snapshots without disconnecting and reconnecting (due to the fact that you can't go from transmission back to negotiation), but I'm not sure that's a problem. Doing so has two advantages: - If a client is accidentally (due to misconfiguration or implementation bugs or whatnot) connecting to the wrong server after having created a snapshot through a management protocol, we have an opportunity to detect this error, due to the fact that the "foreign" identifiers passed to the client during negotiation will not match with what the client was expecting. - A future version of the protocol could possibly include an extended version of the read command, allowing a client to read information from multiple storage snapshots without requiring a reconnect, and allowing current clients information about allocation status across various snapshots (although a first implementation could very well limit itself to only having one snapshot). [...] > I'm arguing that the NBD protocol doesn't need to support the > incremental backup workflow since it's a complex control plane concept. > > Being able to read dirty information via NBD is useful for other block > backup applications, not just QEMU. It could be used for syncing LVM > volumes across machines, for example, if someone implements an NBD+LVM > server. Indeed, and I was considering adding a basic implementation to go with the copy-on-write support in stock nbd-server, too. > Another issue with adding control plane operations is that you need to > begin considering privilege separation. Should all NBD clients be able > to initiate snapshots, dirty tracking, etc or is some kind of access > control required to limit certain commands? Not all clients require the > same privileges and so they shouldn't have access to the same set of > operations. Sure, which is why I wasn't suggesting anything of the sorts :-) -- < ron> I mean, the main *practical* problem with C++, is there's like a dozen people in the world who think they really understand all of its rules, and pretty much all of them are just lying to themselves too. -- #debian-devel, OFTC, 2016-02-12