From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37163) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1alxNz-0005CY-Rc for qemu-devel@nongnu.org; Fri, 01 Apr 2016 07:37:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1alxNv-0005t6-JI for qemu-devel@nongnu.org; Fri, 01 Apr 2016 07:37:07 -0400 Received: from barbershop.grep.be ([89.106.240.122]:36638) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1alxNv-0005sb-AA for qemu-devel@nongnu.org; Fri, 01 Apr 2016 07:37:03 -0400 Date: Fri, 1 Apr 2016 13:11:26 +0200 From: Wouter Verhelst Message-ID: <20160401111126.GA24838@grep.be> References: <1459465399-56203-1-git-send-email-alex@alex.org.uk> <20160401083522.GC25514@grep.be> <2073BBC6-CD9E-427D-831C-05E9B4599A4B@alex.org.uk> <20160401101019.GL25514@grep.be> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] [Nbd] [PATCH] Improve documentation of FUA and FLUSH List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alex Bligh Cc: "nbd-general@lists.sourceforge.net" , "qemu-devel@nongnu.org" On Fri, Apr 01, 2016 at 11:17:29AM +0100, Alex Bligh wrote: > On 1 Apr 2016, at 11:10, Wouter Verhelst wrote: > > On Fri, Apr 01, 2016 at 10:28:03AM +0100, Alex Bligh wrote: > >> On 1 Apr 2016, at 09:35, Wouter Verhelst wrote: > >>>> +* All write commands (that includes both `NBD_CMD_WRITE` and > >>>> + `NBD_CMD_TRIM`) that the server completes (i.e. replies to) > >>>> + prior to processing to a `NBD_CMD_FLUSH` MUST be written to non-volatile > >>>> + storage prior to replying to that `NBD_CMD_FLUSH`. The server SHOULD ensure > >>>> + that all write command received prior to processing the `NBD_CMD_FLUSH` > >>>> + (whether they are replied to or not) are written to non-volatile > >>>> + storage prior to processing an `NBD_CMD_FLUSH`; note this is a > >>>> + stronger condition than the previous 'MUST' condition. This > >>> > >>> This seems to make little sense. Are you saying that suddenly now > >>> sending a reply for FLUSH with outstanding writes is wrong? If not, the > >>> above should be clarified. > >> > >> The MUST sentence does not cover that situation as it only refers > >> to completed writes. > >> > >> The SHOULD sentence says that's a 'SHOULD NOT' situation in respect > >> of writes that have PROCESSED (i.e actioned) whether or not they > >> have been replied to. Of course the client has no way of knowing > >> whether they have been PROCESSED without a reply. > >> > >> Personally I think the SHOULD clause is pretty pointless and is > >> unnecessary, but that's where the conversation got to n years > >> ago I believe. > > > > I'm still not sure what it's supposed to mean, though. Clearly, you > > should at the very least reword it, if not... > > > >> Happy to delete the last sentence if that's wrong. > > > > ... remove it instead. > > If I can't even explain it, it doesn't bode well! :-) > I think there are three types of writes that are relevant at the point > of replying to a FLUSH: > > Type A: writes that are truly 'in flight', i.e. have been sent, but have > not been 'processed' (i.e. write(2) has not been called). Right. > Type B: writes that are have been processed (i.e. write(2) has not been ITYM s/not// here? > called) but the reply has not yet been sent to the client - either the > reply hasn't been made yet or (more likely) it's sitting in some queue. > > Type C: writes that have been processed and a reply sent. > > > The first sentence (the 'MUST') refers to Type C writes and says these > MUST be persisted to permanent storage prior to replying to the FLUSH. > > Type A writes may be in a buffer server side or even client side, so > no one expects anything to happen with respect to those. > > The 'SHOULD' bit concerns Type B writes. It's saying that if you've > actually done the write call (but not yet replied) you SHOULD persist > these to disk prior to replying to the FLUSH. This type of situation > doesn't (last time I looked) happen in nbd-server.c because it does > not process requests in parallel. Actually, it does, since a few months or so :-) > However it may happen elsewhere. The question is what to do with them. Right. As far as the client (and as a direct result, the protocol) is concerned, however, there is no substantial difference between type A and type B. The request has been sent, and the reply hasn't been received yet. This is the same for A as well as for B. I don't think it is at all useful to tell the server what it should do in that situation. If the server implements FLUSH with fsync(), then type B replies are handled exactly like type C replies. If the server implements FLUSH with something else, however, then flushing type B replies could potentially be much more expensive then flushing type C replies. I think this is one of those cases where the protocol spec simply shouldn't say anything (at most, the protocol spec could explicitly say that it doesn't prescribe any particular behaviour). Requests that have been sent to the server MAY be flushed by a FLUSH request, but MAY equally well not be. There is no reason to use SHOULD regarding anything about that. (I believe that is what the spec currently says, fwiw, but I didn't check and may be wrong) -- < ron> I mean, the main *practical* problem with C++, is there's like a dozen people in the world who think they really understand all of its rules, and pretty much all of them are just lying to themselves too. -- #debian-devel, OFTC, 2016-02-12