From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52377) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1axHLA-0000dz-Sx for qemu-devel@nongnu.org; Mon, 02 May 2016 13:09:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1axHKp-0000Wp-6B for qemu-devel@nongnu.org; Mon, 02 May 2016 13:08:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42424) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1axHKo-0000SU-UR for qemu-devel@nongnu.org; Mon, 02 May 2016 13:08:39 -0400 References: <56FD7B7E.4060004@redhat.com> <64B326DA-CDF4-4537-B38A-46E7B57C319C@alex.org.uk> <56FD8069.7020101@redhat.com> <56FD89D0.5050408@redhat.com> From: Eric Blake Message-ID: <5727898A.1060504@redhat.com> Date: Mon, 2 May 2016 11:08:26 -0600 MIME-Version: 1.0 In-Reply-To: <56FD89D0.5050408@redhat.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="eprcWqhXrFMU3D1VW4kO4mcqt5A3gJDe1" Subject: Re: [Qemu-devel] [Nbd] Is NBD_CMD_FLAG_FUA valid during NBD_CMD_FLUSH? List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alex Bligh Cc: "nbd-general@lists.sourceforge.net" , "qemu-devel@nongnu.org" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --eprcWqhXrFMU3D1VW4kO4mcqt5A3gJDe1 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 03/31/2016 02:34 PM, Eric Blake wrote: > On 03/31/2016 02:17 PM, Alex Bligh wrote: >> OK so I actually went and researched what my answer was last time I >> was asked ( :-) ): >> >> Here was my conclusion last time after trawling through lkml >> on the subject: >> >> From https://sourceforge.net/p/nbd/mailman/message/27569820/ >> >>> Meanwhile, it sounds like FUA is valid on read, write, AND flush >>> (because the kernel supports all three), >> >> Do you have a pointer to what FUA means on kernel reads? Does it >=20 > No clue. I'm not a kernel expert, and was assuming that you knew more > about it than me. >=20 >> mean "force unit access for the read" or does it mean "flush any >> write for that block first"? The first is subtly different if the >> file is remote and being accessed by multiple people (e.g. NFS, Ceph e= tc.) >=20 > I would lean to the latter - FUA on a read seems like it is most useful= > if it means "guarantee that no one else can read something older than > what I read", and NOT "give me possibly stale data because I accessed > the underlying storage rather than paying attention to in-flight writes= > that would change what I read". In other words, I think you should > ALWAYS prefer data from in-flight writes over going to backing storage,= > but USUALLY don't need the overhead of waiting for those writes to > complete; FUA slows down your read, but gives you better data assurance= =2E SCSI defines FUA on both reads and writes. Reading http://www.seagate.com/staticfiles/support/disc/manuals/scsi/100293068a.p= df under READ(10), I see: "The force unit access (FUA) and force unit access non-volatile cache (FUA_NV) bits are defined in table 87. =2E.. The device server shall read the logical blocks from the medium. If a cache contains a more recent version of a logical block, the device server shall write the logical block to the medium before reading it. " Whether the kernel exposes this aspect of SCSI FUA bit through its bio interface (and if not, whether it should) are a different matter, but it sounds like since at least SCSI has a definition for FUA on read, maybe NBD should (someday) likewise add a definition for it. At least for now, we left the door open for it, by at least requiring that servers not reject the flag, although there may be some discovery issues to figure out before a client can reasonably know that its FUA-on-read requests will actually be honored. >=20 >> >>> even if we aren't quite sure >>> what to document of those flags. And that means qemu is correct, and= >>> the NBD protocol has a bug. Since you contributed the FUA flag, is t= hat >>> something you can try to improve? >> >> Yeah. My mess so I should clean it up. I think FUA should be valid >> on essentially everything. >> >> I think I might wait until structured replies is in though! >=20 > It's also tricky because we just barely documented that servers SHOULD > reject invalid flags with EINVAL; and that clients MUST NOT send FUA on= > commands where it is not documented; I don't know if we have an adequat= e > discovery system in place to learn _which_ commands support FUA, > especially if you are proposing that we expand the scope of FUA to be > valid alongside a TRIM request. >=20 > It doesn't have to be solved today, though, so I'm fine if you wait for= > structured replies first. >=20 --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --eprcWqhXrFMU3D1VW4kO4mcqt5A3gJDe1 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJXJ4mKAAoJEKeha0olJ0NqPwoH/1Jq3XDR45G2svziE5Ub+EnB g1L5FBA3n6u4rHtAmKU78W5RAoh82xnr5wk+9DcbcBTpm4w5fLNTgnQDNcrDbXmg OUHGoc5qoOpbboNh9su7ZLooTuG7myYr2wOlwOqH+ek1SY5hj+9yEX+2FXQG9407 OMcncbBCDkQEV8Zz9+p35dLGZYruFPIseC8j50ZQUy7l9b+zCtil9QMowtSjHb0x PtCNfl1EGd2jHn7s0msMeVJjVaZx3OexpWVVchi+J8MWJy/kU235jIXRY1NbBAtC Vkuz28Cx6WI2iKLt0brYN60TRRDo8lbEFcrOS38QoC/moklUgNfg9oxyrHXUalQ= =fGvb -----END PGP SIGNATURE----- --eprcWqhXrFMU3D1VW4kO4mcqt5A3gJDe1--