[Qemu-devel] [PATCHv9] Improve documentation for TLS

[Alex Bligh <alex@alex.org.uk>]

* Call out TLS into a separate section

* Add details of the TLS protocol itself

* Emphasise that actual TLS session initiation (i.e. the TLS handshake) can
  be initiated from either side (as required by the TLS standard I believe
  and as actually works in practice)

* Clarify what is a requirement on servers, and what is a requirement on
  clients, separately, specifying their behaviour in a single place
  in the document.

* Document the three possible modes of operation of a server.

* Add text defining what 'terminate the session' means during
  negotiation, and when it is available.

Signed-off-by: Alex Bligh <alex@alex.org.uk>
---
 doc/proto.md | 352 +++++++++++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 318 insertions(+), 34 deletions(-)

Changes since v8:

* Reword section on disconnection, as per mail to list save with Eric Blake's
  change of 're' to 'regarding'.

Changes since v7

* I missed committing the changes re consistent use of 'option' rather than 'command'
 in v7. They are here now.

Changes from v6:

* Introduced language mandating a server to reply with NBD_ERR_INVALID
to NBD_OPT_STARTTLS if TLS is already negotiatied.

* Removed some duplication in SELECTIVETLS over the prohibition on
servers not returning NBD_ERR_TLSREQD to options other than
NBD_OPT_EXPORTNAME, NBD_OPT_INFO and NBD_OPT_GO. The same thing
was said a different way a couple of paragraphs below.

* Consistently refer to 'options' rather than 'commands' in the
negotiation phase.

* Eric Blake's nits

Changes from v5:

* Delete OPTIONALTLS (RIP)

* Add NBD_REP_ERR_POLICY

* s/NBD_ERR_REP/NBD_REP_ERR/ in one place

* Consistently use the phrase 'terminate the session' to mean dropping
the connection, as per Wouter. Note there are other inconsistent
uses of 'dropping the connection', 'disconnecting' etc. elsewhere
which I haven't touched.

* Similarly refer to the connection as a 'session' when it doesn't
explicitly mean the L3 TCP connection (TLS section only).

* Introduce a paragraph under newstyle negotiation emphasising that
terminating the session is legal and sometimes required, and defining
it.

Changes from v4

* Minor grammar nit

Changes from v3:

* Delete confusing text about server omitting entries from NBD_OPT_LIST
if TLS is not negotiated and FORCETLS is used, as that (of course)
requires NBD_REP_ERR_TLS_REQD elsewhere in the text.

* Further nits from Eric Blake

Changes from v2:

* The response to a command is a response, not a NBD_REP_ACK

* Make it clear that the response can be errored

* Nits from Eric Blake

Changes from v1:

* Make a NBD_CMD_CLOSE imply a flush

* Nits from Eric Blake

diff --git a/doc/proto.md b/doc/proto.md
index f117394..255fd11 100644
--- a/doc/proto.md
+++ b/doc/proto.md
@@ -195,6 +195,23 @@ request before sending the next one of the same type. The server MAY
 send replies in the order that the requests were received, but is not
 required to.
 
+There is no requirement for the client or server to complete a
+negotiation if it does not wish to do so. Either end may simply
+close the TCP connection (though see below regarding prior use
+of NBD_OPT_ABORT). Under certain circumstances either
+the client or the server may be required by this document to close
+the TCP connection. In each case, this is referred to as 'terminate
+the session'.
+
+If the client wishes to terminate the session in the negotiation
+phase, and is not doing so because it is required to do so
+by this document, it SHOULD send NBD_OPT_ABORT first if the protocol
+permits. There are instances where this is impossible, such as after
+an NBD_OPT_EXPORTNAME has been issued, or on an unsuccessful
+negotiation of TLS.  For instance, if the client does not find an
+export it is looking for, it may simply send an NBD_OPT_ABORT
+and close the TCP connection.
+
 ### Transmission
 
 There are three message types in the transmission phase: the request,
@@ -286,6 +303,287 @@ S: (*length* bytes of data if the request is of type `NBD_CMD_READ`)
 This reply type MUST NOT be used except as documented by the
 experimental `STRUCTURED_REPLY` extension; see below.
 
+## TLS support
+
+The NBD protocol supports Transport Layer Security (TLS) (see
+[RFC5246](https://tools.ietf.org/html/rfc5246)
+as updated by
+[RFC6176](https://tools.ietf.org/html/rfc6176)
+).
+
+TLS is negotiated with the `NBD_OPT_STARTTLS`
+option. This is performed as an in-session upgrade. Below the term
+'negotiation' is used to refer to the sending and receiving of
+NBD options and option replies, and the term 'initiation' of TLS
+is used to refer to the actual upgrade to TLS.
+
+### Certificates, authentication and authorisation
+
+This standard does not specify what encryption, certification
+and signature algorithms are used. This standard does not
+specify authentication and authorisation (for instance
+whether client and/or server certificates are required and
+what they should contain); this is implementation dependent.
+
+TLS requires fixed newstyle negotiation to have completed.
+
+### Server-side requirements
+
+There are three modes of operation for a server. The
+server MUST support one of these modes.
+
+* The server operates entirely without TLS ('NOTLS'); OR
+
+* The server insists upon TLS, and forces the client to
+  upgrade by erroring any NBD options other than `NBD_OPT_STARTTLS`
+  with `NBD_REP_ERR_TLS_REQD` ('FORCEDTLS'); this in practice means
+  that all option negotiation (apart from the `NBD_OPT_STARTTLS`
+  itself) is carried out with TLS; OR
+
+* The server provides TLS, and it is mandatory on zero or more
+  exports, and is available at the client's option on all
+  other exports ('SELECTIVETLS'). The server does not force
+  the client to upgrade to TLS during option haggling (as
+  if the client ultimately were to choose a non-TLS-only export,
+  stopping TLS is not possible). Instead it permits the client
+  to upgrade as and when it chooses, but unless an upgrade to
+  TLS has already taken place, the server errors attempts
+  to enter transmission mode on TLS-only exports, MAY
+  refuse to provide information about TLS-only exports
+  via `NBD_OPT_INFO`, MAY refuse to provide information
+  about non-existent exports via `NBD_OPT_INFO`, and MAY omit
+  exports that are TLS-only from `NBD_OPT_LIST`.
+
+The server MAY determine the mode in which it operates
+dependent upon the session (for instance it might be
+more liberal with TCP connections made over the loopback
+interface) but it MUST be consistent in its mode
+of operation across the lifespan of a single TCP connection
+to the server. A client MUST NOT assume indications from
+a prior TCP session to a given server will be relevant
+to a subsequent session.
+
+The server MUST operate in NOTLS mode unless the server
+set flag NBD_FLAG_FIXED_NEWSTYLE and the client replied
+with NBD_FLAG_C_FIXED_NEWSTYLE in the fixed newstyle
+negotiation.
+
+These modes of operations are described in detail below.
+
+#### NOTLS mode
+
+If the server receives `NBD_OPT_STARTTLS` it MUST respond with
+`NBD_REP_ERR_POLICY` (if it does not support TLS for
+policy reasons) or `NBD_REP_ERR_UNSUP` (if it does not
+support the `NBD_OPT_STARTTLS` option at all). The server MUST NOT
+respond to any option request with `NBD_REP_ERR_TLS_REQD`.
+
+#### FORCEDTLS mode
+
+If the server receives `NBD_OPT_STARTTLS` prior to negotiating
+TLS, it MUST reply with `NBD_REP_ACK`. If the server receives
+`NBD_OPT_STARTTLS` when TLS has already been negotiated, it
+it MUST reply with `NBD_REP_ERR_INVALID`.
+
+After an `NBD_REP_ACK` reply has been sent, the server MUST be
+prepared for a TLS handshake, and all further data MUST be sent
+and received over TLS. There is no downgrade to a non-TLS session.
+
+As per the TLS standard, the handshake MAY be initiated either
+by the server (having sent the `NBD_REP_ACK`) or by the client.
+If the handshake is unsuccessful (for instance the client's
+certificate does not match) the server MUST terminate the
+session as by this stage it is too late to continue without TLS
+as the acknowledgement has been sent.
+
+If the server receives any other option, including `NBD_OPT_INFO`
+and unsupported options, it MUST reply with `NBD_REP_ERR_TLS_REQD`
+if TLS has not been initiated; `NBD_OPT_INFO` is included as in this
+mode, all exports are TLS-only. If the server receives a request to
+enter transmission mode via `NBD_OPT_EXPORT_NAME` when TLS has not
+been initiated, then as this request cannot error, it MUST
+terminate the session. If the server receives a request to
+enter transmission mode via `NBD_OPT_GO` when TLS has not been
+initiated, it MUST error with `NBD_REP_ERR_TLS_REQD`.
+
+The server MUST NOT send `NBD_REP_ERR_TLS_REQD` in reply to
+any option if TLS has already been initiated.
+
+The FORCEDTLS mode of operation has an implementation problem in
+that the client MAY legally simply send a `NBD_OPT_EXPORT_NAME`
+to enter transmission mode without previously sending any options.
+Therefore, if a server uses FORCEDTLS, it SHOULD implement the
+INFO extension.
+
+#### SELECTIVETLS mode
+
+If the server receives `NBD_OPT_STARTTLS` prior to negotiating
+TLS, it MUST reply with `NBD_REP_ACK` and initiate TLS as set
+out under 'FORCEDTLS' above. If the server receives
+`NBD_OPT_STARTTLS` when TLS has already been negotiated, it
+it MUST reply with `NBD_REP_ERR_INVALID`.
+
+If the server receives `NBD_OPT_INFO` or `NBD_OPT_GO` and TLS
+has not been initiated, it MAY reply with `NBD_REP_ERR_TLS_REQD`
+if that export is non-existent, and MUST reply with
+`NBD_REP_ERR_TLS_REQD` if that export is TLS-only.
+
+If the server receives a request to enter transmission mode
+via `NBD_OPT_EXPORT_NAME` on a TLS-only export when TLS has not
+been initiated, then as this request cannot error, it MUST
+terminate the session.
+
+The server MUST NOT send `NBD_REP_ERR_TLS_REQD` in reply to
+any option if TLS has already been negotiated. The server
+MUST NOT send `NBD_REP_ERR_TLS_REQD` in response to any
+option other than `NBD_OPT_INFO`, `NBD_OPT_GO` and
+`NBD_OPT_EXPORT_NAME`, and only in those cases in respect of
+a TLS-only or non-existent export.
+
+There is a degenerate case of SELECTIVETLS where all
+exports are TLS-only. This is permitted in part to make programming
+of servers easier. Operation is a little different from FORCEDTLS,
+as the client is not forced to upgrade to TLS prior to any options
+being processed, and the server MAY choose to give information on
+non-existent exports via NBD_OPT_INFO exports prior to an upgrade
+to TLS.
+
+The SELECTIVETLS mode of operation has an implementation problem
+in that unless the INFO extension is supported, the client that
+does not use TLS may have its access to exports denied without
+it being able to ascertain the reason. For instance it may
+go into transmission mode using `NBD_OPT_EXPORT_NAME` - which
+does not return an error as no options will be denied with
+`NBD_REP_ERR_TLS_REQD`. Further there is no way to remotely
+determine whether an export requires TLS, and therefore this
+must be initiated between client and server out of band.
+Therefore, if a server uses SELECTIVETLS, it MUST implement
+the INFO extension.
+
+## Client-side requirements
+
+If the client supports TLS at all, it MUST be prepared
+to deal with servers operating in any of the above modes.
+Notwithstanding, a client MAY always terminate the session or
+refuse to connect to a particular export if TLS is
+not available and the user requires TLS.
+
+The client MUST NOT issue `NBD_OPT_STARTTLS` unless the server
+set flag NBD_FLAG_FIXED_NEWSTYLE and the client replied
+with NBD_FLAG_C_FIXED_NEWSTYLE in the fixed newstyle
+negotiation.
+
+The client MUST NOT issue `NBD_OPT_STARTTLS` if TLS has already
+been initiated.
+
+Subject to the above two limitations, the client MAY send
+`NBD_OPT_STARTTLS` at any time to initiate a TLS session. If the
+client receives `NBD_REP_ACK` in response, it MUST immediately
+upgrade the session to TLS. If it receives `NBD_REP_ERR_UNSUP`,
+`NBD_REP_ERR_POLICY` or any other error in response, it indicates
+that the server cannot or will not upgrade the session to TLS,
+and therefore the client MUST either continue the session
+without TLS, or terminate the session.
+
+A client that prefers to use TLS irrespective of whether
+the server makes TLS mandatory SHOULD send `NBD_OPT_STARTTLS`
+as the first option. This will ensure option haggling is subject
+to TLS, and will thus prevent the possibility of options being
+compromised by a Man-in-the-Middle attack. Note that the
+`NBD_OPT_STARTTLS` itself may be compromised - see 'downgrade
+attacks' for more details. For this reason, a client which only
+wishes to use TLS SHOULD terminate the session if the
+`NBD_OPT_STARTTLS` replies with an error.
+
+If the TLS handshake is unsuccessful (for instance the server's
+certificate does not validate) the client MUST terminate the
+session as by this stage it is too late to continue without TLS.
+
+If the client receives an `NBD_REP_ERR_TLS_REQD` in response
+to any option, it implies that this option cannot be executed
+unless a TLS upgrade is performed. If the option is any
+option other than `NBD_OPT_INFO` or `NBD_OPT_GO`, this
+indicates that no option will succeed unless a TLS upgrade
+is performed; the client MAY therefore choose to issue
+an `NBD_OPT_STARTTLS`, or MAY terminate the session (if
+for instance it does not support TLS or does not have
+appropriate credentials for this server). If the client
+receives `NBD_REP_ERR_TLS_REQD` in response to
+`NBD_OPT_INFO` or `NBD_OPT_GO` this indicates that the
+export referred to within the option is either non-existent
+or requires TLS; the client MAY therefore choose to issue
+an `NBD_OPT_STARTTLS`, MAY terminate the session (if
+for instance it does not support TLS or does not have
+appropriate credentials for this server), or MAY continue
+in another manner without TLS, for instance by querying
+or using other exports.
+
+If a client supports TLS, it SHOULD also support the INFO
+extension, and SHOULD use `NBD_OPT_GO` if available in place
+of `NBD_OPT_EXPORT_NAME`. The reason for this is set out in
+the final paragraphs of the sections under 'FORCEDTLS'
+and 'SELECTIVETLS': this gives an opportunity for the
+server to transmit that an error going into transmission
+mode is due to the client's failure to initiate TLS,
+and the fact that the client may obtain information about
+which exports are TLS-only through `NBD_OPT_INFO`.
+
+### Security considerations
+
+#### TLS versions
+
+NBD implementations supporting TLS MUST support TLS version 1.2,
+SHOULD support any later versions. NBD implementations
+MAY support older versions but SHOULD NOT do so by default
+(i.e. they SHOULD only be available by a configuration change).
+Older versions SHOULD NOT be used where there is a risk of security
+problems with those older versions or of a downgrade attack
+against TLS versions.
+
+#### Protocol downgrade attacks
+
+A danger inherent in any scheme relying on the negotiation
+of whether TLS should be employed is downgrade attacks within
+the NBD protocol.
+
+There are two main dangers:
+
+* A Man-in-the-Middle (MitM) hijacks a session and impersonates
+  the server (possibly by proxying it) claiming not to support
+  TLS. In this manner, the client is confused into operating
+  in a plain-text manner with the MitM (with the session possibly
+  being proxied in plain-text to the server using the method
+  below).
+
+* The MitM hijacks a session and impersonates the client
+  (possibly by proxying it) claiming not to support TLS. In
+  this manner the server is confused into operating in a plain-text
+  manner with the MitM (with the session being possibly
+  proxied to the client with the method above).
+
+With regard to the first, any client that does not wish
+to be subject to potential downgrade attack SHOULD ensure
+that if a TLS endpoint is specified by the client, it
+ensures that TLS is negotiated prior to sending or
+requesting sensitive data. To recap, the client MAY send
+`NBD_OPT_STARTTLS` at any point during option haggling,
+and MAY terminate the session if `NBD_REP_ACK` is not
+provided.
+
+With regard to the second, any server that does not wish
+to be subject to a potential downgrade attack SHOULD either
+used FORCEDTLS mode, or should force TLS on those exports
+it is concerned about using SELECTIVE mode and TLS-only
+exports. It is not possible to avoid downgrade attacks
+on exports which may be served either via TLS or in plain
+text unless the client insists on TLS.
+
+### Status
+
+This functionality has not yet been implemented by the reference
+implementation, but was implemented by qemu and subsequently
+by other users, so has been moved out of the "experimental" section.
+
 ## Values
 
 This section describes the value and meaning of constants (other than
@@ -366,7 +664,7 @@ of the newstyle negotiation.
     Data: String, name of the export, as free-form text.
     The length of the name is determined from the option header. If the
     chosen export does not exist or requirements for the chosen export
-    are not met (e.g., the client did not negotiate TLS for an export
+    are not met (e.g., the client did not initiate TLS for an export
     where the server requires it), the server should close the
     connection.
 
@@ -391,7 +689,9 @@ of the newstyle negotiation.
 - `NBD_OPT_LIST` (3)
 
     Return a number of `NBD_REP_SERVER` replies, one for each export,
-    followed by `NBD_REP_ACK`.
+    followed by `NBD_REP_ACK`. The server SHOULD omit entries from this
+    list if TLS has not been negotiated, the server is operating in
+    SELECTIVETLS mode, and the entry concerned is a TLS-only export.
 
 - `NBD_OPT_PEEK_EXPORT` (4)
 
@@ -400,21 +700,15 @@ of the newstyle negotiation.
 
 - `NBD_OPT_STARTTLS` (5)
 
-    The client wishes to initiate TLS. If the server replies with
-    `NBD_REP_ACK`, then the client should immediately initiate a TLS
-    handshake and continue the negotiation in the encrypted channel. If
-    the server is unwilling to perform TLS, it should reply with
-    `NBD_REP_ERR_POLICY`. For backwards compatibility, a client should
-    also be prepared to handle `NBD_REP_ERR_UNSUP`. If the client sent
-    along any data with the request, the server should send back
-    `NBD_REP_ERR_INVALID`. The client MUST NOT send this option if
-    it has already negotiated TLS; if the server receives
-    `NBD_OPT_STARTTLS` when TLS has already been negotiated, the server
-    MUST send back `NBD_REP_ERR_INVALID`.
-
-    This functionality has not yet been implemented by the reference
-    implementation, but was implemented by qemu so has been moved out of
-    the "experimental" section.
+    The client wishes to initiate TLS.
+
+    The server MUST either reply with `NBD_REP_ACK` after which
+    point the connection is upgraded to TLS, or reply with
+    `NBD_REP_ERR_POLICY` (or if it does not support the option
+    at all, `NBD_REP_ERR_UNSUP`, or if TLS has already been
+    negotiated, `NBD_REP_ERR_INVALID`).
+
+    See the section on TLS above for further details.
 
 - `NBD_OPT_INFO` (6)
 
@@ -489,20 +783,10 @@ case that data is an error message string suitable for display to the user.
 * `NBD_REP_ERR_TLS_REQD` (2^31 + 5)
 
     The server is unwilling to continue negotiation unless TLS is
-    negotiated first. A server MUST NOT send this error if it has one or
-    more exports that do not require TLS; not even if the client indicated
-    interest (by way of `NBD_OPT_PEEK_EXPORT`) in an export which requires
-    TLS.
-
-    If this reply is used, servers SHOULD send it in reply to each and every
-    unencrypted `NBD_OPT_*` message (apart from `NBD_OPT_STARTTLS`).
-
-    This functionality has not yet been implemented by the reference
-    implementation, but was implemented by qemu so has been moved out of
-    the "experimental" section.
-
-    The experimental `INFO` extension makes small but compatible
-    changes to the semantics of this error message; see below.
+    initiated first. In the case of `NBD_OPT_INFO` and `NBD_OPT_GO`
+    this unwillingness MAY (depending on the TLS mode) be limited
+    to the export in question. See the section on TLS above for
+    further details.
 
 * `NBD_REP_ERR_UNKNOWN` (2^31 + 6)
 
@@ -735,13 +1019,13 @@ Therefore these commands share common documentation.
     - `NBD_REP_ERR_UNKNOWN`: The chosen export does not exist on this
       server.
     - `NBD_REP_ERR_TLS_REQD`: The server does not wish to export this
-      block device unless the client negotiates TLS first.
+      block device unless the client initiates TLS first.
     - `NBD_REP_SERVER`: The server accepts the chosen export.
 
-    Additionally, if TLS has not been negotiated, the server MAY reply
+    Additionally, if TLS has not been initiated, the server MAY reply
     with `NBD_REP_ERR_TLS_REQD` (instead of `NBD_REP_ERR_UNKNOWN`)
     to requests for exports that are unknown. This is so that clients
-    that have not negotiated TLS cannot enumerate exports.
+    that have not initiated TLS cannot enumerate exports.
 
     In the case of `NBD_REP_SERVER`, the message's data takes on a different
     interpretation than the default (so as to provide additional
-- 
1.9.1