kernel-tls-handshake.lists.linux.dev archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH 00/18] nvme: In-kernel TLS support for TCP
@ 2023-03-21 12:43 Hannes Reinecke
  2023-03-21 12:43 ` [PATCH 01/18] nvme-keyring: register '.nvme' keyring Hannes Reinecke
                   ` (18 more replies)
  0 siblings, 19 replies; 90+ messages in thread
From: Hannes Reinecke @ 2023-03-21 12:43 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Sagi Grimberg, Keith Busch, linux-nvme, Chuck Lever,
	kernel-tls-handshake, Hannes Reinecke

Hi all,

finally I've managed to put all things together and enable in-kernel
TLS support for NVMe-over-TCP.

The patchset is based on the TLS upcall mechanism from Chuck Lever
(cf '[PATCH v7 0/2] Another crack at a handshake upcall mechanism'
posted to the linux netdev list), and requires the 'tlshd' userspace
daemon (https://github.com/oracle/ktls-utils) for the actual TLS handshake.

Theory of operation:
A dedicated '.nvme' keyring is created to hold the pre-shared keys (PSKs)
for the TLS handshake. Keys will have to be provisioned before TLS handshake
is attempted; I'll be sending a patch for nvme-cli separately.
After connection to the remote TCP port the client side will check if there are
matching PSKs, and call the TLS userspace daemon to initiate a TLS handshake.
The server side will be doing a MSG_PEEK on the first incoming PDU after
accept(), and check if it's an TCP ICREQ packet. If not it's assumed to be
a TLS ClientHello and the TLS userspace daemon is invoked for the TLS
handshake.
If the TLS handshake succeeds the userspace daemon will be activating
kTLS on the socket, and control is passed back to the kernel.

The one issue of note is the multiple identity handling.
The NVMe TCP spec defines up to 4 PSK identities, and
TLS 1.3 allows for several identities to be sent with the
ClientHello. So in theory we could sent the all in one go.
Sadly none of the userspace libraries implement this feature,
so we have to test each possible identity and terminate the
connection on failure.
With this patchset all possible identities need to be part of
the keyring, and the client side will be trying to establish
a TLS connection with each matching PSK from the keyring.
The beauty is that this method works without modification
to the existing nvme-cli; one only needs to provision PSKs
in the .nvme keyring and TLS handshake will be attempted.
As I'm not sure if that approach meets with general approval
I'm sending out this patchset as an RFC for now.

As usual, comments and reviews are welcome.

Hannes Reinecke (18):
  nvme-keyring: register '.nvme' keyring
  nvme-keyring: define a 'psk' keytype
  nvme: add TCP TSAS definitions
  nvme-tcp: add definitions for TLS cipher suites
  nvme-tcp: implement recvmsg rx flow for TLS
  nvme-tcp: call 'queue->data_ready()' in nvme_tcp_data_ready()
  nvme/tcp: allocate socket file
  nvme-tcp: enable TLS handshake upcall
  nvme-tcp: add connect option 'tls'
  nvme-tcp: fixup send workflow for kTLS
  nvme-tcp: control message handling for recvmsg()
  nvmet: make TCP sectype settable via configfs
  nvmet-tcp: allocate socket file
  security/keys: export key_lookup()
  nvmet-tcp: enable TLS handshake upcall
  nvmet-tcp: rework sendpage for kTLS
  nvmet-tcp: control messages for recvmsg()
  nvmet-tcp: peek icreq before starting TLS

 drivers/nvme/common/Makefile   |   2 +-
 drivers/nvme/common/keyring.c  | 132 ++++++++++
 drivers/nvme/host/core.c       |  10 +-
 drivers/nvme/host/fabrics.c    |   5 +
 drivers/nvme/host/fabrics.h    |   2 +
 drivers/nvme/host/tcp.c        | 450 +++++++++++++++++++++++++--------
 drivers/nvme/target/configfs.c |  65 +++++
 drivers/nvme/target/tcp.c      | 407 ++++++++++++++++++++++++++---
 include/linux/nvme-keyring.h   |  20 ++
 include/linux/nvme-tcp.h       |   6 +
 include/linux/nvme.h           |  10 +
 security/keys/key.c            |   1 +
 12 files changed, 965 insertions(+), 145 deletions(-)
 create mode 100644 drivers/nvme/common/keyring.c
 create mode 100644 include/linux/nvme-keyring.h

-- 
2.35.3


^ permalink raw reply	[flat|nested] 90+ messages in thread
* [PATCHv2 00/18] nvme: In-kernel TLS support for TCP
@ 2023-03-29 13:59 Hannes Reinecke
  2023-03-29 13:59 ` [PATCH 15/18] nvmet-tcp: enable TLS handshake upcall Hannes Reinecke
  0 siblings, 1 reply; 90+ messages in thread
From: Hannes Reinecke @ 2023-03-29 13:59 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Sagi Grimberg, Keith Busch, linux-nvme, Chuck Lever,
	kernel-tls-handshake, Hannes Reinecke

Hi all,

finally I've managed to put all things together and enable in-kernel
TLS support for NVMe-over-TCP.

The patchset is based on the TLS upcall mechanism from Chuck Lever
(cf '[PATCH v7 0/2] Another crack at a handshake upcall mechanism'
posted to the linux netdev list), and requires the 'tlshd' userspace
daemon (https://github.com/oracle/ktls-utils) for the actual TLS handshake.
Changes for nvme-cli are already included in the upstream repository.

Theory of operation:
A dedicated '.nvme' keyring is created to hold the pre-shared keys (PSKs)
for the TLS handshake. Keys will have to be provisioned before TLS handshake
is attempted; that can be done with the 'nvme gen-tls-key' command for nvme-cli
(patches are already merged upstream).
After connection to the remote TCP port the client side will use the
'best' PSK (as inferred from the NVMe TCP spec) or the PSK specified
by the '--tls_key' option to nvme-cli and call the TLS userspace daemon
to initiate a TLS handshake.
The server side will then invoke the TLS userspace daemon to run the TLS
handshake.
If the TLS handshake succeeds the userspace daemon will be activating
kTLS on the socket, and control is passed back to the kernel.

To make this work I had to implement the 'read_sock()' functionality
for TLS; it seems to be holding up well enough (for me), but it really
could do with reviews from persons with more network stack knowledge.

As usual, comments and reviews are welcome.

Changes to the original RFC:
- Add a CONFIG_NVME_TLS config option
- Use a single PSK for the TLS handshake
- Make TLS connections mandatory
- Do not peek messages for the server
- Simplify data_ready callback
- Implement read_sock() for TLS

Hannes Reinecke (18):
  nvme-keyring: register '.nvme' keyring and add CONFIG_NVME_TLS
  nvme-keyring: define a 'psk' keytype
  nvme: add TCP TSAS definitions
  nvme-tcp: add definitions for TLS cipher suites
  net/tls: implement ->read_sock()
  nvme/tcp: allocate socket file
  nvme-keyring: implement nvme_tls_psk_default()
  security/keys: export key_lookup()
  nvme-tcp: enable TLS handshake upcall
  nvme-tcp: fixup send workflow for kTLS
  nvme-tcp: control message handling for recvmsg()
  nvme-fabrics: parse options 'keyring' and 'tls_key'
  nvmet: make TCP sectype settable via configfs
  nvmet-tcp: allocate socket file
  nvmet-tcp: enable TLS handshake upcall
  nvmet-tcp: rework sendpage for kTLS
  nvmet-tcp: control messages for recvmsg()
  nvmet-tcp: add configfs attribute 'param_keyring'

 drivers/nvme/common/Kconfig    |   9 ++
 drivers/nvme/common/Makefile   |   1 +
 drivers/nvme/common/keyring.c  | 181 ++++++++++++++++++++++
 drivers/nvme/host/core.c       |  44 +++++-
 drivers/nvme/host/fabrics.c    |  90 ++++++++++-
 drivers/nvme/host/fabrics.h    |   9 ++
 drivers/nvme/host/nvme.h       |   4 +-
 drivers/nvme/host/tcp.c        | 213 +++++++++++++++++++++++---
 drivers/nvme/target/configfs.c | 165 ++++++++++++++++++++-
 drivers/nvme/target/nvmet.h    |   1 +
 drivers/nvme/target/tcp.c      | 264 ++++++++++++++++++++++++++++++---
 include/linux/nvme-keyring.h   |  20 +++
 include/linux/nvme-tcp.h       |   6 +
 include/linux/nvme.h           |  10 ++
 net/tls/tls.h                  |   2 +
 net/tls/tls_main.c             |   2 +
 net/tls/tls_sw.c               |  71 +++++++++
 security/keys/key.c            |   1 +
 18 files changed, 1050 insertions(+), 43 deletions(-)
 create mode 100644 drivers/nvme/common/keyring.c
 create mode 100644 include/linux/nvme-keyring.h

-- 
2.35.3


^ permalink raw reply	[flat|nested] 90+ messages in thread

end of thread, other threads:[~2023-04-03 14:05 UTC | newest]

Thread overview: 90+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-21 12:43 [RFC PATCH 00/18] nvme: In-kernel TLS support for TCP Hannes Reinecke
2023-03-21 12:43 ` [PATCH 01/18] nvme-keyring: register '.nvme' keyring Hannes Reinecke
2023-03-21 13:50   ` Sagi Grimberg
2023-03-21 14:11     ` Hannes Reinecke
2023-03-21 12:43 ` [PATCH 02/18] nvme-keyring: define a 'psk' keytype Hannes Reinecke
2023-03-22  8:29   ` Sagi Grimberg
2023-03-22  8:38     ` Hannes Reinecke
2023-03-22  8:49       ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 03/18] nvme: add TCP TSAS definitions Hannes Reinecke
2023-03-21 13:46   ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 04/18] nvme-tcp: add definitions for TLS cipher suites Hannes Reinecke
2023-03-22  8:18   ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 05/18] nvme-tcp: implement recvmsg rx flow for TLS Hannes Reinecke
2023-03-21 13:39   ` Sagi Grimberg
2023-03-21 13:59     ` Hannes Reinecke
2023-03-22  8:01       ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 06/18] nvme-tcp: call 'queue->data_ready()' in nvme_tcp_data_ready() Hannes Reinecke
2023-03-21 13:44   ` Sagi Grimberg
2023-03-21 14:09     ` Hannes Reinecke
2023-03-22  0:18       ` Chris Leech
2023-03-22  6:59         ` Hannes Reinecke
2023-03-22  8:12           ` Sagi Grimberg
2023-03-22  8:08       ` Sagi Grimberg
2023-03-22  8:26         ` Hannes Reinecke
2023-03-22 10:13           ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 07/18] nvme/tcp: allocate socket file Hannes Reinecke
2023-03-21 13:52   ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 08/18] nvme-tcp: enable TLS handshake upcall Hannes Reinecke
2023-03-22  8:45   ` Sagi Grimberg
2023-03-22  9:12     ` Hannes Reinecke
2023-03-22 10:56       ` Sagi Grimberg
2023-03-22 12:54         ` Hannes Reinecke
2023-03-22 13:16           ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 09/18] nvme-tcp: add connect option 'tls' Hannes Reinecke
2023-03-22  9:24   ` Sagi Grimberg
2023-03-22  9:59     ` Hannes Reinecke
2023-03-22 10:09       ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 10/18] nvme-tcp: fixup send workflow for kTLS Hannes Reinecke
2023-03-22  9:31   ` Sagi Grimberg
2023-03-22 10:08     ` Hannes Reinecke
2023-03-22 11:18       ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 11/18] nvme-tcp: control message handling for recvmsg() Hannes Reinecke
2023-03-22 11:33   ` Sagi Grimberg
2023-03-22 11:48     ` Hannes Reinecke
2023-03-22 11:50       ` Sagi Grimberg
2023-03-22 12:17         ` Hannes Reinecke
2023-03-22 12:29           ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 12/18] nvmet: make TCP sectype settable via configfs Hannes Reinecke
2023-03-22 11:38   ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 13/18] nvmet-tcp: allocate socket file Hannes Reinecke
2023-03-22 11:46   ` Sagi Grimberg
2023-03-22 12:07     ` Hannes Reinecke
2023-03-21 12:43 ` [PATCH 14/18] security/keys: export key_lookup() Hannes Reinecke
2023-03-21 12:43 ` [PATCH 15/18] nvmet-tcp: enable TLS handshake upcall Hannes Reinecke
2023-03-22 12:13   ` Sagi Grimberg
2023-03-22 12:34     ` Hannes Reinecke
2023-03-22 12:51       ` Sagi Grimberg
2023-03-22 13:47         ` Hannes Reinecke
2023-03-22 15:42           ` Sagi Grimberg
2023-03-22 16:43             ` Hannes Reinecke
2023-03-22 16:49               ` Chuck Lever III
2023-03-23  7:21                 ` Sagi Grimberg
2023-03-24 11:29                   ` Hannes Reinecke
2023-03-26  7:18                     ` Sagi Grimberg
2023-03-27  6:20                       ` Hannes Reinecke
2023-03-28  8:44                         ` Sagi Grimberg
2023-03-28  9:20                           ` Hannes Reinecke
2023-03-28  9:43                             ` Sagi Grimberg
2023-03-28 10:04                               ` Hannes Reinecke
2023-03-28 13:22                           ` Chuck Lever III
2023-03-28 15:29                             ` Sagi Grimberg
2023-03-28 15:56                               ` Chuck Lever III
2023-03-29  6:33                                 ` Sagi Grimberg
2023-03-23  7:44               ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 16/18] nvmet-tcp: rework sendpage for kTLS Hannes Reinecke
2023-03-22 12:16   ` Sagi Grimberg
2023-03-21 12:43 ` [PATCH 17/18] nvmet-tcp: control messages for recvmsg() Hannes Reinecke
2023-03-21 12:43 ` [PATCH 18/18] nvmet-tcp: peek icreq before starting TLS Hannes Reinecke
2023-03-22 12:24   ` Sagi Grimberg
2023-03-22 12:38     ` Hannes Reinecke
2023-03-21 13:12 ` [RFC PATCH 00/18] nvme: In-kernel TLS support for TCP Sagi Grimberg
2023-03-21 13:30   ` Hannes Reinecke
2023-03-22  8:16     ` Sagi Grimberg
2023-03-22  8:28       ` Hannes Reinecke
2023-03-22 12:53         ` Sagi Grimberg
2023-03-22 15:10           ` Hannes Reinecke
2023-03-22 15:43             ` Sagi Grimberg
2023-03-29 13:59 [PATCHv2 " Hannes Reinecke
2023-03-29 13:59 ` [PATCH 15/18] nvmet-tcp: enable TLS handshake upcall Hannes Reinecke
2023-04-03 12:51   ` Sagi Grimberg
2023-04-03 14:05     ` Hannes Reinecke

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).