linux-cifs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v4 0/6] multichannel
@ 2019-09-20  5:01 Aurelien Aptel
  2019-09-20  5:01 ` [PATCH v4 1/6] cifs: add multichannel mount options and data structs Aurelien Aptel
                   ` (5 more replies)
  0 siblings, 6 replies; 9+ messages in thread
From: Aurelien Aptel @ 2019-09-20  5:01 UTC (permalink / raw)
  To: linux-cifs; +Cc: smfrench, Aurelien Aptel

This patchset adds multichannel support. It is a cleaned up version of
v3. I've removed debug messages and split the patch in multiple commits.

It still needs more reconnection testing and some tweaks (use 3.0
instead of 3.1.1 as minimum requirement).

Long Li sucessfully tried the patchset with RDMA (opening RDMA
channels).

Aurelien Aptel (6):
  cifs: add multichannel mount options and data structs
  cifs: add server param
  cifs: switch servers depending on binding state
  cifs: sort interface list by speed
  cifs: try opening channels after mounting
  cifs: mention if an interface has a channel connected to it

 fs/cifs/cifs_debug.c    |   6 +-
 fs/cifs/cifs_spnego.c   |   2 +-
 fs/cifs/cifsfs.c        |   3 +
 fs/cifs/cifsglob.h      |  29 ++++++-
 fs/cifs/cifsproto.h     |   8 ++
 fs/cifs/connect.c       |  83 +++++++++++++++----
 fs/cifs/sess.c          | 216 +++++++++++++++++++++++++++++++++++++++++++++++-
 fs/cifs/smb2misc.c      |  37 ++++++---
 fs/cifs/smb2ops.c       |  13 ++-
 fs/cifs/smb2pdu.c       | 106 ++++++++++++++----------
 fs/cifs/smb2proto.h     |   3 +-
 fs/cifs/smb2transport.c | 150 ++++++++++++++++++++++++---------
 fs/cifs/transport.c     |  14 +++-
 13 files changed, 551 insertions(+), 119 deletions(-)

-- 
2.16.4


^ permalink raw reply	[flat|nested] 9+ messages in thread
* [PATCH v4 0/6] add multichannel support
@ 2019-11-03  1:21 Aurelien Aptel
  2019-11-03  1:21 ` [PATCH v4 5/6] cifs: try opening channels after mounting Aurelien Aptel
  0 siblings, 1 reply; 9+ messages in thread
From: Aurelien Aptel @ 2019-11-03  1:21 UTC (permalink / raw)
  To: linux-cifs; +Cc: smfrench, Aurelien Aptel

Hello,

I have very little time to work on this during the work week due to
annoying^W lovely customers, company stuff and backporting work but
during this long weekend of festivities I somehow mustered the courage
to dig in again...

So without further ado, here is another iteration of the multichannel
patchset.

Couple of changes since last time:
* do round-robin on the channels (cycle over them on requests)
* reuse master client guid for new channel (fixes binding against samba)
* simplification of crypto key lookup and potential race condition fix
* properly discard channels on session closing
* better display of channels info in DebugData

Usage
=====

Make sure your server exposes multiple interfaces and mount with:

    -o multichannel,max_channels=2

To have a total of 2 connections. One master and one extra channel.
* Each call to ...send_recv() will alternate between the 2 channels.
* You can check /proc/fs/cifs/DebugData for channel infos

Overview
========

Multichannel is part of SMB3 and above. It lets the client "share" the
same SMB session over multiple TCP (or RDMA) connections.

* First, a regular connection is made.

client                                                       server
  |							       |
  | ----------- negprot req (unsigned, guid=0xABC)--------->   |
  |							       |
  | <---------- negprot res (unsigned)	-------------------    |
  |							       |
  | ----------  sesssetup req (sesid=0x000, unsigned) ------>  |
  |							       |
  | <---------  sesssetup res (sesid=0x123, signed) ---------  |
  |                                                            |
  |  (client now has signing/encryption/decryption keys)       |
  |                                                            |
  |							       |
  | ----------- ... (signed) ------------------------------->  |
  | <-------------------------------------------------------   |
  |							       |

* The client then queries the network interface available on the
  server via an FSCTL (this already implemented in cifs.ko)

* For each new channel, the client opens another connection, but
  reuses the client guid, and at session setup it sets a binding flag
  to say "this is a new channel, please make this session an alias to
  sess id 0x123"

* Since the session is an alias, you can use TreeCons, files, etc
  interchangeably between channels as if it was the same transport.

* Channels use the same encryption/decryption keys from the session
  BUT each channel has its own signing key.

* When you see "signed" in lowercase, it's signed using the session/master key.
  When you see "SIGNED" in uppercase, it's signed with the CHANNEL key.

Here is the traffic for opening and using new channels:
                                                              |
  | ---------  negprot req (unsigned, guid=0xABC) ----------> |  o
  |                                                           |  o
  | <--------  negprog res (unsigned) ----------------------  |  o BINDING
  |                                                           |  o  PART
  | ---------  sesssetup req (sesid=0x123, binding, signed)-> |  o
  |                                                           |  o
  | <--------  sesssetup res (sesid=0x123, SIGNED) ---------  |  o
  |                                                           |
  | (from then on, client needs to use CHANNEL key            |
  |  for signing)                                             |
  |                                                           |
  | ---------  ... (SIGNED) --------------------------------> |
  | <-------------------------------------------------------- |
  |                                                           |

This signing key business is tricky to get right. As usual refer to
MS-SMB2 for more boring details.

cifs.ko implementation
======================

Data structure
--------------

We add an array of struct cifs_chan to cifs_ses. A channel has a
server pointer and a signing key.

struct cifs_ses {
    struct TCP_Server_Info *server; <-- master con
    u8 smb3signingkey[...];         <-- master sign key
    u8 smb3encryptionkey[...];      <-- extra channels & master
    u8 smb3decryptionkey[...];      <-- extra channels & master
    bool binding;                   <--- true during BINDING PART
    ...
    struct cifs_chan {
           struct TCP_Server_Info *server; <-- channel con
           u8 signkey[SMB3_SIGN_KEY_SIZE]; <-- channel sign key
    } chans[16];
};

New channel connections (TCP_Server_Info) end up in the usual global
cifs_tcp_ses_list linked list but also in the ses channel array.

 ,-------------> global cifs_tcp_ses_list <-------------------------.
 |                                                                  |
 '- TCP_Server_Info  <-->  TCP_Server_Info  <-->  TCP_Server_Info <-'
       (master con)           (chan#1 con)         (chan#2 con)
       |      ^                    ^                    ^
       v      '--------------------|--------------------'
    cifs_ses                       |
    - chan_count = 3               |
    - chans[].server --------------'

Some important details:

* ses->server still points to the master con.
* ses->server == chans[0].server
* chan#1 and chan#2 have an empty server->smb_ses_list

Channel/session establishment
-----------------------------

* When opening channels we check ses->binding to know that we are in the
  BINDING PART. In which case we don't create a new session object but
  merely reuse the existing one.

* chan_count only keeps track of fully established channels. During
  the BINDING PART there is actually chan_count+1 channels, the last
  one being the channel in the process of being created.

NOTE: we could add a state field to cifs_chan and make cifs_count
      always accurate. Not sure if it's better.

Channel dispatch
----------------

On syscalls, as we get lower and lower in the bowels of cifs.ko we
eventually reach the transport layer. There we often pass a ses
pointer to code that assumes the transport can be reached via
ses->server.

* A big change was to decouple ses from server in those codepaths. So
  that we can select a channel and make the code use that one instead
  of the master.
* That often means passing ses AND server and replacing ses->server by
  server.

Signing and encryption
----------------------

* When generating the keys we make sure to not overwrite enc/dec keys
  for new channels and store the sign key in cifs_chan.

* When sending and receiving a message we have to search for the right
  keys by searching the whole cifs_tcp_ses_list instead of just
  looking at the master con sessions.


Aurelien Aptel (6):
  cifs: sort interface list by speed
  cifs: add multichannel mount options and data structs
  cifs: add server param
  cifs: switch servers depending on binding state
  cifs: try opening channels after mounting
  cifs: dump channel info in DebugData

 fs/cifs/cifs_debug.c    |  35 +++++++-
 fs/cifs/cifs_spnego.c   |   2 +-
 fs/cifs/cifsfs.c        |   4 +
 fs/cifs/cifsglob.h      |  45 +++++++++-
 fs/cifs/cifsproto.h     |   8 ++
 fs/cifs/connect.c       |  92 ++++++++++++++++----
 fs/cifs/sess.c          | 218 +++++++++++++++++++++++++++++++++++++++++++++++-
 fs/cifs/smb2misc.c      |  37 +++++---
 fs/cifs/smb2ops.c       |  31 +++++--
 fs/cifs/smb2pdu.c       | 109 ++++++++++++++----------
 fs/cifs/smb2proto.h     |   3 +-
 fs/cifs/smb2transport.c | 165 +++++++++++++++++++++++++++---------
 fs/cifs/transport.c     |  18 +++-
 13 files changed, 636 insertions(+), 131 deletions(-)

--
2.16.4

^ permalink raw reply	[flat|nested] 9+ messages in thread
end of thread, other threads:[~2019-11-03  1:21 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-20  5:01 [PATCH v4 0/6] multichannel Aurelien Aptel
2019-09-20  5:01 ` [PATCH v4 1/6] cifs: add multichannel mount options and data structs Aurelien Aptel
2019-09-20  5:01 ` [PATCH v4 2/6] cifs: add server param Aurelien Aptel
2019-09-20  5:01 ` [PATCH v4 3/6] cifs: switch servers depending on binding state Aurelien Aptel
2019-09-20  5:01 ` [PATCH v4 4/6] cifs: sort interface list by speed Aurelien Aptel
2019-09-20  5:01 ` [PATCH v4 5/6] cifs: try opening channels after mounting Aurelien Aptel
2019-09-20  7:30   ` Steve French
2019-09-20  5:01 ` [PATCH v4 6/6] cifs: mention if an interface has a channel connected to it Aurelien Aptel
2019-11-03  1:21 [PATCH v4 0/6] add multichannel support Aurelien Aptel
2019-11-03  1:21 ` [PATCH v4 5/6] cifs: try opening channels after mounting Aurelien Aptel

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).