From: Aurelien Aptel <aaptel@suse.com>
To: linux-cifs@vger.kernel.org
Cc: smfrench@gmail.com, Aurelien Aptel <aaptel@suse.com>
Subject: [PATCH v4 0/6] add multichannel support
Date: Sun, 3 Nov 2019 02:21:06 +0100 [thread overview]
Message-ID: <20191103012112.12212-1-aaptel@suse.com> (raw)
Hello,
I have very little time to work on this during the work week due to
annoying^W lovely customers, company stuff and backporting work but
during this long weekend of festivities I somehow mustered the courage
to dig in again...
So without further ado, here is another iteration of the multichannel
patchset.
Couple of changes since last time:
* do round-robin on the channels (cycle over them on requests)
* reuse master client guid for new channel (fixes binding against samba)
* simplification of crypto key lookup and potential race condition fix
* properly discard channels on session closing
* better display of channels info in DebugData
Usage
=====
Make sure your server exposes multiple interfaces and mount with:
-o multichannel,max_channels=2
To have a total of 2 connections. One master and one extra channel.
* Each call to ...send_recv() will alternate between the 2 channels.
* You can check /proc/fs/cifs/DebugData for channel infos
Overview
========
Multichannel is part of SMB3 and above. It lets the client "share" the
same SMB session over multiple TCP (or RDMA) connections.
* First, a regular connection is made.
client server
| |
| ----------- negprot req (unsigned, guid=0xABC)---------> |
| |
| <---------- negprot res (unsigned) ------------------- |
| |
| ---------- sesssetup req (sesid=0x000, unsigned) ------> |
| |
| <--------- sesssetup res (sesid=0x123, signed) --------- |
| |
| (client now has signing/encryption/decryption keys) |
| |
| |
| ----------- ... (signed) -------------------------------> |
| <------------------------------------------------------- |
| |
* The client then queries the network interface available on the
server via an FSCTL (this already implemented in cifs.ko)
* For each new channel, the client opens another connection, but
reuses the client guid, and at session setup it sets a binding flag
to say "this is a new channel, please make this session an alias to
sess id 0x123"
* Since the session is an alias, you can use TreeCons, files, etc
interchangeably between channels as if it was the same transport.
* Channels use the same encryption/decryption keys from the session
BUT each channel has its own signing key.
* When you see "signed" in lowercase, it's signed using the session/master key.
When you see "SIGNED" in uppercase, it's signed with the CHANNEL key.
Here is the traffic for opening and using new channels:
|
| --------- negprot req (unsigned, guid=0xABC) ----------> | o
| | o
| <-------- negprog res (unsigned) ---------------------- | o BINDING
| | o PART
| --------- sesssetup req (sesid=0x123, binding, signed)-> | o
| | o
| <-------- sesssetup res (sesid=0x123, SIGNED) --------- | o
| |
| (from then on, client needs to use CHANNEL key |
| for signing) |
| |
| --------- ... (SIGNED) --------------------------------> |
| <-------------------------------------------------------- |
| |
This signing key business is tricky to get right. As usual refer to
MS-SMB2 for more boring details.
cifs.ko implementation
======================
Data structure
--------------
We add an array of struct cifs_chan to cifs_ses. A channel has a
server pointer and a signing key.
struct cifs_ses {
struct TCP_Server_Info *server; <-- master con
u8 smb3signingkey[...]; <-- master sign key
u8 smb3encryptionkey[...]; <-- extra channels & master
u8 smb3decryptionkey[...]; <-- extra channels & master
bool binding; <--- true during BINDING PART
...
struct cifs_chan {
struct TCP_Server_Info *server; <-- channel con
u8 signkey[SMB3_SIGN_KEY_SIZE]; <-- channel sign key
} chans[16];
};
New channel connections (TCP_Server_Info) end up in the usual global
cifs_tcp_ses_list linked list but also in the ses channel array.
,-------------> global cifs_tcp_ses_list <-------------------------.
| |
'- TCP_Server_Info <--> TCP_Server_Info <--> TCP_Server_Info <-'
(master con) (chan#1 con) (chan#2 con)
| ^ ^ ^
v '--------------------|--------------------'
cifs_ses |
- chan_count = 3 |
- chans[].server --------------'
Some important details:
* ses->server still points to the master con.
* ses->server == chans[0].server
* chan#1 and chan#2 have an empty server->smb_ses_list
Channel/session establishment
-----------------------------
* When opening channels we check ses->binding to know that we are in the
BINDING PART. In which case we don't create a new session object but
merely reuse the existing one.
* chan_count only keeps track of fully established channels. During
the BINDING PART there is actually chan_count+1 channels, the last
one being the channel in the process of being created.
NOTE: we could add a state field to cifs_chan and make cifs_count
always accurate. Not sure if it's better.
Channel dispatch
----------------
On syscalls, as we get lower and lower in the bowels of cifs.ko we
eventually reach the transport layer. There we often pass a ses
pointer to code that assumes the transport can be reached via
ses->server.
* A big change was to decouple ses from server in those codepaths. So
that we can select a channel and make the code use that one instead
of the master.
* That often means passing ses AND server and replacing ses->server by
server.
Signing and encryption
----------------------
* When generating the keys we make sure to not overwrite enc/dec keys
for new channels and store the sign key in cifs_chan.
* When sending and receiving a message we have to search for the right
keys by searching the whole cifs_tcp_ses_list instead of just
looking at the master con sessions.
Aurelien Aptel (6):
cifs: sort interface list by speed
cifs: add multichannel mount options and data structs
cifs: add server param
cifs: switch servers depending on binding state
cifs: try opening channels after mounting
cifs: dump channel info in DebugData
fs/cifs/cifs_debug.c | 35 +++++++-
fs/cifs/cifs_spnego.c | 2 +-
fs/cifs/cifsfs.c | 4 +
fs/cifs/cifsglob.h | 45 +++++++++-
fs/cifs/cifsproto.h | 8 ++
fs/cifs/connect.c | 92 ++++++++++++++++----
fs/cifs/sess.c | 218 +++++++++++++++++++++++++++++++++++++++++++++++-
fs/cifs/smb2misc.c | 37 +++++---
fs/cifs/smb2ops.c | 31 +++++--
fs/cifs/smb2pdu.c | 109 ++++++++++++++----------
fs/cifs/smb2proto.h | 3 +-
fs/cifs/smb2transport.c | 165 +++++++++++++++++++++++++++---------
fs/cifs/transport.c | 18 +++-
13 files changed, 636 insertions(+), 131 deletions(-)
--
2.16.4
next reply other threads:[~2019-11-03 1:21 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-03 1:21 Aurelien Aptel [this message]
2019-11-03 1:21 ` [PATCH v4 1/6] cifs: sort interface list by speed Aurelien Aptel
2019-11-25 21:29 ` Pavel Shilovsky
2019-11-26 16:04 ` [EXTERNAL] " Tom Talpey
2019-11-26 16:54 ` Aurélien Aptel
2019-11-26 18:38 ` Tom Talpey
2019-11-03 1:21 ` [PATCH v4 2/6] cifs: add multichannel mount options and data structs Aurelien Aptel
2019-11-03 1:21 ` [PATCH v4 3/6] cifs: add server param Aurelien Aptel
2019-11-03 1:21 ` [PATCH v4 4/6] cifs: switch servers depending on binding state Aurelien Aptel
2019-11-03 1:21 ` [PATCH v4 5/6] cifs: try opening channels after mounting Aurelien Aptel
2019-11-03 1:21 ` [PATCH v4 6/6] cifs: dump channel info in DebugData Aurelien Aptel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191103012112.12212-1-aaptel@suse.com \
--to=aaptel@suse.com \
--cc=linux-cifs@vger.kernel.org \
--cc=smfrench@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).