linux-cifs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
@ 2021-05-25 14:14 ` samba-bugs
  2021-05-25 18:19 ` samba-bugs
                   ` (31 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 14:14 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

Björn Jacke <bjacke@samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         QA Contact|samba-qa@samba.org          |cifs-qa@samba.org
            Version|4.13.3                      |5.x
            Product|Samba 4.1 and newer         |CifsVFS
          Component|libsmbclient                |kernel fs

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
  2021-05-25 14:14 ` [Bug 14713] SMBv3 negotiation fails with a Solaris server samba-bugs
@ 2021-05-25 18:19 ` samba-bugs
  2021-05-25 18:26 ` samba-bugs
                   ` (30 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 18:19 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #7 from Steve French <sfrench@samba.org> ---
There isn't much we can see in the traces you provide except the following:
1) for the "445broken.pcap" wireshark trace we can see that the server hung up
the session (probably server bug but hard to prove since the request they hung
up on (SMB3 tree connect) is encrypted when you mount with "seal").  If you
have access to that system and can reproduce it, it might be helpful to dump
the decryption keys so we can see the failing request and make sure it is not
malformed.  See https://wiki.samba.org/index.php/Wireshark_Decryption for
details on how to dump decryption keys

2) for the debug trace you attached all we can see is the "EINVAL" being
returned

   [  205.645184] CIFS: fs/cifs/connect.c: Received no data or error: 0
   [  205.645187] CIFS: fs/cifs/connect.c: cifs_reconnect: will not do DFS 
                  failover: rc = -22

presumably the same issue (the server hung up due to a bug processing the tree
connect request (the first encrypted request) when encryption is specified on
mount).

Could you retry without specifying "seal" on mount and include (or send to my
gmail) the wireshark traces?  In addition the dynamic trace info may be useful:

1) In one process type "trace-cmd record -e cifs"
2) Run the test in another window
3) dump the dynamic trace info "trace-cmd show"
4) kill the trace from step 1

If it can not be tried without encryption ("seal" on mount) then can you dump
the decryption keys for the trace you run (as described in the link above)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
  2021-05-25 14:14 ` [Bug 14713] SMBv3 negotiation fails with a Solaris server samba-bugs
  2021-05-25 18:19 ` samba-bugs
@ 2021-05-25 18:26 ` samba-bugs
  2021-05-25 18:51 ` samba-bugs
                   ` (29 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 18:26 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #8 from Steve French <sfrench@samba.org> ---
Two additional questions:
1) As long as you are running a reasonably recent kernel or distro update (5.0
or later kernel should be fine, and current RHEL/CentOS/Oracle client ie
RHEL/CentOS/Oracle 8.4 or 8.3 should be fine as well since Redhat backports
many fixes to their older kernel) can you try not specifying "vers=" at all and
see what it negotiates with the server?  (should be 3.1.1 - you can see in
/proc/fs/cifs/DebugData)

2) Does mounting with "vers=2.1" work?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (2 preceding siblings ...)
  2021-05-25 18:26 ` samba-bugs
@ 2021-05-25 18:51 ` samba-bugs
  2021-05-25 19:03 ` samba-bugs
                   ` (28 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 18:51 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #9 from Richard Flint <richard.flint@gmail.com> ---
1) As long as you are running a reasonably recent kernel or distro update (5.0
or later kernel should be fine, and current RHEL/CentOS/Oracle client ie
RHEL/CentOS/Oracle 8.4 or 8.3 should be fine as well since Redhat backports
many fixes to their older kernel) can you try not specifying "vers=" at all and
see what it negotiates with the server?  (should be 3.1.1 - you can see in
/proc/fs/cifs/DebugData)

If I fail to specify the vers= option. Mounting fails completely. Dmesg says
our usual:

[162979.482987] CIFS: VFS: \\nonsuch failed to connect to IPC (rc=-11)
[162979.484449] CIFS: VFS: session 0000000083b7840c has no tcon available for a
dfs referral request
[162979.485899] CIFS: VFS: cifs_mount failed w/return code = -2

and the Solaris server says:

May 24 23:09:46 nonsuch smbcmn: [ID 997540 kern.warning] WARNING:
../../common/fs/smbsrv/smb2_dispatch.c:smb2_dispatch_message:134:Decryption
failure (71)!
May 25 19:44:21 nonsuch smbcmn: [ID 997540 kern.warning] WARNING:
../../common/fs/smbsrv/smb2_dispatch.c:smb2_dispatch_message:134:Decryption
failure (72)!

I have also seen the above Solaris server side messages intermittently and
think they may be related to the issue I am experiencing. But if SMBv3 is
broken on Solaris, I do not understand why it works fine with MacOS.

2) Does mounting with "vers=2.1" work?

Yes, it works fine. In fact, it is required to make anything work as omitting
it results in the above behaviour.


Regarding your other comments about traces and encryption keys, I will need to
time to get these, but will indeed do so.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (3 preceding siblings ...)
  2021-05-25 18:51 ` samba-bugs
@ 2021-05-25 19:03 ` samba-bugs
  2021-05-25 19:09 ` samba-bugs
                   ` (27 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 19:03 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #10 from Steve French <sfrench@samba.org> ---
Since the Solaris server logs:


May 24 23:09:46 nonsuch smbcmn: [ID 997540 kern.warning] WARNING:
../../common/fs/smbsrv/smb2_dispatch.c:smb2_dispatch_message:134:Decryption
failure (71)!

are you sure that you mounted without encryption (ie did not specify "seal=" on
mount)?

If this connection is defaulting to encryption whether or not the client
specifies it on mount, that implies that the server is configured with
encryption as required ... which is odd - because the server allowed vers=2.1
(which is not encrypted, encryption was added in the SMB3 and later versions of
the protocol and not supported with SMB2.1) but fails with vers=3.0 or 3.1.1
(smb3.1.1 is the typical default) which presumably means the server is
negotiating with encryption required (but only for a subset of dialects). 
Strange server configuration.

Can you send or attach the vers=3.1.1 (or default with no vers= specified)
wireshark trace so we can see what crypto algorithm the server is defaulting to
(even if we can't see the keys - we can see how it is trying to encrypt/decrypt
if smb3.1.1 is used instead of smb3.0)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (4 preceding siblings ...)
  2021-05-25 19:03 ` samba-bugs
@ 2021-05-25 19:09 ` samba-bugs
  2021-05-25 19:09 ` samba-bugs
                   ` (26 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 19:09 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #11 from Richard Flint <richard.flint@gmail.com> ---
When I omitted vers= as you requested, I definitely did not specify seal. I
have attached the server SMB configuration for completeness but the most
relevant settings are below:

server_lmauth_level=5
server_minprotocol=2.1
server_maxprotocol=3.1
server_encrypt_data=true
server_reject_unencrypt=false
server_signing_enabled=true
server_signing_required=false
restrict_anonymous=true

I will get a wireshark trace with vers=3.1.1 as you requested shortly.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (5 preceding siblings ...)
  2021-05-25 19:09 ` samba-bugs
@ 2021-05-25 19:09 ` samba-bugs
  2021-05-25 19:13 ` samba-bugs
                   ` (25 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 19:09 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #12 from Richard Flint <richard.flint@gmail.com> ---
Created attachment 16628
  --> https://bugzilla.samba.org/attachment.cgi?id=16628&action=edit
Solaris server SMB configuration

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (6 preceding siblings ...)
  2021-05-25 19:09 ` samba-bugs
@ 2021-05-25 19:13 ` samba-bugs
  2021-05-25 19:19 ` samba-bugs
                   ` (24 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 19:13 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #13 from Steve French <sfrench@samba.org> ---
Will be interesting to see which crypto algorithm they negotiate - but at least
one way around this is to mount with vers=2.1 or change the config line:
        server_encrypt_data=true
so that it doesn't end up causing SMB3.0 and later to encrypt until we figure
out where the bug is in encryption (presumably is on the server side since
encrypted mounts work to every other server from Linux).

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (7 preceding siblings ...)
  2021-05-25 19:13 ` samba-bugs
@ 2021-05-25 19:19 ` samba-bugs
  2021-05-25 19:27 ` samba-bugs
                   ` (23 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 19:19 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #14 from Richard Flint <richard.flint@gmail.com> ---
Unfortunately, I spent quite a bit of time trying various combination of
options and I have to say I have never managed to get SMBv3 to work between
Linux and Solaris with or without encryption. E.g. setting vers=3.1.1 without
seal and with server_encrypt_data=false on Solaris does not result in
successful mounting:

[164849.013736] CIFS: VFS: \\seraphix-3.achelon.net failed to connect to IPC
(rc=-13)
[164849.015760] CIFS: VFS: cifs_mount failed w/return code = -13


But let's try and debug one test case at a time so things don't get confusing.
I have a wireshark dump with vers=3.1.1 and seal options. I will email to your
gmail.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (8 preceding siblings ...)
  2021-05-25 19:19 ` samba-bugs
@ 2021-05-25 19:27 ` samba-bugs
  2021-05-25 19:37 ` samba-bugs
                   ` (22 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 19:27 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #15 from Richard Flint <richard.flint@gmail.com> ---
If there is a bug in Solaris SMBv3 encryption handling, I'm perplexed as to why
it works fine when doing this from smbclient. E.g. the below appears to work:

smbclient //nonsuch/myshare --debuglevel=10 --encrypt --user=myuser

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (9 preceding siblings ...)
  2021-05-25 19:27 ` samba-bugs
@ 2021-05-25 19:37 ` samba-bugs
  2021-05-25 19:42 ` samba-bugs
                   ` (21 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 19:37 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #16 from Steve French <sfrench@samba.org> ---
Based on the trace you sent - can see that when mounting with 3.1.1 (or default
which ends up the same thing), the server responds with
SMB2_ENCRYPTION_CAPABILITIES set to CipherId: AES-128-GCM which is interesting
because that is the 'normal' case we see (Windows, Azure, current Samba server
etc.) so this is less likely to be a bug in the client due to falling back to
something different than the more common GCM.

Do you have the equivalent trace from smbclient (the Samba userspace tool) to
the same share (trying to negotiate SMB3.1.1) for comparison?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (10 preceding siblings ...)
  2021-05-25 19:37 ` samba-bugs
@ 2021-05-25 19:42 ` samba-bugs
  2021-05-25 19:43 ` samba-bugs
                   ` (20 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 19:42 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #17 from Steve French <sfrench@samba.org> ---
In both trace you sent (seal and not specifying seal on the mount) you can see
the server is requiring encryption (unless you mount with smb2.1 or earlier). 
See attached screenshot

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (11 preceding siblings ...)
  2021-05-25 19:42 ` samba-bugs
@ 2021-05-25 19:43 ` samba-bugs
  2021-05-25 19:47 ` samba-bugs
                   ` (19 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 19:43 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #18 from Steve French <sfrench@samba.org> ---
Created attachment 16629
  --> https://bugzilla.samba.org/attachment.cgi?id=16629&action=edit
session-setup-response-when-no-seal-on-mount

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (12 preceding siblings ...)
  2021-05-25 19:43 ` samba-bugs
@ 2021-05-25 19:47 ` samba-bugs
  2021-05-25 20:31 ` samba-bugs
                   ` (18 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 19:47 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #19 from Richard Flint <richard.flint@gmail.com> ---
Understood. Please see your gmail for a pcap and a debug level 10 trace of
smbclient successfully negotiating an encrypted SMB 3.1.1 connection with the
same Solaris share, whereas the kernel mount for the same failed.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (13 preceding siblings ...)
  2021-05-25 19:47 ` samba-bugs
@ 2021-05-25 20:31 ` samba-bugs
  2021-05-25 20:38 ` samba-bugs
                   ` (17 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 20:31 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #20 from Steve French <sfrench@samba.org> ---
Comparing with smbclient, there are a few interesting things which differ:
a) smbclient sets a default domain name ("SAMBA").  To make this identical for
the kernel mount ("mount -t cifs ...") case you could try setting domain=
parameter to the same.  I doubt this will make a difference because in neither
case does the server indicate in its SessionSetup response that authentication
ended up as 'guest' so presumably Solaris server thinks the user authenticated
properly in both cases (albeit it could be a very unlikely case where
"SAMBA/username" is different than "username")
b) there are some NegotiateFlags (NTLMSSP flags) set differently during
negotiation:
     1) smbclient sets "Negotiate Version"
     2) cifs.ko sets "Negotiate Seal" and "Negotiate Target Info" and
"Negotiate 56"
but otherwise the flags look the same.  
c) smbclient sends both an old Lanman (Lanmanv2) and NTLM (NTLMv2) response in
the NTLMSSP_AUTH SessionSetup request, but zeroes the Lanman field, while
cifs.ko doesn't send Lanman.  This is unlikely to be related

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (14 preceding siblings ...)
  2021-05-25 20:31 ` samba-bugs
@ 2021-05-25 20:38 ` samba-bugs
  2021-05-25 20:55 ` samba-bugs
                   ` (16 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 20:38 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #21 from Steve French <sfrench@samba.org> ---
Key next steps:
1) seeing if it is possible to decrypt the wireshark trace (see link provided
earlier for instructions) although this may require rebuilding the cifs.ko to
dump keys (does Solaris have a way to view encrypted traces taken on the server
side?)
2) looking in more detail at the server.  It doesn't indicate why it was
rejected:
./../common/fs/smbsrv/smb2_dispatch.c:smb2_dispatch_message:134:Decryption
failure (71)!
May 25 19:44:21 nonsuch smbcmn: [ID 997540 kern.warning] WARNING:
../../common/fs/smbsrv/smb2_dispatch.c:smb2_dispatch_message:134:Decryption
failure (72)!

Do you have any contacts with Solaris support to see if they can see if they
can provide more information?  My guess is that they don't like the format or
the flags of something specified in the tree connect (since this works to every
other server type) - but they may also have some subtle rounding error with
decryption on their server side.

Googling for the error I do see one other report of similar sounding problem to
Solaris so it has been around a long time (see
https://bugs.centos.org/view.php?id=16531)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (15 preceding siblings ...)
  2021-05-25 20:38 ` samba-bugs
@ 2021-05-25 20:55 ` samba-bugs
  2021-05-25 21:40 ` samba-bugs
                   ` (15 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 20:55 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #22 from Richard Flint <richard.flint@gmail.com> ---
I'm sure it won't shock you, but adding domain=SAMBA to the mount options
hasn't miraculously fixed this, but at least it's another data point.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (16 preceding siblings ...)
  2021-05-25 20:55 ` samba-bugs
@ 2021-05-25 21:40 ` samba-bugs
  2021-05-25 22:33 ` samba-bugs
                   ` (14 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 21:40 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #23 from Richard Flint <richard.flint@gmail.com> ---
I am going to see if I can rebuild cifs.ko with the right options to dump the
keys. But this is a lot of work - is it likely to lead to anything useful?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (17 preceding siblings ...)
  2021-05-25 21:40 ` samba-bugs
@ 2021-05-25 22:33 ` samba-bugs
  2021-05-26  8:57 ` samba-bugs
                   ` (13 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-25 22:33 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #24 from Steve French <sfrench@samba.org> ---
The reason that it is of some value is that if wireshark can decrypt it and
shows no errors (ie decrypt the first encrypted frame, the SMB3.1.1 tree
connect request) then it is even more likely a server bug ... (perhaps some
strange case where they expect a padded response that has a length divisible by
8 or some such bug)

if you have access to the source RPM then rebuilding it might only take a few
minutes (you could e.g. just remove the 2 ifdef CONFIG_CIFS_DEBUG_DUMP_KEYS in
fs/cifs/smb2transport.c

e.g. remove the ifdef and endif here (and the one before that in the same file)

#ifdef CONFIG_CIFS_DEBUG_DUMP_KEYS
        cifs_dbg(VFS, "%s: dumping generated AES session keys\n", __func__);
        /*
         * The session id is opaque in terms of endianness, so we can't
         * print it as a long long. we dump it as we got it on the wire
         */
        cifs_dbg(VFS, "Session Id    %*ph\n", (int)sizeof(ses->Suid),
                        &ses->Suid);
        cifs_dbg(VFS, "Cipher type   %d\n", server->cipher_type);
        cifs_dbg(VFS, "Session Key   %*ph\n",
                 SMB2_NTLMV2_SESSKEY_SIZE, ses->auth_key.response);
        cifs_dbg(VFS, "Signing Key   %*ph\n",
                 SMB3_SIGN_KEY_SIZE, ses->smb3signingkey);
        if ((server->cipher_type == SMB2_ENCRYPTION_AES256_CCM) ||
                (server->cipher_type == SMB2_ENCRYPTION_AES256_GCM)) {
                cifs_dbg(VFS, "ServerIn Key  %*ph\n",
                                SMB3_GCM256_CRYPTKEY_SIZE,
ses->smb3encryptionkey);
                cifs_dbg(VFS, "ServerOut Key %*ph\n",
                                SMB3_GCM256_CRYPTKEY_SIZE,
ses->smb3decryptionkey);
        } else {
                cifs_dbg(VFS, "ServerIn Key  %*ph\n",
                                SMB3_GCM128_CRYPTKEY_SIZE,
ses->smb3encryptionkey);
                cifs_dbg(VFS, "ServerOut Key %*ph\n",
                                SMB3_GCM128_CRYPTKEY_SIZE,
ses->smb3decryptionkey);
        }
#endif

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (18 preceding siblings ...)
  2021-05-25 22:33 ` samba-bugs
@ 2021-05-26  8:57 ` samba-bugs
  2021-05-26  9:00 ` samba-bugs
                   ` (12 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-26  8:57 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

Aurélien Aptel <aaptel@samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |aaptel@samba.org

--- Comment #25 from Aurélien Aptel <aaptel@samba.org> ---
You might not need to recompile your kernel.

I have a GDB script (made for 4.4) you might be able to run as root

https://lists.samba.org/archive/samba-technical/attachments/20170524/2950140e/cifs_dump_keys.py

It reads the keys from kernel memory. The offset (OFF var) might have to be
updated for your kernel (or not) but it's worth a try.

(this script just reads from memory, worst case it prints garbage)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (19 preceding siblings ...)
  2021-05-26  8:57 ` samba-bugs
@ 2021-05-26  9:00 ` samba-bugs
  2021-05-26  9:47 ` samba-bugs
                   ` (11 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-26  9:00 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #26 from Aurélien Aptel <aaptel@samba.org> ---
Ah sorry, this script assumes you have a successful mount point. If mount fails
it won't be of any use.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (20 preceding siblings ...)
  2021-05-26  9:00 ` samba-bugs
@ 2021-05-26  9:47 ` samba-bugs
  2021-05-26 15:36 ` samba-bugs
                   ` (10 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-26  9:47 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #27 from Richard Flint <richard.flint@gmail.com> ---
I will look into getting the decryption keys. I thought maybe I could dump the
TreeConnect on the Solaris side using the Dtrace capability but if it's
possible I haven't figured it out yet. 

I'm baffled as to how MacOS and smbclient work fine but Linux kernel mounts
don't.
I guess my fear is that some flag is set wrong, maybe during negotiation, and
there is no simple knob we can turn to set it just to test.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (21 preceding siblings ...)
  2021-05-26  9:47 ` samba-bugs
@ 2021-05-26 15:36 ` samba-bugs
  2021-05-26 18:59 ` samba-bugs
                   ` (9 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-26 15:36 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #28 from Steve French <sfrench@samba.org> ---
> fear is that some flag is set wrong, maybe during negotiation

There isn't an obvious reason why any of the flag differences would matter
(unless server bug), but it should be possible to test mount with smb3.1.1
(without encryption) by changing the server config line
      server_encrypt_data=true
and make sure server doesn't hang up on tree connect (as it does with
encryption)

If we verify that wireshark can decrypt it, then the only strange guesses I can
think of that would cause the server to give up on the tree connect are:
1) difference in tree connect flags with smb3.1.1
2) differences in padding of the tree connect request that confuse the server

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (22 preceding siblings ...)
  2021-05-26 15:36 ` samba-bugs
@ 2021-05-26 18:59 ` samba-bugs
  2021-05-26 21:51 ` samba-bugs
                   ` (8 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-26 18:59 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #29 from Richard Flint <richard.flint@gmail.com> ---
Unfortunately, as I mentioned by email. The connection does indeed also fail
with server_encrypt_data=false.

After setting server_encrypt_data=false I try to mount with vers=3.1.1 and
without the seal option (since encryption is off on the server). This results
in the following mysterious error:

mount error(13): Permission denied
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) and kernel log
messages (dmesg)

[249955.014343] CIFS: Attempting to mount //nonsuch/myshare
[249955.020762] CIFS: VFS: \\nonsuch failed to connect to IPC (rc=-13)
[249955.023383] CIFS: VFS: cifs_mount failed w/return code = -13

see wireshark trace in your email "3.1.1_encryptionoff.pcap".

Trying with smbclient on the same share with the same user and same password
with server_encrypt_data=false on the server and no --encrypt option results in
success:

smbclient //nonsuch/myshare --debuglevel=10 --user=myuser

see wireshark trace in your email "3.1.1_smbclient.pcap".

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (23 preceding siblings ...)
  2021-05-26 18:59 ` samba-bugs
@ 2021-05-26 21:51 ` samba-bugs
  2021-05-26 21:54 ` samba-bugs
                   ` (7 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-26 21:51 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #30 from Steve French <sfrench@samba.org> ---
Presumably Solaris server doesn't like the signature (which is odd).  Let's
compare with SMB3.0 as well if possible.  Was the SMB2.1 mounts (which
succeeded) signed?  That is weird if signed works with 2.1 and fails with 3.0
and 3.1.1

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (24 preceding siblings ...)
  2021-05-26 21:51 ` samba-bugs
@ 2021-05-26 21:54 ` samba-bugs
  2021-05-26 22:09 ` samba-bugs
                   ` (6 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-26 21:54 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #31 from Richard Flint <richard.flint@gmail.com> ---
The server says:

May 26 19:53:58 nonsuch smbsrv: [ID 211007 kern.warning] WARNING: bad
signature, cmd=0x3

so you are right, it doesn't like the signature.

I will obtain wireshark traces with vers=3.0 (and equivalent smb.conf
option for smbclient) set for comparison.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (25 preceding siblings ...)
  2021-05-26 21:54 ` samba-bugs
@ 2021-05-26 22:09 ` samba-bugs
  2021-05-26 22:26 ` samba-bugs
                   ` (5 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-26 22:09 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #32 from Ronnie Sahlberg <ronniesahlberg@gmail.com> ---
Note that in vers=3.1.1  the TreeConnect calls are always signed
which means that client and server must agree on what the session key is.
Just like when encryption is used.

Try to see what happens if you use vers=3.0 in this scenario as the TreeConnect
will not be signed.


It does sound like there is something wrong with the session key and client and
server disagree on it.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (26 preceding siblings ...)
  2021-05-26 22:09 ` samba-bugs
@ 2021-05-26 22:26 ` samba-bugs
  2021-05-26 22:39 ` samba-bugs
                   ` (4 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-26 22:26 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #33 from Richard Flint <richard.flint@gmail.com> ---
with vers=3.0 (and without seal) specified on the client and
server_encrypt_data=false specified on the server, the following new fun
happens:

mount error(2): No such file or directory
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) and kernel log
messages (dmesg)

and dmesg says: 

[261829.875860] CIFS: Attempting to mount //nonsuch/myshare
[261829.891575] CIFS: VFS: \\nonsuch\IPC$ validate protocol negotiate failed:
-13
[261829.893269] CIFS: VFS: \\nonsuch failed to connect to IPC (rc=-5)
[261829.895940] CIFS: VFS: \\nonsuch\myshare validate protocol negotiate
failed: -13
[261829.897752] CIFS: VFS: session 00000000bfe08581 has no tcon available for a
dfs referral request
[261829.900102] CIFS: VFS: cifs_mount failed w/return code = -2

I also notice a new message on the server server. Note that cmd=0x3 is now
cmd=0xb - whatever that signifies.

May 26 23:11:53 nonsuch smbsrv: [ID 211007 kern.warning] WARNING: bad
signature, cmd=0xb

A wireshark trace for the above has been emailed to you "3.0_mount.pcap"

As usual, for comparison smbclient works flawless (hopefully -m SMB3 was the
right way to force SMB 3.0 dialect...)
smbclient //nonsuch/myshare -m SMB3 --user=myuser

A wireshark trace for the above smbclient interaction has been emailed to you:
"3.0_smbclient.pcap"

Throughout all this dialogue with you, signing on the server has been
'enabled', but not 'required'.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (27 preceding siblings ...)
  2021-05-26 22:26 ` samba-bugs
@ 2021-05-26 22:39 ` samba-bugs
  2021-05-27 17:33 ` samba-bugs
                   ` (3 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-26 22:39 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #34 from Ronnie Sahlberg <ronniesahlberg@gmail.com> ---
Signing.  It is not required on the server config,

BUT in 3.1.1 signing is required for all TreeConnect calls (0x03) always, by
the protocol.

In 3.0 and 3.0.2 signing is required for all
Ioctl:fsctl_validate_negotiate_info calls. Also by protocol requirements.

The command code for Ioctl is 0x0b, which you saw in the log.


At least it got the mount process a bit further by using vers=3.0 since we got
past the TreeConnect.   I forgot that in 3.0 we have that Ioctl call.
It could have worked, but I forgot about the Ioctl :-(


Anyway, this is another datapoint that tells us that the issue is that it is
related to the session key imo.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (28 preceding siblings ...)
  2021-05-26 22:39 ` samba-bugs
@ 2021-05-27 17:33 ` samba-bugs
  2021-05-30 13:51 ` samba-bugs
                   ` (2 subsequent siblings)
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-27 17:33 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #35 from Richard Flint <richard.flint@gmail.com> ---
Thanks for your help it - does seem that you've narrowed down the cause a
little. Is it still required for me to dump those keys?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (29 preceding siblings ...)
  2021-05-27 17:33 ` samba-bugs
@ 2021-05-30 13:51 ` samba-bugs
  2021-06-04  6:42 ` samba-bugs
  2021-06-04 10:17 ` samba-bugs
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-05-30 13:51 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #36 from Richard Flint <richard.flint@gmail.com> ---
As requested I am emailing you details of the successful SMB2.1 negotiation
using both smbclient:

smbclient //nonsuch/myshare -m SMB2 --user=myuser

and the CIFS mount command (with vers=2.1 and without seal).

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (30 preceding siblings ...)
  2021-05-30 13:51 ` samba-bugs
@ 2021-06-04  6:42 ` samba-bugs
  2021-06-04 10:17 ` samba-bugs
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-06-04  6:42 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #37 from Stefan Metzmacher <metze@samba.org> ---
(In reply to Richard Flint from comment #33)

From 'man smb.conf'

...
SMB2: Re-implementation of the SMB protocol. Used by Windows Vista and later
versions of Windows. SMB2 has sub protocols available.

    SMB2_02: The earliest SMB2 version.

    SMB2_10: Windows 7 SMB2 version.

    SMB2_22: Early Windows 8 SMB2 version.

    SMB2_24: Windows 8 beta SMB2 version.

By default SMB2 selects the SMB2_10 variant.

SMB3: The same as SMB2. Used by Windows 8. SMB3 has sub protocols available.

    SMB3_00: Windows 8 SMB3 version. (mostly the same as SMB2_24)

    SMB3_02: Windows 8.1 SMB3 version.

    SMB3_10: early Windows 10 technical preview SMB3 version.

    SMB3_11: Windows 10 technical preview SMB3 version (maybe final).

By default SMB3 selects the SMB3_11 variant.
...

So '-m SMB3' is the same as '-m SMB3_11', I guess to want
'-m SMB3_00' instead in order to force 3.0.0

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* [Bug 14713] SMBv3 negotiation fails with a Solaris server
       [not found] <bug-14713-10630@https.bugzilla.samba.org/>
                   ` (31 preceding siblings ...)
  2021-06-04  6:42 ` samba-bugs
@ 2021-06-04 10:17 ` samba-bugs
  32 siblings, 0 replies; 33+ messages in thread
From: samba-bugs @ 2021-06-04 10:17 UTC (permalink / raw)
  To: cifs-qa

https://bugzilla.samba.org/show_bug.cgi?id=14713

--- Comment #38 from Richard Flint <richard.flint@gmail.com> ---
Thanks for the correction, you are quite right, the comparison file should have
been done with -m SMB3_00 not -m SMB3.

I have created two new files:

SMB3_00_smbclient.pcap created with command:

smbclient //nonsuch/myshare -m SMB3_00 --user=myuser

and

SMB3_00_encrypt_smbclient.pcap created with command:

smbclient //nonsuch/myshare -m SMB3_00 --encrypt --user=myuser

I am emailing the traces to Steve French's gmail.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

^ permalink raw reply	[flat|nested] 33+ messages in thread

end of thread, other threads:[~2021-06-04 10:18 UTC | newest]

Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <bug-14713-10630@https.bugzilla.samba.org/>
2021-05-25 14:14 ` [Bug 14713] SMBv3 negotiation fails with a Solaris server samba-bugs
2021-05-25 18:19 ` samba-bugs
2021-05-25 18:26 ` samba-bugs
2021-05-25 18:51 ` samba-bugs
2021-05-25 19:03 ` samba-bugs
2021-05-25 19:09 ` samba-bugs
2021-05-25 19:09 ` samba-bugs
2021-05-25 19:13 ` samba-bugs
2021-05-25 19:19 ` samba-bugs
2021-05-25 19:27 ` samba-bugs
2021-05-25 19:37 ` samba-bugs
2021-05-25 19:42 ` samba-bugs
2021-05-25 19:43 ` samba-bugs
2021-05-25 19:47 ` samba-bugs
2021-05-25 20:31 ` samba-bugs
2021-05-25 20:38 ` samba-bugs
2021-05-25 20:55 ` samba-bugs
2021-05-25 21:40 ` samba-bugs
2021-05-25 22:33 ` samba-bugs
2021-05-26  8:57 ` samba-bugs
2021-05-26  9:00 ` samba-bugs
2021-05-26  9:47 ` samba-bugs
2021-05-26 15:36 ` samba-bugs
2021-05-26 18:59 ` samba-bugs
2021-05-26 21:51 ` samba-bugs
2021-05-26 21:54 ` samba-bugs
2021-05-26 22:09 ` samba-bugs
2021-05-26 22:26 ` samba-bugs
2021-05-26 22:39 ` samba-bugs
2021-05-27 17:33 ` samba-bugs
2021-05-30 13:51 ` samba-bugs
2021-06-04  6:42 ` samba-bugs
2021-06-04 10:17 ` samba-bugs

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).