[PATCH 01/14] cifs: print server capabilities in DebugData

[PATCH 01/14] cifs: print server capabilities in DebugData

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

In the output of /proc/fs/cifs/DebugData, we do not
print the server->capabilities field today.
With this change, we will do that.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/cifs_debug.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/fs/smb/client/cifs_debug.c b/fs/smb/client/cifs_debug.c
index 76922fcc4bc6..a9dfecc397a8 100644
--- a/fs/smb/client/cifs_debug.c
+++ b/fs/smb/client/cifs_debug.c
@@ -427,6 +427,8 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
 		if (server->nosharesock)
 			seq_printf(m, " nosharesock");
 
+		seq_printf(m, "\nServer capability: 0x%x", server->capabilities);
+
 		if (server->rdma)
 			seq_printf(m, "\nRDMA ");
 		seq_printf(m, "\nTCP status: %d Instance: %d"
-- 
2.34.1

[PATCH 02/14] cifs: add xid to query server interface call

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

We were passing 0 as the xid for the call to query
server interfaces. This is not great for debugging.
This change adds a real xid.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/connect.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index 7b923e36501b..15a1c86482ed 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -119,6 +119,7 @@ static int reconn_set_ipaddr_from_hostname(struct TCP_Server_Info *server)
 static void smb2_query_server_interfaces(struct work_struct *work)
 {
 	int rc;
+	int xid;
 	struct cifs_tcon *tcon = container_of(work,
 					struct cifs_tcon,
 					query_interfaces.work);
@@ -126,7 +127,10 @@ static void smb2_query_server_interfaces(struct work_struct *work)
 	/*
 	 * query server network interfaces, in case they change
 	 */
-	rc = SMB3_request_interfaces(0, tcon, false);
+	xid = get_xid();
+	rc = SMB3_request_interfaces(xid, tcon, false);
+	free_xid(xid);
+
 	if (rc) {
 		cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
 				__func__, rc);
-- 
2.34.1

Re: [PATCH 02/14] cifs: add xid to query server interface call

[Bharath SM]

Looks good to me.

On Mon, Oct 30, 2023 at 4:30 PM <nspmangalore@gmail.com> wrote:
>
> From: Shyam Prasad N <sprasad@microsoft.com>
>
> We were passing 0 as the xid for the call to query
> server interfaces. This is not great for debugging.
> This change adds a real xid.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/connect.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
> index 7b923e36501b..15a1c86482ed 100644
> --- a/fs/smb/client/connect.c
> +++ b/fs/smb/client/connect.c
> @@ -119,6 +119,7 @@ static int reconn_set_ipaddr_from_hostname(struct TCP_Server_Info *server)
>  static void smb2_query_server_interfaces(struct work_struct *work)
>  {
>         int rc;
> +       int xid;
>         struct cifs_tcon *tcon = container_of(work,
>                                         struct cifs_tcon,
>                                         query_interfaces.work);
> @@ -126,7 +127,10 @@ static void smb2_query_server_interfaces(struct work_struct *work)
>         /*
>          * query server network interfaces, in case they change
>          */
> -       rc = SMB3_request_interfaces(0, tcon, false);
> +       xid = get_xid();
> +       rc = SMB3_request_interfaces(xid, tcon, false);
> +       free_xid(xid);
> +
>         if (rc) {
>                 cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
>                                 __func__, rc);
> --
> 2.34.1
>

[PATCH 03/14] cifs: reconnect helper should set reconnect for the right channel

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

We introduced a helper function to be used by non-cifsd threads to
mark the connection for reconnect. For multichannel, when only
a particular channel needs to be reconnected, this had a bug.

This change fixes that by marking that particular channel
for reconnect.

Fixes: dca65818c80c ("cifs: use a different reconnect helper for non-cifsd threads")
Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/connect.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index 15a1c86482ed..c993c7a3415a 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -160,13 +160,14 @@ cifs_signal_cifsd_for_reconnect(struct TCP_Server_Info *server,
 	/* If server is a channel, select the primary channel */
 	pserver = SERVER_IS_CHAN(server) ? server->primary_server : server;
 
-	spin_lock(&pserver->srv_lock);
+	/* if we need to signal just this channel */
 	if (!all_channels) {
-		pserver->tcpStatus = CifsNeedReconnect;
-		spin_unlock(&pserver->srv_lock);
+		spin_lock(&server->srv_lock);
+		if (server->tcpStatus != CifsExiting)
+			server->tcpStatus = CifsNeedReconnect;
+		spin_unlock(&server->srv_lock);
 		return;
 	}
-	spin_unlock(&pserver->srv_lock);
 
 	spin_lock(&cifs_tcp_ses_lock);
 	list_for_each_entry(ses, &pserver->smb_ses_list, smb_ses_list) {
-- 
2.34.1

Re: [PATCH 03/14] cifs: reconnect helper should set reconnect for the right channel

[Paulo Alcantara]

nspmangalore@gmail.com writes:

> From: Shyam Prasad N <sprasad@microsoft.com>
>
> We introduced a helper function to be used by non-cifsd threads to
> mark the connection for reconnect. For multichannel, when only
> a particular channel needs to be reconnected, this had a bug.
>
> This change fixes that by marking that particular channel
> for reconnect.
>
> Fixes: dca65818c80c ("cifs: use a different reconnect helper for non-cifsd threads")
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/connect.c | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)

Reviewed-by: Paulo Alcantara (SUSE) <pc@manguebit.com>

Re: [PATCH 03/14] cifs: reconnect helper should set reconnect for the right channel

[Steve French]

tentatively added to for-next pending testing

(and added Cc: stable and Paulo's RB)

On Tue, Oct 31, 2023 at 10:27 AM Paulo Alcantara <pc@manguebit.com> wrote:
>
> nspmangalore@gmail.com writes:
>
> > From: Shyam Prasad N <sprasad@microsoft.com>
> >
> > We introduced a helper function to be used by non-cifsd threads to
> > mark the connection for reconnect. For multichannel, when only
> > a particular channel needs to be reconnected, this had a bug.
> >
> > This change fixes that by marking that particular channel
> > for reconnect.
> >
> > Fixes: dca65818c80c ("cifs: use a different reconnect helper for non-cifsd threads")
> > Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> > ---
> >  fs/smb/client/connect.c | 9 +++++----
> >  1 file changed, 5 insertions(+), 4 deletions(-)
>
> Reviewed-by: Paulo Alcantara (SUSE) <pc@manguebit.com>



-- 
Thanks,

Steve

[PATCH 04/14] cifs: do not reset chan_max if multichannel is not supported at mount

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

If the mount command has specified multichannel as a mount option,
but multichannel is found to be unsupported by the server at the time
of mount, we set chan_max to 1. Which means that the user needs to
remount the share if the server starts supporting multichannel.

This change removes this reset. What it means is that if the user
specified multichannel or max_channels during mount, and at this
time, multichannel is not supported, but the server starts supporting
it at a later point, the client will be capable of scaling out the
number of channels.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/sess.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
index 79f26c560edf..c899b05c92f7 100644
--- a/fs/smb/client/sess.c
+++ b/fs/smb/client/sess.c
@@ -186,7 +186,6 @@ int cifs_try_adding_channels(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses)
 	}
 
 	if (!(server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL)) {
-		ses->chan_max = 1;
 		spin_unlock(&ses->chan_lock);
 		cifs_server_dbg(VFS, "no multichannel support\n");
 		return 0;
-- 
2.34.1

Re: [PATCH 04/14] cifs: do not reset chan_max if multichannel is not supported at mount

[Steve French]

merged into cifs-2.6.git for-next pending more testing

On Mon, Oct 30, 2023 at 6:00 AM <nspmangalore@gmail.com> wrote:
>
> From: Shyam Prasad N <sprasad@microsoft.com>
>
> If the mount command has specified multichannel as a mount option,
> but multichannel is found to be unsupported by the server at the time
> of mount, we set chan_max to 1. Which means that the user needs to
> remount the share if the server starts supporting multichannel.
>
> This change removes this reset. What it means is that if the user
> specified multichannel or max_channels during mount, and at this
> time, multichannel is not supported, but the server starts supporting
> it at a later point, the client will be capable of scaling out the
> number of channels.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/sess.c | 1 -
>  1 file changed, 1 deletion(-)
>
> diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
> index 79f26c560edf..c899b05c92f7 100644
> --- a/fs/smb/client/sess.c
> +++ b/fs/smb/client/sess.c
> @@ -186,7 +186,6 @@ int cifs_try_adding_channels(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses)
>         }
>
>         if (!(server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL)) {
> -               ses->chan_max = 1;
>                 spin_unlock(&ses->chan_lock);
>                 cifs_server_dbg(VFS, "no multichannel support\n");
>                 return 0;
> --
> 2.34.1
>


-- 
Thanks,

Steve

Re: [PATCH 04/14] cifs: do not reset chan_max if multichannel is not supported at mount

[Steve French]

merged into cifs-2.6.git for-next pending testing

And added cc:stable

On Mon, Oct 30, 2023 at 6:00 AM <nspmangalore@gmail.com> wrote:
>
> From: Shyam Prasad N <sprasad@microsoft.com>
>
> If the mount command has specified multichannel as a mount option,
> but multichannel is found to be unsupported by the server at the time
> of mount, we set chan_max to 1. Which means that the user needs to
> remount the share if the server starts supporting multichannel.
>
> This change removes this reset. What it means is that if the user
> specified multichannel or max_channels during mount, and at this
> time, multichannel is not supported, but the server starts supporting
> it at a later point, the client will be capable of scaling out the
> number of channels.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/sess.c | 1 -
>  1 file changed, 1 deletion(-)
>
> diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
> index 79f26c560edf..c899b05c92f7 100644
> --- a/fs/smb/client/sess.c
> +++ b/fs/smb/client/sess.c
> @@ -186,7 +186,6 @@ int cifs_try_adding_channels(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses)
>         }
>
>         if (!(server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL)) {
> -               ses->chan_max = 1;
>                 spin_unlock(&ses->chan_lock);
>                 cifs_server_dbg(VFS, "no multichannel support\n");
>                 return 0;
> --
> 2.34.1
>


-- 
Thanks,

Steve

[PATCH 05/14] cifs: force interface update before a fresh session setup

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

During a session reconnect, it is possible that the
server moved to another physical server (happens in case
of Azure files). So at this time, force a query of server
interfaces again (in case of multichannel session), such
that the secondary channels connect to the right
IP addresses (possibly updated now).

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/connect.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index c993c7a3415a..97c9a32cff36 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -3854,8 +3854,12 @@ cifs_setup_session(const unsigned int xid, struct cifs_ses *ses,
 	is_binding = !CIFS_ALL_CHANS_NEED_RECONNECT(ses);
 	spin_unlock(&ses->chan_lock);
 
-	if (!is_binding)
+	if (!is_binding) {
 		ses->ses_status = SES_IN_SETUP;
+
+		/* force iface_list refresh */
+		ses->iface_last_update = 0;
+	}
 	spin_unlock(&ses->ses_lock);
 
 	/* update ses ip_addr only for primary chan */
-- 
2.34.1

Re: [PATCH 05/14] cifs: force interface update before a fresh session setup

[Steve French]

merged into cifs-2.6.git for-next pending testing

And added cc:stable

On Mon, Oct 30, 2023 at 6:00 AM <nspmangalore@gmail.com> wrote:
>
> From: Shyam Prasad N <sprasad@microsoft.com>
>
> During a session reconnect, it is possible that the
> server moved to another physical server (happens in case
> of Azure files). So at this time, force a query of server
> interfaces again (in case of multichannel session), such
> that the secondary channels connect to the right
> IP addresses (possibly updated now).
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/connect.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
> index c993c7a3415a..97c9a32cff36 100644
> --- a/fs/smb/client/connect.c
> +++ b/fs/smb/client/connect.c
> @@ -3854,8 +3854,12 @@ cifs_setup_session(const unsigned int xid, struct cifs_ses *ses,
>         is_binding = !CIFS_ALL_CHANS_NEED_RECONNECT(ses);
>         spin_unlock(&ses->chan_lock);
>
> -       if (!is_binding)
> +       if (!is_binding) {
>                 ses->ses_status = SES_IN_SETUP;
> +
> +               /* force iface_list refresh */
> +               ses->iface_last_update = 0;
> +       }
>         spin_unlock(&ses->ses_lock);
>
>         /* update ses ip_addr only for primary chan */
> --
> 2.34.1
>


-- 
Thanks,

Steve

[PATCH 06/14] cifs: handle cases where a channel is closed

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

So far, SMB multichannel could only scale up, but not
scale down the number of channels. In this series of
patch, we now allow the client to deal with the case
of multichannel disabled on the server when the share
is mounted. With that change, we now need the ability
to scale down the channels.

This change allows the client to deal with cases of
missing channels more gracefully.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/cifs_debug.c    |  5 +++++
 fs/smb/client/cifsglob.h      |  1 +
 fs/smb/client/cifsproto.h     |  2 +-
 fs/smb/client/connect.c       | 10 ++++++++--
 fs/smb/client/sess.c          | 25 +++++++++++++++++++++----
 fs/smb/client/smb2transport.c |  8 +++++++-
 6 files changed, 43 insertions(+), 8 deletions(-)

diff --git a/fs/smb/client/cifs_debug.c b/fs/smb/client/cifs_debug.c
index a9dfecc397a8..9fca09539728 100644
--- a/fs/smb/client/cifs_debug.c
+++ b/fs/smb/client/cifs_debug.c
@@ -136,6 +136,11 @@ cifs_dump_channel(struct seq_file *m, int i, struct cifs_chan *chan)
 {
 	struct TCP_Server_Info *server = chan->server;
 
+	if (!server) {
+		seq_printf(m, "\n\n\t\tChannel: %d DISABLED", i+1);
+		return;
+	}
+
 	seq_printf(m, "\n\n\t\tChannel: %d ConnectionId: 0x%llx"
 		   "\n\t\tNumber of credits: %d,%d,%d Dialect 0x%x"
 		   "\n\t\tTCP status: %d Instance: %d"
diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
index 02082621d8e0..552ed441281a 100644
--- a/fs/smb/client/cifsglob.h
+++ b/fs/smb/client/cifsglob.h
@@ -1050,6 +1050,7 @@ struct cifs_ses {
 	spinlock_t chan_lock;
 	/* ========= begin: protected by chan_lock ======== */
 #define CIFS_MAX_CHANNELS 16
+#define CIFS_INVAL_CHAN_INDEX (-1)
 #define CIFS_ALL_CHANNELS_SET(ses)	\
 	((1UL << (ses)->chan_count) - 1)
 #define CIFS_ALL_CHANS_GOOD(ses)		\
diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
index 0c37eefa18a5..65c84b3d1a65 100644
--- a/fs/smb/client/cifsproto.h
+++ b/fs/smb/client/cifsproto.h
@@ -616,7 +616,7 @@ bool is_server_using_iface(struct TCP_Server_Info *server,
 bool is_ses_using_iface(struct cifs_ses *ses, struct cifs_server_iface *iface);
 void cifs_ses_mark_for_reconnect(struct cifs_ses *ses);
 
-unsigned int
+int
 cifs_ses_get_chan_index(struct cifs_ses *ses,
 			struct TCP_Server_Info *server);
 void
diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index 97c9a32cff36..8393977e21ee 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -173,8 +173,12 @@ cifs_signal_cifsd_for_reconnect(struct TCP_Server_Info *server,
 	list_for_each_entry(ses, &pserver->smb_ses_list, smb_ses_list) {
 		spin_lock(&ses->chan_lock);
 		for (i = 0; i < ses->chan_count; i++) {
+			if (!ses->chans[i].server)
+				continue;
+
 			spin_lock(&ses->chans[i].server->srv_lock);
-			ses->chans[i].server->tcpStatus = CifsNeedReconnect;
+			if (ses->chans[i].server->tcpStatus != CifsExiting)
+				ses->chans[i].server->tcpStatus = CifsNeedReconnect;
 			spin_unlock(&ses->chans[i].server->srv_lock);
 		}
 		spin_unlock(&ses->chan_lock);
@@ -2033,7 +2037,9 @@ void __cifs_put_smb_ses(struct cifs_ses *ses)
 				kref_put(&ses->chans[i].iface->refcount, release_iface);
 				ses->chans[i].iface = NULL;
 			}
-			cifs_put_tcp_session(ses->chans[i].server, 0);
+
+			if (ses->chans[i].server)
+				cifs_put_tcp_session(ses->chans[i].server, 0);
 			ses->chans[i].server = NULL;
 		}
 	}
diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
index c899b05c92f7..9d2228c2d7e5 100644
--- a/fs/smb/client/sess.c
+++ b/fs/smb/client/sess.c
@@ -69,7 +69,7 @@ bool is_ses_using_iface(struct cifs_ses *ses, struct cifs_server_iface *iface)
 
 /* channel helper functions. assumed that chan_lock is held by caller. */
 
-unsigned int
+int
 cifs_ses_get_chan_index(struct cifs_ses *ses,
 			struct TCP_Server_Info *server)
 {
@@ -85,14 +85,16 @@ cifs_ses_get_chan_index(struct cifs_ses *ses,
 		cifs_dbg(VFS, "unable to get chan index for server: 0x%llx",
 			 server->conn_id);
 	WARN_ON(1);
-	return 0;
+	return CIFS_INVAL_CHAN_INDEX;
 }
 
 void
 cifs_chan_set_in_reconnect(struct cifs_ses *ses,
 			     struct TCP_Server_Info *server)
 {
-	unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
+	int chan_index = cifs_ses_get_chan_index(ses, server);
+	if (chan_index == CIFS_INVAL_CHAN_INDEX)
+		return;
 
 	ses->chans[chan_index].in_reconnect = true;
 }
@@ -102,6 +104,8 @@ cifs_chan_clear_in_reconnect(struct cifs_ses *ses,
 			     struct TCP_Server_Info *server)
 {
 	unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
+	if (chan_index == CIFS_INVAL_CHAN_INDEX)
+		return;
 
 	ses->chans[chan_index].in_reconnect = false;
 }
@@ -111,6 +115,8 @@ cifs_chan_in_reconnect(struct cifs_ses *ses,
 			  struct TCP_Server_Info *server)
 {
 	unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
+	if (chan_index == CIFS_INVAL_CHAN_INDEX)
+		return true;	/* err on the safer side */
 
 	return CIFS_CHAN_IN_RECONNECT(ses, chan_index);
 }
@@ -120,6 +126,8 @@ cifs_chan_set_need_reconnect(struct cifs_ses *ses,
 			     struct TCP_Server_Info *server)
 {
 	unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
+	if (chan_index == CIFS_INVAL_CHAN_INDEX)
+		return;
 
 	set_bit(chan_index, &ses->chans_need_reconnect);
 	cifs_dbg(FYI, "Set reconnect bitmask for chan %u; now 0x%lx\n",
@@ -131,6 +139,8 @@ cifs_chan_clear_need_reconnect(struct cifs_ses *ses,
 			       struct TCP_Server_Info *server)
 {
 	unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
+	if (chan_index == CIFS_INVAL_CHAN_INDEX)
+		return;
 
 	clear_bit(chan_index, &ses->chans_need_reconnect);
 	cifs_dbg(FYI, "Cleared reconnect bitmask for chan %u; now 0x%lx\n",
@@ -142,6 +152,8 @@ cifs_chan_needs_reconnect(struct cifs_ses *ses,
 			  struct TCP_Server_Info *server)
 {
 	unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
+	if (chan_index == CIFS_INVAL_CHAN_INDEX)
+		return true;	/* err on the safer side */
 
 	return CIFS_CHAN_NEEDS_RECONNECT(ses, chan_index);
 }
@@ -151,6 +163,8 @@ cifs_chan_is_iface_active(struct cifs_ses *ses,
 			  struct TCP_Server_Info *server)
 {
 	unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
+	if (chan_index == CIFS_INVAL_CHAN_INDEX)
+		return true;	/* err on the safer side */
 
 	return ses->chans[chan_index].iface &&
 		ses->chans[chan_index].iface->is_active;
@@ -269,7 +283,7 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 
 	spin_lock(&ses->chan_lock);
 	chan_index = cifs_ses_get_chan_index(ses, server);
-	if (!chan_index) {
+	if (chan_index == CIFS_INVAL_CHAN_INDEX) {
 		spin_unlock(&ses->chan_lock);
 		return 0;
 	}
@@ -319,6 +333,9 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 
 	spin_lock(&ses->chan_lock);
 	chan_index = cifs_ses_get_chan_index(ses, server);
+	if (chan_index == CIFS_INVAL_CHAN_INDEX)
+		return 0;
+
 	ses->chans[chan_index].iface = iface;
 
 	/* No iface is found. if secondary chan, drop connection */
diff --git a/fs/smb/client/smb2transport.c b/fs/smb/client/smb2transport.c
index 23c50ed7d4b5..84ea67301303 100644
--- a/fs/smb/client/smb2transport.c
+++ b/fs/smb/client/smb2transport.c
@@ -413,7 +413,13 @@ generate_smb3signingkey(struct cifs_ses *ses,
 		      ses->ses_status == SES_GOOD);
 
 	chan_index = cifs_ses_get_chan_index(ses, server);
-	/* TODO: introduce ref counting for channels when the can be freed */
+	if (chan_index == CIFS_INVAL_CHAN_INDEX) {
+		spin_unlock(&ses->chan_lock);
+		spin_unlock(&ses->ses_lock);
+
+		return -EINVAL;
+	}
+
 	spin_unlock(&ses->chan_lock);
 	spin_unlock(&ses->ses_lock);
 
-- 
2.34.1

Re: [PATCH 06/14] cifs: handle cases where a channel is closed

[Steve French]

This patch wouldn't merge due to a merge conflict here.  Am I missing
an earlier patch of yours before the series?

I have these in for-next:
d1ea8f36fa51 (HEAD -> for-next) cifs: force interface update before a
fresh session setup
1e27cedcaf4e (origin/for-next) cifs: do not reset chan_max if
multichannel is not supported at mount
e257df806ae0 cifs: display the endpoint IP details in DebugData
9d95130c9f78 cifs: reconnect helper should set reconnect for the right channel
7ac6866076bd smb: client: fix use-after-free in smb2_query_info_compound()
0779365fc10d smb: client: remove extra @chan_count check in __cifs_put_smb_ses()
4cf6e1101a25 cifs: add xid to query server interface call
52768695d36a cifs: print server capabilities in DebugData


--- fs/smb/client/connect.c
+++ fs/smb/client/connect.c
@@ -2037,7 +2041,9 @@ void __cifs_put_smb_ses(struct cifs_ses *ses)

kref_put(&ses->chans[i].iface->refcount, release_iface);
                                ses->chans[i].iface = NULL;
                        }
-                       cifs_put_tcp_session(ses->chans[i].server, 0);
+
+                       if (ses->chans[i].server)
+                               cifs_put_tcp_session(ses->chans[i].server, 0);
                        ses->chans[i].server = NULL;
                }
        }

On Mon, Oct 30, 2023 at 6:00 AM <nspmangalore@gmail.com> wrote:
>
> From: Shyam Prasad N <sprasad@microsoft.com>
>
> So far, SMB multichannel could only scale up, but not
> scale down the number of channels. In this series of
> patch, we now allow the client to deal with the case
> of multichannel disabled on the server when the share
> is mounted. With that change, we now need the ability
> to scale down the channels.
>
> This change allows the client to deal with cases of
> missing channels more gracefully.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/cifs_debug.c    |  5 +++++
>  fs/smb/client/cifsglob.h      |  1 +
>  fs/smb/client/cifsproto.h     |  2 +-
>  fs/smb/client/connect.c       | 10 ++++++++--
>  fs/smb/client/sess.c          | 25 +++++++++++++++++++++----
>  fs/smb/client/smb2transport.c |  8 +++++++-
>  6 files changed, 43 insertions(+), 8 deletions(-)
>
> diff --git a/fs/smb/client/cifs_debug.c b/fs/smb/client/cifs_debug.c
> index a9dfecc397a8..9fca09539728 100644
> --- a/fs/smb/client/cifs_debug.c
> +++ b/fs/smb/client/cifs_debug.c
> @@ -136,6 +136,11 @@ cifs_dump_channel(struct seq_file *m, int i, struct cifs_chan *chan)
>  {
>         struct TCP_Server_Info *server = chan->server;
>
> +       if (!server) {
> +               seq_printf(m, "\n\n\t\tChannel: %d DISABLED", i+1);
> +               return;
> +       }
> +
>         seq_printf(m, "\n\n\t\tChannel: %d ConnectionId: 0x%llx"
>                    "\n\t\tNumber of credits: %d,%d,%d Dialect 0x%x"
>                    "\n\t\tTCP status: %d Instance: %d"
> diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
> index 02082621d8e0..552ed441281a 100644
> --- a/fs/smb/client/cifsglob.h
> +++ b/fs/smb/client/cifsglob.h
> @@ -1050,6 +1050,7 @@ struct cifs_ses {
>         spinlock_t chan_lock;
>         /* ========= begin: protected by chan_lock ======== */
>  #define CIFS_MAX_CHANNELS 16
> +#define CIFS_INVAL_CHAN_INDEX (-1)
>  #define CIFS_ALL_CHANNELS_SET(ses)     \
>         ((1UL << (ses)->chan_count) - 1)
>  #define CIFS_ALL_CHANS_GOOD(ses)               \
> diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
> index 0c37eefa18a5..65c84b3d1a65 100644
> --- a/fs/smb/client/cifsproto.h
> +++ b/fs/smb/client/cifsproto.h
> @@ -616,7 +616,7 @@ bool is_server_using_iface(struct TCP_Server_Info *server,
>  bool is_ses_using_iface(struct cifs_ses *ses, struct cifs_server_iface *iface);
>  void cifs_ses_mark_for_reconnect(struct cifs_ses *ses);
>
> -unsigned int
> +int
>  cifs_ses_get_chan_index(struct cifs_ses *ses,
>                         struct TCP_Server_Info *server);
>  void
> diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
> index 97c9a32cff36..8393977e21ee 100644
> --- a/fs/smb/client/connect.c
> +++ b/fs/smb/client/connect.c
> @@ -173,8 +173,12 @@ cifs_signal_cifsd_for_reconnect(struct TCP_Server_Info *server,
>         list_for_each_entry(ses, &pserver->smb_ses_list, smb_ses_list) {
>                 spin_lock(&ses->chan_lock);
>                 for (i = 0; i < ses->chan_count; i++) {
> +                       if (!ses->chans[i].server)
> +                               continue;
> +
>                         spin_lock(&ses->chans[i].server->srv_lock);
> -                       ses->chans[i].server->tcpStatus = CifsNeedReconnect;
> +                       if (ses->chans[i].server->tcpStatus != CifsExiting)
> +                               ses->chans[i].server->tcpStatus = CifsNeedReconnect;
>                         spin_unlock(&ses->chans[i].server->srv_lock);
>                 }
>                 spin_unlock(&ses->chan_lock);
> @@ -2033,7 +2037,9 @@ void __cifs_put_smb_ses(struct cifs_ses *ses)
>                                 kref_put(&ses->chans[i].iface->refcount, release_iface);
>                                 ses->chans[i].iface = NULL;
>                         }
> -                       cifs_put_tcp_session(ses->chans[i].server, 0);
> +
> +                       if (ses->chans[i].server)
> +                               cifs_put_tcp_session(ses->chans[i].server, 0);
>                         ses->chans[i].server = NULL;
>                 }
>         }
> diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
> index c899b05c92f7..9d2228c2d7e5 100644
> --- a/fs/smb/client/sess.c
> +++ b/fs/smb/client/sess.c
> @@ -69,7 +69,7 @@ bool is_ses_using_iface(struct cifs_ses *ses, struct cifs_server_iface *iface)
>
>  /* channel helper functions. assumed that chan_lock is held by caller. */
>
> -unsigned int
> +int
>  cifs_ses_get_chan_index(struct cifs_ses *ses,
>                         struct TCP_Server_Info *server)
>  {
> @@ -85,14 +85,16 @@ cifs_ses_get_chan_index(struct cifs_ses *ses,
>                 cifs_dbg(VFS, "unable to get chan index for server: 0x%llx",
>                          server->conn_id);
>         WARN_ON(1);
> -       return 0;
> +       return CIFS_INVAL_CHAN_INDEX;
>  }
>
>  void
>  cifs_chan_set_in_reconnect(struct cifs_ses *ses,
>                              struct TCP_Server_Info *server)
>  {
> -       unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> +       int chan_index = cifs_ses_get_chan_index(ses, server);
> +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> +               return;
>
>         ses->chans[chan_index].in_reconnect = true;
>  }
> @@ -102,6 +104,8 @@ cifs_chan_clear_in_reconnect(struct cifs_ses *ses,
>                              struct TCP_Server_Info *server)
>  {
>         unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> +               return;
>
>         ses->chans[chan_index].in_reconnect = false;
>  }
> @@ -111,6 +115,8 @@ cifs_chan_in_reconnect(struct cifs_ses *ses,
>                           struct TCP_Server_Info *server)
>  {
>         unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> +               return true;    /* err on the safer side */
>
>         return CIFS_CHAN_IN_RECONNECT(ses, chan_index);
>  }
> @@ -120,6 +126,8 @@ cifs_chan_set_need_reconnect(struct cifs_ses *ses,
>                              struct TCP_Server_Info *server)
>  {
>         unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> +               return;
>
>         set_bit(chan_index, &ses->chans_need_reconnect);
>         cifs_dbg(FYI, "Set reconnect bitmask for chan %u; now 0x%lx\n",
> @@ -131,6 +139,8 @@ cifs_chan_clear_need_reconnect(struct cifs_ses *ses,
>                                struct TCP_Server_Info *server)
>  {
>         unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> +               return;
>
>         clear_bit(chan_index, &ses->chans_need_reconnect);
>         cifs_dbg(FYI, "Cleared reconnect bitmask for chan %u; now 0x%lx\n",
> @@ -142,6 +152,8 @@ cifs_chan_needs_reconnect(struct cifs_ses *ses,
>                           struct TCP_Server_Info *server)
>  {
>         unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> +               return true;    /* err on the safer side */
>
>         return CIFS_CHAN_NEEDS_RECONNECT(ses, chan_index);
>  }
> @@ -151,6 +163,8 @@ cifs_chan_is_iface_active(struct cifs_ses *ses,
>                           struct TCP_Server_Info *server)
>  {
>         unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> +               return true;    /* err on the safer side */
>
>         return ses->chans[chan_index].iface &&
>                 ses->chans[chan_index].iface->is_active;
> @@ -269,7 +283,7 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
>
>         spin_lock(&ses->chan_lock);
>         chan_index = cifs_ses_get_chan_index(ses, server);
> -       if (!chan_index) {
> +       if (chan_index == CIFS_INVAL_CHAN_INDEX) {
>                 spin_unlock(&ses->chan_lock);
>                 return 0;
>         }
> @@ -319,6 +333,9 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
>
>         spin_lock(&ses->chan_lock);
>         chan_index = cifs_ses_get_chan_index(ses, server);
> +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> +               return 0;
> +
>         ses->chans[chan_index].iface = iface;
>
>         /* No iface is found. if secondary chan, drop connection */
> diff --git a/fs/smb/client/smb2transport.c b/fs/smb/client/smb2transport.c
> index 23c50ed7d4b5..84ea67301303 100644
> --- a/fs/smb/client/smb2transport.c
> +++ b/fs/smb/client/smb2transport.c
> @@ -413,7 +413,13 @@ generate_smb3signingkey(struct cifs_ses *ses,
>                       ses->ses_status == SES_GOOD);
>
>         chan_index = cifs_ses_get_chan_index(ses, server);
> -       /* TODO: introduce ref counting for channels when the can be freed */
> +       if (chan_index == CIFS_INVAL_CHAN_INDEX) {
> +               spin_unlock(&ses->chan_lock);
> +               spin_unlock(&ses->ses_lock);
> +
> +               return -EINVAL;
> +       }
> +
>         spin_unlock(&ses->chan_lock);
>         spin_unlock(&ses->ses_lock);
>
> --
> 2.34.1
>


-- 
Thanks,

Steve

Re: [PATCH 06/14] cifs: handle cases where a channel is closed

[Shyam Prasad N]

On Wed, Nov 1, 2023 at 8:39 AM Steve French <smfrench@gmail.com> wrote:
>
> This patch wouldn't merge due to a merge conflict here.  Am I missing
> an earlier patch of yours before the series?
>

I think the conflict is with:
0779365fc10d smb: client: remove extra @chan_count check in __cifs_put_smb_ses()

Let me resolve the conflict and give you an updated patch.

> I have these in for-next:
> d1ea8f36fa51 (HEAD -> for-next) cifs: force interface update before a
> fresh session setup
> 1e27cedcaf4e (origin/for-next) cifs: do not reset chan_max if
> multichannel is not supported at mount
> e257df806ae0 cifs: display the endpoint IP details in DebugData
> 9d95130c9f78 cifs: reconnect helper should set reconnect for the right channel
> 7ac6866076bd smb: client: fix use-after-free in smb2_query_info_compound()
> 0779365fc10d smb: client: remove extra @chan_count check in __cifs_put_smb_ses()
> 4cf6e1101a25 cifs: add xid to query server interface call
> 52768695d36a cifs: print server capabilities in DebugData
>
>
> --- fs/smb/client/connect.c
> +++ fs/smb/client/connect.c
> @@ -2037,7 +2041,9 @@ void __cifs_put_smb_ses(struct cifs_ses *ses)
>
> kref_put(&ses->chans[i].iface->refcount, release_iface);
>                                 ses->chans[i].iface = NULL;
>                         }
> -                       cifs_put_tcp_session(ses->chans[i].server, 0);
> +
> +                       if (ses->chans[i].server)
> +                               cifs_put_tcp_session(ses->chans[i].server, 0);
>                         ses->chans[i].server = NULL;
>                 }
>         }
>
> On Mon, Oct 30, 2023 at 6:00 AM <nspmangalore@gmail.com> wrote:
> >
> > From: Shyam Prasad N <sprasad@microsoft.com>
> >
> > So far, SMB multichannel could only scale up, but not
> > scale down the number of channels. In this series of
> > patch, we now allow the client to deal with the case
> > of multichannel disabled on the server when the share
> > is mounted. With that change, we now need the ability
> > to scale down the channels.
> >
> > This change allows the client to deal with cases of
> > missing channels more gracefully.
> >
> > Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> > ---
> >  fs/smb/client/cifs_debug.c    |  5 +++++
> >  fs/smb/client/cifsglob.h      |  1 +
> >  fs/smb/client/cifsproto.h     |  2 +-
> >  fs/smb/client/connect.c       | 10 ++++++++--
> >  fs/smb/client/sess.c          | 25 +++++++++++++++++++++----
> >  fs/smb/client/smb2transport.c |  8 +++++++-
> >  6 files changed, 43 insertions(+), 8 deletions(-)
> >
> > diff --git a/fs/smb/client/cifs_debug.c b/fs/smb/client/cifs_debug.c
> > index a9dfecc397a8..9fca09539728 100644
> > --- a/fs/smb/client/cifs_debug.c
> > +++ b/fs/smb/client/cifs_debug.c
> > @@ -136,6 +136,11 @@ cifs_dump_channel(struct seq_file *m, int i, struct cifs_chan *chan)
> >  {
> >         struct TCP_Server_Info *server = chan->server;
> >
> > +       if (!server) {
> > +               seq_printf(m, "\n\n\t\tChannel: %d DISABLED", i+1);
> > +               return;
> > +       }
> > +
> >         seq_printf(m, "\n\n\t\tChannel: %d ConnectionId: 0x%llx"
> >                    "\n\t\tNumber of credits: %d,%d,%d Dialect 0x%x"
> >                    "\n\t\tTCP status: %d Instance: %d"
> > diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
> > index 02082621d8e0..552ed441281a 100644
> > --- a/fs/smb/client/cifsglob.h
> > +++ b/fs/smb/client/cifsglob.h
> > @@ -1050,6 +1050,7 @@ struct cifs_ses {
> >         spinlock_t chan_lock;
> >         /* ========= begin: protected by chan_lock ======== */
> >  #define CIFS_MAX_CHANNELS 16
> > +#define CIFS_INVAL_CHAN_INDEX (-1)
> >  #define CIFS_ALL_CHANNELS_SET(ses)     \
> >         ((1UL << (ses)->chan_count) - 1)
> >  #define CIFS_ALL_CHANS_GOOD(ses)               \
> > diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
> > index 0c37eefa18a5..65c84b3d1a65 100644
> > --- a/fs/smb/client/cifsproto.h
> > +++ b/fs/smb/client/cifsproto.h
> > @@ -616,7 +616,7 @@ bool is_server_using_iface(struct TCP_Server_Info *server,
> >  bool is_ses_using_iface(struct cifs_ses *ses, struct cifs_server_iface *iface);
> >  void cifs_ses_mark_for_reconnect(struct cifs_ses *ses);
> >
> > -unsigned int
> > +int
> >  cifs_ses_get_chan_index(struct cifs_ses *ses,
> >                         struct TCP_Server_Info *server);
> >  void
> > diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
> > index 97c9a32cff36..8393977e21ee 100644
> > --- a/fs/smb/client/connect.c
> > +++ b/fs/smb/client/connect.c
> > @@ -173,8 +173,12 @@ cifs_signal_cifsd_for_reconnect(struct TCP_Server_Info *server,
> >         list_for_each_entry(ses, &pserver->smb_ses_list, smb_ses_list) {
> >                 spin_lock(&ses->chan_lock);
> >                 for (i = 0; i < ses->chan_count; i++) {
> > +                       if (!ses->chans[i].server)
> > +                               continue;
> > +
> >                         spin_lock(&ses->chans[i].server->srv_lock);
> > -                       ses->chans[i].server->tcpStatus = CifsNeedReconnect;
> > +                       if (ses->chans[i].server->tcpStatus != CifsExiting)
> > +                               ses->chans[i].server->tcpStatus = CifsNeedReconnect;
> >                         spin_unlock(&ses->chans[i].server->srv_lock);
> >                 }
> >                 spin_unlock(&ses->chan_lock);
> > @@ -2033,7 +2037,9 @@ void __cifs_put_smb_ses(struct cifs_ses *ses)
> >                                 kref_put(&ses->chans[i].iface->refcount, release_iface);
> >                                 ses->chans[i].iface = NULL;
> >                         }
> > -                       cifs_put_tcp_session(ses->chans[i].server, 0);
> > +
> > +                       if (ses->chans[i].server)
> > +                               cifs_put_tcp_session(ses->chans[i].server, 0);
> >                         ses->chans[i].server = NULL;
> >                 }
> >         }
> > diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
> > index c899b05c92f7..9d2228c2d7e5 100644
> > --- a/fs/smb/client/sess.c
> > +++ b/fs/smb/client/sess.c
> > @@ -69,7 +69,7 @@ bool is_ses_using_iface(struct cifs_ses *ses, struct cifs_server_iface *iface)
> >
> >  /* channel helper functions. assumed that chan_lock is held by caller. */
> >
> > -unsigned int
> > +int
> >  cifs_ses_get_chan_index(struct cifs_ses *ses,
> >                         struct TCP_Server_Info *server)
> >  {
> > @@ -85,14 +85,16 @@ cifs_ses_get_chan_index(struct cifs_ses *ses,
> >                 cifs_dbg(VFS, "unable to get chan index for server: 0x%llx",
> >                          server->conn_id);
> >         WARN_ON(1);
> > -       return 0;
> > +       return CIFS_INVAL_CHAN_INDEX;
> >  }
> >
> >  void
> >  cifs_chan_set_in_reconnect(struct cifs_ses *ses,
> >                              struct TCP_Server_Info *server)
> >  {
> > -       unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> > +       int chan_index = cifs_ses_get_chan_index(ses, server);
> > +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> > +               return;
> >
> >         ses->chans[chan_index].in_reconnect = true;
> >  }
> > @@ -102,6 +104,8 @@ cifs_chan_clear_in_reconnect(struct cifs_ses *ses,
> >                              struct TCP_Server_Info *server)
> >  {
> >         unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> > +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> > +               return;
> >
> >         ses->chans[chan_index].in_reconnect = false;
> >  }
> > @@ -111,6 +115,8 @@ cifs_chan_in_reconnect(struct cifs_ses *ses,
> >                           struct TCP_Server_Info *server)
> >  {
> >         unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> > +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> > +               return true;    /* err on the safer side */
> >
> >         return CIFS_CHAN_IN_RECONNECT(ses, chan_index);
> >  }
> > @@ -120,6 +126,8 @@ cifs_chan_set_need_reconnect(struct cifs_ses *ses,
> >                              struct TCP_Server_Info *server)
> >  {
> >         unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> > +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> > +               return;
> >
> >         set_bit(chan_index, &ses->chans_need_reconnect);
> >         cifs_dbg(FYI, "Set reconnect bitmask for chan %u; now 0x%lx\n",
> > @@ -131,6 +139,8 @@ cifs_chan_clear_need_reconnect(struct cifs_ses *ses,
> >                                struct TCP_Server_Info *server)
> >  {
> >         unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> > +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> > +               return;
> >
> >         clear_bit(chan_index, &ses->chans_need_reconnect);
> >         cifs_dbg(FYI, "Cleared reconnect bitmask for chan %u; now 0x%lx\n",
> > @@ -142,6 +152,8 @@ cifs_chan_needs_reconnect(struct cifs_ses *ses,
> >                           struct TCP_Server_Info *server)
> >  {
> >         unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> > +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> > +               return true;    /* err on the safer side */
> >
> >         return CIFS_CHAN_NEEDS_RECONNECT(ses, chan_index);
> >  }
> > @@ -151,6 +163,8 @@ cifs_chan_is_iface_active(struct cifs_ses *ses,
> >                           struct TCP_Server_Info *server)
> >  {
> >         unsigned int chan_index = cifs_ses_get_chan_index(ses, server);
> > +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> > +               return true;    /* err on the safer side */
> >
> >         return ses->chans[chan_index].iface &&
> >                 ses->chans[chan_index].iface->is_active;
> > @@ -269,7 +283,7 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
> >
> >         spin_lock(&ses->chan_lock);
> >         chan_index = cifs_ses_get_chan_index(ses, server);
> > -       if (!chan_index) {
> > +       if (chan_index == CIFS_INVAL_CHAN_INDEX) {
> >                 spin_unlock(&ses->chan_lock);
> >                 return 0;
> >         }
> > @@ -319,6 +333,9 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
> >
> >         spin_lock(&ses->chan_lock);
> >         chan_index = cifs_ses_get_chan_index(ses, server);
> > +       if (chan_index == CIFS_INVAL_CHAN_INDEX)
> > +               return 0;
> > +
> >         ses->chans[chan_index].iface = iface;
> >
> >         /* No iface is found. if secondary chan, drop connection */
> > diff --git a/fs/smb/client/smb2transport.c b/fs/smb/client/smb2transport.c
> > index 23c50ed7d4b5..84ea67301303 100644
> > --- a/fs/smb/client/smb2transport.c
> > +++ b/fs/smb/client/smb2transport.c
> > @@ -413,7 +413,13 @@ generate_smb3signingkey(struct cifs_ses *ses,
> >                       ses->ses_status == SES_GOOD);
> >
> >         chan_index = cifs_ses_get_chan_index(ses, server);
> > -       /* TODO: introduce ref counting for channels when the can be freed */
> > +       if (chan_index == CIFS_INVAL_CHAN_INDEX) {
> > +               spin_unlock(&ses->chan_lock);
> > +               spin_unlock(&ses->ses_lock);
> > +
> > +               return -EINVAL;
> > +       }
> > +
> >         spin_unlock(&ses->chan_lock);
> >         spin_unlock(&ses->ses_lock);
> >
> > --
> > 2.34.1
> >
>
>
> --
> Thanks,
>
> Steve



-- 
Regards,
Shyam

[PATCH 07/14] cifs: distribute channels across interfaces based on speed

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

Today, if the server interfaces RSS capable, we simply
choose the fastest interface to setup a channel. This is not
a scalable approach, and does not make a lot of attempt to
distribute the connections.

This change does a weighted distribution of channels across
all the available server interfaces, where the weight is
a function of the advertised interface speed.

Also make sure that we don't mix rdma and non-rdma for channels.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/cifs_debug.c | 16 ++++++++
 fs/smb/client/cifsglob.h   |  2 +
 fs/smb/client/sess.c       | 83 +++++++++++++++++++++++++++++++-------
 3 files changed, 87 insertions(+), 14 deletions(-)

diff --git a/fs/smb/client/cifs_debug.c b/fs/smb/client/cifs_debug.c
index 9fca09539728..e23fcabb78d6 100644
--- a/fs/smb/client/cifs_debug.c
+++ b/fs/smb/client/cifs_debug.c
@@ -284,6 +284,8 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
 	struct cifs_ses *ses;
 	struct cifs_tcon *tcon;
 	struct cifs_server_iface *iface;
+	size_t iface_weight = 0, iface_min_speed = 0;
+	struct cifs_server_iface *last_iface = NULL;
 	int c, i, j;
 
 	seq_puts(m,
@@ -543,11 +545,25 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
 					   "\tLast updated: %lu seconds ago",
 					   ses->iface_count,
 					   (jiffies - ses->iface_last_update) / HZ);
+
+			last_iface = list_last_entry(&ses->iface_list,
+						     struct cifs_server_iface,
+						     iface_head);
+			iface_min_speed = last_iface->speed;
+
 			j = 0;
 			list_for_each_entry(iface, &ses->iface_list,
 						 iface_head) {
 				seq_printf(m, "\n\t%d)", ++j);
 				cifs_dump_iface(m, iface);
+
+				iface_weight = iface->speed / iface_min_speed;
+				seq_printf(m, "\t\tWeight (cur,total): (%zu,%zu)"
+					   "\n\t\tAllocated channels: %u\n",
+					   iface->weight_fulfilled,
+					   iface_weight,
+					   iface->num_channels);
+
 				if (is_ses_using_iface(ses, iface))
 					seq_puts(m, "\t\t[CONNECTED]\n");
 			}
diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
index 552ed441281a..81e7a45f413d 100644
--- a/fs/smb/client/cifsglob.h
+++ b/fs/smb/client/cifsglob.h
@@ -969,6 +969,8 @@ struct cifs_server_iface {
 	struct list_head iface_head;
 	struct kref refcount;
 	size_t speed;
+	size_t weight_fulfilled;
+	unsigned int num_channels;
 	unsigned int rdma_capable : 1;
 	unsigned int rss_capable : 1;
 	unsigned int is_active : 1; /* unset if non existent */
diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
index 9d2228c2d7e5..d009994f82cf 100644
--- a/fs/smb/client/sess.c
+++ b/fs/smb/client/sess.c
@@ -178,7 +178,9 @@ int cifs_try_adding_channels(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses)
 	int left;
 	int rc = 0;
 	int tries = 0;
+	size_t iface_weight = 0, iface_min_speed = 0;
 	struct cifs_server_iface *iface = NULL, *niface = NULL;
+	struct cifs_server_iface *last_iface = NULL;
 
 	spin_lock(&ses->chan_lock);
 
@@ -206,21 +208,11 @@ int cifs_try_adding_channels(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses)
 	}
 	spin_unlock(&ses->chan_lock);
 
-	/*
-	 * Keep connecting to same, fastest, iface for all channels as
-	 * long as its RSS. Try next fastest one if not RSS or channel
-	 * creation fails.
-	 */
-	spin_lock(&ses->iface_lock);
-	iface = list_first_entry(&ses->iface_list, struct cifs_server_iface,
-				 iface_head);
-	spin_unlock(&ses->iface_lock);
-
 	while (left > 0) {
 
 		tries++;
 		if (tries > 3*ses->chan_max) {
-			cifs_dbg(FYI, "too many channel open attempts (%d channels left to open)\n",
+			cifs_dbg(VFS, "too many channel open attempts (%d channels left to open)\n",
 				 left);
 			break;
 		}
@@ -228,17 +220,34 @@ int cifs_try_adding_channels(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses)
 		spin_lock(&ses->iface_lock);
 		if (!ses->iface_count) {
 			spin_unlock(&ses->iface_lock);
+			cifs_dbg(VFS, "server %s does not advertise interfaces\n", ses->server->hostname);
 			break;
 		}
 
+		if (!iface)
+			iface = list_first_entry(&ses->iface_list, struct cifs_server_iface,
+						 iface_head);
+		last_iface = list_last_entry(&ses->iface_list, struct cifs_server_iface,
+					     iface_head);
+		iface_min_speed = last_iface->speed;
+
 		list_for_each_entry_safe_from(iface, niface, &ses->iface_list,
 				    iface_head) {
+			/* do not mix rdma and non-rdma interfaces */
+			if (iface->rdma_capable != ses->server->rdma)
+				continue;
+
 			/* skip ifaces that are unusable */
 			if (!iface->is_active ||
 			    (is_ses_using_iface(ses, iface) &&
-			     !iface->rss_capable)) {
+			     !iface->rss_capable))
+				continue;
+
+			/* check if we already allocated enough channels */
+			iface_weight = iface->speed / iface_min_speed;
+
+			if (iface->weight_fulfilled >= iface_weight)
 				continue;
-			}
 
 			/* take ref before unlock */
 			kref_get(&iface->refcount);
@@ -255,10 +264,21 @@ int cifs_try_adding_channels(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses)
 				continue;
 			}
 
-			cifs_dbg(FYI, "successfully opened new channel on iface:%pIS\n",
+			iface->num_channels++;
+			iface->weight_fulfilled++;
+			cifs_dbg(VFS, "successfully opened new channel on iface:%pIS\n",
 				 &iface->sockaddr);
 			break;
 		}
+
+		/* reached end of list. reset weight_fulfilled and start over */
+		if (list_entry_is_head(iface, &ses->iface_list, iface_head)) {
+			list_for_each_entry(iface, &ses->iface_list, iface_head)
+				iface->weight_fulfilled = 0;
+			spin_unlock(&ses->iface_lock);
+			iface = NULL;
+			continue;
+		}
 		spin_unlock(&ses->iface_lock);
 
 		left--;
@@ -277,8 +297,10 @@ int
 cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 {
 	unsigned int chan_index;
+	size_t iface_weight = 0, iface_min_speed = 0;
 	struct cifs_server_iface *iface = NULL;
 	struct cifs_server_iface *old_iface = NULL;
+	struct cifs_server_iface *last_iface = NULL;
 	int rc = 0;
 
 	spin_lock(&ses->chan_lock);
@@ -298,13 +320,34 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 	spin_unlock(&ses->chan_lock);
 
 	spin_lock(&ses->iface_lock);
+	if (!ses->iface_count) {
+		spin_unlock(&ses->iface_lock);
+		cifs_dbg(VFS, "server %s does not advertise interfaces\n", ses->server->hostname);
+		return 0;
+	}
+
+	last_iface = list_last_entry(&ses->iface_list, struct cifs_server_iface,
+				     iface_head);
+	iface_min_speed = last_iface->speed;
+
 	/* then look for a new one */
 	list_for_each_entry(iface, &ses->iface_list, iface_head) {
+		/* do not mix rdma and non-rdma interfaces */
+		if (iface->rdma_capable != server->rdma)
+			continue;
+
 		if (!iface->is_active ||
 		    (is_ses_using_iface(ses, iface) &&
 		     !iface->rss_capable)) {
 			continue;
 		}
+
+		/* check if we already allocated enough channels */
+		iface_weight = iface->speed / iface_min_speed;
+
+		if (iface->weight_fulfilled >= iface_weight)
+			continue;
+
 		kref_get(&iface->refcount);
 		break;
 	}
@@ -320,10 +363,22 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 		cifs_dbg(FYI, "replacing iface: %pIS with %pIS\n",
 			 &old_iface->sockaddr,
 			 &iface->sockaddr);
+
+		old_iface->num_channels--;
+		if (old_iface->weight_fulfilled)
+			old_iface->weight_fulfilled--;
+		iface->num_channels++;
+		iface->weight_fulfilled++;
+
 		kref_put(&old_iface->refcount, release_iface);
 	} else if (old_iface) {
 		cifs_dbg(FYI, "releasing ref to iface: %pIS\n",
 			 &old_iface->sockaddr);
+
+		old_iface->num_channels--;
+		if (old_iface->weight_fulfilled)
+			old_iface->weight_fulfilled--;
+
 		kref_put(&old_iface->refcount, release_iface);
 	} else {
 		WARN_ON(!iface);
-- 
2.34.1

[PATCH 08/14] cifs: account for primary channel in the interface list

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

The refcounting of server interfaces should account
for the primary channel too. Although this is not
strictly necessary, doing so will account for the primary
channel in DebugData.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/sess.c    | 23 +++++++++++++++++++++++
 fs/smb/client/smb2ops.c |  6 ++++++
 2 files changed, 29 insertions(+)

diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
index d009994f82cf..6843deed6119 100644
--- a/fs/smb/client/sess.c
+++ b/fs/smb/client/sess.c
@@ -332,6 +332,16 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 
 	/* then look for a new one */
 	list_for_each_entry(iface, &ses->iface_list, iface_head) {
+		if (!chan_index) {
+			/* if we're trying to get the updated iface for primary channel */
+			if (!cifs_match_ipaddr((struct sockaddr *) &server->dstaddr,
+					       (struct sockaddr *) &iface->sockaddr))
+				continue;
+
+			kref_get(&iface->refcount);
+			break;
+		}
+
 		/* do not mix rdma and non-rdma interfaces */
 		if (iface->rdma_capable != server->rdma)
 			continue;
@@ -358,6 +368,13 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 		cifs_dbg(FYI, "unable to find a suitable iface\n");
 	}
 
+	if (!chan_index && !iface) {
+		cifs_dbg(VFS, "unable to get the interface matching: %pIS\n",
+			 &server->dstaddr);
+		spin_unlock(&ses->iface_lock);
+		return 0;
+	}
+
 	/* now drop the ref to the current iface */
 	if (old_iface && iface) {
 		cifs_dbg(FYI, "replacing iface: %pIS with %pIS\n",
@@ -380,6 +397,12 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 			old_iface->weight_fulfilled--;
 
 		kref_put(&old_iface->refcount, release_iface);
+	} else if (!chan_index) {
+		/* special case: update interface for primary channel */
+		cifs_dbg(FYI, "referencing primary channel iface: %pIS\n",
+			 &iface->sockaddr);
+		iface->num_channels++;
+		iface->weight_fulfilled++;
 	} else {
 		WARN_ON(!iface);
 		cifs_dbg(FYI, "adding new iface: %pIS\n", &iface->sockaddr);
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index 9aeecee6b91b..e7e1e0e5bdfe 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -756,6 +756,7 @@ SMB3_request_interfaces(const unsigned int xid, struct cifs_tcon *tcon, bool in_
 	unsigned int ret_data_len = 0;
 	struct network_interface_info_ioctl_rsp *out_buf = NULL;
 	struct cifs_ses *ses = tcon->ses;
+	struct TCP_Server_Info *pserver;
 
 	/* do not query too frequently */
 	if (ses->iface_last_update &&
@@ -780,6 +781,11 @@ SMB3_request_interfaces(const unsigned int xid, struct cifs_tcon *tcon, bool in_
 	if (rc)
 		goto out;
 
+	/* check if iface is still active */
+	pserver = ses->chans[0].server;
+	if (pserver && !cifs_chan_is_iface_active(ses, pserver))
+		cifs_chan_update_iface(ses, pserver);
+
 out:
 	kfree(out_buf);
 	return rc;
-- 
2.34.1

Re: [PATCH 08/14] cifs: account for primary channel in the interface list

[Paulo Alcantara]

nspmangalore@gmail.com writes:

> From: Shyam Prasad N <sprasad@microsoft.com>
>
> The refcounting of server interfaces should account
> for the primary channel too. Although this is not
> strictly necessary, doing so will account for the primary
> channel in DebugData.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/sess.c    | 23 +++++++++++++++++++++++
>  fs/smb/client/smb2ops.c |  6 ++++++
>  2 files changed, 29 insertions(+)
>
> diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
> index d009994f82cf..6843deed6119 100644
> --- a/fs/smb/client/sess.c
> +++ b/fs/smb/client/sess.c
> @@ -332,6 +332,16 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
>  
>  	/* then look for a new one */
>  	list_for_each_entry(iface, &ses->iface_list, iface_head) {
> +		if (!chan_index) {
> +			/* if we're trying to get the updated iface for primary channel */
> +			if (!cifs_match_ipaddr((struct sockaddr *) &server->dstaddr,
> +					       (struct sockaddr *) &iface->sockaddr))
> +				continue;

You should hold @server->srv_lock to protect access of @server->dstaddr
as it might change over reconnect or mount.

> +
> +			kref_get(&iface->refcount);
> +			break;
> +		}
> +
>  		/* do not mix rdma and non-rdma interfaces */
>  		if (iface->rdma_capable != server->rdma)
>  			continue;
> @@ -358,6 +368,13 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
>  		cifs_dbg(FYI, "unable to find a suitable iface\n");
>  	}
>  
> +	if (!chan_index && !iface) {
> +		cifs_dbg(VFS, "unable to get the interface matching: %pIS\n",
> +			 &server->dstaddr);

Ditto.

Also, I think you should log in FYI level as the above doesn't seem
unlikely to happen.

Re: [PATCH 08/14] cifs: account for primary channel in the interface list

[Steve French]

[-- Attachment #1: Type: text/plain, Size: 2490 bytes --]

Updated patch attached. Let me know if any objections.

Paulo,
I made minor updates to Shyam's patch following your suggestion of
changing the logging level you suggested, and saving off dstaddr so we
don't have to hold a lock on it

On Wed, Nov 8, 2023 at 9:44 AM Paulo Alcantara <pc@manguebit.com> wrote:
>
> nspmangalore@gmail.com writes:
>
> > From: Shyam Prasad N <sprasad@microsoft.com>
> >
> > The refcounting of server interfaces should account
> > for the primary channel too. Although this is not
> > strictly necessary, doing so will account for the primary
> > channel in DebugData.
> >
> > Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> > ---
> >  fs/smb/client/sess.c    | 23 +++++++++++++++++++++++
> >  fs/smb/client/smb2ops.c |  6 ++++++
> >  2 files changed, 29 insertions(+)
> >
> > diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
> > index d009994f82cf..6843deed6119 100644
> > --- a/fs/smb/client/sess.c
> > +++ b/fs/smb/client/sess.c
> > @@ -332,6 +332,16 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
> >
> >       /* then look for a new one */
> >       list_for_each_entry(iface, &ses->iface_list, iface_head) {
> > +             if (!chan_index) {
> > +                     /* if we're trying to get the updated iface for primary channel */
> > +                     if (!cifs_match_ipaddr((struct sockaddr *) &server->dstaddr,
> > +                                            (struct sockaddr *) &iface->sockaddr))
> > +                             continue;
>
> You should hold @server->srv_lock to protect access of @server->dstaddr
> as it might change over reconnect or mount.
>
> > +
> > +                     kref_get(&iface->refcount);
> > +                     break;
> > +             }
> > +
> >               /* do not mix rdma and non-rdma interfaces */
> >               if (iface->rdma_capable != server->rdma)
> >                       continue;
> > @@ -358,6 +368,13 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
> >               cifs_dbg(FYI, "unable to find a suitable iface\n");
> >       }
> >
> > +     if (!chan_index && !iface) {
> > +             cifs_dbg(VFS, "unable to get the interface matching: %pIS\n",
> > +                      &server->dstaddr);
>
> Ditto.
>
> Also, I think you should log in FYI level as the above doesn't seem
> unlikely to happen.



-- 
Thanks,

Steve

[-- Attachment #2: 0001-cifs-account-for-primary-channel-in-the-interface-li.patch --]
[-- Type: text/x-patch, Size: 3829 bytes --]

From 8023cb3ca155551abcef7125cd53e626022fc53b Mon Sep 17 00:00:00 2001
From: Shyam Prasad N <sprasad@microsoft.com>
Date: Tue, 14 Mar 2023 11:14:58 +0000
Subject: [PATCH] cifs: account for primary channel in the interface list

The refcounting of server interfaces should account
for the primary channel too. Although this is not
strictly necessary, doing so will account for the primary
channel in DebugData.

Cc: stable@vger.kernel.org
Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
Signed-off-by: Steve French <stfrench@microsoft.com>
---
 fs/smb/client/sess.c    | 28 ++++++++++++++++++++++++++++
 fs/smb/client/smb2ops.c |  6 ++++++
 2 files changed, 34 insertions(+)

diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
index 04d300e9a779..21e75ca47151 100644
--- a/fs/smb/client/sess.c
+++ b/fs/smb/client/sess.c
@@ -303,6 +303,7 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 	struct cifs_server_iface *iface = NULL;
 	struct cifs_server_iface *old_iface = NULL;
 	struct cifs_server_iface *last_iface = NULL;
+	struct sockaddr_storage ss;
 	int rc = 0;
 
 	spin_lock(&ses->chan_lock);
@@ -321,6 +322,10 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 	}
 	spin_unlock(&ses->chan_lock);
 
+	spin_lock(&server->srv_lock);
+	ss = server->dstaddr;
+	spin_unlock(&server->srv_lock);
+
 	spin_lock(&ses->iface_lock);
 	if (!ses->iface_count) {
 		spin_unlock(&ses->iface_lock);
@@ -334,6 +339,16 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 
 	/* then look for a new one */
 	list_for_each_entry(iface, &ses->iface_list, iface_head) {
+		if (!chan_index) {
+			/* if we're trying to get the updated iface for primary channel */
+			if (!cifs_match_ipaddr((struct sockaddr *) &ss,
+					       (struct sockaddr *) &iface->sockaddr))
+				continue;
+
+			kref_get(&iface->refcount);
+			break;
+		}
+
 		/* do not mix rdma and non-rdma interfaces */
 		if (iface->rdma_capable != server->rdma)
 			continue;
@@ -360,6 +375,13 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 		cifs_dbg(FYI, "unable to find a suitable iface\n");
 	}
 
+	if (!chan_index && !iface) {
+		cifs_dbg(FYI, "unable to get the interface matching: %pIS\n",
+			 &ss);
+		spin_unlock(&ses->iface_lock);
+		return 0;
+	}
+
 	/* now drop the ref to the current iface */
 	if (old_iface && iface) {
 		cifs_dbg(FYI, "replacing iface: %pIS with %pIS\n",
@@ -382,6 +404,12 @@ cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server)
 			old_iface->weight_fulfilled--;
 
 		kref_put(&old_iface->refcount, release_iface);
+	} else if (!chan_index) {
+		/* special case: update interface for primary channel */
+		cifs_dbg(FYI, "referencing primary channel iface: %pIS\n",
+			 &iface->sockaddr);
+		iface->num_channels++;
+		iface->weight_fulfilled++;
 	} else {
 		WARN_ON(!iface);
 		cifs_dbg(FYI, "adding new iface: %pIS\n", &iface->sockaddr);
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index 601e7a187f87..a959ed2c9b22 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -756,6 +756,7 @@ SMB3_request_interfaces(const unsigned int xid, struct cifs_tcon *tcon, bool in_
 	unsigned int ret_data_len = 0;
 	struct network_interface_info_ioctl_rsp *out_buf = NULL;
 	struct cifs_ses *ses = tcon->ses;
+	struct TCP_Server_Info *pserver;
 
 	/* do not query too frequently */
 	if (ses->iface_last_update &&
@@ -780,6 +781,11 @@ SMB3_request_interfaces(const unsigned int xid, struct cifs_tcon *tcon, bool in_
 	if (rc)
 		goto out;
 
+	/* check if iface is still active */
+	pserver = ses->chans[0].server;
+	if (pserver && !cifs_chan_is_iface_active(ses, pserver))
+		cifs_chan_update_iface(ses, pserver);
+
 out:
 	kfree(out_buf);
 	return rc;
-- 
2.39.2

Re: [PATCH 08/14] cifs: account for primary channel in the interface list

[Paulo Alcantara]

Steve French <smfrench@gmail.com> writes:

> Updated patch attached. Let me know if any objections.
>
> I made minor updates to Shyam's patch following your suggestion of
> changing the logging level you suggested, and saving off dstaddr so we
> don't have to hold a lock on it

Looks good, thanks!

[PATCH 09/14] cifs: add a back pointer to cifs_sb from tcon

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

Today, we have no way to access the cifs_sb when we
just have pointers to struct tcon. This is very
limiting as many functions deal with cifs_sb, and
these calls do not directly originate from VFS.

This change introduces a new cifs_sb field in cifs_tcon
that points to the cifs_sb for the tcon. The assumption
here is that a tcon will always map to this cifs_sb and
will never change.

Also, refcounting should not be necessary, since cifs_sb
will never be freed before tcon.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/cifsglob.h | 1 +
 fs/smb/client/connect.c  | 2 ++
 2 files changed, 3 insertions(+)

diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
index 81e7a45f413d..cdbc2cd207dc 100644
--- a/fs/smb/client/cifsglob.h
+++ b/fs/smb/client/cifsglob.h
@@ -1134,6 +1134,7 @@ struct cifs_tcon {
 	int tc_count;
 	struct list_head rlist; /* reconnect list */
 	spinlock_t tc_lock;  /* protect anything here that is not protected */
+	struct cifs_sb_info *cifs_sb; /* back pointer to cifs super block */
 	atomic_t num_local_opens;  /* num of all opens including disconnected */
 	atomic_t num_remote_opens; /* num of all network opens on server */
 	struct list_head openFileList;
diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index 8393977e21ee..184075da5c6e 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -3355,6 +3355,7 @@ int cifs_mount_get_tcon(struct cifs_mount_ctx *mnt_ctx)
 		tcon = NULL;
 		goto out;
 	}
+	tcon->cifs_sb = cifs_sb;
 
 	/* if new SMB3.11 POSIX extensions are supported do not remap / and \ */
 	if (tcon->posix_extensions)
@@ -3986,6 +3987,7 @@ cifs_construct_tcon(struct cifs_sb_info *cifs_sb, kuid_t fsuid)
 		cifs_put_smb_ses(ses);
 		goto out;
 	}
+	tcon->cifs_sb = cifs_sb;
 
 #ifdef CONFIG_CIFS_ALLOW_INSECURE_LEGACY
 	if (cap_unix(ses))
-- 
2.34.1

Re: [PATCH 09/14] cifs: add a back pointer to cifs_sb from tcon

[Steve French]

merged into cifs-2.6.git for-next pending more testing

added cc:stable

On Mon, Oct 30, 2023 at 6:00 AM <nspmangalore@gmail.com> wrote:
>
> From: Shyam Prasad N <sprasad@microsoft.com>
>
> Today, we have no way to access the cifs_sb when we
> just have pointers to struct tcon. This is very
> limiting as many functions deal with cifs_sb, and
> these calls do not directly originate from VFS.
>
> This change introduces a new cifs_sb field in cifs_tcon
> that points to the cifs_sb for the tcon. The assumption
> here is that a tcon will always map to this cifs_sb and
> will never change.
>
> Also, refcounting should not be necessary, since cifs_sb
> will never be freed before tcon.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/cifsglob.h | 1 +
>  fs/smb/client/connect.c  | 2 ++
>  2 files changed, 3 insertions(+)
>
> diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
> index 81e7a45f413d..cdbc2cd207dc 100644
> --- a/fs/smb/client/cifsglob.h
> +++ b/fs/smb/client/cifsglob.h
> @@ -1134,6 +1134,7 @@ struct cifs_tcon {
>         int tc_count;
>         struct list_head rlist; /* reconnect list */
>         spinlock_t tc_lock;  /* protect anything here that is not protected */
> +       struct cifs_sb_info *cifs_sb; /* back pointer to cifs super block */
>         atomic_t num_local_opens;  /* num of all opens including disconnected */
>         atomic_t num_remote_opens; /* num of all network opens on server */
>         struct list_head openFileList;
> diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
> index 8393977e21ee..184075da5c6e 100644
> --- a/fs/smb/client/connect.c
> +++ b/fs/smb/client/connect.c
> @@ -3355,6 +3355,7 @@ int cifs_mount_get_tcon(struct cifs_mount_ctx *mnt_ctx)
>                 tcon = NULL;
>                 goto out;
>         }
> +       tcon->cifs_sb = cifs_sb;
>
>         /* if new SMB3.11 POSIX extensions are supported do not remap / and \ */
>         if (tcon->posix_extensions)
> @@ -3986,6 +3987,7 @@ cifs_construct_tcon(struct cifs_sb_info *cifs_sb, kuid_t fsuid)
>                 cifs_put_smb_ses(ses);
>                 goto out;
>         }
> +       tcon->cifs_sb = cifs_sb;
>
>  #ifdef CONFIG_CIFS_ALLOW_INSECURE_LEGACY
>         if (cap_unix(ses))
> --
> 2.34.1
>


-- 
Thanks,

Steve

Re: [PATCH 09/14] cifs: add a back pointer to cifs_sb from tcon

[Paulo Alcantara]

nspmangalore@gmail.com writes:

> From: Shyam Prasad N <sprasad@microsoft.com>
>
> Today, we have no way to access the cifs_sb when we
> just have pointers to struct tcon. This is very
> limiting as many functions deal with cifs_sb, and
> these calls do not directly originate from VFS.
>
> This change introduces a new cifs_sb field in cifs_tcon
> that points to the cifs_sb for the tcon. The assumption
> here is that a tcon will always map to this cifs_sb and
> will never change.
>
> Also, refcounting should not be necessary, since cifs_sb
> will never be freed before tcon.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/cifsglob.h | 1 +
>  fs/smb/client/connect.c  | 2 ++
>  2 files changed, 3 insertions(+)

This is wrong as a single tcon may be shared among different
superblocks.  You can, however, map those superblocks to a tcon by using
the cifs_sb_master_tcon() helper.

If you do something like this

	mount.cifs //srv/share /mnt/1 -o ...
	mount.cifs //srv/share /mnt/1 -o ... -> -EBUSY

tcon->cifs_sb will end up with the already freed superblock pointer that
was compared to the existing one.  So, you'll get an use-after-free when
you dereference tcon->cifs_sb as in patch 11/14.

Re: [PATCH 09/14] cifs: add a back pointer to cifs_sb from tcon

[Shyam Prasad N]

On Sat, Nov 4, 2023 at 2:33 AM Paulo Alcantara <pc@manguebit.com> wrote:
>
> nspmangalore@gmail.com writes:
>
> > From: Shyam Prasad N <sprasad@microsoft.com>
> >
> > Today, we have no way to access the cifs_sb when we
> > just have pointers to struct tcon. This is very
> > limiting as many functions deal with cifs_sb, and
> > these calls do not directly originate from VFS.
> >
> > This change introduces a new cifs_sb field in cifs_tcon
> > that points to the cifs_sb for the tcon. The assumption
> > here is that a tcon will always map to this cifs_sb and
> > will never change.
> >
> > Also, refcounting should not be necessary, since cifs_sb
> > will never be freed before tcon.
> >
> > Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> > ---
> >  fs/smb/client/cifsglob.h | 1 +
> >  fs/smb/client/connect.c  | 2 ++
> >  2 files changed, 3 insertions(+)
>
> This is wrong as a single tcon may be shared among different
> superblocks.  You can, however, map those superblocks to a tcon by using
> the cifs_sb_master_tcon() helper.
>
> If you do something like this
>
>         mount.cifs //srv/share /mnt/1 -o ...
>         mount.cifs //srv/share /mnt/1 -o ... -> -EBUSY
>
> tcon->cifs_sb will end up with the already freed superblock pointer that
> was compared to the existing one.  So, you'll get an use-after-free when
> you dereference tcon->cifs_sb as in patch 11/14.

Hi Steve,
I discussed this one with Paulo separately. You can drop this patch.
I'll have another patch in place of this one. And then send you an
updated patch for the patch which depends on it.

-- 
Regards,
Shyam

Re: [PATCH 09/14] cifs: add a back pointer to cifs_sb from tcon

[Shyam Prasad N]

[-- Attachment #1: Type: text/plain, Size: 1931 bytes --]

On Mon, Nov 6, 2023 at 9:42 PM Shyam Prasad N <nspmangalore@gmail.com> wrote:
>
> On Sat, Nov 4, 2023 at 2:33 AM Paulo Alcantara <pc@manguebit.com> wrote:
> >
> > nspmangalore@gmail.com writes:
> >
> > > From: Shyam Prasad N <sprasad@microsoft.com>
> > >
> > > Today, we have no way to access the cifs_sb when we
> > > just have pointers to struct tcon. This is very
> > > limiting as many functions deal with cifs_sb, and
> > > these calls do not directly originate from VFS.
> > >
> > > This change introduces a new cifs_sb field in cifs_tcon
> > > that points to the cifs_sb for the tcon. The assumption
> > > here is that a tcon will always map to this cifs_sb and
> > > will never change.
> > >
> > > Also, refcounting should not be necessary, since cifs_sb
> > > will never be freed before tcon.
> > >
> > > Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> > > ---
> > >  fs/smb/client/cifsglob.h | 1 +
> > >  fs/smb/client/connect.c  | 2 ++
> > >  2 files changed, 3 insertions(+)
> >
> > This is wrong as a single tcon may be shared among different
> > superblocks.  You can, however, map those superblocks to a tcon by using
> > the cifs_sb_master_tcon() helper.
> >
> > If you do something like this
> >
> >         mount.cifs //srv/share /mnt/1 -o ...
> >         mount.cifs //srv/share /mnt/1 -o ... -> -EBUSY
> >
> > tcon->cifs_sb will end up with the already freed superblock pointer that
> > was compared to the existing one.  So, you'll get an use-after-free when
> > you dereference tcon->cifs_sb as in patch 11/14.
>
> Hi Steve,
> I discussed this one with Paulo separately. You can drop this patch.
> I'll have another patch in place of this one. And then send you an
> updated patch for the patch which depends on it.
>
> --
> Regards,
> Shyam

Here's the replacement patch for "cifs: add a back pointer to cifs_sb from tcon"

-- 
Regards,
Shyam

[-- Attachment #2: 0004-cifs-do-not-pass-cifs_sb-when-trying-to-add-channels.patch --]
[-- Type: application/octet-stream, Size: 4004 bytes --]

From 833e5471212056e107757f1df908e0980ccd9112 Mon Sep 17 00:00:00 2001
From: Shyam Prasad N <sprasad@microsoft.com>
Date: Mon, 6 Nov 2023 16:22:11 +0000
Subject: [PATCH 4/9] cifs: do not pass cifs_sb when trying to add channels

The only reason why cifs_sb gets passed today to cifs_try_adding_channels
is to pass the local_nls field for the new channels and binding session.
However, the ses struct already has local_nls field that is setup during
the first cifs_setup_session. So there is no need to pass cifs_sb.

This change removes cifs_sb from the arg list for this and the functions
that it calls and uses ses->local_nls instead.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/cifsproto.h |  2 +-
 fs/smb/client/connect.c   |  2 +-
 fs/smb/client/sess.c      | 12 ++++++------
 3 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
index c1f71c6be7e3..eed8dbb6b2fb 100644
--- a/fs/smb/client/cifsproto.h
+++ b/fs/smb/client/cifsproto.h
@@ -610,7 +610,7 @@ void cifs_free_hash(struct shash_desc **sdesc);
 
 struct cifs_chan *
 cifs_ses_find_chan(struct cifs_ses *ses, struct TCP_Server_Info *server);
-int cifs_try_adding_channels(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses);
+int cifs_try_adding_channels(struct cifs_ses *ses);
 bool is_server_using_iface(struct TCP_Server_Info *server,
 			   struct cifs_server_iface *iface);
 bool is_ses_using_iface(struct cifs_ses *ses, struct cifs_server_iface *iface);
diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index 3ff82f0aa00e..947e3c362beb 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -3564,7 +3564,7 @@ int cifs_mount(struct cifs_sb_info *cifs_sb, struct smb3_fs_context *ctx)
 	ctx->prepath = NULL;
 
 out:
-	cifs_try_adding_channels(cifs_sb, mnt_ctx.ses);
+	cifs_try_adding_channels(mnt_ctx.ses);
 	rc = mount_setup_tlink(cifs_sb, mnt_ctx.ses, mnt_ctx.tcon);
 	if (rc)
 		goto error;
diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
index f454ee1de9ef..b4e60dcf3a94 100644
--- a/fs/smb/client/sess.c
+++ b/fs/smb/client/sess.c
@@ -24,7 +24,7 @@
 #include "fs_context.h"
 
 static int
-cifs_ses_add_channel(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses,
+cifs_ses_add_channel(struct cifs_ses *ses,
 		     struct cifs_server_iface *iface);
 
 bool
@@ -171,7 +171,7 @@ cifs_chan_is_iface_active(struct cifs_ses *ses,
 }
 
 /* returns number of channels added */
-int cifs_try_adding_channels(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses)
+int cifs_try_adding_channels(struct cifs_ses *ses)
 {
 	struct TCP_Server_Info *server = ses->server;
 	int old_chan_count, new_chan_count;
@@ -253,7 +253,7 @@ int cifs_try_adding_channels(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses)
 			kref_get(&iface->refcount);
 
 			spin_unlock(&ses->iface_lock);
-			rc = cifs_ses_add_channel(cifs_sb, ses, iface);
+			rc = cifs_ses_add_channel(ses, iface);
 			spin_lock(&ses->iface_lock);
 
 			if (rc) {
@@ -449,7 +449,7 @@ cifs_ses_find_chan(struct cifs_ses *ses, struct TCP_Server_Info *server)
 }
 
 static int
-cifs_ses_add_channel(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses,
+cifs_ses_add_channel(struct cifs_ses *ses,
 		     struct cifs_server_iface *iface)
 {
 	struct TCP_Server_Info *chan_server;
@@ -528,7 +528,7 @@ cifs_ses_add_channel(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses,
 	 * This will be used for encoding/decoding user/domain/pw
 	 * during sess setup auth.
 	 */
-	ctx->local_nls = cifs_sb->local_nls;
+	ctx->local_nls = ses->local_nls;
 
 	/* Use RDMA if possible */
 	ctx->rdma = iface->rdma_capable;
@@ -574,7 +574,7 @@ cifs_ses_add_channel(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses,
 
 	rc = cifs_negotiate_protocol(xid, ses, chan->server);
 	if (!rc)
-		rc = cifs_setup_session(xid, ses, chan->server, cifs_sb->local_nls);
+		rc = cifs_setup_session(xid, ses, chan->server, ses->local_nls);
 
 	mutex_unlock(&ses->session_mutex);
 
-- 
2.34.1

Fwd: [PATCH 09/14] cifs: add a back pointer to cifs_sb from tcon

[Steve French]

[-- Attachment #1: Type: text/plain, Size: 3655 bytes --]

resending to list

---------- Forwarded message ---------
From: Steve French <smfrench@gmail.com>
Date: Mon, Nov 6, 2023 at 1:13 PM
Subject: Re: [PATCH 09/14] cifs: add a back pointer to cifs_sb from tcon
To: Shyam Prasad N <nspmangalore@gmail.com>
Cc: Paulo Alcantara <pc@manguebit.com>, <bharathsm.hsk@gmail.com>,
<linux-cifs@vger.kernel.org>, Shyam Prasad N <sprasad@microsoft.com>


Updated patches again to address compile/sparse warning and pushed to
for-next pending more testing and review

Attached are updated patches 1 and 2

On Mon, Nov 6, 2023 at 12:45 PM Steve French <smfrench@gmail.com> wrote:
>
> lightly updated patch 1 and patch 2 in your series to fix checkpatch warnings
>
> Was thinking about patch 2 though and whether it could hurt cases where there is little parallelism - we could randomly pick channels to send things on so we could overuse channels with higher latency rather than preferring the first two channels which are the "faster" ones (and presumably lower latencies).   Was wondering if we end up picking e.g. 4 channels, 1 of which is "fast" and 3 are slower - we could end up with cases where no traffic on channel 1 but we end up sending on a longer latency channlel - ie the round robin approach could end up using channel 2 or 3 or 4 which have longer latencies even if no traffic on the "fastest" channel ...
>
>
> On Mon, Nov 6, 2023 at 11:04 AM Shyam Prasad N <nspmangalore@gmail.com> wrote:
>>
>> On Mon, Nov 6, 2023 at 9:42 PM Shyam Prasad N <nspmangalore@gmail.com> wrote:
>> >
>> > On Sat, Nov 4, 2023 at 2:33 AM Paulo Alcantara <pc@manguebit.com> wrote:
>> > >
>> > > nspmangalore@gmail.com writes:
>> > >
>> > > > From: Shyam Prasad N <sprasad@microsoft.com>
>> > > >
>> > > > Today, we have no way to access the cifs_sb when we
>> > > > just have pointers to struct tcon. This is very
>> > > > limiting as many functions deal with cifs_sb, and
>> > > > these calls do not directly originate from VFS.
>> > > >
>> > > > This change introduces a new cifs_sb field in cifs_tcon
>> > > > that points to the cifs_sb for the tcon. The assumption
>> > > > here is that a tcon will always map to this cifs_sb and
>> > > > will never change.
>> > > >
>> > > > Also, refcounting should not be necessary, since cifs_sb
>> > > > will never be freed before tcon.
>> > > >
>> > > > Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
>> > > > ---
>> > > >  fs/smb/client/cifsglob.h | 1 +
>> > > >  fs/smb/client/connect.c  | 2 ++
>> > > >  2 files changed, 3 insertions(+)
>> > >
>> > > This is wrong as a single tcon may be shared among different
>> > > superblocks.  You can, however, map those superblocks to a tcon by using
>> > > the cifs_sb_master_tcon() helper.
>> > >
>> > > If you do something like this
>> > >
>> > >         mount.cifs //srv/share /mnt/1 -o ...
>> > >         mount.cifs //srv/share /mnt/1 -o ... -> -EBUSY
>> > >
>> > > tcon->cifs_sb will end up with the already freed superblock pointer that
>> > > was compared to the existing one.  So, you'll get an use-after-free when
>> > > you dereference tcon->cifs_sb as in patch 11/14.
>> >
>> > Hi Steve,
>> > I discussed this one with Paulo separately. You can drop this patch.
>> > I'll have another patch in place of this one. And then send you an
>> > updated patch for the patch which depends on it.
>> >
>> > --
>> > Regards,
>> > Shyam
>>
>> Here's the replacement patch for "cifs: add a back pointer to cifs_sb from tcon"
>>
>> --
>> Regards,
>> Shyam
>
>
>
> --
> Thanks,
>
> Steve



-- 
Thanks,

Steve


-- 
Thanks,

Steve

[-- Attachment #2: 0002-cifs-distribute-channels-across-interfaces-based-on-.patch --]
[-- Type: application/x-patch, Size: 7981 bytes --]

[-- Attachment #3: 0001-cifs-handle-cases-where-a-channel-is-closed.patch --]
[-- Type: application/x-patch, Size: 7593 bytes --]

Re: [PATCH 09/14] cifs: add a back pointer to cifs_sb from tcon

[Paulo Alcantara]

Shyam Prasad N <nspmangalore@gmail.com> writes:

> Here's the replacement patch for "cifs: add a back pointer to cifs_sb from tcon"

Looks good,

Reviewed-by: Paulo Alcantara (SUSE) <pc@manguebit.com>

Re: [PATCH 09/14] cifs: add a back pointer to cifs_sb from tcon

[Steve French]

added RB for you for this and for "cifs: Fix encryption of cleared,
but unset rq_iter ..."

Let me know if any more RB or Acked-by for the other 11 in for-next

fce2374a6ce1 (HEAD -> for-next) smb3: fix caching of ctime on setxattr
8cc431e2afba smb3: more minor cleanups for session handling routines
2828c39da33f smb3: minor cleanup of session handling code
e8c286d1fb82 cifs: update internal module version number for cifs.ko
a25ebba8a4f2 smb3: minor RDMA cleanup
091578169aa4 cifs: handle when server stops supporting multichannel
a340e7145da5 cifs: handle when server starts supporting multichannel
693358495264 cifs: reconnect work should have reference on server struct
e935ee282ccf cifs: do not pass cifs_sb when trying to add channels
4266585b23f4 cifs: account for primary channel in the interface list
6dbc7a50c5e4 cifs: distribute channels across interfaces based on speed
44a65e388107 cifs: handle cases where a channel is closed
37de5a80e932 cifs: Fix encryption of cleared, but unset rq_iter data buffers

Am trying to work on a few patches today (e.g. caching root handle
across mount when no dir leases support to reduce open and close)
and how we might save off an "altpassword" when servers do "key
rotation" (where you move from one password to another but don't want
to risk sessions going down for a while for that user).  Let me know
if any patches/features for you in next few days of merge window

On Wed, Nov 8, 2023 at 9:24 AM Paulo Alcantara <pc@manguebit.com> wrote:
>
> Shyam Prasad N <nspmangalore@gmail.com> writes:
>
> > Here's the replacement patch for "cifs: add a back pointer to cifs_sb from tcon"
>
> Looks good,
>
> Reviewed-by: Paulo Alcantara (SUSE) <pc@manguebit.com>



-- 
Thanks,

Steve

[PATCH 10/14] cifs: reconnect work should have reference on server struct

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

The delayed work for reconnect takes server struct
as a parameter. But it does so without holding a ref
to it. Normally, this may not show a problem as
the reconnect work is only cancelled on umount.

However, since we now plan to support scaling down of
channels, and the scale down can happen from reconnect
work itself, we need to fix it.

This change takes a reference on the server struct
before it is passed to the delayed work. And drops
the reference in the delayed work itself. Or if
the delayed work is successfully cancelled, by the
process that cancels it.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/connect.c | 27 +++++++++++++++++++++------
 fs/smb/client/smb2pdu.c | 23 +++++++++++++----------
 2 files changed, 34 insertions(+), 16 deletions(-)

diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index 184075da5c6e..e71aa33bf026 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -389,7 +389,13 @@ static int __cifs_reconnect(struct TCP_Server_Info *server,
 			spin_unlock(&server->srv_lock);
 			cifs_swn_reset_server_dstaddr(server);
 			cifs_server_unlock(server);
-			mod_delayed_work(cifsiod_wq, &server->reconnect, 0);
+
+			/* increase ref count which reconnect work will drop */
+			spin_lock(&cifs_tcp_ses_lock);
+			server->srv_count++;
+			spin_unlock(&cifs_tcp_ses_lock);
+			if (mod_delayed_work(cifsiod_wq, &server->reconnect, 0))
+				cifs_put_tcp_session(server, false);
 		}
 	} while (server->tcpStatus == CifsNeedReconnect);
 
@@ -519,7 +525,13 @@ static int reconnect_dfs_server(struct TCP_Server_Info *server)
 		spin_unlock(&server->srv_lock);
 		cifs_swn_reset_server_dstaddr(server);
 		cifs_server_unlock(server);
-		mod_delayed_work(cifsiod_wq, &server->reconnect, 0);
+
+		/* increase ref count which reconnect work will drop */
+		spin_lock(&cifs_tcp_ses_lock);
+		server->srv_count++;
+		spin_unlock(&cifs_tcp_ses_lock);
+		if (mod_delayed_work(cifsiod_wq, &server->reconnect, 0))
+			cifs_put_tcp_session(server, false);
 	} while (server->tcpStatus == CifsNeedReconnect);
 
 	mutex_lock(&server->refpath_lock);
@@ -1601,16 +1613,19 @@ cifs_put_tcp_session(struct TCP_Server_Info *server, int from_reconnect)
 
 	cancel_delayed_work_sync(&server->echo);
 
-	if (from_reconnect)
+	if (from_reconnect) {
 		/*
 		 * Avoid deadlock here: reconnect work calls
 		 * cifs_put_tcp_session() at its end. Need to be sure
 		 * that reconnect work does nothing with server pointer after
 		 * that step.
 		 */
-		cancel_delayed_work(&server->reconnect);
-	else
-		cancel_delayed_work_sync(&server->reconnect);
+		if (cancel_delayed_work(&server->reconnect))
+			cifs_put_tcp_session(server, from_reconnect);
+	} else {
+		if (cancel_delayed_work_sync(&server->reconnect))
+			cifs_put_tcp_session(server, from_reconnect);
+	}
 
 	spin_lock(&server->srv_lock);
 	server->tcpStatus = CifsExiting;
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index c75a80bb6d9e..b7665155f4e2 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -3852,12 +3852,6 @@ void smb2_reconnect_server(struct work_struct *work)
 		}
 		spin_unlock(&ses->chan_lock);
 	}
-	/*
-	 * Get the reference to server struct to be sure that the last call of
-	 * cifs_put_tcon() in the loop below won't release the server pointer.
-	 */
-	if (tcon_exist || ses_exist)
-		server->srv_count++;
 
 	spin_unlock(&cifs_tcp_ses_lock);
 
@@ -3905,13 +3899,17 @@ void smb2_reconnect_server(struct work_struct *work)
 
 done:
 	cifs_dbg(FYI, "Reconnecting tcons and channels finished\n");
-	if (resched)
+	if (resched) {
 		queue_delayed_work(cifsiod_wq, &server->reconnect, 2 * HZ);
+		mutex_unlock(&pserver->reconnect_mutex);
+
+		/* no need to put tcp session as we're retrying */
+		return;
+	}
 	mutex_unlock(&pserver->reconnect_mutex);
 
 	/* now we can safely release srv struct */
-	if (tcon_exist || ses_exist)
-		cifs_put_tcp_session(server, 1);
+	cifs_put_tcp_session(server, true);
 }
 
 int
@@ -3931,7 +3929,12 @@ SMB2_echo(struct TCP_Server_Info *server)
 	    server->ops->need_neg(server)) {
 		spin_unlock(&server->srv_lock);
 		/* No need to send echo on newly established connections */
-		mod_delayed_work(cifsiod_wq, &server->reconnect, 0);
+		spin_lock(&cifs_tcp_ses_lock);
+		server->srv_count++;
+		spin_unlock(&cifs_tcp_ses_lock);
+		if (mod_delayed_work(cifsiod_wq, &server->reconnect, 0))
+			cifs_put_tcp_session(server, false);
+
 		return rc;
 	}
 	spin_unlock(&server->srv_lock);
-- 
2.34.1

Re: [PATCH 10/14] cifs: reconnect work should have reference on server struct

[Paulo Alcantara]

nspmangalore@gmail.com writes:

> From: Shyam Prasad N <sprasad@microsoft.com>
>
> The delayed work for reconnect takes server struct
> as a parameter. But it does so without holding a ref
> to it. Normally, this may not show a problem as
> the reconnect work is only cancelled on umount.
>
> However, since we now plan to support scaling down of
> channels, and the scale down can happen from reconnect
> work itself, we need to fix it.
>
> This change takes a reference on the server struct
> before it is passed to the delayed work. And drops
> the reference in the delayed work itself. Or if
> the delayed work is successfully cancelled, by the
> process that cancels it.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/connect.c | 27 +++++++++++++++++++++------
>  fs/smb/client/smb2pdu.c | 23 +++++++++++++----------
>  2 files changed, 34 insertions(+), 16 deletions(-)

Several DFS tests are failing with current for-next (5eef12c4e323),
either crashing or leaking server connections.

I've managed to reproduce one of the problems without multichannel
against Windows Server 2022.  Can't reproduce it on v6.6.

DFS layout:

  //dom/dfs
    -> /srv1/dfs
    -> /srv2/dfs

$ mount //dom/dfs /mnt -o ...,vers=3.1.1,echo_interval=10
# disconnect root server that client connected first (srv1 or srv2)
$ ls /mnt/1
ls: cannot access '/mnt/1': Host is down
ls /mnt/1

[   54.455794] CIFS: No dialect specified on mount. Default has changed to a more secure dialect, SMB2.1 or later (e.g. SMB3.1.1), from CIFS (SMB1). To use the less secure SMB1 dialect to access old servers which do not support SMB3.1.1 (or even SMB3 or SMB2.1) specify vers=1.0 on mount.
[   54.958219] CIFS: VFS: Autodisabling the use of server inode numbers on new server
[   54.959049] CIFS: VFS: The server doesn't seem to support them properly or the files might be on different servers (DFS)
[   54.960031] CIFS: VFS: Hardlinks will not be recognized on this mount. Consider mounting with the "noserverino" option to silence this message.
[  100.935813] CIFS: VFS: \\W22-ROOT1.GANDALF.TEST has not responded in 30 seconds. Reconnecting...
[  106.258652] ls (899) used greatest stack depth: 21920 bytes left
[  113.252212] ==================================================================
[  113.253154] BUG: KASAN: slab-use-after-free in cifs_get_fattr+0x546/0xe70 [cifs]
[  113.254721] Read of size 8 at addr ffff88800aa76088 by task ls/942
[  113.255508] 
[  113.255720] CPU: 2 PID: 942 Comm: ls Not tainted 6.7.0-rc1 #4
[  113.256451] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
[  113.257809] Call Trace:
[  113.258132]  <TASK>
[  113.258413]  dump_stack_lvl+0x4a/0x80
[  113.258897]  print_report+0xcf/0x650
[  113.259368]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.259980]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.260375]  ? __phys_addr+0x46/0x90
[  113.260676]  kasan_report+0xd8/0x110
[  113.260976]  ? cifs_get_fattr+0x546/0xe70 [cifs]
[  113.261745]  ? cifs_get_fattr+0x546/0xe70 [cifs]
[  113.262521]  cifs_get_fattr+0x546/0xe70 [cifs]
[  113.263284]  ? __pfx_cifs_get_fattr+0x10/0x10 [cifs]
[  113.264079]  ? irq_work_queue+0xe/0x40
[  113.264391]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.264785]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.265177]  ? vprintk_emit+0xf8/0x310
[  113.265496]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.265886]  ? _printk+0xc0/0xf0
[  113.266159]  ? __pfx__printk+0x10/0x10
[  113.266479]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.266870]  ? __dynamic_pr_debug+0x1f5/0x260
[  113.267232]  ? __pfx___dynamic_pr_debug+0x10/0x10
[  113.267616]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.268011]  cifs_get_inode_info+0x157/0x210 [cifs]
[  113.268807]  ? __pfx_cifs_get_inode_info+0x10/0x10 [cifs]
[  113.269642]  ? __pfx____ratelimit+0x10/0x10
[  113.269994]  cifs_revalidate_dentry_attr+0x2c5/0x450 [cifs]
[  113.270836]  ? __pfx_cifs_revalidate_dentry_attr+0x10/0x10 [cifs]
[  113.271720]  cifs_getattr+0x28f/0x450 [cifs]
[  113.272473]  vfs_statx+0x1f6/0x240
[  113.272760]  ? __pfx_vfs_statx+0x10/0x10
[  113.273083]  ? lock_acquire+0x14a/0x3a0
[  113.273405]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.273797]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.274192]  do_statx+0xac/0x110
[  113.274466]  ? __pfx_do_statx+0x10/0x10
[  113.274783]  ? __pfx_lock_release+0x10/0x10
[  113.275138]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.275532]  ? getname_flags.part.0+0xd6/0x260
[  113.275896]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.276289]  __x64_sys_statx+0xa0/0xc0
[  113.276603]  do_syscall_64+0x47/0xf0
[  113.276904]  entry_SYSCALL_64_after_hwframe+0x6f/0x77
[  113.277315] RIP: 0033:0x7f844366c8de
[  113.277611] Code: a5 fd ff ff e8 b1 e4 01 00 0f 1f 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 90 90 41 89 ca b8 4c 01 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2a 89 c1 85 c0 74 0f 48 8b 05 0d 05 0e 00 64
[  113.279073] RSP: 002b:00007fff09d8cdb8 EFLAGS: 00000246 ORIG_RAX: 000000000000014c
[  113.279677] RAX: ffffffffffffffda RBX: 000055f7065b37b8 RCX: 00007f844366c8de
[  113.280244] RDX: 0000000000000800 RSI: 00007fff09d8eb1c RDI: 00000000ffffff9c
[  113.280811] RBP: 0000000000000002 R08: 00007fff09d8cdc0 R09: 0000000000000001
[  113.281378] R10: 0000000000000002 R11: 0000000000000246 R12: 000055f7065b37b8
[  113.281946] R13: 000000000000002f R14: 00007fff09d8eb1a R15: 000055f7065b37a0
[  113.282525]  </TASK>
[  113.282713] 
[  113.282847] Allocated by task 880:
[  113.283128]  kasan_save_stack+0x33/0x60
[  113.283446]  kasan_set_track+0x25/0x30
[  113.283756]  __kasan_kmalloc+0x8f/0xa0
[  113.284065]  cifs_get_tcp_session+0xbc/0xc70 [cifs]
[  113.284850]  cifs_mount_get_session+0x70/0x220 [cifs]
[  113.285651]  dfs_mount_share+0xa60/0x1060 [cifs]
[  113.286420]  cifs_mount+0xda/0x4c0 [cifs]
[  113.287141]  cifs_smb3_do_mount+0x1e5/0xcc0 [cifs]
[  113.287919]  smb3_get_tree+0x16d/0x380 [cifs]
[  113.288670]  vfs_get_tree+0x4d/0x190
[  113.288973]  path_mount+0x3c4/0xf90
[  113.289265]  __x64_sys_mount+0x1aa/0x1f0
[  113.289589]  do_syscall_64+0x47/0xf0
[  113.289892]  entry_SYSCALL_64_after_hwframe+0x6f/0x77
[  113.290311] 
[  113.290447] Freed by task 893:
[  113.290702]  kasan_save_stack+0x33/0x60
[  113.291019]  kasan_set_track+0x25/0x30
[  113.291329]  kasan_save_free_info+0x2b/0x50
[  113.291673]  ____kasan_slab_free+0x126/0x170
[  113.292025]  slab_free_freelist_hook+0x9d/0x1e0
[  113.292400]  __kmem_cache_free+0x9d/0x190
[  113.292731]  clean_demultiplex_info+0x3bb/0x640 [cifs]
[  113.293536]  cifs_demultiplex_thread+0x3de/0x1270 [cifs]
[  113.294346]  kthread+0x18d/0x1d0
[  113.294616]  ret_from_fork+0x34/0x60
[  113.294912]  ret_from_fork_asm+0x1b/0x30
[  113.295234] 
[  113.295366] Last potentially related work creation:
[  113.295759]  kasan_save_stack+0x33/0x60
[  113.296075]  __kasan_record_aux_stack+0x94/0xa0
[  113.296442]  __queue_work+0x334/0x8a0
[  113.296746]  mod_delayed_work_on+0xa5/0x100
[  113.297087]  smb2_reconnect+0x735/0xcb0 [cifs]
[  113.297836]  SMB2_open_init+0xf8/0x13f0 [cifs]
[  113.298584]  smb2_compound_op+0x620/0x35b0 [cifs]
[  113.299363]  smb2_query_path_info+0x1c7/0x470 [cifs]
[  113.300154]  cifs_get_fattr+0x580/0xe70 [cifs]
[  113.300899]  cifs_get_inode_info+0x157/0x210 [cifs]
[  113.301677]  cifs_revalidate_dentry_attr+0x2c5/0x450 [cifs]
[  113.302512]  cifs_getattr+0x28f/0x450 [cifs]
[  113.303244]  vfs_statx+0x1f6/0x240
[  113.303523]  do_statx+0xac/0x110
[  113.303791]  __x64_sys_statx+0xa0/0xc0
[  113.304099]  do_syscall_64+0x47/0xf0
[  113.304395]  entry_SYSCALL_64_after_hwframe+0x6f/0x77
[  113.304803] 
[  113.304934] Second to last potentially related work creation:
[  113.305390]  kasan_save_stack+0x33/0x60
[  113.305711]  __kasan_record_aux_stack+0x94/0xa0
[  113.306078]  __queue_work+0x334/0x8a0
[  113.306380]  mod_delayed_work_on+0xa5/0x100
[  113.306722]  reconnect_dfs_server+0x69f/0x6b0 [cifs]
[  113.307505]  cifs_readv_from_socket+0x335/0x490 [cifs]
[  113.308300]  cifs_read_from_socket+0xb5/0x100 [cifs]
[  113.309085]  cifs_demultiplex_thread+0x274/0x1270 [cifs]
[  113.309894]  kthread+0x18d/0x1d0
[  113.310163]  ret_from_fork+0x34/0x60
[  113.310456]  ret_from_fork_asm+0x1b/0x30
[  113.310779] 
[  113.310912] The buggy address belongs to the object at ffff88800aa76000
[  113.310912]  which belongs to the cache kmalloc-4k of size 4096
[  113.311884] The buggy address is located 136 bytes inside of
[  113.311884]  freed 4096-byte region [ffff88800aa76000, ffff88800aa77000)
[  113.312845] 
[  113.312979] The buggy address belongs to the physical page:
[  113.313420] page:00000000c46c12fc refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0xaa70
[  113.314141] head:00000000c46c12fc order:3 entire_mapcount:0 nr_pages_mapped:0 pincount:0
[  113.314775] flags: 0x100000000000840(slab|head|node=0|zone=1)
[  113.315234] page_type: 0xffffffff()
[  113.315525] raw: 0100000000000840 ffff888006443040 ffffea0000430400 dead000000000002
[  113.316138] raw: 0000000000000000 0000000000040004 00000001ffffffff 0000000000000000
[  113.316747] page dumped because: kasan: bad access detected
[  113.317195] 
[  113.317329] Memory state around the buggy address:
[  113.317721]  ffff88800aa75f80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[  113.318305]  ffff88800aa76000: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[  113.318889] >ffff88800aa76080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[  113.319468]                       ^
[  113.319758]  ffff88800aa76100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[  113.320342]  ffff88800aa76180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[  113.320926] ==================================================================
[  113.321531] Kernel panic - not syncing: kasan.fault=panic set ...
[  113.322029] CPU: 2 PID: 942 Comm: ls Not tainted 6.7.0-rc1 #4
[  113.322507] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
[  113.323393] Call Trace:
[  113.323604]  <TASK>
[  113.323787]  dump_stack_lvl+0x4a/0x80
[  113.324099]  panic+0x41f/0x460
[  113.324363]  ? __pfx_panic+0x10/0x10
[  113.324665]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.325070]  ? check_panic_on_warn+0x2f/0x80
[  113.325437]  end_report+0x125/0x130
[  113.325737]  kasan_report+0xe8/0x110
[  113.326043]  ? cifs_get_fattr+0x546/0xe70 [cifs]
[  113.326827]  ? cifs_get_fattr+0x546/0xe70 [cifs]
[  113.327612]  cifs_get_fattr+0x546/0xe70 [cifs]
[  113.328408]  ? __pfx_cifs_get_fattr+0x10/0x10 [cifs]
[  113.329221]  ? irq_work_queue+0xe/0x40
[  113.329538]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.329935]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.330332]  ? vprintk_emit+0xf8/0x310
[  113.330650]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.331046]  ? _printk+0xc0/0xf0
[  113.331323]  ? __pfx__printk+0x10/0x10
[  113.331646]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.332046]  ? __dynamic_pr_debug+0x1f5/0x260
[  113.332416]  ? __pfx___dynamic_pr_debug+0x10/0x10
[  113.332807]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.333208]  cifs_get_inode_info+0x157/0x210 [cifs]
[  113.334013]  ? __pfx_cifs_get_inode_info+0x10/0x10 [cifs]
[  113.334859]  ? __pfx____ratelimit+0x10/0x10
[  113.335213]  cifs_revalidate_dentry_attr+0x2c5/0x450 [cifs]
[  113.336074]  ? __pfx_cifs_revalidate_dentry_attr+0x10/0x10 [cifs]
[  113.336973]  cifs_getattr+0x28f/0x450 [cifs]
[  113.337727]  vfs_statx+0x1f6/0x240
[  113.338017]  ? __pfx_vfs_statx+0x10/0x10
[  113.338343]  ? lock_acquire+0x14a/0x3a0
[  113.338665]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.339064]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.339463]  do_statx+0xac/0x110
[  113.339737]  ? __pfx_do_statx+0x10/0x10
[  113.340057]  ? __pfx_lock_release+0x10/0x10
[  113.340417]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.340813]  ? getname_flags.part.0+0xd6/0x260
[  113.341186]  ? srso_alias_return_thunk+0x5/0xfbef5
[  113.341585]  __x64_sys_statx+0xa0/0xc0
[  113.341903]  do_syscall_64+0x47/0xf0
[  113.342211]  entry_SYSCALL_64_after_hwframe+0x6f/0x77
[  113.342628] RIP: 0033:0x7f844366c8de
[  113.342926] Code: a5 fd ff ff e8 b1 e4 01 00 0f 1f 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 90 90 41 89 ca b8 4c 01 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2a 89 c1 85 c0 74 0f 48 8b 05 0d 05 0e 00 64
[  113.344532] RSP: 002b:00007fff09d8cdb8 EFLAGS: 00000246 ORIG_RAX: 000000000000014c
[  113.345218] RAX: ffffffffffffffda RBX: 000055f7065b37b8 RCX: 00007f844366c8de
[  113.345854] RDX: 0000000000000800 RSI: 00007fff09d8eb1c RDI: 00000000ffffff9c
[  113.346482] RBP: 0000000000000002 R08: 00007fff09d8cdc0 R09: 0000000000000001
[  113.347243] R10: 0000000000000002 R11: 0000000000000246 R12: 000055f7065b37b8
[  113.347984] R13: 000000000000002f R14: 00007fff09d8eb1a R15: 000055f7065b37a0
[  113.348624]  </TASK>
[  113.348974] Kernel Offset: disabled
[  113.349272] ---[ end Kernel panic - not syncing: kasan.fault=panic set ... ]---

~/g/linux (cifs.work)> ./scripts/faddr2line --list fs/smb/client/cifs.o cifs_get_fattr+0x546
cifs_get_fattr+0x546/0xe70:

cifs_get_fattr at /home/pc/g/linux/fs/smb/client/inode.c:1089
 1084           /*
 1085            * 1. Fetch file metadata if not provided (data)
 1086            */
 1087 
 1088           if (!data) {
>1089<                  rc = server->ops->query_path_info(xid, tcon, cifs_sb,
 1090                                                     full_path, &tmp_data);
 1091                   data = &tmp_data;
 1092           }
 1093 
 1094           /*

Bisect lead to this patch, which is

	19a4b9d6c372 ("cifs: reconnect work should have reference on server struct")

in v6.7-rc1.

Re: [PATCH 10/14] cifs: reconnect work should have reference on server struct

[Paulo Alcantara]

Steve French <smfrench@gmail.com> writes:

> I haven't seen the use after free you see - but I do see the mount/umount
> failure around tests generic/044 and generic/045 with and without
> multichannel (which predates 6.6, probably introduced in 6.3 or 6.4).  I
> also have seen some refcount warning in code paths called from query
> interfaces - but it hits it only a few of the tests in the full test run.

You haven't seen it because xfstests don't excercise reconnect or DFS
failover code paths.  It will only if client looses connection to server
while tests are running.

> Any thoughts on this warning?
>
> "[18326.477305] workqueue: smb2_deferred_work_close [cifs] hogged CPU for
>>10000us 8 times, consider switching to WQ_UNBOUND" (e.g. I saw it in test
> generic/048 without multichannel)

Never saw this one.  Can you bisect it?

[PATCH 11/14] cifs: handle when server starts supporting multichannel

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

When the user mounts with multichannel option, but the
server does not support it, there can be a time in future
where it can be supported.

With this change, such a case is handled.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/cifsproto.h |  4 ++++
 fs/smb/client/connect.c   |  6 +++++-
 fs/smb/client/smb2pdu.c   | 31 ++++++++++++++++++++++++++++---
 3 files changed, 37 insertions(+), 4 deletions(-)

diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
index 65c84b3d1a65..5a4c1f1e0d91 100644
--- a/fs/smb/client/cifsproto.h
+++ b/fs/smb/client/cifsproto.h
@@ -132,6 +132,10 @@ extern int SendReceiveBlockingLock(const unsigned int xid,
 			struct smb_hdr *in_buf,
 			struct smb_hdr *out_buf,
 			int *bytes_returned);
+
+void
+smb2_query_server_interfaces(struct work_struct *work);
+
 void
 cifs_signal_cifsd_for_reconnect(struct TCP_Server_Info *server,
 				      bool all_channels);
diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index e71aa33bf026..149cde77500e 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -116,7 +116,8 @@ static int reconn_set_ipaddr_from_hostname(struct TCP_Server_Info *server)
 	return rc;
 }
 
-static void smb2_query_server_interfaces(struct work_struct *work)
+void
+smb2_query_server_interfaces(struct work_struct *work)
 {
 	int rc;
 	int xid;
@@ -134,6 +135,9 @@ static void smb2_query_server_interfaces(struct work_struct *work)
 	if (rc) {
 		cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
 				__func__, rc);
+
+		if (rc == -EOPNOTSUPP)
+			return;
 	}
 
 	queue_delayed_work(cifsiod_wq, &tcon->query_interfaces,
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index b7665155f4e2..2617437a4627 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -163,6 +163,7 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 	int rc = 0;
 	struct nls_table *nls_codepage = NULL;
 	struct cifs_ses *ses;
+	int xid;
 
 	/*
 	 * SMB2s NegProt, SessSetup, Logoff do not have tcon yet so
@@ -307,17 +308,41 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 		tcon->need_reopen_files = true;
 
 	rc = cifs_tree_connect(0, tcon, nls_codepage);
-	mutex_unlock(&ses->session_mutex);
 
 	cifs_dbg(FYI, "reconnect tcon rc = %d\n", rc);
 	if (rc) {
 		/* If sess reconnected but tcon didn't, something strange ... */
+		mutex_unlock(&ses->session_mutex);
 		cifs_dbg(VFS, "reconnect tcon failed rc = %d\n", rc);
 		goto out;
 	}
 
-	if (smb2_command != SMB2_INTERNAL_CMD)
-		mod_delayed_work(cifsiod_wq, &server->reconnect, 0);
+	if (!rc &&
+	    (server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL)) {
+		mutex_unlock(&ses->session_mutex);
+
+		/*
+		 * query server network interfaces, in case they change
+		 */
+		xid = get_xid();
+		rc = SMB3_request_interfaces(xid, tcon, false);
+		free_xid(xid);
+
+		if (rc)
+			cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
+				 __func__, rc);
+
+		if (ses->chan_max > ses->chan_count &&
+		    !SERVER_IS_CHAN(server)) {
+			if (ses->chan_count == 1)
+				cifs_dbg(VFS, "server %s supports multichannel now\n",
+					 ses->server->hostname);
+
+			cifs_try_adding_channels(tcon->cifs_sb, ses);
+		}
+	} else {
+		mutex_unlock(&ses->session_mutex);
+	}
 
 	atomic_inc(&tconInfoReconnectCount);
 out:
-- 
2.34.1

Re: [PATCH 11/14] cifs: handle when server starts supporting multichannel

[Steve French]

merged into cifs-2.6.git for-next pending more testing

added cc:stable

On Mon, Oct 30, 2023 at 6:00 AM <nspmangalore@gmail.com> wrote:
>
> From: Shyam Prasad N <sprasad@microsoft.com>
>
> When the user mounts with multichannel option, but the
> server does not support it, there can be a time in future
> where it can be supported.
>
> With this change, such a case is handled.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/cifsproto.h |  4 ++++
>  fs/smb/client/connect.c   |  6 +++++-
>  fs/smb/client/smb2pdu.c   | 31 ++++++++++++++++++++++++++++---
>  3 files changed, 37 insertions(+), 4 deletions(-)
>
> diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
> index 65c84b3d1a65..5a4c1f1e0d91 100644
> --- a/fs/smb/client/cifsproto.h
> +++ b/fs/smb/client/cifsproto.h
> @@ -132,6 +132,10 @@ extern int SendReceiveBlockingLock(const unsigned int xid,
>                         struct smb_hdr *in_buf,
>                         struct smb_hdr *out_buf,
>                         int *bytes_returned);
> +
> +void
> +smb2_query_server_interfaces(struct work_struct *work);
> +
>  void
>  cifs_signal_cifsd_for_reconnect(struct TCP_Server_Info *server,
>                                       bool all_channels);
> diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
> index e71aa33bf026..149cde77500e 100644
> --- a/fs/smb/client/connect.c
> +++ b/fs/smb/client/connect.c
> @@ -116,7 +116,8 @@ static int reconn_set_ipaddr_from_hostname(struct TCP_Server_Info *server)
>         return rc;
>  }
>
> -static void smb2_query_server_interfaces(struct work_struct *work)
> +void
> +smb2_query_server_interfaces(struct work_struct *work)
>  {
>         int rc;
>         int xid;
> @@ -134,6 +135,9 @@ static void smb2_query_server_interfaces(struct work_struct *work)
>         if (rc) {
>                 cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
>                                 __func__, rc);
> +
> +               if (rc == -EOPNOTSUPP)
> +                       return;
>         }
>
>         queue_delayed_work(cifsiod_wq, &tcon->query_interfaces,
> diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
> index b7665155f4e2..2617437a4627 100644
> --- a/fs/smb/client/smb2pdu.c
> +++ b/fs/smb/client/smb2pdu.c
> @@ -163,6 +163,7 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
>         int rc = 0;
>         struct nls_table *nls_codepage = NULL;
>         struct cifs_ses *ses;
> +       int xid;
>
>         /*
>          * SMB2s NegProt, SessSetup, Logoff do not have tcon yet so
> @@ -307,17 +308,41 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
>                 tcon->need_reopen_files = true;
>
>         rc = cifs_tree_connect(0, tcon, nls_codepage);
> -       mutex_unlock(&ses->session_mutex);
>
>         cifs_dbg(FYI, "reconnect tcon rc = %d\n", rc);
>         if (rc) {
>                 /* If sess reconnected but tcon didn't, something strange ... */
> +               mutex_unlock(&ses->session_mutex);
>                 cifs_dbg(VFS, "reconnect tcon failed rc = %d\n", rc);
>                 goto out;
>         }
>
> -       if (smb2_command != SMB2_INTERNAL_CMD)
> -               mod_delayed_work(cifsiod_wq, &server->reconnect, 0);
> +       if (!rc &&
> +           (server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL)) {
> +               mutex_unlock(&ses->session_mutex);
> +
> +               /*
> +                * query server network interfaces, in case they change
> +                */
> +               xid = get_xid();
> +               rc = SMB3_request_interfaces(xid, tcon, false);
> +               free_xid(xid);
> +
> +               if (rc)
> +                       cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
> +                                __func__, rc);
> +
> +               if (ses->chan_max > ses->chan_count &&
> +                   !SERVER_IS_CHAN(server)) {
> +                       if (ses->chan_count == 1)
> +                               cifs_dbg(VFS, "server %s supports multichannel now\n",
> +                                        ses->server->hostname);
> +
> +                       cifs_try_adding_channels(tcon->cifs_sb, ses);
> +               }
> +       } else {
> +               mutex_unlock(&ses->session_mutex);
> +       }
>
>         atomic_inc(&tconInfoReconnectCount);
>  out:
> --
> 2.34.1
>


-- 
Thanks,

Steve

Re: [PATCH 11/14] cifs: handle when server starts supporting multichannel

[Paulo Alcantara]

nspmangalore@gmail.com writes:

> From: Shyam Prasad N <sprasad@microsoft.com>
>
> When the user mounts with multichannel option, but the
> server does not support it, there can be a time in future
> where it can be supported.
>
> With this change, such a case is handled.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/cifsproto.h |  4 ++++
>  fs/smb/client/connect.c   |  6 +++++-
>  fs/smb/client/smb2pdu.c   | 31 ++++++++++++++++++++++++++++---
>  3 files changed, 37 insertions(+), 4 deletions(-)
>
> diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
> index 65c84b3d1a65..5a4c1f1e0d91 100644
> --- a/fs/smb/client/cifsproto.h
> +++ b/fs/smb/client/cifsproto.h
> @@ -132,6 +132,10 @@ extern int SendReceiveBlockingLock(const unsigned int xid,
>  			struct smb_hdr *in_buf,
>  			struct smb_hdr *out_buf,
>  			int *bytes_returned);
> +
> +void
> +smb2_query_server_interfaces(struct work_struct *work);
> +

Why are you exporting this?  smb2_query_server_interfaces() seems to be
used only in fs/smb/client/connect.c.

>  void
>  cifs_signal_cifsd_for_reconnect(struct TCP_Server_Info *server,
>  				      bool all_channels);
> diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
> index e71aa33bf026..149cde77500e 100644
> --- a/fs/smb/client/connect.c
> +++ b/fs/smb/client/connect.c
> @@ -116,7 +116,8 @@ static int reconn_set_ipaddr_from_hostname(struct TCP_Server_Info *server)
>  	return rc;
>  }
>  
> -static void smb2_query_server_interfaces(struct work_struct *work)
> +void
> +smb2_query_server_interfaces(struct work_struct *work)
>  {

Ditto.

>  	int rc;
>  	int xid;
> @@ -134,6 +135,9 @@ static void smb2_query_server_interfaces(struct work_struct *work)
>  	if (rc) {
>  		cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
>  				__func__, rc);
> +
> +		if (rc == -EOPNOTSUPP)
> +			return;

Maybe also get rid of cifs_dbg() when rc == -EOPNOTSUPP?

>  	}
>  
>  	queue_delayed_work(cifsiod_wq, &tcon->query_interfaces,
> diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
> index b7665155f4e2..2617437a4627 100644
> --- a/fs/smb/client/smb2pdu.c
> +++ b/fs/smb/client/smb2pdu.c
> @@ -163,6 +163,7 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
>  	int rc = 0;
>  	struct nls_table *nls_codepage = NULL;
>  	struct cifs_ses *ses;
> +	int xid;
>  
>  	/*
>  	 * SMB2s NegProt, SessSetup, Logoff do not have tcon yet so
> @@ -307,17 +308,41 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
>  		tcon->need_reopen_files = true;
>  
>  	rc = cifs_tree_connect(0, tcon, nls_codepage);
> -	mutex_unlock(&ses->session_mutex);
>  
>  	cifs_dbg(FYI, "reconnect tcon rc = %d\n", rc);
>  	if (rc) {
>  		/* If sess reconnected but tcon didn't, something strange ... */
> +		mutex_unlock(&ses->session_mutex);
>  		cifs_dbg(VFS, "reconnect tcon failed rc = %d\n", rc);
>  		goto out;
>  	}
>  
> -	if (smb2_command != SMB2_INTERNAL_CMD)
> -		mod_delayed_work(cifsiod_wq, &server->reconnect, 0);

Why are you removing this optimisation?  For example, session/tcon
reconnect will no longer be triggered after returning back to userspace,
only when next I/O comes in.

Re: [PATCH 11/14] cifs: handle when server starts supporting multichannel

[Shyam Prasad N]

On Wed, Nov 1, 2023 at 9:22 PM Paulo Alcantara <pc@manguebit.com> wrote:
>
> nspmangalore@gmail.com writes:
>
> > From: Shyam Prasad N <sprasad@microsoft.com>
> >
> > When the user mounts with multichannel option, but the
> > server does not support it, there can be a time in future
> > where it can be supported.
> >
> > With this change, such a case is handled.
> >
> > Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> > ---
> >  fs/smb/client/cifsproto.h |  4 ++++
> >  fs/smb/client/connect.c   |  6 +++++-
> >  fs/smb/client/smb2pdu.c   | 31 ++++++++++++++++++++++++++++---
> >  3 files changed, 37 insertions(+), 4 deletions(-)
> >
> > diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
> > index 65c84b3d1a65..5a4c1f1e0d91 100644
> > --- a/fs/smb/client/cifsproto.h
> > +++ b/fs/smb/client/cifsproto.h
> > @@ -132,6 +132,10 @@ extern int SendReceiveBlockingLock(const unsigned int xid,
> >                       struct smb_hdr *in_buf,
> >                       struct smb_hdr *out_buf,
> >                       int *bytes_returned);
> > +
> > +void
> > +smb2_query_server_interfaces(struct work_struct *work);
> > +
>
> Why are you exporting this?  smb2_query_server_interfaces() seems to be
> used only in fs/smb/client/connect.c.

In an earlier version of this change, I was calling
smb2_query_server_interfaces from smb2_reconnect when multichannel is
reenabled.
That needed the export. However, I had to change that.
I will reset this back.

>
> >  void
> >  cifs_signal_cifsd_for_reconnect(struct TCP_Server_Info *server,
> >                                     bool all_channels);
> > diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
> > index e71aa33bf026..149cde77500e 100644
> > --- a/fs/smb/client/connect.c
> > +++ b/fs/smb/client/connect.c
> > @@ -116,7 +116,8 @@ static int reconn_set_ipaddr_from_hostname(struct TCP_Server_Info *server)
> >       return rc;
> >  }
> >
> > -static void smb2_query_server_interfaces(struct work_struct *work)
> > +void
> > +smb2_query_server_interfaces(struct work_struct *work)
> >  {
>
> Ditto.
>
> >       int rc;
> >       int xid;
> > @@ -134,6 +135,9 @@ static void smb2_query_server_interfaces(struct work_struct *work)
> >       if (rc) {
> >               cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
> >                               __func__, rc);
> > +
> > +             if (rc == -EOPNOTSUPP)
> > +                     return;
>
> Maybe also get rid of cifs_dbg() when rc == -EOPNOTSUPP?
>
> >       }
> >
> >       queue_delayed_work(cifsiod_wq, &tcon->query_interfaces,
> > diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
> > index b7665155f4e2..2617437a4627 100644
> > --- a/fs/smb/client/smb2pdu.c
> > +++ b/fs/smb/client/smb2pdu.c
> > @@ -163,6 +163,7 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
> >       int rc = 0;
> >       struct nls_table *nls_codepage = NULL;
> >       struct cifs_ses *ses;
> > +     int xid;
> >
> >       /*
> >        * SMB2s NegProt, SessSetup, Logoff do not have tcon yet so
> > @@ -307,17 +308,41 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
> >               tcon->need_reopen_files = true;
> >
> >       rc = cifs_tree_connect(0, tcon, nls_codepage);
> > -     mutex_unlock(&ses->session_mutex);
> >
> >       cifs_dbg(FYI, "reconnect tcon rc = %d\n", rc);
> >       if (rc) {
> >               /* If sess reconnected but tcon didn't, something strange ... */
> > +             mutex_unlock(&ses->session_mutex);
> >               cifs_dbg(VFS, "reconnect tcon failed rc = %d\n", rc);
> >               goto out;
> >       }
> >
> > -     if (smb2_command != SMB2_INTERNAL_CMD)
> > -             mod_delayed_work(cifsiod_wq, &server->reconnect, 0);
>
> Why are you removing this optimisation?  For example, session/tcon
> reconnect will no longer be triggered after returning back to userspace,
> only when next I/O comes in.



-- 
Regards,
Shyam

Re: [PATCH 11/14] cifs: handle when server starts supporting multichannel

[Paulo Alcantara]

nspmangalore@gmail.com writes:

> @@ -307,17 +308,41 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
>  		tcon->need_reopen_files = true;
>  
>  	rc = cifs_tree_connect(0, tcon, nls_codepage);
> -	mutex_unlock(&ses->session_mutex);
>  
>  	cifs_dbg(FYI, "reconnect tcon rc = %d\n", rc);
>  	if (rc) {
>  		/* If sess reconnected but tcon didn't, something strange ... */
> +		mutex_unlock(&ses->session_mutex);
>  		cifs_dbg(VFS, "reconnect tcon failed rc = %d\n", rc);
>  		goto out;
>  	}
>  
> -	if (smb2_command != SMB2_INTERNAL_CMD)
> -		mod_delayed_work(cifsiod_wq, &server->reconnect, 0);
> +	if (!rc &&
> +	    (server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL)) {
> +		mutex_unlock(&ses->session_mutex);
> +
> +		/*
> +		 * query server network interfaces, in case they change
> +		 */
> +		xid = get_xid();
> +		rc = SMB3_request_interfaces(xid, tcon, false);
> +		free_xid(xid);
> +
> +		if (rc)
> +			cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
> +				 __func__, rc);
> +
> +		if (ses->chan_max > ses->chan_count &&
> +		    !SERVER_IS_CHAN(server)) {
> +			if (ses->chan_count == 1)
> +				cifs_dbg(VFS, "server %s supports multichannel now\n",
> +					 ses->server->hostname);

Sorry, forgot to mention that you should call cifs_dbg_server() which
protects access of @server->hostname as cifsd thread could have it freed
before you access it.

Re: [PATCH 11/14] cifs: handle when server starts supporting multichannel

[Steve French]

[-- Attachment #1: Type: text/plain, Size: 2299 bytes --]

removed cc:stable and changed

> +                             cifs_dbg(VFS, "server %s supports multichannel now\n",
> +                                      ses->server->hostname);

to`

+                               cifs_server_dbg(VFS, "supports
multichannel now\n");

Let me know if that is ok for you.  (See attached updated patch)


On Thu, Nov 2, 2023 at 3:28 PM Paulo Alcantara <pc@manguebit.com> wrote:
>
> nspmangalore@gmail.com writes:
>
> > @@ -307,17 +308,41 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
> >               tcon->need_reopen_files = true;
> >
> >       rc = cifs_tree_connect(0, tcon, nls_codepage);
> > -     mutex_unlock(&ses->session_mutex);
> >
> >       cifs_dbg(FYI, "reconnect tcon rc = %d\n", rc);
> >       if (rc) {
> >               /* If sess reconnected but tcon didn't, something strange ... */
> > +             mutex_unlock(&ses->session_mutex);
> >               cifs_dbg(VFS, "reconnect tcon failed rc = %d\n", rc);
> >               goto out;
> >       }
> >
> > -     if (smb2_command != SMB2_INTERNAL_CMD)
> > -             mod_delayed_work(cifsiod_wq, &server->reconnect, 0);
> > +     if (!rc &&
> > +         (server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL)) {
> > +             mutex_unlock(&ses->session_mutex);
> > +
> > +             /*
> > +              * query server network interfaces, in case they change
> > +              */
> > +             xid = get_xid();
> > +             rc = SMB3_request_interfaces(xid, tcon, false);
> > +             free_xid(xid);
> > +
> > +             if (rc)
> > +                     cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
> > +                              __func__, rc);
> > +
> > +             if (ses->chan_max > ses->chan_count &&
> > +                 !SERVER_IS_CHAN(server)) {
> > +                     if (ses->chan_count == 1)
> > +                             cifs_dbg(VFS, "server %s supports multichannel now\n",
> > +                                      ses->server->hostname);
>
> Sorry, forgot to mention that you should call cifs_dbg_server() which
> protects access of @server->hostname as cifsd thread could have it freed
> before you access it.



--
Thanks,

Steve

[-- Attachment #2: 0001-cifs-handle-when-server-starts-supporting-multichann.patch --]
[-- Type: text/x-patch, Size: 3595 bytes --]

From d012d89c90a47f1250956ef2d9b515927bcc4ec9 Mon Sep 17 00:00:00 2001
From: Shyam Prasad N <sprasad@microsoft.com>
Date: Mon, 30 Oct 2023 11:00:17 +0000
Subject: [PATCH] cifs: handle when server starts supporting multichannel

When the user mounts with multichannel option, but the
server does not support it, there can be a time in future
where it can be supported.

With this change, such a case is handled.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
Signed-off-by: Steve French <stfrench@microsoft.com>
---
 fs/smb/client/cifsproto.h |  4 ++++
 fs/smb/client/connect.c   |  6 +++++-
 fs/smb/client/smb2pdu.c   | 30 +++++++++++++++++++++++++++---
 3 files changed, 36 insertions(+), 4 deletions(-)

diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
index 890ceddae07e..bb3bf3cac828 100644
--- a/fs/smb/client/cifsproto.h
+++ b/fs/smb/client/cifsproto.h
@@ -132,6 +132,10 @@ extern int SendReceiveBlockingLock(const unsigned int xid,
 			struct smb_hdr *in_buf,
 			struct smb_hdr *out_buf,
 			int *bytes_returned);
+
+void
+smb2_query_server_interfaces(struct work_struct *work);
+
 void
 cifs_signal_cifsd_for_reconnect(struct TCP_Server_Info *server,
 				      bool all_channels);
diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index 6c7701422d4c..6d6e334a079b 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -116,7 +116,8 @@ static int reconn_set_ipaddr_from_hostname(struct TCP_Server_Info *server)
 	return rc;
 }
 
-static void smb2_query_server_interfaces(struct work_struct *work)
+void
+smb2_query_server_interfaces(struct work_struct *work)
 {
 	int rc;
 	int xid;
@@ -134,6 +135,9 @@ static void smb2_query_server_interfaces(struct work_struct *work)
 	if (rc) {
 		cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
 				__func__, rc);
+
+		if (rc == -EOPNOTSUPP)
+			return;
 	}
 
 	queue_delayed_work(cifsiod_wq, &tcon->query_interfaces,
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index c75a80bb6d9e..3a1651772486 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -163,6 +163,7 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 	int rc = 0;
 	struct nls_table *nls_codepage = NULL;
 	struct cifs_ses *ses;
+	int xid;
 
 	/*
 	 * SMB2s NegProt, SessSetup, Logoff do not have tcon yet so
@@ -307,17 +308,40 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 		tcon->need_reopen_files = true;
 
 	rc = cifs_tree_connect(0, tcon, nls_codepage);
-	mutex_unlock(&ses->session_mutex);
 
 	cifs_dbg(FYI, "reconnect tcon rc = %d\n", rc);
 	if (rc) {
 		/* If sess reconnected but tcon didn't, something strange ... */
+		mutex_unlock(&ses->session_mutex);
 		cifs_dbg(VFS, "reconnect tcon failed rc = %d\n", rc);
 		goto out;
 	}
 
-	if (smb2_command != SMB2_INTERNAL_CMD)
-		mod_delayed_work(cifsiod_wq, &server->reconnect, 0);
+	if (!rc &&
+	    (server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL)) {
+		mutex_unlock(&ses->session_mutex);
+
+		/*
+		 * query server network interfaces, in case they change
+		 */
+		xid = get_xid();
+		rc = SMB3_request_interfaces(xid, tcon, false);
+		free_xid(xid);
+
+		if (rc)
+			cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
+				 __func__, rc);
+
+		if (ses->chan_max > ses->chan_count &&
+		    !SERVER_IS_CHAN(server)) {
+			if (ses->chan_count == 1)
+				cifs_server_dbg(VFS, "supports multichannel now\n");
+
+			cifs_try_adding_channels(tcon->cifs_sb, ses);
+		}
+	} else {
+		mutex_unlock(&ses->session_mutex);
+	}
 
 	atomic_inc(&tconInfoReconnectCount);
 out:
-- 
2.39.2

Re: [PATCH 11/14] cifs: handle when server starts supporting multichannel

[Paulo Alcantara]

Steve French <smfrench@gmail.com> writes:

> removed cc:stable and changed
>
>> +                             cifs_dbg(VFS, "server %s supports multichannel now\n",
>> +                                      ses->server->hostname);
>
> to`
>
> +                               cifs_server_dbg(VFS, "supports
> multichannel now\n");

Looks good, thanks.

> Let me know if that is ok for you.  (See attached updated patch)

For the s/cifs_dbg/cifs_server_dbg/ change, it is.

Re: [PATCH 11/14] cifs: handle when server starts supporting multichannel

[Paulo Alcantara]

Paulo Alcantara <pc@manguebit.com> writes:

> Steve French <smfrench@gmail.com> writes:
>
>> removed cc:stable and changed
>>
>>> +                             cifs_dbg(VFS, "server %s supports multichannel now\n",
>>> +                                      ses->server->hostname);
>>
>> to`
>>
>> +                               cifs_server_dbg(VFS, "supports
>> multichannel now\n");
>
> Looks good, thanks.
>
>> Let me know if that is ok for you.  (See attached updated patch)
>
> For the s/cifs_dbg/cifs_server_dbg/ change, it is.

BTW, this patch in for-next branch still contains

        (1) misleading export of smb2_query_server_interfaces()

        (2) removal of mod_delayed_work() when reconnect failed

        (3) logging of failed network interface queries even when server
        doesn't support it.

Re: [PATCH 11/14] cifs: handle when server starts supporting multichannel

[Steve French]

[-- Attachment #1: Type: text/plain, Size: 1431 bytes --]

I just updated the patch in for-next (see attached)

I removed the unneeded export of smb2_query_server_interfaces() but it
looks like the logging of failed network interface queries is fine
(and is an FYI message so not turned on automatically) - I could
definitely see cases where FYI logging is turned on to see why query
server interfaces failed even in the EOPNOTSUPP case - so the debug
FYI looks ok to me


On Wed, Nov 8, 2023 at 10:02 AM Paulo Alcantara <pc@manguebit.com> wrote:
>
> Paulo Alcantara <pc@manguebit.com> writes:
>
> > Steve French <smfrench@gmail.com> writes:
> >
> >> removed cc:stable and changed
> >>
> >>> +                             cifs_dbg(VFS, "server %s supports multichannel now\n",
> >>> +                                      ses->server->hostname);
> >>
> >> to`
> >>
> >> +                               cifs_server_dbg(VFS, "supports
> >> multichannel now\n");
> >
> > Looks good, thanks.
> >
> >> Let me know if that is ok for you.  (See attached updated patch)
> >
> > For the s/cifs_dbg/cifs_server_dbg/ change, it is.
>
> BTW, this patch in for-next branch still contains
>
>         (1) misleading export of smb2_query_server_interfaces()
>
>         (2) removal of mod_delayed_work() when reconnect failed
>
>         (3) logging of failed network interface queries even when server
>         doesn't support it.
>


-- 
Thanks,

Steve

[-- Attachment #2: 0001-cifs-handle-when-server-starts-supporting-multichann.patch --]
[-- Type: text/x-patch, Size: 2766 bytes --]

From 07a290c9d27bdedfba3cf0910768969ea553157e Mon Sep 17 00:00:00 2001
From: Shyam Prasad N <sprasad@microsoft.com>
Date: Fri, 13 Oct 2023 11:33:21 +0000
Subject: [PATCH] cifs: handle when server starts supporting multichannel

When the user mounts with multichannel option, but the
server does not support it, there can be a time in future
where it can be supported.

With this change, such a case is handled.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
Signed-off-by: Steve French <stfrench@microsoft.com>
---
 fs/smb/client/connect.c |  3 +++
 fs/smb/client/smb2pdu.c | 30 +++++++++++++++++++++++++++---
 2 files changed, 30 insertions(+), 3 deletions(-)

diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index b514b41cc9f0..55cf70800bb5 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -134,6 +134,9 @@ static void smb2_query_server_interfaces(struct work_struct *work)
 	if (rc) {
 		cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
 				__func__, rc);
+
+		if (rc == -EOPNOTSUPP)
+			return;
 	}
 
 	queue_delayed_work(cifsiod_wq, &tcon->query_interfaces,
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index b7665155f4e2..65636bdf6cdf 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -163,6 +163,7 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 	int rc = 0;
 	struct nls_table *nls_codepage = NULL;
 	struct cifs_ses *ses;
+	int xid;
 
 	/*
 	 * SMB2s NegProt, SessSetup, Logoff do not have tcon yet so
@@ -307,17 +308,40 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 		tcon->need_reopen_files = true;
 
 	rc = cifs_tree_connect(0, tcon, nls_codepage);
-	mutex_unlock(&ses->session_mutex);
 
 	cifs_dbg(FYI, "reconnect tcon rc = %d\n", rc);
 	if (rc) {
 		/* If sess reconnected but tcon didn't, something strange ... */
+		mutex_unlock(&ses->session_mutex);
 		cifs_dbg(VFS, "reconnect tcon failed rc = %d\n", rc);
 		goto out;
 	}
 
-	if (smb2_command != SMB2_INTERNAL_CMD)
-		mod_delayed_work(cifsiod_wq, &server->reconnect, 0);
+	if (!rc &&
+	    (server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL)) {
+		mutex_unlock(&ses->session_mutex);
+
+		/*
+		 * query server network interfaces, in case they change
+		 */
+		xid = get_xid();
+		rc = SMB3_request_interfaces(xid, tcon, false);
+		free_xid(xid);
+
+		if (rc)
+			cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
+				 __func__, rc);
+
+		if (ses->chan_max > ses->chan_count &&
+		    !SERVER_IS_CHAN(server)) {
+			if (ses->chan_count == 1)
+				cifs_server_dbg(VFS, "supports multichannel now\n");
+
+			cifs_try_adding_channels(ses);
+		}
+	} else {
+		mutex_unlock(&ses->session_mutex);
+	}
 
 	atomic_inc(&tconInfoReconnectCount);
 out:
-- 
2.39.2

Re: [PATCH 11/14] cifs: handle when server starts supporting multichannel

[Paulo Alcantara]

Steve French <smfrench@gmail.com> writes:

> I just updated the patch in for-next (see attached)

Thanks.

> I removed the unneeded export of smb2_query_server_interfaces() but it
> looks like the logging of failed network interface queries is fine
> (and is an FYI message so not turned on automatically) - I could
> definitely see cases where FYI logging is turned on to see why query
> server interfaces failed even in the EOPNOTSUPP case - so the debug
> FYI looks ok to me

Yeah, no problem.

[PATCH 12/14] cifs: handle when server stops supporting multichannel

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

When a server stops supporting multichannel, we will
keep attempting reconnects to the secondary channels today.
Avoid this by freeing extra channels when negotiate
returns no multichannel support.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/cifsglob.h  |  1 +
 fs/smb/client/cifsproto.h |  2 ++
 fs/smb/client/connect.c   | 10 ++++++
 fs/smb/client/sess.c      | 64 ++++++++++++++++++++++++++++++-----
 fs/smb/client/smb2pdu.c   | 70 ++++++++++++++++++++++++++++++++++++++-
 fs/smb/client/transport.c |  2 +-
 6 files changed, 139 insertions(+), 10 deletions(-)

diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
index cdbc2cd207dc..1e1a5f3ab600 100644
--- a/fs/smb/client/cifsglob.h
+++ b/fs/smb/client/cifsglob.h
@@ -650,6 +650,7 @@ struct TCP_Server_Info {
 	bool noautotune;		/* do not autotune send buf sizes */
 	bool nosharesock;
 	bool tcp_nodelay;
+	bool terminate;
 	unsigned int credits;  /* send no more requests at once */
 	unsigned int max_credits; /* can override large 32000 default at mnt */
 	unsigned int in_flight;  /* number of requests on the wire to server */
diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
index 5a4c1f1e0d91..828c3916cb88 100644
--- a/fs/smb/client/cifsproto.h
+++ b/fs/smb/client/cifsproto.h
@@ -644,6 +644,8 @@ cifs_chan_needs_reconnect(struct cifs_ses *ses,
 bool
 cifs_chan_is_iface_active(struct cifs_ses *ses,
 			  struct TCP_Server_Info *server);
+void
+cifs_disable_secondary_channels(struct cifs_ses *ses);
 int
 cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server);
 int
diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index 149cde77500e..6fe80d5d7c23 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -220,6 +220,14 @@ cifs_mark_tcp_ses_conns_for_reconnect(struct TCP_Server_Info *server,
 
 	spin_lock(&cifs_tcp_ses_lock);
 	list_for_each_entry_safe(ses, nses, &pserver->smb_ses_list, smb_ses_list) {
+		/*
+		 * if channel has been marked for termination, nothing to do
+		 * for the channel. in fact, we cannot find the channel for the
+		 * server. So safe to exit here
+		 */
+		if (server->terminate)
+			break;
+
 		/* check if iface is still active */
 		if (!cifs_chan_is_iface_active(ses, server))
 			cifs_chan_update_iface(ses, server);
@@ -254,6 +262,8 @@ cifs_mark_tcp_ses_conns_for_reconnect(struct TCP_Server_Info *server,
 			spin_lock(&tcon->tc_lock);
 			tcon->status = TID_NEED_RECON;
 			spin_unlock(&tcon->tc_lock);
+
+			cancel_delayed_work_sync(&tcon->query_interfaces);
 		}
 		if (ses->tcon_ipc) {
 			ses->tcon_ipc->need_reconnect = true;
diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
index 6843deed6119..51bcd78fc5bf 100644
--- a/fs/smb/client/sess.c
+++ b/fs/smb/client/sess.c
@@ -288,6 +288,60 @@ int cifs_try_adding_channels(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses)
 	return new_chan_count - old_chan_count;
 }
 
+/*
+ * called when multichannel is disabled by the server.
+ * this always gets called from smb2_reconnect
+ * and cannot get called in parallel threads.
+ */
+void
+cifs_disable_secondary_channels(struct cifs_ses *ses)
+{
+	int i, chan_count;
+	struct TCP_Server_Info *server;
+	struct cifs_server_iface *iface;
+
+	spin_lock(&ses->chan_lock);
+	chan_count = ses->chan_count;
+	if (chan_count == 1)
+		goto done;
+
+	ses->chan_count = 1;
+
+	/* for all secondary channels reset the need reconnect bit */
+	ses->chans_need_reconnect &= 1;
+
+	for (i = 1; i < chan_count; i++) {
+		iface = ses->chans[i].iface;
+		server = ses->chans[i].server;
+
+		if (iface) {
+			spin_lock(&ses->iface_lock);
+			kref_put(&iface->refcount, release_iface);
+			ses->chans[i].iface = NULL;
+			iface->num_channels--;
+			if (iface->weight_fulfilled)
+				iface->weight_fulfilled--;
+			spin_unlock(&ses->iface_lock);
+		}
+
+		spin_unlock(&ses->chan_lock);
+		if (server && !server->terminate) {
+			server->terminate = true;
+			cifs_signal_cifsd_for_reconnect(server, false);
+		}
+		spin_lock(&ses->chan_lock);
+
+		if (server) {
+			ses->chans[i].server = NULL;
+			cifs_put_tcp_session(server, false);
+		}
+
+	}
+
+done:
+	spin_unlock(&ses->chan_lock);
+}
+
 /*
  * update the iface for the channel if necessary.
  * will return 0 when iface is updated, 1 if removed, 2 otherwise
@@ -580,14 +634,10 @@ cifs_ses_add_channel(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses,
 
 out:
 	if (rc && chan->server) {
-		/*
-		 * we should avoid race with these delayed works before we
-		 * remove this channel
-		 */
-		cancel_delayed_work_sync(&chan->server->echo);
-		cancel_delayed_work_sync(&chan->server->reconnect);
+		cifs_put_tcp_session(chan->server, 0);
 
 		spin_lock(&ses->chan_lock);
+
 		/* we rely on all bits beyond chan_count to be clear */
 		cifs_chan_clear_need_reconnect(ses, chan->server);
 		ses->chan_count--;
@@ -597,8 +647,6 @@ cifs_ses_add_channel(struct cifs_sb_info *cifs_sb, struct cifs_ses *ses,
 		 */
 		WARN_ON(ses->chan_count < 1);
 		spin_unlock(&ses->chan_lock);
-
-		cifs_put_tcp_session(chan->server, 0);
 	}
 
 	kfree(ctx->UNC);
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index 2617437a4627..d1d4d9100870 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -164,6 +164,8 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 	struct nls_table *nls_codepage = NULL;
 	struct cifs_ses *ses;
 	int xid;
+	struct TCP_Server_Info *pserver;
+	unsigned int chan_index;
 
 	/*
 	 * SMB2s NegProt, SessSetup, Logoff do not have tcon yet so
@@ -224,6 +226,12 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 			return -EAGAIN;
 		}
 	}
+
+	/* if server is marked for termination, cifsd will cleanup */
+	if (server->terminate) {
+		spin_unlock(&server->srv_lock);
+		return -EHOSTDOWN;
+	}
 	spin_unlock(&server->srv_lock);
 
 again:
@@ -242,12 +250,24 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 		 tcon->need_reconnect);
 
 	mutex_lock(&ses->session_mutex);
+	/*
+	 * if this is called by delayed work, and the channel has been disabled
+	 * in parallel, the delayed work can continue to execute in parallel
+	 * there's a chance that this channel may not exist anymore
+	 */
+	spin_lock(&server->srv_lock);
+	if (server->tcpStatus == CifsExiting) {
+		spin_unlock(&server->srv_lock);
+		mutex_unlock(&ses->session_mutex);
+		rc = -EHOSTDOWN;
+		goto out;
+	}
+
 	/*
 	 * Recheck after acquire mutex. If another thread is negotiating
 	 * and the server never sends an answer the socket will be closed
 	 * and tcpStatus set to reconnect.
 	 */
-	spin_lock(&server->srv_lock);
 	if (server->tcpStatus == CifsNeedReconnect) {
 		spin_unlock(&server->srv_lock);
 		mutex_unlock(&ses->session_mutex);
@@ -284,6 +304,47 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 
 	rc = cifs_negotiate_protocol(0, ses, server);
 	if (!rc) {
+		/*
+		 * if server stopped supporting multichannel
+		 * and the first channel reconnected, disable all the others.
+		 */
+		if (ses->chan_count > 1 &&
+		    !(server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL)) {
+			if (SERVER_IS_CHAN(server)) {
+				cifs_dbg(VFS, "server %s does not support " \
+					 "multichannel anymore. skipping secondary channel\n",
+					 ses->server->hostname);
+
+				spin_lock(&ses->chan_lock);
+				chan_index = cifs_ses_get_chan_index(ses, server);
+				if (chan_index == CIFS_INVAL_CHAN_INDEX) {
+					spin_unlock(&ses->chan_lock);
+					goto skip_terminate;
+				}
+
+				ses->chans[chan_index].server = NULL;
+				cifs_put_tcp_session(server, 1);
+				spin_unlock(&ses->chan_lock);
+
+				server->terminate = true;
+				cifs_signal_cifsd_for_reconnect(server, false);
+
+				/* mark primary server as needing reconnect */
+				pserver = server->primary_server;
+				cifs_signal_cifsd_for_reconnect(pserver, false);
+
+skip_terminate:
+				mutex_unlock(&ses->session_mutex);
+				rc = -EHOSTDOWN;
+				goto out;
+			} else {
+				cifs_dbg(VFS, "server %s does not support " \
+					 "multichannel anymore. disabling all other channels\n",
+					 ses->server->hostname);
+				cifs_disable_secondary_channels(ses);
+			}
+		}
+
 		rc = cifs_setup_session(0, ses, server, nls_codepage);
 		if ((rc == -EACCES) && !tcon->retry) {
 			mutex_unlock(&ses->session_mutex);
@@ -3833,6 +3894,13 @@ void smb2_reconnect_server(struct work_struct *work)
 	/* Prevent simultaneous reconnects that can corrupt tcon->rlist list */
 	mutex_lock(&pserver->reconnect_mutex);
 
+	/* if the server is marked for termination, drop the ref count here */
+	if (server->terminate) {
+		cifs_put_tcp_session(server, true);
+		mutex_unlock(&pserver->reconnect_mutex);
+		return;
+	}
+
 	INIT_LIST_HEAD(&tmp_list);
 	INIT_LIST_HEAD(&tmp_ses_list);
 	cifs_dbg(FYI, "Reconnecting tcons and channels\n");
diff --git a/fs/smb/client/transport.c b/fs/smb/client/transport.c
index 14710afdc2a3..c60c65926933 100644
--- a/fs/smb/client/transport.c
+++ b/fs/smb/client/transport.c
@@ -1032,7 +1032,7 @@ struct TCP_Server_Info *cifs_pick_channel(struct cifs_ses *ses)
 	spin_lock(&ses->chan_lock);
 	for (i = 0; i < ses->chan_count; i++) {
 		server = ses->chans[i].server;
-		if (!server)
+		if (!server || server->terminate)
 			continue;
 
 		/*
-- 
2.34.1

Re: [PATCH 12/14] cifs: handle when server stops supporting multichannel

[Paulo Alcantara]

nspmangalore@gmail.com writes:

> From: Shyam Prasad N <sprasad@microsoft.com>
>
> When a server stops supporting multichannel, we will
> keep attempting reconnects to the secondary channels today.
> Avoid this by freeing extra channels when negotiate
> returns no multichannel support.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/cifsglob.h  |  1 +
>  fs/smb/client/cifsproto.h |  2 ++
>  fs/smb/client/connect.c   | 10 ++++++
>  fs/smb/client/sess.c      | 64 ++++++++++++++++++++++++++++++-----
>  fs/smb/client/smb2pdu.c   | 70 ++++++++++++++++++++++++++++++++++++++-
>  fs/smb/client/transport.c |  2 +-
>  6 files changed, 139 insertions(+), 10 deletions(-)

Several reconnect tests are triggering this when running against
for-next branch

[  360.487827] dftest>> [*] mount opts: vers=3.1.1,echo_interval=10,nohandlecache,noserverino
[  361.570676] dftest>> [*] mount opts: vers=3.1.1,echo_interval=10,nohandlecache,noserverino
[  361.572133] dftest>> [*] TEST: mount //gandalf.test/dfstest3 , cd /mnt/dfs/link , ls ., expect:["target\d_file" "tsub"]
[  361.572133]       disconnect /mnt/dfs/link , ls#1 . (fail here is ok),  ls#2 (fail here NOT ok)
[  361.613964] CIFS: Attempting to mount //gandalf.test/dfstest3
[  361.637663] virtio_net virtio1 enp1s0: entered promiscuous mode
[  362.262990] CIFS: Attempting to mount //gandalf.test/dfstest3/link
[  362.483166] ls (978) used greatest stack depth: 21904 bytes left
[  398.362399] CIFS: VFS: \\w22-root2.gandalf.test has not responded in 30 seconds. Reconnecting...
[  398.364632] BUG: sleeping function called from invalid context at kernel/workqueue.c:3344
[  398.366399] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 981, name: cifsd
[  398.367318] preempt_count: 1, expected: 0
[  398.367826] RCU nest depth: 0, expected: 0
[  398.368414] 1 lock held by cifsd/981:
[  398.368905]  #0: ffffffffc028fb58 (&cifs_tcp_ses_lock){+.+.}-{2:2}, at: cifs_mark_tcp_ses_conns_for_reconnect+0x5f/0x3b0 [cifs]
[  398.371053] CPU: 3 PID: 981 Comm: cifsd Not tainted 6.6.0 #1
[  398.371720] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
[  398.373072] Call Trace:
[  398.373404]  <TASK>
[  398.373700]  dump_stack_lvl+0x64/0x80
[  398.374188]  __might_resched+0x23c/0x360
[  398.374708]  ? __pfx___might_resched+0x10/0x10
[  398.375188]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.375688]  ? __might_sleep+0x2b/0xb0
[  398.376086]  ? __flush_work+0xc5/0x640
[  398.376539]  __flush_work+0xfd/0x640
[  398.377019]  ? __flush_work+0xc5/0x640
[  398.377524]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.378155]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.378798]  ? __pfx___flush_work+0x10/0x10
[  398.379408]  ? hlock_class+0x32/0xc0
[  398.379840]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.380355]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.380959]  ? mark_held_locks+0x5d/0x90
[  398.381537]  __cancel_work_timer+0x210/0x2c0
[  398.382117]  ? __pfx___cancel_work_timer+0x10/0x10
[  398.382757]  ? do_raw_spin_trylock+0xd1/0x120
[  398.383363]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.383921]  ? do_raw_spin_unlock+0x9b/0x100
[  398.384382]  cifs_mark_tcp_ses_conns_for_reconnect+0x227/0x3b0 [cifs]
[  398.385834]  reconnect_dfs_server+0x162/0x6b0 [cifs]
[  398.387107]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.387715]  ? __mutex_unlock_slowpath+0x11e/0x400
[  398.388219]  ? __pfx___mutex_lock+0x10/0x10
[  398.388736]  ? __pfx_reconnect_dfs_server+0x10/0x10 [cifs]
[  398.390057]  ? __pfx____ratelimit+0x10/0x10
[  398.390637]  cifs_readv_from_socket+0x335/0x490 [cifs]
[  398.391898]  ? __pfx_cifs_readv_from_socket+0x10/0x10 [cifs]
[  398.393084]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.393730]  cifs_read_from_socket+0xb5/0x100 [cifs]
[  398.394985]  ? __pfx_cifs_read_from_socket+0x10/0x10 [cifs]
[  398.396176]  ? __pfx_mempool_alloc+0x10/0x10
[  398.396734]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.397386]  ? cifs_small_buf_get+0x53/0x70 [cifs]
[  398.398619]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.399245]  ? allocate_buffers+0xa7/0x1d0 [cifs]
[  398.400294]  cifs_demultiplex_thread+0x274/0x1270 [cifs]
[  398.401346]  ? __pfx_cifs_demultiplex_thread+0x10/0x10 [cifs]
[  398.402680]  ? __pfx_lock_release+0x10/0x10
[  398.403053]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.403464]  ? mark_held_locks+0x1a/0x90
[  398.403813]  ? lockdep_hardirqs_on_prepare+0x136/0x210
[  398.404250]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.404663]  ? srso_alias_return_thunk+0x5/0xfbef5
[  398.405079]  ? __kthread_parkme+0xce/0xf0
[  398.405433]  ? __pfx_cifs_demultiplex_thread+0x10/0x10 [cifs]
[  398.406293]  kthread+0x18d/0x1d0
[  398.406582]  ? kthread+0xdb/0x1d0
[  398.406885]  ? __pfx_kthread+0x10/0x10
[  398.407237]  ret_from_fork+0x34/0x60
[  398.407551]  ? __pfx_kthread+0x10/0x10
[  398.407881]  ret_from_fork_asm+0x1b/0x30
[  398.408258]  </TASK>

Re: [PATCH 12/14] cifs: handle when server stops supporting multichannel

[Paulo Alcantara]

Also found memory leaks

        ...
	unreferenced object 0xffff88801c8ca600 (size 192):
	  comm "python3", pid 14002, jiffies 4300343883 (age 4393.481s)
	  hex dump (first 32 bytes):
	    f0 a2 8b 17 80 88 ff ff f0 a2 8b 17 80 88 ff ff  ................
	    01 00 00 00 00 00 00 00 00 ca 9a 3b 00 00 00 00  ...........;....
	  backtrace:
	    [<ffffffff8144e3f5>] __kmem_cache_alloc_node+0x295/0x2d0
	    [<ffffffff813ab1aa>] kmalloc_trace+0x2a/0xc0
	    [<ffffffffc016178d>] parse_server_interfaces+0x4ed/0xcc0 [cifs]
	    [<ffffffffc016ae13>] SMB3_request_interfaces+0x163/0x2b0 [cifs]
	    [<ffffffffc016b0dd>] smb3_qfs_tcon+0x16d/0x2c0 [cifs]
	    [<ffffffffc01102c1>] cifs_mount_get_tcon+0x3b1/0x550 [cifs]
	    [<ffffffffc01a2471>] dfs_mount_share+0xcc1/0xfa0 [cifs]
	    [<ffffffffc011089a>] cifs_mount+0xda/0x4c0 [cifs]
	    [<ffffffffc01006d5>] cifs_smb3_do_mount+0x1e5/0xcc0 [cifs]
	    [<ffffffffc01954cd>] smb3_get_tree+0x16d/0x380 [cifs]
	    [<ffffffff8147c1ad>] vfs_get_tree+0x4d/0x190
	    [<ffffffff814bdbc3>] fc_mount+0x13/0x60
	    [<ffffffffc019aae4>] cifs_do_automount.isra.0+0x3c4/0x4d0 [cifs]
	    [<ffffffffc019ae3d>] cifs_d_automount+0x4d/0x1b0 [cifs]
	    [<ffffffff8148b01c>] __traverse_mounts+0xcc/0x300
	    [<ffffffff8149350b>] step_into+0x54b/0xd30

and leaked server connections

	Display Internal CIFS Data Structures for Debugging
	---------------------------------------------------
	CIFS Version 2.46
	Features: DFS,FSCACHE,STATS2,DEBUG2,ALLOW_INSECURE_LEGACY,CIFS_POSIX,UPCALL(SPNEGO),XATTR,ACL,WITNESS
	CIFSMaxBufSize: 16384
	Active VFS Requests: 0
	
	Servers: 
	1) ConnectionId: 0x70e Hostname: w22-root2.gandalf.test 
	ClientGUID: 44DAE383-3E91-3042-85FE-87D6F17298B7
	Number of credits: 1,1,1 Dialect 0x210
	Server capabilities: 0x300007
	TCP status: 4 Instance: 77
	Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840 
	In Send: 0 In MaxReq Wait: 0
	DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link10
	
	        Sessions: 
	                [NONE]
	2) ConnectionId: 0x706 Hostname: w22-root2.gandalf.test 
	ClientGUID: C8CF45E4-F70D-DF40-8821-0234A2E20DD4
	Number of credits: 1,1,1 Dialect 0x210
	Server capabilities: 0x300007
	TCP status: 4 Instance: 81
	Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840 
	In Send: 0 In MaxReq Wait: 0
	DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link6
	
	        Sessions: 
	                [NONE]
	3) ConnectionId: 0x6ae Hostname: w22-root1.gandalf.test 
	ClientGUID: AB059CDD-12FF-B94D-B30C-9E1928ACBA95
	Number of credits: 1,1,1 Dialect 0x210
	Server capabilities: 0x300007
	TCP status: 4 Instance: 96
	Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840 
	In Send: 0 In MaxReq Wait: 0
	DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link9
	
	        Sessions: 
	                [NONE]
        ...

while running reconnect tests.

Re: [PATCH 12/14] cifs: handle when server stops supporting multichannel

[Paulo Alcantara]

> and leaked server connections
>
> 	Display Internal CIFS Data Structures for Debugging
> 	---------------------------------------------------
> 	CIFS Version 2.46
> 	Features: DFS,FSCACHE,STATS2,DEBUG2,ALLOW_INSECURE_LEGACY,CIFS_POSIX,UPCALL(SPNEGO),XATTR,ACL,WITNESS
> 	CIFSMaxBufSize: 16384
> 	Active VFS Requests: 0
> 	
> 	Servers: 
> 	1) ConnectionId: 0x70e Hostname: w22-root2.gandalf.test 
> 	ClientGUID: 44DAE383-3E91-3042-85FE-87D6F17298B7
> 	Number of credits: 1,1,1 Dialect 0x210
> 	Server capabilities: 0x300007
> 	TCP status: 4 Instance: 77
> 	Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840 
> 	In Send: 0 In MaxReq Wait: 0
> 	DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link10
> 	
> 	        Sessions: 
> 	                [NONE]
> 	2) ConnectionId: 0x706 Hostname: w22-root2.gandalf.test 
> 	ClientGUID: C8CF45E4-F70D-DF40-8821-0234A2E20DD4
> 	Number of credits: 1,1,1 Dialect 0x210
> 	Server capabilities: 0x300007
> 	TCP status: 4 Instance: 81
> 	Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840 
> 	In Send: 0 In MaxReq Wait: 0
> 	DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link6
> 	
> 	        Sessions: 
> 	                [NONE]
> 	3) ConnectionId: 0x6ae Hostname: w22-root1.gandalf.test 
> 	ClientGUID: AB059CDD-12FF-B94D-B30C-9E1928ACBA95
> 	Number of credits: 1,1,1 Dialect 0x210
> 	Server capabilities: 0x300007
> 	TCP status: 4 Instance: 96
> 	Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840 
> 	In Send: 0 In MaxReq Wait: 0
> 	DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link9
> 	
> 	        Sessions: 
> 	                [NONE]
>         ...

I ended up with a simple reproducer for the above

        mount.cifs //srv/share /mnt/1 -o ...,echo_interval=10
        iptables -I INPUT -s $srvaddr -j DROP
        stat -f /mnt/1
        # check dmesg for "BUG: sleeping function called from invalid context"
        iptables -I INPUT -s $srvaddr -j ACCEPT
        stat -f /mnt/1
        umount /mnt/1
        # check /proc/fs/cifs/DebugData for leaked server connection

Re: [PATCH 12/14] cifs: handle when server stops supporting multichannel

[Shyam Prasad N]

On Thu, Nov 9, 2023 at 1:11 AM Paulo Alcantara <pc@manguebit.com> wrote:
>
> > and leaked server connections
> >
> >       Display Internal CIFS Data Structures for Debugging
> >       ---------------------------------------------------
> >       CIFS Version 2.46
> >       Features: DFS,FSCACHE,STATS2,DEBUG2,ALLOW_INSECURE_LEGACY,CIFS_POSIX,UPCALL(SPNEGO),XATTR,ACL,WITNESS
> >       CIFSMaxBufSize: 16384
> >       Active VFS Requests: 0
> >
> >       Servers:
> >       1) ConnectionId: 0x70e Hostname: w22-root2.gandalf.test
> >       ClientGUID: 44DAE383-3E91-3042-85FE-87D6F17298B7
> >       Number of credits: 1,1,1 Dialect 0x210
> >       Server capabilities: 0x300007
> >       TCP status: 4 Instance: 77
> >       Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840
> >       In Send: 0 In MaxReq Wait: 0
> >       DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link10
> >
> >               Sessions:
> >                       [NONE]
> >       2) ConnectionId: 0x706 Hostname: w22-root2.gandalf.test
> >       ClientGUID: C8CF45E4-F70D-DF40-8821-0234A2E20DD4
> >       Number of credits: 1,1,1 Dialect 0x210
> >       Server capabilities: 0x300007
> >       TCP status: 4 Instance: 81
> >       Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840
> >       In Send: 0 In MaxReq Wait: 0
> >       DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link6
> >
> >               Sessions:
> >                       [NONE]
> >       3) ConnectionId: 0x6ae Hostname: w22-root1.gandalf.test
> >       ClientGUID: AB059CDD-12FF-B94D-B30C-9E1928ACBA95
> >       Number of credits: 1,1,1 Dialect 0x210
> >       Server capabilities: 0x300007
> >       TCP status: 4 Instance: 96
> >       Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840
> >       In Send: 0 In MaxReq Wait: 0
> >       DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link9
> >
> >               Sessions:
> >                       [NONE]
> >         ...
>
> I ended up with a simple reproducer for the above
>
>         mount.cifs //srv/share /mnt/1 -o ...,echo_interval=10
>         iptables -I INPUT -s $srvaddr -j DROP
>         stat -f /mnt/1
>         # check dmesg for "BUG: sleeping function called from invalid context"
>         iptables -I INPUT -s $srvaddr -j ACCEPT
>         stat -f /mnt/1
>         umount /mnt/1
>         # check /proc/fs/cifs/DebugData for leaked server connection

Ack. I'll try and get a repro locally and debug this one.

-- 
Regards,
Shyam

Re: [PATCH 12/14] cifs: handle when server stops supporting multichannel

[Paulo Alcantara]

Shyam Prasad N <nspmangalore@gmail.com> writes:

> On Thu, Nov 9, 2023 at 1:11 AM Paulo Alcantara <pc@manguebit.com> wrote:
>>
>> > and leaked server connections
>> >
>> >       Display Internal CIFS Data Structures for Debugging
>> >       ---------------------------------------------------
>> >       CIFS Version 2.46
>> >       Features: DFS,FSCACHE,STATS2,DEBUG2,ALLOW_INSECURE_LEGACY,CIFS_POSIX,UPCALL(SPNEGO),XATTR,ACL,WITNESS
>> >       CIFSMaxBufSize: 16384
>> >       Active VFS Requests: 0
>> >
>> >       Servers:
>> >       1) ConnectionId: 0x70e Hostname: w22-root2.gandalf.test
>> >       ClientGUID: 44DAE383-3E91-3042-85FE-87D6F17298B7
>> >       Number of credits: 1,1,1 Dialect 0x210
>> >       Server capabilities: 0x300007
>> >       TCP status: 4 Instance: 77
>> >       Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840
>> >       In Send: 0 In MaxReq Wait: 0
>> >       DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link10
>> >
>> >               Sessions:
>> >                       [NONE]
>> >       2) ConnectionId: 0x706 Hostname: w22-root2.gandalf.test
>> >       ClientGUID: C8CF45E4-F70D-DF40-8821-0234A2E20DD4
>> >       Number of credits: 1,1,1 Dialect 0x210
>> >       Server capabilities: 0x300007
>> >       TCP status: 4 Instance: 81
>> >       Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840
>> >       In Send: 0 In MaxReq Wait: 0
>> >       DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link6
>> >
>> >               Sessions:
>> >                       [NONE]
>> >       3) ConnectionId: 0x6ae Hostname: w22-root1.gandalf.test
>> >       ClientGUID: AB059CDD-12FF-B94D-B30C-9E1928ACBA95
>> >       Number of credits: 1,1,1 Dialect 0x210
>> >       Server capabilities: 0x300007
>> >       TCP status: 4 Instance: 96
>> >       Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840
>> >       In Send: 0 In MaxReq Wait: 0
>> >       DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link9
>> >
>> >               Sessions:
>> >                       [NONE]
>> >         ...
>>
>> I ended up with a simple reproducer for the above
>>
>>         mount.cifs //srv/share /mnt/1 -o ...,echo_interval=10
>>         iptables -I INPUT -s $srvaddr -j DROP
>>         stat -f /mnt/1
>>         # check dmesg for "BUG: sleeping function called from invalid context"
>>         iptables -I INPUT -s $srvaddr -j ACCEPT
>>         stat -f /mnt/1
>>         umount /mnt/1
>>         # check /proc/fs/cifs/DebugData for leaked server connection
>
> Ack. I'll try and get a repro locally and debug this one.

This should be related to patch 10/14 as you're taking an extra
reference of @server over reconnect, and when the client reconnects and
umount, that reference don't seem to be put afterwards.

Re: [PATCH 12/14] cifs: handle when server stops supporting multichannel

[Shyam Prasad N]

On Thu, Nov 9, 2023 at 6:58 PM Paulo Alcantara <pc@manguebit.com> wrote:
>
> Shyam Prasad N <nspmangalore@gmail.com> writes:
>
> > On Thu, Nov 9, 2023 at 1:11 AM Paulo Alcantara <pc@manguebit.com> wrote:
> >>
> >> > and leaked server connections
> >> >
> >> >       Display Internal CIFS Data Structures for Debugging
> >> >       ---------------------------------------------------
> >> >       CIFS Version 2.46
> >> >       Features: DFS,FSCACHE,STATS2,DEBUG2,ALLOW_INSECURE_LEGACY,CIFS_POSIX,UPCALL(SPNEGO),XATTR,ACL,WITNESS
> >> >       CIFSMaxBufSize: 16384
> >> >       Active VFS Requests: 0
> >> >
> >> >       Servers:
> >> >       1) ConnectionId: 0x70e Hostname: w22-root2.gandalf.test
> >> >       ClientGUID: 44DAE383-3E91-3042-85FE-87D6F17298B7
> >> >       Number of credits: 1,1,1 Dialect 0x210
> >> >       Server capabilities: 0x300007
> >> >       TCP status: 4 Instance: 77
> >> >       Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840
> >> >       In Send: 0 In MaxReq Wait: 0
> >> >       DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link10
> >> >
> >> >               Sessions:
> >> >                       [NONE]
> >> >       2) ConnectionId: 0x706 Hostname: w22-root2.gandalf.test
> >> >       ClientGUID: C8CF45E4-F70D-DF40-8821-0234A2E20DD4
> >> >       Number of credits: 1,1,1 Dialect 0x210
> >> >       Server capabilities: 0x300007
> >> >       TCP status: 4 Instance: 81
> >> >       Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840
> >> >       In Send: 0 In MaxReq Wait: 0
> >> >       DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link6
> >> >
> >> >               Sessions:
> >> >                       [NONE]
> >> >       3) ConnectionId: 0x6ae Hostname: w22-root1.gandalf.test
> >> >       ClientGUID: AB059CDD-12FF-B94D-B30C-9E1928ACBA95
> >> >       Number of credits: 1,1,1 Dialect 0x210
> >> >       Server capabilities: 0x300007
> >> >       TCP status: 4 Instance: 96
> >> >       Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840
> >> >       In Send: 0 In MaxReq Wait: 0
> >> >       DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link9
> >> >
> >> >               Sessions:
> >> >                       [NONE]
> >> >         ...
> >>
> >> I ended up with a simple reproducer for the above
> >>
> >>         mount.cifs //srv/share /mnt/1 -o ...,echo_interval=10
> >>         iptables -I INPUT -s $srvaddr -j DROP
> >>         stat -f /mnt/1
> >>         # check dmesg for "BUG: sleeping function called from invalid context"
> >>         iptables -I INPUT -s $srvaddr -j ACCEPT
> >>         stat -f /mnt/1
> >>         umount /mnt/1
> >>         # check /proc/fs/cifs/DebugData for leaked server connection
> >
> > Ack. I'll try and get a repro locally and debug this one.
>
> This should be related to patch 10/14 as you're taking an extra
> reference of @server over reconnect, and when the client reconnects and
> umount, that reference don't seem to be put afterwards.

The idea is that the reference is put in the reconnect worker.
I think the issue is that I don't drop the final reference if
cancel_delayed_work_sync was able to cancel a work.
Let me work on the updated patch.

Also, I found the reason for "BUG: sleeping function called from
invalid context". I've enabled CONFIG_DEBUG_ATOMIC_SLEEP in my test
config now.

-- 
Regards,
Shyam

Re: [PATCH 12/14] cifs: handle when server stops supporting multichannel

[Shyam Prasad N]

[-- Attachment #1: Type: text/plain, Size: 3954 bytes --]

On Thu, Nov 9, 2023 at 7:19 PM Shyam Prasad N <nspmangalore@gmail.com> wrote:
>
> On Thu, Nov 9, 2023 at 6:58 PM Paulo Alcantara <pc@manguebit.com> wrote:
> >
> > Shyam Prasad N <nspmangalore@gmail.com> writes:
> >
> > > On Thu, Nov 9, 2023 at 1:11 AM Paulo Alcantara <pc@manguebit.com> wrote:
> > >>
> > >> > and leaked server connections
> > >> >
> > >> >       Display Internal CIFS Data Structures for Debugging
> > >> >       ---------------------------------------------------
> > >> >       CIFS Version 2.46
> > >> >       Features: DFS,FSCACHE,STATS2,DEBUG2,ALLOW_INSECURE_LEGACY,CIFS_POSIX,UPCALL(SPNEGO),XATTR,ACL,WITNESS
> > >> >       CIFSMaxBufSize: 16384
> > >> >       Active VFS Requests: 0
> > >> >
> > >> >       Servers:
> > >> >       1) ConnectionId: 0x70e Hostname: w22-root2.gandalf.test
> > >> >       ClientGUID: 44DAE383-3E91-3042-85FE-87D6F17298B7
> > >> >       Number of credits: 1,1,1 Dialect 0x210
> > >> >       Server capabilities: 0x300007
> > >> >       TCP status: 4 Instance: 77
> > >> >       Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840
> > >> >       In Send: 0 In MaxReq Wait: 0
> > >> >       DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link10
> > >> >
> > >> >               Sessions:
> > >> >                       [NONE]
> > >> >       2) ConnectionId: 0x706 Hostname: w22-root2.gandalf.test
> > >> >       ClientGUID: C8CF45E4-F70D-DF40-8821-0234A2E20DD4
> > >> >       Number of credits: 1,1,1 Dialect 0x210
> > >> >       Server capabilities: 0x300007
> > >> >       TCP status: 4 Instance: 81
> > >> >       Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840
> > >> >       In Send: 0 In MaxReq Wait: 0
> > >> >       DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link6
> > >> >
> > >> >               Sessions:
> > >> >                       [NONE]
> > >> >       3) ConnectionId: 0x6ae Hostname: w22-root1.gandalf.test
> > >> >       ClientGUID: AB059CDD-12FF-B94D-B30C-9E1928ACBA95
> > >> >       Number of credits: 1,1,1 Dialect 0x210
> > >> >       Server capabilities: 0x300007
> > >> >       TCP status: 4 Instance: 96
> > >> >       Local Users To Server: 1 SecMode: 0x1 Req On Wire: 0 Net namespace: 4026531840
> > >> >       In Send: 0 In MaxReq Wait: 0
> > >> >       DFS leaf full path: \\W22-ROOT1.GANDALF.TEST\dfstest3\link9
> > >> >
> > >> >               Sessions:
> > >> >                       [NONE]
> > >> >         ...
> > >>
> > >> I ended up with a simple reproducer for the above
> > >>
> > >>         mount.cifs //srv/share /mnt/1 -o ...,echo_interval=10
> > >>         iptables -I INPUT -s $srvaddr -j DROP
> > >>         stat -f /mnt/1
> > >>         # check dmesg for "BUG: sleeping function called from invalid context"
> > >>         iptables -I INPUT -s $srvaddr -j ACCEPT
> > >>         stat -f /mnt/1
> > >>         umount /mnt/1
> > >>         # check /proc/fs/cifs/DebugData for leaked server connection
> > >
> > > Ack. I'll try and get a repro locally and debug this one.
> >
> > This should be related to patch 10/14 as you're taking an extra
> > reference of @server over reconnect, and when the client reconnects and
> > umount, that reference don't seem to be put afterwards.
>
> The idea is that the reference is put in the reconnect worker.
> I think the issue is that I don't drop the final reference if
> cancel_delayed_work_sync was able to cancel a work.
This was already taken care of.

> Let me work on the updated patch.
>
> Also, I found the reason for "BUG: sleeping function called from
> invalid context". I've enabled CONFIG_DEBUG_ATOMIC_SLEEP in my test
> config now.
>
> --
> Regards,
> Shyam

Hi Paulo,

Can you please check if the problem is still seen with these updated patches?
I was unable to reproduce the issue with the steps you provided.

-- 
Regards,
Shyam

[-- Attachment #2: 0007-cifs-handle-when-server-stops-supporting-multichanne.patch --]
[-- Type: application/octet-stream, Size: 9561 bytes --]

From 5106086473124167e9e0bdcd5895c2f48e5eb816 Mon Sep 17 00:00:00 2001
From: Shyam Prasad N <sprasad@microsoft.com>
Date: Fri, 13 Oct 2023 11:40:09 +0000
Subject: [PATCH 7/9] cifs: handle when server stops supporting multichannel

When a server stops supporting multichannel, we will
keep attempting reconnects to the secondary channels today.
Avoid this by freeing extra channels when negotiate
returns no multichannel support.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/cifsglob.h  |  1 +
 fs/smb/client/cifsproto.h |  2 ++
 fs/smb/client/connect.c   | 10 ++++++
 fs/smb/client/sess.c      | 64 ++++++++++++++++++++++++++++-----
 fs/smb/client/smb2pdu.c   | 76 ++++++++++++++++++++++++++++++++++++++-
 fs/smb/client/transport.c |  2 +-
 6 files changed, 145 insertions(+), 10 deletions(-)

diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
index 81e7a45f413d..62d5a5495bb8 100644
--- a/fs/smb/client/cifsglob.h
+++ b/fs/smb/client/cifsglob.h
@@ -650,6 +650,7 @@ struct TCP_Server_Info {
 	bool noautotune;		/* do not autotune send buf sizes */
 	bool nosharesock;
 	bool tcp_nodelay;
+	bool terminate;
 	unsigned int credits;  /* send no more requests at once */
 	unsigned int max_credits; /* can override large 32000 default at mnt */
 	unsigned int in_flight;  /* number of requests on the wire to server */
diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
index 00ad41633e26..d87e2c26cce2 100644
--- a/fs/smb/client/cifsproto.h
+++ b/fs/smb/client/cifsproto.h
@@ -641,6 +641,8 @@ cifs_chan_needs_reconnect(struct cifs_ses *ses,
 bool
 cifs_chan_is_iface_active(struct cifs_ses *ses,
 			  struct TCP_Server_Info *server);
+void
+cifs_disable_secondary_channels(struct cifs_ses *ses);
 int
 cifs_chan_update_iface(struct cifs_ses *ses, struct TCP_Server_Info *server);
 int
diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index 1490547ec493..57c2a7df3457 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -219,6 +219,14 @@ cifs_mark_tcp_ses_conns_for_reconnect(struct TCP_Server_Info *server,
 
 	spin_lock(&cifs_tcp_ses_lock);
 	list_for_each_entry_safe(ses, nses, &pserver->smb_ses_list, smb_ses_list) {
+		/*
+		 * if channel has been marked for termination, nothing to do
+		 * for the channel. in fact, we cannot find the channel for the
+		 * server. So safe to exit here
+		 */
+		if (server->terminate)
+			break;
+
 		/* check if iface is still active */
 		if (!cifs_chan_is_iface_active(ses, server))
 			cifs_chan_update_iface(ses, server);
@@ -253,6 +261,8 @@ cifs_mark_tcp_ses_conns_for_reconnect(struct TCP_Server_Info *server,
 			spin_lock(&tcon->tc_lock);
 			tcon->status = TID_NEED_RECON;
 			spin_unlock(&tcon->tc_lock);
+
+			cancel_delayed_work(&tcon->query_interfaces);
 		}
 		if (ses->tcon_ipc) {
 			ses->tcon_ipc->need_reconnect = true;
diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c
index 1457f81af0cf..26784e338388 100644
--- a/fs/smb/client/sess.c
+++ b/fs/smb/client/sess.c
@@ -289,6 +289,60 @@ int cifs_try_adding_channels(struct cifs_ses *ses)
 	return new_chan_count - old_chan_count;
 }
 
+/*
+ * called when multichannel is disabled by the server.
+ * this always gets called from smb2_reconnect
+ * and cannot get called in parallel threads.
+ */
+void
+cifs_disable_secondary_channels(struct cifs_ses *ses)
+{
+	int i, chan_count;
+	struct TCP_Server_Info *server;
+	struct cifs_server_iface *iface;
+
+	spin_lock(&ses->chan_lock);
+	chan_count = ses->chan_count;
+	if (chan_count == 1)
+		goto done;
+
+	ses->chan_count = 1;
+
+	/* for all secondary channels reset the need reconnect bit */
+	ses->chans_need_reconnect &= 1;
+
+	for (i = 1; i < chan_count; i++) {
+		iface = ses->chans[i].iface;
+		server = ses->chans[i].server;
+
+		if (iface) {
+			spin_lock(&ses->iface_lock);
+			kref_put(&iface->refcount, release_iface);
+			ses->chans[i].iface = NULL;
+			iface->num_channels--;
+			if (iface->weight_fulfilled)
+				iface->weight_fulfilled--;
+			spin_unlock(&ses->iface_lock);
+		}
+
+		spin_unlock(&ses->chan_lock);
+		if (server && !server->terminate) {
+			server->terminate = true;
+			cifs_signal_cifsd_for_reconnect(server, false);
+		}
+		spin_lock(&ses->chan_lock);
+
+		if (server) {
+			ses->chans[i].server = NULL;
+			cifs_put_tcp_session(server, false);
+		}
+
+	}
+
+done:
+	spin_unlock(&ses->chan_lock);
+}
+
 /*
  * update the iface for the channel if necessary.
  * will return 0 when iface is updated, 1 if removed, 2 otherwise
@@ -586,14 +640,10 @@ cifs_ses_add_channel(struct cifs_ses *ses,
 
 out:
 	if (rc && chan->server) {
-		/*
-		 * we should avoid race with these delayed works before we
-		 * remove this channel
-		 */
-		cancel_delayed_work_sync(&chan->server->echo);
-		cancel_delayed_work_sync(&chan->server->reconnect);
+		cifs_put_tcp_session(chan->server, 0);
 
 		spin_lock(&ses->chan_lock);
+
 		/* we rely on all bits beyond chan_count to be clear */
 		cifs_chan_clear_need_reconnect(ses, chan->server);
 		ses->chan_count--;
@@ -603,8 +653,6 @@ cifs_ses_add_channel(struct cifs_ses *ses,
 		 */
 		WARN_ON(ses->chan_count < 1);
 		spin_unlock(&ses->chan_lock);
-
-		cifs_put_tcp_session(chan->server, 0);
 	}
 
 	kfree(ctx->UNC);
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index 0d4351e43069..2eb29fa278c3 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -164,6 +164,8 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 	struct nls_table *nls_codepage = NULL;
 	struct cifs_ses *ses;
 	int xid;
+	struct TCP_Server_Info *pserver;
+	unsigned int chan_index;
 
 	/*
 	 * SMB2s NegProt, SessSetup, Logoff do not have tcon yet so
@@ -224,6 +226,12 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 			return -EAGAIN;
 		}
 	}
+
+	/* if server is marked for termination, cifsd will cleanup */
+	if (server->terminate) {
+		spin_unlock(&server->srv_lock);
+		return -EHOSTDOWN;
+	}
 	spin_unlock(&server->srv_lock);
 
 again:
@@ -242,12 +250,24 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 		 tcon->need_reconnect);
 
 	mutex_lock(&ses->session_mutex);
+	/*
+	 * if this is called by delayed work, and the channel has been disabled
+	 * in parallel, the delayed work can continue to execute in parallel
+	 * there's a chance that this channel may not exist anymore
+	 */
+	spin_lock(&server->srv_lock);
+	if (server->tcpStatus == CifsExiting) {
+		spin_unlock(&server->srv_lock);
+		mutex_unlock(&ses->session_mutex);
+		rc = -EHOSTDOWN;
+		goto out;
+	}
+
 	/*
 	 * Recheck after acquire mutex. If another thread is negotiating
 	 * and the server never sends an answer the socket will be closed
 	 * and tcpStatus set to reconnect.
 	 */
-	spin_lock(&server->srv_lock);
 	if (server->tcpStatus == CifsNeedReconnect) {
 		spin_unlock(&server->srv_lock);
 		mutex_unlock(&ses->session_mutex);
@@ -284,6 +304,53 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 
 	rc = cifs_negotiate_protocol(0, ses, server);
 	if (!rc) {
+		/*
+		 * if server stopped supporting multichannel
+		 * and the first channel reconnected, disable all the others.
+		 */
+		if (ses->chan_count > 1 &&
+		    !(server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL)) {
+			if (SERVER_IS_CHAN(server)) {
+				cifs_dbg(VFS, "server %s does not support " \
+					 "multichannel anymore. skipping secondary channel\n",
+					 ses->server->hostname);
+
+				spin_lock(&ses->chan_lock);
+				chan_index = cifs_ses_get_chan_index(ses, server);
+				if (chan_index == CIFS_INVAL_CHAN_INDEX) {
+					spin_unlock(&ses->chan_lock);
+					goto skip_terminate;
+				}
+
+				ses->chans[chan_index].server = NULL;
+				spin_unlock(&ses->chan_lock);
+
+				/*
+				 * the above reference of server by channel
+				 * needs to be dropped without holding chan_lock
+				 * as cifs_put_tcp_session takes a higher lock
+				 * i.e. cifs_tcp_ses_lock
+				 */
+				cifs_put_tcp_session(server, 1);
+
+				server->terminate = true;
+				cifs_signal_cifsd_for_reconnect(server, false);
+
+				/* mark primary server as needing reconnect */
+				pserver = server->primary_server;
+				cifs_signal_cifsd_for_reconnect(pserver, false);
+
+skip_terminate:
+				mutex_unlock(&ses->session_mutex);
+				rc = -EHOSTDOWN;
+				goto out;
+			} else {
+				cifs_server_dbg(VFS, "does not support " \
+					 "multichannel anymore. disabling all other channels\n");
+				cifs_disable_secondary_channels(ses);
+			}
+		}
+
 		rc = cifs_setup_session(0, ses, server, nls_codepage);
 		if ((rc == -EACCES) && !tcon->retry) {
 			mutex_unlock(&ses->session_mutex);
@@ -3836,6 +3903,13 @@ void smb2_reconnect_server(struct work_struct *work)
 	/* Prevent simultaneous reconnects that can corrupt tcon->rlist list */
 	mutex_lock(&pserver->reconnect_mutex);
 
+	/* if the server is marked for termination, drop the ref count here */
+	if (server->terminate) {
+		cifs_put_tcp_session(server, true);
+		mutex_unlock(&pserver->reconnect_mutex);
+		return;
+	}
+
 	INIT_LIST_HEAD(&tmp_list);
 	INIT_LIST_HEAD(&tmp_ses_list);
 	cifs_dbg(FYI, "Reconnecting tcons and channels\n");
diff --git a/fs/smb/client/transport.c b/fs/smb/client/transport.c
index d553b7a54621..4f717ad7c21b 100644
--- a/fs/smb/client/transport.c
+++ b/fs/smb/client/transport.c
@@ -1023,7 +1023,7 @@ struct TCP_Server_Info *cifs_pick_channel(struct cifs_ses *ses)
 	spin_lock(&ses->chan_lock);
 	for (i = 0; i < ses->chan_count; i++) {
 		server = ses->chans[i].server;
-		if (!server)
+		if (!server || server->terminate)
 			continue;
 
 		/*
-- 
2.34.1


[-- Attachment #3: 0006-cifs-handle-when-server-starts-supporting-multichann.patch --]
[-- Type: application/octet-stream, Size: 3256 bytes --]

From b6f410cb8635d01875938f3991d7b093d83e264d Mon Sep 17 00:00:00 2001
From: Shyam Prasad N <sprasad@microsoft.com>
Date: Fri, 13 Oct 2023 11:33:21 +0000
Subject: [PATCH 6/9] cifs: handle when server starts supporting multichannel

When the user mounts with multichannel option, but the
server does not support it, there can be a time in future
where it can be supported.

With this change, such a case is handled.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/cifsproto.h |  1 +
 fs/smb/client/connect.c   |  3 +++
 fs/smb/client/smb2pdu.c   | 32 ++++++++++++++++++++++++++++++--
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
index eed8dbb6b2fb..00ad41633e26 100644
--- a/fs/smb/client/cifsproto.h
+++ b/fs/smb/client/cifsproto.h
@@ -132,6 +132,7 @@ extern int SendReceiveBlockingLock(const unsigned int xid,
 			struct smb_hdr *in_buf,
 			struct smb_hdr *out_buf,
 			int *bytes_returned);
+
 void
 cifs_signal_cifsd_for_reconnect(struct TCP_Server_Info *server,
 				      bool all_channels);
diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index b514b41cc9f0..1490547ec493 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -132,6 +132,9 @@ static void smb2_query_server_interfaces(struct work_struct *work)
 	free_xid(xid);
 
 	if (rc) {
+		if (rc == -EOPNOTSUPP)
+			return;
+
 		cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
 				__func__, rc);
 	}
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index b7665155f4e2..0d4351e43069 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -163,6 +163,7 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 	int rc = 0;
 	struct nls_table *nls_codepage = NULL;
 	struct cifs_ses *ses;
+	int xid;
 
 	/*
 	 * SMB2s NegProt, SessSetup, Logoff do not have tcon yet so
@@ -307,17 +308,44 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon,
 		tcon->need_reopen_files = true;
 
 	rc = cifs_tree_connect(0, tcon, nls_codepage);
-	mutex_unlock(&ses->session_mutex);
 
 	cifs_dbg(FYI, "reconnect tcon rc = %d\n", rc);
 	if (rc) {
 		/* If sess reconnected but tcon didn't, something strange ... */
+		mutex_unlock(&ses->session_mutex);
 		cifs_dbg(VFS, "reconnect tcon failed rc = %d\n", rc);
 		goto out;
 	}
 
+	if (!rc &&
+	    (server->capabilities & SMB2_GLOBAL_CAP_MULTI_CHANNEL)) {
+		mutex_unlock(&ses->session_mutex);
+
+		/*
+		 * query server network interfaces, in case they change
+		 */
+		xid = get_xid();
+		rc = SMB3_request_interfaces(xid, tcon, false);
+		free_xid(xid);
+
+		if (rc)
+			cifs_dbg(FYI, "%s: failed to query server interfaces: %d\n",
+				 __func__, rc);
+
+		if (ses->chan_max > ses->chan_count &&
+		    !SERVER_IS_CHAN(server)) {
+			if (ses->chan_count == 1)
+				cifs_server_dbg(VFS, "supports multichannel now\n");
+
+			cifs_try_adding_channels(ses);
+		}
+	} else {
+		mutex_unlock(&ses->session_mutex);
+	}
+
 	if (smb2_command != SMB2_INTERNAL_CMD)
-		mod_delayed_work(cifsiod_wq, &server->reconnect, 0);
+		if (mod_delayed_work(cifsiod_wq, &server->reconnect, 0))
+			cifs_put_tcp_session(server, false);
 
 	atomic_inc(&tconInfoReconnectCount);
 out:
-- 
2.34.1

Re: [PATCH 12/14] cifs: handle when server stops supporting multichannel

[Paulo Alcantara]

Shyam Prasad N <nspmangalore@gmail.com> writes:

> Can you please check if the problem is still seen with these updated patches?
> I was unable to reproduce the issue with the steps you provided.

I couldn't reproduce that reconnect issue anymore.

However, some other problems as shown below with lockdep, kmemleak and
kasan enabled.

$ mount.cifs //w22-root1.gandalf.test/test /mnt/1 -o username=administrator,password=***,multichannel,echo_interval=10
# disable multichannel on w22-root1.gandalf.test server
#   Set-SmbServerConfiguration -EnableMultichannel $false
$ iptables -I INPUT -s 192.168.1.11 -j DROP
$ stat -f /mnt/1
stat: cannot read file system information for '/mnt/1': Host is down
$ iptables -I INPUT -s 192.168.1.11 -j ACCEPT
$ stat -f /mnt/1
stat: cannot read file system information for '/mnt/1': Resource temporarily unavailable

...
[ 1465.290096] CIFS: VFS: \\w22-root1.gandalf.test does not support multichannel anymore. disabling all other channels
[ 1465.292488] 
[ 1465.292863] ======================================================
[ 1465.294183] WARNING: possible circular locking dependency detected
[ 1465.295309] 6.6.0 #2 Not tainted
[ 1465.295863] ------------------------------------------------------
[ 1465.296876] kworker/2:1/1127 is trying to acquire lock:
[ 1465.297745] ffff8880129482c0 (&ret_buf->iface_lock){+.+.}-{2:2}, at: cifs_disable_secondary_channels+0x117/0x280 [cifs]
[ 1465.300314] 
[ 1465.300314] but task is already holding lock:
[ 1465.301267] ffff888012948328 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_disable_secondary_channels+0x29/0x280 [cifs]
[ 1465.303804] 
[ 1465.303804] which lock already depends on the new lock.
[ 1465.303804] 
[ 1465.305107] 
[ 1465.305107] the existing dependency chain (in reverse order) is:
[ 1465.306309] 
[ 1465.306309] -> #1 (&ret_buf->chan_lock){+.+.}-{2:2}:
[ 1465.307391]        _raw_spin_lock+0x34/0x80
[ 1465.308093]        cifs_try_adding_channels+0x205/0x1290 [cifs]
[ 1465.309865]        cifs_mount+0xfb/0x4c0 [cifs]
[ 1465.311423]        cifs_smb3_do_mount+0x1e5/0xcc0 [cifs]
[ 1465.313094]        smb3_get_tree+0x16d/0x380 [cifs]
[ 1465.314702]        vfs_get_tree+0x4d/0x190
[ 1465.315275]        path_mount+0x3c4/0xf90
[ 1465.315811]        __x64_sys_mount+0x1aa/0x1f0
[ 1465.316392]        do_syscall_64+0x47/0xf0
[ 1465.316950]        entry_SYSCALL_64_after_hwframe+0x6f/0x77
[ 1465.317681] 
[ 1465.317681] -> #0 (&ret_buf->iface_lock){+.+.}-{2:2}:
[ 1465.318548]        __lock_acquire+0x1793/0x2110
[ 1465.319156]        lock_acquire+0x14a/0x3a0
[ 1465.319714]        _raw_spin_lock+0x34/0x80
[ 1465.320266]        cifs_disable_secondary_channels+0x117/0x280 [cifs]
[ 1465.321729]        smb2_reconnect+0x520/0xcb0 [cifs]
[ 1465.323001]        smb2_reconnect_server+0x771/0xb00 [cifs]
[ 1465.324352]        process_one_work+0x43c/0x8e0
[ 1465.324959]        worker_thread+0x397/0x690
[ 1465.325522]        kthread+0x18d/0x1d0
[ 1465.326028]        ret_from_fork+0x34/0x60
[ 1465.326564]        ret_from_fork_asm+0x1b/0x30
[ 1465.327164] 
[ 1465.327164] other info that might help us debug this:
[ 1465.327164] 
[ 1465.328180]  Possible unsafe locking scenario:
[ 1465.328180] 
[ 1465.328943]        CPU0                    CPU1
[ 1465.329523]        ----                    ----
[ 1465.330119]   lock(&ret_buf->chan_lock);
[ 1465.330649]                                lock(&ret_buf->iface_lock);
[ 1465.331493]                                lock(&ret_buf->chan_lock);
[ 1465.332326]   lock(&ret_buf->iface_lock);
[ 1465.332865] 
[ 1465.332865]  *** DEADLOCK ***
[ 1465.332865] 
[ 1465.333623] 5 locks held by kworker/2:1/1127:
[ 1465.334199]  #0: ffff88800dd09d48 ((wq_completion)cifsiod){+.+.}-{0:0}, at: process_one_work+0x39a/0x8e0
[ 1465.335430]  #1: ffff88800de97dc0 ((work_completion)(&(&tcp_ses->reconnect)->work)){+.+.}-{0:0}, at: process_one_work+0x39a/0x8e0
[ 1465.336947]  #2: ffff88800dd7a8c8 (&tcp_ses->reconnect_mutex){+.+.}-{3:3}, at: smb2_reconnect_server+0xde/0xb00 [cifs]
[ 1465.338953]  #3: ffff8880129480f0 (&ret_buf->session_mutex){+.+.}-{3:3}, at: smb2_reconnect+0x234/0xcb0 [cifs]
[ 1465.340838]  #4: ffff888012948328 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_disable_secondary_channels+0x29/0x280 [cifs]
[ 1465.342073] 
[ 1465.342073] stack backtrace:
[ 1465.342421] CPU: 2 PID: 1127 Comm: kworker/2:1 Not tainted 6.6.0 #2
[ 1465.342917] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
[ 1465.343782] Workqueue: cifsiod smb2_reconnect_server [cifs]
[ 1465.344595] Call Trace:
[ 1465.344845]  <TASK>
[ 1465.345025]  dump_stack_lvl+0x4a/0x80
[ 1465.345329]  check_noncircular+0x269/0x2b0
[ 1465.345672]  ? __pfx_check_noncircular+0x10/0x10
[ 1465.346047]  ? __pfx_stack_trace_save+0x10/0x10
[ 1465.346420]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.346814]  ? lockdep_lock+0xa3/0x160
[ 1465.347128]  ? __pfx_lockdep_lock+0x10/0x10
[ 1465.347465]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.347858]  ? add_chain_block+0x1d8/0x280
[ 1465.348191]  __lock_acquire+0x1793/0x2110
[ 1465.348524]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.348918]  ? __pfx___lock_acquire+0x10/0x10
[ 1465.349271]  ? __pfx_prb_read_valid+0x10/0x10
[ 1465.349624]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.350018]  ? llist_add_batch+0x9d/0xf0
[ 1465.350344]  ? __pfx_llist_add_batch+0x10/0x10
[ 1465.350712]  lock_acquire+0x14a/0x3a0
[ 1465.351014]  ? cifs_disable_secondary_channels+0x117/0x280 [cifs]
[ 1465.351874]  ? __pfx_lock_acquire+0x10/0x10
[ 1465.352214]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.352600]  ? hlock_class+0x32/0xc0
[ 1465.352900]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.353286]  ? lock_acquired+0x2df/0x510
[ 1465.353612]  ? do_raw_spin_trylock+0xd1/0x120
[ 1465.353973]  ? __pfx_lock_acquired+0x10/0x10
[ 1465.354323]  _raw_spin_lock+0x34/0x80
[ 1465.354622]  ? cifs_disable_secondary_channels+0x117/0x280 [cifs]
[ 1465.355484]  cifs_disable_secondary_channels+0x117/0x280 [cifs]
[ 1465.356332]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.356730]  smb2_reconnect+0x520/0xcb0 [cifs]
[ 1465.357477]  smb2_reconnect_server+0x771/0xb00 [cifs]
[ 1465.358269]  ? __pfx_smb2_reconnect_server+0x10/0x10 [cifs]
[ 1465.359089]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.359478]  ? lock_is_held_type+0x90/0x100
[ 1465.359830]  ? mark_held_locks+0x1a/0x90
[ 1465.360155]  process_one_work+0x43c/0x8e0
[ 1465.360493]  ? __pfx_process_one_work+0x10/0x10
[ 1465.360912]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.361359]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.361754]  ? __list_add_valid_or_report+0x37/0xf0
[ 1465.362155]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.362544]  worker_thread+0x397/0x690
[ 1465.362863]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.363250]  ? __kthread_parkme+0xce/0xf0
[ 1465.363583]  ? __pfx_worker_thread+0x10/0x10
[ 1465.363942]  kthread+0x18d/0x1d0
[ 1465.364210]  ? kthread+0xdb/0x1d0
[ 1465.364483]  ? __pfx_kthread+0x10/0x10
[ 1465.364798]  ret_from_fork+0x34/0x60
[ 1465.365093]  ? __pfx_kthread+0x10/0x10
[ 1465.365400]  ret_from_fork_asm+0x1b/0x30
[ 1465.365733]  </TASK>
[ 1465.365954] BUG: sleeping function called from invalid context at kernel/workqueue.c:3344
[ 1465.366821] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1127, name: kworker/2:1
[ 1465.367690] preempt_count: 1, expected: 0
[ 1465.368115] RCU nest depth: 0, expected: 0
[ 1465.368547] INFO: lockdep is turned off.
[ 1465.368984] CPU: 2 PID: 1127 Comm: kworker/2:1 Not tainted 6.6.0 #2
[ 1465.369642] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
[ 1465.370788] Workqueue: cifsiod smb2_reconnect_server [cifs]
[ 1465.371888] Call Trace:
[ 1465.372164]  <TASK>
[ 1465.372401]  dump_stack_lvl+0x64/0x80
[ 1465.372809]  __might_resched+0x23c/0x360
[ 1465.373236]  ? __pfx___might_resched+0x10/0x10
[ 1465.373722]  ? rcu_is_watching+0x23/0x50
[ 1465.374149]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.374661]  ? __might_sleep+0x2b/0xb0
[ 1465.375119]  ? __flush_work+0xc5/0x640
[ 1465.375524]  __flush_work+0xfd/0x640
[ 1465.375914]  ? __flush_work+0xc5/0x640
[ 1465.376320]  ? add_chain_block+0x1d8/0x280
[ 1465.376771]  ? __pfx___flush_work+0x10/0x10
[ 1465.377240]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.377686]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.378072]  ? try_to_grab_pending+0x47/0x3a0
[ 1465.378430]  __cancel_work_timer+0x210/0x2c0
[ 1465.378785]  ? __pfx___cancel_work_timer+0x10/0x10
[ 1465.379176]  ? do_raw_spin_trylock+0xd1/0x120
[ 1465.379537]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.379933]  ? do_raw_spin_unlock+0x9b/0x100
[ 1465.380293]  cifs_put_tcp_session+0x118/0x290 [cifs]
[ 1465.381076]  cifs_disable_secondary_channels+0xdb/0x280 [cifs]
[ 1465.381930]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.382323]  smb2_reconnect+0x520/0xcb0 [cifs]
[ 1465.383072]  smb2_reconnect_server+0x771/0xb00 [cifs]
[ 1465.383870]  ? __pfx_smb2_reconnect_server+0x10/0x10 [cifs]
[ 1465.384694]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.385084]  ? lock_is_held_type+0x90/0x100
[ 1465.385424]  ? mark_held_locks+0x1a/0x90
[ 1465.385708] CIFS: fs/smb/client/cifsfs.c: VFS: in cifs_statfs as Xid: 9 with uid: 0
[ 1465.385754]  process_one_work+0x43c/0x8e0
[ 1465.387019]  ? __pfx_process_one_work+0x10/0x10
[ 1465.387387]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.387821]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.388334]  ? __list_add_valid_or_report+0x37/0xf0
[ 1465.388870]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.389385]  worker_thread+0x397/0x690
[ 1465.389753]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.390144]  ? __kthread_parkme+0xce/0xf0
[ 1465.390476]  ? __pfx_worker_thread+0x10/0x10
[ 1465.390833]  kthread+0x18d/0x1d0
[ 1465.391103]  ? kthread+0xdb/0x1d0
[ 1465.391378]  ? __pfx_kthread+0x10/0x10
[ 1465.391695]  ret_from_fork+0x34/0x60
[ 1465.391991]  ? __pfx_kthread+0x10/0x10
[ 1465.392302]  ret_from_fork_asm+0x1b/0x30
[ 1465.392632]  </TASK>
[ 1465.393120] CIFS: fs/smb/client/connect.c: cifs_setup_session: channel connect bitmap: 0x1
[ 1465.393959] CIFS: fs/smb/client/connect.c: Free previous auth_key.response = 0000000018de71c6
[ 1465.393960] CIFS: fs/smb/client/smb2pdu.c: sess reconnect mask: 0x1, tcon reconnect: 0
[ 1465.394666] CIFS: fs/smb/client/connect.c: Security Mode: 0x1 Capabilities: 0x300067 TimeAdjust: 0
[ 1465.396191] CIFS: fs/smb/client/smb2pdu.c: Session Setup
[ 1465.396675] CIFS: fs/smb/client/smb2pdu.c: sess setup type 2
[ 1465.397269] CIFS: fs/smb/client/smb2pdu.c: Fresh session. Previous: 1e800d8000049
[ 1465.399253] CIFS: fs/smb/client/smb2maperror.c: Mapping SMB2 status code 0xc0000016 to POSIX err -5
[ 1465.400409] CIFS: fs/smb/client/sess.c: decode_ntlmssp_challenge: negotiate=0xe2088235 challenge=0xe2898235
[ 1465.401245] CIFS: fs/smb/client/smb2pdu.c: rawntlmssp session setup challenge phase
[ 1465.402058] CIFS: fs/smb/client/smb2pdu.c: Fresh session. Previous: 1e800d8000049
[ 1465.404585] CIFS: fs/smb/client/smb2pdu.c: SMB2/3 session established successfully
[ 1465.405543] CIFS: fs/smb/client/sess.c: Cleared reconnect bitmask for chan 0; now 0x0
[ 1465.406375] CIFS: fs/smb/client/connect.c: __cifs_put_smb_ses: ses_count=2
[ 1465.407092] CIFS: fs/smb/client/connect.c: __cifs_put_smb_ses: ses ipc: \\w22-root1.gandalf.test\IPC$
[ 1465.408048] CIFS: fs/smb/client/smb2pdu.c: Reconnecting tcons and channels finished
[ 1465.408794] CIFS: fs/smb/client/smb2pdu.c: Reconnecting tcons and channels
[ 1465.409349] CIFS: fs/smb/client/smb2pdu.c: Reconnecting tcons and channels finished
[ 1465.409949] CIFS: Server share \\w22-root1.gandalf.test\test deleted.
[ 1465.410961] CIFS: fs/smb/client/smb2maperror.c: Mapping SMB2 status code 0xc00000c9 to POSIX err -78
[ 1465.412204] CIFS: server share \\w22-root1.gandalf.test\test deleted
[ 1465.413298] CIFS: fs/smb/client/smb2pdu.c: sess reconnect mask: 0x0, tcon reconnect: 1
[ 1465.413311] CIFS: fs/smb/client/smb2pdu.c: reconnect tcon rc = 0
[ 1465.415272] CIFS: fs/smb/client/smb2pdu.c: sess reconnect mask: 0x0, tcon reconnect: 1
[ 1465.415285] CIFS: fs/smb/client/smb2pdu.c: reconnect tcon rc = 0
[ 1465.417047] CIFS: fs/smb/client/smb2pdu.c: Reconnecting tcons and channels
[ 1465.417911] CIFS: fs/smb/client/smb2pdu.c: sess reconnect mask: 0x0, tcon reconnect: 1
[ 1465.417924] CIFS: fs/smb/client/smb2pdu.c: reconnect tcon rc = 0
[ 1465.421431] CIFS: fs/smb/client/connect.c: cifs_put_tcon: tc_count=2
[ 1465.422397] CIFS: fs/smb/client/smb2pdu.c: Reconnecting tcons and channels finished
[ 1465.423725] ------------[ cut here ]------------
[ 1465.424158] WARNING: CPU: 3 PID: 85 at fs/smb/client/connect.c:1616 cifs_put_tcp_session+0x27a/0x290 [cifs]
[ 1465.425407] Modules linked in: cifs cifs_arc4 nls_ucs2_utils fscache cifs_md4 [last unloaded: cifs]
[ 1465.426242] CPU: 3 PID: 85 Comm: kworker/3:3 Tainted: G        W          6.6.0 #2
[ 1465.426929] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
[ 1465.427915] Workqueue: cifsiod smb2_reconnect_server [cifs]
[ 1465.428828] RIP: 0010:cifs_put_tcp_session+0x27a/0x290 [cifs]
[ 1465.429754] Code: b1 15 c2 e9 32 fe ff ff 89 ee 48 89 df e8 9e fd ff ff e9 cd fe ff ff be 03 00 00 00 4c 89 ef e8 ac 6d ad c1 e9 11 fe ff ff 90 <0f> 0b 90 e9 c0 fd ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00
[ 1465.431382] RSP: 0018:ffff88800b8afc10 EFLAGS: 00010286
[ 1465.431872] RAX: 00000000ffffffff RBX: ffff88800dd7a000 RCX: dffffc0000000000
[ 1465.432498] RDX: 0000000000000003 RSI: ffffffffc00e901b RDI: ffff88800dd7a070
[ 1465.433134] RBP: 0000000000000001 R08: 0000000000000000 R09: fffffbfff082290a
[ 1465.433788] R10: ffffffff84114857 R11: 0000000000000000 R12: 1ffff11001715f90
[ 1465.434416] R13: ffff88800b8afca0 R14: ffff88800dd7a7a0 R15: ffff88800b8afc88
[ 1465.435086] FS:  0000000000000000(0000) GS:ffff88805b000000(0000) knlGS:0000000000000000
[ 1465.435818] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1465.436330] CR2: 00007f00682affae CR3: 0000000009fe6000 CR4: 0000000000750ef0
[ 1465.436981] PKRU: 55555554
[ 1465.437227] Call Trace:
[ 1465.437458]  <TASK>
[ 1465.437676]  ? __warn+0xa5/0x200
[ 1465.437979]  ? cifs_put_tcp_session+0x27a/0x290 [cifs]
[ 1465.438833]  ? report_bug+0x1b2/0x1e0
[ 1465.439168]  ? handle_bug+0x6f/0x90
[ 1465.439484]  ? exc_invalid_op+0x17/0x50
[ 1465.439846]  ? asm_exc_invalid_op+0x1a/0x20
[ 1465.440247]  ? cifs_put_tcp_session+0x2b/0x290 [cifs]
[ 1465.441102]  ? cifs_put_tcp_session+0x27a/0x290 [cifs]
[ 1465.441970]  smb2_reconnect_server+0x646/0xb00 [cifs]
[ 1465.442823]  ? lock_sync+0xd0/0xe0
[ 1465.443138]  ? __pfx_smb2_reconnect_server+0x10/0x10 [cifs]
[ 1465.444042]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.444467]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.444912]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.445342]  ? read_word_at_a_time+0xe/0x20
[ 1465.445727]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.446150]  process_one_work+0x43c/0x8e0
[ 1465.446513]  ? __pfx_process_one_work+0x10/0x10
[ 1465.446946]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.447376]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.447826]  ? __list_add_valid_or_report+0x37/0xf0
[ 1465.448278]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.448731]  worker_thread+0x397/0x690
[ 1465.449080]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1465.449515]  ? __kthread_parkme+0xce/0xf0
[ 1465.449991]  ? __pfx_worker_thread+0x10/0x10
[ 1465.450390]  kthread+0x18d/0x1d0
[ 1465.450712]  ? kthread+0xdb/0x1d0
[ 1465.451022]  ? __pfx_kthread+0x10/0x10
[ 1465.451362]  ret_from_fork+0x34/0x60
[ 1465.451699]  ? __pfx_kthread+0x10/0x10
[ 1465.452038]  ret_from_fork_asm+0x1b/0x30
[ 1465.452405]  </TASK>
[ 1465.452605] irq event stamp: 87714
[ 1465.452927] hardirqs last  enabled at (87713): [<ffffffff82864168>] _raw_spin_unlock_irq+0x28/0x50
[ 1465.453732] hardirqs last disabled at (87714): [<ffffffff82856f8d>] __schedule+0xc0d/0x1560
[ 1465.454468] softirqs last  enabled at (87624): [<ffffffff8110918c>] process_one_work+0x43c/0x8e0
[ 1465.455257] softirqs last disabled at (87620): [<ffffffff8227f67a>] neigh_managed_work+0x2a/0x110
[ 1465.456055] ---[ end trace 0000000000000000 ]---
[ 1465.456937] CIFS: fs/smb/client/cifsfs.c: VFS: leaving cifs_statfs (xid = 9) rc = -11
[ 1528.811802] CIFS: fs/smb/client/connect.c: VFS: in smb2_query_server_interfaces as Xid: 10 with uid: 0
[ 1528.813395] CIFS: fs/smb/client/smb2pdu.c: SMB2 IOCTL
[ 1528.814246] ==================================================================
[ 1528.815401] BUG: KASAN: slab-use-after-free in cifs_pick_channel+0xa2/0x170 [cifs]
[ 1528.817404] Read of size 1 at addr ffff88800dd7a2c4 by task kworker/0:2/1114
[ 1528.818539] 
[ 1528.818810] CPU: 0 PID: 1114 Comm: kworker/0:2 Tainted: G        W          6.6.0 #2
[ 1528.820040] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
[ 1528.821789] Workqueue: cifsiod smb2_query_server_interfaces [cifs]
[ 1528.823573] Call Trace:
[ 1528.823991]  <TASK>
[ 1528.824360]  dump_stack_lvl+0x4a/0x80
[ 1528.824976]  print_report+0xcf/0x650
[ 1528.825577]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.826361]  ? rcu_is_watching+0x23/0x50
[ 1528.827011]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.827804]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.828590]  ? __phys_addr+0x46/0x90
[ 1528.829193]  kasan_report+0xd8/0x110
[ 1528.829794]  ? cifs_pick_channel+0xa2/0x170 [cifs]
[ 1528.831379]  ? cifs_pick_channel+0xa2/0x170 [cifs]
[ 1528.832952]  cifs_pick_channel+0xa2/0x170 [cifs]
[ 1528.834506]  SMB2_ioctl+0x1b5/0x6f0 [cifs]
[ 1528.835969]  ? __pfx_console_unlock+0x10/0x10
[ 1528.836692]  ? tick_nohz_tick_stopped+0x21/0x30
[ 1528.837441]  ? __pfx_SMB2_ioctl+0x10/0x10 [cifs]
[ 1528.838992]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.839779]  ? irq_work_queue+0x2c/0x40
[ 1528.840416]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.841210]  ? lock_acquire+0xc1/0x3a0
[ 1528.841840]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.842626]  ? _printk+0xc0/0xf0
[ 1528.843191]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.843977]  ? __dynamic_pr_debug+0x1f5/0x260
[ 1528.844705]  ? __pfx___dynamic_pr_debug+0x10/0x10
[ 1528.845476]  ? lock_release+0xb6/0x5a0
[ 1528.846104]  ? __pfx_lock_acquire+0x10/0x10
[ 1528.846802]  ? SMB3_request_interfaces+0x137/0x2b0 [cifs]
[ 1528.848468]  SMB3_request_interfaces+0x137/0x2b0 [cifs]
[ 1528.850103]  ? __pfx_SMB3_request_interfaces+0x10/0x10 [cifs]
[ 1528.851828]  ? ___ratelimit+0x133/0x210
[ 1528.852535]  ? __pfx____ratelimit+0x10/0x10
[ 1528.853207]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.853839]  smb2_query_server_interfaces+0x54/0x1f0 [cifs]
[ 1528.855179]  process_one_work+0x43c/0x8e0
[ 1528.855848]  ? __pfx_process_one_work+0x10/0x10
[ 1528.856595]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.857380]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.858170]  ? __list_add_valid_or_report+0x37/0xf0
[ 1528.858985]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.859773]  worker_thread+0x397/0x690
[ 1528.860410]  ? __pfx_worker_thread+0x10/0x10
[ 1528.861124]  kthread+0x18d/0x1d0
[ 1528.861668]  ? kthread+0xdb/0x1d0
[ 1528.862226]  ? __pfx_kthread+0x10/0x10
[ 1528.862851]  ret_from_fork+0x34/0x60
[ 1528.863449]  ? __pfx_kthread+0x10/0x10
[ 1528.864074]  ret_from_fork_asm+0x1b/0x30
[ 1528.864745]  </TASK>
[ 1528.865120] 
[ 1528.865389] Allocated by task 1253:
[ 1528.865965]  kasan_save_stack+0x33/0x60
[ 1528.866602]  kasan_set_track+0x25/0x30
[ 1528.867226]  __kasan_kmalloc+0x8f/0xa0
[ 1528.867856]  cifs_get_tcp_session+0xbc/0xc70 [cifs]
[ 1528.869161]  cifs_mount_get_session+0x70/0x220 [cifs]
[ 1528.870425]  dfs_mount_share+0x249/0x1060 [cifs]
[ 1528.871650]  cifs_mount+0xda/0x4c0 [cifs]
[ 1528.872785]  cifs_smb3_do_mount+0x1e5/0xcc0 [cifs]
[ 1528.874016]  smb3_get_tree+0x16d/0x380 [cifs]
[ 1528.875208]  vfs_get_tree+0x4d/0x190
[ 1528.875684]  path_mount+0x3c4/0xf90
[ 1528.876149]  __x64_sys_mount+0x1aa/0x1f0
[ 1528.876664]  do_syscall_64+0x47/0xf0
[ 1528.877140]  entry_SYSCALL_64_after_hwframe+0x6f/0x77
[ 1528.877800] 
[ 1528.878013] Freed by task 1255:
[ 1528.878428]  kasan_save_stack+0x33/0x60
[ 1528.878934]  kasan_set_track+0x25/0x30
[ 1528.879430]  kasan_save_free_info+0x2b/0x50
[ 1528.879975]  ____kasan_slab_free+0x126/0x170
[ 1528.880536]  slab_free_freelist_hook+0x9d/0x1e0
[ 1528.881131]  __kmem_cache_free+0x9d/0x190
[ 1528.881506]  clean_demultiplex_info+0x3bb/0x640 [cifs]
[ 1528.882306]  cifs_demultiplex_thread+0x3de/0x1270 [cifs]
[ 1528.883086]  kthread+0x18d/0x1d0
[ 1528.883351]  ret_from_fork+0x34/0x60
[ 1528.883643]  ret_from_fork_asm+0x1b/0x30
[ 1528.883960] 
[ 1528.884092] Last potentially related work creation:
[ 1528.884481]  kasan_save_stack+0x33/0x60
[ 1528.884792]  __kasan_record_aux_stack+0x94/0xa0
[ 1528.885155]  __queue_work+0x334/0x8a0
[ 1528.885454]  mod_delayed_work_on+0xa5/0x100
[ 1528.885792]  smb2_reconnect+0x735/0xcb0 [cifs]
[ 1528.886518]  SMB2_query_info_init+0xca/0x250 [cifs]
[ 1528.887273]  smb2_query_info_compound+0x473/0x6d0 [cifs]
[ 1528.888065]  smb2_queryfs+0xc2/0x2c0 [cifs]
[ 1528.888763]  smb311_queryfs+0x210/0x220 [cifs]
[ 1528.889485]  cifs_statfs+0x164/0x290 [cifs]
[ 1528.890185]  statfs_by_dentry+0x9b/0xf0
[ 1528.890497]  user_statfs+0xab/0x130
[ 1528.890783]  __do_sys_statfs+0x81/0xe0
[ 1528.891090]  do_syscall_64+0x47/0xf0
[ 1528.891383]  entry_SYSCALL_64_after_hwframe+0x6f/0x77
[ 1528.891786] 
[ 1528.891918] Second to last potentially related work creation:
[ 1528.892368]  kasan_save_stack+0x33/0x60
[ 1528.892679]  __kasan_record_aux_stack+0x94/0xa0
[ 1528.893043]  __queue_work+0x334/0x8a0
[ 1528.893342]  mod_delayed_work_on+0xa5/0x100
[ 1528.893680]  smb2_reconnect+0x735/0xcb0 [cifs]
[ 1528.894409]  SMB2_open_init+0xf8/0x13f0 [cifs]
[ 1528.895135]  smb2_query_info_compound+0x2b6/0x6d0 [cifs]
[ 1528.895925]  smb2_queryfs+0xc2/0x2c0 [cifs]
[ 1528.896628]  smb311_queryfs+0x210/0x220 [cifs]
[ 1528.897353]  cifs_statfs+0x164/0x290 [cifs]
[ 1528.898060]  statfs_by_dentry+0x9b/0xf0
[ 1528.898371]  user_statfs+0xab/0x130
[ 1528.898656]  __do_sys_statfs+0x81/0xe0
[ 1528.898961]  do_syscall_64+0x47/0xf0
[ 1528.899255]  entry_SYSCALL_64_after_hwframe+0x6f/0x77
[ 1528.899658] 
[ 1528.899790] The buggy address belongs to the object at ffff88800dd7a000
[ 1528.899790]  which belongs to the cache kmalloc-4k of size 4096
[ 1528.900751] The buggy address is located 708 bytes inside of
[ 1528.900751]  freed 4096-byte region [ffff88800dd7a000, ffff88800dd7b000)
[ 1528.901699] 
[ 1528.901830] The buggy address belongs to the physical page:
[ 1528.902268] page:00000000b974ece7 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0xdd78
[ 1528.902981] head:00000000b974ece7 order:3 entire_mapcount:0 nr_pages_mapped:0 pincount:0
[ 1528.903609] anon flags: 0x100000000000840(slab|head|node=0|zone=1)
[ 1528.904098] page_type: 0xffffffff()
[ 1528.904386] raw: 0100000000000840 ffff888006443040 0000000000000000 dead000000000001
[ 1528.904989] raw: 0000000000000000 0000000000040004 00000001ffffffff 0000000000000000
[ 1528.905591] page dumped because: kasan: bad access detected
[ 1528.906030] 
[ 1528.906161] Memory state around the buggy address:
[ 1528.906542]  ffff88800dd7a180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[ 1528.907107]  ffff88800dd7a200: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[ 1528.907675] >ffff88800dd7a280: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[ 1528.908240]                                            ^
[ 1528.908660]  ffff88800dd7a300: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[ 1528.909225]  ffff88800dd7a380: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[ 1528.909787] ==================================================================
[ 1528.910379] Kernel panic - not syncing: kasan.fault=panic set ...
[ 1528.910867] CPU: 0 PID: 1114 Comm: kworker/0:2 Tainted: G        W          6.6.0 #2
[ 1528.911475] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
[ 1528.912368] Workqueue: cifsiod smb2_query_server_interfaces [cifs]
[ 1528.913235] Call Trace:
[ 1528.913439]  <TASK>
[ 1528.913616]  dump_stack_lvl+0x4a/0x80
[ 1528.913917]  panic+0x41f/0x460
[ 1528.914177]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.914563]  ? __pfx_panic+0x10/0x10
[ 1528.914855]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.915248]  ? check_panic_on_warn+0x2f/0x80
[ 1528.915594]  end_report+0x125/0x130
[ 1528.915882]  kasan_report+0xe8/0x110
[ 1528.916176]  ? cifs_pick_channel+0xa2/0x170 [cifs]
[ 1528.916935]  ? cifs_pick_channel+0xa2/0x170 [cifs]
[ 1528.917695]  cifs_pick_channel+0xa2/0x170 [cifs]
[ 1528.918437]  SMB2_ioctl+0x1b5/0x6f0 [cifs]
[ 1528.919138]  ? __pfx_console_unlock+0x10/0x10
[ 1528.919490]  ? tick_nohz_tick_stopped+0x21/0x30
[ 1528.919857]  ? __pfx_SMB2_ioctl+0x10/0x10 [cifs]
[ 1528.920596]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.920985]  ? irq_work_queue+0x2c/0x40
[ 1528.921298]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.921685]  ? lock_acquire+0xc1/0x3a0
[ 1528.921998]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.922416]  ? _printk+0xc0/0xf0
[ 1528.922726]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.923157]  ? __dynamic_pr_debug+0x1f5/0x260
[ 1528.923546]  ? __pfx___dynamic_pr_debug+0x10/0x10
[ 1528.924097]  ? lock_release+0xb6/0x5a0
[ 1528.924707]  ? __pfx_lock_acquire+0x10/0x10
[ 1528.925558]  ? SMB3_request_interfaces+0x137/0x2b0 [cifs]
[ 1528.926794]  SMB3_request_interfaces+0x137/0x2b0 [cifs]
[ 1528.927962]  ? __pfx_SMB3_request_interfaces+0x10/0x10 [cifs]
[ 1528.929230]  ? ___ratelimit+0x133/0x210
[ 1528.929847]  ? __pfx____ratelimit+0x10/0x10
[ 1528.930217]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.930730]  smb2_query_server_interfaces+0x54/0x1f0 [cifs]
[ 1528.931564]  process_one_work+0x43c/0x8e0
[ 1528.931900]  ? __pfx_process_one_work+0x10/0x10
[ 1528.932273]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.932663]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.933055]  ? __list_add_valid_or_report+0x37/0xf0
[ 1528.933459]  ? srso_alias_return_thunk+0x5/0xfbef5
[ 1528.933852]  worker_thread+0x397/0x690
[ 1528.934173]  ? __pfx_worker_thread+0x10/0x10
[ 1528.934526]  kthread+0x18d/0x1d0
[ 1528.934799]  ? kthread+0xdb/0x1d0
[ 1528.935077]  ? __pfx_kthread+0x10/0x10
[ 1528.935388]  ret_from_fork+0x34/0x60
[ 1528.935686]  ? __pfx_kthread+0x10/0x10
[ 1528.935997]  ret_from_fork_asm+0x1b/0x30
[ 1528.936328]  </TASK>
[ 1528.937856] Kernel Offset: disabled
[ 1528.938146] ---[ end Kernel panic - not syncing: kasan.fault=panic set ... ]---

$ mount.cifs //w22-root1.gandalf.test/test /mnt/1 -o username=administrator,password=***
# ...wait until smb2_query_server_interfaces() is executed at least once...
$ cat /sys/kernel/debug/kmemleak
unreferenced object 0xffff888007821c00 (size 192):
  comm "mount.cifs", pid 885, jiffies 4294765292 (age 656.515s)
  hex dump (first 32 bytes):
    f0 52 da 16 80 88 ff ff f0 52 da 16 80 88 ff ff  .R.......R......
    01 00 00 00 00 00 00 00 00 ca 9a 3b 00 00 00 00  ...........;....
  backtrace:
    [<ffffffff8144e3f5>] __kmem_cache_alloc_node+0x295/0x2d0
    [<ffffffff813ab1aa>] kmalloc_trace+0x2a/0xc0
    [<ffffffffc0141c9d>] parse_server_interfaces+0x4ed/0xcc0 [cifs]
    [<ffffffffc014b323>] SMB3_request_interfaces+0x163/0x2b0 [cifs]
    [<ffffffffc014b5ed>] smb3_qfs_tcon+0x16d/0x2c0 [cifs]
    [<ffffffffc00f02c1>] cifs_mount_get_tcon+0x3b1/0x550 [cifs]
    [<ffffffffc0181f7a>] dfs_mount_share+0x1da/0x1060 [cifs]
    [<ffffffffc00f089a>] cifs_mount+0xda/0x4c0 [cifs]
    [<ffffffffc00e06d5>] cifs_smb3_do_mount+0x1e5/0xcc0 [cifs]
    [<ffffffffc0175a1d>] smb3_get_tree+0x16d/0x380 [cifs]
    [<ffffffff8147c1ad>] vfs_get_tree+0x4d/0x190
    [<ffffffff814c66d4>] path_mount+0x3c4/0xf90
    [<ffffffff814c7b5a>] __x64_sys_mount+0x1aa/0x1f0
    [<ffffffff8284c3d7>] do_syscall_64+0x47/0xf0
    [<ffffffff82a000eb>] entry_SYSCALL_64_after_hwframe+0x6f/0x77
    ...

$ ./scripts/faddr2line --list fs/smb/client/cifs.o parse_server_interfaces+0x4ed
parse_server_interfaces+0x4ed/0xcc0:

kmalloc at /home/pc/g/linux/./include/linux/slab.h:600
 595 
 596                    if (size > KMALLOC_MAX_CACHE_SIZE)
 597                            return kmalloc_large(size, flags);
 598 
 599                    index = kmalloc_index(size);
>600<                   return kmalloc_trace(
 601                                    kmalloc_caches[kmalloc_type(flags, _RET_IP_)][index],
 602                                    flags, size);
 603            }
 604            return __kmalloc(size, flags);
 605    }

(inlined by) parse_server_interfaces at /home/pc/g/linux/fs/smb/client/smb2ops.c:694
 689                            }
 690                    }
 691                    spin_unlock(&ses->iface_lock);
 692 
 693                    /* no match. insert the entry in the list */
>694<                   info = kmalloc(sizeof(struct cifs_server_iface),
 695                                   GFP_KERNEL);
 696                    if (!info) {
 697                            rc = -ENOMEM;
 698                            goto out;
 699                    }

Re: [PATCH 12/14] cifs: handle when server stops supporting multichannel

[Steve French]

I did reproduce a problem (memory leak) though.  After umount in the
scenario described below (similar to Paulo's repro).  I can umount but
am unable to do an rmmod, and the memory leak detector did spot the
following:

root@smfrench-ThinkPad-P52:/home/smfrench# cat /sys/kernel/debug/kmemleak
unreferenced object 0xffff8881a1186a00 (size 192):
  comm "apparmor_parser", pid 987, jiffies 4294916984 (age 2878.840s)
  hex dump (first 32 bytes):
    00 00 00 00 00 00 00 00 00 07 01 9f 81 88 ff ff  ................
    00 80 40 7a 81 88 ff ff 14 00 00 00 00 00 00 00  ..@z............
  backtrace:
    [<ffffffff8ac40b4a>] kmemleak_alloc+0x4a/0x90
    [<ffffffff898a94c3>] __kmem_cache_alloc_node+0x233/0x350
    [<ffffffff897cc67a>] kmalloc_trace+0x2a/0xf0
    [<ffffffff89ce725c>] aa_alloc_pdb+0x5c/0xd0
    [<ffffffff89cece8e>] unpack_pdb+0xbe/0x1080
    [<ffffffff89cee1da>] unpack_profile+0x37a/0x1fe0
    [<ffffffff89cf0704>] aa_unpack+0x254/0xba0
    [<ffffffff89ce9681>] aa_replace_profiles+0x151/0x1e50
    [<ffffffff89cd2bef>] policy_update+0x13f/0x1b0
    [<ffffffff89cd2da9>] profile_replace+0x139/0x180
    [<ffffffff899275ae>] vfs_write+0x1de/0x720
    [<ffffffff8992812f>] ksys_write+0xdf/0x190
    [<ffffffff89928232>] __x64_sys_write+0x42/0x60
    [<ffffffff8ac2fdcc>] do_syscall_64+0x5c/0xf0
    [<ffffffff8ae000e6>] entry_SYSCALL_64_after_hwframe+0x6e/0x76
unreferenced object 0xffff8881a1187b00 (size 192):
  comm "apparmor_parser", pid 987, jiffies 4294916984 (age 2878.840s)
  hex dump (first 32 bytes):
    00 00 00 00 00 00 00 00 00 09 01 9f 81 88 ff ff  ................
    00 90 40 7a 81 88 ff ff 24 00 00 00 00 00 00 00  ..@z....$.......
  backtrace:
    [<ffffffff8ac40b4a>] kmemleak_alloc+0x4a/0x90
    [<ffffffff898a94c3>] __kmem_cache_alloc_node+0x233/0x350
    [<ffffffff897cc67a>] kmalloc_trace+0x2a/0xf0
    [<ffffffff89ce725c>] aa_alloc_pdb+0x5c/0xd0
    [<ffffffff89cece8e>] unpack_pdb+0xbe/0x1080
    [<ffffffff89cef26b>] unpack_profile+0x140b/0x1fe0
    [<ffffffff89cf0704>] aa_unpack+0x254/0xba0
    [<ffffffff89ce9681>] aa_replace_profiles+0x151/0x1e50
    [<ffffffff89cd2bef>] policy_update+0x13f/0x1b0
    [<ffffffff89cd2da9>] profile_replace+0x139/0x180
    [<ffffffff899275ae>] vfs_write+0x1de/0x720
    [<ffffffff8992812f>] ksys_write+0xdf/0x190
    [<ffffffff89928232>] __x64_sys_write+0x42/0x60
    [<ffffffff8ac2fdcc>] do_syscall_64+0x5c/0xf0
    [<ffffffff8ae000e6>] entry_SYSCALL_64_after_hwframe+0x6e/0x76
unreferenced object 0xffff8881824d9600 (size 192):
  comm "apparmor_parser", pid 987, jiffies 4294916984 (age 2878.844s)
  hex dump (first 32 bytes):
    00 00 00 00 00 00 00 00 00 0a 01 9f 81 88 ff ff  ................
    00 00 78 65 81 88 ff ff e8 0b 00 00 00 00 00 00  ..xe............
  backtrace:
    [<ffffffff8ac40b4a>] kmemleak_alloc+0x4a/0x90
    [<ffffffff898a94c3>] __kmem_cache_alloc_node+0x233/0x350
    [<ffffffff897cc67a>] kmalloc_trace+0x2a/0xf0
    [<ffffffff89ce725c>] aa_alloc_pdb+0x5c/0xd0
    [<ffffffff89cece8e>] unpack_pdb+0xbe/0x1080
    [<ffffffff89ceeaae>] unpack_profile+0xc4e/0x1fe0
    [<ffffffff89cf0704>] aa_unpack+0x254/0xba0
    [<ffffffff89ce9681>] aa_replace_profiles+0x151/0x1e50
    [<ffffffff89cd2bef>] policy_update+0x13f/0x1b0
    [<ffffffff89cd2da9>] profile_replace+0x139/0x180
    [<ffffffff899275ae>] vfs_write+0x1de/0x720
    [<ffffffff8992812f>] ksys_write+0xdf/0x190
    [<ffffffff89928232>] __x64_sys_write+0x42/0x60
    [<ffffffff8ac2fdcc>] do_syscall_64+0x5c/0xf0
    [<ffffffff8ae000e6>] entry_SYSCALL_64_after_hwframe+0x6e/0x76
unreferenced object 0xffff888105f81600 (size 192):
  comm "mount.cifs", pid 4420, jiffies 4295067087 (age 2279.232s)
  hex dump (first 32 bytes):
    d0 02 16 90 88 88 ff ff d0 02 16 90 88 88 ff ff  ................
    01 00 00 00 00 00 00 00 00 e4 0b 54 02 00 00 00  ...........T....
  backtrace:
    [<ffffffff8ac40b4a>] kmemleak_alloc+0x4a/0x90
    [<ffffffff898a94c3>] __kmem_cache_alloc_node+0x233/0x350
    [<ffffffff897cc67a>] kmalloc_trace+0x2a/0xf0
    [<ffffffffc4360b85>] parse_server_interfaces+0x4c5/0xc20 [cifs]
    [<ffffffffc436bbce>] SMB3_request_interfaces+0x17e/0x2e0 [cifs]
    [<ffffffffc436becb>] smb3_qfs_tcon+0x18b/0x2d0 [cifs]
    [<ffffffffc43062b8>] cifs_mount_get_tcon+0x3c8/0x570 [cifs]
    [<ffffffffc43a776d>] dfs_mount_share+0x15d/0x1190 [cifs]
    [<ffffffffc43068b2>] cifs_mount+0xe2/0x5a0 [cifs]
    [<ffffffffc42f489f>] cifs_smb3_do_mount+0x1ef/0xd60 [cifs]
    [<ffffffffc439a501>] smb3_get_tree+0x1f1/0x3e0 [cifs]
    [<ffffffff8992bccd>] vfs_get_tree+0x4d/0x190
    [<ffffffff8997bc15>] path_mount+0x675/0x1040
    [<ffffffff8997c7e8>] __x64_sys_mount+0x1f8/0x240
    [<ffffffff8ac2fdcc>] do_syscall_64+0x5c/0xf0
    [<ffffffff8ae000e6>] entry_SYSCALL_64_after_hwframe+0x6e/0x76
unreferenced object 0xffff888104ac2a00 (size 192):
  comm "mount.cifs", pid 4734, jiffies 4295145078 (age 1967.296s)
  hex dump (first 32 bytes):
    d0 e2 51 33 81 88 ff ff d0 e2 51 33 81 88 ff ff  ..Q3......Q3....
    01 00 00 00 00 00 00 00 00 e4 0b 54 02 00 00 00  ...........T....
  backtrace:
    [<ffffffff8ac40b4a>] kmemleak_alloc+0x4a/0x90
    [<ffffffff898a94c3>] __kmem_cache_alloc_node+0x233/0x350
    [<ffffffff897cc67a>] kmalloc_trace+0x2a/0xf0
    [<ffffffffc4360b85>] parse_server_interfaces+0x4c5/0xc20 [cifs]
    [<ffffffffc436bbce>] SMB3_request_interfaces+0x17e/0x2e0 [cifs]
    [<ffffffffc436becb>] smb3_qfs_tcon+0x18b/0x2d0 [cifs]
    [<ffffffffc43062b8>] cifs_mount_get_tcon+0x3c8/0x570 [cifs]
    [<ffffffffc43a776d>] dfs_mount_share+0x15d/0x1190 [cifs]
    [<ffffffffc43068b2>] cifs_mount+0xe2/0x5a0 [cifs]
    [<ffffffffc42f489f>] cifs_smb3_do_mount+0x1ef/0xd60 [cifs]
    [<ffffffffc439a501>] smb3_get_tree+0x1f1/0x3e0 [cifs]
    [<ffffffff8992bccd>] vfs_get_tree+0x4d/0x190
    [<ffffffff8997bc15>] path_mount+0x675/0x1040
    [<ffffffff8997c7e8>] __x64_sys_mount+0x1f8/0x240
    [<ffffffff8ac2fdcc>] do_syscall_64+0x5c/0xf0
    [<ffffffff8ae000e6>] entry_SYSCALL_64_after_hwframe+0x6e/0x76
unreferenced object 0xffff8881103ece00 (size 192):
  comm "mount.cifs", pid 5653, jiffies 4295317051 (age 1279.476s)
  hex dump (first 32 bytes):
    d0 e2 b4 25 81 88 ff ff d0 e2 b4 25 81 88 ff ff  ...%.......%....
    01 00 00 00 00 00 00 00 80 5e 9b 08 00 00 00 00  .........^......
  backtrace:
    [<ffffffff8ac40b4a>] kmemleak_alloc+0x4a/0x90
    [<ffffffff898a94c3>] __kmem_cache_alloc_node+0x233/0x350
    [<ffffffff897cc67a>] kmalloc_trace+0x2a/0xf0
    [<ffffffffc4360b85>] parse_server_interfaces+0x4c5/0xc20 [cifs]
    [<ffffffffc436bbce>] SMB3_request_interfaces+0x17e/0x2e0 [cifs]
    [<ffffffffc436becb>] smb3_qfs_tcon+0x18b/0x2d0 [cifs]
    [<ffffffffc43062b8>] cifs_mount_get_tcon+0x3c8/0x570 [cifs]
    [<ffffffffc43a776d>] dfs_mount_share+0x15d/0x1190 [cifs]
    [<ffffffffc43068b2>] cifs_mount+0xe2/0x5a0 [cifs]
    [<ffffffffc42f489f>] cifs_smb3_do_mount+0x1ef/0xd60 [cifs]
    [<ffffffffc439a501>] smb3_get_tree+0x1f1/0x3e0 [cifs]
    [<ffffffff8992bccd>] vfs_get_tree+0x4d/0x190
    [<ffffffff8997bc15>] path_mount+0x675/0x1040
    [<ffffffff8997c7e8>] __x64_sys_mount+0x1f8/0x240
    [<ffffffff8ac2fdcc>] do_syscall_64+0x5c/0xf0
    [<ffffffff8ae000e6>] entry_SYSCALL_64_after_hwframe+0x6e/0x76


On Sat, Nov 11, 2023 at 11:23 AM Paulo Alcantara <pc@manguebit.com> wrote:
>
> Shyam Prasad N <nspmangalore@gmail.com> writes:
>
> > Can you please check if the problem is still seen with these updated patches?
> > I was unable to reproduce the issue with the steps you provided.
>
> I couldn't reproduce that reconnect issue anymore.
>
> However, some other problems as shown below with lockdep, kmemleak and
> kasan enabled.
>
> $ mount.cifs //w22-root1.gandalf.test/test /mnt/1 -o username=administrator,password=***,multichannel,echo_interval=10
> # disable multichannel on w22-root1.gandalf.test server
> #   Set-SmbServerConfiguration -EnableMultichannel $false
> $ iptables -I INPUT -s 192.168.1.11 -j DROP
> $ stat -f /mnt/1
> stat: cannot read file system information for '/mnt/1': Host is down
> $ iptables -I INPUT -s 192.168.1.11 -j ACCEPT
> $ stat -f /mnt/1
> stat: cannot read file system information for '/mnt/1': Resource temporarily unavailable
>
> ...
> [ 1465.290096] CIFS: VFS: \\w22-root1.gandalf.test does not support multichannel anymore. disabling all other channels
> [ 1465.292488]
> [ 1465.292863] ======================================================
> [ 1465.294183] WARNING: possible circular locking dependency detected
> [ 1465.295309] 6.6.0 #2 Not tainted
> [ 1465.295863] ------------------------------------------------------
> [ 1465.296876] kworker/2:1/1127 is trying to acquire lock:
> [ 1465.297745] ffff8880129482c0 (&ret_buf->iface_lock){+.+.}-{2:2}, at: cifs_disable_secondary_channels+0x117/0x280 [cifs]
> [ 1465.300314]
> [ 1465.300314] but task is already holding lock:
> [ 1465.301267] ffff888012948328 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_disable_secondary_channels+0x29/0x280 [cifs]
> [ 1465.303804]
> [ 1465.303804] which lock already depends on the new lock.
> [ 1465.303804]
> [ 1465.305107]
> [ 1465.305107] the existing dependency chain (in reverse order) is:
> [ 1465.306309]
> [ 1465.306309] -> #1 (&ret_buf->chan_lock){+.+.}-{2:2}:
> [ 1465.307391]        _raw_spin_lock+0x34/0x80
> [ 1465.308093]        cifs_try_adding_channels+0x205/0x1290 [cifs]
> [ 1465.309865]        cifs_mount+0xfb/0x4c0 [cifs]
> [ 1465.311423]        cifs_smb3_do_mount+0x1e5/0xcc0 [cifs]
> [ 1465.313094]        smb3_get_tree+0x16d/0x380 [cifs]
> [ 1465.314702]        vfs_get_tree+0x4d/0x190
> [ 1465.315275]        path_mount+0x3c4/0xf90
> [ 1465.315811]        __x64_sys_mount+0x1aa/0x1f0
> [ 1465.316392]        do_syscall_64+0x47/0xf0
> [ 1465.316950]        entry_SYSCALL_64_after_hwframe+0x6f/0x77
> [ 1465.317681]
> [ 1465.317681] -> #0 (&ret_buf->iface_lock){+.+.}-{2:2}:
> [ 1465.318548]        __lock_acquire+0x1793/0x2110
> [ 1465.319156]        lock_acquire+0x14a/0x3a0
> [ 1465.319714]        _raw_spin_lock+0x34/0x80
> [ 1465.320266]        cifs_disable_secondary_channels+0x117/0x280 [cifs]
> [ 1465.321729]        smb2_reconnect+0x520/0xcb0 [cifs]
> [ 1465.323001]        smb2_reconnect_server+0x771/0xb00 [cifs]
> [ 1465.324352]        process_one_work+0x43c/0x8e0
> [ 1465.324959]        worker_thread+0x397/0x690
> [ 1465.325522]        kthread+0x18d/0x1d0
> [ 1465.326028]        ret_from_fork+0x34/0x60
> [ 1465.326564]        ret_from_fork_asm+0x1b/0x30
> [ 1465.327164]
> [ 1465.327164] other info that might help us debug this:
> [ 1465.327164]
> [ 1465.328180]  Possible unsafe locking scenario:
> [ 1465.328180]
> [ 1465.328943]        CPU0                    CPU1
> [ 1465.329523]        ----                    ----
> [ 1465.330119]   lock(&ret_buf->chan_lock);
> [ 1465.330649]                                lock(&ret_buf->iface_lock);
> [ 1465.331493]                                lock(&ret_buf->chan_lock);
> [ 1465.332326]   lock(&ret_buf->iface_lock);
> [ 1465.332865]
> [ 1465.332865]  *** DEADLOCK ***
> [ 1465.332865]
> [ 1465.333623] 5 locks held by kworker/2:1/1127:
> [ 1465.334199]  #0: ffff88800dd09d48 ((wq_completion)cifsiod){+.+.}-{0:0}, at: process_one_work+0x39a/0x8e0
> [ 1465.335430]  #1: ffff88800de97dc0 ((work_completion)(&(&tcp_ses->reconnect)->work)){+.+.}-{0:0}, at: process_one_work+0x39a/0x8e0
> [ 1465.336947]  #2: ffff88800dd7a8c8 (&tcp_ses->reconnect_mutex){+.+.}-{3:3}, at: smb2_reconnect_server+0xde/0xb00 [cifs]
> [ 1465.338953]  #3: ffff8880129480f0 (&ret_buf->session_mutex){+.+.}-{3:3}, at: smb2_reconnect+0x234/0xcb0 [cifs]
> [ 1465.340838]  #4: ffff888012948328 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_disable_secondary_channels+0x29/0x280 [cifs]
> [ 1465.342073]
> [ 1465.342073] stack backtrace:
> [ 1465.342421] CPU: 2 PID: 1127 Comm: kworker/2:1 Not tainted 6.6.0 #2
> [ 1465.342917] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
> [ 1465.343782] Workqueue: cifsiod smb2_reconnect_server [cifs]
> [ 1465.344595] Call Trace:
> [ 1465.344845]  <TASK>
> [ 1465.345025]  dump_stack_lvl+0x4a/0x80
> [ 1465.345329]  check_noncircular+0x269/0x2b0
> [ 1465.345672]  ? __pfx_check_noncircular+0x10/0x10
> [ 1465.346047]  ? __pfx_stack_trace_save+0x10/0x10
> [ 1465.346420]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.346814]  ? lockdep_lock+0xa3/0x160
> [ 1465.347128]  ? __pfx_lockdep_lock+0x10/0x10
> [ 1465.347465]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.347858]  ? add_chain_block+0x1d8/0x280
> [ 1465.348191]  __lock_acquire+0x1793/0x2110
> [ 1465.348524]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.348918]  ? __pfx___lock_acquire+0x10/0x10
> [ 1465.349271]  ? __pfx_prb_read_valid+0x10/0x10
> [ 1465.349624]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.350018]  ? llist_add_batch+0x9d/0xf0
> [ 1465.350344]  ? __pfx_llist_add_batch+0x10/0x10
> [ 1465.350712]  lock_acquire+0x14a/0x3a0
> [ 1465.351014]  ? cifs_disable_secondary_channels+0x117/0x280 [cifs]
> [ 1465.351874]  ? __pfx_lock_acquire+0x10/0x10
> [ 1465.352214]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.352600]  ? hlock_class+0x32/0xc0
> [ 1465.352900]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.353286]  ? lock_acquired+0x2df/0x510
> [ 1465.353612]  ? do_raw_spin_trylock+0xd1/0x120
> [ 1465.353973]  ? __pfx_lock_acquired+0x10/0x10
> [ 1465.354323]  _raw_spin_lock+0x34/0x80
> [ 1465.354622]  ? cifs_disable_secondary_channels+0x117/0x280 [cifs]
> [ 1465.355484]  cifs_disable_secondary_channels+0x117/0x280 [cifs]
> [ 1465.356332]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.356730]  smb2_reconnect+0x520/0xcb0 [cifs]
> [ 1465.357477]  smb2_reconnect_server+0x771/0xb00 [cifs]
> [ 1465.358269]  ? __pfx_smb2_reconnect_server+0x10/0x10 [cifs]
> [ 1465.359089]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.359478]  ? lock_is_held_type+0x90/0x100
> [ 1465.359830]  ? mark_held_locks+0x1a/0x90
> [ 1465.360155]  process_one_work+0x43c/0x8e0
> [ 1465.360493]  ? __pfx_process_one_work+0x10/0x10
> [ 1465.360912]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.361359]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.361754]  ? __list_add_valid_or_report+0x37/0xf0
> [ 1465.362155]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.362544]  worker_thread+0x397/0x690
> [ 1465.362863]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.363250]  ? __kthread_parkme+0xce/0xf0
> [ 1465.363583]  ? __pfx_worker_thread+0x10/0x10
> [ 1465.363942]  kthread+0x18d/0x1d0
> [ 1465.364210]  ? kthread+0xdb/0x1d0
> [ 1465.364483]  ? __pfx_kthread+0x10/0x10
> [ 1465.364798]  ret_from_fork+0x34/0x60
> [ 1465.365093]  ? __pfx_kthread+0x10/0x10
> [ 1465.365400]  ret_from_fork_asm+0x1b/0x30
> [ 1465.365733]  </TASK>
> [ 1465.365954] BUG: sleeping function called from invalid context at kernel/workqueue.c:3344
> [ 1465.366821] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1127, name: kworker/2:1
> [ 1465.367690] preempt_count: 1, expected: 0
> [ 1465.368115] RCU nest depth: 0, expected: 0
> [ 1465.368547] INFO: lockdep is turned off.
> [ 1465.368984] CPU: 2 PID: 1127 Comm: kworker/2:1 Not tainted 6.6.0 #2
> [ 1465.369642] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
> [ 1465.370788] Workqueue: cifsiod smb2_reconnect_server [cifs]
> [ 1465.371888] Call Trace:
> [ 1465.372164]  <TASK>
> [ 1465.372401]  dump_stack_lvl+0x64/0x80
> [ 1465.372809]  __might_resched+0x23c/0x360
> [ 1465.373236]  ? __pfx___might_resched+0x10/0x10
> [ 1465.373722]  ? rcu_is_watching+0x23/0x50
> [ 1465.374149]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.374661]  ? __might_sleep+0x2b/0xb0
> [ 1465.375119]  ? __flush_work+0xc5/0x640
> [ 1465.375524]  __flush_work+0xfd/0x640
> [ 1465.375914]  ? __flush_work+0xc5/0x640
> [ 1465.376320]  ? add_chain_block+0x1d8/0x280
> [ 1465.376771]  ? __pfx___flush_work+0x10/0x10
> [ 1465.377240]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.377686]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.378072]  ? try_to_grab_pending+0x47/0x3a0
> [ 1465.378430]  __cancel_work_timer+0x210/0x2c0
> [ 1465.378785]  ? __pfx___cancel_work_timer+0x10/0x10
> [ 1465.379176]  ? do_raw_spin_trylock+0xd1/0x120
> [ 1465.379537]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.379933]  ? do_raw_spin_unlock+0x9b/0x100
> [ 1465.380293]  cifs_put_tcp_session+0x118/0x290 [cifs]
> [ 1465.381076]  cifs_disable_secondary_channels+0xdb/0x280 [cifs]
> [ 1465.381930]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.382323]  smb2_reconnect+0x520/0xcb0 [cifs]
> [ 1465.383072]  smb2_reconnect_server+0x771/0xb00 [cifs]
> [ 1465.383870]  ? __pfx_smb2_reconnect_server+0x10/0x10 [cifs]
> [ 1465.384694]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.385084]  ? lock_is_held_type+0x90/0x100
> [ 1465.385424]  ? mark_held_locks+0x1a/0x90
> [ 1465.385708] CIFS: fs/smb/client/cifsfs.c: VFS: in cifs_statfs as Xid: 9 with uid: 0
> [ 1465.385754]  process_one_work+0x43c/0x8e0
> [ 1465.387019]  ? __pfx_process_one_work+0x10/0x10
> [ 1465.387387]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.387821]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.388334]  ? __list_add_valid_or_report+0x37/0xf0
> [ 1465.388870]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.389385]  worker_thread+0x397/0x690
> [ 1465.389753]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.390144]  ? __kthread_parkme+0xce/0xf0
> [ 1465.390476]  ? __pfx_worker_thread+0x10/0x10
> [ 1465.390833]  kthread+0x18d/0x1d0
> [ 1465.391103]  ? kthread+0xdb/0x1d0
> [ 1465.391378]  ? __pfx_kthread+0x10/0x10
> [ 1465.391695]  ret_from_fork+0x34/0x60
> [ 1465.391991]  ? __pfx_kthread+0x10/0x10
> [ 1465.392302]  ret_from_fork_asm+0x1b/0x30
> [ 1465.392632]  </TASK>
> [ 1465.393120] CIFS: fs/smb/client/connect.c: cifs_setup_session: channel connect bitmap: 0x1
> [ 1465.393959] CIFS: fs/smb/client/connect.c: Free previous auth_key.response = 0000000018de71c6
> [ 1465.393960] CIFS: fs/smb/client/smb2pdu.c: sess reconnect mask: 0x1, tcon reconnect: 0
> [ 1465.394666] CIFS: fs/smb/client/connect.c: Security Mode: 0x1 Capabilities: 0x300067 TimeAdjust: 0
> [ 1465.396191] CIFS: fs/smb/client/smb2pdu.c: Session Setup
> [ 1465.396675] CIFS: fs/smb/client/smb2pdu.c: sess setup type 2
> [ 1465.397269] CIFS: fs/smb/client/smb2pdu.c: Fresh session. Previous: 1e800d8000049
> [ 1465.399253] CIFS: fs/smb/client/smb2maperror.c: Mapping SMB2 status code 0xc0000016 to POSIX err -5
> [ 1465.400409] CIFS: fs/smb/client/sess.c: decode_ntlmssp_challenge: negotiate=0xe2088235 challenge=0xe2898235
> [ 1465.401245] CIFS: fs/smb/client/smb2pdu.c: rawntlmssp session setup challenge phase
> [ 1465.402058] CIFS: fs/smb/client/smb2pdu.c: Fresh session. Previous: 1e800d8000049
> [ 1465.404585] CIFS: fs/smb/client/smb2pdu.c: SMB2/3 session established successfully
> [ 1465.405543] CIFS: fs/smb/client/sess.c: Cleared reconnect bitmask for chan 0; now 0x0
> [ 1465.406375] CIFS: fs/smb/client/connect.c: __cifs_put_smb_ses: ses_count=2
> [ 1465.407092] CIFS: fs/smb/client/connect.c: __cifs_put_smb_ses: ses ipc: \\w22-root1.gandalf.test\IPC$
> [ 1465.408048] CIFS: fs/smb/client/smb2pdu.c: Reconnecting tcons and channels finished
> [ 1465.408794] CIFS: fs/smb/client/smb2pdu.c: Reconnecting tcons and channels
> [ 1465.409349] CIFS: fs/smb/client/smb2pdu.c: Reconnecting tcons and channels finished
> [ 1465.409949] CIFS: Server share \\w22-root1.gandalf.test\test deleted.
> [ 1465.410961] CIFS: fs/smb/client/smb2maperror.c: Mapping SMB2 status code 0xc00000c9 to POSIX err -78
> [ 1465.412204] CIFS: server share \\w22-root1.gandalf.test\test deleted
> [ 1465.413298] CIFS: fs/smb/client/smb2pdu.c: sess reconnect mask: 0x0, tcon reconnect: 1
> [ 1465.413311] CIFS: fs/smb/client/smb2pdu.c: reconnect tcon rc = 0
> [ 1465.415272] CIFS: fs/smb/client/smb2pdu.c: sess reconnect mask: 0x0, tcon reconnect: 1
> [ 1465.415285] CIFS: fs/smb/client/smb2pdu.c: reconnect tcon rc = 0
> [ 1465.417047] CIFS: fs/smb/client/smb2pdu.c: Reconnecting tcons and channels
> [ 1465.417911] CIFS: fs/smb/client/smb2pdu.c: sess reconnect mask: 0x0, tcon reconnect: 1
> [ 1465.417924] CIFS: fs/smb/client/smb2pdu.c: reconnect tcon rc = 0
> [ 1465.421431] CIFS: fs/smb/client/connect.c: cifs_put_tcon: tc_count=2
> [ 1465.422397] CIFS: fs/smb/client/smb2pdu.c: Reconnecting tcons and channels finished
> [ 1465.423725] ------------[ cut here ]------------
> [ 1465.424158] WARNING: CPU: 3 PID: 85 at fs/smb/client/connect.c:1616 cifs_put_tcp_session+0x27a/0x290 [cifs]
> [ 1465.425407] Modules linked in: cifs cifs_arc4 nls_ucs2_utils fscache cifs_md4 [last unloaded: cifs]
> [ 1465.426242] CPU: 3 PID: 85 Comm: kworker/3:3 Tainted: G        W          6.6.0 #2
> [ 1465.426929] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
> [ 1465.427915] Workqueue: cifsiod smb2_reconnect_server [cifs]
> [ 1465.428828] RIP: 0010:cifs_put_tcp_session+0x27a/0x290 [cifs]
> [ 1465.429754] Code: b1 15 c2 e9 32 fe ff ff 89 ee 48 89 df e8 9e fd ff ff e9 cd fe ff ff be 03 00 00 00 4c 89 ef e8 ac 6d ad c1 e9 11 fe ff ff 90 <0f> 0b 90 e9 c0 fd ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00
> [ 1465.431382] RSP: 0018:ffff88800b8afc10 EFLAGS: 00010286
> [ 1465.431872] RAX: 00000000ffffffff RBX: ffff88800dd7a000 RCX: dffffc0000000000
> [ 1465.432498] RDX: 0000000000000003 RSI: ffffffffc00e901b RDI: ffff88800dd7a070
> [ 1465.433134] RBP: 0000000000000001 R08: 0000000000000000 R09: fffffbfff082290a
> [ 1465.433788] R10: ffffffff84114857 R11: 0000000000000000 R12: 1ffff11001715f90
> [ 1465.434416] R13: ffff88800b8afca0 R14: ffff88800dd7a7a0 R15: ffff88800b8afc88
> [ 1465.435086] FS:  0000000000000000(0000) GS:ffff88805b000000(0000) knlGS:0000000000000000
> [ 1465.435818] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 1465.436330] CR2: 00007f00682affae CR3: 0000000009fe6000 CR4: 0000000000750ef0
> [ 1465.436981] PKRU: 55555554
> [ 1465.437227] Call Trace:
> [ 1465.437458]  <TASK>
> [ 1465.437676]  ? __warn+0xa5/0x200
> [ 1465.437979]  ? cifs_put_tcp_session+0x27a/0x290 [cifs]
> [ 1465.438833]  ? report_bug+0x1b2/0x1e0
> [ 1465.439168]  ? handle_bug+0x6f/0x90
> [ 1465.439484]  ? exc_invalid_op+0x17/0x50
> [ 1465.439846]  ? asm_exc_invalid_op+0x1a/0x20
> [ 1465.440247]  ? cifs_put_tcp_session+0x2b/0x290 [cifs]
> [ 1465.441102]  ? cifs_put_tcp_session+0x27a/0x290 [cifs]
> [ 1465.441970]  smb2_reconnect_server+0x646/0xb00 [cifs]
> [ 1465.442823]  ? lock_sync+0xd0/0xe0
> [ 1465.443138]  ? __pfx_smb2_reconnect_server+0x10/0x10 [cifs]
> [ 1465.444042]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.444467]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.444912]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.445342]  ? read_word_at_a_time+0xe/0x20
> [ 1465.445727]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.446150]  process_one_work+0x43c/0x8e0
> [ 1465.446513]  ? __pfx_process_one_work+0x10/0x10
> [ 1465.446946]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.447376]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.447826]  ? __list_add_valid_or_report+0x37/0xf0
> [ 1465.448278]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.448731]  worker_thread+0x397/0x690
> [ 1465.449080]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1465.449515]  ? __kthread_parkme+0xce/0xf0
> [ 1465.449991]  ? __pfx_worker_thread+0x10/0x10
> [ 1465.450390]  kthread+0x18d/0x1d0
> [ 1465.450712]  ? kthread+0xdb/0x1d0
> [ 1465.451022]  ? __pfx_kthread+0x10/0x10
> [ 1465.451362]  ret_from_fork+0x34/0x60
> [ 1465.451699]  ? __pfx_kthread+0x10/0x10
> [ 1465.452038]  ret_from_fork_asm+0x1b/0x30
> [ 1465.452405]  </TASK>
> [ 1465.452605] irq event stamp: 87714
> [ 1465.452927] hardirqs last  enabled at (87713): [<ffffffff82864168>] _raw_spin_unlock_irq+0x28/0x50
> [ 1465.453732] hardirqs last disabled at (87714): [<ffffffff82856f8d>] __schedule+0xc0d/0x1560
> [ 1465.454468] softirqs last  enabled at (87624): [<ffffffff8110918c>] process_one_work+0x43c/0x8e0
> [ 1465.455257] softirqs last disabled at (87620): [<ffffffff8227f67a>] neigh_managed_work+0x2a/0x110
> [ 1465.456055] ---[ end trace 0000000000000000 ]---
> [ 1465.456937] CIFS: fs/smb/client/cifsfs.c: VFS: leaving cifs_statfs (xid = 9) rc = -11
> [ 1528.811802] CIFS: fs/smb/client/connect.c: VFS: in smb2_query_server_interfaces as Xid: 10 with uid: 0
> [ 1528.813395] CIFS: fs/smb/client/smb2pdu.c: SMB2 IOCTL
> [ 1528.814246] ==================================================================
> [ 1528.815401] BUG: KASAN: slab-use-after-free in cifs_pick_channel+0xa2/0x170 [cifs]
> [ 1528.817404] Read of size 1 at addr ffff88800dd7a2c4 by task kworker/0:2/1114
> [ 1528.818539]
> [ 1528.818810] CPU: 0 PID: 1114 Comm: kworker/0:2 Tainted: G        W          6.6.0 #2
> [ 1528.820040] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
> [ 1528.821789] Workqueue: cifsiod smb2_query_server_interfaces [cifs]
> [ 1528.823573] Call Trace:
> [ 1528.823991]  <TASK>
> [ 1528.824360]  dump_stack_lvl+0x4a/0x80
> [ 1528.824976]  print_report+0xcf/0x650
> [ 1528.825577]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.826361]  ? rcu_is_watching+0x23/0x50
> [ 1528.827011]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.827804]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.828590]  ? __phys_addr+0x46/0x90
> [ 1528.829193]  kasan_report+0xd8/0x110
> [ 1528.829794]  ? cifs_pick_channel+0xa2/0x170 [cifs]
> [ 1528.831379]  ? cifs_pick_channel+0xa2/0x170 [cifs]
> [ 1528.832952]  cifs_pick_channel+0xa2/0x170 [cifs]
> [ 1528.834506]  SMB2_ioctl+0x1b5/0x6f0 [cifs]
> [ 1528.835969]  ? __pfx_console_unlock+0x10/0x10
> [ 1528.836692]  ? tick_nohz_tick_stopped+0x21/0x30
> [ 1528.837441]  ? __pfx_SMB2_ioctl+0x10/0x10 [cifs]
> [ 1528.838992]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.839779]  ? irq_work_queue+0x2c/0x40
> [ 1528.840416]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.841210]  ? lock_acquire+0xc1/0x3a0
> [ 1528.841840]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.842626]  ? _printk+0xc0/0xf0
> [ 1528.843191]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.843977]  ? __dynamic_pr_debug+0x1f5/0x260
> [ 1528.844705]  ? __pfx___dynamic_pr_debug+0x10/0x10
> [ 1528.845476]  ? lock_release+0xb6/0x5a0
> [ 1528.846104]  ? __pfx_lock_acquire+0x10/0x10
> [ 1528.846802]  ? SMB3_request_interfaces+0x137/0x2b0 [cifs]
> [ 1528.848468]  SMB3_request_interfaces+0x137/0x2b0 [cifs]
> [ 1528.850103]  ? __pfx_SMB3_request_interfaces+0x10/0x10 [cifs]
> [ 1528.851828]  ? ___ratelimit+0x133/0x210
> [ 1528.852535]  ? __pfx____ratelimit+0x10/0x10
> [ 1528.853207]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.853839]  smb2_query_server_interfaces+0x54/0x1f0 [cifs]
> [ 1528.855179]  process_one_work+0x43c/0x8e0
> [ 1528.855848]  ? __pfx_process_one_work+0x10/0x10
> [ 1528.856595]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.857380]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.858170]  ? __list_add_valid_or_report+0x37/0xf0
> [ 1528.858985]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.859773]  worker_thread+0x397/0x690
> [ 1528.860410]  ? __pfx_worker_thread+0x10/0x10
> [ 1528.861124]  kthread+0x18d/0x1d0
> [ 1528.861668]  ? kthread+0xdb/0x1d0
> [ 1528.862226]  ? __pfx_kthread+0x10/0x10
> [ 1528.862851]  ret_from_fork+0x34/0x60
> [ 1528.863449]  ? __pfx_kthread+0x10/0x10
> [ 1528.864074]  ret_from_fork_asm+0x1b/0x30
> [ 1528.864745]  </TASK>
> [ 1528.865120]
> [ 1528.865389] Allocated by task 1253:
> [ 1528.865965]  kasan_save_stack+0x33/0x60
> [ 1528.866602]  kasan_set_track+0x25/0x30
> [ 1528.867226]  __kasan_kmalloc+0x8f/0xa0
> [ 1528.867856]  cifs_get_tcp_session+0xbc/0xc70 [cifs]
> [ 1528.869161]  cifs_mount_get_session+0x70/0x220 [cifs]
> [ 1528.870425]  dfs_mount_share+0x249/0x1060 [cifs]
> [ 1528.871650]  cifs_mount+0xda/0x4c0 [cifs]
> [ 1528.872785]  cifs_smb3_do_mount+0x1e5/0xcc0 [cifs]
> [ 1528.874016]  smb3_get_tree+0x16d/0x380 [cifs]
> [ 1528.875208]  vfs_get_tree+0x4d/0x190
> [ 1528.875684]  path_mount+0x3c4/0xf90
> [ 1528.876149]  __x64_sys_mount+0x1aa/0x1f0
> [ 1528.876664]  do_syscall_64+0x47/0xf0
> [ 1528.877140]  entry_SYSCALL_64_after_hwframe+0x6f/0x77
> [ 1528.877800]
> [ 1528.878013] Freed by task 1255:
> [ 1528.878428]  kasan_save_stack+0x33/0x60
> [ 1528.878934]  kasan_set_track+0x25/0x30
> [ 1528.879430]  kasan_save_free_info+0x2b/0x50
> [ 1528.879975]  ____kasan_slab_free+0x126/0x170
> [ 1528.880536]  slab_free_freelist_hook+0x9d/0x1e0
> [ 1528.881131]  __kmem_cache_free+0x9d/0x190
> [ 1528.881506]  clean_demultiplex_info+0x3bb/0x640 [cifs]
> [ 1528.882306]  cifs_demultiplex_thread+0x3de/0x1270 [cifs]
> [ 1528.883086]  kthread+0x18d/0x1d0
> [ 1528.883351]  ret_from_fork+0x34/0x60
> [ 1528.883643]  ret_from_fork_asm+0x1b/0x30
> [ 1528.883960]
> [ 1528.884092] Last potentially related work creation:
> [ 1528.884481]  kasan_save_stack+0x33/0x60
> [ 1528.884792]  __kasan_record_aux_stack+0x94/0xa0
> [ 1528.885155]  __queue_work+0x334/0x8a0
> [ 1528.885454]  mod_delayed_work_on+0xa5/0x100
> [ 1528.885792]  smb2_reconnect+0x735/0xcb0 [cifs]
> [ 1528.886518]  SMB2_query_info_init+0xca/0x250 [cifs]
> [ 1528.887273]  smb2_query_info_compound+0x473/0x6d0 [cifs]
> [ 1528.888065]  smb2_queryfs+0xc2/0x2c0 [cifs]
> [ 1528.888763]  smb311_queryfs+0x210/0x220 [cifs]
> [ 1528.889485]  cifs_statfs+0x164/0x290 [cifs]
> [ 1528.890185]  statfs_by_dentry+0x9b/0xf0
> [ 1528.890497]  user_statfs+0xab/0x130
> [ 1528.890783]  __do_sys_statfs+0x81/0xe0
> [ 1528.891090]  do_syscall_64+0x47/0xf0
> [ 1528.891383]  entry_SYSCALL_64_after_hwframe+0x6f/0x77
> [ 1528.891786]
> [ 1528.891918] Second to last potentially related work creation:
> [ 1528.892368]  kasan_save_stack+0x33/0x60
> [ 1528.892679]  __kasan_record_aux_stack+0x94/0xa0
> [ 1528.893043]  __queue_work+0x334/0x8a0
> [ 1528.893342]  mod_delayed_work_on+0xa5/0x100
> [ 1528.893680]  smb2_reconnect+0x735/0xcb0 [cifs]
> [ 1528.894409]  SMB2_open_init+0xf8/0x13f0 [cifs]
> [ 1528.895135]  smb2_query_info_compound+0x2b6/0x6d0 [cifs]
> [ 1528.895925]  smb2_queryfs+0xc2/0x2c0 [cifs]
> [ 1528.896628]  smb311_queryfs+0x210/0x220 [cifs]
> [ 1528.897353]  cifs_statfs+0x164/0x290 [cifs]
> [ 1528.898060]  statfs_by_dentry+0x9b/0xf0
> [ 1528.898371]  user_statfs+0xab/0x130
> [ 1528.898656]  __do_sys_statfs+0x81/0xe0
> [ 1528.898961]  do_syscall_64+0x47/0xf0
> [ 1528.899255]  entry_SYSCALL_64_after_hwframe+0x6f/0x77
> [ 1528.899658]
> [ 1528.899790] The buggy address belongs to the object at ffff88800dd7a000
> [ 1528.899790]  which belongs to the cache kmalloc-4k of size 4096
> [ 1528.900751] The buggy address is located 708 bytes inside of
> [ 1528.900751]  freed 4096-byte region [ffff88800dd7a000, ffff88800dd7b000)
> [ 1528.901699]
> [ 1528.901830] The buggy address belongs to the physical page:
> [ 1528.902268] page:00000000b974ece7 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0xdd78
> [ 1528.902981] head:00000000b974ece7 order:3 entire_mapcount:0 nr_pages_mapped:0 pincount:0
> [ 1528.903609] anon flags: 0x100000000000840(slab|head|node=0|zone=1)
> [ 1528.904098] page_type: 0xffffffff()
> [ 1528.904386] raw: 0100000000000840 ffff888006443040 0000000000000000 dead000000000001
> [ 1528.904989] raw: 0000000000000000 0000000000040004 00000001ffffffff 0000000000000000
> [ 1528.905591] page dumped because: kasan: bad access detected
> [ 1528.906030]
> [ 1528.906161] Memory state around the buggy address:
> [ 1528.906542]  ffff88800dd7a180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> [ 1528.907107]  ffff88800dd7a200: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> [ 1528.907675] >ffff88800dd7a280: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> [ 1528.908240]                                            ^
> [ 1528.908660]  ffff88800dd7a300: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> [ 1528.909225]  ffff88800dd7a380: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> [ 1528.909787] ==================================================================
> [ 1528.910379] Kernel panic - not syncing: kasan.fault=panic set ...
> [ 1528.910867] CPU: 0 PID: 1114 Comm: kworker/0:2 Tainted: G        W          6.6.0 #2
> [ 1528.911475] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
> [ 1528.912368] Workqueue: cifsiod smb2_query_server_interfaces [cifs]
> [ 1528.913235] Call Trace:
> [ 1528.913439]  <TASK>
> [ 1528.913616]  dump_stack_lvl+0x4a/0x80
> [ 1528.913917]  panic+0x41f/0x460
> [ 1528.914177]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.914563]  ? __pfx_panic+0x10/0x10
> [ 1528.914855]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.915248]  ? check_panic_on_warn+0x2f/0x80
> [ 1528.915594]  end_report+0x125/0x130
> [ 1528.915882]  kasan_report+0xe8/0x110
> [ 1528.916176]  ? cifs_pick_channel+0xa2/0x170 [cifs]
> [ 1528.916935]  ? cifs_pick_channel+0xa2/0x170 [cifs]
> [ 1528.917695]  cifs_pick_channel+0xa2/0x170 [cifs]
> [ 1528.918437]  SMB2_ioctl+0x1b5/0x6f0 [cifs]
> [ 1528.919138]  ? __pfx_console_unlock+0x10/0x10
> [ 1528.919490]  ? tick_nohz_tick_stopped+0x21/0x30
> [ 1528.919857]  ? __pfx_SMB2_ioctl+0x10/0x10 [cifs]
> [ 1528.920596]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.920985]  ? irq_work_queue+0x2c/0x40
> [ 1528.921298]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.921685]  ? lock_acquire+0xc1/0x3a0
> [ 1528.921998]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.922416]  ? _printk+0xc0/0xf0
> [ 1528.922726]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.923157]  ? __dynamic_pr_debug+0x1f5/0x260
> [ 1528.923546]  ? __pfx___dynamic_pr_debug+0x10/0x10
> [ 1528.924097]  ? lock_release+0xb6/0x5a0
> [ 1528.924707]  ? __pfx_lock_acquire+0x10/0x10
> [ 1528.925558]  ? SMB3_request_interfaces+0x137/0x2b0 [cifs]
> [ 1528.926794]  SMB3_request_interfaces+0x137/0x2b0 [cifs]
> [ 1528.927962]  ? __pfx_SMB3_request_interfaces+0x10/0x10 [cifs]
> [ 1528.929230]  ? ___ratelimit+0x133/0x210
> [ 1528.929847]  ? __pfx____ratelimit+0x10/0x10
> [ 1528.930217]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.930730]  smb2_query_server_interfaces+0x54/0x1f0 [cifs]
> [ 1528.931564]  process_one_work+0x43c/0x8e0
> [ 1528.931900]  ? __pfx_process_one_work+0x10/0x10
> [ 1528.932273]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.932663]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.933055]  ? __list_add_valid_or_report+0x37/0xf0
> [ 1528.933459]  ? srso_alias_return_thunk+0x5/0xfbef5
> [ 1528.933852]  worker_thread+0x397/0x690
> [ 1528.934173]  ? __pfx_worker_thread+0x10/0x10
> [ 1528.934526]  kthread+0x18d/0x1d0
> [ 1528.934799]  ? kthread+0xdb/0x1d0
> [ 1528.935077]  ? __pfx_kthread+0x10/0x10
> [ 1528.935388]  ret_from_fork+0x34/0x60
> [ 1528.935686]  ? __pfx_kthread+0x10/0x10
> [ 1528.935997]  ret_from_fork_asm+0x1b/0x30
> [ 1528.936328]  </TASK>
> [ 1528.937856] Kernel Offset: disabled
> [ 1528.938146] ---[ end Kernel panic - not syncing: kasan.fault=panic set ... ]---
>
> $ mount.cifs //w22-root1.gandalf.test/test /mnt/1 -o username=administrator,password=***
> # ...wait until smb2_query_server_interfaces() is executed at least once...
> $ cat /sys/kernel/debug/kmemleak
> unreferenced object 0xffff888007821c00 (size 192):
>   comm "mount.cifs", pid 885, jiffies 4294765292 (age 656.515s)
>   hex dump (first 32 bytes):
>     f0 52 da 16 80 88 ff ff f0 52 da 16 80 88 ff ff  .R.......R......
>     01 00 00 00 00 00 00 00 00 ca 9a 3b 00 00 00 00  ...........;....
>   backtrace:
>     [<ffffffff8144e3f5>] __kmem_cache_alloc_node+0x295/0x2d0
>     [<ffffffff813ab1aa>] kmalloc_trace+0x2a/0xc0
>     [<ffffffffc0141c9d>] parse_server_interfaces+0x4ed/0xcc0 [cifs]
>     [<ffffffffc014b323>] SMB3_request_interfaces+0x163/0x2b0 [cifs]
>     [<ffffffffc014b5ed>] smb3_qfs_tcon+0x16d/0x2c0 [cifs]
>     [<ffffffffc00f02c1>] cifs_mount_get_tcon+0x3b1/0x550 [cifs]
>     [<ffffffffc0181f7a>] dfs_mount_share+0x1da/0x1060 [cifs]
>     [<ffffffffc00f089a>] cifs_mount+0xda/0x4c0 [cifs]
>     [<ffffffffc00e06d5>] cifs_smb3_do_mount+0x1e5/0xcc0 [cifs]
>     [<ffffffffc0175a1d>] smb3_get_tree+0x16d/0x380 [cifs]
>     [<ffffffff8147c1ad>] vfs_get_tree+0x4d/0x190
>     [<ffffffff814c66d4>] path_mount+0x3c4/0xf90
>     [<ffffffff814c7b5a>] __x64_sys_mount+0x1aa/0x1f0
>     [<ffffffff8284c3d7>] do_syscall_64+0x47/0xf0
>     [<ffffffff82a000eb>] entry_SYSCALL_64_after_hwframe+0x6f/0x77
>     ...
>
> $ ./scripts/faddr2line --list fs/smb/client/cifs.o parse_server_interfaces+0x4ed
> parse_server_interfaces+0x4ed/0xcc0:
>
> kmalloc at /home/pc/g/linux/./include/linux/slab.h:600
>  595
>  596                    if (size > KMALLOC_MAX_CACHE_SIZE)
>  597                            return kmalloc_large(size, flags);
>  598
>  599                    index = kmalloc_index(size);
> >600<                   return kmalloc_trace(
>  601                                    kmalloc_caches[kmalloc_type(flags, _RET_IP_)][index],
>  602                                    flags, size);
>  603            }
>  604            return __kmalloc(size, flags);
>  605    }
>
> (inlined by) parse_server_interfaces at /home/pc/g/linux/fs/smb/client/smb2ops.c:694
>  689                            }
>  690                    }
>  691                    spin_unlock(&ses->iface_lock);
>  692
>  693                    /* no match. insert the entry in the list */
> >694<                   info = kmalloc(sizeof(struct cifs_server_iface),
>  695                                   GFP_KERNEL);
>  696                    if (!info) {
>  697                            rc = -ENOMEM;
>  698                            goto out;
>  699                    }



-- 
Thanks,

Steve

Re: [PATCH 12/14] cifs: handle when server stops supporting multichannel

[Paulo Alcantara]

Steve French <smfrench@gmail.com> writes:

> I tried reproducing this to Windows (a little trickier than I expected
> because I had to block the IPv6, not just the IPv4 addresses as in your
> example).  In my example (and built with the stricter kernel config you
> suggested) I do see expected warning messages (although puzzled at first
> why server name is blank in some debug messages - although I did mount with
> IPv4 address not hostname).  After reconnect I can access the mount and it
> seems to work fine.

You could also bring the ifaces down rather than blocking the specific
IP addresses.

I'd expect to see the other warnings in case you have lockdep and
CONFIG_DEBUG_ATOMIC_SLEEP=y enabled.

The blank hostnames are due to secondary channels being reconnected as
we only set it for the primary one.

[PATCH 13/14] cifs: display the endpoint IP details in DebugData

[nspmangalore]

From: Shyam Prasad N <sprasad@microsoft.com>

With multichannel, it is useful to know the src port details
for each channel. This change will print the ip addr and
port details for both the socket dest and src endpoints.

Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
---
 fs/smb/client/cifs_debug.c | 73 ++++++++++++++++++++++++++++++++++++--
 1 file changed, 71 insertions(+), 2 deletions(-)

diff --git a/fs/smb/client/cifs_debug.c b/fs/smb/client/cifs_debug.c
index e23fcabb78d6..d8362e098310 100644
--- a/fs/smb/client/cifs_debug.c
+++ b/fs/smb/client/cifs_debug.c
@@ -13,6 +13,7 @@
 #include <linux/proc_fs.h>
 #include <linux/uaccess.h>
 #include <uapi/linux/ethtool.h>
+#include <net/inet_sock.h>
 #include "cifspdu.h"
 #include "cifsglob.h"
 #include "cifsproto.h"
@@ -158,11 +159,37 @@ cifs_dump_channel(struct seq_file *m, int i, struct cifs_chan *chan)
 		   in_flight(server),
 		   atomic_read(&server->in_send),
 		   atomic_read(&server->num_waiters));
+
 #ifdef CONFIG_NET_NS
 	if (server->net)
 		seq_printf(m, " Net namespace: %u ", server->net->ns.inum);
 #endif /* NET_NS */
 
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	if (!server->rdma)
+		goto skip_rdma;
+
+	if (server->smbd_conn && server->smbd_conn->id) {
+		struct rdma_addr *addr =
+			&server->smbd_conn->id->route.addr;
+		seq_printf(m, "\n\t\tIP addr: dst: %pISpc, src: %pISpc",
+			   &addr->dst_addr, &addr->src_addr);
+	}
+
+skip_rdma:
+#endif
+	if (server->ssocket) {
+		struct sockaddr src;
+		int addrlen;
+
+		addrlen = kernel_getsockname(server->ssocket, &src);
+		if (addrlen != sizeof(struct sockaddr_in) &&
+		    addrlen != sizeof(struct sockaddr_in6))
+			return;
+
+		seq_printf(m, "\n\t\tIP addr: dst: %pISpc, src: %pISpc",
+			   &server->dstaddr, &src);
+	}
 }
 
 static inline const char *smb_speed_to_str(size_t bps)
@@ -279,7 +306,7 @@ static int cifs_debug_files_proc_show(struct seq_file *m, void *v)
 static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
 {
 	struct mid_q_entry *mid_entry;
-	struct TCP_Server_Info *server;
+	struct TCP_Server_Info *server, *nserver;
 	struct TCP_Server_Info *chan_server;
 	struct cifs_ses *ses;
 	struct cifs_tcon *tcon;
@@ -336,7 +363,7 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
 
 	c = 0;
 	spin_lock(&cifs_tcp_ses_lock);
-	list_for_each_entry(server, &cifs_tcp_ses_list, tcp_ses_list) {
+	list_for_each_entry_safe(server, nserver, &cifs_tcp_ses_list, tcp_ses_list) {
 		/* channel info will be printed as a part of sessions below */
 		if (SERVER_IS_CHAN(server))
 			continue;
@@ -414,8 +441,39 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
 		seq_printf(m, "\nMR mr_ready_count: %x mr_used_count: %x",
 			atomic_read(&server->smbd_conn->mr_ready_count),
 			atomic_read(&server->smbd_conn->mr_used_count));
+		if (server->smbd_conn->id) {
+			struct rdma_addr *addr =
+				&server->smbd_conn->id->route.addr;
+			seq_printf(m, "\nIP addr: dst: %pISpc, src: %pISpc",
+				   &addr->dst_addr, &addr->src_addr);
+		}
 skip_rdma:
 #endif
+		if (server->ssocket) {
+			struct sockaddr src;
+			int addrlen;
+
+			/* kernel_getsockname can block. so drop the lock first */
+			server->srv_count++;
+			spin_unlock(&cifs_tcp_ses_lock);
+
+			addrlen = kernel_getsockname(server->ssocket, &src);
+			if (addrlen != sizeof(struct sockaddr_in) &&
+			    addrlen != sizeof(struct sockaddr_in6)) {
+				cifs_put_tcp_session(server, 0);
+				spin_lock(&cifs_tcp_ses_lock);
+
+				goto skip_addr_details;
+			}
+
+			seq_printf(m, "\nIP addr: dst: %pISpc, src: %pISpc",
+				   &server->dstaddr, &src);
+
+			cifs_put_tcp_session(server, 0);
+			spin_lock(&cifs_tcp_ses_lock);
+		}
+
+skip_addr_details:
 		seq_printf(m, "\nNumber of credits: %d,%d,%d Dialect 0x%x",
 			server->credits,
 			server->echo_credits,
@@ -515,7 +573,18 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
 				seq_printf(m, "\n\n\tExtra Channels: %zu ",
 					   ses->chan_count-1);
 				for (j = 1; j < ses->chan_count; j++) {
+					/*
+					 * kernel_getsockname can block inside
+					 * cifs_dump_channel. so drop the lock first
+					 */
+					server->srv_count++;
+					spin_unlock(&cifs_tcp_ses_lock);
+
 					cifs_dump_channel(m, j, &ses->chans[j]);
+
+					cifs_put_tcp_session(server, 0);
+					spin_lock(&cifs_tcp_ses_lock);
+
 					if (CIFS_CHAN_NEEDS_RECONNECT(ses, j))
 						seq_puts(m, "\tDISCONNECTED ");
 					if (CIFS_CHAN_IN_RECONNECT(ses, j))
-- 
2.34.1

Re: [PATCH 13/14] cifs: display the endpoint IP details in DebugData

[Paulo Alcantara]

nspmangalore@gmail.com writes:

> From: Shyam Prasad N <sprasad@microsoft.com>
>
> With multichannel, it is useful to know the src port details
> for each channel. This change will print the ip addr and
> port details for both the socket dest and src endpoints.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/cifs_debug.c | 73 ++++++++++++++++++++++++++++++++++++--
>  1 file changed, 71 insertions(+), 2 deletions(-)
>
> diff --git a/fs/smb/client/cifs_debug.c b/fs/smb/client/cifs_debug.c
> index e23fcabb78d6..d8362e098310 100644
> --- a/fs/smb/client/cifs_debug.c
> +++ b/fs/smb/client/cifs_debug.c
> @@ -13,6 +13,7 @@
>  #include <linux/proc_fs.h>
>  #include <linux/uaccess.h>
>  #include <uapi/linux/ethtool.h>
> +#include <net/inet_sock.h>
>  #include "cifspdu.h"
>  #include "cifsglob.h"
>  #include "cifsproto.h"
> @@ -158,11 +159,37 @@ cifs_dump_channel(struct seq_file *m, int i, struct cifs_chan *chan)
>  		   in_flight(server),
>  		   atomic_read(&server->in_send),
>  		   atomic_read(&server->num_waiters));
> +
>  #ifdef CONFIG_NET_NS
>  	if (server->net)
>  		seq_printf(m, " Net namespace: %u ", server->net->ns.inum);
>  #endif /* NET_NS */
>  
> +#ifdef CONFIG_CIFS_SMB_DIRECT

No need to check for CONFIG_CIFS_SMB_DIRECT as @server->{rdma,smbd_conn}
are always defined.

> +	if (!server->rdma)

cifs_rdma_enabled()?  To be consistent with other places.

> +		goto skip_rdma;
> +
> +	if (server->smbd_conn && server->smbd_conn->id) {
> +		struct rdma_addr *addr =
> +			&server->smbd_conn->id->route.addr;
> +		seq_printf(m, "\n\t\tIP addr: dst: %pISpc, src: %pISpc",
> +			   &addr->dst_addr, &addr->src_addr);
> +	}
> +
> +skip_rdma:
> +#endif

The goto is no longer necessary when removing above #ifdef.

> +	if (server->ssocket) {
> +		struct sockaddr src;
> +		int addrlen;
> +
> +		addrlen = kernel_getsockname(server->ssocket, &src);
> +		if (addrlen != sizeof(struct sockaddr_in) &&
> +		    addrlen != sizeof(struct sockaddr_in6))
> +			return;
> +
> +		seq_printf(m, "\n\t\tIP addr: dst: %pISpc, src: %pISpc",
> +			   &server->dstaddr, &src);
> +	}
>  }
>  
>  static inline const char *smb_speed_to_str(size_t bps)
> @@ -279,7 +306,7 @@ static int cifs_debug_files_proc_show(struct seq_file *m, void *v)
>  static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
>  {
>  	struct mid_q_entry *mid_entry;
> -	struct TCP_Server_Info *server;
> +	struct TCP_Server_Info *server, *nserver;
>  	struct TCP_Server_Info *chan_server;
>  	struct cifs_ses *ses;
>  	struct cifs_tcon *tcon;
> @@ -336,7 +363,7 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
>  
>  	c = 0;
>  	spin_lock(&cifs_tcp_ses_lock);
> -	list_for_each_entry(server, &cifs_tcp_ses_list, tcp_ses_list) {
> +	list_for_each_entry_safe(server, nserver, &cifs_tcp_ses_list, tcp_ses_list) {
>  		/* channel info will be printed as a part of sessions below */
>  		if (SERVER_IS_CHAN(server))
>  			continue;
> @@ -414,8 +441,39 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
>  		seq_printf(m, "\nMR mr_ready_count: %x mr_used_count: %x",
>  			atomic_read(&server->smbd_conn->mr_ready_count),
>  			atomic_read(&server->smbd_conn->mr_used_count));
> +		if (server->smbd_conn->id) {
> +			struct rdma_addr *addr =
> +				&server->smbd_conn->id->route.addr;
> +			seq_printf(m, "\nIP addr: dst: %pISpc, src: %pISpc",
> +				   &addr->dst_addr, &addr->src_addr);
> +		}
>  skip_rdma:
>  #endif
> +		if (server->ssocket) {
> +			struct sockaddr src;
> +			int addrlen;
> +
> +			/* kernel_getsockname can block. so drop the lock first */
> +			server->srv_count++;
> +			spin_unlock(&cifs_tcp_ses_lock);
> +
> +			addrlen = kernel_getsockname(server->ssocket, &src);
> +			if (addrlen != sizeof(struct sockaddr_in) &&
> +			    addrlen != sizeof(struct sockaddr_in6)) {
> +				cifs_put_tcp_session(server, 0);
> +				spin_lock(&cifs_tcp_ses_lock);
> +
> +				goto skip_addr_details;
> +			}

What about

			addrlen = kernel_getsockname(server->ssocket, &src);
			if (addrlen == sizeof(struct sockaddr_in) &&
			    addrlen == sizeof(struct sockaddr_in6)) {
                                seq_printf(m, "\nIP addr: dst: %pISpc, src: %pISpc",
				           &server->dstaddr, &src);
                                cifs_put_tcp_session(server, 0);
                                spin_lock(&cifs_tcp_ses_lock);
                        }

> +
> +			seq_printf(m, "\nIP addr: dst: %pISpc, src: %pISpc",
> +				   &server->dstaddr, &src);
> +
> +			cifs_put_tcp_session(server, 0);
> +			spin_lock(&cifs_tcp_ses_lock);
> +		}
> +
> +skip_addr_details:

Then you can get rid of this goto as well.

>  		seq_printf(m, "\nNumber of credits: %d,%d,%d Dialect 0x%x",
>  			server->credits,
>  			server->echo_credits,
> @@ -515,7 +573,18 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
>  				seq_printf(m, "\n\n\tExtra Channels: %zu ",
>  					   ses->chan_count-1);
>  				for (j = 1; j < ses->chan_count; j++) {
> +					/*
> +					 * kernel_getsockname can block inside
> +					 * cifs_dump_channel. so drop the lock first
> +					 */
> +					server->srv_count++;
> +					spin_unlock(&cifs_tcp_ses_lock);
> +
>  					cifs_dump_channel(m, j, &ses->chans[j]);
> +
> +					cifs_put_tcp_session(server, 0);
> +					spin_lock(&cifs_tcp_ses_lock);

Here you are re-acquiring @cifs_tcp_ses_lock spinlock under
@ses->chan_lock, which will introduce deadlocks in threads calling
cifs_match_super(), cifs_signal_cifsd_for_reconnect(),
cifs_mark_tcp_ses_conns_for_reconnect(), cifs_find_smb_ses(), ...

Re: [PATCH 13/14] cifs: display the endpoint IP details in DebugData

[Paulo Alcantara]

Paulo Alcantara <pc@manguebit.com> writes:

>> @@ -515,7 +573,18 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
>>  				seq_printf(m, "\n\n\tExtra Channels: %zu ",
>>  					   ses->chan_count-1);
>>  				for (j = 1; j < ses->chan_count; j++) {
>> +					/*
>> +					 * kernel_getsockname can block inside
>> +					 * cifs_dump_channel. so drop the lock first
>> +					 */
>> +					server->srv_count++;
>> +					spin_unlock(&cifs_tcp_ses_lock);
>> +
>>  					cifs_dump_channel(m, j, &ses->chans[j]);
>> +
>> +					cifs_put_tcp_session(server, 0);
>> +					spin_lock(&cifs_tcp_ses_lock);
>
> Here you are re-acquiring @cifs_tcp_ses_lock spinlock under
> @ses->chan_lock, which will introduce deadlocks in threads calling
> cifs_match_super(), cifs_signal_cifsd_for_reconnect(),
> cifs_mark_tcp_ses_conns_for_reconnect(), cifs_find_smb_ses(), ...

A simple reproducer

  $ mount.cifs //srv/share /mnt -o ...,multichannel
  $ cat /proc/fs/cifs/DebugData
  
  [ 1293.512572] BUG: sleeping function called from invalid context at net/core/sock.c:3507
  [ 1293.513915] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1068, name: cat
  [ 1293.515381] preempt_count: 1, expected: 0
  [ 1293.516321] RCU nest depth: 0, expected: 0
  [ 1293.517294] 3 locks held by cat/1068:
  [ 1293.518165]  #0: ffff88800818fc48 (&p->lock){+.+.}-{3:3}, at: seq_read_iter+0x59/0x470
  [ 1293.519383]  #1: ffff88800aed2b28 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_debug_data_proc_show+0x555/0xee0 [cifs]
  [ 1293.520865]  #2: ffff888011c9a540 (sk_lock-AF_INET-CIFS){+.+.}-{0:0}, at: inet_getname+0x29/0xa0
  [ 1293.522098] CPU: 3 PID: 1068 Comm: cat Not tainted 6.6.0-rc7 #2
  [ 1293.522901] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
  [ 1293.524368] Call Trace:
  [ 1293.524711]  <TASK>
  [ 1293.525015]  dump_stack_lvl+0x64/0x80
  [ 1293.525519]  __might_resched+0x173/0x280
  [ 1293.526059]  lock_sock_nested+0x43/0x80
  [ 1293.526578]  ? inet_getname+0x29/0xa0
  [ 1293.527097]  inet_getname+0x29/0xa0
  [ 1293.527584]  cifs_debug_data_proc_show+0xcf9/0xee0 [cifs]
  [ 1293.528360]  seq_read_iter+0x118/0x470
  [ 1293.528877]  proc_reg_read_iter+0x53/0x90
  [ 1293.529419]  ? srso_alias_return_thunk+0x5/0x7f
  [ 1293.530037]  vfs_read+0x201/0x350
  [ 1293.530507]  ksys_read+0x75/0x100
  [ 1293.530968]  do_syscall_64+0x3f/0x90
  [ 1293.531461]  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
  [ 1293.532138] RIP: 0033:0x7f71d767e381
  [ 1293.532630] Code: ff ff eb c3 e8 0e ea 01 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 90 90 80 3d a5 f6 0e 00 00 74 13 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 57 c3 66 0f 1f 44 00 00 48 83 ec 28 48 89 54
  [ 1293.535095] RSP: 002b:00007ffc312d65a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
  [ 1293.536106] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007f71d767e381
  [ 1293.537056] RDX: 0000000000020000 RSI: 00007f71d74f8000 RDI: 0000000000000003
  [ 1293.538003] RBP: 0000000000020000 R08: 00000000ffffffff R09: 0000000000000000
  [ 1293.538957] R10: 0000000000000022 R11: 0000000000000246 R12: 00007f71d74f8000
  [ 1293.539908] R13: 0000000000000003 R14: 0000000000000000 R15: 0000000000020000
  [ 1293.540877]  </TASK>
  [ 1293.541233] 
  [ 1293.541449] ======================================================
  [ 1293.542270] WARNING: possible circular locking dependency detected
  [ 1293.543098] 6.6.0-rc7 #2 Tainted: G        W         
  [ 1293.543782] ------------------------------------------------------
  [ 1293.544606] cat/1068 is trying to acquire lock:
  [ 1293.545214] ffffffffc015b5f8 (&cifs_tcp_ses_lock){+.+.}-{2:2}, at: cifs_put_tcp_session+0x1c/0x180 [cifs]
  [ 1293.546516] 
  [ 1293.546516] but task is already holding lock:
  [ 1293.547292] ffff88800aed2b28 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_debug_data_proc_show+0x555/0xee0 [cifs]
  [ 1293.548454] 
  [ 1293.548454] which lock already depends on the new lock.
  [ 1293.548454] 
  [ 1293.549350] 
  [ 1293.549350] the existing dependency chain (in reverse order) is:
  [ 1293.550183] 
  [ 1293.550183] -> #1 (&ret_buf->chan_lock){+.+.}-{2:2}:
  [ 1293.550899]        _raw_spin_lock+0x34/0x80
  [ 1293.551401]        cifs_debug_data_proc_show+0x555/0xee0 [cifs]
  [ 1293.552082]        seq_read_iter+0x118/0x470
  [ 1293.552556]        proc_reg_read_iter+0x53/0x90
  [ 1293.553054]        vfs_read+0x201/0x350
  [ 1293.553490]        ksys_read+0x75/0x100
  [ 1293.553925]        do_syscall_64+0x3f/0x90
  [ 1293.554389]        entry_SYSCALL_64_after_hwframe+0x6e/0xd8
  [ 1293.555004] 
  [ 1293.555004] -> #0 (&cifs_tcp_ses_lock){+.+.}-{2:2}:
  [ 1293.555709]        __lock_acquire+0x1521/0x2660
  [ 1293.556218]        lock_acquire+0xbf/0x2b0
  [ 1293.556680]        _raw_spin_lock+0x34/0x80
  [ 1293.557148]        cifs_put_tcp_session+0x1c/0x180 [cifs]
  [ 1293.557773]        cifs_debug_data_proc_show+0xd15/0xee0 [cifs]
  [ 1293.558463]        seq_read_iter+0x118/0x470
  [ 1293.558945]        proc_reg_read_iter+0x53/0x90
  [ 1293.559450]        vfs_read+0x201/0x350
  [ 1293.559882]        ksys_read+0x75/0x100
  [ 1293.560317]        do_syscall_64+0x3f/0x90
  [ 1293.560773]        entry_SYSCALL_64_after_hwframe+0x6e/0xd8
  [ 1293.561390] 
  [ 1293.561390] other info that might help us debug this:
  [ 1293.561390] 
  [ 1293.562267]  Possible unsafe locking scenario:
  [ 1293.562267] 
  [ 1293.562927]        CPU0                    CPU1
  [ 1293.563394]        ----                    ----
  [ 1293.563754]   lock(&ret_buf->chan_lock);
  [ 1293.564068]                                lock(&cifs_tcp_ses_lock);
  [ 1293.564573]                                lock(&ret_buf->chan_lock);
  [ 1293.565077]   lock(&cifs_tcp_ses_lock);
  [ 1293.565387] 
  [ 1293.565387]  *** DEADLOCK ***
  [ 1293.565387] 
  [ 1293.565852] 2 locks held by cat/1068:
  [ 1293.566147]  #0: ffff88800818fc48 (&p->lock){+.+.}-{3:3}, at: seq_read_iter+0x59/0x470
  [ 1293.566767]  #1: ffff88800aed2b28 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_debug_data_proc_show+0x555/0xee0 [cifs]
  [ 1293.567611] 
  [ 1293.567611] stack backtrace:
  [ 1293.567954] CPU: 3 PID: 1068 Comm: cat Tainted: G        W          6.6.0-rc7 #2
  [ 1293.568536] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
  [ 1293.569387] Call Trace:
  [ 1293.569585]  <TASK>
  [ 1293.569755]  dump_stack_lvl+0x4a/0x80
  [ 1293.570047]  check_noncircular+0x14e/0x170
  [ 1293.570373]  ? save_trace+0x3e/0x390
  [ 1293.570659]  __lock_acquire+0x1521/0x2660
  [ 1293.570982]  lock_acquire+0xbf/0x2b0
  [ 1293.571268]  ? cifs_put_tcp_session+0x1c/0x180 [cifs]
  [ 1293.571687]  _raw_spin_lock+0x34/0x80
  [ 1293.571977]  ? cifs_put_tcp_session+0x1c/0x180 [cifs]
  [ 1293.572394]  cifs_put_tcp_session+0x1c/0x180 [cifs]
  [ 1293.572795]  cifs_debug_data_proc_show+0xd15/0xee0 [cifs]
  [ 1293.573241]  seq_read_iter+0x118/0x470
  [ 1293.573546]  proc_reg_read_iter+0x53/0x90
  [ 1293.573861]  ? srso_alias_return_thunk+0x5/0x7f
  [ 1293.574218]  vfs_read+0x201/0x350
  [ 1293.574489]  ksys_read+0x75/0x100
  [ 1293.574752]  do_syscall_64+0x3f/0x90
  [ 1293.575030]  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
  [ 1293.575428] RIP: 0033:0x7f71d767e381
  [ 1293.575716] Code: ff ff eb c3 e8 0e ea 01 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 90 90 80 3d a5 f6 0e 00 00 74 13 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 57 c3 66 0f 1f 44 00 00 48 83 ec 28 48 89 54
  [ 1293.577151] RSP: 002b:00007ffc312d65a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
  [ 1293.577736] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007f71d767e381
  [ 1293.578286] RDX: 0000000000020000 RSI: 00007f71d74f8000 RDI: 0000000000000003
  [ 1293.578839] RBP: 0000000000020000 R08: 00000000ffffffff R09: 0000000000000000
  [ 1293.579391] R10: 0000000000000022 R11: 0000000000000246 R12: 00007f71d74f8000
  [ 1293.579951] R13: 0000000000000003 R14: 0000000000000000 R15: 0000000000020000
  [ 1293.580511]  </TASK>

Re: [PATCH 13/14] cifs: display the endpoint IP details in DebugData

[Steve French]

removed this patch from for-next until patch updated to address this

On Wed, Nov 1, 2023 at 9:12 AM Paulo Alcantara <pc@manguebit.com> wrote:
>
> Paulo Alcantara <pc@manguebit.com> writes:
>
> >> @@ -515,7 +573,18 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
> >>                              seq_printf(m, "\n\n\tExtra Channels: %zu ",
> >>                                         ses->chan_count-1);
> >>                              for (j = 1; j < ses->chan_count; j++) {
> >> +                                    /*
> >> +                                     * kernel_getsockname can block inside
> >> +                                     * cifs_dump_channel. so drop the lock first
> >> +                                     */
> >> +                                    server->srv_count++;
> >> +                                    spin_unlock(&cifs_tcp_ses_lock);
> >> +
> >>                                      cifs_dump_channel(m, j, &ses->chans[j]);
> >> +
> >> +                                    cifs_put_tcp_session(server, 0);
> >> +                                    spin_lock(&cifs_tcp_ses_lock);
> >
> > Here you are re-acquiring @cifs_tcp_ses_lock spinlock under
> > @ses->chan_lock, which will introduce deadlocks in threads calling
> > cifs_match_super(), cifs_signal_cifsd_for_reconnect(),
> > cifs_mark_tcp_ses_conns_for_reconnect(), cifs_find_smb_ses(), ...
>
> A simple reproducer
>
>   $ mount.cifs //srv/share /mnt -o ...,multichannel
>   $ cat /proc/fs/cifs/DebugData
>
>   [ 1293.512572] BUG: sleeping function called from invalid context at net/core/sock.c:3507
>   [ 1293.513915] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1068, name: cat
>   [ 1293.515381] preempt_count: 1, expected: 0
>   [ 1293.516321] RCU nest depth: 0, expected: 0
>   [ 1293.517294] 3 locks held by cat/1068:
>   [ 1293.518165]  #0: ffff88800818fc48 (&p->lock){+.+.}-{3:3}, at: seq_read_iter+0x59/0x470
>   [ 1293.519383]  #1: ffff88800aed2b28 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_debug_data_proc_show+0x555/0xee0 [cifs]
>   [ 1293.520865]  #2: ffff888011c9a540 (sk_lock-AF_INET-CIFS){+.+.}-{0:0}, at: inet_getname+0x29/0xa0
>   [ 1293.522098] CPU: 3 PID: 1068 Comm: cat Not tainted 6.6.0-rc7 #2
>   [ 1293.522901] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
>   [ 1293.524368] Call Trace:
>   [ 1293.524711]  <TASK>
>   [ 1293.525015]  dump_stack_lvl+0x64/0x80
>   [ 1293.525519]  __might_resched+0x173/0x280
>   [ 1293.526059]  lock_sock_nested+0x43/0x80
>   [ 1293.526578]  ? inet_getname+0x29/0xa0
>   [ 1293.527097]  inet_getname+0x29/0xa0
>   [ 1293.527584]  cifs_debug_data_proc_show+0xcf9/0xee0 [cifs]
>   [ 1293.528360]  seq_read_iter+0x118/0x470
>   [ 1293.528877]  proc_reg_read_iter+0x53/0x90
>   [ 1293.529419]  ? srso_alias_return_thunk+0x5/0x7f
>   [ 1293.530037]  vfs_read+0x201/0x350
>   [ 1293.530507]  ksys_read+0x75/0x100
>   [ 1293.530968]  do_syscall_64+0x3f/0x90
>   [ 1293.531461]  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
>   [ 1293.532138] RIP: 0033:0x7f71d767e381
>   [ 1293.532630] Code: ff ff eb c3 e8 0e ea 01 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 90 90 80 3d a5 f6 0e 00 00 74 13 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 57 c3 66 0f 1f 44 00 00 48 83 ec 28 48 89 54
>   [ 1293.535095] RSP: 002b:00007ffc312d65a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
>   [ 1293.536106] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007f71d767e381
>   [ 1293.537056] RDX: 0000000000020000 RSI: 00007f71d74f8000 RDI: 0000000000000003
>   [ 1293.538003] RBP: 0000000000020000 R08: 00000000ffffffff R09: 0000000000000000
>   [ 1293.538957] R10: 0000000000000022 R11: 0000000000000246 R12: 00007f71d74f8000
>   [ 1293.539908] R13: 0000000000000003 R14: 0000000000000000 R15: 0000000000020000
>   [ 1293.540877]  </TASK>
>   [ 1293.541233]
>   [ 1293.541449] ======================================================
>   [ 1293.542270] WARNING: possible circular locking dependency detected
>   [ 1293.543098] 6.6.0-rc7 #2 Tainted: G        W
>   [ 1293.543782] ------------------------------------------------------
>   [ 1293.544606] cat/1068 is trying to acquire lock:
>   [ 1293.545214] ffffffffc015b5f8 (&cifs_tcp_ses_lock){+.+.}-{2:2}, at: cifs_put_tcp_session+0x1c/0x180 [cifs]
>   [ 1293.546516]
>   [ 1293.546516] but task is already holding lock:
>   [ 1293.547292] ffff88800aed2b28 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_debug_data_proc_show+0x555/0xee0 [cifs]
>   [ 1293.548454]
>   [ 1293.548454] which lock already depends on the new lock.
>   [ 1293.548454]
>   [ 1293.549350]
>   [ 1293.549350] the existing dependency chain (in reverse order) is:
>   [ 1293.550183]
>   [ 1293.550183] -> #1 (&ret_buf->chan_lock){+.+.}-{2:2}:
>   [ 1293.550899]        _raw_spin_lock+0x34/0x80
>   [ 1293.551401]        cifs_debug_data_proc_show+0x555/0xee0 [cifs]
>   [ 1293.552082]        seq_read_iter+0x118/0x470
>   [ 1293.552556]        proc_reg_read_iter+0x53/0x90
>   [ 1293.553054]        vfs_read+0x201/0x350
>   [ 1293.553490]        ksys_read+0x75/0x100
>   [ 1293.553925]        do_syscall_64+0x3f/0x90
>   [ 1293.554389]        entry_SYSCALL_64_after_hwframe+0x6e/0xd8
>   [ 1293.555004]
>   [ 1293.555004] -> #0 (&cifs_tcp_ses_lock){+.+.}-{2:2}:
>   [ 1293.555709]        __lock_acquire+0x1521/0x2660
>   [ 1293.556218]        lock_acquire+0xbf/0x2b0
>   [ 1293.556680]        _raw_spin_lock+0x34/0x80
>   [ 1293.557148]        cifs_put_tcp_session+0x1c/0x180 [cifs]
>   [ 1293.557773]        cifs_debug_data_proc_show+0xd15/0xee0 [cifs]
>   [ 1293.558463]        seq_read_iter+0x118/0x470
>   [ 1293.558945]        proc_reg_read_iter+0x53/0x90
>   [ 1293.559450]        vfs_read+0x201/0x350
>   [ 1293.559882]        ksys_read+0x75/0x100
>   [ 1293.560317]        do_syscall_64+0x3f/0x90
>   [ 1293.560773]        entry_SYSCALL_64_after_hwframe+0x6e/0xd8
>   [ 1293.561390]
>   [ 1293.561390] other info that might help us debug this:
>   [ 1293.561390]
>   [ 1293.562267]  Possible unsafe locking scenario:
>   [ 1293.562267]
>   [ 1293.562927]        CPU0                    CPU1
>   [ 1293.563394]        ----                    ----
>   [ 1293.563754]   lock(&ret_buf->chan_lock);
>   [ 1293.564068]                                lock(&cifs_tcp_ses_lock);
>   [ 1293.564573]                                lock(&ret_buf->chan_lock);
>   [ 1293.565077]   lock(&cifs_tcp_ses_lock);
>   [ 1293.565387]
>   [ 1293.565387]  *** DEADLOCK ***
>   [ 1293.565387]
>   [ 1293.565852] 2 locks held by cat/1068:
>   [ 1293.566147]  #0: ffff88800818fc48 (&p->lock){+.+.}-{3:3}, at: seq_read_iter+0x59/0x470
>   [ 1293.566767]  #1: ffff88800aed2b28 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_debug_data_proc_show+0x555/0xee0 [cifs]
>   [ 1293.567611]
>   [ 1293.567611] stack backtrace:
>   [ 1293.567954] CPU: 3 PID: 1068 Comm: cat Tainted: G        W          6.6.0-rc7 #2
>   [ 1293.568536] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
>   [ 1293.569387] Call Trace:
>   [ 1293.569585]  <TASK>
>   [ 1293.569755]  dump_stack_lvl+0x4a/0x80
>   [ 1293.570047]  check_noncircular+0x14e/0x170
>   [ 1293.570373]  ? save_trace+0x3e/0x390
>   [ 1293.570659]  __lock_acquire+0x1521/0x2660
>   [ 1293.570982]  lock_acquire+0xbf/0x2b0
>   [ 1293.571268]  ? cifs_put_tcp_session+0x1c/0x180 [cifs]
>   [ 1293.571687]  _raw_spin_lock+0x34/0x80
>   [ 1293.571977]  ? cifs_put_tcp_session+0x1c/0x180 [cifs]
>   [ 1293.572394]  cifs_put_tcp_session+0x1c/0x180 [cifs]
>   [ 1293.572795]  cifs_debug_data_proc_show+0xd15/0xee0 [cifs]
>   [ 1293.573241]  seq_read_iter+0x118/0x470
>   [ 1293.573546]  proc_reg_read_iter+0x53/0x90
>   [ 1293.573861]  ? srso_alias_return_thunk+0x5/0x7f
>   [ 1293.574218]  vfs_read+0x201/0x350
>   [ 1293.574489]  ksys_read+0x75/0x100
>   [ 1293.574752]  do_syscall_64+0x3f/0x90
>   [ 1293.575030]  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
>   [ 1293.575428] RIP: 0033:0x7f71d767e381
>   [ 1293.575716] Code: ff ff eb c3 e8 0e ea 01 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 90 90 80 3d a5 f6 0e 00 00 74 13 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 57 c3 66 0f 1f 44 00 00 48 83 ec 28 48 89 54
>   [ 1293.577151] RSP: 002b:00007ffc312d65a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
>   [ 1293.577736] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007f71d767e381
>   [ 1293.578286] RDX: 0000000000020000 RSI: 00007f71d74f8000 RDI: 0000000000000003
>   [ 1293.578839] RBP: 0000000000020000 R08: 00000000ffffffff R09: 0000000000000000
>   [ 1293.579391] R10: 0000000000000022 R11: 0000000000000246 R12: 00007f71d74f8000
>   [ 1293.579951] R13: 0000000000000003 R14: 0000000000000000 R15: 0000000000020000
>   [ 1293.580511]  </TASK>



-- 
Thanks,

Steve

Re: [PATCH 13/14] cifs: display the endpoint IP details in DebugData

[Shyam Prasad N]

On Wed, Nov 1, 2023 at 7:42 PM Paulo Alcantara <pc@manguebit.com> wrote:
>
> Paulo Alcantara <pc@manguebit.com> writes:
>
> >> @@ -515,7 +573,18 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
> >>                              seq_printf(m, "\n\n\tExtra Channels: %zu ",
> >>                                         ses->chan_count-1);
> >>                              for (j = 1; j < ses->chan_count; j++) {
> >> +                                    /*
> >> +                                     * kernel_getsockname can block inside
> >> +                                     * cifs_dump_channel. so drop the lock first
> >> +                                     */
> >> +                                    server->srv_count++;
> >> +                                    spin_unlock(&cifs_tcp_ses_lock);
> >> +
> >>                                      cifs_dump_channel(m, j, &ses->chans[j]);
> >> +
> >> +                                    cifs_put_tcp_session(server, 0);
> >> +                                    spin_lock(&cifs_tcp_ses_lock);
> >
> > Here you are re-acquiring @cifs_tcp_ses_lock spinlock under
> > @ses->chan_lock, which will introduce deadlocks in threads calling
> > cifs_match_super(), cifs_signal_cifsd_for_reconnect(),
> > cifs_mark_tcp_ses_conns_for_reconnect(), cifs_find_smb_ses(), ...
>

Good points.
I'm just wondering why I'm unable to repro the same though.
I have lockdep enabled on my kernel too. But the same steps do not
throw this warning.

> A simple reproducer
>
>   $ mount.cifs //srv/share /mnt -o ...,multichannel
>   $ cat /proc/fs/cifs/DebugData
>
>   [ 1293.512572] BUG: sleeping function called from invalid context at net/core/sock.c:3507
>   [ 1293.513915] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1068, name: cat
>   [ 1293.515381] preempt_count: 1, expected: 0
>   [ 1293.516321] RCU nest depth: 0, expected: 0
>   [ 1293.517294] 3 locks held by cat/1068:
>   [ 1293.518165]  #0: ffff88800818fc48 (&p->lock){+.+.}-{3:3}, at: seq_read_iter+0x59/0x470
>   [ 1293.519383]  #1: ffff88800aed2b28 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_debug_data_proc_show+0x555/0xee0 [cifs]
>   [ 1293.520865]  #2: ffff888011c9a540 (sk_lock-AF_INET-CIFS){+.+.}-{0:0}, at: inet_getname+0x29/0xa0
>   [ 1293.522098] CPU: 3 PID: 1068 Comm: cat Not tainted 6.6.0-rc7 #2
>   [ 1293.522901] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
>   [ 1293.524368] Call Trace:
>   [ 1293.524711]  <TASK>
>   [ 1293.525015]  dump_stack_lvl+0x64/0x80
>   [ 1293.525519]  __might_resched+0x173/0x280
>   [ 1293.526059]  lock_sock_nested+0x43/0x80
>   [ 1293.526578]  ? inet_getname+0x29/0xa0
>   [ 1293.527097]  inet_getname+0x29/0xa0
>   [ 1293.527584]  cifs_debug_data_proc_show+0xcf9/0xee0 [cifs]
>   [ 1293.528360]  seq_read_iter+0x118/0x470
>   [ 1293.528877]  proc_reg_read_iter+0x53/0x90
>   [ 1293.529419]  ? srso_alias_return_thunk+0x5/0x7f
>   [ 1293.530037]  vfs_read+0x201/0x350
>   [ 1293.530507]  ksys_read+0x75/0x100
>   [ 1293.530968]  do_syscall_64+0x3f/0x90
>   [ 1293.531461]  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
>   [ 1293.532138] RIP: 0033:0x7f71d767e381
>   [ 1293.532630] Code: ff ff eb c3 e8 0e ea 01 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 90 90 80 3d a5 f6 0e 00 00 74 13 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 57 c3 66 0f 1f 44 00 00 48 83 ec 28 48 89 54
>   [ 1293.535095] RSP: 002b:00007ffc312d65a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
>   [ 1293.536106] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007f71d767e381
>   [ 1293.537056] RDX: 0000000000020000 RSI: 00007f71d74f8000 RDI: 0000000000000003
>   [ 1293.538003] RBP: 0000000000020000 R08: 00000000ffffffff R09: 0000000000000000
>   [ 1293.538957] R10: 0000000000000022 R11: 0000000000000246 R12: 00007f71d74f8000
>   [ 1293.539908] R13: 0000000000000003 R14: 0000000000000000 R15: 0000000000020000
>   [ 1293.540877]  </TASK>
>   [ 1293.541233]
>   [ 1293.541449] ======================================================
>   [ 1293.542270] WARNING: possible circular locking dependency detected
>   [ 1293.543098] 6.6.0-rc7 #2 Tainted: G        W
>   [ 1293.543782] ------------------------------------------------------
>   [ 1293.544606] cat/1068 is trying to acquire lock:
>   [ 1293.545214] ffffffffc015b5f8 (&cifs_tcp_ses_lock){+.+.}-{2:2}, at: cifs_put_tcp_session+0x1c/0x180 [cifs]
>   [ 1293.546516]
>   [ 1293.546516] but task is already holding lock:
>   [ 1293.547292] ffff88800aed2b28 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_debug_data_proc_show+0x555/0xee0 [cifs]
>   [ 1293.548454]
>   [ 1293.548454] which lock already depends on the new lock.
>   [ 1293.548454]
>   [ 1293.549350]
>   [ 1293.549350] the existing dependency chain (in reverse order) is:
>   [ 1293.550183]
>   [ 1293.550183] -> #1 (&ret_buf->chan_lock){+.+.}-{2:2}:
>   [ 1293.550899]        _raw_spin_lock+0x34/0x80
>   [ 1293.551401]        cifs_debug_data_proc_show+0x555/0xee0 [cifs]
>   [ 1293.552082]        seq_read_iter+0x118/0x470
>   [ 1293.552556]        proc_reg_read_iter+0x53/0x90
>   [ 1293.553054]        vfs_read+0x201/0x350
>   [ 1293.553490]        ksys_read+0x75/0x100
>   [ 1293.553925]        do_syscall_64+0x3f/0x90
>   [ 1293.554389]        entry_SYSCALL_64_after_hwframe+0x6e/0xd8
>   [ 1293.555004]
>   [ 1293.555004] -> #0 (&cifs_tcp_ses_lock){+.+.}-{2:2}:
>   [ 1293.555709]        __lock_acquire+0x1521/0x2660
>   [ 1293.556218]        lock_acquire+0xbf/0x2b0
>   [ 1293.556680]        _raw_spin_lock+0x34/0x80
>   [ 1293.557148]        cifs_put_tcp_session+0x1c/0x180 [cifs]
>   [ 1293.557773]        cifs_debug_data_proc_show+0xd15/0xee0 [cifs]
>   [ 1293.558463]        seq_read_iter+0x118/0x470
>   [ 1293.558945]        proc_reg_read_iter+0x53/0x90
>   [ 1293.559450]        vfs_read+0x201/0x350
>   [ 1293.559882]        ksys_read+0x75/0x100
>   [ 1293.560317]        do_syscall_64+0x3f/0x90
>   [ 1293.560773]        entry_SYSCALL_64_after_hwframe+0x6e/0xd8
>   [ 1293.561390]
>   [ 1293.561390] other info that might help us debug this:
>   [ 1293.561390]
>   [ 1293.562267]  Possible unsafe locking scenario:
>   [ 1293.562267]
>   [ 1293.562927]        CPU0                    CPU1
>   [ 1293.563394]        ----                    ----
>   [ 1293.563754]   lock(&ret_buf->chan_lock);
>   [ 1293.564068]                                lock(&cifs_tcp_ses_lock);
>   [ 1293.564573]                                lock(&ret_buf->chan_lock);
>   [ 1293.565077]   lock(&cifs_tcp_ses_lock);
>   [ 1293.565387]
>   [ 1293.565387]  *** DEADLOCK ***
>   [ 1293.565387]
>   [ 1293.565852] 2 locks held by cat/1068:
>   [ 1293.566147]  #0: ffff88800818fc48 (&p->lock){+.+.}-{3:3}, at: seq_read_iter+0x59/0x470
>   [ 1293.566767]  #1: ffff88800aed2b28 (&ret_buf->chan_lock){+.+.}-{2:2}, at: cifs_debug_data_proc_show+0x555/0xee0 [cifs]
>   [ 1293.567611]
>   [ 1293.567611] stack backtrace:
>   [ 1293.567954] CPU: 3 PID: 1068 Comm: cat Tainted: G        W          6.6.0-rc7 #2
>   [ 1293.568536] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.2-3-gd478f380-rebuilt.opensuse.org 04/01/2014
>   [ 1293.569387] Call Trace:
>   [ 1293.569585]  <TASK>
>   [ 1293.569755]  dump_stack_lvl+0x4a/0x80
>   [ 1293.570047]  check_noncircular+0x14e/0x170
>   [ 1293.570373]  ? save_trace+0x3e/0x390
>   [ 1293.570659]  __lock_acquire+0x1521/0x2660
>   [ 1293.570982]  lock_acquire+0xbf/0x2b0
>   [ 1293.571268]  ? cifs_put_tcp_session+0x1c/0x180 [cifs]
>   [ 1293.571687]  _raw_spin_lock+0x34/0x80
>   [ 1293.571977]  ? cifs_put_tcp_session+0x1c/0x180 [cifs]
>   [ 1293.572394]  cifs_put_tcp_session+0x1c/0x180 [cifs]
>   [ 1293.572795]  cifs_debug_data_proc_show+0xd15/0xee0 [cifs]
>   [ 1293.573241]  seq_read_iter+0x118/0x470
>   [ 1293.573546]  proc_reg_read_iter+0x53/0x90
>   [ 1293.573861]  ? srso_alias_return_thunk+0x5/0x7f
>   [ 1293.574218]  vfs_read+0x201/0x350
>   [ 1293.574489]  ksys_read+0x75/0x100
>   [ 1293.574752]  do_syscall_64+0x3f/0x90
>   [ 1293.575030]  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
>   [ 1293.575428] RIP: 0033:0x7f71d767e381
>   [ 1293.575716] Code: ff ff eb c3 e8 0e ea 01 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 90 90 80 3d a5 f6 0e 00 00 74 13 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 57 c3 66 0f 1f 44 00 00 48 83 ec 28 48 89 54
>   [ 1293.577151] RSP: 002b:00007ffc312d65a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
>   [ 1293.577736] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007f71d767e381
>   [ 1293.578286] RDX: 0000000000020000 RSI: 00007f71d74f8000 RDI: 0000000000000003
>   [ 1293.578839] RBP: 0000000000020000 R08: 00000000ffffffff R09: 0000000000000000
>   [ 1293.579391] R10: 0000000000000022 R11: 0000000000000246 R12: 00007f71d74f8000
>   [ 1293.579951] R13: 0000000000000003 R14: 0000000000000000 R15: 0000000000020000
>   [ 1293.580511]  </TASK>



-- 
Regards,
Shyam

Re: [PATCH 13/14] cifs: display the endpoint IP details in DebugData

[Paulo Alcantara]

Hi Shyam,

Shyam Prasad N <nspmangalore@gmail.com> writes:

> On Wed, Nov 1, 2023 at 7:42 PM Paulo Alcantara <pc@manguebit.com> wrote:
>>
>> Paulo Alcantara <pc@manguebit.com> writes:
>>
>> >> @@ -515,7 +573,18 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
>> >>                              seq_printf(m, "\n\n\tExtra Channels: %zu ",
>> >>                                         ses->chan_count-1);
>> >>                              for (j = 1; j < ses->chan_count; j++) {
>> >> +                                    /*
>> >> +                                     * kernel_getsockname can block inside
>> >> +                                     * cifs_dump_channel. so drop the lock first
>> >> +                                     */
>> >> +                                    server->srv_count++;
>> >> +                                    spin_unlock(&cifs_tcp_ses_lock);
>> >> +
>> >>                                      cifs_dump_channel(m, j, &ses->chans[j]);
>> >> +
>> >> +                                    cifs_put_tcp_session(server, 0);
>> >> +                                    spin_lock(&cifs_tcp_ses_lock);
>> >
>> > Here you are re-acquiring @cifs_tcp_ses_lock spinlock under
>> > @ses->chan_lock, which will introduce deadlocks in threads calling
>> > cifs_match_super(), cifs_signal_cifsd_for_reconnect(),
>> > cifs_mark_tcp_ses_conns_for_reconnect(), cifs_find_smb_ses(), ...
>>
>
> Good points.
> I'm just wondering why I'm unable to repro the same though.
> I have lockdep enabled on my kernel too. But the same steps do not
> throw this warning.

That's weird as I can always reproduce it when mounting a Windows Server
2022 share with 'username=foo,password=bar,multichannel' options
followed by `cat /proc/fs/cifs/DebugData`.

Here is the relevant part of my .config

#
# Lock Debugging (spinlocks, mutexes, etc...)
#
CONFIG_LOCK_DEBUGGING_SUPPORT=y
CONFIG_PROVE_LOCKING=y
# CONFIG_PROVE_RAW_LOCK_NESTING is not set
CONFIG_LOCK_STAT=y
CONFIG_DEBUG_RT_MUTEXES=y
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
CONFIG_DEBUG_WW_MUTEX_SLOWPATH=y
CONFIG_DEBUG_RWSEMS=y
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_LOCKDEP=y
CONFIG_LOCKDEP_BITS=15
CONFIG_LOCKDEP_CHAINS_BITS=16
CONFIG_LOCKDEP_STACK_TRACE_BITS=19
CONFIG_LOCKDEP_STACK_TRACE_HASH_BITS=14
CONFIG_LOCKDEP_CIRCULAR_QUEUE_BITS=12
# CONFIG_DEBUG_LOCKDEP is not set
CONFIG_DEBUG_ATOMIC_SLEEP=y
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
# CONFIG_LOCK_TORTURE_TEST is not set
# CONFIG_WW_MUTEX_SELFTEST is not set
# CONFIG_SCF_TORTURE_TEST is not set
# CONFIG_CSD_LOCK_WAIT_DEBUG is not set
# end of Lock Debugging (spinlocks, mutexes, etc...)

Re: [PATCH 01/14] cifs: print server capabilities in DebugData

[Bharath SM]

On Mon, Oct 30, 2023 at 4:30 PM <nspmangalore@gmail.com> wrote:
>
> From: Shyam Prasad N <sprasad@microsoft.com>
>
> In the output of /proc/fs/cifs/DebugData, we do not
> print the server->capabilities field today.
> With this change, we will do that.

We already print session capabilities in DebugData. How is it
different from that.?

Re: [PATCH 01/14] cifs: print server capabilities in DebugData

[Shyam Prasad N]

On Mon, Oct 30, 2023 at 6:04 PM Bharath SM <bharathsm.hsk@gmail.com> wrote:
>
> On Mon, Oct 30, 2023 at 4:30 PM <nspmangalore@gmail.com> wrote:
> >
> > From: Shyam Prasad N <sprasad@microsoft.com>
> >
> > In the output of /proc/fs/cifs/DebugData, we do not
> > print the server->capabilities field today.
> > With this change, we will do that.
>
> We already print session capabilities in DebugData. How is it
> different from that.?

These are capabilities at the server level. That one is at session level.
Actually, there's tcon capabilities as well, which I think we should
also dump in DebugData.
I'll submit a revised version for that as well.

-- 
Regards,
Shyam

Re: [PATCH 01/14] cifs: print server capabilities in DebugData

[Shyam Prasad N]

On Mon, Oct 30, 2023 at 6:10 PM Shyam Prasad N <nspmangalore@gmail.com> wrote:
>
> On Mon, Oct 30, 2023 at 6:04 PM Bharath SM <bharathsm.hsk@gmail.com> wrote:
> >
> > On Mon, Oct 30, 2023 at 4:30 PM <nspmangalore@gmail.com> wrote:
> > >
> > > From: Shyam Prasad N <sprasad@microsoft.com>
> > >
> > > In the output of /proc/fs/cifs/DebugData, we do not
> > > print the server->capabilities field today.
> > > With this change, we will do that.
> >
> > We already print session capabilities in DebugData. How is it
> > different from that.?
>
> These are capabilities at the server level. That one is at session level.
> Actually, there's tcon capabilities as well, which I think we should
> also dump in DebugData.
> I'll submit a revised version for that as well.

Actually, it looks like that's already done in smb2_dump_share_caps.
It would be even better to print server capabilities in the symbolic
form similar to how it's done there.
But this works too. I think we can do a follow up patch for that if needed.

>
> --
> Regards,
> Shyam



-- 
Regards,
Shyam

Re: [PATCH 01/14] cifs: print server capabilities in DebugData

[Steve French]

Made a minor change "Server capability: "  --> "Server capabilities:"
to make it consistent with the places we print session and tcon
capabilities in the DebugData

Let me know if any objections

On Mon, Oct 30, 2023 at 6:00 AM <nspmangalore@gmail.com> wrote:
>
> From: Shyam Prasad N <sprasad@microsoft.com>
>
> In the output of /proc/fs/cifs/DebugData, we do not
> print the server->capabilities field today.
> With this change, we will do that.
>
> Signed-off-by: Shyam Prasad N <sprasad@microsoft.com>
> ---
>  fs/smb/client/cifs_debug.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/fs/smb/client/cifs_debug.c b/fs/smb/client/cifs_debug.c
> index 76922fcc4bc6..a9dfecc397a8 100644
> --- a/fs/smb/client/cifs_debug.c
> +++ b/fs/smb/client/cifs_debug.c
> @@ -427,6 +427,8 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
>                 if (server->nosharesock)
>                         seq_printf(m, " nosharesock");
>
> +               seq_printf(m, "\nServer capability: 0x%x", server->capabilities);
> +
>                 if (server->rdma)
>                         seq_printf(m, "\nRDMA ");
>                 seq_printf(m, "\nTCP status: %d Instance: %d"
> --
> 2.34.1
>


-- 
Thanks,

Steve