From: "dust.li" <dust.li@linux.alibaba.com>
To: Wen Gu <guwen@linux.alibaba.com>,
kgraul@linux.ibm.com, wenjia@linux.ibm.com, davem@davemloft.net,
edumazet@google.com, kuba@kernel.org, pabeni@redhat.com
Cc: linux-s390@vger.kernel.org, netdev@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH net-next 1/2] net/smc: Introduce a specific sysctl for TEST_LINK time
Date: Tue, 20 Sep 2022 12:55:20 +0800 [thread overview]
Message-ID: <20220920045520.GC108825@linux.alibaba.com> (raw)
In-Reply-To: <1663642434-30035-1-git-send-email-guwen@linux.alibaba.com>
On Tue, Sep 20, 2022 at 10:53:54AM +0800, Wen Gu wrote:
>SMC-R tests the viability of link by sending out TEST_LINK LLC
>messages over RoCE fabric when connections on link have been
>idle for a time longer than keepalive interval (testlink time).
>
>But using tcp_keepalive_time as testlink time maybe not quite
>suitable because it is default no less than two hours[1], which
>is too long for single link to find peer dead. The active host
>will still use peer-dead link (QP) sending messages, and can't
>find out until get IB_WC_RETRY_EXC_ERR error CQEs, which takes
>more time than TEST_LINK timeout (SMC_LLC_WAIT_TIME) normally.
>
>So this patch introduces a independent sysctl for SMC-R to set
>link keepalive time, in order to detect link down in time. The
>default value is 30 seconds.
>
>[1] https://www.rfc-editor.org/rfc/rfc1122#page-101
>
>Signed-off-by: Wen Gu <guwen@linux.alibaba.com>
>---
> Documentation/networking/smc-sysctl.rst | 7 +++++++
> include/net/netns/smc.h | 1 +
> net/smc/smc_llc.c | 2 +-
> net/smc/smc_llc.h | 1 +
> net/smc/smc_sysctl.c | 14 ++++++++++++++
> 5 files changed, 24 insertions(+), 1 deletion(-)
>
>diff --git a/Documentation/networking/smc-sysctl.rst b/Documentation/networking/smc-sysctl.rst
>index 742e90e..f8c3d59 100644
>--- a/Documentation/networking/smc-sysctl.rst
>+++ b/Documentation/networking/smc-sysctl.rst
>@@ -34,3 +34,10 @@ smcr_buf_type - INTEGER
> - 1 - Use virtually contiguous buffers
> - 2 - Mixed use of the two types. Try physically contiguous buffers first.
> If not available, use virtually contiguous buffers then.
>+
>+smcr_testlink_time - INTEGER
>+ How frequently SMC-R link sends out TEST_LINK LLC messages to confirm
>+ viability, after the last activity of connections on it. The maximum
>+ value is (INT_MAX / HZ) seconds, the minimum value is 1 second.
>+
>+ Default: 30 seconds.
>diff --git a/include/net/netns/smc.h b/include/net/netns/smc.h
>index 2adbe2b..d295e2c 100644
>--- a/include/net/netns/smc.h
>+++ b/include/net/netns/smc.h
>@@ -19,5 +19,6 @@ struct netns_smc {
> #endif
> unsigned int sysctl_autocorking_size;
> unsigned int sysctl_smcr_buf_type;
>+ int sysctl_smcr_testlink_time;
> };
> #endif
>diff --git a/net/smc/smc_llc.c b/net/smc/smc_llc.c
>index 175026a..388bd2e 100644
>--- a/net/smc/smc_llc.c
>+++ b/net/smc/smc_llc.c
>@@ -2127,7 +2127,7 @@ void smc_llc_lgr_init(struct smc_link_group *lgr, struct smc_sock *smc)
> init_waitqueue_head(&lgr->llc_flow_waiter);
> init_waitqueue_head(&lgr->llc_msg_waiter);
> mutex_init(&lgr->llc_conf_mutex);
>- lgr->llc_testlink_time = READ_ONCE(net->ipv4.sysctl_tcp_keepalive_time);
>+ lgr->llc_testlink_time = READ_ONCE(net->smc.sysctl_smcr_testlink_time) * HZ;
> }
>
> /* called after lgr was removed from lgr_list */
>diff --git a/net/smc/smc_llc.h b/net/smc/smc_llc.h
>index 4404e52..1de9a29 100644
>--- a/net/smc/smc_llc.h
>+++ b/net/smc/smc_llc.h
>@@ -19,6 +19,7 @@
>
> #define SMC_LLC_WAIT_FIRST_TIME (5 * HZ)
> #define SMC_LLC_WAIT_TIME (2 * HZ)
>+#define SMC_LLC_TESTLINK_DEFAULT_TIME 30
I'm wondering why we don't follow the upper to macros using (30 * HZ) ?
>
> enum smc_llc_reqresp {
> SMC_LLC_REQ,
>diff --git a/net/smc/smc_sysctl.c b/net/smc/smc_sysctl.c
>index 0613868..7f68520 100644
>--- a/net/smc/smc_sysctl.c
>+++ b/net/smc/smc_sysctl.c
>@@ -16,8 +16,12 @@
>
> #include "smc.h"
> #include "smc_core.h"
>+#include "smc_llc.h"
> #include "smc_sysctl.h"
>
>+static int smcr_testlink_time_min = 1;
>+static int smcr_testlink_time_max = (INT_MAX / HZ);
>+
> static struct ctl_table smc_table[] = {
> {
> .procname = "autocorking_size",
>@@ -35,6 +39,15 @@
> .extra1 = SYSCTL_ZERO,
> .extra2 = SYSCTL_TWO,
> },
>+ {
>+ .procname = "smcr_testlink_time",
>+ .data = &init_net.smc.sysctl_smcr_testlink_time,
>+ .maxlen = sizeof(int),
>+ .mode = 0644,
>+ .proc_handler = proc_dointvec_minmax,
>+ .extra1 = &smcr_testlink_time_min,
>+ .extra2 = &smcr_testlink_time_max,
>+ },
> { }
> };
>
>@@ -60,6 +73,7 @@ int __net_init smc_sysctl_net_init(struct net *net)
>
> net->smc.sysctl_autocorking_size = SMC_AUTOCORKING_DEFAULT_SIZE;
> net->smc.sysctl_smcr_buf_type = SMCR_PHYS_CONT_BUFS;
>+ net->smc.sysctl_smcr_testlink_time = SMC_LLC_TESTLINK_DEFAULT_TIME;
>
> return 0;
>
>--
>1.8.3.1
next prev parent reply other threads:[~2022-09-20 4:55 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-20 2:45 [PATCH net-next 0/2] Separate SMC parameter settings from TCP sysctls Wen Gu
2022-09-20 2:53 ` [PATCH net-next 1/2] net/smc: Introduce a specific sysctl for TEST_LINK time Wen Gu
2022-09-20 2:57 ` Tony Lu
2022-09-20 4:55 ` dust.li [this message]
2022-09-20 6:23 ` Wen Gu
2022-09-20 8:21 ` dust.li
2022-09-20 9:03 ` Wen Gu
2022-09-20 2:54 ` [PATCH net-next 2/2] net/smc: Unbind r/w buffer size from clcsock and make them tunable Wen Gu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220920045520.GC108825@linux.alibaba.com \
--to=dust.li@linux.alibaba.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=guwen@linux.alibaba.com \
--cc=kgraul@linux.ibm.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=wenjia@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).