* [PATCH v2 00/13] create sysfs files for changing IP address
@ 2021-04-16 3:52 Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 01/13] sunrpc: Create a sunrpc directory under /sys/kernel/ Olga Kornievskaia
` (12 more replies)
0 siblings, 13 replies; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Olga Kornievskaia <kolga@netapp.com>
This is the same patch series that was introduced by Anna Schumaker
and slightly redone with also including some elements of the proposed
additions by Dan Alohi.
The main motivation behind is a situation where an NFS server
goes down and then comes back up with a different IP. These patches
provide a way for administrators to handle this issue by providing
a new IP address for one ore more existing transports to connect to.
Sysfs structure has changed slightly. Everything is still off the
/sys/kernel/sunrpc which has 2 main subdirectories "rpc-clients" and
"xprt-switches".
"xprt-switches" contain one or more subdirectories
"switch-<uniqueid>" that represent a group of transports created
for a given NFS server. Under this directory is one or more directories
named "xprt-<uniqueid>-type" where type is "udp, tcp, rdma, local".
Under each transport directory are 2 files. "dstaddr" which can
be queried to see what's this transport is connected to and "dstaddr"
can be changed by providing a new IP for the server. The setting
of the new address is allowed only for "tcp" and "rdma" transport
types.
There is also "xprt_info" file which contains a list of possibly
useful information about the transport: xprt state, last time use,
etc (for the full list see an individual commit). At the
"switch-<uniqueid>" directory there is also "xprt_switch_info" which
contains the info about number of transports and active transports
and xprt_switch's queue len.
Going back to the "rpc-clients" directory, it contains a subdirectory
for each rpc client "clnt-<uniqueid>" and inside is a symlink to
the xprt_switch directory this rpc client is using.
Some of Anna's and Dan's patches were slightly modified to satisfy
checkpatch script.
v2: fixed kernel test robot issues.
Anna Schumaker (1):
sunrpc: Prepare xs_connect() for taking NULL tasks
Dan Aloni (1):
sunrpc: add xprt id
Olga Kornievskaia (11):
sunrpc: Create a sunrpc directory under /sys/kernel/
sunrpc: Create a client/ subdirectory in the sunrpc sysfs
sunrpc: Create per-rpc_clnt sysfs kobjects
sunrpc: add IDs to multipath
sunrpc: keep track of the xprt_class in rpc_xprt structure
sunrpc: add xprt_switch direcotry to sunrpc's sysfs
sunrpc: add a symlink from rpc-client directory to the xprt_switch
sunrpc: add add sysfs directory per xprt under each xprt_switch
sunrpc: add dst_attr attributes to the sysfs xprt directory
sunrpc: provide transport info in the sysfs directory
sunrpc: provide multipath info in the sysfs directory
include/linux/sunrpc/clnt.h | 1 +
include/linux/sunrpc/xprt.h | 5 +
include/linux/sunrpc/xprtmultipath.h | 5 +
net/sunrpc/Makefile | 2 +-
net/sunrpc/clnt.c | 5 +
net/sunrpc/sunrpc_syms.c | 10 +
net/sunrpc/sysfs.c | 476 +++++++++++++++++++++++++++
net/sunrpc/sysfs.h | 42 +++
net/sunrpc/xprt.c | 26 ++
net/sunrpc/xprtmultipath.c | 32 ++
net/sunrpc/xprtrdma/transport.c | 2 +
net/sunrpc/xprtsock.c | 11 +-
12 files changed, 615 insertions(+), 2 deletions(-)
create mode 100644 net/sunrpc/sysfs.c
create mode 100644 net/sunrpc/sysfs.h
--
2.27.0
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v2 01/13] sunrpc: Create a sunrpc directory under /sys/kernel/
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 02/13] sunrpc: Create a client/ subdirectory in the sunrpc sysfs Olga Kornievskaia
` (11 subsequent siblings)
12 siblings, 0 replies; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Olga Kornievskaia <kolga@netapp.com>
This is where we'll put per-rpc_client related files
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
---
net/sunrpc/Makefile | 2 +-
net/sunrpc/sunrpc_syms.c | 8 ++++++++
net/sunrpc/sysfs.c | 20 ++++++++++++++++++++
net/sunrpc/sysfs.h | 11 +++++++++++
4 files changed, 40 insertions(+), 1 deletion(-)
create mode 100644 net/sunrpc/sysfs.c
create mode 100644 net/sunrpc/sysfs.h
diff --git a/net/sunrpc/Makefile b/net/sunrpc/Makefile
index 9488600451e8..1c8de397d6ad 100644
--- a/net/sunrpc/Makefile
+++ b/net/sunrpc/Makefile
@@ -12,7 +12,7 @@ sunrpc-y := clnt.o xprt.o socklib.o xprtsock.o sched.o \
auth.o auth_null.o auth_unix.o \
svc.o svcsock.o svcauth.o svcauth_unix.o \
addr.o rpcb_clnt.o timer.o xdr.o \
- sunrpc_syms.o cache.o rpc_pipe.o \
+ sunrpc_syms.o cache.o rpc_pipe.o sysfs.o \
svc_xprt.o \
xprtmultipath.o
sunrpc-$(CONFIG_SUNRPC_DEBUG) += debugfs.o
diff --git a/net/sunrpc/sunrpc_syms.c b/net/sunrpc/sunrpc_syms.c
index 236fadc4a439..3b57efc692ec 100644
--- a/net/sunrpc/sunrpc_syms.c
+++ b/net/sunrpc/sunrpc_syms.c
@@ -24,6 +24,7 @@
#include <linux/sunrpc/xprtsock.h>
#include "sunrpc.h"
+#include "sysfs.h"
#include "netns.h"
unsigned int sunrpc_net_id;
@@ -103,6 +104,10 @@ init_sunrpc(void)
if (err)
goto out4;
+ err = rpc_sysfs_init();
+ if (err)
+ goto out5;
+
sunrpc_debugfs_init();
#if IS_ENABLED(CONFIG_SUNRPC_DEBUG)
rpc_register_sysctl();
@@ -111,6 +116,8 @@ init_sunrpc(void)
init_socket_xprt(); /* clnt sock transport */
return 0;
+out5:
+ unregister_rpc_pipefs();
out4:
unregister_pernet_subsys(&sunrpc_net_ops);
out3:
@@ -124,6 +131,7 @@ init_sunrpc(void)
static void __exit
cleanup_sunrpc(void)
{
+ rpc_sysfs_exit();
rpc_cleanup_clids();
rpcauth_remove_module();
cleanup_socket_xprt();
diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c
new file mode 100644
index 000000000000..27eda180ac5e
--- /dev/null
+++ b/net/sunrpc/sysfs.c
@@ -0,0 +1,20 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2020 Anna Schumaker <Anna.Schumaker@Netapp.com>
+ */
+#include <linux/kobject.h>
+
+static struct kset *rpc_sunrpc_kset;
+
+int rpc_sysfs_init(void)
+{
+ rpc_sunrpc_kset = kset_create_and_add("sunrpc", NULL, kernel_kobj);
+ if (!rpc_sunrpc_kset)
+ return -ENOMEM;
+ return 0;
+}
+
+void rpc_sysfs_exit(void)
+{
+ kset_unregister(rpc_sunrpc_kset);
+}
diff --git a/net/sunrpc/sysfs.h b/net/sunrpc/sysfs.h
new file mode 100644
index 000000000000..f181c650aab8
--- /dev/null
+++ b/net/sunrpc/sysfs.h
@@ -0,0 +1,11 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2020 Anna Schumaker <Anna.Schumaker@Netapp.com>
+ */
+#ifndef __SUNRPC_SYSFS_H
+#define __SUNRPC_SYSFS_H
+
+int rpc_sysfs_init(void);
+void rpc_sysfs_exit(void);
+
+#endif
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 02/13] sunrpc: Create a client/ subdirectory in the sunrpc sysfs
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 01/13] sunrpc: Create a sunrpc directory under /sys/kernel/ Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 03/13] sunrpc: Create per-rpc_clnt sysfs kobjects Olga Kornievskaia
` (10 subsequent siblings)
12 siblings, 0 replies; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Olga Kornievskaia <kolga@netapp.com>
For network namespace separation.
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
---
net/sunrpc/sysfs.c | 42 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 42 insertions(+)
diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c
index 27eda180ac5e..6be3f4cfac95 100644
--- a/net/sunrpc/sysfs.c
+++ b/net/sunrpc/sysfs.c
@@ -2,19 +2,61 @@
/*
* Copyright (c) 2020 Anna Schumaker <Anna.Schumaker@Netapp.com>
*/
+#include <linux/sunrpc/clnt.h>
#include <linux/kobject.h>
static struct kset *rpc_sunrpc_kset;
+static struct kobject *rpc_sunrpc_client_kobj;
+
+static void rpc_sysfs_object_release(struct kobject *kobj)
+{
+ kfree(kobj);
+}
+
+static const struct kobj_ns_type_operations *
+rpc_sysfs_object_child_ns_type(struct kobject *kobj)
+{
+ return &net_ns_type_operations;
+}
+
+static struct kobj_type rpc_sysfs_object_type = {
+ .release = rpc_sysfs_object_release,
+ .sysfs_ops = &kobj_sysfs_ops,
+ .child_ns_type = rpc_sysfs_object_child_ns_type,
+};
+
+static struct kobject *rpc_sysfs_object_alloc(const char *name,
+ struct kset *kset, struct kobject *parent)
+{
+ struct kobject *kobj;
+
+ kobj = kzalloc(sizeof(*kobj), GFP_KERNEL);
+ if (kobj) {
+ kobj->kset = kset;
+ if (kobject_init_and_add(kobj, &rpc_sysfs_object_type,
+ parent, "%s", name) == 0)
+ return kobj;
+ kobject_put(kobj);
+ }
+ return NULL;
+}
int rpc_sysfs_init(void)
{
rpc_sunrpc_kset = kset_create_and_add("sunrpc", NULL, kernel_kobj);
if (!rpc_sunrpc_kset)
return -ENOMEM;
+ rpc_sunrpc_client_kobj = rpc_sysfs_object_alloc("client", rpc_sunrpc_kset, NULL);
+ if (!rpc_sunrpc_client_kobj) {
+ kset_unregister(rpc_sunrpc_kset);
+ rpc_sunrpc_client_kobj = NULL;
+ return -ENOMEM;
+ }
return 0;
}
void rpc_sysfs_exit(void)
{
+ kobject_put(rpc_sunrpc_client_kobj);
kset_unregister(rpc_sunrpc_kset);
}
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 03/13] sunrpc: Create per-rpc_clnt sysfs kobjects
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 01/13] sunrpc: Create a sunrpc directory under /sys/kernel/ Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 02/13] sunrpc: Create a client/ subdirectory in the sunrpc sysfs Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 04/13] sunrpc: Prepare xs_connect() for taking NULL tasks Olga Kornievskaia
` (9 subsequent siblings)
12 siblings, 0 replies; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Olga Kornievskaia <kolga@netapp.com>
These will eventually have files placed under them for sysfs operations.
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
---
include/linux/sunrpc/clnt.h | 1 +
net/sunrpc/clnt.c | 5 +++
net/sunrpc/sysfs.c | 61 +++++++++++++++++++++++++++++++++++++
net/sunrpc/sysfs.h | 8 +++++
4 files changed, 75 insertions(+)
diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index 02e7a5863d28..503653720e18 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -71,6 +71,7 @@ struct rpc_clnt {
#if IS_ENABLED(CONFIG_SUNRPC_DEBUG)
struct dentry *cl_debugfs; /* debugfs directory */
#endif
+ void *cl_sysfs; /* sysfs directory */
/* cl_work is only needed after cl_xpi is no longer used,
* and that are of similar size
*/
diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
index 612f0a641f4c..ceb8d19d4cb4 100644
--- a/net/sunrpc/clnt.c
+++ b/net/sunrpc/clnt.c
@@ -41,6 +41,7 @@
#include <trace/events/sunrpc.h>
#include "sunrpc.h"
+#include "sysfs.h"
#include "netns.h"
#if IS_ENABLED(CONFIG_SUNRPC_DEBUG)
@@ -300,6 +301,7 @@ static int rpc_client_register(struct rpc_clnt *clnt,
int err;
rpc_clnt_debugfs_register(clnt);
+ rpc_sysfs_client_setup(clnt, net);
pipefs_sb = rpc_get_sb_net(net);
if (pipefs_sb) {
@@ -327,6 +329,7 @@ static int rpc_client_register(struct rpc_clnt *clnt,
out:
if (pipefs_sb)
rpc_put_sb_net(net);
+ rpc_sysfs_client_destroy(clnt);
rpc_clnt_debugfs_unregister(clnt);
return err;
}
@@ -733,6 +736,7 @@ int rpc_switch_client_transport(struct rpc_clnt *clnt,
rpc_unregister_client(clnt);
__rpc_clnt_remove_pipedir(clnt);
+ rpc_sysfs_client_destroy(clnt);
rpc_clnt_debugfs_unregister(clnt);
/*
@@ -879,6 +883,7 @@ static void rpc_free_client_work(struct work_struct *work)
* so they cannot be called in rpciod, so they are handled separately
* here.
*/
+ rpc_sysfs_client_destroy(clnt);
rpc_clnt_debugfs_unregister(clnt);
rpc_free_clid(clnt);
rpc_clnt_remove_pipedir(clnt);
diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c
index 6be3f4cfac95..d14d54f33c65 100644
--- a/net/sunrpc/sysfs.c
+++ b/net/sunrpc/sysfs.c
@@ -4,6 +4,7 @@
*/
#include <linux/sunrpc/clnt.h>
#include <linux/kobject.h>
+#include "sysfs.h"
static struct kset *rpc_sunrpc_kset;
static struct kobject *rpc_sunrpc_client_kobj;
@@ -55,8 +56,68 @@ int rpc_sysfs_init(void)
return 0;
}
+static void rpc_sysfs_client_release(struct kobject *kobj)
+{
+ struct rpc_sysfs_client *c;
+
+ c = container_of(kobj, struct rpc_sysfs_client, kobject);
+ kfree(c);
+}
+
+static const void *rpc_sysfs_client_namespace(struct kobject *kobj)
+{
+ return container_of(kobj, struct rpc_sysfs_client, kobject)->net;
+}
+
+static struct kobj_type rpc_sysfs_client_type = {
+ .release = rpc_sysfs_client_release,
+ .sysfs_ops = &kobj_sysfs_ops,
+ .namespace = rpc_sysfs_client_namespace,
+};
+
void rpc_sysfs_exit(void)
{
kobject_put(rpc_sunrpc_client_kobj);
kset_unregister(rpc_sunrpc_kset);
}
+
+static struct rpc_sysfs_client *rpc_sysfs_client_alloc(struct kobject *parent,
+ struct net *net,
+ int clid)
+{
+ struct rpc_sysfs_client *p;
+
+ p = kzalloc(sizeof(*p), GFP_KERNEL);
+ if (p) {
+ p->net = net;
+ p->kobject.kset = rpc_sunrpc_kset;
+ if (kobject_init_and_add(&p->kobject, &rpc_sysfs_client_type,
+ parent, "clnt-%d", clid) == 0)
+ return p;
+ kobject_put(&p->kobject);
+ }
+ return NULL;
+}
+
+void rpc_sysfs_client_setup(struct rpc_clnt *clnt, struct net *net)
+{
+ struct rpc_sysfs_client *rpc_client;
+
+ rpc_client = rpc_sysfs_client_alloc(rpc_sunrpc_client_kobj, net, clnt->cl_clid);
+ if (rpc_client) {
+ clnt->cl_sysfs = rpc_client;
+ kobject_uevent(&rpc_client->kobject, KOBJ_ADD);
+ }
+}
+
+void rpc_sysfs_client_destroy(struct rpc_clnt *clnt)
+{
+ struct rpc_sysfs_client *rpc_client = clnt->cl_sysfs;
+
+ if (rpc_client) {
+ kobject_uevent(&rpc_client->kobject, KOBJ_REMOVE);
+ kobject_del(&rpc_client->kobject);
+ kobject_put(&rpc_client->kobject);
+ clnt->cl_sysfs = NULL;
+ }
+}
diff --git a/net/sunrpc/sysfs.h b/net/sunrpc/sysfs.h
index f181c650aab8..c46afc848993 100644
--- a/net/sunrpc/sysfs.h
+++ b/net/sunrpc/sysfs.h
@@ -5,7 +5,15 @@
#ifndef __SUNRPC_SYSFS_H
#define __SUNRPC_SYSFS_H
+struct rpc_sysfs_client {
+ struct kobject kobject;
+ struct net *net;
+};
+
int rpc_sysfs_init(void);
void rpc_sysfs_exit(void);
+void rpc_sysfs_client_setup(struct rpc_clnt *clnt, struct net *net);
+void rpc_sysfs_client_destroy(struct rpc_clnt *clnt);
+
#endif
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 04/13] sunrpc: Prepare xs_connect() for taking NULL tasks
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
` (2 preceding siblings ...)
2021-04-16 3:52 ` [PATCH v2 03/13] sunrpc: Create per-rpc_clnt sysfs kobjects Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 05/13] sunrpc: add xprt id Olga Kornievskaia
` (8 subsequent siblings)
12 siblings, 0 replies; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Anna Schumaker <Anna.Schumaker@Netapp.com>
We won't have a task structure when we go to change IP addresses, so
check for one before calling the WARN_ON() to avoid crashing.
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
---
net/sunrpc/xprtsock.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index 47aa47a2b07c..2bcb80c19339 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -2317,7 +2317,7 @@ static void xs_connect(struct rpc_xprt *xprt, struct rpc_task *task)
struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt);
unsigned long delay = 0;
- WARN_ON_ONCE(!xprt_lock_connect(xprt, task, transport));
+ WARN_ON_ONCE(task && !xprt_lock_connect(xprt, task, transport));
if (transport->sock != NULL) {
dprintk("RPC: xs_connect delayed xprt %p for %lu "
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 05/13] sunrpc: add xprt id
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
` (3 preceding siblings ...)
2021-04-16 3:52 ` [PATCH v2 04/13] sunrpc: Prepare xs_connect() for taking NULL tasks Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 06/13] sunrpc: add IDs to multipath Olga Kornievskaia
` (7 subsequent siblings)
12 siblings, 0 replies; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Dan Aloni <dan@kernelim.com>
This adds a unique identifier for a sunrpc transport in sysfs, which is
similarly managed to the unique IDs of clients.
Signed-off-by: Dan Aloni <dan@kernelim.com>
---
include/linux/sunrpc/xprt.h | 2 ++
net/sunrpc/sunrpc_syms.c | 1 +
net/sunrpc/xprt.c | 26 ++++++++++++++++++++++++++
3 files changed, 29 insertions(+)
diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h
index d81fe8b364d0..82294d06075c 100644
--- a/include/linux/sunrpc/xprt.h
+++ b/include/linux/sunrpc/xprt.h
@@ -185,6 +185,7 @@ enum xprt_transports {
struct rpc_xprt {
struct kref kref; /* Reference count */
const struct rpc_xprt_ops *ops; /* transport methods */
+ unsigned int id; /* transport id */
const struct rpc_timeout *timeout; /* timeout parms */
struct sockaddr_storage addr; /* server address */
@@ -368,6 +369,7 @@ struct rpc_xprt * xprt_alloc(struct net *net, size_t size,
unsigned int num_prealloc,
unsigned int max_req);
void xprt_free(struct rpc_xprt *);
+void xprt_cleanup_ids(void);
static inline int
xprt_enable_swap(struct rpc_xprt *xprt)
diff --git a/net/sunrpc/sunrpc_syms.c b/net/sunrpc/sunrpc_syms.c
index 3b57efc692ec..b61b74c00483 100644
--- a/net/sunrpc/sunrpc_syms.c
+++ b/net/sunrpc/sunrpc_syms.c
@@ -133,6 +133,7 @@ cleanup_sunrpc(void)
{
rpc_sysfs_exit();
rpc_cleanup_clids();
+ xprt_cleanup_ids();
rpcauth_remove_module();
cleanup_socket_xprt();
svc_cleanup_xprt_sock();
diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c
index a853f75d4968..6181792aec23 100644
--- a/net/sunrpc/xprt.c
+++ b/net/sunrpc/xprt.c
@@ -1719,6 +1719,30 @@ static void xprt_free_all_slots(struct rpc_xprt *xprt)
}
}
+static DEFINE_IDA(rpc_xprt_ids);
+
+void xprt_cleanup_ids(void)
+{
+ ida_destroy(&rpc_xprt_ids);
+}
+
+static int xprt_alloc_id(struct rpc_xprt *xprt)
+{
+ int id;
+
+ id = ida_simple_get(&rpc_xprt_ids, 0, 0, GFP_KERNEL);
+ if (id < 0)
+ return id;
+
+ xprt->id = id;
+ return 0;
+}
+
+static void xprt_free_id(struct rpc_xprt *xprt)
+{
+ ida_simple_remove(&rpc_xprt_ids, xprt->id);
+}
+
struct rpc_xprt *xprt_alloc(struct net *net, size_t size,
unsigned int num_prealloc,
unsigned int max_alloc)
@@ -1731,6 +1755,7 @@ struct rpc_xprt *xprt_alloc(struct net *net, size_t size,
if (xprt == NULL)
goto out;
+ xprt_alloc_id(xprt);
xprt_init(xprt, net);
for (i = 0; i < num_prealloc; i++) {
@@ -1759,6 +1784,7 @@ void xprt_free(struct rpc_xprt *xprt)
{
put_net(xprt->xprt_net);
xprt_free_all_slots(xprt);
+ xprt_free_id(xprt);
kfree_rcu(xprt, rcu);
}
EXPORT_SYMBOL_GPL(xprt_free);
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 06/13] sunrpc: add IDs to multipath
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
` (4 preceding siblings ...)
2021-04-16 3:52 ` [PATCH v2 05/13] sunrpc: add xprt id Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
2021-04-23 21:16 ` Trond Myklebust
2021-04-16 3:52 ` [PATCH v2 07/13] sunrpc: keep track of the xprt_class in rpc_xprt structure Olga Kornievskaia
` (6 subsequent siblings)
12 siblings, 1 reply; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Olga Kornievskaia <kolga@netapp.com>
This is used to uniquely identify sunrpc multipath objects in /sys.
Signed-off-by: Dan Aloni <dan@kernelim.com>
---
include/linux/sunrpc/xprtmultipath.h | 4 ++++
net/sunrpc/sunrpc_syms.c | 1 +
net/sunrpc/xprtmultipath.c | 26 ++++++++++++++++++++++++++
3 files changed, 31 insertions(+)
diff --git a/include/linux/sunrpc/xprtmultipath.h b/include/linux/sunrpc/xprtmultipath.h
index c6cce3fbf29d..ef95a6f18ccf 100644
--- a/include/linux/sunrpc/xprtmultipath.h
+++ b/include/linux/sunrpc/xprtmultipath.h
@@ -14,6 +14,7 @@ struct rpc_xprt_switch {
spinlock_t xps_lock;
struct kref xps_kref;
+ unsigned int xps_id;
unsigned int xps_nxprts;
unsigned int xps_nactive;
atomic_long_t xps_queuelen;
@@ -71,4 +72,7 @@ extern struct rpc_xprt *xprt_iter_get_next(struct rpc_xprt_iter *xpi);
extern bool rpc_xprt_switch_has_addr(struct rpc_xprt_switch *xps,
const struct sockaddr *sap);
+
+extern void xprt_multipath_cleanup_ids(void);
+
#endif
diff --git a/net/sunrpc/sunrpc_syms.c b/net/sunrpc/sunrpc_syms.c
index b61b74c00483..691c0000e9ea 100644
--- a/net/sunrpc/sunrpc_syms.c
+++ b/net/sunrpc/sunrpc_syms.c
@@ -134,6 +134,7 @@ cleanup_sunrpc(void)
rpc_sysfs_exit();
rpc_cleanup_clids();
xprt_cleanup_ids();
+ xprt_multipath_cleanup_ids();
rpcauth_remove_module();
cleanup_socket_xprt();
svc_cleanup_xprt_sock();
diff --git a/net/sunrpc/xprtmultipath.c b/net/sunrpc/xprtmultipath.c
index 78c075a68c04..b71dd95ad7de 100644
--- a/net/sunrpc/xprtmultipath.c
+++ b/net/sunrpc/xprtmultipath.c
@@ -86,6 +86,30 @@ void rpc_xprt_switch_remove_xprt(struct rpc_xprt_switch *xps,
xprt_put(xprt);
}
+static DEFINE_IDA(rpc_xprtswitch_ids);
+
+void xprt_multipath_cleanup_ids(void)
+{
+ ida_destroy(&rpc_xprtswitch_ids);
+}
+
+static int xprt_switch_alloc_id(struct rpc_xprt_switch *xps)
+{
+ int id;
+
+ id = ida_simple_get(&rpc_xprtswitch_ids, 0, 0, GFP_KERNEL);
+ if (id < 0)
+ return id;
+
+ xps->xps_id = id;
+ return 0;
+}
+
+static void xprt_switch_free_id(struct rpc_xprt_switch *xps)
+{
+ ida_simple_remove(&rpc_xprtswitch_ids, xps->xps_id);
+}
+
/**
* xprt_switch_alloc - Allocate a new struct rpc_xprt_switch
* @xprt: pointer to struct rpc_xprt
@@ -103,6 +127,7 @@ struct rpc_xprt_switch *xprt_switch_alloc(struct rpc_xprt *xprt,
if (xps != NULL) {
spin_lock_init(&xps->xps_lock);
kref_init(&xps->xps_kref);
+ xprt_switch_alloc_id(xps);
xps->xps_nxprts = xps->xps_nactive = 0;
atomic_long_set(&xps->xps_queuelen, 0);
xps->xps_net = NULL;
@@ -136,6 +161,7 @@ static void xprt_switch_free(struct kref *kref)
struct rpc_xprt_switch, xps_kref);
xprt_switch_free_entries(xps);
+ xprt_switch_free_id(xps);
kfree_rcu(xps, xps_rcu);
}
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 07/13] sunrpc: keep track of the xprt_class in rpc_xprt structure
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
` (5 preceding siblings ...)
2021-04-16 3:52 ` [PATCH v2 06/13] sunrpc: add IDs to multipath Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 08/13] sunrpc: add xprt_switch direcotry to sunrpc's sysfs Olga Kornievskaia
` (5 subsequent siblings)
12 siblings, 0 replies; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Olga Kornievskaia <kolga@netapp.com>
We need to keep track of the type for a given transport.
Signed-off-by: Olga Kornievskaia <kolga@netapp.com>
---
include/linux/sunrpc/xprt.h | 2 ++
net/sunrpc/xprtrdma/transport.c | 2 ++
net/sunrpc/xprtsock.c | 9 +++++++++
3 files changed, 13 insertions(+)
diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h
index 82294d06075c..a2edcc42e6c4 100644
--- a/include/linux/sunrpc/xprt.h
+++ b/include/linux/sunrpc/xprt.h
@@ -53,6 +53,7 @@ enum rpc_display_format_t {
struct rpc_task;
struct rpc_xprt;
+struct xprt_class;
struct seq_file;
struct svc_serv;
struct net;
@@ -289,6 +290,7 @@ struct rpc_xprt {
atomic_t inject_disconnect;
#endif
struct rcu_head rcu;
+ const struct xprt_class *xprt_class;
};
#if defined(CONFIG_SUNRPC_BACKCHANNEL)
diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c
index 78d29d1bcc20..de0dec5f6d5b 100644
--- a/net/sunrpc/xprtrdma/transport.c
+++ b/net/sunrpc/xprtrdma/transport.c
@@ -73,6 +73,7 @@ unsigned int xprt_rdma_max_inline_read = RPCRDMA_DEF_INLINE;
unsigned int xprt_rdma_max_inline_write = RPCRDMA_DEF_INLINE;
unsigned int xprt_rdma_memreg_strategy = RPCRDMA_FRWR;
int xprt_rdma_pad_optimize;
+static struct xprt_class xprt_rdma;
#if IS_ENABLED(CONFIG_SUNRPC_DEBUG)
@@ -347,6 +348,7 @@ xprt_setup_rdma(struct xprt_create *args)
/* Ensure xprt->addr holds valid server TCP (not RDMA)
* address, for any side protocols which peek at it */
xprt->prot = IPPROTO_TCP;
+ xprt->xprt_class = &xprt_rdma;
xprt->addrlen = args->addrlen;
memcpy(&xprt->addr, sap, xprt->addrlen);
diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index 2bcb80c19339..5ff37badd335 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -91,6 +91,11 @@ static unsigned int xprt_max_resvport_limit = RPC_MAX_RESVPORT;
static struct ctl_table_header *sunrpc_table_header;
+static struct xprt_class xs_local_transport;
+static struct xprt_class xs_udp_transport;
+static struct xprt_class xs_tcp_transport;
+static struct xprt_class xs_bc_tcp_transport;
+
/*
* FIXME: changing the UDP slot table size should also resize the UDP
* socket buffers for existing UDP transports
@@ -2777,6 +2782,7 @@ static struct rpc_xprt *xs_setup_local(struct xprt_create *args)
transport = container_of(xprt, struct sock_xprt, xprt);
xprt->prot = 0;
+ xprt->xprt_class = &xs_local_transport;
xprt->max_payload = RPC_MAX_FRAGMENT_SIZE;
xprt->bind_timeout = XS_BIND_TO;
@@ -2846,6 +2852,7 @@ static struct rpc_xprt *xs_setup_udp(struct xprt_create *args)
transport = container_of(xprt, struct sock_xprt, xprt);
xprt->prot = IPPROTO_UDP;
+ xprt->xprt_class = &xs_udp_transport;
/* XXX: header size can vary due to auth type, IPv6, etc. */
xprt->max_payload = (1U << 16) - (MAX_HEADER << 3);
@@ -2926,6 +2933,7 @@ static struct rpc_xprt *xs_setup_tcp(struct xprt_create *args)
transport = container_of(xprt, struct sock_xprt, xprt);
xprt->prot = IPPROTO_TCP;
+ xprt->xprt_class = &xs_tcp_transport;
xprt->max_payload = RPC_MAX_FRAGMENT_SIZE;
xprt->bind_timeout = XS_BIND_TO;
@@ -2999,6 +3007,7 @@ static struct rpc_xprt *xs_setup_bc_tcp(struct xprt_create *args)
transport = container_of(xprt, struct sock_xprt, xprt);
xprt->prot = IPPROTO_TCP;
+ xprt->xprt_class = &xs_bc_tcp_transport;
xprt->max_payload = RPC_MAX_FRAGMENT_SIZE;
xprt->timeout = &xs_tcp_default_timeout;
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 08/13] sunrpc: add xprt_switch direcotry to sunrpc's sysfs
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
` (6 preceding siblings ...)
2021-04-16 3:52 ` [PATCH v2 07/13] sunrpc: keep track of the xprt_class in rpc_xprt structure Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
2021-04-23 21:20 ` Trond Myklebust
2021-04-16 3:52 ` [PATCH v2 09/13] sunrpc: add a symlink from rpc-client directory to the xprt_switch Olga Kornievskaia
` (4 subsequent siblings)
12 siblings, 1 reply; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Olga Kornievskaia <kolga@netapp.com>
Add xprt_switch directory to the sysfs and create individual
xprt_swith subdirectories for multipath transport group.
Signed-off-by: Olga Kornievskaia <kolga@netapp.com>
---
include/linux/sunrpc/xprtmultipath.h | 1 +
net/sunrpc/sysfs.c | 97 ++++++++++++++++++++++++++--
net/sunrpc/sysfs.h | 10 +++
net/sunrpc/xprtmultipath.c | 4 ++
4 files changed, 105 insertions(+), 7 deletions(-)
diff --git a/include/linux/sunrpc/xprtmultipath.h b/include/linux/sunrpc/xprtmultipath.h
index ef95a6f18ccf..47b0a85cdcfa 100644
--- a/include/linux/sunrpc/xprtmultipath.h
+++ b/include/linux/sunrpc/xprtmultipath.h
@@ -24,6 +24,7 @@ struct rpc_xprt_switch {
const struct rpc_xprt_iter_ops *xps_iter_ops;
+ void *xps_sysfs;
struct rcu_head xps_rcu;
};
diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c
index d14d54f33c65..0c34330714ab 100644
--- a/net/sunrpc/sysfs.c
+++ b/net/sunrpc/sysfs.c
@@ -7,7 +7,7 @@
#include "sysfs.h"
static struct kset *rpc_sunrpc_kset;
-static struct kobject *rpc_sunrpc_client_kobj;
+static struct kobject *rpc_sunrpc_client_kobj, *rpc_sunrpc_xprt_switch_kobj;
static void rpc_sysfs_object_release(struct kobject *kobj)
{
@@ -47,13 +47,22 @@ int rpc_sysfs_init(void)
rpc_sunrpc_kset = kset_create_and_add("sunrpc", NULL, kernel_kobj);
if (!rpc_sunrpc_kset)
return -ENOMEM;
- rpc_sunrpc_client_kobj = rpc_sysfs_object_alloc("client", rpc_sunrpc_kset, NULL);
- if (!rpc_sunrpc_client_kobj) {
- kset_unregister(rpc_sunrpc_kset);
- rpc_sunrpc_client_kobj = NULL;
- return -ENOMEM;
- }
+ rpc_sunrpc_client_kobj =
+ rpc_sysfs_object_alloc("rpc-clients", rpc_sunrpc_kset, NULL);
+ if (!rpc_sunrpc_client_kobj)
+ goto err_client;
+ rpc_sunrpc_xprt_switch_kobj =
+ rpc_sysfs_object_alloc("xprt-switches", rpc_sunrpc_kset, NULL);
+ if (!rpc_sunrpc_xprt_switch_kobj)
+ goto err_switch;
return 0;
+err_switch:
+ kobject_put(rpc_sunrpc_client_kobj);
+ rpc_sunrpc_client_kobj = NULL;
+err_client:
+ kset_unregister(rpc_sunrpc_kset);
+ rpc_sunrpc_kset = NULL;
+ return -ENOMEM;
}
static void rpc_sysfs_client_release(struct kobject *kobj)
@@ -64,20 +73,40 @@ static void rpc_sysfs_client_release(struct kobject *kobj)
kfree(c);
}
+static void rpc_sysfs_xprt_switch_release(struct kobject *kobj)
+{
+ struct rpc_sysfs_xprt_switch *xprt_switch;
+
+ xprt_switch = container_of(kobj, struct rpc_sysfs_xprt_switch, kobject);
+ kfree(xprt_switch);
+}
+
static const void *rpc_sysfs_client_namespace(struct kobject *kobj)
{
return container_of(kobj, struct rpc_sysfs_client, kobject)->net;
}
+static const void *rpc_sysfs_xprt_switch_namespace(struct kobject *kobj)
+{
+ return container_of(kobj, struct rpc_sysfs_xprt_switch, kobject)->net;
+}
+
static struct kobj_type rpc_sysfs_client_type = {
.release = rpc_sysfs_client_release,
.sysfs_ops = &kobj_sysfs_ops,
.namespace = rpc_sysfs_client_namespace,
};
+static struct kobj_type rpc_sysfs_xprt_switch_type = {
+ .release = rpc_sysfs_xprt_switch_release,
+ .sysfs_ops = &kobj_sysfs_ops,
+ .namespace = rpc_sysfs_xprt_switch_namespace,
+};
+
void rpc_sysfs_exit(void)
{
kobject_put(rpc_sunrpc_client_kobj);
+ kobject_put(rpc_sunrpc_xprt_switch_kobj);
kset_unregister(rpc_sunrpc_kset);
}
@@ -99,6 +128,27 @@ static struct rpc_sysfs_client *rpc_sysfs_client_alloc(struct kobject *parent,
return NULL;
}
+static struct rpc_sysfs_xprt_switch *
+rpc_sysfs_xprt_switch_alloc(struct kobject *parent,
+ struct rpc_xprt_switch *xprt_switch,
+ struct net *net)
+{
+ struct rpc_sysfs_xprt_switch *p;
+
+ p = kzalloc(sizeof(*p), GFP_KERNEL);
+ if (p) {
+ p->net = net;
+ p->kobject.kset = rpc_sunrpc_kset;
+ if (kobject_init_and_add(&p->kobject,
+ &rpc_sysfs_xprt_switch_type,
+ parent, "switch-%d",
+ xprt_switch->xps_id) == 0)
+ return p;
+ kobject_put(&p->kobject);
+ }
+ return NULL;
+}
+
void rpc_sysfs_client_setup(struct rpc_clnt *clnt, struct net *net)
{
struct rpc_sysfs_client *rpc_client;
@@ -110,6 +160,27 @@ void rpc_sysfs_client_setup(struct rpc_clnt *clnt, struct net *net)
}
}
+void rpc_sysfs_xprt_switch_setup(struct rpc_xprt_switch *xprt_switch,
+ struct rpc_xprt *xprt)
+{
+ struct rpc_sysfs_xprt_switch *rpc_xprt_switch;
+ struct net *net;
+
+ if (xprt_switch->xps_net)
+ net = xprt_switch->xps_net;
+ else
+ net = xprt->xprt_net;
+ rpc_xprt_switch =
+ rpc_sysfs_xprt_switch_alloc(rpc_sunrpc_xprt_switch_kobj,
+ xprt_switch, net);
+ if (rpc_xprt_switch) {
+ xprt_switch->xps_sysfs = rpc_xprt_switch;
+ rpc_xprt_switch->xprt_switch = xprt_switch;
+ rpc_xprt_switch->xprt = xprt;
+ kobject_uevent(&rpc_xprt_switch->kobject, KOBJ_ADD);
+ }
+}
+
void rpc_sysfs_client_destroy(struct rpc_clnt *clnt)
{
struct rpc_sysfs_client *rpc_client = clnt->cl_sysfs;
@@ -121,3 +192,15 @@ void rpc_sysfs_client_destroy(struct rpc_clnt *clnt)
clnt->cl_sysfs = NULL;
}
}
+
+void rpc_sysfs_xprt_switch_destroy(struct rpc_xprt_switch *xprt_switch)
+{
+ struct rpc_sysfs_xprt_switch *rpc_xprt_switch = xprt_switch->xps_sysfs;
+
+ if (rpc_xprt_switch) {
+ kobject_uevent(&rpc_xprt_switch->kobject, KOBJ_REMOVE);
+ kobject_del(&rpc_xprt_switch->kobject);
+ kobject_put(&rpc_xprt_switch->kobject);
+ xprt_switch->xps_sysfs = NULL;
+ }
+}
diff --git a/net/sunrpc/sysfs.h b/net/sunrpc/sysfs.h
index c46afc848993..9b6acd3fd3dc 100644
--- a/net/sunrpc/sysfs.h
+++ b/net/sunrpc/sysfs.h
@@ -10,10 +10,20 @@ struct rpc_sysfs_client {
struct net *net;
};
+struct rpc_sysfs_xprt_switch {
+ struct kobject kobject;
+ struct net *net;
+ struct rpc_xprt_switch *xprt_switch;
+ struct rpc_xprt *xprt;
+};
+
int rpc_sysfs_init(void);
void rpc_sysfs_exit(void);
void rpc_sysfs_client_setup(struct rpc_clnt *clnt, struct net *net);
void rpc_sysfs_client_destroy(struct rpc_clnt *clnt);
+void rpc_sysfs_xprt_switch_setup(struct rpc_xprt_switch *xprt_switch,
+ struct rpc_xprt *xprt);
+void rpc_sysfs_xprt_switch_destroy(struct rpc_xprt_switch *xprt);
#endif
diff --git a/net/sunrpc/xprtmultipath.c b/net/sunrpc/xprtmultipath.c
index b71dd95ad7de..1ed16e4cc465 100644
--- a/net/sunrpc/xprtmultipath.c
+++ b/net/sunrpc/xprtmultipath.c
@@ -19,6 +19,8 @@
#include <linux/sunrpc/addr.h>
#include <linux/sunrpc/xprtmultipath.h>
+#include "sysfs.h"
+
typedef struct rpc_xprt *(*xprt_switch_find_xprt_t)(struct rpc_xprt_switch *xps,
const struct rpc_xprt *cur);
@@ -133,6 +135,7 @@ struct rpc_xprt_switch *xprt_switch_alloc(struct rpc_xprt *xprt,
xps->xps_net = NULL;
INIT_LIST_HEAD(&xps->xps_xprt_list);
xps->xps_iter_ops = &rpc_xprt_iter_singular;
+ rpc_sysfs_xprt_switch_setup(xps, xprt);
xprt_switch_add_xprt_locked(xps, xprt);
}
@@ -161,6 +164,7 @@ static void xprt_switch_free(struct kref *kref)
struct rpc_xprt_switch, xps_kref);
xprt_switch_free_entries(xps);
+ rpc_sysfs_xprt_switch_destroy(xps);
xprt_switch_free_id(xps);
kfree_rcu(xps, xps_rcu);
}
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 09/13] sunrpc: add a symlink from rpc-client directory to the xprt_switch
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
` (7 preceding siblings ...)
2021-04-16 3:52 ` [PATCH v2 08/13] sunrpc: add xprt_switch direcotry to sunrpc's sysfs Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 10/13] sunrpc: add add sysfs directory per xprt under each xprt_switch Olga Kornievskaia
` (3 subsequent siblings)
12 siblings, 0 replies; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Olga Kornievskaia <kolga@netapp.com>
An rpc client uses a transport switch and one ore more transports
associated with that switch. Since transports are shared among
rpc clients, create a symlink into the xprt_switch directory
instead of duplicating entries under each rpc client.
Signed-off-by: Olga Kornievskaia <kolga@netapp.com>
---
net/sunrpc/clnt.c | 2 +-
net/sunrpc/sysfs.c | 25 +++++++++++++++++++++++--
net/sunrpc/sysfs.h | 6 +++++-
3 files changed, 29 insertions(+), 4 deletions(-)
diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
index ceb8d19d4cb4..fad87dba5114 100644
--- a/net/sunrpc/clnt.c
+++ b/net/sunrpc/clnt.c
@@ -301,7 +301,6 @@ static int rpc_client_register(struct rpc_clnt *clnt,
int err;
rpc_clnt_debugfs_register(clnt);
- rpc_sysfs_client_setup(clnt, net);
pipefs_sb = rpc_get_sb_net(net);
if (pipefs_sb) {
@@ -426,6 +425,7 @@ static struct rpc_clnt * rpc_new_client(const struct rpc_create_args *args,
/* save the nodename */
rpc_clnt_set_nodename(clnt, nodename);
+ rpc_sysfs_client_setup(clnt, xps, rpc_net_ns(clnt));
err = rpc_client_register(clnt, args->authflavor, args->client_name);
if (err)
goto out_no_path;
diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c
index 0c34330714ab..ce2cad1b6aa6 100644
--- a/net/sunrpc/sysfs.c
+++ b/net/sunrpc/sysfs.c
@@ -149,14 +149,30 @@ rpc_sysfs_xprt_switch_alloc(struct kobject *parent,
return NULL;
}
-void rpc_sysfs_client_setup(struct rpc_clnt *clnt, struct net *net)
+void rpc_sysfs_client_setup(struct rpc_clnt *clnt,
+ struct rpc_xprt_switch *xprt_switch,
+ struct net *net)
{
struct rpc_sysfs_client *rpc_client;
- rpc_client = rpc_sysfs_client_alloc(rpc_sunrpc_client_kobj, net, clnt->cl_clid);
+ rpc_client = rpc_sysfs_client_alloc(rpc_sunrpc_client_kobj,
+ net, clnt->cl_clid);
if (rpc_client) {
+ char name[23];
+ struct rpc_sysfs_xprt_switch *xswitch =
+ (struct rpc_sysfs_xprt_switch *)xprt_switch->xps_sysfs;
+ int ret;
+
clnt->cl_sysfs = rpc_client;
+ rpc_client->clnt = clnt;
+ rpc_client->xprt_switch = xprt_switch;
kobject_uevent(&rpc_client->kobject, KOBJ_ADD);
+ snprintf(name, sizeof(name), "switch-%d", xprt_switch->xps_id);
+ ret = sysfs_create_link_nowarn(&rpc_client->kobject,
+ &xswitch->kobject, name);
+ if (ret)
+ pr_warn("can't create link to %s in sysfs (%d)\n",
+ name, ret);
}
}
@@ -186,6 +202,11 @@ void rpc_sysfs_client_destroy(struct rpc_clnt *clnt)
struct rpc_sysfs_client *rpc_client = clnt->cl_sysfs;
if (rpc_client) {
+ char name[23];
+
+ snprintf(name, sizeof(name), "switch-%d",
+ rpc_client->xprt_switch->xps_id);
+ sysfs_remove_link(&rpc_client->kobject, name);
kobject_uevent(&rpc_client->kobject, KOBJ_REMOVE);
kobject_del(&rpc_client->kobject);
kobject_put(&rpc_client->kobject);
diff --git a/net/sunrpc/sysfs.h b/net/sunrpc/sysfs.h
index 9b6acd3fd3dc..9a0625b1cd65 100644
--- a/net/sunrpc/sysfs.h
+++ b/net/sunrpc/sysfs.h
@@ -8,6 +8,8 @@
struct rpc_sysfs_client {
struct kobject kobject;
struct net *net;
+ struct rpc_clnt *clnt;
+ struct rpc_xprt_switch *xprt_switch;
};
struct rpc_sysfs_xprt_switch {
@@ -20,7 +22,9 @@ struct rpc_sysfs_xprt_switch {
int rpc_sysfs_init(void);
void rpc_sysfs_exit(void);
-void rpc_sysfs_client_setup(struct rpc_clnt *clnt, struct net *net);
+void rpc_sysfs_client_setup(struct rpc_clnt *clnt,
+ struct rpc_xprt_switch *xprt_switch,
+ struct net *net);
void rpc_sysfs_client_destroy(struct rpc_clnt *clnt);
void rpc_sysfs_xprt_switch_setup(struct rpc_xprt_switch *xprt_switch,
struct rpc_xprt *xprt);
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 10/13] sunrpc: add add sysfs directory per xprt under each xprt_switch
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
` (8 preceding siblings ...)
2021-04-16 3:52 ` [PATCH v2 09/13] sunrpc: add a symlink from rpc-client directory to the xprt_switch Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
2021-04-23 21:21 ` Trond Myklebust
2021-04-16 3:52 ` [PATCH v2 11/13] sunrpc: add dst_attr attributes to the sysfs xprt directory Olga Kornievskaia
` (2 subsequent siblings)
12 siblings, 1 reply; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Olga Kornievskaia <kolga@netapp.com>
Add individual transport directories under each transport switch
group. For instance, for each nconnect=X connections there will be
a transport directory. Naming conventions also identifies transport
type -- xprt-<id>-<type> where type is udp, tcp, rdma, local, bc.
Signed-off-by: Olga Kornievskaia <kolga@netapp.com>
---
include/linux/sunrpc/xprt.h | 1 +
net/sunrpc/sysfs.c | 83 +++++++++++++++++++++++++++++++++++++
net/sunrpc/sysfs.h | 9 ++++
net/sunrpc/xprtmultipath.c | 2 +
4 files changed, 95 insertions(+)
diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h
index a2edcc42e6c4..1e4906759a6a 100644
--- a/include/linux/sunrpc/xprt.h
+++ b/include/linux/sunrpc/xprt.h
@@ -291,6 +291,7 @@ struct rpc_xprt {
#endif
struct rcu_head rcu;
const struct xprt_class *xprt_class;
+ void *xprt_sysfs;
};
#if defined(CONFIG_SUNRPC_BACKCHANNEL)
diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c
index ce2cad1b6aa6..5410d8fe1181 100644
--- a/net/sunrpc/sysfs.c
+++ b/net/sunrpc/sysfs.c
@@ -81,6 +81,14 @@ static void rpc_sysfs_xprt_switch_release(struct kobject *kobj)
kfree(xprt_switch);
}
+static void rpc_sysfs_xprt_switch_xprt_release(struct kobject *kobj)
+{
+ struct rpc_sysfs_xprt_switch_xprt *xprt;
+
+ xprt = container_of(kobj, struct rpc_sysfs_xprt_switch_xprt, kobject);
+ kfree(xprt);
+}
+
static const void *rpc_sysfs_client_namespace(struct kobject *kobj)
{
return container_of(kobj, struct rpc_sysfs_client, kobject)->net;
@@ -91,6 +99,12 @@ static const void *rpc_sysfs_xprt_switch_namespace(struct kobject *kobj)
return container_of(kobj, struct rpc_sysfs_xprt_switch, kobject)->net;
}
+static const void *rpc_sysfs_xprt_switch_xprt_namespace(struct kobject *kobj)
+{
+ return container_of(kobj, struct rpc_sysfs_xprt_switch_xprt,
+ kobject)->net;
+}
+
static struct kobj_type rpc_sysfs_client_type = {
.release = rpc_sysfs_client_release,
.sysfs_ops = &kobj_sysfs_ops,
@@ -103,6 +117,12 @@ static struct kobj_type rpc_sysfs_xprt_switch_type = {
.namespace = rpc_sysfs_xprt_switch_namespace,
};
+static struct kobj_type rpc_sysfs_xprt_switch_xprt_type = {
+ .release = rpc_sysfs_xprt_switch_xprt_release,
+ .sysfs_ops = &kobj_sysfs_ops,
+ .namespace = rpc_sysfs_xprt_switch_xprt_namespace,
+};
+
void rpc_sysfs_exit(void)
{
kobject_put(rpc_sunrpc_client_kobj);
@@ -149,6 +169,39 @@ rpc_sysfs_xprt_switch_alloc(struct kobject *parent,
return NULL;
}
+static struct rpc_sysfs_xprt_switch_xprt *
+rpc_sysfs_xprt_switch_xprt_alloc(struct kobject *parent,
+ struct rpc_xprt *xprt,
+ struct net *net)
+{
+ struct rpc_sysfs_xprt_switch_xprt *p;
+
+ p = kzalloc(sizeof(*p), GFP_KERNEL);
+ if (p) {
+ char type[6];
+
+ p->net = net;
+ p->kobject.kset = rpc_sunrpc_kset;
+ if (xprt->xprt_class->ident == XPRT_TRANSPORT_RDMA)
+ snprintf(type, sizeof(type), "rdma");
+ else if (xprt->xprt_class->ident == XPRT_TRANSPORT_TCP)
+ snprintf(type, sizeof(type), "tcp");
+ else if (xprt->xprt_class->ident == XPRT_TRANSPORT_UDP)
+ snprintf(type, sizeof(type), "udp");
+ else if (xprt->xprt_class->ident == XPRT_TRANSPORT_LOCAL)
+ snprintf(type, sizeof(type), "local");
+ else if (xprt->xprt_class->ident == XPRT_TRANSPORT_BC_TCP)
+ snprintf(type, sizeof(type), "bc");
+ if (kobject_init_and_add(&p->kobject,
+ &rpc_sysfs_xprt_switch_xprt_type,
+ parent, "xprt-%d-%s", xprt->id,
+ type) == 0)
+ return p;
+ kobject_put(&p->kobject);
+ }
+ return NULL;
+}
+
void rpc_sysfs_client_setup(struct rpc_clnt *clnt,
struct rpc_xprt_switch *xprt_switch,
struct net *net)
@@ -197,6 +250,23 @@ void rpc_sysfs_xprt_switch_setup(struct rpc_xprt_switch *xprt_switch,
}
}
+void rpc_sysfs_xprt_switch_xprt_setup(struct rpc_xprt_switch *xprt_switch,
+ struct rpc_xprt *xprt)
+{
+ struct rpc_sysfs_xprt_switch_xprt *rpc_xprt_switch_xprt;
+ struct rpc_sysfs_xprt_switch *switch_obj =
+ (struct rpc_sysfs_xprt_switch *)xprt_switch->xps_sysfs;
+
+ rpc_xprt_switch_xprt =
+ rpc_sysfs_xprt_switch_xprt_alloc(&switch_obj->kobject,
+ xprt, xprt->xprt_net);
+ if (rpc_xprt_switch_xprt) {
+ xprt->xprt_sysfs = rpc_xprt_switch_xprt;
+ rpc_xprt_switch_xprt->xprt = xprt;
+ kobject_uevent(&rpc_xprt_switch_xprt->kobject, KOBJ_ADD);
+ }
+}
+
void rpc_sysfs_client_destroy(struct rpc_clnt *clnt)
{
struct rpc_sysfs_client *rpc_client = clnt->cl_sysfs;
@@ -225,3 +295,16 @@ void rpc_sysfs_xprt_switch_destroy(struct rpc_xprt_switch *xprt_switch)
xprt_switch->xps_sysfs = NULL;
}
}
+
+void rpc_sysfs_xprt_switch_xprt_destroy(struct rpc_xprt *xprt)
+{
+ struct rpc_sysfs_xprt_switch_xprt *rpc_xprt_switch_xprt =
+ xprt->xprt_sysfs;
+
+ if (rpc_xprt_switch_xprt) {
+ kobject_uevent(&rpc_xprt_switch_xprt->kobject, KOBJ_REMOVE);
+ kobject_del(&rpc_xprt_switch_xprt->kobject);
+ kobject_put(&rpc_xprt_switch_xprt->kobject);
+ xprt->xprt_sysfs = NULL;
+ }
+}
diff --git a/net/sunrpc/sysfs.h b/net/sunrpc/sysfs.h
index 9a0625b1cd65..52abe443ee8d 100644
--- a/net/sunrpc/sysfs.h
+++ b/net/sunrpc/sysfs.h
@@ -19,6 +19,12 @@ struct rpc_sysfs_xprt_switch {
struct rpc_xprt *xprt;
};
+struct rpc_sysfs_xprt_switch_xprt {
+ struct kobject kobject;
+ struct net *net;
+ struct rpc_xprt *xprt;
+};
+
int rpc_sysfs_init(void);
void rpc_sysfs_exit(void);
@@ -29,5 +35,8 @@ void rpc_sysfs_client_destroy(struct rpc_clnt *clnt);
void rpc_sysfs_xprt_switch_setup(struct rpc_xprt_switch *xprt_switch,
struct rpc_xprt *xprt);
void rpc_sysfs_xprt_switch_destroy(struct rpc_xprt_switch *xprt);
+void rpc_sysfs_xprt_switch_xprt_setup(struct rpc_xprt_switch *xprt_switch,
+ struct rpc_xprt *xprt);
+void rpc_sysfs_xprt_switch_xprt_destroy(struct rpc_xprt *xprt);
#endif
diff --git a/net/sunrpc/xprtmultipath.c b/net/sunrpc/xprtmultipath.c
index 1ed16e4cc465..eba45cbf8448 100644
--- a/net/sunrpc/xprtmultipath.c
+++ b/net/sunrpc/xprtmultipath.c
@@ -33,6 +33,7 @@ static void xprt_switch_add_xprt_locked(struct rpc_xprt_switch *xps,
{
if (unlikely(xprt_get(xprt) == NULL))
return;
+ rpc_sysfs_xprt_switch_xprt_setup(xps, xprt);
list_add_tail_rcu(&xprt->xprt_switch, &xps->xps_xprt_list);
smp_wmb();
if (xps->xps_nxprts == 0)
@@ -66,6 +67,7 @@ static void xprt_switch_remove_xprt_locked(struct rpc_xprt_switch *xps,
return;
xps->xps_nactive--;
xps->xps_nxprts--;
+ rpc_sysfs_xprt_switch_xprt_destroy(xprt);
if (xps->xps_nxprts == 0)
xps->xps_net = NULL;
smp_wmb();
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 11/13] sunrpc: add dst_attr attributes to the sysfs xprt directory
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
` (9 preceding siblings ...)
2021-04-16 3:52 ` [PATCH v2 10/13] sunrpc: add add sysfs directory per xprt under each xprt_switch Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 12/13] sunrpc: provide transport info in the sysfs directory Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 13/13] sunrpc: provide multipath " Olga Kornievskaia
12 siblings, 0 replies; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Olga Kornievskaia <kolga@netapp.com>
Allow to query and set the destination's address of a transport.
Setting of the destination address is allowed only for TCP or RDMA
based connections.
Signed-off-by: Olga Kornievskaia <kolga@netapp.com>
---
net/sunrpc/sysfs.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 72 insertions(+)
diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c
index 5410d8fe1181..20c622c3330e 100644
--- a/net/sunrpc/sysfs.c
+++ b/net/sunrpc/sysfs.c
@@ -4,6 +4,9 @@
*/
#include <linux/sunrpc/clnt.h>
#include <linux/kobject.h>
+#include <linux/sunrpc/addr.h>
+#include <linux/sunrpc/xprtsock.h>
+
#include "sysfs.h"
static struct kset *rpc_sunrpc_kset;
@@ -42,6 +45,66 @@ static struct kobject *rpc_sysfs_object_alloc(const char *name,
return NULL;
}
+static inline struct rpc_xprt *
+rpc_sysfs_xprt_kobj_get_xprt(struct kobject *kobj)
+{
+ struct rpc_sysfs_xprt_switch_xprt *x = container_of(kobj,
+ struct rpc_sysfs_xprt_switch_xprt, kobject);
+
+ return xprt_get(x->xprt);
+}
+
+static ssize_t rpc_sysfs_xprt_dstaddr_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf)
+{
+ struct rpc_xprt *xprt = rpc_sysfs_xprt_kobj_get_xprt(kobj);
+ ssize_t ret;
+
+ if (!xprt)
+ return 0;
+ if (xprt->xprt_class->ident == XPRT_TRANSPORT_LOCAL)
+ ret = sprintf(buf, "localhost");
+ else
+ ret = rpc_ntop((struct sockaddr *)&xprt->addr, buf, PAGE_SIZE);
+ buf[ret] = '\n';
+ xprt_put(xprt);
+ return ret + 1;
+}
+
+static ssize_t rpc_sysfs_xprt_dstaddr_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct rpc_xprt *xprt = rpc_sysfs_xprt_kobj_get_xprt(kobj);
+ struct sockaddr *saddr;
+ int port;
+
+ if (!xprt)
+ return 0;
+ if (!(xprt->xprt_class->ident == XPRT_TRANSPORT_TCP ||
+ xprt->xprt_class->ident == XPRT_TRANSPORT_RDMA)) {
+ xprt_put(xprt);
+ return -EOPNOTSUPP;
+ }
+
+ wait_on_bit_lock(&xprt->state, XPRT_LOCKED, TASK_KILLABLE);
+ saddr = (struct sockaddr *)&xprt->addr;
+ port = rpc_get_port(saddr);
+
+ kfree(xprt->address_strings[RPC_DISPLAY_ADDR]);
+ xprt->address_strings[RPC_DISPLAY_ADDR] = kstrndup(buf, count - 1,
+ GFP_KERNEL);
+ xprt->addrlen = rpc_pton(xprt->xprt_net, buf, count - 1, saddr,
+ sizeof(*saddr));
+ rpc_set_port(saddr, port);
+
+ xprt->ops->connect(xprt, NULL);
+ clear_bit(XPRT_LOCKED, &xprt->state);
+ xprt_put(xprt);
+ return count;
+}
+
int rpc_sysfs_init(void)
{
rpc_sunrpc_kset = kset_create_and_add("sunrpc", NULL, kernel_kobj);
@@ -105,6 +168,14 @@ static const void *rpc_sysfs_xprt_switch_xprt_namespace(struct kobject *kobj)
kobject)->net;
}
+static struct kobj_attribute rpc_sysfs_xprt_dstaddr = __ATTR(dstaddr,
+ 0644, rpc_sysfs_xprt_dstaddr_show, rpc_sysfs_xprt_dstaddr_store);
+
+static struct attribute *rpc_sysfs_xprt_attrs[] = {
+ &rpc_sysfs_xprt_dstaddr.attr,
+ NULL,
+};
+
static struct kobj_type rpc_sysfs_client_type = {
.release = rpc_sysfs_client_release,
.sysfs_ops = &kobj_sysfs_ops,
@@ -119,6 +190,7 @@ static struct kobj_type rpc_sysfs_xprt_switch_type = {
static struct kobj_type rpc_sysfs_xprt_switch_xprt_type = {
.release = rpc_sysfs_xprt_switch_xprt_release,
+ .default_attrs = rpc_sysfs_xprt_attrs,
.sysfs_ops = &kobj_sysfs_ops,
.namespace = rpc_sysfs_xprt_switch_xprt_namespace,
};
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 12/13] sunrpc: provide transport info in the sysfs directory
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
` (10 preceding siblings ...)
2021-04-16 3:52 ` [PATCH v2 11/13] sunrpc: add dst_attr attributes to the sysfs xprt directory Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 13/13] sunrpc: provide multipath " Olga Kornievskaia
12 siblings, 0 replies; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Olga Kornievskaia <kolga@netapp.com>
Allow to query transport's attributes. Currently showing following
fields of the rpc_xprt structure: state, last_used, cong, cwnd,
max_reqs, min_reqs, num_reqs, sizes of queues binding, sending,
pending, backlog.
Signed-off-by: Olga Kornievskaia <kolga@netapp.com>
---
net/sunrpc/sysfs.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 54 insertions(+)
diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c
index 20c622c3330e..e7a728da8e9c 100644
--- a/net/sunrpc/sysfs.c
+++ b/net/sunrpc/sysfs.c
@@ -72,6 +72,56 @@ static ssize_t rpc_sysfs_xprt_dstaddr_show(struct kobject *kobj,
return ret + 1;
}
+static ssize_t rpc_sysfs_xprt_info_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf)
+{
+ struct rpc_xprt *xprt = rpc_sysfs_xprt_kobj_get_xprt(kobj);
+ ssize_t ret;
+ int locked, connected, connecting, close_wait, bound, binding,
+ closing, congested, cwnd_wait, write_space;
+
+ if (!xprt)
+ return 0;
+
+ if (!xprt->state) {
+ ret = sprintf(buf, "state=CLOSED\n");
+ } else {
+ locked = test_bit(XPRT_LOCKED, &xprt->state);
+ connected = test_bit(XPRT_CONNECTED, &xprt->state);
+ connecting = test_bit(XPRT_CONNECTING, &xprt->state);
+ close_wait = test_bit(XPRT_CLOSE_WAIT, &xprt->state);
+ bound = test_bit(XPRT_BOUND, &xprt->state);
+ binding = test_bit(XPRT_BINDING, &xprt->state);
+ closing = test_bit(XPRT_CLOSING, &xprt->state);
+ congested = test_bit(XPRT_CONGESTED, &xprt->state);
+ cwnd_wait = test_bit(XPRT_CWND_WAIT, &xprt->state);
+ write_space = test_bit(XPRT_WRITE_SPACE, &xprt->state);
+
+ ret = sprintf(buf, "state=%s %s %s %s %s %s %s %s %s %s\n",
+ locked ? "LOCKED" : "",
+ connected ? "CONNECTED" : "",
+ connecting ? "CONNECTING" : "",
+ close_wait ? "CLOSE_WAIT" : "",
+ bound ? "BOUND" : "",
+ binding ? "BOUNDING" : "",
+ closing ? "CLOSING" : "",
+ congested ? "CONGESTED" : "",
+ cwnd_wait ? "CWND_WAIT" : "",
+ write_space ? "WRITE_SPACE" : "");
+ }
+ ret += sprintf(buf + ret, "last_used=%lu\ncur_cong=%lu\ncong_win=%lu\n"
+ "max_num_slots=%u\nmin_num_slots=%u\nnum_reqs=%u\n"
+ "binding_q_len=%u\nsending_q_len=%u\npending_q_len=%u\n"
+ "backlog_q_len=%u", xprt->last_used, xprt->cong,
+ xprt->cwnd, xprt->max_reqs, xprt->min_reqs,
+ xprt->num_reqs, xprt->binding.qlen, xprt->sending.qlen,
+ xprt->pending.qlen, xprt->backlog.qlen);
+ buf[ret] = '\n';
+ xprt_put(xprt);
+ return ret + 1;
+}
+
static ssize_t rpc_sysfs_xprt_dstaddr_store(struct kobject *kobj,
struct kobj_attribute *attr,
const char *buf, size_t count)
@@ -171,8 +221,12 @@ static const void *rpc_sysfs_xprt_switch_xprt_namespace(struct kobject *kobj)
static struct kobj_attribute rpc_sysfs_xprt_dstaddr = __ATTR(dstaddr,
0644, rpc_sysfs_xprt_dstaddr_show, rpc_sysfs_xprt_dstaddr_store);
+static struct kobj_attribute rpc_sysfs_xprt_info = __ATTR(xprt_info,
+ 0444, rpc_sysfs_xprt_info_show, NULL);
+
static struct attribute *rpc_sysfs_xprt_attrs[] = {
&rpc_sysfs_xprt_dstaddr.attr,
+ &rpc_sysfs_xprt_info.attr,
NULL,
};
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 13/13] sunrpc: provide multipath info in the sysfs directory
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
` (11 preceding siblings ...)
2021-04-16 3:52 ` [PATCH v2 12/13] sunrpc: provide transport info in the sysfs directory Olga Kornievskaia
@ 2021-04-16 3:52 ` Olga Kornievskaia
12 siblings, 0 replies; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-16 3:52 UTC (permalink / raw)
To: trond.myklebust, anna.schumaker; +Cc: linux-nfs
From: Olga Kornievskaia <kolga@netapp.com>
Allow to query xrpt_switch attributes. Currently showing the following
fields of the rpc_xprt_switch structure: xps_nxprts, xps_nactive,
xps_queuelen.
Signed-off-by: Olga Kornievskaia <kolga@netapp.com>
---
net/sunrpc/sysfs.c | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c
index e7a728da8e9c..b6a2484cf067 100644
--- a/net/sunrpc/sysfs.c
+++ b/net/sunrpc/sysfs.c
@@ -54,6 +54,19 @@ rpc_sysfs_xprt_kobj_get_xprt(struct kobject *kobj)
return xprt_get(x->xprt);
}
+static inline struct rpc_xprt_switch *
+rpc_sysfs_xprt_switch_kobj_get_xprt(struct kobject *kobj)
+{
+ struct rpc_sysfs_xprt_switch *x = container_of(kobj,
+ struct rpc_sysfs_xprt_switch, kobject);
+ struct rpc_xprt_switch *xprt_switch;
+
+ rcu_read_lock();
+ xprt_switch = xprt_switch_get(rcu_dereference(x->xprt_switch));
+ rcu_read_unlock();
+ return xprt_switch;
+}
+
static ssize_t rpc_sysfs_xprt_dstaddr_show(struct kobject *kobj,
struct kobj_attribute *attr,
char *buf)
@@ -122,6 +135,24 @@ static ssize_t rpc_sysfs_xprt_info_show(struct kobject *kobj,
return ret + 1;
}
+static ssize_t rpc_sysfs_xprt_switch_info_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf)
+{
+ struct rpc_xprt_switch *xprt_switch =
+ rpc_sysfs_xprt_switch_kobj_get_xprt(kobj);
+ ssize_t ret;
+
+ if (!xprt_switch)
+ return 0;
+ ret = sprintf(buf, "num_xprts=%u\nnum_active=%u\nqueue_len=%ld",
+ xprt_switch->xps_nxprts, xprt_switch->xps_nactive,
+ atomic_long_read(&xprt_switch->xps_queuelen));
+ buf[ret] = '\n';
+ xprt_switch_put(xprt_switch);
+ return ret + 1;
+}
+
static ssize_t rpc_sysfs_xprt_dstaddr_store(struct kobject *kobj,
struct kobj_attribute *attr,
const char *buf, size_t count)
@@ -230,6 +261,14 @@ static struct attribute *rpc_sysfs_xprt_attrs[] = {
NULL,
};
+static struct kobj_attribute rpc_sysfs_xprt_switch_info =
+ __ATTR(xprt_switch_info, 0444, rpc_sysfs_xprt_switch_info_show, NULL);
+
+static struct attribute *rpc_sysfs_xprt_switch_attrs[] = {
+ &rpc_sysfs_xprt_switch_info.attr,
+ NULL,
+};
+
static struct kobj_type rpc_sysfs_client_type = {
.release = rpc_sysfs_client_release,
.sysfs_ops = &kobj_sysfs_ops,
@@ -238,6 +277,7 @@ static struct kobj_type rpc_sysfs_client_type = {
static struct kobj_type rpc_sysfs_xprt_switch_type = {
.release = rpc_sysfs_xprt_switch_release,
+ .default_attrs = rpc_sysfs_xprt_switch_attrs,
.sysfs_ops = &kobj_sysfs_ops,
.namespace = rpc_sysfs_xprt_switch_namespace,
};
--
2.27.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH v2 06/13] sunrpc: add IDs to multipath
2021-04-16 3:52 ` [PATCH v2 06/13] sunrpc: add IDs to multipath Olga Kornievskaia
@ 2021-04-23 21:16 ` Trond Myklebust
0 siblings, 0 replies; 18+ messages in thread
From: Trond Myklebust @ 2021-04-23 21:16 UTC (permalink / raw)
To: anna.schumaker, olga.kornievskaia; +Cc: linux-nfs
On Thu, 2021-04-15 at 23:52 -0400, Olga Kornievskaia wrote:
> From: Olga Kornievskaia <kolga@netapp.com>
>
> This is used to uniquely identify sunrpc multipath objects in /sys.
>
> Signed-off-by: Dan Aloni <dan@kernelim.com>
> ---
> include/linux/sunrpc/xprtmultipath.h | 4 ++++
> net/sunrpc/sunrpc_syms.c | 1 +
> net/sunrpc/xprtmultipath.c | 26 ++++++++++++++++++++++++++
> 3 files changed, 31 insertions(+)
>
> diff --git a/include/linux/sunrpc/xprtmultipath.h
> b/include/linux/sunrpc/xprtmultipath.h
> index c6cce3fbf29d..ef95a6f18ccf 100644
> --- a/include/linux/sunrpc/xprtmultipath.h
> +++ b/include/linux/sunrpc/xprtmultipath.h
> @@ -14,6 +14,7 @@ struct rpc_xprt_switch {
> spinlock_t xps_lock;
> struct kref xps_kref;
>
> + unsigned int xps_id;
> unsigned int xps_nxprts;
> unsigned int xps_nactive;
> atomic_long_t xps_queuelen;
> @@ -71,4 +72,7 @@ extern struct rpc_xprt *xprt_iter_get_next(struct
> rpc_xprt_iter *xpi);
>
> extern bool rpc_xprt_switch_has_addr(struct rpc_xprt_switch *xps,
> const struct sockaddr *sap);
> +
> +extern void xprt_multipath_cleanup_ids(void);
> +
> #endif
> diff --git a/net/sunrpc/sunrpc_syms.c b/net/sunrpc/sunrpc_syms.c
> index b61b74c00483..691c0000e9ea 100644
> --- a/net/sunrpc/sunrpc_syms.c
> +++ b/net/sunrpc/sunrpc_syms.c
> @@ -134,6 +134,7 @@ cleanup_sunrpc(void)
> rpc_sysfs_exit();
> rpc_cleanup_clids();
> xprt_cleanup_ids();
> + xprt_multipath_cleanup_ids();
> rpcauth_remove_module();
> cleanup_socket_xprt();
> svc_cleanup_xprt_sock();
> diff --git a/net/sunrpc/xprtmultipath.c b/net/sunrpc/xprtmultipath.c
> index 78c075a68c04..b71dd95ad7de 100644
> --- a/net/sunrpc/xprtmultipath.c
> +++ b/net/sunrpc/xprtmultipath.c
> @@ -86,6 +86,30 @@ void rpc_xprt_switch_remove_xprt(struct
> rpc_xprt_switch *xps,
> xprt_put(xprt);
> }
>
> +static DEFINE_IDA(rpc_xprtswitch_ids);
> +
> +void xprt_multipath_cleanup_ids(void)
> +{
> + ida_destroy(&rpc_xprtswitch_ids);
> +}
> +
> +static int xprt_switch_alloc_id(struct rpc_xprt_switch *xps)
> +{
> + int id;
> +
> + id = ida_simple_get(&rpc_xprtswitch_ids, 0, 0, GFP_KERNEL);
This really needs to use the same allocation mode as the caller in
xprt_switch_alloc()
> + if (id < 0)
> + return id;
> +
> + xps->xps_id = id;
> + return 0;
> +}
> +
> +static void xprt_switch_free_id(struct rpc_xprt_switch *xps)
> +{
> + ida_simple_remove(&rpc_xprtswitch_ids, xps->xps_id);
> +}
> +
> /**
> * xprt_switch_alloc - Allocate a new struct rpc_xprt_switch
> * @xprt: pointer to struct rpc_xprt
> @@ -103,6 +127,7 @@ struct rpc_xprt_switch *xprt_switch_alloc(struct
> rpc_xprt *xprt,
> if (xps != NULL) {
> spin_lock_init(&xps->xps_lock);
> kref_init(&xps->xps_kref);
> + xprt_switch_alloc_id(xps);
> xps->xps_nxprts = xps->xps_nactive = 0;
> atomic_long_set(&xps->xps_queuelen, 0);
> xps->xps_net = NULL;
> @@ -136,6 +161,7 @@ static void xprt_switch_free(struct kref *kref)
> struct rpc_xprt_switch, xps_kref);
>
> xprt_switch_free_entries(xps);
> + xprt_switch_free_id(xps);
> kfree_rcu(xps, xps_rcu);
> }
>
--
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 08/13] sunrpc: add xprt_switch direcotry to sunrpc's sysfs
2021-04-16 3:52 ` [PATCH v2 08/13] sunrpc: add xprt_switch direcotry to sunrpc's sysfs Olga Kornievskaia
@ 2021-04-23 21:20 ` Trond Myklebust
2021-04-26 15:30 ` Olga Kornievskaia
0 siblings, 1 reply; 18+ messages in thread
From: Trond Myklebust @ 2021-04-23 21:20 UTC (permalink / raw)
To: anna.schumaker, olga.kornievskaia; +Cc: linux-nfs
On Thu, 2021-04-15 at 23:52 -0400, Olga Kornievskaia wrote:
> From: Olga Kornievskaia <kolga@netapp.com>
>
> Add xprt_switch directory to the sysfs and create individual
> xprt_swith subdirectories for multipath transport group.
>
> Signed-off-by: Olga Kornievskaia <kolga@netapp.com>
> ---
> include/linux/sunrpc/xprtmultipath.h | 1 +
> net/sunrpc/sysfs.c | 97
> ++++++++++++++++++++++++++--
> net/sunrpc/sysfs.h | 10 +++
> net/sunrpc/xprtmultipath.c | 4 ++
> 4 files changed, 105 insertions(+), 7 deletions(-)
>
> diff --git a/include/linux/sunrpc/xprtmultipath.h
> b/include/linux/sunrpc/xprtmultipath.h
> index ef95a6f18ccf..47b0a85cdcfa 100644
> --- a/include/linux/sunrpc/xprtmultipath.h
> +++ b/include/linux/sunrpc/xprtmultipath.h
> @@ -24,6 +24,7 @@ struct rpc_xprt_switch {
>
> const struct rpc_xprt_iter_ops *xps_iter_ops;
>
> + void *xps_sysfs;
> struct rcu_head xps_rcu;
> };
>
> diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c
> index d14d54f33c65..0c34330714ab 100644
> --- a/net/sunrpc/sysfs.c
> +++ b/net/sunrpc/sysfs.c
> @@ -7,7 +7,7 @@
> #include "sysfs.h"
>
> static struct kset *rpc_sunrpc_kset;
> -static struct kobject *rpc_sunrpc_client_kobj;
> +static struct kobject *rpc_sunrpc_client_kobj,
> *rpc_sunrpc_xprt_switch_kobj;
>
> static void rpc_sysfs_object_release(struct kobject *kobj)
> {
> @@ -47,13 +47,22 @@ int rpc_sysfs_init(void)
> rpc_sunrpc_kset = kset_create_and_add("sunrpc", NULL,
> kernel_kobj);
> if (!rpc_sunrpc_kset)
> return -ENOMEM;
> - rpc_sunrpc_client_kobj = rpc_sysfs_object_alloc("client",
> rpc_sunrpc_kset, NULL);
> - if (!rpc_sunrpc_client_kobj) {
> - kset_unregister(rpc_sunrpc_kset);
> - rpc_sunrpc_client_kobj = NULL;
> - return -ENOMEM;
> - }
> + rpc_sunrpc_client_kobj =
> + rpc_sysfs_object_alloc("rpc-clients",
> rpc_sunrpc_kset, NULL);
> + if (!rpc_sunrpc_client_kobj)
> + goto err_client;
> + rpc_sunrpc_xprt_switch_kobj =
> + rpc_sysfs_object_alloc("xprt-switches",
> rpc_sunrpc_kset, NULL);
> + if (!rpc_sunrpc_xprt_switch_kobj)
> + goto err_switch;
> return 0;
> +err_switch:
> + kobject_put(rpc_sunrpc_client_kobj);
> + rpc_sunrpc_client_kobj = NULL;
> +err_client:
> + kset_unregister(rpc_sunrpc_kset);
> + rpc_sunrpc_kset = NULL;
> + return -ENOMEM;
> }
>
> static void rpc_sysfs_client_release(struct kobject *kobj)
> @@ -64,20 +73,40 @@ static void rpc_sysfs_client_release(struct
> kobject *kobj)
> kfree(c);
> }
>
> +static void rpc_sysfs_xprt_switch_release(struct kobject *kobj)
> +{
> + struct rpc_sysfs_xprt_switch *xprt_switch;
> +
> + xprt_switch = container_of(kobj, struct
> rpc_sysfs_xprt_switch, kobject);
> + kfree(xprt_switch);
> +}
> +
> static const void *rpc_sysfs_client_namespace(struct kobject *kobj)
> {
> return container_of(kobj, struct rpc_sysfs_client, kobject)-
> >net;
> }
>
> +static const void *rpc_sysfs_xprt_switch_namespace(struct kobject
> *kobj)
> +{
> + return container_of(kobj, struct rpc_sysfs_xprt_switch,
> kobject)->net;
> +}
> +
> static struct kobj_type rpc_sysfs_client_type = {
> .release = rpc_sysfs_client_release,
> .sysfs_ops = &kobj_sysfs_ops,
> .namespace = rpc_sysfs_client_namespace,
> };
>
> +static struct kobj_type rpc_sysfs_xprt_switch_type = {
> + .release = rpc_sysfs_xprt_switch_release,
> + .sysfs_ops = &kobj_sysfs_ops,
> + .namespace = rpc_sysfs_xprt_switch_namespace,
> +};
> +
> void rpc_sysfs_exit(void)
> {
> kobject_put(rpc_sunrpc_client_kobj);
> + kobject_put(rpc_sunrpc_xprt_switch_kobj);
> kset_unregister(rpc_sunrpc_kset);
> }
>
> @@ -99,6 +128,27 @@ static struct rpc_sysfs_client
> *rpc_sysfs_client_alloc(struct kobject *parent,
> return NULL;
> }
>
> +static struct rpc_sysfs_xprt_switch *
> +rpc_sysfs_xprt_switch_alloc(struct kobject *parent,
> + struct rpc_xprt_switch *xprt_switch,
> + struct net *net)
> +{
> + struct rpc_sysfs_xprt_switch *p;
> +
> + p = kzalloc(sizeof(*p), GFP_KERNEL);
Again, this needs to use the allocation mode of xprt_switch_alloc().
> + if (p) {
> + p->net = net;
> + p->kobject.kset = rpc_sunrpc_kset;
> + if (kobject_init_and_add(&p->kobject,
> + &rpc_sysfs_xprt_switch_type,
> + parent, "switch-%d",
> + xprt_switch->xps_id) == 0)
> + return p;
> + kobject_put(&p->kobject);
> + }
> + return NULL;
> +}
> +
> void rpc_sysfs_client_setup(struct rpc_clnt *clnt, struct net *net)
> {
> struct rpc_sysfs_client *rpc_client;
> @@ -110,6 +160,27 @@ void rpc_sysfs_client_setup(struct rpc_clnt
> *clnt, struct net *net)
> }
> }
>
> +void rpc_sysfs_xprt_switch_setup(struct rpc_xprt_switch
> *xprt_switch,
> + struct rpc_xprt *xprt)
> +{
> + struct rpc_sysfs_xprt_switch *rpc_xprt_switch;
> + struct net *net;
> +
> + if (xprt_switch->xps_net)
> + net = xprt_switch->xps_net;
> + else
> + net = xprt->xprt_net;
> + rpc_xprt_switch =
> + rpc_sysfs_xprt_switch_alloc(rpc_sunrpc_xprt_switch_ko
> bj,
> + xprt_switch, net);
> + if (rpc_xprt_switch) {
> + xprt_switch->xps_sysfs = rpc_xprt_switch;
> + rpc_xprt_switch->xprt_switch = xprt_switch;
> + rpc_xprt_switch->xprt = xprt;
> + kobject_uevent(&rpc_xprt_switch->kobject, KOBJ_ADD);
This probably cannot be called from a locked environment.
> + }
> +}
> +
> void rpc_sysfs_client_destroy(struct rpc_clnt *clnt)
> {
> struct rpc_sysfs_client *rpc_client = clnt->cl_sysfs;
> @@ -121,3 +192,15 @@ void rpc_sysfs_client_destroy(struct rpc_clnt
> *clnt)
> clnt->cl_sysfs = NULL;
> }
> }
> +
> +void rpc_sysfs_xprt_switch_destroy(struct rpc_xprt_switch
> *xprt_switch)
> +{
> + struct rpc_sysfs_xprt_switch *rpc_xprt_switch = xprt_switch-
> >xps_sysfs;
> +
> + if (rpc_xprt_switch) {
> + kobject_uevent(&rpc_xprt_switch->kobject,
> KOBJ_REMOVE);
> + kobject_del(&rpc_xprt_switch->kobject);
> + kobject_put(&rpc_xprt_switch->kobject);
> + xprt_switch->xps_sysfs = NULL;
> + }
> +}
> diff --git a/net/sunrpc/sysfs.h b/net/sunrpc/sysfs.h
> index c46afc848993..9b6acd3fd3dc 100644
> --- a/net/sunrpc/sysfs.h
> +++ b/net/sunrpc/sysfs.h
> @@ -10,10 +10,20 @@ struct rpc_sysfs_client {
> struct net *net;
> };
>
> +struct rpc_sysfs_xprt_switch {
> + struct kobject kobject;
> + struct net *net;
> + struct rpc_xprt_switch *xprt_switch;
> + struct rpc_xprt *xprt;
> +};
> +
> int rpc_sysfs_init(void);
> void rpc_sysfs_exit(void);
>
> void rpc_sysfs_client_setup(struct rpc_clnt *clnt, struct net *net);
> void rpc_sysfs_client_destroy(struct rpc_clnt *clnt);
> +void rpc_sysfs_xprt_switch_setup(struct rpc_xprt_switch
> *xprt_switch,
> + struct rpc_xprt *xprt);
> +void rpc_sysfs_xprt_switch_destroy(struct rpc_xprt_switch *xprt);
>
> #endif
> diff --git a/net/sunrpc/xprtmultipath.c b/net/sunrpc/xprtmultipath.c
> index b71dd95ad7de..1ed16e4cc465 100644
> --- a/net/sunrpc/xprtmultipath.c
> +++ b/net/sunrpc/xprtmultipath.c
> @@ -19,6 +19,8 @@
> #include <linux/sunrpc/addr.h>
> #include <linux/sunrpc/xprtmultipath.h>
>
> +#include "sysfs.h"
> +
> typedef struct rpc_xprt *(*xprt_switch_find_xprt_t)(struct
> rpc_xprt_switch *xps,
> const struct rpc_xprt *cur);
>
> @@ -133,6 +135,7 @@ struct rpc_xprt_switch *xprt_switch_alloc(struct
> rpc_xprt *xprt,
> xps->xps_net = NULL;
> INIT_LIST_HEAD(&xps->xps_xprt_list);
> xps->xps_iter_ops = &rpc_xprt_iter_singular;
> + rpc_sysfs_xprt_switch_setup(xps, xprt);
> xprt_switch_add_xprt_locked(xps, xprt);
> }
>
> @@ -161,6 +164,7 @@ static void xprt_switch_free(struct kref *kref)
> struct rpc_xprt_switch, xps_kref);
>
> xprt_switch_free_entries(xps);
> + rpc_sysfs_xprt_switch_destroy(xps);
> xprt_switch_free_id(xps);
> kfree_rcu(xps, xps_rcu);
> }
--
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 10/13] sunrpc: add add sysfs directory per xprt under each xprt_switch
2021-04-16 3:52 ` [PATCH v2 10/13] sunrpc: add add sysfs directory per xprt under each xprt_switch Olga Kornievskaia
@ 2021-04-23 21:21 ` Trond Myklebust
0 siblings, 0 replies; 18+ messages in thread
From: Trond Myklebust @ 2021-04-23 21:21 UTC (permalink / raw)
To: anna.schumaker, olga.kornievskaia; +Cc: linux-nfs
On Thu, 2021-04-15 at 23:52 -0400, Olga Kornievskaia wrote:
> From: Olga Kornievskaia <kolga@netapp.com>
>
> Add individual transport directories under each transport switch
> group. For instance, for each nconnect=X connections there will be
> a transport directory. Naming conventions also identifies transport
> type -- xprt-<id>-<type> where type is udp, tcp, rdma, local, bc.
>
> Signed-off-by: Olga Kornievskaia <kolga@netapp.com>
> ---
> include/linux/sunrpc/xprt.h | 1 +
> net/sunrpc/sysfs.c | 83
> +++++++++++++++++++++++++++++++++++++
> net/sunrpc/sysfs.h | 9 ++++
> net/sunrpc/xprtmultipath.c | 2 +
> 4 files changed, 95 insertions(+)
>
> diff --git a/include/linux/sunrpc/xprt.h
> b/include/linux/sunrpc/xprt.h
> index a2edcc42e6c4..1e4906759a6a 100644
> --- a/include/linux/sunrpc/xprt.h
> +++ b/include/linux/sunrpc/xprt.h
> @@ -291,6 +291,7 @@ struct rpc_xprt {
> #endif
> struct rcu_head rcu;
> const struct xprt_class *xprt_class;
> + void *xprt_sysfs;
> };
>
> #if defined(CONFIG_SUNRPC_BACKCHANNEL)
> diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c
> index ce2cad1b6aa6..5410d8fe1181 100644
> --- a/net/sunrpc/sysfs.c
> +++ b/net/sunrpc/sysfs.c
> @@ -81,6 +81,14 @@ static void rpc_sysfs_xprt_switch_release(struct
> kobject *kobj)
> kfree(xprt_switch);
> }
>
> +static void rpc_sysfs_xprt_switch_xprt_release(struct kobject *kobj)
> +{
> + struct rpc_sysfs_xprt_switch_xprt *xprt;
> +
> + xprt = container_of(kobj, struct rpc_sysfs_xprt_switch_xprt,
> kobject);
> + kfree(xprt);
> +}
> +
> static const void *rpc_sysfs_client_namespace(struct kobject *kobj)
> {
> return container_of(kobj, struct rpc_sysfs_client, kobject)-
> >net;
> @@ -91,6 +99,12 @@ static const void
> *rpc_sysfs_xprt_switch_namespace(struct kobject *kobj)
> return container_of(kobj, struct rpc_sysfs_xprt_switch,
> kobject)->net;
> }
>
> +static const void *rpc_sysfs_xprt_switch_xprt_namespace(struct
> kobject *kobj)
> +{
> + return container_of(kobj, struct rpc_sysfs_xprt_switch_xprt,
> + kobject)->net;
> +}
> +
> static struct kobj_type rpc_sysfs_client_type = {
> .release = rpc_sysfs_client_release,
> .sysfs_ops = &kobj_sysfs_ops,
> @@ -103,6 +117,12 @@ static struct kobj_type
> rpc_sysfs_xprt_switch_type = {
> .namespace = rpc_sysfs_xprt_switch_namespace,
> };
>
> +static struct kobj_type rpc_sysfs_xprt_switch_xprt_type = {
> + .release = rpc_sysfs_xprt_switch_xprt_release,
> + .sysfs_ops = &kobj_sysfs_ops,
> + .namespace = rpc_sysfs_xprt_switch_xprt_namespace,
> +};
> +
> void rpc_sysfs_exit(void)
> {
> kobject_put(rpc_sunrpc_client_kobj);
> @@ -149,6 +169,39 @@ rpc_sysfs_xprt_switch_alloc(struct kobject
> *parent,
> return NULL;
> }
>
> +static struct rpc_sysfs_xprt_switch_xprt *
> +rpc_sysfs_xprt_switch_xprt_alloc(struct kobject *parent,
> + struct rpc_xprt *xprt,
> + struct net *net)
> +{
> + struct rpc_sysfs_xprt_switch_xprt *p;
> +
> + p = kzalloc(sizeof(*p), GFP_KERNEL);
Ditto. This cannot be called from a locked environment.
> + if (p) {
> + char type[6];
> +
> + p->net = net;
> + p->kobject.kset = rpc_sunrpc_kset;
> + if (xprt->xprt_class->ident == XPRT_TRANSPORT_RDMA)
> + snprintf(type, sizeof(type), "rdma");
> + else if (xprt->xprt_class->ident ==
> XPRT_TRANSPORT_TCP)
> + snprintf(type, sizeof(type), "tcp");
> + else if (xprt->xprt_class->ident ==
> XPRT_TRANSPORT_UDP)
> + snprintf(type, sizeof(type), "udp");
> + else if (xprt->xprt_class->ident ==
> XPRT_TRANSPORT_LOCAL)
> + snprintf(type, sizeof(type), "local");
> + else if (xprt->xprt_class->ident ==
> XPRT_TRANSPORT_BC_TCP)
> + snprintf(type, sizeof(type), "bc");
> + if (kobject_init_and_add(&p->kobject,
> +
> &rpc_sysfs_xprt_switch_xprt_type,
> + parent, "xprt-%d-%s", xprt-
> >id,
> + type) == 0)
> + return p;
> + kobject_put(&p->kobject);
> + }
> + return NULL;
> +}
> +
> void rpc_sysfs_client_setup(struct rpc_clnt *clnt,
> struct rpc_xprt_switch *xprt_switch,
> struct net *net)
> @@ -197,6 +250,23 @@ void rpc_sysfs_xprt_switch_setup(struct
> rpc_xprt_switch *xprt_switch,
> }
> }
>
> +void rpc_sysfs_xprt_switch_xprt_setup(struct rpc_xprt_switch
> *xprt_switch,
> + struct rpc_xprt *xprt)
> +{
> + struct rpc_sysfs_xprt_switch_xprt *rpc_xprt_switch_xprt;
> + struct rpc_sysfs_xprt_switch *switch_obj =
> + (struct rpc_sysfs_xprt_switch *)xprt_switch-
> >xps_sysfs;
> +
> + rpc_xprt_switch_xprt =
> + rpc_sysfs_xprt_switch_xprt_alloc(&switch_obj-
> >kobject,
> + xprt, xprt-
> >xprt_net);
> + if (rpc_xprt_switch_xprt) {
> + xprt->xprt_sysfs = rpc_xprt_switch_xprt;
> + rpc_xprt_switch_xprt->xprt = xprt;
> + kobject_uevent(&rpc_xprt_switch_xprt->kobject,
> KOBJ_ADD);
> + }
> +}
> +
> void rpc_sysfs_client_destroy(struct rpc_clnt *clnt)
> {
> struct rpc_sysfs_client *rpc_client = clnt->cl_sysfs;
> @@ -225,3 +295,16 @@ void rpc_sysfs_xprt_switch_destroy(struct
> rpc_xprt_switch *xprt_switch)
> xprt_switch->xps_sysfs = NULL;
> }
> }
> +
> +void rpc_sysfs_xprt_switch_xprt_destroy(struct rpc_xprt *xprt)
> +{
> + struct rpc_sysfs_xprt_switch_xprt *rpc_xprt_switch_xprt =
> + xprt->xprt_sysfs;
> +
> + if (rpc_xprt_switch_xprt) {
> + kobject_uevent(&rpc_xprt_switch_xprt->kobject,
> KOBJ_REMOVE);
> + kobject_del(&rpc_xprt_switch_xprt->kobject);
> + kobject_put(&rpc_xprt_switch_xprt->kobject);
> + xprt->xprt_sysfs = NULL;
> + }
> +}
> diff --git a/net/sunrpc/sysfs.h b/net/sunrpc/sysfs.h
> index 9a0625b1cd65..52abe443ee8d 100644
> --- a/net/sunrpc/sysfs.h
> +++ b/net/sunrpc/sysfs.h
> @@ -19,6 +19,12 @@ struct rpc_sysfs_xprt_switch {
> struct rpc_xprt *xprt;
> };
>
> +struct rpc_sysfs_xprt_switch_xprt {
> + struct kobject kobject;
> + struct net *net;
> + struct rpc_xprt *xprt;
> +};
> +
> int rpc_sysfs_init(void);
> void rpc_sysfs_exit(void);
>
> @@ -29,5 +35,8 @@ void rpc_sysfs_client_destroy(struct rpc_clnt
> *clnt);
> void rpc_sysfs_xprt_switch_setup(struct rpc_xprt_switch
> *xprt_switch,
> struct rpc_xprt *xprt);
> void rpc_sysfs_xprt_switch_destroy(struct rpc_xprt_switch *xprt);
> +void rpc_sysfs_xprt_switch_xprt_setup(struct rpc_xprt_switch
> *xprt_switch,
> + struct rpc_xprt *xprt);
> +void rpc_sysfs_xprt_switch_xprt_destroy(struct rpc_xprt *xprt);
>
> #endif
> diff --git a/net/sunrpc/xprtmultipath.c b/net/sunrpc/xprtmultipath.c
> index 1ed16e4cc465..eba45cbf8448 100644
> --- a/net/sunrpc/xprtmultipath.c
> +++ b/net/sunrpc/xprtmultipath.c
> @@ -33,6 +33,7 @@ static void xprt_switch_add_xprt_locked(struct
> rpc_xprt_switch *xps,
> {
> if (unlikely(xprt_get(xprt) == NULL))
> return;
> + rpc_sysfs_xprt_switch_xprt_setup(xps, xprt);
> list_add_tail_rcu(&xprt->xprt_switch, &xps->xps_xprt_list);
> smp_wmb();
> if (xps->xps_nxprts == 0)
> @@ -66,6 +67,7 @@ static void xprt_switch_remove_xprt_locked(struct
> rpc_xprt_switch *xps,
> return;
> xps->xps_nactive--;
> xps->xps_nxprts--;
> + rpc_sysfs_xprt_switch_xprt_destroy(xprt);
> if (xps->xps_nxprts == 0)
> xps->xps_net = NULL;
> smp_wmb();
--
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 08/13] sunrpc: add xprt_switch direcotry to sunrpc's sysfs
2021-04-23 21:20 ` Trond Myklebust
@ 2021-04-26 15:30 ` Olga Kornievskaia
0 siblings, 0 replies; 18+ messages in thread
From: Olga Kornievskaia @ 2021-04-26 15:30 UTC (permalink / raw)
To: Trond Myklebust; +Cc: anna.schumaker, linux-nfs
On Fri, Apr 23, 2021 at 5:20 PM Trond Myklebust <trondmy@hammerspace.com> wrote:
>
> On Thu, 2021-04-15 at 23:52 -0400, Olga Kornievskaia wrote:
> > From: Olga Kornievskaia <kolga@netapp.com>
> >
> > Add xprt_switch directory to the sysfs and create individual
> > xprt_swith subdirectories for multipath transport group.
> >
> > Signed-off-by: Olga Kornievskaia <kolga@netapp.com>
> > ---
> > include/linux/sunrpc/xprtmultipath.h | 1 +
> > net/sunrpc/sysfs.c | 97
> > ++++++++++++++++++++++++++--
> > net/sunrpc/sysfs.h | 10 +++
> > net/sunrpc/xprtmultipath.c | 4 ++
> > 4 files changed, 105 insertions(+), 7 deletions(-)
> >
> > diff --git a/include/linux/sunrpc/xprtmultipath.h
> > b/include/linux/sunrpc/xprtmultipath.h
> > index ef95a6f18ccf..47b0a85cdcfa 100644
> > --- a/include/linux/sunrpc/xprtmultipath.h
> > +++ b/include/linux/sunrpc/xprtmultipath.h
> > @@ -24,6 +24,7 @@ struct rpc_xprt_switch {
> >
> > const struct rpc_xprt_iter_ops *xps_iter_ops;
> >
> > + void *xps_sysfs;
> > struct rcu_head xps_rcu;
> > };
> >
> > diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c
> > index d14d54f33c65..0c34330714ab 100644
> > --- a/net/sunrpc/sysfs.c
> > +++ b/net/sunrpc/sysfs.c
> > @@ -7,7 +7,7 @@
> > #include "sysfs.h"
> >
> > static struct kset *rpc_sunrpc_kset;
> > -static struct kobject *rpc_sunrpc_client_kobj;
> > +static struct kobject *rpc_sunrpc_client_kobj,
> > *rpc_sunrpc_xprt_switch_kobj;
> >
> > static void rpc_sysfs_object_release(struct kobject *kobj)
> > {
> > @@ -47,13 +47,22 @@ int rpc_sysfs_init(void)
> > rpc_sunrpc_kset = kset_create_and_add("sunrpc", NULL,
> > kernel_kobj);
> > if (!rpc_sunrpc_kset)
> > return -ENOMEM;
> > - rpc_sunrpc_client_kobj = rpc_sysfs_object_alloc("client",
> > rpc_sunrpc_kset, NULL);
> > - if (!rpc_sunrpc_client_kobj) {
> > - kset_unregister(rpc_sunrpc_kset);
> > - rpc_sunrpc_client_kobj = NULL;
> > - return -ENOMEM;
> > - }
> > + rpc_sunrpc_client_kobj =
> > + rpc_sysfs_object_alloc("rpc-clients",
> > rpc_sunrpc_kset, NULL);
> > + if (!rpc_sunrpc_client_kobj)
> > + goto err_client;
> > + rpc_sunrpc_xprt_switch_kobj =
> > + rpc_sysfs_object_alloc("xprt-switches",
> > rpc_sunrpc_kset, NULL);
> > + if (!rpc_sunrpc_xprt_switch_kobj)
> > + goto err_switch;
> > return 0;
> > +err_switch:
> > + kobject_put(rpc_sunrpc_client_kobj);
> > + rpc_sunrpc_client_kobj = NULL;
> > +err_client:
> > + kset_unregister(rpc_sunrpc_kset);
> > + rpc_sunrpc_kset = NULL;
> > + return -ENOMEM;
> > }
> >
> > static void rpc_sysfs_client_release(struct kobject *kobj)
> > @@ -64,20 +73,40 @@ static void rpc_sysfs_client_release(struct
> > kobject *kobj)
> > kfree(c);
> > }
> >
> > +static void rpc_sysfs_xprt_switch_release(struct kobject *kobj)
> > +{
> > + struct rpc_sysfs_xprt_switch *xprt_switch;
> > +
> > + xprt_switch = container_of(kobj, struct
> > rpc_sysfs_xprt_switch, kobject);
> > + kfree(xprt_switch);
> > +}
> > +
> > static const void *rpc_sysfs_client_namespace(struct kobject *kobj)
> > {
> > return container_of(kobj, struct rpc_sysfs_client, kobject)-
> > >net;
> > }
> >
> > +static const void *rpc_sysfs_xprt_switch_namespace(struct kobject
> > *kobj)
> > +{
> > + return container_of(kobj, struct rpc_sysfs_xprt_switch,
> > kobject)->net;
> > +}
> > +
> > static struct kobj_type rpc_sysfs_client_type = {
> > .release = rpc_sysfs_client_release,
> > .sysfs_ops = &kobj_sysfs_ops,
> > .namespace = rpc_sysfs_client_namespace,
> > };
> >
> > +static struct kobj_type rpc_sysfs_xprt_switch_type = {
> > + .release = rpc_sysfs_xprt_switch_release,
> > + .sysfs_ops = &kobj_sysfs_ops,
> > + .namespace = rpc_sysfs_xprt_switch_namespace,
> > +};
> > +
> > void rpc_sysfs_exit(void)
> > {
> > kobject_put(rpc_sunrpc_client_kobj);
> > + kobject_put(rpc_sunrpc_xprt_switch_kobj);
> > kset_unregister(rpc_sunrpc_kset);
> > }
> >
> > @@ -99,6 +128,27 @@ static struct rpc_sysfs_client
> > *rpc_sysfs_client_alloc(struct kobject *parent,
> > return NULL;
> > }
> >
> > +static struct rpc_sysfs_xprt_switch *
> > +rpc_sysfs_xprt_switch_alloc(struct kobject *parent,
> > + struct rpc_xprt_switch *xprt_switch,
> > + struct net *net)
> > +{
> > + struct rpc_sysfs_xprt_switch *p;
> > +
> > + p = kzalloc(sizeof(*p), GFP_KERNEL);
>
> Again, this needs to use the allocation mode of xprt_switch_alloc().
>
> > + if (p) {
> > + p->net = net;
> > + p->kobject.kset = rpc_sunrpc_kset;
> > + if (kobject_init_and_add(&p->kobject,
> > + &rpc_sysfs_xprt_switch_type,
> > + parent, "switch-%d",
> > + xprt_switch->xps_id) == 0)
> > + return p;
> > + kobject_put(&p->kobject);
> > + }
> > + return NULL;
> > +}
> > +
> > void rpc_sysfs_client_setup(struct rpc_clnt *clnt, struct net *net)
> > {
> > struct rpc_sysfs_client *rpc_client;
> > @@ -110,6 +160,27 @@ void rpc_sysfs_client_setup(struct rpc_clnt
> > *clnt, struct net *net)
> > }
> > }
> >
> > +void rpc_sysfs_xprt_switch_setup(struct rpc_xprt_switch
> > *xprt_switch,
> > + struct rpc_xprt *xprt)
> > +{
> > + struct rpc_sysfs_xprt_switch *rpc_xprt_switch;
> > + struct net *net;
> > +
> > + if (xprt_switch->xps_net)
> > + net = xprt_switch->xps_net;
> > + else
> > + net = xprt->xprt_net;
> > + rpc_xprt_switch =
> > + rpc_sysfs_xprt_switch_alloc(rpc_sunrpc_xprt_switch_ko
> > bj,
> > + xprt_switch, net);
> > + if (rpc_xprt_switch) {
> > + xprt_switch->xps_sysfs = rpc_xprt_switch;
> > + rpc_xprt_switch->xprt_switch = xprt_switch;
> > + rpc_xprt_switch->xprt = xprt;
> > + kobject_uevent(&rpc_xprt_switch->kobject, KOBJ_ADD);
>
> This probably cannot be called from a locked environment.
rpc_sysfs_xprt_switch_setup() isn't called from a locked environment.
Going backwards:
rpc_sysfs_xprt_switch_setup() from xprt_switch_alloc() from
rpc_create_xprt()/rpc_switch_client_transport() none of those are
called with a lock.
>
> > + }
> > +}
> > +
> > void rpc_sysfs_client_destroy(struct rpc_clnt *clnt)
> > {
> > struct rpc_sysfs_client *rpc_client = clnt->cl_sysfs;
> > @@ -121,3 +192,15 @@ void rpc_sysfs_client_destroy(struct rpc_clnt
> > *clnt)
> > clnt->cl_sysfs = NULL;
> > }
> > }
> > +
> > +void rpc_sysfs_xprt_switch_destroy(struct rpc_xprt_switch
> > *xprt_switch)
> > +{
> > + struct rpc_sysfs_xprt_switch *rpc_xprt_switch = xprt_switch-
> > >xps_sysfs;
> > +
> > + if (rpc_xprt_switch) {
> > + kobject_uevent(&rpc_xprt_switch->kobject,
> > KOBJ_REMOVE);
> > + kobject_del(&rpc_xprt_switch->kobject);
> > + kobject_put(&rpc_xprt_switch->kobject);
> > + xprt_switch->xps_sysfs = NULL;
> > + }
> > +}
> > diff --git a/net/sunrpc/sysfs.h b/net/sunrpc/sysfs.h
> > index c46afc848993..9b6acd3fd3dc 100644
> > --- a/net/sunrpc/sysfs.h
> > +++ b/net/sunrpc/sysfs.h
> > @@ -10,10 +10,20 @@ struct rpc_sysfs_client {
> > struct net *net;
> > };
> >
> > +struct rpc_sysfs_xprt_switch {
> > + struct kobject kobject;
> > + struct net *net;
> > + struct rpc_xprt_switch *xprt_switch;
> > + struct rpc_xprt *xprt;
> > +};
> > +
> > int rpc_sysfs_init(void);
> > void rpc_sysfs_exit(void);
> >
> > void rpc_sysfs_client_setup(struct rpc_clnt *clnt, struct net *net);
> > void rpc_sysfs_client_destroy(struct rpc_clnt *clnt);
> > +void rpc_sysfs_xprt_switch_setup(struct rpc_xprt_switch
> > *xprt_switch,
> > + struct rpc_xprt *xprt);
> > +void rpc_sysfs_xprt_switch_destroy(struct rpc_xprt_switch *xprt);
> >
> > #endif
> > diff --git a/net/sunrpc/xprtmultipath.c b/net/sunrpc/xprtmultipath.c
> > index b71dd95ad7de..1ed16e4cc465 100644
> > --- a/net/sunrpc/xprtmultipath.c
> > +++ b/net/sunrpc/xprtmultipath.c
> > @@ -19,6 +19,8 @@
> > #include <linux/sunrpc/addr.h>
> > #include <linux/sunrpc/xprtmultipath.h>
> >
> > +#include "sysfs.h"
> > +
> > typedef struct rpc_xprt *(*xprt_switch_find_xprt_t)(struct
> > rpc_xprt_switch *xps,
> > const struct rpc_xprt *cur);
> >
> > @@ -133,6 +135,7 @@ struct rpc_xprt_switch *xprt_switch_alloc(struct
> > rpc_xprt *xprt,
> > xps->xps_net = NULL;
> > INIT_LIST_HEAD(&xps->xps_xprt_list);
> > xps->xps_iter_ops = &rpc_xprt_iter_singular;
> > + rpc_sysfs_xprt_switch_setup(xps, xprt);
> > xprt_switch_add_xprt_locked(xps, xprt);
> > }
> >
> > @@ -161,6 +164,7 @@ static void xprt_switch_free(struct kref *kref)
> > struct rpc_xprt_switch, xps_kref);
> >
> > xprt_switch_free_entries(xps);
> > + rpc_sysfs_xprt_switch_destroy(xps);
> > xprt_switch_free_id(xps);
> > kfree_rcu(xps, xps_rcu);
> > }
>
> --
> Trond Myklebust
> Linux NFS client maintainer, Hammerspace
> trond.myklebust@hammerspace.com
>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2021-04-26 15:30 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-16 3:52 [PATCH v2 00/13] create sysfs files for changing IP address Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 01/13] sunrpc: Create a sunrpc directory under /sys/kernel/ Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 02/13] sunrpc: Create a client/ subdirectory in the sunrpc sysfs Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 03/13] sunrpc: Create per-rpc_clnt sysfs kobjects Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 04/13] sunrpc: Prepare xs_connect() for taking NULL tasks Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 05/13] sunrpc: add xprt id Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 06/13] sunrpc: add IDs to multipath Olga Kornievskaia
2021-04-23 21:16 ` Trond Myklebust
2021-04-16 3:52 ` [PATCH v2 07/13] sunrpc: keep track of the xprt_class in rpc_xprt structure Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 08/13] sunrpc: add xprt_switch direcotry to sunrpc's sysfs Olga Kornievskaia
2021-04-23 21:20 ` Trond Myklebust
2021-04-26 15:30 ` Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 09/13] sunrpc: add a symlink from rpc-client directory to the xprt_switch Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 10/13] sunrpc: add add sysfs directory per xprt under each xprt_switch Olga Kornievskaia
2021-04-23 21:21 ` Trond Myklebust
2021-04-16 3:52 ` [PATCH v2 11/13] sunrpc: add dst_attr attributes to the sysfs xprt directory Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 12/13] sunrpc: provide transport info in the sysfs directory Olga Kornievskaia
2021-04-16 3:52 ` [PATCH v2 13/13] sunrpc: provide multipath " Olga Kornievskaia
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).