[lustre-devel] [PATCH 00/12] lustre: backport OpenSFS work Dec 12, 2021

[lustre-devel] [PATCH 00/12] lustre: backport OpenSFS work Dec 12, 2021

[James Simmons]

Backport of the latest work on the OpenSFS branch as of Dec 12
2021

Andreas Dilger (1):
  lustre: llite: avoid needless large stats alloc

Chris Horn (5):
  lustre: ptlrpc: Do not unlink difficult reply until sent
  lnet: Allow specifying a source NID for lnetctl ping
  lnet: Fix source specified send to different net
  lnet: Fix source specified to routed destination
  lnet: o2iblnd: Default map_on_demand to 1

James Simmons (2):
  lnet: uapi: move out kernel only code.
  lustre: obdclass: make niduuid for lustre_stop_mgc() static

Qian Yingjin (1):
  lustre: pcc: disable PCC for encrypted files

Sebastien Buisson (1):
  lustre: llite: properly detect SELinux disabled case

Sergey Cheremencev (1):
  lustre: obdclass: cosmetic changes in pool handling

Vladimir Saveliev (1):
  lustre: llite: do not take mod rpc slot for getxattr

 fs/lustre/include/lu_object.h              |   2 +-
 fs/lustre/include/lustre_net.h             |   3 +-
 fs/lustre/include/obd_support.h            |   1 +
 fs/lustre/ldlm/ldlm_lib.c                  |  11 +-
 fs/lustre/llite/file.c                     |   3 +
 fs/lustre/llite/llite_internal.h           |   7 +-
 fs/lustre/llite/llite_lib.c                |  10 +-
 fs/lustre/llite/lproc_llite.c              | 238 +++++++++++++++++++++--------
 fs/lustre/llite/pcc.c                      |   3 +
 fs/lustre/llite/xattr_cache.c              |   2 +
 fs/lustre/llite/xattr_security.c           |   8 +-
 fs/lustre/mdc/mdc_locks.c                  |   2 +-
 fs/lustre/obdclass/lu_tgt_pool.c           |  15 +-
 fs/lustre/obdclass/obd_mount.c             |  25 +--
 fs/lustre/ptlrpc/events.c                  |  19 ++-
 fs/lustre/ptlrpc/pack_generic.c            |   2 +-
 fs/lustre/ptlrpc/service.c                 |  10 +-
 include/linux/lnet/lib-types.h             |  16 ++
 include/uapi/linux/lnet/lnet-dlc.h         |   1 +
 include/uapi/linux/lnet/nidstr.h           |  13 --
 net/lnet/klnds/o2iblnd/o2iblnd_modparams.c |   3 +-
 net/lnet/lnet/api-ni.c                     |  29 +++-
 net/lnet/lnet/lib-move.c                   |  28 ++--
 23 files changed, 302 insertions(+), 149 deletions(-)

-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

[lustre-devel] [PATCH 01/12] lustre: llite: do not take mod rpc slot for getxattr

[James Simmons]

From: Vladimir Saveliev <vlaidimir.saveliev@hpe.com>

The following scenario may lead to client eviction:
clientA                clientB                  MDS
threadA1: write to file F1, get
and hold DoM MDC LDLM lock L1:
   ->cl_io_loop()
    ->cl_io_lock()
     :
     ->mdc_lock_granted()
      ->lock->l_writers++
     [hold ref until write done]

threadA2-A8: create files F2-F8:
   ->ll_file_open()
    ->mdc_enqueue_base()
     ->ldlm_cli_enqueue()
      ->ptlrpc_get_mod_rpc_slot()
      ->ptlrpc_queue_wait()
      [hold RPC slot until create done]

                                                OST(s) in recovery.
                                                MDS waiting on OST(s) to
                                                precreate new objects.

threadA1:
    -> cl_io_start()
     -> __generic_file_aio_write()
      -> file_remove_suid()
       -> ll_xattr_cache_refill()
        -> mdc_xattr_common()
         -> ptlrpc_get_mod_rpc_slot()
         [blocked waiting for RPC slot]

                        threadB1: write file F1,
                    enqueue DoM MDC lock L1

                                                MDS sends blocking AST
                                                to clientA for lock L1

ldlm_threadA3: cannot cancel busy lock L1:
   -> ldlm_handle_bl_callback()
   ["Lock L1 referenced, will be cancelled later"]

                                                MDS evicts clientA for
                                                not cancelling lock L1

threadA1: never completes write:
  ->cl_io_end()
   ->cl_io_unlock()
    ->osc_lock_cancel()
     ->lock->l_writers--;

The fix is to add IT_GETXATTR to list of operations which do not
need mod rpc slot.

Tests to illustrate the issue is added.

wait_for_function(): total sleep time (wait) is to be equal to max
when 1 is returned.

HPE-bug-id: LUS-7271
WC-bug-id: https://jira.whamcloud.com/browse/LU-12347
Lustre-commit: eb64594e4473af85 ("LU-12347 llite: do not take mod rpc slot for getxattr")
Signed-off-by: Vladimir Saveliev <vlaidimir.saveliev@hpe.com>
Reviewed-on: https://review.whamcloud.com/44151
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 fs/lustre/include/obd_support.h | 1 +
 fs/lustre/llite/xattr_cache.c   | 2 ++
 fs/lustre/mdc/mdc_locks.c       | 2 +-
 3 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h
index 540e1e0..d57c25c 100644
--- a/fs/lustre/include/obd_support.h
+++ b/fs/lustre/include/obd_support.h
@@ -484,6 +484,7 @@
 #define OBD_FAIL_LLITE_RACE_MOUNT			0x1417
 #define OBD_FAIL_LLITE_PAGE_ALLOC			0x1418
 #define OBD_FAIL_LLITE_OPEN_DELAY			0x1419
+#define OBD_FAIL_LLITE_XATTR_PAUSE			0x1420
 
 #define OBD_FAIL_FID_INDIR				0x1501
 #define OBD_FAIL_FID_INLMA				0x1502
diff --git a/fs/lustre/llite/xattr_cache.c b/fs/lustre/llite/xattr_cache.c
index b044c89..7c1f5b7 100644
--- a/fs/lustre/llite/xattr_cache.c
+++ b/fs/lustre/llite/xattr_cache.c
@@ -396,6 +396,8 @@ static int ll_xattr_cache_refill(struct inode *inode)
 	u32 *xsizes;
 	int rc, i;
 
+	CFS_FAIL_TIMEOUT(OBD_FAIL_LLITE_XATTR_PAUSE, cfs_fail_val ?: 2);
+
 	rc = ll_xattr_find_get_lock(inode, &oit, &req);
 	if (rc)
 		goto err_req;
diff --git a/fs/lustre/mdc/mdc_locks.c b/fs/lustre/mdc/mdc_locks.c
index 66f0039..2c344d7 100644
--- a/fs/lustre/mdc/mdc_locks.c
+++ b/fs/lustre/mdc/mdc_locks.c
@@ -886,7 +886,7 @@ static inline bool mdc_skip_mod_rpc_slot(const struct lookup_intent *it)
 {
 	if (it &&
 	    (it->it_op == IT_GETATTR || it->it_op == IT_LOOKUP ||
-	     it->it_op == IT_READDIR ||
+	     it->it_op == IT_READDIR || it->it_op == IT_GETXATTR ||
 	     (it->it_op == IT_LAYOUT && !(it->it_flags & MDS_FMODE_WRITE))))
 		return true;
 	return false;
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

[lustre-devel] [PATCH 02/12] lnet: uapi: move out kernel only code.

[James Simmons]

Userland doesn't use the new IPv6 function in the UAPI nidstr.h.
So move this to the kernel header lib-types.h. Normally this
wouldn't matter but the python wrappers break with the LUTF
project. With the move to Netlink in the future for the user land
API we shouldn't need these functions anyways.

WC-bug-id: https://jira.whamcloud.com/browse/LU-10391
Lustre-commit: 0631caae7d804720d ("LU-10391 uapi: move out kernel only code.")
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/44844
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
---
 include/linux/lnet/lib-types.h   | 16 ++++++++++++++++
 include/uapi/linux/lnet/nidstr.h | 13 -------------
 2 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/include/linux/lnet/lib-types.h b/include/linux/lnet/lib-types.h
index 7631044..eb736a5 100644
--- a/include/linux/lnet/lib-types.h
+++ b/include/linux/lnet/lib-types.h
@@ -51,6 +51,22 @@
 #include <uapi/linux/lnet/lnet-dlc.h>
 #include <uapi/linux/lnet/nidstr.h>
 
+char *libcfs_nidstr_r(const struct lnet_nid *nid,
+		      char *buf, size_t buf_size);
+
+static inline char *libcfs_nidstr(const struct lnet_nid *nid)
+{
+	return libcfs_nidstr_r(nid, libcfs_next_nidstring(),
+			       LNET_NIDSTR_SIZE);
+}
+
+int libcfs_strnid(struct lnet_nid *nid, const char *str);
+char *libcfs_idstr(struct lnet_processid *id);
+
+int cfs_match_nid_net(struct lnet_nid *nid, u32 net,
+		      struct list_head *net_num_list,
+		      struct list_head *addr);
+
 /* Max payload size */
 #define LNET_MAX_PAYLOAD	LNET_MTU
 
diff --git a/include/uapi/linux/lnet/nidstr.h b/include/uapi/linux/lnet/nidstr.h
index 482cfb2..80be2eb 100644
--- a/include/uapi/linux/lnet/nidstr.h
+++ b/include/uapi/linux/lnet/nidstr.h
@@ -90,27 +90,14 @@ static inline char *libcfs_nid2str(lnet_nid_t nid)
 				LNET_NIDSTR_SIZE);
 }
 
-char *libcfs_nidstr_r(const struct lnet_nid *nid,
-		      char *buf, __kernel_size_t buf_size);
-static inline char *libcfs_nidstr(const struct lnet_nid *nid)
-{
-	return libcfs_nidstr_r(nid, libcfs_next_nidstring(),
-			       LNET_NIDSTR_SIZE);
-}
-
 __u32 libcfs_str2net(const char *str);
 lnet_nid_t libcfs_str2nid(const char *str);
-int libcfs_strnid(struct lnet_nid *nid, const char *str);
 int libcfs_str2anynid(lnet_nid_t *nid, const char *str);
 char *libcfs_id2str(struct lnet_process_id id);
-char *libcfs_idstr(struct lnet_processid *id);
 void cfs_free_nidlist(struct list_head *list);
 int cfs_parse_nidlist(char *str, int len, struct list_head *list);
 int cfs_print_nidlist(char *buffer, int count, struct list_head *list);
 int cfs_match_nid(lnet_nid_t nid, struct list_head *list);
-int cfs_match_nid_net(struct lnet_nid *nid, __u32 net,
-		       struct list_head *net_num_list,
-		      struct list_head *addr);
 int cfs_match_net(__u32 net_id, __u32 net_type,
 		  struct list_head *net_num_list);
 
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

[lustre-devel] [PATCH 03/12] lustre: ptlrpc: Do not unlink difficult reply until sent

[James Simmons]

From: Chris Horn <chris.horn@hpe.com>

If a difficult reply is queued in LNet, or the PUT for it is
otherwise delayed, then it is possible for the commit callback
to unlink the reply MD which will abort the send. This results in
client hitting "slow reply" timeout for the associated RPC and
an unnecessary reconnect (and possibly resend).

This patch replaces the rs_on_net flag with rs_sent and rs_unlinked.
These flags indicate whether the send event for the reply MD has
been generated, and whether the MD has been unlinked, respectively.

If rs_sent is set, but rs_unlinked has not been set, then ptlrpc_hr
is free to unlink the reply MD as a result of the commit callback.
The reply-ack will simply be dropped by the server.

If ptlrpc_hr is processing the reply because of commit callback, and
rs_sent has not been set, then ptlrpc_hr will not unlink the reply
MD. This means that the reply_out_callback must also be modified to
check for this case when the send event occurs. Otherwise, if the ACK
never arrives from the client, then the MD would never be unlinked.
Thus when the send event occurs, and rs_handled is set, the
reply_out_callback will schedule the reply for handling by ptlrpc_hr.

HPE-bug-id: LUS-10505
WC-bug-id: https://jira.whamcloud.com/browse/LU-15068
Lustre-commit: 5c156b48425aae245 ("LU-15068 ptlrpc: Do not unlink difficult reply until sent")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Reviewed-on: https://review.whamcloud.com/45138
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 fs/lustre/include/lustre_net.h  |  3 ++-
 fs/lustre/ldlm/ldlm_lib.c       | 11 +++++++----
 fs/lustre/ptlrpc/events.c       | 19 ++++++++++++++++---
 fs/lustre/ptlrpc/pack_generic.c |  2 +-
 fs/lustre/ptlrpc/service.c      | 10 +++++++---
 5 files changed, 33 insertions(+), 12 deletions(-)

diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h
index f72f7c6..78df59b 100644
--- a/fs/lustre/include/lustre_net.h
+++ b/fs/lustre/include/lustre_net.h
@@ -492,7 +492,8 @@ struct ptlrpc_reply_state {
 	unsigned long		rs_scheduled:1; /* being handled? */
 	unsigned long		rs_scheduled_ever:1; /* any schedule attempts? */
 	unsigned long		rs_handled:1;	/* been handled yet? */
-	unsigned long		rs_on_net:1;	/* reply_out_callback pending? */
+	unsigned long		rs_sent:1;	/* Got LNET_EVENT_SEND? */
+	unsigned long		rs_unlinked:1;	/* Reply MD unlinked? */
 	unsigned long		rs_prealloc:1;	/* rs from prealloc list */
 	unsigned long		rs_committed:1;	/* the transaction was committed
 						 * and the rs was dispatched
diff --git a/fs/lustre/ldlm/ldlm_lib.c b/fs/lustre/ldlm/ldlm_lib.c
index 90bad71..9aa87d1 100644
--- a/fs/lustre/ldlm/ldlm_lib.c
+++ b/fs/lustre/ldlm/ldlm_lib.c
@@ -813,7 +813,8 @@ void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id)
 	LASSERT(!rs->rs_scheduled);
 	LASSERT(!rs->rs_scheduled_ever);
 	LASSERT(!rs->rs_handled);
-	LASSERT(!rs->rs_on_net);
+	LASSERT(!rs->rs_sent);
+	LASSERT(!rs->rs_unlinked);
 	LASSERT(!rs->rs_export);
 	LASSERT(list_empty(&rs->rs_obd_list));
 	LASSERT(list_empty(&rs->rs_exp_list));
@@ -822,7 +823,8 @@ void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id)
 
 	/* disable reply scheduling while I'm setting up */
 	rs->rs_scheduled = 1;
-	rs->rs_on_net = 1;
+	rs->rs_sent = 0;
+	rs->rs_unlinked = 0;
 	rs->rs_xid = req->rq_xid;
 	rs->rs_transno = req->rq_transno;
 	rs->rs_export = exp;
@@ -856,13 +858,14 @@ void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id)
 		 * would have been +1 ref for the net, which
 		 * reply_out_callback leaves alone)
 		 */
-		rs->rs_on_net = 0;
+		rs->rs_sent = 1;
+		rs->rs_unlinked = 1;
 		ptlrpc_rs_addref(rs);
 	}
 
 	spin_lock(&rs->rs_lock);
 	if (rs->rs_transno <= exp->exp_last_committed ||
-	    (!rs->rs_on_net && !rs->rs_no_ack) ||
+	    (rs->rs_unlinked && !rs->rs_no_ack) ||
 	    list_empty(&rs->rs_exp_list) ||     /* completed already */
 	    list_empty(&rs->rs_obd_list)) {
 		CDEBUG(D_HA, "Schedule reply immediately\n");
diff --git a/fs/lustre/ptlrpc/events.c b/fs/lustre/ptlrpc/events.c
index 559d811..dbf9f9d 100644
--- a/fs/lustre/ptlrpc/events.c
+++ b/fs/lustre/ptlrpc/events.c
@@ -401,6 +401,7 @@ void reply_out_callback(struct lnet_event *ev)
 	struct ptlrpc_cb_id *cbid = ev->md_user_ptr;
 	struct ptlrpc_reply_state *rs = cbid->cbid_arg;
 	struct ptlrpc_service_part *svcpt = rs->rs_svcpt;
+	bool need_schedule = false;
 
 	LASSERT(ev->type == LNET_EVENT_SEND ||
 		ev->type == LNET_EVENT_ACK ||
@@ -415,16 +416,28 @@ void reply_out_callback(struct lnet_event *ev)
 		return;
 	}
 
-	LASSERT(rs->rs_on_net);
+	if (ev->type == LNET_EVENT_SEND) {
+		spin_lock(&rs->rs_lock);
+		rs->rs_sent = 1;
+		/* If transaction was committed before the SEND, and the ACK
+		 * is lost, then we need to schedule so ptlrpc_hr can unlink
+		 * the MD.
+		 */
+		if (rs->rs_handled)
+			need_schedule = true;
+		spin_unlock(&rs->rs_lock);
+	}
+
+	if (ev->unlinked || need_schedule) {
+		LASSERT(rs->rs_sent);
 
-	if (ev->unlinked) {
 		/* Last network callback. The net's ref on 'rs' stays put
 		 * until ptlrpc_handle_rs() is done with it
 		 */
 		spin_lock(&svcpt->scp_rep_lock);
 		spin_lock(&rs->rs_lock);
 
-		rs->rs_on_net = 0;
+		rs->rs_unlinked = ev->unlinked;
 		if (!rs->rs_no_ack ||
 		    rs->rs_transno <=
 		    rs->rs_export->exp_obd->obd_last_committed ||
diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c
index 62e060d..23b36de 100644
--- a/fs/lustre/ptlrpc/pack_generic.c
+++ b/fs/lustre/ptlrpc/pack_generic.c
@@ -458,7 +458,7 @@ void lustre_free_reply_state(struct ptlrpc_reply_state *rs)
 
 	LASSERT(atomic_read(&rs->rs_refcount) == 0);
 	LASSERT(!rs->rs_difficult || rs->rs_handled);
-	LASSERT(!rs->rs_on_net);
+	LASSERT(!rs->rs_difficult || rs->rs_unlinked);
 	LASSERT(!rs->rs_scheduled);
 	LASSERT(!rs->rs_export);
 	LASSERT(rs->rs_nlocks == 0);
diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c
index 2917ca3..dbe2347 100644
--- a/fs/lustre/ptlrpc/service.c
+++ b/fs/lustre/ptlrpc/service.c
@@ -1940,10 +1940,14 @@ static int ptlrpc_handle_rs(struct ptlrpc_reply_state *rs)
 		       libcfs_nid2str(exp->exp_connection->c_peer.nid));
 	}
 
-	if ((!been_handled && rs->rs_on_net) || nlocks > 0) {
+	if ((rs->rs_sent && !rs->rs_unlinked) || nlocks > 0) {
 		spin_unlock(&rs->rs_lock);
 
-		if (!been_handled && rs->rs_on_net) {
+		/* We can unlink if the LNET_EVENT_SEND has occurred.
+		 * If rs_unlinked is set then MD is already unlinked and no
+		 * need to do so here.
+		 */
+		if ((rs->rs_sent && !rs->rs_unlinked)) {
 			LNetMDUnlink(rs->rs_md_h);
 			/* Ignore return code; we're racing with completion */
 		}
@@ -1957,7 +1961,7 @@ static int ptlrpc_handle_rs(struct ptlrpc_reply_state *rs)
 
 	rs->rs_scheduled = 0;
 
-	if (!rs->rs_on_net) {
+	if (rs->rs_unlinked) {
 		/* Off the net */
 		spin_unlock(&rs->rs_lock);
 
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

[lustre-devel] [PATCH 04/12] lustre: obdclass: make niduuid for lustre_stop_mgc() static

[James Simmons]

The process to create a proper string for niduuid can be made
simpler and avoid a memory allocation.

WC-bug-id: https://jira.whamcloud.com/browse/LU-9325
Lustre-commit: 85b400b67b0d8d493 ("LU-9325 obdclass: make niduuid for lustre_stop_mgc() static")
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/33617
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
---
 fs/lustre/obdclass/obd_mount.c | 25 ++++++++-----------------
 1 file changed, 8 insertions(+), 17 deletions(-)

diff --git a/fs/lustre/obdclass/obd_mount.c b/fs/lustre/obdclass/obd_mount.c
index 19684fb..58ca72d 100644
--- a/fs/lustre/obdclass/obd_mount.c
+++ b/fs/lustre/obdclass/obd_mount.c
@@ -431,8 +431,8 @@ static int lustre_stop_mgc(struct super_block *sb)
 {
 	struct lustre_sb_info *lsi = s2lsi(sb);
 	struct obd_device *obd;
-	char *niduuid = NULL, *ptr = NULL;
-	int i, rc = 0, len = 0;
+	char niduuid[MAX_OBD_NAME + 6], *ptr = NULL;
+	int i, rc = 0;
 
 	if (!lsi)
 		return -ENOENT;
@@ -467,24 +467,17 @@ static int lustre_stop_mgc(struct super_block *sb)
 			CDEBUG(D_MOUNT, "disconnect failed %d\n", rc);
 	}
 
-	/* Save the obdname for cleaning the nid uuids, which are obdname_XX */
-	len = strlen(obd->obd_name) + 6;
-	niduuid = kzalloc(len, GFP_NOFS);
-	if (niduuid) {
-		strcpy(niduuid, obd->obd_name);
-		ptr = niduuid + strlen(niduuid);
-	}
+	/*
+	 * Cache the obdname for cleaning the nid uuids, which are
+	 * obdname_XX before calling class_manual_cleanup
+	 */
+	strcpy(niduuid, obd->obd_name);
+	ptr = niduuid + strlen(niduuid);
 
 	rc = class_manual_cleanup(obd);
 	if (rc)
 		goto out;
 
-	/* Clean the nid uuids */
-	if (!niduuid) {
-		rc = -ENOMEM;
-		goto out;
-	}
-
 	for (i = 0; i < lsi->lsi_lmd->lmd_mgs_failnodes; i++) {
 		sprintf(ptr, "_%x", i);
 		rc = do_lcfg(LUSTRE_MGC_OBDNAME, 0, LCFG_DEL_UUID,
@@ -494,8 +487,6 @@ static int lustre_stop_mgc(struct super_block *sb)
 			       niduuid, rc);
 	}
 out:
-	kfree(niduuid);
-
 	/* class_import_put will get rid of the additional connections */
 	mutex_unlock(&mgc_start_lock);
 	return rc;
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

[lustre-devel] [PATCH 05/12] lnet: Allow specifying a source NID for lnetctl ping

[James Simmons]

From: Chris Horn <chris.horn@hpe.com>

Add a new --source option for lnetctl ping command. This allows the
user to specify a local NI from which to send the ping. This also
ensures that the specified destination NID is also used. Otherwise,
pings to multi-rail peers may end up going to a different peer NI
based on the multi-rail selection algorithm. The ability to specify
a source NI, and thus fix the destination NI, is a great help in
troubleshooting communication issues between multi-rail peers.

Add test to exercise lnetctl ping --source option.

HPE-bug-id: LUS-10296
WC-bug-id: https://jira.whamcloud.com/browse/LU-14939
Lustre-commit: 48ef9982c474a02c4 ("LU-14939 lnet: Allow specifying a source NID for lnetctl ping")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Reviewed-on: https://review.whamcloud.com/44727
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 include/uapi/linux/lnet/lnet-dlc.h |  1 +
 net/lnet/lnet/api-ni.c             | 29 ++++++++++++++++++++++-------
 2 files changed, 23 insertions(+), 7 deletions(-)

diff --git a/include/uapi/linux/lnet/lnet-dlc.h b/include/uapi/linux/lnet/lnet-dlc.h
index 2ca70eb..415968a 100644
--- a/include/uapi/linux/lnet/lnet-dlc.h
+++ b/include/uapi/linux/lnet/lnet-dlc.h
@@ -132,6 +132,7 @@ struct lnet_ioctl_ping_data {
 	__u32 mr_info;
 	struct lnet_process_id ping_id;
 	struct lnet_process_id __user *ping_buf;
+	lnet_nid_t ping_src;
 };
 
 struct lnet_ioctl_config_data {
diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c
index 3ed3f0b..550f035 100644
--- a/net/lnet/lnet/api-ni.c
+++ b/net/lnet/lnet/api-ni.c
@@ -205,8 +205,9 @@ static void lnet_set_lnd_timeout(void)
  */
 static atomic_t lnet_dlc_seq_no = ATOMIC_INIT(0);
 
-static int lnet_ping(struct lnet_process_id id, signed long timeout,
-		     struct lnet_process_id __user *ids, int n_ids);
+static int lnet_ping(struct lnet_process_id id, lnet_nid_t src_nid,
+		     signed long timeout, struct lnet_process_id __user *ids,
+		     int n_ids);
 
 static int lnet_discover(struct lnet_process_id id, u32 force,
 			 struct lnet_process_id __user *ids, int n_ids);
@@ -4267,7 +4268,7 @@ u32 lnet_get_dlc_seq_locked(void)
 		else
 			timeout = msecs_to_jiffies(data->ioc_u32[1]);
 
-		rc = lnet_ping(id, timeout, data->ioc_pbuf1,
+		rc = lnet_ping(id, LNET_NID_ANY, timeout, data->ioc_pbuf1,
 			       data->ioc_plen1 / sizeof(struct lnet_process_id));
 
 		if (rc < 0)
@@ -4281,6 +4282,19 @@ u32 lnet_get_dlc_seq_locked(void)
 		struct lnet_ioctl_ping_data *ping = arg;
 		struct lnet_peer *lp;
 		signed long timeout;
+		lnet_nid_t src_nid = LNET_NID_ANY;
+
+		/* Check if the supplied ping data supports source nid
+		 * NB: This check is sufficient if lnet_ioctl_ping_data has
+		 * additional fields added, but if they are re-ordered or
+		 * fields removed then this will break. It is expected that
+		 * these ioctls will be replaced with netlink implementation, so
+		 * it is probably not worth coming up with a more robust version
+		 * compatibility scheme.
+		 */
+		if (ping->ping_hdr.ioc_len >=
+		    sizeof(struct lnet_ioctl_ping_data))
+			src_nid = ping->ping_src;
 
 		/* If timeout is negative then set default of 3 minutes */
 		if (((s32)ping->op_param) <= 0 ||
@@ -4289,7 +4303,7 @@ u32 lnet_get_dlc_seq_locked(void)
 		else
 			timeout = msecs_to_jiffies(ping->op_param);
 
-		rc = lnet_ping(ping->ping_id, timeout,
+		rc = lnet_ping(ping->ping_id, src_nid, timeout,
 			       ping->ping_buf,
 			       ping->ping_count);
 		if (rc < 0)
@@ -4526,8 +4540,9 @@ struct ping_data {
 		complete(&pd->completion);
 }
 
-static int lnet_ping(struct lnet_process_id id, signed long timeout,
-		     struct lnet_process_id __user *ids, int n_ids)
+static int lnet_ping(struct lnet_process_id id, lnet_nid_t src_nid,
+		     signed long timeout, struct lnet_process_id __user *ids,
+		     int n_ids)
 {
 	struct lnet_md md = { NULL };
 	struct ping_data pd = { 0 };
@@ -4572,7 +4587,7 @@ static int lnet_ping(struct lnet_process_id id, signed long timeout,
 		goto fail_ping_buffer_decref;
 	}
 
-	rc = LNetGet(LNET_NID_ANY, pd.mdh, id,
+	rc = LNetGet(src_nid, pd.mdh, id,
 		     LNET_RESERVED_PORTAL,
 		     LNET_PROTO_PING_MATCHBITS, 0, false);
 	if (rc) {
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

[lustre-devel] [PATCH 06/12] lnet: Fix source specified send to different net

[James Simmons]

From: Chris Horn <chris.horn@hpe.com>

The destination NI is fixed for all source-specified sends. Thus, in
order for a source-specified send to be considered "local", i.e. a
send that does not require a route, the destination NID must be on
the same net as the specified source.

HPE-bug-id: LUS-10303
WC-bug-id: https://jira.whamcloud.com/browse/LU-14940
Lustre-commit: 3e3563f719ce89de2 ("LU-14940 lnet: Fix source specified send to different net")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Reviewed-on: https://review.whamcloud.com/44728
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 net/lnet/lnet/lib-move.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c
index 088a754..a411724 100644
--- a/net/lnet/lnet/lib-move.c
+++ b/net/lnet/lnet/lib-move.c
@@ -2733,15 +2733,19 @@ struct lnet_ni *
 
 	/* Identify the different send cases
 	 */
-	if (src_nid == LNET_NID_ANY)
+	if (src_nid == LNET_NID_ANY) {
 		send_case |= SRC_ANY;
-	else
+		if (lnet_get_net_locked(LNET_NIDNET(dst_nid)))
+			send_case |= LOCAL_DST;
+		else
+			send_case |= REMOTE_DST;
+	} else {
 		send_case |= SRC_SPEC;
-
-	if (lnet_get_net_locked(LNET_NIDNET(dst_nid)))
-		send_case |= LOCAL_DST;
-	else
-		send_case |= REMOTE_DST;
+		if (LNET_NIDNET(src_nid) == LNET_NIDNET(dst_nid))
+			send_case |= LOCAL_DST;
+		else
+			send_case |= REMOTE_DST;
+	}
 
 	final_hop = false;
 	if (msg->msg_routing && (send_case & LOCAL_DST))
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

[lustre-devel] [PATCH 07/12] lnet: Fix source specified to routed destination

[James Simmons]

From: Chris Horn <chris.horn@hpe.com>

If a source NI is specified for a send then we should not modify the
destination NID that was passed to lnet_send().

HPE-bug-id: LUS-10301
WC-bug-id: https://jira.whamcloud.com/browse/LU-14941
Lustre-commit: 98da4ace43a6c4c59 ("LU-14941 lnet: Fix source specified to routed destination")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Reviewed-on: https://review.whamcloud.com/44730
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 net/lnet/lnet/lib-move.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c
index a411724..caffa30 100644
--- a/net/lnet/lnet/lib-move.c
+++ b/net/lnet/lnet/lib-move.c
@@ -1989,13 +1989,19 @@ struct lnet_ni *
 	}
 
 	if (!route_found) {
-		if (sd->sd_msg->msg_routing) {
+		if (sd->sd_msg->msg_routing || src_nid != LNET_NID_ANY) {
 			/* If I'm routing this message then I need to find the
 			 * next hop based on the destination NID
+			 *
+			 * We also find next hop based on the destination NID
+			 * if the source NI was specified
 			 */
 			best_rnet = lnet_find_rnet_locked(LNET_NIDNET(sd->sd_dst_nid));
 			if (!best_rnet) {
-				CERROR("Unable to route message to %s - Route table may be misconfigured\n",
+				CERROR("Unable to send message from %s to %s - Route table may be misconfigured\n",
+				       src_nid != LNET_NID_ANY ?
+						libcfs_nid2str(src_nid) :
+						"any local NI",
 				       libcfs_nid2str(sd->sd_dst_nid));
 				return -EHOSTUNREACH;
 			}
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

[lustre-devel] [PATCH 08/12] lustre: obdclass: cosmetic changes in pool handling

[James Simmons]

From: Sergey Cheremencev <sergey.cheremencev@hpe.com>

cosmetic changes in pool handling:
- make tgt_pool_free void
- set default rc value in lu_tgt_check_index.

HPE-bug-id: LUS-9547
WC-bug-id: https://jira.whamcloud.com/browse/LU-15110
Lustre-commit: d01cae3ec8758fc27 ("LU-15110 quota: cosmetic changes in PQ")
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/45258
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 fs/lustre/include/lu_object.h    |  2 +-
 fs/lustre/obdclass/lu_tgt_pool.c | 15 +++++++--------
 2 files changed, 8 insertions(+), 9 deletions(-)

diff --git a/fs/lustre/include/lu_object.h b/fs/lustre/include/lu_object.h
index 146398a..3fb40c6 100644
--- a/fs/lustre/include/lu_object.h
+++ b/fs/lustre/include/lu_object.h
@@ -1423,7 +1423,7 @@ struct lu_tgt_pool {
 int lu_tgt_pool_init(struct lu_tgt_pool *op, unsigned int count);
 int lu_tgt_pool_add(struct lu_tgt_pool *op, u32 idx, unsigned int min_count);
 int lu_tgt_pool_remove(struct lu_tgt_pool *op, u32 idx);
-int lu_tgt_pool_free(struct lu_tgt_pool *op);
+void lu_tgt_pool_free(struct lu_tgt_pool *op);
 int lu_tgt_check_index(int idx, struct lu_tgt_pool *osts);
 int lu_tgt_pool_extend(struct lu_tgt_pool *op, unsigned int min_count);
 
diff --git a/fs/lustre/obdclass/lu_tgt_pool.c b/fs/lustre/obdclass/lu_tgt_pool.c
index 17bae54..38ab83d 100644
--- a/fs/lustre/obdclass/lu_tgt_pool.c
+++ b/fs/lustre/obdclass/lu_tgt_pool.c
@@ -197,14 +197,15 @@ int lu_tgt_pool_remove(struct lu_tgt_pool *op, u32 idx)
 
 int lu_tgt_check_index(int idx, struct lu_tgt_pool *osts)
 {
-	int rc = 0, i;
+	int i, rc = -ENOENT;
 
 	down_read(&osts->op_rw_sem);
 	for (i = 0; i < osts->op_count; i++) {
-		if (osts->op_array[i] == idx)
+		if (osts->op_array[i] == idx) {
+			rc = 0;
 			goto out;
+		}
 	}
-	rc = -ENOENT;
 out:
 	up_read(&osts->op_rw_sem);
 	return rc;
@@ -219,13 +220,11 @@ int lu_tgt_check_index(int idx, struct lu_tgt_pool *osts)
  * deleted from memory.
  *
  * @op		pool to be freed.
- *
- * Return:	0 on success or if pool was already freed
  */
-int lu_tgt_pool_free(struct lu_tgt_pool *op)
+void lu_tgt_pool_free(struct lu_tgt_pool *op)
 {
 	if (op->op_size == 0)
-		return 0;
+		return;
 
 	down_write(&op->op_rw_sem);
 
@@ -235,6 +234,6 @@ int lu_tgt_pool_free(struct lu_tgt_pool *op)
 	op->op_size = 0;
 
 	up_write(&op->op_rw_sem);
-	return 0;
+	return;
 }
 EXPORT_SYMBOL(lu_tgt_pool_free);
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

[lustre-devel] [PATCH 09/12] lustre: llite: properly detect SELinux disabled case

[James Simmons]

From: Sebastien Buisson <sbuisson@ddn.com>

Usually, security_dentry_init_security() returns -EOPNOTSUPP when
SELinux is disabled. But on some kernels it returns 0 when SELinux
is disabled, and in this case the security context is empty.
So in both cases make sure the security context name is not set, which
means "SELinux is disabled" for the rest of the code.

WC-bug-id: https://jira.whamcloud.com/browse/LU-15184
Lustre-commit: 42661f7ba106b7d2e ("LU-15184 llite: properly detect SELinux disabled case")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/45501
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 fs/lustre/llite/xattr_security.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/fs/lustre/llite/xattr_security.c b/fs/lustre/llite/xattr_security.c
index e4fb64a..f14021d 100644
--- a/fs/lustre/llite/xattr_security.c
+++ b/fs/lustre/llite/xattr_security.c
@@ -60,7 +60,13 @@ int ll_dentry_init_security(struct dentry *dentry, int mode, struct qstr *name,
 
 	rc = security_dentry_init_security(dentry, mode, name, secctx,
 					   secctx_size);
-	if (rc == -EOPNOTSUPP)
+	/* Usually, security_dentry_init_security() returns -EOPNOTSUPP when
+	 * SELinux is disabled.
+	 * But on some kernels (e.g. rhel 8.5) it returns 0 when SELinux is
+	 * disabled, and in this case the security context is empty.
+	 */
+	if (rc == -EOPNOTSUPP || (rc == 0 && *secctx_size == 0))
+		/* do nothing */
 		return 0;
 	if (rc < 0)
 		return rc;
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

[lustre-devel] [PATCH 10/12] lnet: o2iblnd: Default map_on_demand to 1

[James Simmons]

From: Chris Horn <chris.horn@hpe.com>

On kernels that provide global MR we default to using that exclusively
even if FMR/FastReg is available. This causes an interop issue if the
active side of a connection request has a higher fragment count than
the passive side  because FMR/FastReg may be needed to map the higher
fragment count. We should change the default map_on_demand to 1 so
that FMR/FastReg is used by default. map_on)demand can still be set
to 0 if needed.

WC-bug-id: https://jira.whamcloud.com/browse/LU-15186
Lustre-commit: 21fdd616bd4784e4e ("LU-15186 o2iblnd: Default map_on_demand to 1")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Reviewed-on: https://review.whamcloud.com/45431
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 net/lnet/klnds/o2iblnd/o2iblnd_modparams.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c b/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c
index 81cde1b..022ed02 100644
--- a/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c
+++ b/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c
@@ -141,8 +141,7 @@
  *     the behavior when transmit with GAPS verses contiguous.
  */
 
-#define IBLND_DEFAULT_MAP_ON_DEMAND 1
-static int map_on_demand = IBLND_DEFAULT_MAP_ON_DEMAND;
+static int map_on_demand = 1;
 module_param(map_on_demand, int, 0444);
 MODULE_PARM_DESC(map_on_demand, "map on demand (obsolete)");
 
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

[lustre-devel] [PATCH 11/12] lustre: pcc: disable PCC for encrypted files

[James Simmons]

From: Qian Yingjin <qian@ddn.com>

When files are encrypted in Lustre using fscrypt, they should
normally not be accessible to users without the proper encyrption
key. However, if a user has then encryption key loaded when they
read a file, it may be decrypted in memory and saved to the PCC
backend in unencrypted form.

Due to the above reason, we just disable PCC caching for encrypted
files.

DDN-bug-id: EX-3571
WC-bug-id: https://jira.whamcloud.com/browse/LU-15217
Lustre-commit: f8c79eea11ac96019 ("LU-15217 pcc: disable PCC for encrypted files")
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/45545
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 fs/lustre/llite/file.c | 3 +++
 fs/lustre/llite/pcc.c  | 3 +++
 2 files changed, 6 insertions(+)

diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c
index d3374232..898db80 100644
--- a/fs/lustre/llite/file.c
+++ b/fs/lustre/llite/file.c
@@ -3598,6 +3598,9 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc,
 		if (ioc->lil_count != 1)
 			return -EINVAL;
 
+		if (IS_ENCRYPTED(inode))
+			return -EOPNOTSUPP;
+
 		arg += sizeof(*ioc);
 		if (copy_from_user(&param.pa_archive_id, (void __user *)arg,
 				   sizeof(u32))) {
diff --git a/fs/lustre/llite/pcc.c b/fs/lustre/llite/pcc.c
index 8430fff..85114b8 100644
--- a/fs/lustre/llite/pcc.c
+++ b/fs/lustre/llite/pcc.c
@@ -1472,6 +1472,9 @@ int pcc_file_open(struct inode *inode, struct file *file)
 	if (!S_ISREG(inode->i_mode))
 		return 0;
 
+	if (IS_ENCRYPTED(inode))
+		return 0;
+
 	pcc_inode_lock(inode);
 	pcci = ll_i2pcci(inode);
 
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

[lustre-devel] [PATCH 12/12] lustre: llite: avoid needless large stats alloc

[James Simmons]

From: Andreas Dilger <adilger@whamcloud.com>

Allocate the ll_rw_extents_info (5896 bytes), ll_rw_offset_info
(6400 bytes), and ll_rw_process_info (640 bytes) structs only
when these stats are enabled, which is very rarely.

WC-bug-id: https://jira.whamcloud.com/browse/LU-15217
Lustre-commit: 9490fd9bb84dc277b ("LU-13601 llite: avoid needless large stats alloc")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-on: https://review.whamcloud.com/40901
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 fs/lustre/llite/llite_internal.h |   7 +-
 fs/lustre/llite/llite_lib.c      |  10 +-
 fs/lustre/llite/lproc_llite.c    | 238 ++++++++++++++++++++++++++++-----------
 3 files changed, 177 insertions(+), 78 deletions(-)

diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h
index ce7431f..01672b8 100644
--- a/fs/lustre/llite/llite_internal.h
+++ b/fs/lustre/llite/llite_internal.h
@@ -701,12 +701,12 @@ struct ll_sb_info {
 	struct cl_device	*ll_cl;
 
 	/* Statistics */
-	struct ll_rw_extents_info ll_rw_extents_info;
+	struct ll_rw_extents_info *ll_rw_extents_info;
 	int			ll_extent_process_count;
 	unsigned int		ll_offset_process_count;
+	struct ll_rw_process_info *ll_rw_process_info;
+	struct ll_rw_process_info *ll_rw_offset_info;
 	ktime_t			ll_process_stats_init;
-	struct ll_rw_process_info ll_rw_process_info[LL_PROCESS_HIST_MAX];
-	struct ll_rw_process_info ll_rw_offset_info[LL_OFFSET_HIST_MAX];
 	unsigned int		ll_rw_offset_entry_count;
 	int			ll_stats_track_id;
 	enum stats_track_type	ll_stats_track_type;
@@ -997,6 +997,7 @@ int cl_get_grouplock(struct cl_object *obj, unsigned long gid, int nonblock,
 void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid,
 		       struct ll_file_data *file, loff_t pos,
 		       size_t count, int rw);
+void ll_free_rw_stats_info(struct ll_sb_info *sbi);
 
 enum {
 	LPROC_LL_READ_BYTES,
diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c
index 147e680..dddbe7a 100644
--- a/fs/lustre/llite/llite_lib.c
+++ b/fs/lustre/llite/llite_lib.c
@@ -85,7 +85,6 @@ static struct ll_sb_info *ll_init_sbi(void)
 	unsigned long lru_page_max;
 	struct sysinfo si;
 	int rc;
-	int i;
 
 	sbi = kzalloc(sizeof(*sbi), GFP_NOFS);
 	if (!sbi)
@@ -161,14 +160,6 @@ static struct ll_sb_info *ll_init_sbi(void)
 	set_bit(LL_SBI_LRU_RESIZE, sbi->ll_flags);
 	set_bit(LL_SBI_LAZYSTATFS, sbi->ll_flags);
 
-	for (i = 0; i <= LL_PROCESS_HIST_MAX; i++) {
-		struct per_process_info *pp_ext;
-
-		pp_ext = &sbi->ll_rw_extents_info.pp_extents[i];
-		spin_lock_init(&pp_ext->pp_r_hist.oh_lock);
-		spin_lock_init(&pp_ext->pp_w_hist.oh_lock);
-	}
-
 	/* metadata statahead is enabled by default */
 	sbi->ll_sa_running_max = LL_SA_RUNNING_DEF;
 	sbi->ll_sa_max = LL_SA_RPC_DEF;
@@ -241,6 +232,7 @@ static void ll_free_sbi(struct super_block *sb)
 		kvfree(items);
 		sbi->ll_foreign_symlink_upcall_items = NULL;
 	}
+	ll_free_rw_stats_info(sbi);
 	pcc_super_fini(&sbi->ll_pcc_super);
 	kfree(sbi);
 }
diff --git a/fs/lustre/llite/lproc_llite.c b/fs/lustre/llite/lproc_llite.c
index a7eb8e1..ce715b4 100644
--- a/fs/lustre/llite/lproc_llite.c
+++ b/fs/lustre/llite/lproc_llite.c
@@ -1924,15 +1924,16 @@ void ll_debugfs_unregister_super(struct super_block *sb)
 	lprocfs_free_stats(&sbi->ll_stats);
 }
 
-static void ll_display_extents_info(struct ll_rw_extents_info *io_extents,
+static void ll_display_extents_info(struct ll_rw_extents_info *rw_extents,
 				    struct seq_file *seq, int which)
 {
 	unsigned long read_tot = 0, write_tot = 0, read_cum, write_cum;
 	unsigned long start, end, r, w;
 	char *unitp = "KMGTPEZY";
 	int i, units = 10;
-	struct per_process_info *pp_info = &io_extents->pp_extents[which];
+	struct per_process_info *pp_info;
 
+	pp_info = &rw_extents->pp_extents[which];
 	read_cum = 0;
 	write_cum = 0;
 	start = 0;
@@ -1968,39 +1969,103 @@ static void ll_display_extents_info(struct ll_rw_extents_info *io_extents,
 static int ll_rw_extents_stats_pp_seq_show(struct seq_file *seq, void *v)
 {
 	struct ll_sb_info *sbi = seq->private;
-	struct ll_rw_extents_info *io_extents = &sbi->ll_rw_extents_info;
+	struct ll_rw_extents_info *rw_extents = sbi->ll_rw_extents_info;
 	int k;
 
-	if (!sbi->ll_rw_stats_on) {
-		seq_puts(seq, "disabled\n"
-			 "write anything in this file to activate, then '0' or 'disabled' to deactivate\n");
+	if (!sbi->ll_rw_stats_on || !rw_extents) {
+		seq_puts(seq, "disabled\n write anything in this file to activate, then '0' or 'disabled' to deactivate\n");
 		return 0;
 	}
-	lprocfs_stats_header(seq, ktime_get(), io_extents->pp_init, 25, ":", 1);
+
+	spin_lock(&sbi->ll_pp_extent_lock);
+	lprocfs_stats_header(seq, ktime_get(), rw_extents->pp_init, 25, ":", 1);
 	seq_printf(seq, "%15s %19s       | %20s\n", " ", "read", "write");
 	seq_printf(seq, "%13s   %14s %4s %4s  | %14s %4s %4s\n",
-		   "extents", "calls", "%", "cum%",
-		   "calls", "%", "cum%");
-	spin_lock(&sbi->ll_pp_extent_lock);
+		   "extents", "calls", "%", "cum%", "calls", "%", "cum%");
+
 	for (k = 0; k < LL_PROCESS_HIST_MAX; k++) {
-		if (io_extents->pp_extents[k].pid != 0) {
+		if (rw_extents->pp_extents[k].pid != 0) {
 			seq_printf(seq, "\nPID: %d\n",
-				   io_extents->pp_extents[k].pid);
-			ll_display_extents_info(io_extents, seq, k);
+				   rw_extents->pp_extents[k].pid);
+			ll_display_extents_info(rw_extents, seq, k);
 		}
 	}
 	spin_unlock(&sbi->ll_pp_extent_lock);
 	return 0;
 }
 
+static int alloc_rw_stats_info(struct ll_sb_info *sbi)
+{
+	struct ll_rw_extents_info *rw_extents;
+	struct ll_rw_process_info *offset;
+	struct ll_rw_process_info *process;
+	int i, rc = 0;
+
+	rw_extents = kzalloc(sizeof(*rw_extents), GFP_KERNEL);
+	if (!rw_extents)
+		return -ENOMEM;
+
+	for (i = 0; i <= LL_PROCESS_HIST_MAX; i++) {
+		spin_lock_init(&rw_extents->pp_extents[i].pp_r_hist.oh_lock);
+		spin_lock_init(&rw_extents->pp_extents[i].pp_w_hist.oh_lock);
+	}
+
+	spin_lock(&sbi->ll_pp_extent_lock);
+	if (!sbi->ll_rw_extents_info)
+		sbi->ll_rw_extents_info = rw_extents;
+	spin_unlock(&sbi->ll_pp_extent_lock);
+	/* another writer allocated the struct before we got the lock */
+	if (sbi->ll_rw_extents_info != rw_extents)
+		kfree(rw_extents);
+
+	process = kzalloc(sizeof(*process) * LL_PROCESS_HIST_MAX,
+			  GFP_KERNEL);
+	if (!process) {
+		rc = -ENOMEM;
+		goto out;
+	}
+	offset = kzalloc(sizeof(*offset) * LL_OFFSET_HIST_MAX,
+			 GFP_KERNEL);
+	if (!offset) {
+		rc = -ENOMEM;
+		goto out_free;
+	}
+
+	spin_lock(&sbi->ll_process_lock);
+	if (!sbi->ll_rw_process_info)
+		sbi->ll_rw_process_info = process;
+	if (!sbi->ll_rw_offset_info)
+		sbi->ll_rw_offset_info = offset;
+	spin_unlock(&sbi->ll_process_lock);
+
+	/* another writer allocated the structs before we got the lock */
+	if (sbi->ll_rw_offset_info != offset)
+		kfree(offset);
+	if (sbi->ll_rw_process_info != process) {
+out_free:
+		kfree(process);
+	}
+out:
+	return rc;
+}
+
+void ll_free_rw_stats_info(struct ll_sb_info *sbi)
+{
+	kfree(sbi->ll_rw_extents_info);
+	sbi->ll_rw_extents_info = NULL;
+	kfree(sbi->ll_rw_offset_info);
+	sbi->ll_rw_offset_info = NULL;
+	kfree(sbi->ll_rw_process_info);
+	sbi->ll_rw_process_info = NULL;
+}
+
 static ssize_t ll_rw_extents_stats_pp_seq_write(struct file *file,
 						const char __user *buf,
-						size_t len,
-						loff_t *off)
+						size_t len, loff_t *off)
 {
 	struct seq_file *seq = file->private_data;
 	struct ll_sb_info *sbi = seq->private;
-	struct ll_rw_extents_info *io_extents = &sbi->ll_rw_extents_info;
+	struct ll_rw_extents_info *rw_extents;
 	int i;
 	s64 value;
 
@@ -2009,19 +2074,31 @@ static ssize_t ll_rw_extents_stats_pp_seq_write(struct file *file,
 
 	value = ll_stats_pid_write(buf, len);
 
-	if (value == 0)
+	if (value == 0) {
 		sbi->ll_rw_stats_on = 0;
-	else
+	} else {
+		if (!sbi->ll_rw_extents_info) {
+			int rc = alloc_rw_stats_info(sbi);
+
+			if (rc)
+				return rc;
+		}
 		sbi->ll_rw_stats_on = 1;
+	}
+
 
 	spin_lock(&sbi->ll_pp_extent_lock);
-	io_extents->pp_init = ktime_get();
-	for (i = 0; i < LL_PROCESS_HIST_MAX; i++) {
-		io_extents->pp_extents[i].pid = 0;
-		lprocfs_oh_clear(&io_extents->pp_extents[i].pp_r_hist);
-		lprocfs_oh_clear(&io_extents->pp_extents[i].pp_w_hist);
+	rw_extents = sbi->ll_rw_extents_info;
+	if (rw_extents) {
+		rw_extents->pp_init = ktime_get();
+		for (i = 0; i < LL_PROCESS_HIST_MAX; i++) {
+			rw_extents->pp_extents[i].pid = 0;
+			lprocfs_oh_clear(&rw_extents->pp_extents[i].pp_r_hist);
+			lprocfs_oh_clear(&rw_extents->pp_extents[i].pp_w_hist);
+		}
 	}
 	spin_unlock(&sbi->ll_pp_extent_lock);
+
 	return len;
 }
 
@@ -2030,22 +2107,22 @@ static ssize_t ll_rw_extents_stats_pp_seq_write(struct file *file,
 static int ll_rw_extents_stats_seq_show(struct seq_file *seq, void *v)
 {
 	struct ll_sb_info *sbi = seq->private;
-	struct ll_rw_extents_info *io_extents = &sbi->ll_rw_extents_info;
+	struct ll_rw_extents_info *rw_extents = sbi->ll_rw_extents_info;
 
-	if (!sbi->ll_rw_stats_on) {
-		seq_puts(seq, "disabled\n"
-			 "write anything in this file to activate, then '0' or 'disabled' to deactivate\n");
+	if (!sbi->ll_rw_stats_on || !rw_extents) {
+		seq_puts(seq, "disabled\n write anything in this file to activate, then '0' or 'disabled' to deactivate\n");
 		return 0;
 	}
 
-	lprocfs_stats_header(seq, ktime_get(), io_extents->pp_init, 25, ":", 1);
+	spin_lock(&sbi->ll_lock);
+	lprocfs_stats_header(seq, ktime_get(), rw_extents->pp_init, 25, ":", 1);
 
 	seq_printf(seq, "%15s %19s       | %20s\n", " ", "read", "write");
 	seq_printf(seq, "%13s   %14s %4s %4s  | %14s %4s %4s\n",
 		   "extents", "calls", "%", "cum%",
 		   "calls", "%", "cum%");
-	spin_lock(&sbi->ll_lock);
-	ll_display_extents_info(io_extents, seq, LL_PROCESS_HIST_MAX);
+
+	ll_display_extents_info(rw_extents, seq, LL_PROCESS_HIST_MAX);
 	spin_unlock(&sbi->ll_lock);
 
 	return 0;
@@ -2057,7 +2134,7 @@ static ssize_t ll_rw_extents_stats_seq_write(struct file *file,
 {
 	struct seq_file *seq = file->private_data;
 	struct ll_sb_info *sbi = seq->private;
-	struct ll_rw_extents_info *io_extents = &sbi->ll_rw_extents_info;
+	struct ll_rw_extents_info *rw_extents;
 	int i;
 	s64 value;
 
@@ -2066,17 +2143,27 @@ static ssize_t ll_rw_extents_stats_seq_write(struct file *file,
 
 	value = ll_stats_pid_write(buf, len);
 
-	if (value == 0)
+	if (value == 0) {
 		sbi->ll_rw_stats_on = 0;
-	else
+	} else {
+		if (!sbi->ll_rw_extents_info) {
+			int rc = alloc_rw_stats_info(sbi);
+
+			if (rc)
+				return rc;
+		}
 		sbi->ll_rw_stats_on = 1;
+	}
 
 	spin_lock(&sbi->ll_pp_extent_lock);
-	io_extents->pp_init = ktime_get();
-	for (i = 0; i <= LL_PROCESS_HIST_MAX; i++) {
-		io_extents->pp_extents[i].pid = 0;
-		lprocfs_oh_clear(&io_extents->pp_extents[i].pp_r_hist);
-		lprocfs_oh_clear(&io_extents->pp_extents[i].pp_w_hist);
+	rw_extents = sbi->ll_rw_extents_info;
+	if (rw_extents) {
+		rw_extents->pp_init = ktime_get();
+		for (i = 0; i <= LL_PROCESS_HIST_MAX; i++) {
+			rw_extents->pp_extents[i].pid = 0;
+			lprocfs_oh_clear(&rw_extents->pp_extents[i].pp_r_hist);
+			lprocfs_oh_clear(&rw_extents->pp_extents[i].pp_w_hist);
+		}
 	}
 	spin_unlock(&sbi->ll_pp_extent_lock);
 
@@ -2094,17 +2181,21 @@ void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid,
 	struct ll_rw_process_info *offset;
 	int *off_count = &sbi->ll_rw_offset_entry_count;
 	int *process_count = &sbi->ll_offset_process_count;
-	struct ll_rw_extents_info *io_extents = &sbi->ll_rw_extents_info;
+	struct ll_rw_extents_info *rw_extents;
 
 	if (!sbi->ll_rw_stats_on)
 		return;
-	process = sbi->ll_rw_process_info;
-	offset = sbi->ll_rw_offset_info;
 
 	spin_lock(&sbi->ll_pp_extent_lock);
+	rw_extents = sbi->ll_rw_extents_info;
+	if (!rw_extents) {
+		spin_unlock(&sbi->ll_pp_extent_lock);
+		return;
+	}
+
 	/* Extent statistics */
 	for (i = 0; i < LL_PROCESS_HIST_MAX; i++) {
-		if (io_extents->pp_extents[i].pid == pid) {
+		if (rw_extents->pp_extents[i].pid == pid) {
 			cur = i;
 			break;
 		}
@@ -2115,24 +2206,29 @@ void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid,
 		sbi->ll_extent_process_count =
 			(sbi->ll_extent_process_count + 1) % LL_PROCESS_HIST_MAX;
 		cur = sbi->ll_extent_process_count;
-		io_extents->pp_extents[cur].pid = pid;
-		lprocfs_oh_clear(&io_extents->pp_extents[cur].pp_r_hist);
-		lprocfs_oh_clear(&io_extents->pp_extents[cur].pp_w_hist);
+		rw_extents->pp_extents[cur].pid = pid;
+		lprocfs_oh_clear(&rw_extents->pp_extents[cur].pp_r_hist);
+		lprocfs_oh_clear(&rw_extents->pp_extents[cur].pp_w_hist);
 	}
 
 	for (i = 0; (count >= 1 << (LL_HIST_START + i)) &&
 	     (i < (LL_HIST_MAX - 1)); i++)
 		;
 	if (rw == 0) {
-		io_extents->pp_extents[cur].pp_r_hist.oh_buckets[i]++;
-		io_extents->pp_extents[LL_PROCESS_HIST_MAX].pp_r_hist.oh_buckets[i]++;
+		rw_extents->pp_extents[cur].pp_r_hist.oh_buckets[i]++;
+		rw_extents->pp_extents[LL_PROCESS_HIST_MAX].pp_r_hist.oh_buckets[i]++;
 	} else {
-		io_extents->pp_extents[cur].pp_w_hist.oh_buckets[i]++;
-		io_extents->pp_extents[LL_PROCESS_HIST_MAX].pp_w_hist.oh_buckets[i]++;
+		rw_extents->pp_extents[cur].pp_w_hist.oh_buckets[i]++;
+		rw_extents->pp_extents[LL_PROCESS_HIST_MAX].pp_w_hist.oh_buckets[i]++;
 	}
 	spin_unlock(&sbi->ll_pp_extent_lock);
 
 	spin_lock(&sbi->ll_process_lock);
+	process = sbi->ll_rw_process_info;
+	offset = sbi->ll_rw_offset_info;
+	if (!process || !offset)
+		goto out_unlock;
+
 	/* Offset statistics */
 	for (i = 0; i < LL_PROCESS_HIST_MAX; i++) {
 		if (process[i].rw_pid == pid) {
@@ -2143,8 +2239,7 @@ void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid,
 				process[i].rw_largest_extent = count;
 				process[i].rw_offset = 0;
 				process[i].rw_last_file = file;
-				spin_unlock(&sbi->ll_process_lock);
-				return;
+				goto out_unlock;
 			}
 			if (process[i].rw_last_file_pos != pos) {
 				*off_count =
@@ -2173,8 +2268,7 @@ void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid,
 			if (process[i].rw_largest_extent < count)
 				process[i].rw_largest_extent = count;
 			process[i].rw_last_file_pos = pos + count;
-			spin_unlock(&sbi->ll_process_lock);
-			return;
+			goto out_unlock;
 		}
 	}
 	*process_count = (*process_count + 1) % LL_PROCESS_HIST_MAX;
@@ -2186,14 +2280,16 @@ void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid,
 	process[*process_count].rw_largest_extent = count;
 	process[*process_count].rw_offset = 0;
 	process[*process_count].rw_last_file = file;
+
+out_unlock:
 	spin_unlock(&sbi->ll_process_lock);
 }
 
 static int ll_rw_offset_stats_seq_show(struct seq_file *seq, void *v)
 {
 	struct ll_sb_info *sbi = seq->private;
-	struct ll_rw_process_info *offset = sbi->ll_rw_offset_info;
-	struct ll_rw_process_info *process = sbi->ll_rw_process_info;
+	struct ll_rw_process_info *offset;
+	struct ll_rw_process_info *process;
 	int i;
 
 	if (!sbi->ll_rw_stats_on) {
@@ -2204,12 +2300,13 @@ static int ll_rw_offset_stats_seq_show(struct seq_file *seq, void *v)
 	spin_lock(&sbi->ll_process_lock);
 	lprocfs_stats_header(seq, ktime_get(), sbi->ll_process_stats_init, 25,
 			     ":", true);
-
 	seq_printf(seq, "%3s %10s %14s %14s %17s %17s %14s\n",
 		   "R/W", "PID", "RANGE START", "RANGE END",
 		   "SMALLEST EXTENT", "LARGEST EXTENT", "OFFSET");
+
 	/* We stored the discontiguous offsets here; print them first */
-	for (i = 0; i < LL_OFFSET_HIST_MAX; i++) {
+	offset = sbi->ll_rw_offset_info;
+	for (i = 0; offset && i < LL_OFFSET_HIST_MAX; i++) {
 		if (offset[i].rw_pid != 0)
 			seq_printf(seq,
 				   "%3c %10d %14llu %14llu %17lu %17lu %14lld\n",
@@ -2221,8 +2318,10 @@ static int ll_rw_offset_stats_seq_show(struct seq_file *seq, void *v)
 				   (unsigned long)offset[i].rw_largest_extent,
 				   offset[i].rw_offset);
 	}
+
 	/* Then print the current offsets for each process */
-	for (i = 0; i < LL_PROCESS_HIST_MAX; i++) {
+	process = sbi->ll_rw_process_info;
+	for (i = 0; process && i < LL_PROCESS_HIST_MAX; i++) {
 		if (process[i].rw_pid != 0)
 			seq_printf(seq,
 				   "%3c %10d %14llu %14llu %17lu %17lu %14lld\n",
@@ -2245,8 +2344,6 @@ static ssize_t ll_rw_offset_stats_seq_write(struct file *file,
 {
 	struct seq_file *seq = file->private_data;
 	struct ll_sb_info *sbi = seq->private;
-	struct ll_rw_process_info *process_info = sbi->ll_rw_process_info;
-	struct ll_rw_process_info *offset_info = sbi->ll_rw_offset_info;
 	s64 value;
 
 	if (len == 0)
@@ -2254,19 +2351,28 @@ static ssize_t ll_rw_offset_stats_seq_write(struct file *file,
 
 	value = ll_stats_pid_write(buf, len);
 
-	if (value == 0)
+	if (value == 0) {
 		sbi->ll_rw_stats_on = 0;
-	else
+	} else {
+		if (!sbi->ll_rw_process_info || !sbi->ll_rw_offset_info) {
+			int rc = alloc_rw_stats_info(sbi);
+
+			if (rc)
+				return rc;
+		}
 		sbi->ll_rw_stats_on = 1;
+	}
 
 	spin_lock(&sbi->ll_process_lock);
 	sbi->ll_offset_process_count = 0;
 	sbi->ll_rw_offset_entry_count = 0;
 	sbi->ll_process_stats_init = ktime_get();
-	memset(process_info, 0, sizeof(struct ll_rw_process_info) *
-	       LL_PROCESS_HIST_MAX);
-	memset(offset_info, 0, sizeof(struct ll_rw_process_info) *
-	       LL_OFFSET_HIST_MAX);
+	if (sbi->ll_rw_process_info)
+		memset(sbi->ll_rw_process_info, 0,
+		       sizeof(struct ll_rw_process_info) * LL_PROCESS_HIST_MAX);
+	if (sbi->ll_rw_offset_info)
+		memset(sbi->ll_rw_offset_info, 0,
+		       sizeof(struct ll_rw_process_info) * LL_OFFSET_HIST_MAX);
 	spin_unlock(&sbi->ll_process_lock);
 
 	return len;
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org