lustre-devel-lustre.org archive mirror
 help / color / mirror / Atom feed
From: James Simmons <jsimmons@infradead.org>
To: Andreas Dilger <adilger@whamcloud.com>,
	Oleg Drokin <green@whamcloud.com>, NeilBrown <neilb@suse.de>
Cc: Lai Siyao <lai.siyao@whamcloud.com>,
	Lustre Development List <lustre-devel@lists.lustre.org>
Subject: [lustre-devel] [PATCH 05/18] lustre: lmv: compare space to mkdir on parent MDT
Date: Mon, 19 Jul 2021 08:32:00 -0400	[thread overview]
Message-ID: <1626697933-6971-6-git-send-email-jsimmons@infradead.org> (raw)
In-Reply-To: <1626697933-6971-1-git-send-email-jsimmons@infradead.org>

From: Lai Siyao <lai.siyao@whamcloud.com>

In QOS subdirectory creation, subdirectories are kept on parent MDT
if it is less full than average, however it checks weight other than
free space, while "weight = free space - penalty", if MDTs have
different penalties, the result is not accurate, therefore this may
not work.

Check free space instead, and loosen the critirion to allow the
free space within the range of QOS threshold.

Fixes: 6a7e36a787eb ("lustre: lmv: qos stay on current MDT if less full")
WC-bug-id: https://jira.whamcloud.com/browse/LU-14762
Lustre-commit: 002c2a80266b23c1 ("LU-14762 lmv: compare space to mkdir on parent MDT")
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/43997
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 fs/lustre/include/lu_object.h     |  3 ++-
 fs/lustre/lmv/lmv_obd.c           | 17 ++++++++++-------
 fs/lustre/obdclass/lu_tgt_descs.c | 11 ++++++-----
 3 files changed, 18 insertions(+), 13 deletions(-)

diff --git a/fs/lustre/include/lu_object.h b/fs/lustre/include/lu_object.h
index bbc4533..84e0489 100644
--- a/fs/lustre/include/lu_object.h
+++ b/fs/lustre/include/lu_object.h
@@ -1434,7 +1434,7 @@ struct lu_svr_qos {
 	struct obd_uuid		 lsq_uuid;	/* ptlrpc's c_remote_uuid */
 	struct list_head	 lsq_svr_list;	/* link to lq_svr_list */
 	u64			 lsq_bavail;	/* total bytes avail on svr */
-	u64			 lsq_iavail;	/* tital inode avail on svr */
+	u64			 lsq_iavail;	/* total inode avail on svr */
 	u64			 lsq_penalty;	/* current penalty */
 	u64			 lsq_penalty_per_obj; /* penalty decrease
 						       * every obj
@@ -1451,6 +1451,7 @@ struct lu_tgt_qos {
 	u64			 ltq_penalty_per_obj; /* penalty decrease
 						       * every obj
 						       */
+	u64			 ltq_avail;	/* bytes/inode avail */
 	u64			 ltq_weight;	/* net weighting */
 	time64_t		 ltq_used;	/* last used time, seconds */
 	bool			 ltq_usable:1;	/* usable for striping */
diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c
index ac88d20..2f84028 100644
--- a/fs/lustre/lmv/lmv_obd.c
+++ b/fs/lustre/lmv/lmv_obd.c
@@ -1430,6 +1430,7 @@ static int lmv_close(struct obd_export *exp, struct md_op_data *op_data,
 static struct lu_tgt_desc *lmv_locate_tgt_qos(struct lmv_obd *lmv, u32 *mdt)
 {
 	struct lu_tgt_desc *tgt, *cur = NULL;
+	u64 total_avail = 0;
 	u64 total_weight = 0;
 	u64 cur_weight = 0;
 	int total_usable = 0;
@@ -1460,23 +1461,25 @@ static struct lu_tgt_desc *lmv_locate_tgt_qos(struct lmv_obd *lmv, u32 *mdt)
 
 		tgt->ltd_qos.ltq_usable = 1;
 		lu_tgt_qos_weight_calc(tgt);
-		if (tgt->ltd_index == *mdt) {
+		if (tgt->ltd_index == *mdt)
 			cur = tgt;
-			cur_weight = tgt->ltd_qos.ltq_weight;
-		}
+		total_avail += tgt->ltd_qos.ltq_avail;
 		total_weight += tgt->ltd_qos.ltq_weight;
 		total_usable++;
 	}
 
-	/* if current MDT has higher-than-average space, stay on same MDT */
-	rand = total_weight / total_usable;
-	if (cur_weight >= rand) {
+	/* if current MDT has above-average space, within range of the QOS
+	 * threshold, stay on the same MDT to avoid creating needless remote
+	 * MDT directories.
+	 */
+	rand = total_avail * (256 - lmv->lmv_qos.lq_threshold_rr) /
+	       (total_usable * 256);
+	if (cur && cur->ltd_qos.ltq_avail >= rand) {
 		tgt = cur;
 		rc = 0;
 		goto unlock;
 	}
 
-	cur_weight = 0;
 	rand = lu_prandom_u64_max(total_weight);
 
 	lmv_foreach_connected_tgt(lmv, tgt) {
diff --git a/fs/lustre/obdclass/lu_tgt_descs.c b/fs/lustre/obdclass/lu_tgt_descs.c
index 2a2b30a..935cff6 100644
--- a/fs/lustre/obdclass/lu_tgt_descs.c
+++ b/fs/lustre/obdclass/lu_tgt_descs.c
@@ -220,14 +220,15 @@ static inline u64 tgt_statfs_iavail(struct lu_tgt_desc *tgt)
 void lu_tgt_qos_weight_calc(struct lu_tgt_desc *tgt)
 {
 	struct lu_tgt_qos *ltq = &tgt->ltd_qos;
-	u64 temp, temp2;
+	u64 penalty;
 
-	temp = (tgt_statfs_bavail(tgt) >> 16) * (tgt_statfs_iavail(tgt) >> 8);
-	temp2 = ltq->ltq_penalty + ltq->ltq_svr->lsq_penalty;
-	if (temp < temp2)
+	ltq->ltq_avail = (tgt_statfs_bavail(tgt) >> 16) *
+			 (tgt_statfs_iavail(tgt) >> 8);
+	penalty = ltq->ltq_penalty + ltq->ltq_svr->lsq_penalty;
+	if (ltq->ltq_avail < penalty)
 		ltq->ltq_weight = 0;
 	else
-		ltq->ltq_weight = temp - temp2;
+		ltq->ltq_weight = ltq->ltq_avail - penalty;
 }
 EXPORT_SYMBOL(lu_tgt_qos_weight_calc);
 
-- 
1.8.3.1

_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

  parent reply	other threads:[~2021-07-19 12:32 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-19 12:31 [lustre-devel] [PATCH 00/18] lustre: sync to OpenSFS as of July 18, 2021 James Simmons
2021-07-19 12:31 ` [lustre-devel] [PATCH 01/18] lustre: statahead: update task management code James Simmons
2021-07-19 12:31 ` [lustre-devel] [PATCH 02/18] lustre: llite: simplify callback handling for async getattr James Simmons
2021-07-19 12:31 ` [lustre-devel] [PATCH 03/18] lustre: uapi: per-user changelog names and mask James Simmons
2021-07-19 12:31 ` [lustre-devel] [PATCH 04/18] lnet: Correct peer NI recovery age out calculation James Simmons
2021-07-19 12:32 ` James Simmons [this message]
2021-07-19 12:32 ` [lustre-devel] [PATCH 06/18] lnet: annotate LNET_WIRE_HANDLE_COOKIE_NONE as u64 James Simmons
2021-07-19 12:32 ` [lustre-devel] [PATCH 07/18] lnet: libcfs: Add checksum speed under /sys/fs James Simmons
2021-07-19 12:32 ` [lustre-devel] [PATCH 08/18] lnet: use ni fatal error when calculating net health James Simmons
2021-07-19 12:32 ` [lustre-devel] [PATCH 09/18] lustre: quota: add get/set project support for non-dir/file James Simmons
2021-07-19 12:32 ` [lustre-devel] [PATCH 10/18] lustre: readahead: fix to reserve min pages James Simmons
2021-07-19 12:32 ` [lustre-devel] [PATCH 11/18] lnet: RMDA infrastructure updates James Simmons
2021-07-19 12:32 ` [lustre-devel] [PATCH 12/18] lnet: o2iblnd: Move racy NULL assignment James Simmons
2021-07-19 12:32 ` [lustre-devel] [PATCH 13/18] lnet: o2iblnd: Avoid double posting invalidate James Simmons
2021-07-19 12:32 ` [lustre-devel] [PATCH 14/18] lustre: quota: nodemap squashed root cannot bypass quota James Simmons
2021-07-19 12:32 ` [lustre-devel] [PATCH 15/18] lustre: llite: reset pfid after dir migration James Simmons
2021-07-19 12:32 ` [lustre-devel] [PATCH 16/18] lustre: llite: failed ASSERTION(ldlm_has_layout(lock)) James Simmons
2021-07-19 12:32 ` [lustre-devel] [PATCH 17/18] lustre: pcc: introducing OBD_CONNECT2_PCCRO flag James Simmons
2021-07-19 12:32 ` [lustre-devel] [PATCH 18/18] lustre: sec: migrate/extend/split on encrypted file James Simmons

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1626697933-6971-6-git-send-email-jsimmons@infradead.org \
    --to=jsimmons@infradead.org \
    --cc=adilger@whamcloud.com \
    --cc=green@whamcloud.com \
    --cc=lai.siyao@whamcloud.com \
    --cc=lustre-devel@lists.lustre.org \
    --cc=neilb@suse.de \
    --subject='Re: [lustre-devel] [PATCH 05/18] lustre: lmv: compare space to mkdir on parent MDT' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).