From: James Simmons <jsimmons@infradead.org>
To: Andreas Dilger <adilger@whamcloud.com>,
Oleg Drokin <green@whamcloud.com>, NeilBrown <neilb@suse.de>
Cc: Lustre Development List <lustre-devel@lists.lustre.org>
Subject: [lustre-devel] [PATCH 38/39] lustre: lov: correctly set OST obj size
Date: Thu, 21 Jan 2021 12:17:01 -0500 [thread overview]
Message-ID: <1611249422-556-39-git-send-email-jsimmons@infradead.org> (raw)
In-Reply-To: <1611249422-556-1-git-send-email-jsimmons@infradead.org>
From: Bobi Jam <bobijam@whamcloud.com>
When extends a PFL file to a size locating at a boundary of a stripe
in a component, the truncate won't set the size of the OST object
in the prior stripe.
This patch record the prior stripe in
lov_layout_raid0::lo_trunc_stripeno and add the stripe in the
truncate IO and enqueue the lock covering it.
WC-bug-id: https://jira.whamcloud.com/browse/LU-14128
Lustre-commit: 98015004516cad ("LU-14128 lov: correctly set OST obj size")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40581
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
fs/lustre/lov/lov_cl_internal.h | 5 +++
fs/lustre/lov/lov_internal.h | 1 +
fs/lustre/lov/lov_io.c | 97 +++++++++++++++++++++++++++++++++++------
fs/lustre/lov/lov_lock.c | 31 +++++++++----
fs/lustre/lov/lov_object.c | 1 +
fs/lustre/lov/lov_offset.c | 2 +-
6 files changed, 114 insertions(+), 23 deletions(-)
diff --git a/fs/lustre/lov/lov_cl_internal.h b/fs/lustre/lov/lov_cl_internal.h
index 7128224..f86176a 100644
--- a/fs/lustre/lov/lov_cl_internal.h
+++ b/fs/lustre/lov/lov_cl_internal.h
@@ -176,6 +176,11 @@ struct lov_comp_layout_entry_ops {
struct lov_layout_raid0 {
unsigned int lo_nr;
/**
+ * record the stripe no before the truncate size, used for setting OST
+ * object size for truncate. LU-14128.
+ */
+ int lo_trunc_stripeno;
+ /**
* When this is true, lov_object::lo_attr contains
* valid up to date attributes for a top-level
* object. This field is reset to 0 when attributes of
diff --git a/fs/lustre/lov/lov_internal.h b/fs/lustre/lov/lov_internal.h
index 202e4b5..5d726fd 100644
--- a/fs/lustre/lov/lov_internal.h
+++ b/fs/lustre/lov/lov_internal.h
@@ -264,6 +264,7 @@ int lov_merge_lvb_kms(struct lov_stripe_md *lsm, int index,
struct ost_lvb *lvb, u64 *kms_place);
/* lov_offset.c */
+u64 stripe_width(struct lov_stripe_md *lsm, unsigned int index);
u64 lov_stripe_size(struct lov_stripe_md *lsm, int index, u64 ost_size,
int stripeno);
int lov_stripe_offset(struct lov_stripe_md *lsm, int index, u64 lov_off,
diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c
index d4a0c9d..daceab0 100644
--- a/fs/lustre/lov/lov_io.c
+++ b/fs/lustre/lov/lov_io.c
@@ -752,6 +752,24 @@ static u64 lov_offset_mod(u64 val, int delta)
return val;
}
+static int lov_io_add_sub(const struct lu_env *env, struct lov_io *lio,
+ struct lov_io_sub *sub, u64 start, u64 end)
+{
+ int rc;
+
+ end = lov_offset_mod(end, 1);
+ lov_io_sub_inherit(sub, lio, start, end);
+ rc = cl_io_iter_init(sub->sub_env, &sub->sub_io);
+ if (rc != 0) {
+ cl_io_iter_fini(sub->sub_env, &sub->sub_io);
+ return rc;
+ }
+
+ list_add_tail(&sub->sub_linkage, &lio->lis_active);
+
+ return rc;
+}
+
static int lov_io_iter_init(const struct lu_env *env,
const struct cl_io_slice *ios)
{
@@ -768,10 +786,13 @@ static int lov_io_iter_init(const struct lu_env *env,
lov_foreach_io_layout(index, lio, &ext) {
struct lov_layout_entry *le = lov_entry(lio->lis_object, index);
struct lov_layout_raid0 *r0 = &le->lle_raid0;
+ bool tested_trunc_stripe = false;
int stripe;
u64 start;
u64 end;
+ r0->lo_trunc_stripeno = -1;
+
CDEBUG(D_VFSTRACE, "component[%d] flags %#x\n",
index, lsm->lsm_entries[index]->lsme_flags);
if (!lsm_entry_inited(lsm, index)) {
@@ -801,28 +822,76 @@ static int lov_io_iter_init(const struct lu_env *env,
continue;
}
- end = lov_offset_mod(end, 1);
+ if (cl_io_is_trunc(ios->cis_io) &&
+ !tested_trunc_stripe) {
+ int prev;
+ u64 tr_start;
+
+ prev = (stripe == 0) ? r0->lo_nr - 1 :
+ stripe - 1;
+ /**
+ * Only involving previous stripe if the
+ * truncate in this component is at the
+ * beginning of this stripe.
+ */
+ tested_trunc_stripe = true;
+ if (ext.e_start <
+ lsm->lsm_entries[index]->lsme_extent.e_start) {
+ /* need previous stripe involvement */
+ r0->lo_trunc_stripeno = prev;
+ } else {
+ tr_start = ext.e_start;
+ tr_start = lov_do_div64(tr_start,
+ stripe_width(lsm, index));
+ /* tr_start %= stripe_swidth */
+ if (tr_start == stripe * lsm->lsm_entries[index]->lsme_stripe_size)
+ r0->lo_trunc_stripeno = prev;
+ }
+ }
+
+ /* if the last stripe is the trunc stripeno */
+ if (r0->lo_trunc_stripeno == stripe)
+ r0->lo_trunc_stripeno = -1;
+
sub = lov_sub_get(env, lio,
lov_comp_index(index, stripe));
- if (IS_ERR(sub)) {
- rc = PTR_ERR(sub);
- break;
- }
+ if (IS_ERR(sub))
+ return PTR_ERR(sub);
- lov_io_sub_inherit(sub, lio, start, end);
- rc = cl_io_iter_init(sub->sub_env, &sub->sub_io);
- if (rc) {
- cl_io_iter_fini(sub->sub_env, &sub->sub_io);
+ rc = lov_io_add_sub(env, lio, sub, start, end);
+ if (rc)
break;
+ }
+ if (rc != 0)
+ break;
+
+ if (r0->lo_trunc_stripeno != -1) {
+ stripe = r0->lo_trunc_stripeno;
+ if (unlikely(!r0->lo_sub[stripe])) {
+ r0->lo_trunc_stripeno = -1;
+ continue;
}
+ sub = lov_sub_get(env, lio,
+ lov_comp_index(index, stripe));
+ if (IS_ERR(sub))
+ return PTR_ERR(sub);
- CDEBUG(D_VFSTRACE, "shrink: %d [%llu, %llu)\n",
- stripe, start, end);
+ /**
+ * the prev sub could be used by another truncate, we'd
+ * skip it. LU-14128 happends when expand truncate +
+ * read get wrong kms.
+ */
+ if (!list_empty(&sub->sub_linkage)) {
+ r0->lo_trunc_stripeno = -1;
+ continue;
+ }
- list_add_tail(&sub->sub_linkage, &lio->lis_active);
+ (void)lov_stripe_intersects(lsm, index, stripe, &ext,
+ &start, &end);
+ rc = lov_io_add_sub(env, lio, sub, start, end);
+ if (rc != 0)
+ break;
}
- if (rc)
- break;
}
return rc;
}
diff --git a/fs/lustre/lov/lov_lock.c b/fs/lustre/lov/lov_lock.c
index 7dae13f..c79f728 100644
--- a/fs/lustre/lov/lov_lock.c
+++ b/fs/lustre/lov/lov_lock.c
@@ -111,6 +111,7 @@ static int lov_sublock_init(const struct lu_env *env,
* through already created sub-locks (possibly shared with other top-locks).
*/
static struct lov_lock *lov_lock_sub_init(const struct lu_env *env,
+ const struct cl_io *io,
const struct cl_object *obj,
struct cl_lock *lock)
{
@@ -135,10 +136,14 @@ static struct lov_lock *lov_lock_sub_init(const struct lu_env *env,
struct lov_layout_raid0 *r0 = lov_r0(lov, index);
for (i = 0; i < r0->lo_nr; i++) {
- if (likely(r0->lo_sub[i]) && /* spare layout */
- lov_stripe_intersects(lov->lo_lsm, index, i,
- &ext, &start, &end))
- nr++;
+ if (likely(r0->lo_sub[i])) { /* spare layout */
+ if (lov_stripe_intersects(lov->lo_lsm, index, i,
+ &ext, &start, &end))
+ nr++;
+ else if (cl_io_is_trunc(io) &&
+ r0->lo_trunc_stripeno == i)
+ nr++;
+ }
}
}
/**
@@ -160,12 +165,22 @@ static struct lov_lock *lov_lock_sub_init(const struct lu_env *env,
for (i = 0; i < r0->lo_nr; ++i) {
struct lov_lock_sub *lls = &lovlck->lls_sub[nr];
struct cl_lock_descr *descr = &lls->sub_lock.cll_descr;
+ bool intersect = false;
- if (unlikely(!r0->lo_sub[i]) ||
- !lov_stripe_intersects(lov->lo_lsm, index, i,
- &ext, &start, &end))
+ if (unlikely(!r0->lo_sub[i]))
continue;
+ intersect = lov_stripe_intersects(lov->lo_lsm, index, i,
+ &ext, &start, &end);
+ if (intersect)
+ goto init_sublock;
+
+ if (cl_io_is_trunc(io) && i == r0->lo_trunc_stripeno)
+ goto init_sublock;
+
+ continue;
+
+init_sublock:
LASSERT(!descr->cld_obj);
descr->cld_obj = lovsub2cl(r0->lo_sub[i]);
descr->cld_start = cl_index(descr->cld_obj, start);
@@ -308,7 +323,7 @@ int lov_lock_init_composite(const struct lu_env *env, struct cl_object *obj,
struct lov_lock *lck;
int result = 0;
- lck = lov_lock_sub_init(env, obj, lock);
+ lck = lov_lock_sub_init(env, io, obj, lock);
if (!IS_ERR(lck))
cl_lock_slice_add(lock, &lck->lls_cl, obj, &lov_lock_ops);
else
diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c
index 3fcd342..d9729c8 100644
--- a/fs/lustre/lov/lov_object.c
+++ b/fs/lustre/lov/lov_object.c
@@ -215,6 +215,7 @@ static int lov_init_raid0(const struct lu_env *env, struct lov_device *dev,
spin_lock_init(&r0->lo_sub_lock);
r0->lo_nr = lse->lsme_stripe_count;
+ r0->lo_trunc_stripeno = -1;
flags = memalloc_nofs_save();
r0->lo_sub = kvmalloc_array(r0->lo_nr, sizeof(r0->lo_sub[0]),
diff --git a/fs/lustre/lov/lov_offset.c b/fs/lustre/lov/lov_offset.c
index ca763af..2493331 100644
--- a/fs/lustre/lov/lov_offset.c
+++ b/fs/lustre/lov/lov_offset.c
@@ -37,7 +37,7 @@
#include "lov_internal.h"
-static u64 stripe_width(struct lov_stripe_md *lsm, unsigned int index)
+u64 stripe_width(struct lov_stripe_md *lsm, unsigned int index)
{
struct lov_stripe_md_entry *entry = lsm->lsm_entries[index];
--
1.8.3.1
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
next prev parent reply other threads:[~2021-01-21 17:18 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-21 17:16 [lustre-devel] [PATCH 00/39] lustre: update to latest OpenSFS version as of Jan 21 2021 James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 01/39] lustre: ldlm: page discard speedup James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 02/39] lustre: ptlrpc: fixes for RCU-related stalls James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 03/39] lustre: ldlm: Do not wait for lock replay sending if import dsconnected James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 04/39] lustre: ldlm: Do not hang if recovery restarted during lock replay James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 05/39] lnet: Correct handling of NETWORK_TIMEOUT status James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 06/39] lnet: Introduce constant for net ID of LNET_NID_ANY James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 07/39] lustre: ldlm: Don't re-enqueue glimpse lock on read James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 08/39] lustre: osc: prevent overflow of o_dropped James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 09/39] lustre: llite: fix client evicition with DIO James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 10/39] lustre: Use vfree_atomic instead of vfree James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 11/39] lnet: lnd: Use NETWORK_TIMEOUT for txs on ibp_tx_queue James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 12/39] lnet: lnd: Use NETWORK_TIMEOUT for some conn failures James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 13/39] lustre: llite: allow DIO with unaligned IO count James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 14/39] lustre: osc: skip 0 row for rpc_stats James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 15/39] lustre: quota: df should return projid-specific values James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 16/39] lnet: discard the callback James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 17/39] lustre: llite: try to improve mmap performance James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 18/39] lnet: Introduce lnet_recovery_limit parameter James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 19/39] lustre: mdc: avoid easize set to 0 James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 20/39] lustre: lmv: optimize dir shard revalidate James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 21/39] lustre: ldlm: osc_object_ast_clear() is called for mdc object on eviction James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 22/39] lustre: uapi: fix compatibility for LL_IOC_MDC_GETINFO James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 23/39] lustre: llite: don't check layout info for page discard James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 24/39] lustre: update version to 2.13.57 James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 25/39] lnet: o2iblnd: retry qp creation with reduced queue depth James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 26/39] lustre: lov: fix SEEK_HOLE calcs at component end James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 27/39] lustre: lov: instantiate components layout for fallocate James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 28/39] lustre: dom: non-blocking enqueue for DOM locks James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 29/39] lustre: llite: fiemap set flags for encrypted files James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 30/39] lustre: ldlm: don't compute sumsq for pool stats James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 31/39] lustre: lov: FIEMAP support for PFL and FLR file James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 32/39] lustre: mdc: process changelogs_catalog from the oldest rec James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 33/39] lustre: ldlm: Use req_mode while lock cleanup James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 34/39] lnet: socklnd: announce deprecation of 'use_tcp_bonding' James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 35/39] lnet: o2iblnd: remove FMR-pool support James Simmons
2021-01-21 17:16 ` [lustre-devel] [PATCH 36/39] lustre: llite: return EOPNOTSUPP if fallocate is not supported James Simmons
2021-01-21 17:17 ` [lustre-devel] [PATCH 37/39] lnet: use an unbound cred in kiblnd_resolve_addr() James Simmons
2021-01-21 17:17 ` James Simmons [this message]
2021-01-21 17:17 ` [lustre-devel] [PATCH 39/39] lustre: cksum: add lprocfs checksum support in MDC/MDT James Simmons
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1611249422-556-39-git-send-email-jsimmons@infradead.org \
--to=jsimmons@infradead.org \
--cc=adilger@whamcloud.com \
--cc=green@whamcloud.com \
--cc=lustre-devel@lists.lustre.org \
--cc=neilb@suse.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).