* [lustre-devel] [PATCH 1/9] lustre: llite: send file mode with rename RPC
2021-02-09 0:54 [lustre-devel] [PATCH 0/9] Sync to OpenSFS as of Feb 8, 2021 James Simmons
@ 2021-02-09 0:54 ` James Simmons
2021-02-09 0:54 ` [lustre-devel] [PATCH 2/9] lustre: flr: skip unknown FLR component types James Simmons
` (7 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: James Simmons @ 2021-02-09 0:54 UTC (permalink / raw)
To: Andreas Dilger, Oleg Drokin, NeilBrown; +Cc: Lustre Development List
From: Andreas Dilger <adilger@whamcloud.com>
In preparation for parallel rename operations, send renamed file mode
to the MDS in order to allow the rename locking to be more efficient.
WC-bug-id: https://jira.whamcloud.com/browse/LU-12125
Lustre-commit: 47576dc68cd9006 ("LU-12125 llite: send file mode with rename RPC")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41184
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
fs/lustre/llite/namei.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/fs/lustre/llite/namei.c b/fs/lustre/llite/namei.c
index b24f097..1095fa9 100644
--- a/fs/lustre/llite/namei.c
+++ b/fs/lustre/llite/namei.c
@@ -1745,6 +1745,7 @@ static int ll_rename(struct inode *src, struct dentry *src_dchild,
struct ll_sb_info *sbi = ll_i2sbi(src);
struct md_op_data *op_data;
ktime_t kstart = ktime_get();
+ umode_t mode = 0;
int err;
if (flags)
@@ -1762,7 +1763,13 @@ static int ll_rename(struct inode *src, struct dentry *src_dchild,
if (err)
return err;
- op_data = ll_prep_md_op_data(NULL, src, tgt, NULL, 0, 0,
+ if (src_dchild->d_inode)
+ mode = src_dchild->d_inode->i_mode;
+
+ if (tgt_dchild->d_inode)
+ mode = tgt_dchild->d_inode->i_mode;
+
+ op_data = ll_prep_md_op_data(NULL, src, tgt, NULL, 0, mode,
LUSTRE_OPC_ANY, NULL);
if (IS_ERR(op_data))
return PTR_ERR(op_data);
--
1.8.3.1
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
^ permalink raw reply related [flat|nested] 10+ messages in thread* [lustre-devel] [PATCH 2/9] lustre: flr: skip unknown FLR component types
2021-02-09 0:54 [lustre-devel] [PATCH 0/9] Sync to OpenSFS as of Feb 8, 2021 James Simmons
2021-02-09 0:54 ` [lustre-devel] [PATCH 1/9] lustre: llite: send file mode with rename RPC James Simmons
@ 2021-02-09 0:54 ` James Simmons
2021-02-09 0:54 ` [lustre-devel] [PATCH 3/9] lustre: obdclass: connect vs disconnect race James Simmons
` (6 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: James Simmons @ 2021-02-09 0:54 UTC (permalink / raw)
To: Andreas Dilger, Oleg Drokin, NeilBrown; +Cc: Lustre Development List
From: Qian Yingjin <qian@ddn.com>
Currently, in lov_init_composite() it will quit with error when
reading an unknown LOV pattern.
Since FLR will be used for upcoming new features, like PCC-RO,
an old client should be able to read the old format of the
component types, ignore and skip the new types of FLR component.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13602
Lustre-commit: 61a002cd8631ecd ("LU-13602 flr: skip unknown FLR component types")
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/39513
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
fs/lustre/lov/lov_cl_internal.h | 7 +++-
fs/lustre/lov/lov_ea.c | 66 +++++++++++++++++++++++++--------
fs/lustre/lov/lov_internal.h | 15 +++++++-
fs/lustre/lov/lov_io.c | 14 ++++++-
fs/lustre/lov/lov_object.c | 22 ++++++++++-
include/uapi/linux/lustre/lustre_user.h | 1 +
6 files changed, 102 insertions(+), 23 deletions(-)
diff --git a/fs/lustre/lov/lov_cl_internal.h b/fs/lustre/lov/lov_cl_internal.h
index f86176a..e9ef5aa 100644
--- a/fs/lustre/lov/lov_cl_internal.h
+++ b/fs/lustre/lov/lov_cl_internal.h
@@ -151,7 +151,8 @@ static inline char *llt2str(enum lov_layout_type llt)
static inline u32 lov_entry_type(struct lov_stripe_md_entry *lsme)
{
if ((lov_pattern(lsme->lsme_pattern) & LOV_PATTERN_RAID0) ||
- (lov_pattern(lsme->lsme_pattern) == LOV_PATTERN_MDT))
+ (lov_pattern(lsme->lsme_pattern) == LOV_PATTERN_MDT) ||
+ (lov_pattern(lsme->lsme_pattern) == LOV_PATTERN_FOREIGN))
return lov_pattern(lsme->lsme_pattern &
~LOV_PATTERN_OVERSTRIPING);
return 0;
@@ -239,9 +240,11 @@ struct lov_mirror_entry {
unsigned short lre_mirror_id;
unsigned short lre_preferred:1,
lre_stale:1, /* set if any components is stale */
- lre_valid:1; /* set if at least one of components
+ lre_valid:1, /* set if at least one of components
* in this mirror is valid
*/
+ lre_foreign:1; /* set if it is a foreign component */
+
unsigned short lre_start; /* index to lo_entries, start index of
* this mirror
*/
diff --git a/fs/lustre/lov/lov_ea.c b/fs/lustre/lov/lov_ea.c
index 13f47ee..75a19a4 100644
--- a/fs/lustre/lov/lov_ea.c
+++ b/fs/lustre/lov/lov_ea.c
@@ -125,9 +125,22 @@ static int lsm_lmm_verify_v1v3(struct lov_mds_md *lmm, size_t lmm_size,
static void lsme_free(struct lov_stripe_md_entry *lsme)
{
- unsigned int stripe_count = lsme->lsme_stripe_count;
+ unsigned int stripe_count;
unsigned int i;
+ if (lsme->lsme_magic == LOV_MAGIC_FOREIGN) {
+ /*
+ * TODO: In addition to HSM foreign layout, It needs to add
+ * support for other kinds of foreign layout types such as
+ * DAOS, S3. When add these supports, it will use non-inline
+ * @lov_hsm_base to store layout information, and need to
+ * free extra allocated buffer.
+ */
+ kvfree(lsme);
+ return;
+ }
+
+ stripe_count = lsme->lsme_stripe_count;
if (!lsme_inited(lsme) ||
lsme->lsme_pattern & LOV_PATTERN_F_RELEASED)
stripe_count = 0;
@@ -183,8 +196,7 @@ void lsm_free(struct lov_stripe_md *lsm)
else
stripe_count = le16_to_cpu(lmm->lmm_stripe_count);
- if (buf_size < (magic == LOV_MAGIC_V1 ? sizeof(struct lov_mds_md_v1) :
- sizeof(struct lov_mds_md_v3))) {
+ if (buf_size < lov_mds_md_size(stripe_count, magic)) {
CERROR("LOV EA %s too small: %zu, need %u\n",
magic == LOV_MAGIC_V1 ? "V1" : "V3", buf_size,
lov_mds_md_size(stripe_count, magic == LOV_MAGIC_V1 ?
@@ -407,36 +419,58 @@ static int lsm_verify_comp_md_v1(struct lov_comp_md_v1 *lcm,
}
static struct lov_stripe_md_entry *
-lsme_unpack_comp(struct lov_obd *lov, struct lov_mds_md *lmm,
- size_t lmm_buf_size, bool inited, loff_t *maxbytes)
+lsme_unpack_foreign(struct lov_obd *lov, void *buf, size_t buf_size,
+ bool inited, loff_t *maxbytes)
{
- unsigned int stripe_count;
- unsigned int magic;
+ struct lov_stripe_md_entry *lsme;
+ struct lov_foreign_md *lfm = buf;
+ u32 magic;
- stripe_count = le16_to_cpu(lmm->lmm_stripe_count);
- if (stripe_count == 0 &&
- lov_pattern(le32_to_cpu(lmm->lmm_pattern)) != LOV_PATTERN_MDT)
+ magic = le32_to_cpu(lfm->lfm_magic);
+ if (magic != LOV_MAGIC_FOREIGN)
return ERR_PTR(-EINVAL);
- /* un-instantiated lmm contains no ost id info, i.e. lov_ost_data_v1 */
- if (!inited)
- stripe_count = 0;
+ lsme = kvzalloc(sizeof(*lsme), GFP_NOFS);
+ if (!lsme)
+ return ERR_PTR(-ENOMEM);
+
+ lsme->lsme_magic = magic;
+ lsme->lsme_pattern = LOV_PATTERN_FOREIGN;
+ lsme->lsme_flags = 0;
+
+ if (maxbytes)
+ *maxbytes = MAX_LFS_FILESIZE;
+
+ return lsme;
+}
+
+static struct lov_stripe_md_entry *
+lsme_unpack_comp(struct lov_obd *lov, struct lov_mds_md *lmm,
+ size_t lmm_buf_size, bool inited, loff_t *maxbytes)
+{
+ unsigned int magic;
magic = le32_to_cpu(lmm->lmm_magic);
- if (magic != LOV_MAGIC_V1 && magic != LOV_MAGIC_V3)
+ if (magic != LOV_MAGIC_V1 && magic != LOV_MAGIC_V3 &&
+ magic != LOV_MAGIC_FOREIGN)
return ERR_PTR(-EINVAL);
- if (lmm_buf_size < lov_mds_md_size(stripe_count, magic))
+ if (magic != LOV_MAGIC_FOREIGN &&
+ le16_to_cpu(lmm->lmm_stripe_count) == 0 &&
+ lov_pattern(le32_to_cpu(lmm->lmm_pattern)) != LOV_PATTERN_MDT)
return ERR_PTR(-EINVAL);
if (magic == LOV_MAGIC_V1) {
return lsme_unpack(lov, lmm, lmm_buf_size, NULL,
inited, lmm->lmm_objects, maxbytes);
- } else {
+ } else if (magic == LOV_MAGIC_V3) {
struct lov_mds_md_v3 *lmm3 = (struct lov_mds_md_v3 *)lmm;
return lsme_unpack(lov, lmm, lmm_buf_size, lmm3->lmm_pool_name,
inited, lmm3->lmm_objects, maxbytes);
+ } else { /* LOV_MAGIC_FOREIGN */
+ return lsme_unpack_foreign(lov, lmm, lmm_buf_size,
+ inited, maxbytes);
}
}
diff --git a/fs/lustre/lov/lov_internal.h b/fs/lustre/lov/lov_internal.h
index 5d726fd..81adce4 100644
--- a/fs/lustre/lov/lov_internal.h
+++ b/fs/lustre/lov/lov_internal.h
@@ -100,6 +100,17 @@ struct lov_stripe_md {
#define lsm_foreign(lsm) (lsm->lsm_entries[0])
+static inline bool lsme_is_foreign(const struct lov_stripe_md_entry *lsme)
+{
+ return lsme->lsme_magic == LOV_MAGIC_FOREIGN;
+}
+
+static inline bool lsm_entry_is_foreign(const struct lov_stripe_md *lsm,
+ int index)
+{
+ return lsme_is_foreign(lsm->lsm_entries[index]);
+}
+
static inline bool lsme_inited(const struct lov_stripe_md_entry *lsme)
{
return lsme->lsme_flags & LCME_FL_INIT;
@@ -130,7 +141,8 @@ static inline size_t lov_comp_md_size(const struct lov_stripe_md *lsm)
LASSERT(lsm->lsm_magic == LOV_MAGIC_COMP_V1);
- size = sizeof(struct lov_comp_md_v1);
+ size = sizeof(struct lov_comp_md_v1) +
+ sizeof(struct lov_comp_md_entry_v1) * lsm->lsm_entry_count;
for (entry = 0; entry < lsm->lsm_entry_count; entry++) {
u16 stripe_count;
@@ -141,7 +153,6 @@ static inline size_t lov_comp_md_size(const struct lov_stripe_md *lsm)
else
stripe_count = 0;
- size += sizeof(*lsme);
size += lov_mds_md_size(stripe_count,
lsme->lsme_magic);
}
diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c
index daceab0..c9600bc 100644
--- a/fs/lustre/lov/lov_io.c
+++ b/fs/lustre/lov/lov_io.c
@@ -398,6 +398,9 @@ static int lov_io_mirror_init(struct lov_io *lio, struct lov_object *obj,
if (!lre->lre_valid)
continue;
+ if (lre->lre_foreign)
+ continue;
+
lov_foreach_mirror_layout_entry(obj, lle, lre) {
if (!lle->lle_valid)
continue;
@@ -802,6 +805,9 @@ static int lov_io_iter_init(const struct lu_env *env,
continue;
}
+ if (lsm_entry_is_foreign(lsm, index))
+ continue;
+
if (!le->lle_valid && !ios->cis_io->ci_designated_mirror) {
CERROR("I/O to invalid component: %d, mirror: %d\n",
index, lio->lis_mirror_index);
@@ -931,6 +937,9 @@ static int lov_io_rw_iter_init(const struct lu_env *env,
lse = lov_lse(lio->lis_object, index);
+ if (lsme_is_foreign(lse))
+ return -EINVAL;
+
next = MAX_LFS_FILESIZE;
if (lse->lsme_stripe_count > 1) {
unsigned long ssize = lse->lsme_stripe_size;
@@ -1107,7 +1116,8 @@ static int lov_io_read_ahead(const struct lu_env *env,
offset = cl_offset(obj, start);
index = lov_io_layout_at(lio, offset);
- if (index < 0 || !lsm_entry_inited(loo->lo_lsm, index))
+ if (index < 0 || !lsm_entry_inited(loo->lo_lsm, index) ||
+ lsm_entry_is_foreign(loo->lo_lsm, index))
return -ENODATA;
/* avoid readahead to expand to stale components */
@@ -1760,6 +1770,8 @@ int lov_io_layout_at(struct lov_io *lio, u64 offset)
for (i = start_index; i <= end_index; i++) {
struct lov_layout_entry *lle = lov_entry(lov, i);
+ LASSERT(!lsme_is_foreign(lle->lle_lsme));
+
if ((offset >= lle->lle_extent->e_start &&
offset < lle->lle_extent->e_end) ||
(offset == OBD_OBJECT_EOF &&
diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c
index d9729c8..5d0e257 100644
--- a/fs/lustre/lov/lov_object.c
+++ b/fs/lustre/lov/lov_object.c
@@ -681,6 +681,9 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev,
}
lle->lle_comp_ops = &dom_ops;
break;
+ case LOV_PATTERN_FOREIGN:
+ lle->lle_comp_ops = NULL;
+ break;
default:
CERROR("%s: unknown composite layout entry type %i\n",
lov2obd(dev->ld_lov)->obd_name,
@@ -700,6 +703,8 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev,
if (mirror_id == lre->lre_mirror_id) {
lre->lre_valid |= lle->lle_valid;
lre->lre_stale |= !lle->lle_valid;
+ lre->lre_foreign |=
+ lsme_is_foreign(lle->lle_lsme);
lre->lre_end = i;
continue;
}
@@ -722,6 +727,7 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev,
LCME_FL_PREF_RD);
lre->lre_valid = lle->lle_valid;
lre->lre_stale = !lle->lle_valid;
+ lre->lre_foreign = lsme_is_foreign(lle->lle_lsme);
}
/* sanity check for FLR */
@@ -746,6 +752,9 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev,
if (!lsme_inited(lle->lle_lsme))
continue;
+ if (lsme_is_foreign(lle->lle_lsme))
+ continue;
+
result = lle->lle_comp_ops->lco_init(env, dev, lov, index,
conf, lle);
if (result < 0)
@@ -771,6 +780,9 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev,
if (lre->lre_stale)
continue;
+ if (lre->lre_foreign)
+ continue;
+
mirror_count++; /* valid mirror */
if (lre->lre_preferred || comp->lo_preferred_mirror < 0)
@@ -847,8 +859,12 @@ static int lov_delete_composite(const struct lu_env *env,
lov_layout_wait(env, lov);
if (comp->lo_entries)
- lov_foreach_layout_entry(lov, entry)
+ lov_foreach_layout_entry(lov, entry) {
+ if (lsme_is_foreign(entry->lle_lsme))
+ continue;
+
lov_delete_raid0(env, lov, entry);
+ }
return 0;
}
@@ -921,7 +937,9 @@ static int lov_print_composite(const struct lu_env *env, void *cookie,
lse->lsme_id, lse->lsme_pattern, lse->lsme_layout_gen,
lse->lsme_flags, lse->lsme_stripe_count,
lse->lsme_stripe_size);
- lov_print_raid0(env, cookie, p, lle);
+
+ if (!lsme_is_foreign(lse))
+ lov_print_raid0(env, cookie, p, lle);
}
return 0;
diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h
index 835ffce..0f195a4 100644
--- a/include/uapi/linux/lustre/lustre_user.h
+++ b/include/uapi/linux/lustre/lustre_user.h
@@ -422,6 +422,7 @@ struct ll_ioc_lease_id {
#define LOV_PATTERN_RAID1 0x002
#define LOV_PATTERN_MDT 0x100
#define LOV_PATTERN_OVERSTRIPING 0x200
+#define LOV_PATTERN_FOREIGN 0x400
#define LOV_PATTERN_F_MASK 0xffff0000
#define LOV_PATTERN_F_HOLE 0x40000000 /* there is hole in LOV EA */
--
1.8.3.1
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
^ permalink raw reply related [flat|nested] 10+ messages in thread* [lustre-devel] [PATCH 3/9] lustre: obdclass: connect vs disconnect race
2021-02-09 0:54 [lustre-devel] [PATCH 0/9] Sync to OpenSFS as of Feb 8, 2021 James Simmons
2021-02-09 0:54 ` [lustre-devel] [PATCH 1/9] lustre: llite: send file mode with rename RPC James Simmons
2021-02-09 0:54 ` [lustre-devel] [PATCH 2/9] lustre: flr: skip unknown FLR component types James Simmons
@ 2021-02-09 0:54 ` James Simmons
2021-02-09 0:54 ` [lustre-devel] [PATCH 4/9] lustre: osc: correctly update size/kms for fallocate James Simmons
` (5 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: James Simmons @ 2021-02-09 0:54 UTC (permalink / raw)
To: Andreas Dilger, Oleg Drokin, NeilBrown
Cc: Wang Shilong, Lustre Development List
From: Wang Shilong <wshilong@ddn.com>
There might be a possible race if setup (connect)
and cleanup (disconnect) are tangled together(similar
comments in osc_disconnect()):
Thread1: Thread2:
connecting class_cleanup
ptlrpc_connect_interpret
obd->obd_setup = 0
obd_import_event
if (obd->obd_set_up)
osc_init_grant() /*skipped*/
ptlrpc_activate_import..
And If RPC was waked up and send out before
class_disconnect_exports(), It might hit divide zero crash
in osc_announce_cached() because @cl_max_extent_pages is zero.
The problem is we clear @obd_setup too early, It should be cleared
when OBD is really shutdown.
Fixes: cfd750 ("lustre: obdclass: obd_device improvement")
WC-bug-id: https://jira.whamcloud.com/browse/LU-14283
Lustre-commit: 4f689a30fb5d870 ("LU-14283 obdclass: connect vs disconnect race")
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/41256
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
fs/lustre/obdclass/obd_config.c | 4 ----
1 file changed, 4 deletions(-)
diff --git a/fs/lustre/obdclass/obd_config.c b/fs/lustre/obdclass/obd_config.c
index 09dee7b..8f062bb 100644
--- a/fs/lustre/obdclass/obd_config.c
+++ b/fs/lustre/obdclass/obd_config.c
@@ -467,10 +467,6 @@ static int class_cleanup(struct obd_device *obd, struct lustre_cfg *lcfg)
}
/* Leave this on forever */
obd->obd_stopping = 1;
- /* function can't return error after that point, so clear setup flag
- * as early as possible to avoid finding via obd_devs / hash
- */
- obd->obd_set_up = 0;
spin_unlock(&obd->obd_dev_lock);
while (obd->obd_conn_inprogress > 0)
--
1.8.3.1
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
^ permalink raw reply related [flat|nested] 10+ messages in thread* [lustre-devel] [PATCH 4/9] lustre: osc: correctly update size/kms for fallocate
2021-02-09 0:54 [lustre-devel] [PATCH 0/9] Sync to OpenSFS as of Feb 8, 2021 James Simmons
` (2 preceding siblings ...)
2021-02-09 0:54 ` [lustre-devel] [PATCH 3/9] lustre: obdclass: connect vs disconnect race James Simmons
@ 2021-02-09 0:54 ` James Simmons
2021-02-09 0:54 ` [lustre-devel] [PATCH 5/9] lustre: ldlm: don't change GROUP lock GID on client James Simmons
` (4 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: James Simmons @ 2021-02-09 0:54 UTC (permalink / raw)
To: Andreas Dilger, Oleg Drokin, NeilBrown; +Cc: Lustre Development List
From: Bobi Jam <bobijam@whamcloud.com>
* fallocate chose oa->o_size for falloc_offset and o->o_blocks for
falloc_end, but forgot to change attr->cat_size and attr->cat_kms
to use sa_attr.lvb_size to update osc's lvb and kms if it expands
the file's size.
Other setattr IO uses @size (sa_falloc_offset in fallocate case) to
update the lvb and kms.
* lock request extent for fallocate should be
[sa_falloc_offset, sa_falloc_end)
* calculate sa_attr.lvb_size correctly for osc objects
(lov_io_sub_inherit())
Fixes: d748d2ffa1bc ("lustre: fallocate: Implement fallocate preallocate operation")
WC-bug-id: https://jira.whamcloud.com/browse/LU-14326
Lustre-commit: 43979e4e257e78d ("LU-14326 osc: correctly update size/kms for fallocate")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41272
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
fs/lustre/llite/vvp_io.c | 2 +-
fs/lustre/lov/lov_io.c | 11 +++++------
fs/lustre/osc/osc_cache.c | 3 ++-
fs/lustre/osc/osc_io.c | 33 +++++++++++++++++----------------
fs/lustre/osc/osc_object.c | 4 +++-
5 files changed, 28 insertions(+), 25 deletions(-)
diff --git a/fs/lustre/llite/vvp_io.c b/fs/lustre/llite/vvp_io.c
index b0b31c37..ac6aef0 100644
--- a/fs/lustre/llite/vvp_io.c
+++ b/fs/lustre/llite/vvp_io.c
@@ -647,7 +647,7 @@ static int vvp_io_setattr_lock(const struct lu_env *env,
enqflags = CEF_DISCARD_DATA;
} else if (cl_io_is_fallocate(io)) {
lock_start = io->u.ci_setattr.sa_falloc_offset;
- lock_end = lock_start + io->u.ci_setattr.sa_attr.lvb_size;
+ lock_end = io->u.ci_setattr.sa_falloc_end;
} else {
unsigned int valid = io->u.ci_setattr.sa_avalid;
diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c
index c9600bc..2297e53 100644
--- a/fs/lustre/lov/lov_io.c
+++ b/fs/lustre/lov/lov_io.c
@@ -677,17 +677,16 @@ static void lov_io_sub_inherit(struct lov_io_sub *sub, struct lov_io *lio,
parent->u.ci_setattr.sa_parent_fid;
/* For SETATTR(fallocate) pass the subtype to lower IO */
io->u.ci_setattr.sa_subtype = parent->u.ci_setattr.sa_subtype;
- if (cl_io_is_trunc(io)) {
+ if (cl_io_is_fallocate(io)) {
+ io->u.ci_setattr.sa_falloc_offset = start;
+ io->u.ci_setattr.sa_falloc_end = end;
+ }
+ if (cl_io_is_trunc(io) || cl_io_is_fallocate(io)) {
loff_t new_size = parent->u.ci_setattr.sa_attr.lvb_size;
new_size = lov_size_to_stripe(lsm, index, new_size,
stripe);
io->u.ci_setattr.sa_attr.lvb_size = new_size;
- } else if (cl_io_is_fallocate(io)) {
- io->u.ci_setattr.sa_falloc_offset = start;
- io->u.ci_setattr.sa_falloc_end = end;
- io->u.ci_setattr.sa_attr.lvb_size =
- parent->u.ci_setattr.sa_attr.lvb_size;
}
lov_lsm2layout(lsm, lsm->lsm_entries[index],
&io->u.ci_setattr.sa_layout);
diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c
index d511ece..4abe8ba 100644
--- a/fs/lustre/osc/osc_cache.c
+++ b/fs/lustre/osc/osc_cache.c
@@ -1159,7 +1159,8 @@ static int osc_extent_make_ready(const struct lu_env *env,
int last_oap_count = osc_refresh_count(env, last,
OBD_BRW_WRITE);
- LASSERT(last_oap_count > 0);
+ LASSERTF(last_oap_count > 0,
+ "last_oap_count %d\n", last_oap_count);
LASSERT(last->oap_page_off + last_oap_count <= PAGE_SIZE);
last->oap_count = last_oap_count;
spin_lock(&last->oap_lock);
diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c
index a0537b8..ce0f7ec 100644
--- a/fs/lustre/osc/osc_io.c
+++ b/fs/lustre/osc/osc_io.c
@@ -569,8 +569,14 @@ static int osc_io_setattr_start(const struct lu_env *env,
unsigned int cl_valid = 0;
if (ia_avalid & ATTR_SIZE) {
- attr->cat_size = size;
- attr->cat_kms = size;
+ if (io_is_falloc) {
+ attr->cat_size =
+ io->u.ci_setattr.sa_attr.lvb_size;
+ attr->cat_kms = attr->cat_size;
+ } else {
+ attr->cat_size = size;
+ attr->cat_kms = size;
+ }
cl_valid = CAT_SIZE | CAT_KMS;
}
if (ia_avalid & ATTR_MTIME_SET) {
@@ -707,22 +713,17 @@ void osc_io_setattr_end(const struct lu_env *env,
}
if (cl_io_is_fallocate(io)) {
- cl_object_attr_lock(obj);
-
- /* update blocks */
- if (oa->o_valid & OBD_MD_FLBLOCKS) {
- attr->cat_blocks = oa->o_blocks;
- cl_valid |= CAT_BLOCKS;
- }
+ if (result == 0) {
+ cl_object_attr_lock(obj);
+ /* update blocks */
+ if (oa->o_valid & OBD_MD_FLBLOCKS) {
+ attr->cat_blocks = oa->o_blocks;
+ cl_valid |= CAT_BLOCKS;
+ }
- /* update size */
- if (oa->o_valid & OBD_MD_FLSIZE) {
- attr->cat_size = oa->o_size;
- cl_valid |= CAT_SIZE;
+ cl_object_attr_update(env, obj, attr, cl_valid);
+ cl_object_attr_unlock(obj);
}
-
- cl_object_attr_update(env, obj, attr, cl_valid);
- cl_object_attr_unlock(obj);
}
}
EXPORT_SYMBOL(osc_io_setattr_end);
diff --git a/fs/lustre/osc/osc_object.c b/fs/lustre/osc/osc_object.c
index 273098a..00f2800 100644
--- a/fs/lustre/osc/osc_object.c
+++ b/fs/lustre/osc/osc_object.c
@@ -218,7 +218,9 @@ static int osc_object_ast_clear(struct ldlm_lock *lock, void *data)
lvb->lvb_atime, oinfo->loi_lvb.lvb_size,
oinfo->loi_lvb.lvb_blocks, oinfo->loi_lvb.lvb_ctime,
oinfo->loi_lvb.lvb_mtime, oinfo->loi_lvb.lvb_atime);
- LASSERT(oinfo->loi_lvb.lvb_size >= oinfo->loi_kms);
+ LASSERTF(oinfo->loi_lvb.lvb_size >= oinfo->loi_kms,
+ "lvb_size %#llx, loi_kms %#llx\n",
+ oinfo->loi_lvb.lvb_size, oinfo->loi_kms);
cl_object_attr_lock(&osc->oo_cl);
memcpy(lvb, &oinfo->loi_lvb, sizeof(oinfo->loi_lvb));
--
1.8.3.1
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
^ permalink raw reply related [flat|nested] 10+ messages in thread* [lustre-devel] [PATCH 5/9] lustre: ldlm: don't change GROUP lock GID on client
2021-02-09 0:54 [lustre-devel] [PATCH 0/9] Sync to OpenSFS as of Feb 8, 2021 James Simmons
` (3 preceding siblings ...)
2021-02-09 0:54 ` [lustre-devel] [PATCH 4/9] lustre: osc: correctly update size/kms for fallocate James Simmons
@ 2021-02-09 0:54 ` James Simmons
2021-02-09 0:54 ` [lustre-devel] [PATCH 6/9] lustre: sec: get rid of bad rss-counter state messages James Simmons
` (3 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: James Simmons @ 2021-02-09 0:54 UTC (permalink / raw)
To: Andreas Dilger, Oleg Drokin, NeilBrown
Cc: Mikhail Pershin, Lustre Development List
From: Mikhail Pershin <mpershin@whamcloud.com>
GROUP lock GID is part of inodebits policy and is passed
to the server from client in policy li_gid field.
Meanwhile the ldlm_ibits_policy_wire_to_local() is used on
client also when server reply or completion AST is processed,
so client original GID can be overwritten by server value.
This is not problem if both server and client have the same
Lustre version but if server is older then it can have garbage
in li_gid field and client lock policy is updated with it.
Considering that GROUP lock GID is never changed and server should
not do that, the solution is to ignore returned li_gid from server
and never update original GID of GROUP lock on client from server
response.
WC-bug-id: https://jira.whamcloud.com/browse/LU-14312
Lustre-commit: da18ad5628556cb ("LU-14312 ldlm: don't change GROUP lock GID on client")
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41268
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
fs/lustre/ldlm/ldlm_inodebits.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/fs/lustre/ldlm/ldlm_inodebits.c b/fs/lustre/ldlm/ldlm_inodebits.c
index a367ff1..32de9b9 100644
--- a/fs/lustre/ldlm/ldlm_inodebits.c
+++ b/fs/lustre/ldlm/ldlm_inodebits.c
@@ -59,7 +59,10 @@ void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
union ldlm_policy_data *lpolicy)
{
lpolicy->l_inodebits.bits = wpolicy->l_inodebits.bits;
- lpolicy->l_inodebits.li_gid = wpolicy->l_inodebits.li_gid;
+ /**
+ * try_bits and li_gid are to be handled outside of generic
+ * write_to_local due to different behavior on a server and client.
+ */
}
void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
--
1.8.3.1
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
^ permalink raw reply related [flat|nested] 10+ messages in thread* [lustre-devel] [PATCH 6/9] lustre: sec: get rid of bad rss-counter state messages
2021-02-09 0:54 [lustre-devel] [PATCH 0/9] Sync to OpenSFS as of Feb 8, 2021 James Simmons
` (4 preceding siblings ...)
2021-02-09 0:54 ` [lustre-devel] [PATCH 5/9] lustre: ldlm: don't change GROUP lock GID on client James Simmons
@ 2021-02-09 0:54 ` James Simmons
2021-02-09 0:54 ` [lustre-devel] [PATCH 7/9] lustre: obdclass: add option %H for jobid James Simmons
` (2 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: James Simmons @ 2021-02-09 0:54 UTC (permalink / raw)
To: Andreas Dilger, Oleg Drokin, NeilBrown; +Cc: Lustre Development List
From: Sebastien Buisson <sbuisson@ddn.com>
When doing O_DIRECT IOs on encrypted files, messages about bad
rss-counter state can be seen in the console. The mm get confused
because we twist the Lustre pages used for RPCs so that they are
suitable for llcrypt API.
In order to do this properly, the original mapping on these pages
must be preserved outside of the encryption/decryption needs.
Fixes: feca6b62a6 ("lustre: sec: O_DIRECT for encrypted file")
WC-bug-id: https://jira.whamcloud.com/browse/LU-14306
Lustre-commit: a71e0dd7f7aa445 ("LU-14306 sec: get rid of bad rss-counter state messages")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/41199
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
fs/lustre/include/cl_object.h | 6 +++
fs/lustre/include/lustre_crypto.h | 8 ++++
fs/lustre/llite/rw26.c | 23 +++---------
fs/lustre/obdclass/cl_io.c | 11 ------
fs/lustre/obdclass/cl_page.c | 1 +
fs/lustre/osc/osc_request.c | 78 +++++++++++++++++++++++++++++++++------
6 files changed, 86 insertions(+), 41 deletions(-)
diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h
index d2cee34..4f34e5d 100644
--- a/fs/lustre/include/cl_object.h
+++ b/fs/lustre/include/cl_object.h
@@ -741,6 +741,12 @@ struct cl_page {
struct cl_object *cp_obj;
/** vmpage */
struct page *cp_vmpage;
+ /**
+ * Assigned if doing direct IO, because in this case cp_vmpage is not
+ * a valid page cache page, hence the inode cannot be inferred from
+ * cp_vmpage->mapping->host.
+ */
+ struct inode *cp_inode;
/** Linkage of pages within group. Pages must be owned */
struct list_head cp_batch;
/** array of slices offset. Immutable after creation. */
diff --git a/fs/lustre/include/lustre_crypto.h b/fs/lustre/include/lustre_crypto.h
index b80147f..01b5e85 100644
--- a/fs/lustre/include/lustre_crypto.h
+++ b/fs/lustre/include/lustre_crypto.h
@@ -48,8 +48,12 @@ int ll_set_encflags(struct inode *inode, void *encctx, u32 encctxlen,
#define llcrypt_has_encryption_key(inode) fscrypt_has_encryption_key(inode)
#define llcrypt_encrypt_pagecache_blocks(page, len, offs, gfp_flags) \
fscrypt_encrypt_pagecache_blocks(page, len, offs, gfp_flags)
+#define llcrypt_encrypt_block_inplace(inode, page, len, offs, lblk, gfp_flags) \
+ fscrypt_encrypt_block_inplace(inode, page, len, offs, lblk, gfp_flags)
#define llcrypt_decrypt_pagecache_blocks(page, len, offs) \
fscrypt_decrypt_pagecache_blocks(page, len, offs)
+#define llcrypt_decrypt_block_inplace(inode, page, len, offs, lblk_num) \
+ fscrypt_decrypt_block_inplace(inode, page, len, offs, lblk_num)
#define llcrypt_inherit_context(parent, child, fs_data, preload) \
fscrypt_inherit_context(parent, child, fs_data, preload)
#define llcrypt_get_encryption_info(inode) fscrypt_get_encryption_info(inode)
@@ -82,7 +86,11 @@ int ll_set_encflags(struct inode *inode, void *encctx, u32 encctxlen,
#define llcrypt_has_encryption_key(inode) false
#define llcrypt_encrypt_pagecache_blocks(page, len, offs, gfp_flags) \
ERR_PTR(-EOPNOTSUPP)
+#define llcrypt_encrypt_block_inplace(inode, page, len, offs, lblk, gfp_flags) \
+ -EOPNOTSUPP
#define llcrypt_decrypt_pagecache_blocks(page, len, offs) -EOPNOTSUPP
+#define llcrypt_decrypt_block_inplace(inode, page, len, offs, lblk_num) \
+ -EOPNOTSUPP
#define llcrypt_inherit_context(parent, child, fs_data, preload) -EOPNOTSUPP
#define llcrypt_get_encryption_info(inode) -EOPNOTSUPP
#define llcrypt_put_encryption_info(inode) do {} while (0)
diff --git a/fs/lustre/llite/rw26.c b/fs/lustre/llite/rw26.c
index 28c0a75..91a05ac 100644
--- a/fs/lustre/llite/rw26.c
+++ b/fs/lustre/llite/rw26.c
@@ -236,7 +236,6 @@ struct ll_dio_pages {
int io_pages = 0;
size_t page_size = cl_page_size(obj);
int i;
- pgoff_t index = offset >> PAGE_SHIFT;
ssize_t rc = 0;
cl_2queue_init(queue);
@@ -257,26 +256,14 @@ struct ll_dio_pages {
page->cp_sync_io = anchor;
if (inode && IS_ENCRYPTED(inode)) {
- struct page *vmpage = cl_page_vmpage(page);
-
/* In case of Direct IO on encrypted file, we need to
- * set the correct page index, and add a reference to
- * the mapping. This is required by llcrypt to proceed
- * to encryption/decryption, because each block is
- * encrypted independently, and each block's IV is set
- * to the logical block number within the file.
+ * add a reference to the inode on the cl_page.
+ * This info is required by llcrypt to proceed
+ * to encryption/decryption.
* This is safe because we know these pages are private
- * to the thread doing the Direct IO, and despite
- * setting a mapping on the pages, cached lookups will
- * not find them.
- * Set PageChecked to detect special case of Direct IO
- * in osc_brw_fini_request().
- * Reference to the mapping and PageChecked flag are
- * removed in cl_aio_end().
+ * to the thread doing the Direct IO.
*/
- vmpage->index = index++;
- vmpage->mapping = inode->i_mapping;
- SetPageChecked(vmpage);
+ page->cp_inode = inode;
}
cl_page_list_add(&queue->c2_qin, page);
/*
diff --git a/fs/lustre/obdclass/cl_io.c b/fs/lustre/obdclass/cl_io.c
index 37b0828..c57a3766 100644
--- a/fs/lustre/obdclass/cl_io.c
+++ b/fs/lustre/obdclass/cl_io.c
@@ -1081,19 +1081,8 @@ static void cl_aio_end(const struct lu_env *env, struct cl_sync_io *anchor)
/* release pages */
while (aio->cda_pages.pl_nr > 0) {
struct cl_page *page = cl_page_list_first(&aio->cda_pages);
- struct page *vmpage = cl_page_vmpage(page);
- struct inode *inode = vmpage ? page2inode(vmpage) : NULL;
cl_page_get(page);
- /* We end up here in case of Direct IO only. For encrypted file,
- * mapping was set on pages in ll_direct_rw_pages(), so it has
- * to be cleared now before page cleanup.
- * PageChecked flag was also set there, so we clean up here.
- */
- if (inode && IS_ENCRYPTED(inode)) {
- vmpage->mapping = NULL;
- ClearPageChecked(vmpage);
- }
cl_page_list_del(env, &aio->cda_pages, page);
cl_page_delete(env, page);
cl_page_put(env, page);
diff --git a/fs/lustre/obdclass/cl_page.c b/fs/lustre/obdclass/cl_page.c
index 53f88a7..2cf8d30 100644
--- a/fs/lustre/obdclass/cl_page.c
+++ b/fs/lustre/obdclass/cl_page.c
@@ -231,6 +231,7 @@ struct cl_page *cl_page_alloc(const struct lu_env *env, struct cl_object *o,
cl_page->cp_vmpage = vmpage;
cl_page->cp_state = CPS_CACHED;
cl_page->cp_type = type;
+ cl_page->cp_inode = NULL;
INIT_LIST_HEAD(&cl_page->cp_batch);
lu_ref_init(&cl_page->cp_reference);
cl_object_for_each(o2, o) {
diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c
index a6a8cac..8d3688f 100644
--- a/fs/lustre/osc/osc_request.c
+++ b/fs/lustre/osc/osc_request.c
@@ -1409,8 +1409,21 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli,
void *short_io_buf;
const char *obd_name = cli->cl_import->imp_obd->obd_name;
struct inode *inode;
+ bool directio = false;
inode = page2inode(pga[0]->pg);
+ if (!inode) {
+ /* Try to get reference to inode from cl_page if we are
+ * dealing with direct IO, as handled pages are not
+ * actual page cache pages.
+ */
+ struct osc_async_page *oap = brw_page2oap(pga[0]);
+ struct cl_page *clpage = oap2cl_page(oap);
+
+ inode = clpage->cp_inode;
+ if (inode)
+ directio = true;
+ }
if (OBD_FAIL_CHECK(OBD_FAIL_OSC_BRW_PREP_REQ))
return -ENOMEM; /* Recoverable */
if (OBD_FAIL_CHECK(OBD_FAIL_OSC_BRW_PREP_REQ2))
@@ -1435,6 +1448,8 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli,
bool retried = false;
bool lockedbymyself;
u32 nunits = (pg->off & ~PAGE_MASK) + pg->count;
+ struct address_space *map_orig = NULL;
+ pgoff_t index_orig;
retry_encrypt:
if (nunits & ~LUSTRE_ENCRYPTION_MASK)
@@ -1450,10 +1465,20 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli,
* which means only once the page is fully processed.
*/
lockedbymyself = trylock_page(pg->pg);
+ if (directio) {
+ map_orig = pg->pg->mapping;
+ pg->pg->mapping = inode->i_mapping;
+ index_orig = pg->pg->index;
+ pg->pg->index = pg->off >> PAGE_SHIFT;
+ }
data_page =
llcrypt_encrypt_pagecache_blocks(pg->pg,
nunits, 0,
GFP_NOFS);
+ if (directio) {
+ pg->pg->mapping = map_orig;
+ pg->pg->index = index_orig;
+ }
if (lockedbymyself)
unlock_page(pg->pg);
if (IS_ERR(data_page)) {
@@ -1921,6 +1946,7 @@ static int osc_brw_fini_request(struct ptlrpc_request *req, int rc)
struct ost_body *body;
u32 client_cksum = 0;
struct inode *inode;
+ unsigned int blockbits = 0, blocksize = 0;
if (rc < 0 && rc != -EDQUOT) {
DEBUG_REQ(D_INFO, req, "Failed request: rc = %d", rc);
@@ -2113,6 +2139,19 @@ static int osc_brw_fini_request(struct ptlrpc_request *req, int rc)
}
inode = page2inode(aa->aa_ppga[0]->pg);
+ if (!inode) {
+ /* Try to get reference to inode from cl_page if we are
+ * dealing with direct IO, as handled pages are not
+ * actual page cache pages.
+ */
+ struct osc_async_page *oap = brw_page2oap(aa->aa_ppga[0]);
+
+ inode = oap2cl_page(oap)->cp_inode;
+ if (inode) {
+ blockbits = inode->i_blkbits;
+ blocksize = 1 << blockbits;
+ }
+ }
if (inode && IS_ENCRYPTED(inode)) {
int idx;
@@ -2137,18 +2176,33 @@ static int osc_brw_fini_request(struct ptlrpc_request *req, int rc)
break;
}
- /* The page is already locked when we arrive
- * here, except when we deal with a twisted
- * page for specific Direct IO support, in
- * which case PageChecked flag is set on page.
- */
- if (PageChecked(pg->pg))
- lock_page(pg->pg);
- rc = llcrypt_decrypt_pagecache_blocks(pg->pg,
- LUSTRE_ENCRYPTION_UNIT_SIZE,
- offs);
- if (PageChecked(pg->pg))
- unlock_page(pg->pg);
+ if (blockbits) {
+ /* This is direct IO case. Directly call
+ * decrypt function that takes inode as
+ * input parameter. Page does not need
+ * to be locked.
+ */
+ u64 lblk_num =
+ ((u64)(pg->off >> PAGE_SHIFT) <<
+ (PAGE_SHIFT - blockbits)) +
+ (offs >> blockbits);
+ unsigned int i;
+
+ for (i = offs;
+ i < offs + LUSTRE_ENCRYPTION_UNIT_SIZE;
+ i += blocksize, lblk_num++) {
+ rc = llcrypt_decrypt_block_inplace(inode,
+ pg->pg,
+ blocksize, i,
+ lblk_num);
+ if (rc)
+ break;
+ }
+ } else {
+ rc = llcrypt_decrypt_pagecache_blocks(pg->pg,
+ LUSTRE_ENCRYPTION_UNIT_SIZE,
+ offs);
+ }
if (rc)
goto out;
--
1.8.3.1
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
^ permalink raw reply related [flat|nested] 10+ messages in thread* [lustre-devel] [PATCH 7/9] lustre: obdclass: add option %H for jobid
2021-02-09 0:54 [lustre-devel] [PATCH 0/9] Sync to OpenSFS as of Feb 8, 2021 James Simmons
` (5 preceding siblings ...)
2021-02-09 0:54 ` [lustre-devel] [PATCH 6/9] lustre: sec: get rid of bad rss-counter state messages James Simmons
@ 2021-02-09 0:54 ` James Simmons
2021-02-09 0:54 ` [lustre-devel] [PATCH 8/9] lustre: osc: avoid crash if ocd reset James Simmons
2021-02-09 0:54 ` [lustre-devel] [PATCH 9/9] lustre: lov: return stripe_count=1 instead of 0 for DoM files James Simmons
8 siblings, 0 replies; 10+ messages in thread
From: James Simmons @ 2021-02-09 0:54 UTC (permalink / raw)
To: Andreas Dilger, Oleg Drokin, NeilBrown
Cc: Yang Sheng, Lustre Development List
From: Yang Sheng <ys@whamcloud.com>
Add a option %H to avoid jobid too long in some cases.
WC-bug-id: https://jira.whamcloud.com/browse/LU-14339
Lustre-commit: cf72ee174bbf7e6 ("LU-14339 obdclass: add option %H for jobid")
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41262
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
fs/lustre/obdclass/jobid.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/fs/lustre/obdclass/jobid.c b/fs/lustre/obdclass/jobid.c
index 3ee9d40..52ba398 100644
--- a/fs/lustre/obdclass/jobid.c
+++ b/fs/lustre/obdclass/jobid.c
@@ -206,6 +206,7 @@ static void jobid_prune_expedite(void)
* %e = executable
* %g = gid
* %h = hostname
+ * %H = short hostname
* %j = per-session
* %p = pid
* %u = uid
@@ -222,7 +223,7 @@ static int jobid_interpret_string(const char *jobfmt, char *jobid,
char c;
while ((c = *jobfmt++) && joblen > 1) {
- char f;
+ char f, *p;
int l;
if (isspace(c)) /* Don't allow embedded spaces */
@@ -232,6 +233,7 @@ static int jobid_interpret_string(const char *jobfmt, char *jobid,
*jobid = c;
joblen--;
jobid++;
+ *jobid = '\0';
continue;
}
@@ -247,6 +249,15 @@ static int jobid_interpret_string(const char *jobfmt, char *jobid,
l = snprintf(jobid, joblen, "%s",
init_utsname()->nodename);
break;
+ case 'H': /* short hostname. Cut at first dot */
+ l = snprintf(jobid, joblen, "%s",
+ init_utsname()->nodename);
+ p = strnchr(jobid, joblen, '.');
+ if (p) {
+ *p = '\0';
+ l = p - jobid;
+ }
+ break;
case 'j': /* jobid requested by process */
l = snprintf(jobid, joblen, "%s",
jobid_current() ?: "jobid");
--
1.8.3.1
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
^ permalink raw reply related [flat|nested] 10+ messages in thread* [lustre-devel] [PATCH 8/9] lustre: osc: avoid crash if ocd reset
2021-02-09 0:54 [lustre-devel] [PATCH 0/9] Sync to OpenSFS as of Feb 8, 2021 James Simmons
` (6 preceding siblings ...)
2021-02-09 0:54 ` [lustre-devel] [PATCH 7/9] lustre: obdclass: add option %H for jobid James Simmons
@ 2021-02-09 0:54 ` James Simmons
2021-02-09 0:54 ` [lustre-devel] [PATCH 9/9] lustre: lov: return stripe_count=1 instead of 0 for DoM files James Simmons
8 siblings, 0 replies; 10+ messages in thread
From: James Simmons @ 2021-02-09 0:54 UTC (permalink / raw)
To: Andreas Dilger, Oleg Drokin, NeilBrown; +Cc: Lustre Development List
From: Andreas Dilger <adilger@whamcloud.com>
Avoid divide-by-zero if OSC obd_connect_data is not fully initialized.
cl_ocd_grant_param is only set after cl_max_extent_pages is OK to use.
WC-bug-id: https://jira.whamcloud.com/browse/LU-14283
Lustre-commit: 1ec58a2e6ef98ba ("LU-14283 osc: avoid crash if ocd reset")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41225
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
fs/lustre/include/obd.h | 8 +++++---
fs/lustre/osc/osc_request.c | 11 +++++------
2 files changed, 10 insertions(+), 9 deletions(-)
diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h
index de62005..9957105 100644
--- a/fs/lustre/include/obd.h
+++ b/fs/lustre/include/obd.h
@@ -206,6 +206,11 @@ struct client_obd {
*/
u32 cl_dom_min_inline_repsize;
+ /* checksumming for data sent over the network */
+ unsigned int cl_checksum:1, /* 0 = disabled, 1 = enabled */
+ cl_checksum_dump:1, /* same */
+ cl_ocd_grant_param:1;
+ /* supported checksum types that are worked out at connect time */
enum lustre_sec_part cl_sp_me;
enum lustre_sec_part cl_sp_to;
struct sptlrpc_flavor cl_flvr_mgc; /* fixed flavor of mgc->mgs */
@@ -334,9 +339,6 @@ struct client_obd {
struct list_head cl_flight_waiters;
u32 cl_rpcs_in_flight;
- /* checksumming for data sent over the network */
- unsigned int cl_checksum:1, /* 0 = disabled, 1 = enabled */
- cl_checksum_dump:1; /* same */
/* supported checksum types that are worked out at connect time */
u32 cl_supp_cksum_types;
/* checksum algorithm to be used */
diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c
index 8d3688f..be722c9 100644
--- a/fs/lustre/osc/osc_request.c
+++ b/fs/lustre/osc/osc_request.c
@@ -702,7 +702,7 @@ static void osc_announce_cached(struct client_obd *cli, struct obdo *oa,
oa->o_valid |= bits;
spin_lock(&cli->cl_loi_list_lock);
- if (OCD_HAS_FLAG(&cli->cl_import->imp_connect_data, GRANT_PARAM))
+ if (cli->cl_ocd_grant_param)
oa->o_dirty = cli->cl_dirty_grant;
else
oa->o_dirty = cli->cl_dirty_pages << PAGE_SHIFT;
@@ -735,8 +735,7 @@ static void osc_announce_cached(struct client_obd *cli, struct obdo *oa,
nrpages *= cli->cl_max_rpcs_in_flight + 1;
nrpages = max(nrpages, cli->cl_dirty_max_pages);
undirty = nrpages << PAGE_SHIFT;
- if (OCD_HAS_FLAG(&cli->cl_import->imp_connect_data,
- GRANT_PARAM)) {
+ if (cli->cl_ocd_grant_param) {
int nrextents;
/*
@@ -1070,10 +1069,10 @@ void osc_init_grant(struct client_obd *cli, struct obd_connect_data *ocd)
~chunk_mask) & chunk_mask;
/* determine maximum extent size, in #pages */
size = (u64)ocd->ocd_grant_max_blks << ocd->ocd_grant_blkbits;
- cli->cl_max_extent_pages = size >> PAGE_SHIFT;
- if (!cli->cl_max_extent_pages)
- cli->cl_max_extent_pages = 1;
+ cli->cl_max_extent_pages = (size >> PAGE_SHIFT) ? : 1;
+ cli->cl_ocd_grant_param = 1;
} else {
+ cli->cl_ocd_grant_param = 0;
cli->cl_grant_extent_tax = 0;
cli->cl_chunkbits = PAGE_SHIFT;
cli->cl_max_extent_pages = DT_MAX_BRW_PAGES;
--
1.8.3.1
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
^ permalink raw reply related [flat|nested] 10+ messages in thread* [lustre-devel] [PATCH 9/9] lustre: lov: return stripe_count=1 instead of 0 for DoM files
2021-02-09 0:54 [lustre-devel] [PATCH 0/9] Sync to OpenSFS as of Feb 8, 2021 James Simmons
` (7 preceding siblings ...)
2021-02-09 0:54 ` [lustre-devel] [PATCH 8/9] lustre: osc: avoid crash if ocd reset James Simmons
@ 2021-02-09 0:54 ` James Simmons
8 siblings, 0 replies; 10+ messages in thread
From: James Simmons @ 2021-02-09 0:54 UTC (permalink / raw)
To: Andreas Dilger, Oleg Drokin, NeilBrown; +Cc: Lustre Development List
From: Emoly Liu <emoly@whamcloud.com>
Return stripe_count=1 instead of 0 for DoM files to avoid
divide-by-zero for older userspace that calls this ioctl,
e.g. lustre ADIO driver.
WC-bug-id: https://jira.whamcloud.com/browse/LU-14337
Lustre-commit: b538826f2bdb39b ("LU-14337 lov: return stripe_count=1 instead of 0 for DoM files")
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41265
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
fs/lustre/lov/lov_pack.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/fs/lustre/lov/lov_pack.c b/fs/lustre/lov/lov_pack.c
index d200a62..ffe9687 100644
--- a/fs/lustre/lov/lov_pack.c
+++ b/fs/lustre/lov/lov_pack.c
@@ -456,6 +456,14 @@ int lov_getstripe(const struct lu_env *env, struct lov_object *obj,
lmm = lmmk;
lmm_size = lmmk_size;
}
+
+ /**
+ * Return stripe_count=1 instead of 0 for DoM files to avoid
+ * divide-by-zero for older userspace that calls this ioctl,
+ * e.g. lustre ADIO driver.
+ */
+ if ((lum.lmm_stripe_count == 0) && (lum.lmm_pattern & LOV_PATTERN_MDT))
+ lum.lmm_stripe_count = 1;
/**
* User specified limited buffer size, usually the buffer is
* from ll_lov_setstripe(), and the buffer can only hold basic
--
1.8.3.1
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
^ permalink raw reply related [flat|nested] 10+ messages in thread