[PATCH v2 0/2] ceph: switch to 4KB block size if quota size is not aligned to 4MB

[PATCH v2 0/2] ceph: switch to 4KB block size if quota size is not aligned to 4MB

[xiubli]

From: Xiubo Li <xiubli@redhat.com>

V2:
- Switched to IS_ALIGNED() macro
- Added CEPH_4K_BLOCK_SIZE macro
- Rename CEPH_BLOCK to CEPH_BLOCK_SIZE

Xiubo Li (3):
  ceph: make f_bsize always equal to f_frsize
  ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
  ceph: switch to 4KB block size if quota size is not aligned to 4MB

 fs/ceph/quota.c | 32 ++++++++++++++++++++------------
 fs/ceph/super.c | 28 ++++++++++++++--------------
 fs/ceph/super.h |  5 +++--
 3 files changed, 37 insertions(+), 28 deletions(-)

-- 
2.36.0.rc1

[PATCH v2 1/3] ceph: make f_bsize always equal to f_frsize

[xiubli]

From: Xiubo Li <xiubli@redhat.com>

The f_frsize maybe changed in the quota size is less than the defualt
4MB.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
---
 fs/ceph/super.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/fs/ceph/super.c b/fs/ceph/super.c
index 74f9cc5f37e9..88d7e67130b8 100644
--- a/fs/ceph/super.c
+++ b/fs/ceph/super.c
@@ -70,17 +70,6 @@ static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf)
 
 	/* fill in kstatfs */
 	buf->f_type = CEPH_SUPER_MAGIC;  /* ?? */
-
-	/*
-	 * express utilization in terms of large blocks to avoid
-	 * overflow on 32-bit machines.
-	 *
-	 * NOTE: for the time being, we make bsize == frsize to humor
-	 * not-yet-ancient versions of glibc that are broken.
-	 * Someday, we will probably want to report a real block
-	 * size...  whatever that may mean for a network file system!
-	 */
-	buf->f_bsize = 1 << CEPH_BLOCK_SHIFT;
 	buf->f_frsize = 1 << CEPH_BLOCK_SHIFT;
 
 	/*
@@ -95,6 +84,17 @@ static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf)
 		buf->f_bavail = le64_to_cpu(st.kb_avail) >> (CEPH_BLOCK_SHIFT-10);
 	}
 
+	/*
+	 * express utilization in terms of large blocks to avoid
+	 * overflow on 32-bit machines.
+	 *
+	 * NOTE: for the time being, we make bsize == frsize to humor
+	 * not-yet-ancient versions of glibc that are broken.
+	 * Someday, we will probably want to report a real block
+	 * size...  whatever that may mean for a network file system!
+	 */
+	buf->f_bsize = buf->f_frsize;
+
 	buf->f_files = le64_to_cpu(st.num_objects);
 	buf->f_ffree = -1;
 	buf->f_namelen = NAME_MAX;
-- 
2.36.0.rc1

[PATCH v2 2/3] ceph: switch to use CEPH_4K_BLOCK_SHIFT macro

[xiubli]

From: Xiubo Li <xiubli@redhat.com>

At the same time rename CEPH_BLOCK to CEPH_BLOCK_SIZE.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
---
 fs/ceph/super.c | 8 ++++----
 fs/ceph/super.h | 5 +++--
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/fs/ceph/super.c b/fs/ceph/super.c
index 88d7e67130b8..ba835db374a3 100644
--- a/fs/ceph/super.c
+++ b/fs/ceph/super.c
@@ -70,7 +70,7 @@ static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf)
 
 	/* fill in kstatfs */
 	buf->f_type = CEPH_SUPER_MAGIC;  /* ?? */
-	buf->f_frsize = 1 << CEPH_BLOCK_SHIFT;
+	buf->f_frsize = CEPH_BLOCK_SIZE;
 
 	/*
 	 * By default use root quota for stats; fallback to overall filesystem
@@ -79,9 +79,9 @@ static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf)
 	 */
 	if (ceph_test_mount_opt(fsc, NOQUOTADF) ||
 	    !ceph_quota_update_statfs(fsc, buf)) {
-		buf->f_blocks = le64_to_cpu(st.kb) >> (CEPH_BLOCK_SHIFT-10);
-		buf->f_bfree = le64_to_cpu(st.kb_avail) >> (CEPH_BLOCK_SHIFT-10);
-		buf->f_bavail = le64_to_cpu(st.kb_avail) >> (CEPH_BLOCK_SHIFT-10);
+		buf->f_blocks = le64_to_cpu(st.kb) >> CEPH_4K_BLOCK_SHIFT;
+		buf->f_bfree = le64_to_cpu(st.kb_avail) >> CEPH_4K_BLOCK_SHIFT;
+		buf->f_bavail = le64_to_cpu(st.kb_avail) >> CEPH_4K_BLOCK_SHIFT;
 	}
 
 	/*
diff --git a/fs/ceph/super.h b/fs/ceph/super.h
index 9bc34c31831b..02115ed59ff1 100644
--- a/fs/ceph/super.h
+++ b/fs/ceph/super.h
@@ -25,9 +25,10 @@
 
 /* large granularity for statfs utilization stats to facilitate
  * large volume sizes on 32-bit machines. */
-#define CEPH_BLOCK_SHIFT   22  /* 4 MB */
-#define CEPH_BLOCK         (1 << CEPH_BLOCK_SHIFT)
+#define CEPH_BLOCK_SHIFT    22  /* 4 MB */
+#define CEPH_BLOCK_SIZE     (1 << CEPH_BLOCK_SHIFT)
 #define CEPH_4K_BLOCK_SHIFT 12  /* 4 KB */
+#define CEPH_4K_BLOCK_SIZE  (1 << CEPH_4K_BLOCK_SHIFT)
 
 #define CEPH_MOUNT_OPT_CLEANRECOVER    (1<<1) /* auto reonnect (clean mode) after blocklisted */
 #define CEPH_MOUNT_OPT_DIRSTAT         (1<<4) /* `cat dirname` for stats */
-- 
2.36.0.rc1

[PATCH v2 3/3] ceph: switch to 4KB block size if quota size is not aligned to 4MB

[xiubli]

From: Xiubo Li <xiubli@redhat.com>

If the quota size is larger than but not aligned to 4MB, the statfs
will always set the block size to 4MB and round down the fragment
size. For exmaple if the quota size is 6MB, the `df` will always
show 4MB capacity.

Make the block size to 4KB as default if quota size is set unless
the quota size is larger than or equals to 4MB and at the same time
it aligns to 4MB.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
---
 fs/ceph/quota.c | 32 ++++++++++++++++++++------------
 1 file changed, 20 insertions(+), 12 deletions(-)

diff --git a/fs/ceph/quota.c b/fs/ceph/quota.c
index 64592adfe48f..37fe86bfb919 100644
--- a/fs/ceph/quota.c
+++ b/fs/ceph/quota.c
@@ -483,6 +483,7 @@ bool ceph_quota_update_statfs(struct ceph_fs_client *fsc, struct kstatfs *buf)
 	struct inode *in;
 	u64 total = 0, used, free;
 	bool is_updated = false;
+	u32 block_shift = CEPH_4K_BLOCK_SHIFT;
 
 	down_read(&mdsc->snap_rwsem);
 	realm = get_quota_realm(mdsc, d_inode(fsc->sb->s_root),
@@ -498,25 +499,32 @@ bool ceph_quota_update_statfs(struct ceph_fs_client *fsc, struct kstatfs *buf)
 		ci = ceph_inode(in);
 		spin_lock(&ci->i_ceph_lock);
 		if (ci->i_max_bytes) {
-			total = ci->i_max_bytes >> CEPH_BLOCK_SHIFT;
-			used = ci->i_rbytes >> CEPH_BLOCK_SHIFT;
-			/* For quota size less than 4MB, use 4KB block size */
-			if (!total) {
-				total = ci->i_max_bytes >> CEPH_4K_BLOCK_SHIFT;
-				used = ci->i_rbytes >> CEPH_4K_BLOCK_SHIFT;
-	                        buf->f_frsize = 1 << CEPH_4K_BLOCK_SHIFT;
-			}
-			/* It is possible for a quota to be exceeded.
+			/*
+			 * Switch to 4MB block size if quota size is
+			 * larger than or equals to 4MB and at the
+			 * same time is aligned to 4MB.
+			 */
+			if (IS_ALIGNED(ci->i_max_bytes, CEPH_BLOCK_SIZE))
+				block_shift = CEPH_BLOCK_SHIFT;
+
+			total = ci->i_max_bytes >> block_shift;
+			used = ci->i_rbytes >> block_shift;
+			buf->f_frsize = 1 << block_shift;
+
+			/*
+			 * It is possible for a quota to be exceeded.
 			 * Report 'zero' in that case
 			 */
 			free = total > used ? total - used : 0;
-			/* For quota size less than 4KB, report the
+			/*
+			 * For quota size less than 4KB, report the
 			 * total=used=4KB,free=0 when quota is full
-			 * and total=free=4KB, used=0 otherwise */
+			 * and total=free=4KB, used=0 otherwise
+			 */
 			if (!total) {
 				total = 1;
 				free = ci->i_max_bytes > ci->i_rbytes ? 1 : 0;
-	                        buf->f_frsize = 1 << CEPH_4K_BLOCK_SHIFT;
+	                        buf->f_frsize = CEPH_4K_BLOCK_SIZE;
 			}
 		}
 		spin_unlock(&ci->i_ceph_lock);
-- 
2.36.0.rc1

Re: [PATCH v2 0/2] ceph: switch to 4KB block size if quota size is not aligned to 4MB

[Jeff Layton]

On Mon, 2022-06-27 at 10:02 +0800, xiubli@redhat.com wrote:
> From: Xiubo Li <xiubli@redhat.com>
> 
> V2:
> - Switched to IS_ALIGNED() macro
> - Added CEPH_4K_BLOCK_SIZE macro
> - Rename CEPH_BLOCK to CEPH_BLOCK_SIZE
> 
> Xiubo Li (3):
>   ceph: make f_bsize always equal to f_frsize
>   ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
>   ceph: switch to 4KB block size if quota size is not aligned to 4MB
> 
>  fs/ceph/quota.c | 32 ++++++++++++++++++++------------
>  fs/ceph/super.c | 28 ++++++++++++++--------------
>  fs/ceph/super.h |  5 +++--
>  3 files changed, 37 insertions(+), 28 deletions(-)
> 

Reviewed-by: Jeff Layton <jlayton@kernel.org>

Re: [PATCH v2 0/2] ceph: switch to 4KB block size if quota size is not aligned to 4MB

[Yan, Zheng]

NACK,  this change will significantly increase mds load. Inaccuracy is
inherent in current quota design.

On Mon, Jun 27, 2022 at 10:06 AM <xiubli@redhat.com> wrote:
>
> From: Xiubo Li <xiubli@redhat.com>
>
> V2:
> - Switched to IS_ALIGNED() macro
> - Added CEPH_4K_BLOCK_SIZE macro
> - Rename CEPH_BLOCK to CEPH_BLOCK_SIZE
>
> Xiubo Li (3):
>   ceph: make f_bsize always equal to f_frsize
>   ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
>   ceph: switch to 4KB block size if quota size is not aligned to 4MB
>
>  fs/ceph/quota.c | 32 ++++++++++++++++++++------------
>  fs/ceph/super.c | 28 ++++++++++++++--------------
>  fs/ceph/super.h |  5 +++--
>  3 files changed, 37 insertions(+), 28 deletions(-)
>
> --
> 2.36.0.rc1
>

Re: [PATCH v2 0/2] ceph: switch to 4KB block size if quota size is not aligned to 4MB

[Xiubo Li]

On 6/30/22 10:39 AM, Yan, Zheng wrote:
> NACK,  this change will significantly increase mds load. Inaccuracy is
> inherent in current quota design.

Yeah, I was also thinking could we just allow the quota size to be 
aligned to 4KB if it < 4MB, or must be aligned to 4MB ?

Any idea ?

- Xiubo


> On Mon, Jun 27, 2022 at 10:06 AM <xiubli@redhat.com> wrote:
>> From: Xiubo Li <xiubli@redhat.com>
>>
>> V2:
>> - Switched to IS_ALIGNED() macro
>> - Added CEPH_4K_BLOCK_SIZE macro
>> - Rename CEPH_BLOCK to CEPH_BLOCK_SIZE
>>
>> Xiubo Li (3):
>>    ceph: make f_bsize always equal to f_frsize
>>    ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
>>    ceph: switch to 4KB block size if quota size is not aligned to 4MB
>>
>>   fs/ceph/quota.c | 32 ++++++++++++++++++++------------
>>   fs/ceph/super.c | 28 ++++++++++++++--------------
>>   fs/ceph/super.h |  5 +++--
>>   3 files changed, 37 insertions(+), 28 deletions(-)
>>
>> --
>> 2.36.0.rc1
>>

Re: [PATCH v2 0/2] ceph: switch to 4KB block size if quota size is not aligned to 4MB

[Yan, Zheng]

On Thu, Jun 30, 2022 at 11:05 AM Xiubo Li <xiubli@redhat.com> wrote:
>
>
> On 6/30/22 10:39 AM, Yan, Zheng wrote:
> > NACK,  this change will significantly increase mds load. Inaccuracy is
> > inherent in current quota design.
>
> Yeah, I was also thinking could we just allow the quota size to be
> aligned to 4KB if it < 4MB, or must be aligned to 4MB ?
>
> Any idea ?

make sense

>
> - Xiubo
>
>
> > On Mon, Jun 27, 2022 at 10:06 AM <xiubli@redhat.com> wrote:
> >> From: Xiubo Li <xiubli@redhat.com>
> >>
> >> V2:
> >> - Switched to IS_ALIGNED() macro
> >> - Added CEPH_4K_BLOCK_SIZE macro
> >> - Rename CEPH_BLOCK to CEPH_BLOCK_SIZE
> >>
> >> Xiubo Li (3):
> >>    ceph: make f_bsize always equal to f_frsize
> >>    ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
> >>    ceph: switch to 4KB block size if quota size is not aligned to 4MB
> >>
> >>   fs/ceph/quota.c | 32 ++++++++++++++++++++------------
> >>   fs/ceph/super.c | 28 ++++++++++++++--------------
> >>   fs/ceph/super.h |  5 +++--
> >>   3 files changed, 37 insertions(+), 28 deletions(-)
> >>
> >> --
> >> 2.36.0.rc1
> >>
>

Re: [PATCH v2 0/2] ceph: switch to 4KB block size if quota size is not aligned to 4MB

[Xiubo Li]

On 6/30/22 11:30 AM, Yan, Zheng wrote:
> On Thu, Jun 30, 2022 at 11:05 AM Xiubo Li <xiubli@redhat.com> wrote:
>>
>> On 6/30/22 10:39 AM, Yan, Zheng wrote:
>>> NACK,  this change will significantly increase mds load. Inaccuracy is
>>> inherent in current quota design.
>> Yeah, I was also thinking could we just allow the quota size to be
>> aligned to 4KB if it < 4MB, or must be aligned to 4MB ?
>>
>> Any idea ?
> make sense

Cool, Thanks Zheng.

I will work on that.

-- Xiubo


>> - Xiubo
>>
>>
>>> On Mon, Jun 27, 2022 at 10:06 AM <xiubli@redhat.com> wrote:
>>>> From: Xiubo Li <xiubli@redhat.com>
>>>>
>>>> V2:
>>>> - Switched to IS_ALIGNED() macro
>>>> - Added CEPH_4K_BLOCK_SIZE macro
>>>> - Rename CEPH_BLOCK to CEPH_BLOCK_SIZE
>>>>
>>>> Xiubo Li (3):
>>>>     ceph: make f_bsize always equal to f_frsize
>>>>     ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
>>>>     ceph: switch to 4KB block size if quota size is not aligned to 4MB
>>>>
>>>>    fs/ceph/quota.c | 32 ++++++++++++++++++++------------
>>>>    fs/ceph/super.c | 28 ++++++++++++++--------------
>>>>    fs/ceph/super.h |  5 +++--
>>>>    3 files changed, 37 insertions(+), 28 deletions(-)
>>>>
>>>> --
>>>> 2.36.0.rc1
>>>>