[PATCH 0/2] libblkid: add FSSIZE with XFS implementation

[PATCH 0/2] libblkid: add FSSIZE with XFS implementation

[Andrey Albershteyn]

In discussion [1] it was suggested that adding new field FSSIZE
would be useful (size of filesystem, like in lsblk). These patches
add new FSSIZE field together with implementation for XFS.

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=2064810

I had a look into other fs, like ext4 and btrfs, to implement FSSIZE
for them, but I think I don't have enough expertize to do that as
they have not so trivial metadata overhead calculation :)

Andrey Albershteyn (2):
  libblkid: add interface for FSSIZE field
  libblkid: implement FSSIZE calculation for XFS

 libblkid/samples/superblocks.c         |  2 +-
 libblkid/src/blkid.h.in                |  1 +
 libblkid/src/superblocks/superblocks.c | 13 +++++++++++++
 libblkid/src/superblocks/superblocks.h |  1 +
 libblkid/src/superblocks/xfs.c         | 10 ++++++++++
 5 files changed, 26 insertions(+), 1 deletion(-)

-- 
2.27.0

[PATCH 1/2] libblkid: add interface for FSSIZE field

[Andrey Albershteyn]

Add interface to let filesystem probe calculate and set FSSIZE.
Enable that field in the 'superblocks' sample.

Signed-off-by: Andrey Albershteyn <aalbersh@redhat.com>
---
 libblkid/samples/superblocks.c         |  2 +-
 libblkid/src/blkid.h.in                |  1 +
 libblkid/src/superblocks/superblocks.c | 13 +++++++++++++
 libblkid/src/superblocks/superblocks.h |  1 +
 4 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/libblkid/samples/superblocks.c b/libblkid/samples/superblocks.c
index 7d9555771..38903ecee 100644
--- a/libblkid/samples/superblocks.c
+++ b/libblkid/samples/superblocks.c
@@ -44,7 +44,7 @@ int main(int argc, char *argv[])
 			BLKID_SUBLKS_UUID | BLKID_SUBLKS_UUIDRAW |
 			BLKID_SUBLKS_TYPE | BLKID_SUBLKS_SECTYPE |
 			BLKID_SUBLKS_USAGE | BLKID_SUBLKS_VERSION |
-			BLKID_SUBLKS_MAGIC);
+			BLKID_SUBLKS_MAGIC | BLKID_SUBLKS_FSSIZE);
 
 	rc = blkid_do_safeprobe(pr);
 	if (rc == -1)
diff --git a/libblkid/src/blkid.h.in b/libblkid/src/blkid.h.in
index 3cd4116d9..ad4becf0a 100644
--- a/libblkid/src/blkid.h.in
+++ b/libblkid/src/blkid.h.in
@@ -281,6 +281,7 @@ extern int blkid_probe_enable_superblocks(blkid_probe pr, int enable)
 #define BLKID_SUBLKS_VERSION	(1 << 8) /* read FS type from superblock */
 #define BLKID_SUBLKS_MAGIC	(1 << 9) /* define SBMAGIC and SBMAGIC_OFFSET */
 #define BLKID_SUBLKS_BADCSUM	(1 << 10) /* allow a bad checksum */
+#define BLKID_SUBLKS_FSSIZE	(1 << 11) /* read and define FSSIZE from superblock */
 
 #define BLKID_SUBLKS_DEFAULT	(BLKID_SUBLKS_LABEL | BLKID_SUBLKS_UUID | \
 				 BLKID_SUBLKS_TYPE | BLKID_SUBLKS_SECTYPE)
diff --git a/libblkid/src/superblocks/superblocks.c b/libblkid/src/superblocks/superblocks.c
index f21365538..090b618ce 100644
--- a/libblkid/src/superblocks/superblocks.c
+++ b/libblkid/src/superblocks/superblocks.c
@@ -584,6 +584,19 @@ static int blkid_probe_set_usage(blkid_probe pr, int usage)
 	return blkid_probe_set_value(pr, "USAGE", (unsigned char *) u, strlen(u) + 1);
 }
 
+int blkid_probe_set_fssize(blkid_probe pr, unsigned long long size)
+{
+	struct blkid_chain *chn = blkid_probe_get_chain(pr);
+	char u[20];
+
+	if (!(chn->flags & BLKID_SUBLKS_FSSIZE))
+		return 0;
+
+	snprintf(u, sizeof(u), "%llu", size);
+
+	return blkid_probe_set_value(pr, "FSSIZE", (unsigned char *) u, strlen(u) + 1);
+}
+
 int blkid_probe_set_id_label(blkid_probe pr, const char *name,
 			     const unsigned char *data, size_t len)
 {
diff --git a/libblkid/src/superblocks/superblocks.h b/libblkid/src/superblocks/superblocks.h
index 9c489c438..58c11fc02 100644
--- a/libblkid/src/superblocks/superblocks.h
+++ b/libblkid/src/superblocks/superblocks.h
@@ -111,6 +111,7 @@ extern int blkid_probe_set_utf8_id_label(blkid_probe pr, const char *name,
 			     const unsigned char *data, size_t len, int enc);
 
 int blkid_probe_set_block_size(blkid_probe pr, unsigned block_size);
+int blkid_probe_set_fssize(blkid_probe pr, unsigned long long size);
 
 extern int blkid_probe_is_bitlocker(blkid_probe pr);
 extern int blkid_probe_is_ntfs(blkid_probe pr);
-- 
2.27.0

Re: [PATCH 1/2] libblkid: add interface for FSSIZE field

[Karel Zak]

On Thu, Apr 21, 2022 at 03:09:45PM +0200, Andrey Albershteyn wrote:
> +int blkid_probe_set_fssize(blkid_probe pr, unsigned long long size)

I'd prefer uint64_t  rather than unsigned long long


> +{
> +	struct blkid_chain *chn = blkid_probe_get_chain(pr);
> +	char u[20];
> +
> +	if (!(chn->flags & BLKID_SUBLKS_FSSIZE))
> +		return 0;
> +
> +	snprintf(u, sizeof(u), "%llu", size);
> +
> +	return blkid_probe_set_value(pr, "FSSIZE", (unsigned char *) u, strlen(u) + 1);
> +}

  return blkid_probe_sprintf_value(pr, "FSSIZE", "%" PRIu64, size);

and you not need snprintf(), etc.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

[PATCH 2/2] libblkid: implement FSSIZE calculation for XFS

[Andrey Albershteyn]

The implementation is similar to one provided by statfs(2) + lsblk.

Signed-off-by: Andrey Albershteyn <aalbersh@redhat.com>
---
 libblkid/src/superblocks/xfs.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/libblkid/src/superblocks/xfs.c b/libblkid/src/superblocks/xfs.c
index d8c6fb6d4..444050f55 100644
--- a/libblkid/src/superblocks/xfs.c
+++ b/libblkid/src/superblocks/xfs.c
@@ -158,6 +158,15 @@ static int xfs_verify_sb(struct xfs_super_block *ondisk)
 	return 1;
 }
 
+static uint64_t xfs_fssize(struct xfs_super_block *xs)
+{
+	uint32_t lsize = xs->sb_logstart ? xs->sb_logblocks : 0;
+	uint64_t avail_blocks = be64_to_cpu(xs->sb_dblocks) - be32_to_cpu(lsize);
+	uint64_t fssize = avail_blocks*be32_to_cpu(xs->sb_blocksize);
+
+	return fssize;
+}
+
 static int probe_xfs(blkid_probe pr, const struct blkid_idmag *mag)
 {
 	struct xfs_super_block *xs;
@@ -173,6 +182,7 @@ static int probe_xfs(blkid_probe pr, const struct blkid_idmag *mag)
 		blkid_probe_set_label(pr, (unsigned char *) xs->sb_fname,
 				sizeof(xs->sb_fname));
 	blkid_probe_set_uuid(pr, xs->sb_uuid);
+	blkid_probe_set_fssize(pr, xfs_fssize(xs));
 	blkid_probe_set_block_size(pr, be16_to_cpu(xs->sb_sectsize));
 	return 0;
 }
-- 
2.27.0

Re: [PATCH 0/2] libblkid: add FSSIZE with XFS implementation

[Karel Zak]

 Thanks Andrey,

the code looks good.

On Thu, Apr 21, 2022 at 03:09:44PM +0200, Andrey Albershteyn wrote:
> I had a look into other fs, like ext4 and btrfs, to implement FSSIZE
> for them, but I think I don't have enough expertize to do that as
> they have not so trivial metadata overhead calculation :)

But it would be nice to have ext4 and btrfs too. Maybe Eric can help
you :-)

    Karel


-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: [PATCH 0/2] libblkid: add FSSIZE with XFS implementation

[Eric Sandeen]

On 4/25/22 8:43 AM, Karel Zak wrote:
> 
>  Thanks Andrey,
> 
> the code looks good.
> 
> On Thu, Apr 21, 2022 at 03:09:44PM +0200, Andrey Albershteyn wrote:
>> I had a look into other fs, like ext4 and btrfs, to implement FSSIZE
>> for them, but I think I don't have enough expertize to do that as
>> they have not so trivial metadata overhead calculation :)
> 
> But it would be nice to have ext4 and btrfs too. Maybe Eric can help
> you :-)
> 
>     Karel

For ext4 something like e2fsprogs does is probably ok:

static __u64 e2p_free_blocks_count(struct ext2_super_block *super)
{
        return super->s_free_blocks_count |
                (ext2fs_has_feature_64bit(super) ?
                (__u64) super->s_free_blocks_hi << 32 : 0);
}

though I wonder if a little documentation on the intent of the flag is
in order; ext4 can return "df" in different ways (see the minixdf and
bsddf mount options.)

I think best effort seems fine here, and reporting the same number
as the ext4 utilities do should be fine.

As for btrfs - I'm really not sure what this should return on a
multi-device filesystem?

-Eric

Re: [PATCH 0/2] libblkid: add FSSIZE with XFS implementation

[Karel Zak]

On Mon, Apr 25, 2022 at 10:38:02AM -0500, Eric Sandeen wrote:
> On 4/25/22 8:43 AM, Karel Zak wrote:
> > 
> >  Thanks Andrey,
> > 
> > the code looks good.
> > 
> > On Thu, Apr 21, 2022 at 03:09:44PM +0200, Andrey Albershteyn wrote:
> >> I had a look into other fs, like ext4 and btrfs, to implement FSSIZE
> >> for them, but I think I don't have enough expertize to do that as
> >> they have not so trivial metadata overhead calculation :)
> > 
> > But it would be nice to have ext4 and btrfs too. Maybe Eric can help
> > you :-)
> > 
> >     Karel
> 
> For ext4 something like e2fsprogs does is probably ok:
> 
> static __u64 e2p_free_blocks_count(struct ext2_super_block *super)
> {
>         return super->s_free_blocks_count |
>                 (ext2fs_has_feature_64bit(super) ?
>                 (__u64) super->s_free_blocks_hi << 32 : 0);
> }
> 
> though I wonder if a little documentation on the intent of the flag is
> in order; ext4 can return "df" in different ways (see the minixdf and
> bsddf mount options.)
> 
> I think best effort seems fine here, and reporting the same number
> as the ext4 utilities do should be fine.

Yes, that's good enough.

> As for btrfs - I'm really not sure what this should return on a
> multi-device filesystem?

Hmm ... good point. I guess the right answer is return nothing,
because we really don't want any complicated fs-specific logic in
libblkid.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: [PATCH 0/2] libblkid: add FSSIZE with XFS implementation

[Eric Sandeen]

On 4/26/22 5:53 AM, Andrey Albershteyn wrote:
> On Tue, Apr 26, 2022 at 10:21:04AM +0200, Karel Zak wrote:
>> On Mon, Apr 25, 2022 at 10:38:02AM -0500, Eric Sandeen wrote:
>>> On 4/25/22 8:43 AM, Karel Zak wrote:
>>>>
>>>>  Thanks Andrey,
>>>>
>>>> the code looks good.
>>>>
>>>> On Thu, Apr 21, 2022 at 03:09:44PM +0200, Andrey Albershteyn wrote:
>>>>> I had a look into other fs, like ext4 and btrfs, to implement FSSIZE
>>>>> for them, but I think I don't have enough expertize to do that as
>>>>> they have not so trivial metadata overhead calculation :)
>>>>
>>>> But it would be nice to have ext4 and btrfs too. Maybe Eric can help
>>>> you :-)
>>>>
>>>>     Karel
>>>
>>> For ext4 something like e2fsprogs does is probably ok:
>>>
>>> static __u64 e2p_free_blocks_count(struct ext2_super_block *super)
>>> {
>>>         return super->s_free_blocks_count |
>>>                 (ext2fs_has_feature_64bit(super) ?
>>>                 (__u64) super->s_free_blocks_hi << 32 : 0);
>>> }

Whoops, I mis-pasted, that should have been:

static __u64 e2p_blocks_count(struct ext2_super_block *super)
{
        return super->s_blocks_count |
                (ext2fs_has_feature_64bit(super) ?
                (__u64) super->s_blocks_count_hi << 32 : 0);
}

(blocks count, not free blocks count)

> If we want to mimic statfs(2) + lsblk, then, I think this one won't
> be exactly it. I suppose s_free_blocks_count contains number of not
> used blocks which will vary with utilization... The statfs()
> calculates total number of blocks as total_blocks - (journalsize +
> each group free space). As far as I got it, both journalsize and
> group free space is not so trivial to calculate.

(Sorry for mixing this up by pasting e2p_free_blocks_count)

Ok, so I'm remembering what FSSIZE does in lsblk now :) it simply returns:

	dev->fsstat.f_frsize * dev->fsstat.f_blocks

i.e. calculating the value based on statfs values.

This does get a little tricky for ext4, because statfs "f_blocks" is
affected by the minixdf/bsddf mount options.

ext4's f_blocks statfs calculation looks like this:

        buf->f_blocks = ext4_blocks_count(es) - EXT4_C2B(sbi, overhead);

the default behavior, bsddf, subtracts out the overhead.

The overhead is a little tricky to calculate, but I /think/ it's precomputed and
stored in the superblock as well for (newer?) ext4, in the s_overhead_clusters
field.

So I *think* you can just do:

(block size) * (blocks count - overhead)

and get the same answer as statfs and lsblk. Does that work?

(Getting block count is a little tricky, because it may be split across
2 fields; s_blocks_count_lo for the lower 32 bits on smaller filesystems,
and s_blocks_count_hi only if EXT4_FEATURE_INCOMPAT_64BIT is set.)

(ext2 is messier, as a precomputed overhead value is not stored on disk.
Perhaps returning FSSIZE only if the type is ext4 might make sense.)

If I'm wrong about s_overhead_clusters being pre-calculated and stored, perhaps
just ignoring "overhead' altogether and treating FSSIZE in the "minixdf"
sense, and simply return s_blocks_count_lo & s_blocks_count_hi for an
answer.

Thanks,
-Eric

Re: [PATCH 0/2] libblkid: add FSSIZE with XFS implementation

[Karel Zak]

On Tue, Apr 26, 2022 at 11:10:12PM -0500, Eric Sandeen wrote:
> Ok, so I'm remembering what FSSIZE does in lsblk now :) it simply returns:
> 
> 	dev->fsstat.f_frsize * dev->fsstat.f_blocks
> 
> i.e. calculating the value based on statfs values.
> 
> This does get a little tricky for ext4, because statfs "f_blocks" is
> affected by the minixdf/bsddf mount options.
> 
> ext4's f_blocks statfs calculation looks like this:
> 
>         buf->f_blocks = ext4_blocks_count(es) - EXT4_C2B(sbi, overhead);
> 
> the default behavior, bsddf, subtracts out the overhead.

Note. There is no problem add to the man page(s) information that
FSSIZE may be different to numbers from statfs() or df(1). It does not
have to be perfect, but good enough to get basic overview about FS. It
does not make sense to assume that libblkid is able to work with FS
like kernel.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com