[f2fs-dev] [PATCH] fsck.f2fs: Add progression feedback

[f2fs-dev] [PATCH] fsck.f2fs: Add progression feedback

[Antoine Viallon via Linux-f2fs-devel]

On large SSDs filled with lots of data, fsck.f2fs can be very long to finish.
For instance, on my 1TB SSD filled at 99%, it takes literally 5 minutes to
complete.

Currently, the only way to have some feedback is to enable debug output,
but it is very verbose and doesn't tell the actual progress.

This patch implements a simple progress report in the longest
running part of the check (in fsck_chk_node_blk).
The number of checked node / total valid nodes is printed
every 1000 nodes checked, and the percentage of progress
is also calculated and printed.


Signed-off-by: Antoine Viallon <antoine@lesviallon.fr>
---
 fsck/fsck.c | 9 +++++++++
 fsck/fsck.h | 1 +
 2 files changed, 10 insertions(+)

diff --git a/fsck/fsck.c b/fsck/fsck.c
index ecd87af..7a64339 100644
--- a/fsck/fsck.c
+++ b/fsck/fsck.c
@@ -546,6 +546,15 @@ int fsck_chk_node_blk(struct f2fs_sb_info *sbi, struct f2fs_inode *inode,
 	node_blk = (struct f2fs_node *)calloc(BLOCK_SZ, 1);
 	ASSERT(node_blk != NULL);
 
+	/* Progress report */
+	sbi->fsck->chk.checked_blk_cnt++;
+
+	if (sbi->fsck->chk.checked_blk_cnt % 1000 == 0)
+		printf("[FSCK] Checked node %lu / %u (%.2f%%)\n",
+			   sbi->fsck->chk.checked_blk_cnt,
+			   sbi->total_valid_node_count,
+			   100 * (float)(sbi->fsck->chk.checked_blk_cnt) / sbi->total_valid_node_count);
+
 	if (sanity_check_nid(sbi, nid, node_blk, ftype, ntype, &ni))
 		goto err;
 
diff --git a/fsck/fsck.h b/fsck/fsck.h
index 11846e1..dd679b5 100644
--- a/fsck/fsck.h
+++ b/fsck/fsck.h
@@ -91,6 +91,7 @@ struct f2fs_fsck {
 
 	struct orphan_info orphani;
 	struct chk_result {
+		u64 checked_blk_cnt;
 		u64 valid_blk_cnt;
 		u32 valid_nat_entry_cnt;
 		u32 valid_node_cnt;
-- 
2.34.1



_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH] fsck.f2fs: Add progression feedback

[Antoine Viallon via Linux-f2fs-devel]

Some variable naming mistaskes were made in the last patch,
here is an updated version.

---
 fsck/fsck.c | 9 +++++++++
 fsck/fsck.h | 1 +
 2 files changed, 10 insertions(+)

diff --git a/fsck/fsck.c b/fsck/fsck.c
index ecd87af..921db0f 100644
--- a/fsck/fsck.c
+++ b/fsck/fsck.c
@@ -546,6 +546,15 @@ int fsck_chk_node_blk(struct f2fs_sb_info *sbi, struct f2fs_inode *inode,
        node_blk = (struct f2fs_node *)calloc(BLOCK_SZ, 1);
        ASSERT(node_blk != NULL);
 
+       /* Progress report */
+       sbi->fsck->chk.checked_node_cnt++;
+
+       if (sbi->fsck->chk.checked_node_cnt % 1000 == 0)
+               printf("[FSCK] Check node %u / %u (%.2f%%)\n",
+                          sbi->fsck->chk.checked_node_cnt,
+                          sbi->total_valid_node_count,
+                          100 * (float)(sbi->fsck->chk.checked_node_cnt) / sbi->total_valid_node_count);
+
        if (sanity_check_nid(sbi, nid, node_blk, ftype, ntype, &ni))
                goto err;
 
diff --git a/fsck/fsck.h b/fsck/fsck.h
index 11846e1..535d6d5 100644
--- a/fsck/fsck.h
+++ b/fsck/fsck.h
@@ -93,6 +93,7 @@ struct f2fs_fsck {
        struct chk_result {
                u64 valid_blk_cnt;
                u32 valid_nat_entry_cnt;
+               u32 checked_node_cnt;
                u32 valid_node_cnt;
                u32 valid_inode_cnt;
                u32 multi_hard_link_files;
-- 
2.34.1


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH] fsck.f2fs: Add progression feedback

[Antoine Viallon via Linux-f2fs-devel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hello, I know you are probably busy.
However, even though my patch is pretty bad, I'd like to know if the general idea
of having some progress feedback interests you at all.

Perhaps it could be opt-in, using a --progress flag?
Please tell me what is your opinion on the matter,
Antoine Viallon

> On large SSDs filled with lots of data, fsck.f2fs can be very long to finish.
> For instance, on my 1TB SSD filled at 99%, it takes literally 5 minutes to
> complete.
>
> Currently, the only way to have some feedback is to enable debug output,
> but it is very verbose and doesn't tell the actual progress.
>
> This patch implements a simple progress report in the longest
> running part of the check (in fsck_chk_node_blk).
> The number of checked node / total valid nodes is printed
> every 1000 nodes checked, and the percentage of progress
> is also calculated and printed.

-----BEGIN PGP SIGNATURE-----
Version: OpenPGP.js v2.6.2
Comment: https://openpgpjs.org

wsBcBAEBCAAQBQJh1tUaCRDRJrE6tVXhbwAADlsIAJnmen7XeO10Mi82fQwp
LizYs/oT1bxXro564y7ggKXHcEr8cqAgo4yHxQJUMpSYo7Pe+3YO6c7ONQhO
rOi7SfKsEI41YhWIzfv0dpwUgOrkwrkfuytQ4OpfUdR8zqFl2whA962qsCQi
nRgAnOAmhdQl3HFD2ZvcXfdaU/sXzlY9vGCkSeL1aGph1gaPmaWXY4K/y0Ip
7tk/ZssXJgzdtGZL5FHBopnEXAfm3Hk06oKL96VLQJ6OAH4+kUdAVY7P7FYM
BtSUzGzQRJQKfxBR+jYEVPi9x/hpWKGxxekOWPHTrISYX5fQMmrNi8fCSO5Y
cP8YcIwUfCT+WXAdOLrvI58=
=6UQI
-----END PGP SIGNATURE-----


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH] fsck.f2fs: Add progression feedback

[Chao Yu via Linux-f2fs-devel]

On 2021/12/30 8:59, Antoine Viallon via Linux-f2fs-devel wrote:
> Some variable naming mistaskes were made in the last patch,
> here is an updated version.
> 
> ---
>   fsck/fsck.c | 9 +++++++++
>   fsck/fsck.h | 1 +
>   2 files changed, 10 insertions(+)
> 
> diff --git a/fsck/fsck.c b/fsck/fsck.c
> index ecd87af..921db0f 100644
> --- a/fsck/fsck.c
> +++ b/fsck/fsck.c
> @@ -546,6 +546,15 @@ int fsck_chk_node_blk(struct f2fs_sb_info *sbi, struct f2fs_inode *inode,
>          node_blk = (struct f2fs_node *)calloc(BLOCK_SZ, 1);
>          ASSERT(node_blk != NULL);
>   
> +       /* Progress report */
> +       sbi->fsck->chk.checked_node_cnt++;
> +
> +       if (sbi->fsck->chk.checked_node_cnt % 1000 == 0)
> +               printf("[FSCK] Check node %u / %u (%.2f%%)\n",
> +                          sbi->fsck->chk.checked_node_cnt,
> +                          sbi->total_valid_node_count,
> +                          100 * (float)(sbi->fsck->chk.checked_node_cnt) / sbi->total_valid_node_count);

How about showing this message in debug mode? may be under level one?

Thanks,

> +
>          if (sanity_check_nid(sbi, nid, node_blk, ftype, ntype, &ni))
>                  goto err;
>   
> diff --git a/fsck/fsck.h b/fsck/fsck.h
> index 11846e1..535d6d5 100644
> --- a/fsck/fsck.h
> +++ b/fsck/fsck.h
> @@ -93,6 +93,7 @@ struct f2fs_fsck {
>          struct chk_result {
>                  u64 valid_blk_cnt;
>                  u32 valid_nat_entry_cnt;
> +               u32 checked_node_cnt;
>                  u32 valid_node_cnt;
>                  u32 valid_inode_cnt;
>                  u32 multi_hard_link_files;


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH] fsck.f2fs: Add progression feedback

[Chao Yu]

On 2021/12/30 8:59, Antoine Viallon via Linux-f2fs-devel wrote:
> Some variable naming mistaskes were made in the last patch,
> here is an updated version.
> 
> ---
>   fsck/fsck.c | 9 +++++++++
>   fsck/fsck.h | 1 +
>   2 files changed, 10 insertions(+)
> 
> diff --git a/fsck/fsck.c b/fsck/fsck.c
> index ecd87af..921db0f 100644
> --- a/fsck/fsck.c
> +++ b/fsck/fsck.c
> @@ -546,6 +546,15 @@ int fsck_chk_node_blk(struct f2fs_sb_info *sbi, struct f2fs_inode *inode,
>          node_blk = (struct f2fs_node *)calloc(BLOCK_SZ, 1);
>          ASSERT(node_blk != NULL);
>   
> +       /* Progress report */
> +       sbi->fsck->chk.checked_node_cnt++;
> +
> +       if (sbi->fsck->chk.checked_node_cnt % 1000 == 0)
> +               printf("[FSCK] Check node %u / %u (%.2f%%)\n",
> +                          sbi->fsck->chk.checked_node_cnt,
> +                          sbi->total_valid_node_count,
> +                          100 * (float)(sbi->fsck->chk.checked_node_cnt) / sbi->total_valid_node_count);

How about showing this message in debug mode? may be under level one?

Thanks,

> +
>          if (sanity_check_nid(sbi, nid, node_blk, ftype, ntype, &ni))
>                  goto err;
>   
> diff --git a/fsck/fsck.h b/fsck/fsck.h
> index 11846e1..535d6d5 100644
> --- a/fsck/fsck.h
> +++ b/fsck/fsck.h
> @@ -93,6 +93,7 @@ struct f2fs_fsck {
>          struct chk_result {
>                  u64 valid_blk_cnt;
>                  u32 valid_nat_entry_cnt;
> +               u32 checked_node_cnt;
>                  u32 valid_node_cnt;
>                  u32 valid_inode_cnt;
>                  u32 multi_hard_link_files;


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH] fsck.f2fs: Add progression feedback

[Antoine Viallon via Linux-f2fs-devel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Thank you for your reply.

> How about showing this message in debug mode? may be under level one?
>
> Thanks,
>

The debug output is extremely verbose, even at level one (I measured hundreds of messages per SECOND on a SATA SSD).
If we print the progress in this output, it would be completely drawn in the other messages, therefore making it useless.

Thus, we should either decrease the debug 1 verbosity (hard), or add a new, independent flag (or way) to control it.

What do you think?
- --
Antoine Viallon
-----BEGIN PGP SIGNATURE-----
Version: OpenPGP.js v2.6.2
Comment: https://openpgpjs.org

wsBcBAEBCAAQBQJh1wvUCRDRJrE6tVXhbwAAR4MIAIPldpzkLHQbgdegLTTd
ZqrjGpTrcUkKctCJKKwiZ7KrvXsnoPpl1WmlGt/ALeGeXEZgmKY1jY2pfwJK
yhpfiykTZxcypDft6YrezJaclv+RgqADwQmVxsQvN8XYm3M71RM6GQd0+/6Y
I+yWlHZtgnG/5IViGp/eEK7UDGOtneIk7CQ+Znr4TaUXN4S95rFL9gU1Y1At
/Gty7ATbU06xSIFVh0SML3FynPE3wd7tI1F6a/MQFrGJiJyL2Sa5pelwGOSp
w2Y6JDyKgTHre6SdWUYsw23+99Pb7iwF+UkxYcyYWm1w+YY9cdHEM79Sx9ww
G6z0RXAj0unRaT9FogyxJQ4=
=jkgi
-----END PGP SIGNATURE-----


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH] fsck.f2fs: Add progression feedback

[Chao Yu]

On 2022/1/6 23:33, Antoine Viallon wrote:
> Thank you for your reply.
> 
>> How about showing this message in debug mode? may be under level one?
> 
>> Thanks,
> 
> 
> The debug output is extremely verbose, even at level one (I measured hundreds of messages per SECOND on a SATA SSD).
> If we print the progress in this output, it would be completely drawn in the other messages, therefore making it useless.

Yup, :-p

> 
> Thus, we should either decrease the debug 1 verbosity (hard), or add a new, independent flag (or way) to control it.

Maybe -v?

Thanks,

> 
> What do you think?


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH] fsck.f2fs: Add progression feedback

[Jaegeuk Kim]

Hi Antonie,

I actually took a look at the patch yesterday and tested a bit to avoid too many
messages. Could you please check this out?

https://github.com/jaegeuk/f2fs-tools/commit/45b3c75ac3c07d444180a3598f0d48c80abe9934

On 01/06, Chao Yu wrote:
> On 2022/1/6 23:33, Antoine Viallon wrote:
> > Thank you for your reply.
> > 
> > > How about showing this message in debug mode? may be under level one?
> > 
> > > Thanks,
> > 
> > 
> > The debug output is extremely verbose, even at level one (I measured hundreds of messages per SECOND on a SATA SSD).
> > If we print the progress in this output, it would be completely drawn in the other messages, therefore making it useless.
> 
> Yup, :-p
> 
> > 
> > Thus, we should either decrease the debug 1 verbosity (hard), or add a new, independent flag (or way) to control it.
> 
> Maybe -v?
> 
> Thanks,
> 
> > 
> > What do you think?
> 
> 
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] [PATCH] fsck.f2fs: Add progression feedback

[Antoine Viallon via Linux-f2fs-devel]

On large SSDs filled with lots of data, fsck.f2fs can be very long to finish.
For instance, on my 1TB SSD filled at 99%, it takes literally 5 minutes to
complete.

Currently, the only way to have some feedback is to enable debug output,
but it is very verbose and doesn't tell the actual progress.

This patch implements a simple progress report in the longest
running part of the check (in fsck_chk_node_blk).
The number of checked node / total valid nodes is printed
every 1000 nodes checked, and the percentage of progress
is also calculated and printed.


Signed-off-by: Antoine Viallon <antoine@lesviallon.fr>
---
 fsck/fsck.c | 9 +++++++++
 fsck/fsck.h | 1 +
 2 files changed, 10 insertions(+)

diff --git a/fsck/fsck.c b/fsck/fsck.c
index ecd87af..7a64339 100644
--- a/fsck/fsck.c
+++ b/fsck/fsck.c
@@ -546,6 +546,15 @@ int fsck_chk_node_blk(struct f2fs_sb_info *sbi, struct f2fs_inode *inode,
 	node_blk = (struct f2fs_node *)calloc(BLOCK_SZ, 1);
 	ASSERT(node_blk != NULL);
 
+	/* Progress report */
+	sbi->fsck->chk.checked_blk_cnt++;
+
+	if (sbi->fsck->chk.checked_blk_cnt % 1000 == 0)
+		printf("[FSCK] Checked node %lu / %u (%.2f%%)\n",
+			   sbi->fsck->chk.checked_blk_cnt,
+			   sbi->total_valid_node_count,
+			   100 * (float)(sbi->fsck->chk.checked_blk_cnt) / sbi->total_valid_node_count);
+
 	if (sanity_check_nid(sbi, nid, node_blk, ftype, ntype, &ni))
 		goto err;
 
diff --git a/fsck/fsck.h b/fsck/fsck.h
index 11846e1..dd679b5 100644
--- a/fsck/fsck.h
+++ b/fsck/fsck.h
@@ -91,6 +91,7 @@ struct f2fs_fsck {
 
 	struct orphan_info orphani;
 	struct chk_result {
+		u64 checked_blk_cnt;
 		u64 valid_blk_cnt;
 		u32 valid_nat_entry_cnt;
 		u32 valid_node_cnt;
-- 
2.34.1



_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel