[PATCH] debugfs:fix repeated output problem with `logdump -O -n <num_trans>`

[PATCH] debugfs:fix repeated output problem with `logdump -O -n <num_trans>`

[lihaoxiang (F)]

Previously, patch 6e4cc3d5eeb2dfaa055e652b5390beaa6c3d05da introduces
the function of printing the specified number of logs. But there exists
a shortage when n is larger than the total number of logs, it dumped the
duplicated records circulately.

For example, the disk sda only has three records, but using instruction logdump
-On5, it would output the result as follow:
----------------------------------------------------------------------
Journal starts at block 1, transaction 6
Found expected sequence 6, type 1 (descriptor block) at block 1
Found expected sequence 6, type 2 (commit block) at block 4
No magic number at block 5: end of journal.
Found sequence 2 (not 7) at block 7: end of journal.
Found expected sequence 2, type 2 (commit block) at block 7
Found sequence 3 (not 8) at block 8: end of journal.
Found expected sequence 3, type 1 (descriptor block) at block 8
Found sequence 3 (not 8) at block 15: end of journal.
Found expected sequence 3, type 2 (commit block) at block 15
Found sequence 6 (not 9) at block 1: end of journal.       <---------begin loop
Found expected sequence 6, type 1 (descriptor block) at block 1
Found sequence 6 (not 9) at block 4: end of journal.
Found expected sequence 6, type 2 (commit block) at block 4
Found sequence 2 (not 10) at block 7: end of journal.
Found expected sequence 2, type 2 (commit block) at block 7
logdump: short read (read 0, expected 1024) while reading journal

In this commit, we solve the problem above by exiting dumping if the
blocknr had already encountered, displayed the total number of logs
that the disk only possessed.

Signed-off-by: lihaoxiang <lihaoxiang9@huawei.com>
---
 debugfs/logdump.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/debugfs/logdump.c b/debugfs/logdump.c
index 614414e5..036b50ba 100644
--- a/debugfs/logdump.c
+++ b/debugfs/logdump.c
@@ -376,6 +376,7 @@ static void dump_journal(char *cmdname, FILE *out_file,
 	journal_header_t	*header;
 	tid_t			transaction;
 	unsigned int		blocknr = 0;
+	unsigned int		first_transaction_blocknr;
 	int			fc_done;
 	__u64			total_len;
 	__u32			maxlen;
@@ -470,10 +471,18 @@ static void dump_journal(char *cmdname, FILE *out_file,
 			blocknr = 1;
 	}

+	first_transaction_blocknr = blocknr;
+
 	while (1) {
 		if (dump_old && (dump_counts != -1) && (cur_counts >= dump_counts))
 			break;

+		if ((blocknr == first_transaction_blocknr) &&
+		    (cur_counts != 0) && dump_old && (dump_counts != -1)) {
+			fprintf(out_file, "Dump all %lld journal records.\n", cur_counts);
+			break;
+		}
+
 		retval = read_journal_block(cmdname, source,
 				((ext2_loff_t) blocknr) * blocksize,
 				buf, blocksize);
-- 
2.37.0.windows.1

Re: [PATCH] debugfs:fix repeated output problem with `logdump -O -n <num_trans>`

[lihaoxiang (F)]

friendly ping...

On 2022/11/15 16:29, lihaoxiang (F) wrote:
> Previously, patch 6e4cc3d5eeb2dfaa055e652b5390beaa6c3d05da introduces
> the function of printing the specified number of logs. But there exists
> a shortage when n is larger than the total number of logs, it dumped the
> duplicated records circulately.
> 
> For example, the disk sda only has three records, but using instruction logdump
> -On5, it would output the result as follow:
> ----------------------------------------------------------------------
> Journal starts at block 1, transaction 6
> Found expected sequence 6, type 1 (descriptor block) at block 1
> Found expected sequence 6, type 2 (commit block) at block 4
> No magic number at block 5: end of journal.
> Found sequence 2 (not 7) at block 7: end of journal.
> Found expected sequence 2, type 2 (commit block) at block 7
> Found sequence 3 (not 8) at block 8: end of journal.
> Found expected sequence 3, type 1 (descriptor block) at block 8
> Found sequence 3 (not 8) at block 15: end of journal.
> Found expected sequence 3, type 2 (commit block) at block 15
> Found sequence 6 (not 9) at block 1: end of journal.       <---------begin loop
> Found expected sequence 6, type 1 (descriptor block) at block 1
> Found sequence 6 (not 9) at block 4: end of journal.
> Found expected sequence 6, type 2 (commit block) at block 4
> Found sequence 2 (not 10) at block 7: end of journal.
> Found expected sequence 2, type 2 (commit block) at block 7
> logdump: short read (read 0, expected 1024) while reading journal
> 
> In this commit, we solve the problem above by exiting dumping if the
> blocknr had already encountered, displayed the total number of logs
> that the disk only possessed.
> 
> Signed-off-by: lihaoxiang <lihaoxiang9@huawei.com>
> ---
>  debugfs/logdump.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/debugfs/logdump.c b/debugfs/logdump.c
> index 614414e5..036b50ba 100644
> --- a/debugfs/logdump.c
> +++ b/debugfs/logdump.c
> @@ -376,6 +376,7 @@ static void dump_journal(char *cmdname, FILE *out_file,
>  	journal_header_t	*header;
>  	tid_t			transaction;
>  	unsigned int		blocknr = 0;
> +	unsigned int		first_transaction_blocknr;
>  	int			fc_done;
>  	__u64			total_len;
>  	__u32			maxlen;
> @@ -470,10 +471,18 @@ static void dump_journal(char *cmdname, FILE *out_file,
>  			blocknr = 1;
>  	}
> 
> +	first_transaction_blocknr = blocknr;
> +
>  	while (1) {
>  		if (dump_old && (dump_counts != -1) && (cur_counts >= dump_counts))
>  			break;
> 
> +		if ((blocknr == first_transaction_blocknr) &&
> +		    (cur_counts != 0) && dump_old && (dump_counts != -1)) {
> +			fprintf(out_file, "Dump all %lld journal records.\n", cur_counts);
> +			break;
> +		}
> +
>  		retval = read_journal_block(cmdname, source,
>  				((ext2_loff_t) blocknr) * blocksize,
>  				buf, blocksize);

Re: [PATCH] debugfs:fix repeated output problem with `logdump -O -n <num_trans>`

[lihaoxiang (F)]

friendly ping...

On 2022/11/15 16:29, lihaoxiang (F) wrote:
> Previously, patch 6e4cc3d5eeb2dfaa055e652b5390beaa6c3d05da introduces
> the function of printing the specified number of logs. But there exists
> a shortage when n is larger than the total number of logs, it dumped the
> duplicated records circulately.
> 
> For example, the disk sda only has three records, but using instruction logdump
> -On5, it would output the result as follow:
> ----------------------------------------------------------------------
> Journal starts at block 1, transaction 6
> Found expected sequence 6, type 1 (descriptor block) at block 1
> Found expected sequence 6, type 2 (commit block) at block 4
> No magic number at block 5: end of journal.
> Found sequence 2 (not 7) at block 7: end of journal.
> Found expected sequence 2, type 2 (commit block) at block 7
> Found sequence 3 (not 8) at block 8: end of journal.
> Found expected sequence 3, type 1 (descriptor block) at block 8
> Found sequence 3 (not 8) at block 15: end of journal.
> Found expected sequence 3, type 2 (commit block) at block 15
> Found sequence 6 (not 9) at block 1: end of journal.       <---------begin loop
> Found expected sequence 6, type 1 (descriptor block) at block 1
> Found sequence 6 (not 9) at block 4: end of journal.
> Found expected sequence 6, type 2 (commit block) at block 4
> Found sequence 2 (not 10) at block 7: end of journal.
> Found expected sequence 2, type 2 (commit block) at block 7
> logdump: short read (read 0, expected 1024) while reading journal
> 
> In this commit, we solve the problem above by exiting dumping if the
> blocknr had already encountered, displayed the total number of logs
> that the disk only possessed.
> 
> Signed-off-by: lihaoxiang <lihaoxiang9@huawei.com>
> ---
>  debugfs/logdump.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/debugfs/logdump.c b/debugfs/logdump.c
> index 614414e5..036b50ba 100644
> --- a/debugfs/logdump.c
> +++ b/debugfs/logdump.c
> @@ -376,6 +376,7 @@ static void dump_journal(char *cmdname, FILE *out_file,
>  	journal_header_t	*header;
>  	tid_t			transaction;
>  	unsigned int		blocknr = 0;
> +	unsigned int		first_transaction_blocknr;
>  	int			fc_done;
>  	__u64			total_len;
>  	__u32			maxlen;
> @@ -470,10 +471,18 @@ static void dump_journal(char *cmdname, FILE *out_file,
>  			blocknr = 1;
>  	}
> 
> +	first_transaction_blocknr = blocknr;
> +
>  	while (1) {
>  		if (dump_old && (dump_counts != -1) && (cur_counts >= dump_counts))
>  			break;
> 
> +		if ((blocknr == first_transaction_blocknr) &&
> +		    (cur_counts != 0) && dump_old && (dump_counts != -1)) {
> +			fprintf(out_file, "Dump all %lld journal records.\n", cur_counts);
> +			break;
> +		}
> +
>  		retval = read_journal_block(cmdname, source,
>  				((ext2_loff_t) blocknr) * blocksize,
>  				buf, blocksize);

Re: [PATCH] debugfs:fix repeated output problem with `logdump -O -n <num_trans>`

[lihaoxiang (F)]

Dear professor Tytso,the following patch has been tested completely, looking forward to your response~~~

From c6bbaf343479091c3b55b0a54d4452ef16c91c2d Mon Sep 17 00:00:00 2001
From: lihaoxiang <lihaoxiang9@huawei.com>
Date: Thu, 10 Nov 2022 17:09:20 +0800
Subject: [PATCH] debugfs:fix repeated output problem with `logdump -O -n
 <num_trans>`

Previously, patch 6e4cc3d5eeb2dfaa055e652b5390beaa6c3d05da introduces
the function of printing the specified number of logs. But there exists
a shortage when n is larger than the total number of logs, it dumped the
duplicated records circulately.

For example, the disk sda only has three records, but using instruction logdump
-On5, it would output the result as follow:
----------------------------------------------------------------------
Journal starts at block 1, transaction 6
Found expected sequence 6, type 1 (descriptor block) at block 1
Found expected sequence 6, type 2 (commit block) at block 4
No magic number at block 5: end of journal.
Found sequence 2 (not 7) at block 7: end of journal.
Found expected sequence 2, type 2 (commit block) at block 7
Found sequence 3 (not 8) at block 8: end of journal.
Found expected sequence 3, type 1 (descriptor block) at block 8
Found sequence 3 (not 8) at block 15: end of journal.
Found expected sequence 3, type 2 (commit block) at block 15
Found sequence 6 (not 9) at block 1: end of journal.       <---------begin loop
Found expected sequence 6, type 1 (descriptor block) at block 1
Found sequence 6 (not 9) at block 4: end of journal.
Found expected sequence 6, type 2 (commit block) at block 4
Found sequence 2 (not 10) at block 7: end of journal.
Found expected sequence 2, type 2 (commit block) at block 7
logdump: short read (read 0, expected 1024) while reading journal

In this commit, we solve the problem above by exiting dumping if the
blocknr had already encountered, displayed the total number of logs
that the disk only possessed.

Signed-off-by: lihaoxiang <lihaoxiang9@huawei.com>
---
 debugfs/logdump.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/debugfs/logdump.c b/debugfs/logdump.c
index 614414e5..bfb2052e 100644
--- a/debugfs/logdump.c
+++ b/debugfs/logdump.c
@@ -376,6 +376,7 @@ static void dump_journal(char *cmdname, FILE *out_file,
 	journal_header_t	*header;
 	tid_t			transaction;
 	unsigned int		blocknr = 0;
+	unsigned int		first_transaction_blocknr;
 	int			fc_done;
 	__u64			total_len;
 	__u32			maxlen;
@@ -470,10 +471,19 @@ static void dump_journal(char *cmdname, FILE *out_file,
 			blocknr = 1;
 	}

+	first_transaction_blocknr = blocknr;
+
 	while (1) {
 		if (dump_old && (dump_counts != -1) && (cur_counts >= dump_counts))
 			break;

+		if ((blocknr == first_transaction_blocknr) &&
+		    ((cur_counts != 0) || (cur_counts == 0 && exist_no_magic)) &&
+		    dump_old && (dump_counts != -1)) {
+			fprintf(out_file, "Dump all %lld journal records.\n", cur_counts);
+			break;
+		}
+
 		retval = read_journal_block(cmdname, source,
 				((ext2_loff_t) blocknr) * blocksize,
 				buf, blocksize);
-- 
2.37.0.windows.1

Re: [PATCH] debugfs:fix repeated output problem with logdump

[lihaoxiang (F)]

Hi professor Tytso, we found that logdump has the problem to print circulately in the case which logs area are full of valid transactions. We propose the final solution as follow:
---------------------------------------------------------------------
Date: Thu, 10 Nov 2022 17:09:20 +0800
Subject: [PATCH] debugfs:fix repeated output problem with logdump

Currently, the module of logdump with parameter -O or -On might fall
into printing duplicate transactions circulately.

In fact, it would exit when it check the no magic number in transaction's
block head. But sometimes there hasn't no magic number block spanning
the whole log area so it cause the multi meaningless loop until a new no
magic block was founded.

We record the first blocknr of logs for comparing this value to that when
it meet the block again. If the comparison is equal then it exit immediately.

Signed-off-by: lihaoxiang <lihaoxiang9@huawei.com>
---
 debugfs/logdump.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/debugfs/logdump.c b/debugfs/logdump.c
index 614414e5..a6d03f5c 100644
--- a/debugfs/logdump.c
+++ b/debugfs/logdump.c
@@ -376,11 +376,14 @@ static void dump_journal(char *cmdname, FILE *out_file,
 	journal_header_t	*header;
 	tid_t			transaction;
 	unsigned int		blocknr = 0;
+	unsigned int		last_blocknr;
+	unsigned int		first_transaction_blocknr;
 	int			fc_done;
 	__u64			total_len;
 	__u32			maxlen;
 	int64_t			cur_counts = 0;
 	bool			exist_no_magic = false;
+	bool			reverse_flag = false;

 	/* First, check to see if there's an ext2 superblock header */
 	retval = read_journal_block(cmdname, source, 0, buf, 2048);
@@ -470,10 +473,22 @@ static void dump_journal(char *cmdname, FILE *out_file,
 			blocknr = 1;
 	}

+	first_transaction_blocknr = blocknr;
+	last_blocknr = blocknr - 1;
+
 	while (1) {
 		if (dump_old && (dump_counts != -1) && (cur_counts >= dump_counts))
 			break;

+		if (last_blocknr != (blocknr - 1))
+			reverse_flag = true;
+		last_blocknr = blocknr;
+
+		if ((blocknr == first_transaction_blocknr) && reverse_flag) {
+			fprintf(out_file, "Dump all %lld journal records.\n", cur_counts);
+			break;
+		}
+
 		retval = read_journal_block(cmdname, source,
 				((ext2_loff_t) blocknr) * blocksize,
 				buf, blocksize);
-- 
2.37.0.windows.1

Re: [PATCH] debugfs:fix repeated output problem with logdump

[lihaoxiang (F)]

friendly ping...

On 2022/12/15 10:31, lihaoxiang (F) wrote:
> Hi professor Tytso, we found that logdump has the problem to print circulately in the case which logs area are full of valid transactions. We propose the final solution as follow:
> ---------------------------------------------------------------------
> Date: Thu, 10 Nov 2022 17:09:20 +0800
> Subject: [PATCH] debugfs:fix repeated output problem with logdump
> 
> Currently, the module of logdump with parameter -O or -On might fall
> into printing duplicate transactions circulately.
> 
> In fact, it would exit when it check the no magic number in transaction's
> block head. But sometimes there hasn't no magic number block spanning
> the whole log area so it cause the multi meaningless loop until a new no
> magic block was founded.
> 
> We record the first blocknr of logs for comparing this value to that when
> it meet the block again. If the comparison is equal then it exit immediately.
> 
> Signed-off-by: lihaoxiang <lihaoxiang9@huawei.com>
> ---
>  debugfs/logdump.c | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
> 
> diff --git a/debugfs/logdump.c b/debugfs/logdump.c
> index 614414e5..a6d03f5c 100644
> --- a/debugfs/logdump.c
> +++ b/debugfs/logdump.c
> @@ -376,11 +376,14 @@ static void dump_journal(char *cmdname, FILE *out_file,
>  	journal_header_t	*header;
>  	tid_t			transaction;
>  	unsigned int		blocknr = 0;
> +	unsigned int		last_blocknr;
> +	unsigned int		first_transaction_blocknr;
>  	int			fc_done;
>  	__u64			total_len;
>  	__u32			maxlen;
>  	int64_t			cur_counts = 0;
>  	bool			exist_no_magic = false;
> +	bool			reverse_flag = false;
> 
>  	/* First, check to see if there's an ext2 superblock header */
>  	retval = read_journal_block(cmdname, source, 0, buf, 2048);
> @@ -470,10 +473,22 @@ static void dump_journal(char *cmdname, FILE *out_file,
>  			blocknr = 1;
>  	}
> 
> +	first_transaction_blocknr = blocknr;
> +	last_blocknr = blocknr - 1;
> +
>  	while (1) {
>  		if (dump_old && (dump_counts != -1) && (cur_counts >= dump_counts))
>  			break;
> 
> +		if (last_blocknr != (blocknr - 1))
> +			reverse_flag = true;
> +		last_blocknr = blocknr;
> +
> +		if ((blocknr == first_transaction_blocknr) && reverse_flag) {
> +			fprintf(out_file, "Dump all %lld journal records.\n", cur_counts);
> +			break;
> +		}
> +
>  		retval = read_journal_block(cmdname, source,
>  				((ext2_loff_t) blocknr) * blocksize,
>  				buf, blocksize);

Re: [PATCH] debugfs:fix repeated output problem with `logdump -O -n <num_trans>`

[Theodore Ts'o]

On Tue, 15 Nov 2022 16:29:55 +0800, lihaoxiang (F) wrote:
> Previously, patch 6e4cc3d5eeb2dfaa055e652b5390beaa6c3d05da introduces
> the function of printing the specified number of logs. But there exists
> a shortage when n is larger than the total number of logs, it dumped the
> duplicated records circulately.
> 
> For example, the disk sda only has three records, but using instruction logdump
> -On5, it would output the result as follow:
> ----------------------------------------------------------------------
> Journal starts at block 1, transaction 6
> Found expected sequence 6, type 1 (descriptor block) at block 1
> Found expected sequence 6, type 2 (commit block) at block 4
> No magic number at block 5: end of journal.
> Found sequence 2 (not 7) at block 7: end of journal.
> Found expected sequence 2, type 2 (commit block) at block 7
> Found sequence 3 (not 8) at block 8: end of journal.
> Found expected sequence 3, type 1 (descriptor block) at block 8
> Found sequence 3 (not 8) at block 15: end of journal.
> Found expected sequence 3, type 2 (commit block) at block 15
> Found sequence 6 (not 9) at block 1: end of journal.       <---------begin loop
> Found expected sequence 6, type 1 (descriptor block) at block 1
> Found sequence 6 (not 9) at block 4: end of journal.
> Found expected sequence 6, type 2 (commit block) at block 4
> Found sequence 2 (not 10) at block 7: end of journal.
> Found expected sequence 2, type 2 (commit block) at block 7
> logdump: short read (read 0, expected 1024) while reading journal
> 
> [...]

Applied, thanks!

[1/1] debugfs:fix repeated output problem with `logdump -O -n <num_trans>`
      commit: 9562cfca93da6d70494c3a66bddbc9d439410a34

Best regards,
-- 
Theodore Ts'o <tytso@mit.edu>