* Re: [Ocfs2-devel] [PATCH v3 1/2] jbd2: continue to record log between each mount [not found] ` <20230315094826.okdarxaapjyqmlhq@quack3> @ 2023-03-15 12:37 ` Zhang Yi via Ocfs2-devel 2023-03-15 17:28 ` Jan Kara via Ocfs2-devel 0 siblings, 1 reply; 4+ messages in thread From: Zhang Yi via Ocfs2-devel @ 2023-03-15 12:37 UTC (permalink / raw) To: Jan Kara, Zhang Yi Cc: adilger.kernel, linux-ext4, tytso, ocfs2-devel, yukuai3 On 2023/3/15 17:48, Jan Kara wrote: > On Tue 14-03-23 22:05:21, Zhang Yi wrote: >> From: Zhang Yi <yi.zhang@huawei.com> >> >> For a newly mounted file system, the journal committing thread always >> record new transactions from the start of the journal area, no matter >> whether the journal was clean or just has been recovered. So the logdump >> code in debugfs cannot dump continuous logs between each mount, it is >> disadvantageous to analysis corrupted file system image and locate the >> file system inconsistency bugs. >> >> If we get a corrupted file system in the running products and want to >> find out what has happened, besides lookup the system log, one effective >> way is to backtrack the journal log. But we may not always run e2fsck >> before each mount and the default fsck -a mode also cannot always >> checkout all inconsistencies, so it could left over some inconsistencies >> into the next mount until we detect it. Finally, transactions in the >> journal may probably discontinuous and some relatively new transactions >> has been covered, it becomes hard to analyse. If we could record >> transactions continuously between each mount, we could acquire more >> useful info from the journal. Like this: >> >> |Previous mount checkpointed/recovered logs|Current mount logs | >> |{------}{---}{--------} ... {------}| ... |{======}{========}...000000| >> >> And yes the journal area is limited and cannot record everything, the >> problematic transaction may also be covered even if we do this, but >> this is still useful for fuzzy tests and short-running products. >> >> This patch save the head blocknr in the superblock after flushing the >> journal or unmounting the file system, let the next mount could continue >> to record new transaction behind it. This change is backward compatible >> because the old kernel does not care about the head blocknr of the >> journal. It is also fine if we mount a clean old image without valid >> head blocknr, we fail back to set it to s_first just like before. >> Finally, for the case of mount an unclean file system, we could also get >> the journal head easily after scanning/replaying the journal, it will >> continue to record new transaction after the recovered transactions. >> >> Signed-off-by: Zhang Yi <yi.zhang@huawei.com> > > I like this implementation! I even think we could perhaps make ext4 always > behave this way to not increase size of the test matrix. Or do you see any > downside to this option? > Thanks for your suggestion. Indeed, I don't find any side effect on this option both in theory and in the actual use tests on ext4, I added a new option was just from the safe point of view and let user could disable it if they don't want it. I also prefer to make ext4 always behave this way.:) I would like to keep the JBD2_CYCLE_RECORD flag(ocfs2 also use jbd2, I don't want to disturb it until it needs), remove EXT4_MOUNT2_JOURNAL_CYCLE_RECORD and always set JBD2_CYCLE_RECORD on ext4 in patch 2 in the next iteration. Thanks, Yi. _______________________________________________ Ocfs2-devel mailing list Ocfs2-devel@oss.oracle.com https://oss.oracle.com/mailman/listinfo/ocfs2-devel ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Ocfs2-devel] [PATCH v3 1/2] jbd2: continue to record log between each mount 2023-03-15 12:37 ` [Ocfs2-devel] [PATCH v3 1/2] jbd2: continue to record log between each mount Zhang Yi via Ocfs2-devel @ 2023-03-15 17:28 ` Jan Kara via Ocfs2-devel 2023-03-17 11:25 ` Jan Kara via Ocfs2-devel 0 siblings, 1 reply; 4+ messages in thread From: Jan Kara via Ocfs2-devel @ 2023-03-15 17:28 UTC (permalink / raw) To: Zhang Yi Cc: Jan Kara, Zhang Yi, adilger.kernel, yukuai3, tytso, linux-ext4, ocfs2-devel On Wed 15-03-23 20:37:32, Zhang Yi wrote: > On 2023/3/15 17:48, Jan Kara wrote: > > On Tue 14-03-23 22:05:21, Zhang Yi wrote: > >> From: Zhang Yi <yi.zhang@huawei.com> > >> > >> For a newly mounted file system, the journal committing thread always > >> record new transactions from the start of the journal area, no matter > >> whether the journal was clean or just has been recovered. So the logdump > >> code in debugfs cannot dump continuous logs between each mount, it is > >> disadvantageous to analysis corrupted file system image and locate the > >> file system inconsistency bugs. > >> > >> If we get a corrupted file system in the running products and want to > >> find out what has happened, besides lookup the system log, one effective > >> way is to backtrack the journal log. But we may not always run e2fsck > >> before each mount and the default fsck -a mode also cannot always > >> checkout all inconsistencies, so it could left over some inconsistencies > >> into the next mount until we detect it. Finally, transactions in the > >> journal may probably discontinuous and some relatively new transactions > >> has been covered, it becomes hard to analyse. If we could record > >> transactions continuously between each mount, we could acquire more > >> useful info from the journal. Like this: > >> > >> |Previous mount checkpointed/recovered logs|Current mount logs | > >> |{------}{---}{--------} ... {------}| ... |{======}{========}...000000| > >> > >> And yes the journal area is limited and cannot record everything, the > >> problematic transaction may also be covered even if we do this, but > >> this is still useful for fuzzy tests and short-running products. > >> > >> This patch save the head blocknr in the superblock after flushing the > >> journal or unmounting the file system, let the next mount could continue > >> to record new transaction behind it. This change is backward compatible > >> because the old kernel does not care about the head blocknr of the > >> journal. It is also fine if we mount a clean old image without valid > >> head blocknr, we fail back to set it to s_first just like before. > >> Finally, for the case of mount an unclean file system, we could also get > >> the journal head easily after scanning/replaying the journal, it will > >> continue to record new transaction after the recovered transactions. > >> > >> Signed-off-by: Zhang Yi <yi.zhang@huawei.com> > > > > I like this implementation! I even think we could perhaps make ext4 always > > behave this way to not increase size of the test matrix. Or do you see any > > downside to this option? > > > > Thanks for your suggestion. Indeed, I don't find any side effect on this > option both in theory and in the actual use tests on ext4, I added a new > option was just from the safe point of view and let user could disable it if > they don't want it. I also prefer to make ext4 always behave this way.:) > > I would like to keep the JBD2_CYCLE_RECORD flag(ocfs2 also use jbd2, I don't > want to disturb it until it needs), remove EXT4_MOUNT2_JOURNAL_CYCLE_RECORD > and always set JBD2_CYCLE_RECORD on ext4 in patch 2 in the next iteration. Yes, that makes sense. Honza -- Jan Kara <jack@suse.com> SUSE Labs, CR _______________________________________________ Ocfs2-devel mailing list Ocfs2-devel@oss.oracle.com https://oss.oracle.com/mailman/listinfo/ocfs2-devel ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Ocfs2-devel] [PATCH v3 1/2] jbd2: continue to record log between each mount 2023-03-15 17:28 ` Jan Kara via Ocfs2-devel @ 2023-03-17 11:25 ` Jan Kara via Ocfs2-devel 2023-03-18 2:25 ` Zhang Yi via Ocfs2-devel 0 siblings, 1 reply; 4+ messages in thread From: Jan Kara via Ocfs2-devel @ 2023-03-17 11:25 UTC (permalink / raw) To: Zhang Yi Cc: Jan Kara, Zhang Yi, adilger.kernel, yukuai3, tytso, linux-ext4, ocfs2-devel On Wed 15-03-23 18:28:17, Jan Kara wrote: > On Wed 15-03-23 20:37:32, Zhang Yi wrote: > > On 2023/3/15 17:48, Jan Kara wrote: > > > On Tue 14-03-23 22:05:21, Zhang Yi wrote: > > >> From: Zhang Yi <yi.zhang@huawei.com> > > >> > > >> For a newly mounted file system, the journal committing thread always > > >> record new transactions from the start of the journal area, no matter > > >> whether the journal was clean or just has been recovered. So the logdump > > >> code in debugfs cannot dump continuous logs between each mount, it is > > >> disadvantageous to analysis corrupted file system image and locate the > > >> file system inconsistency bugs. > > >> > > >> If we get a corrupted file system in the running products and want to > > >> find out what has happened, besides lookup the system log, one effective > > >> way is to backtrack the journal log. But we may not always run e2fsck > > >> before each mount and the default fsck -a mode also cannot always > > >> checkout all inconsistencies, so it could left over some inconsistencies > > >> into the next mount until we detect it. Finally, transactions in the > > >> journal may probably discontinuous and some relatively new transactions > > >> has been covered, it becomes hard to analyse. If we could record > > >> transactions continuously between each mount, we could acquire more > > >> useful info from the journal. Like this: > > >> > > >> |Previous mount checkpointed/recovered logs|Current mount logs | > > >> |{------}{---}{--------} ... {------}| ... |{======}{========}...000000| > > >> > > >> And yes the journal area is limited and cannot record everything, the > > >> problematic transaction may also be covered even if we do this, but > > >> this is still useful for fuzzy tests and short-running products. > > >> > > >> This patch save the head blocknr in the superblock after flushing the > > >> journal or unmounting the file system, let the next mount could continue > > >> to record new transaction behind it. This change is backward compatible > > >> because the old kernel does not care about the head blocknr of the > > >> journal. It is also fine if we mount a clean old image without valid > > >> head blocknr, we fail back to set it to s_first just like before. > > >> Finally, for the case of mount an unclean file system, we could also get > > >> the journal head easily after scanning/replaying the journal, it will > > >> continue to record new transaction after the recovered transactions. > > >> > > >> Signed-off-by: Zhang Yi <yi.zhang@huawei.com> > > > > > > I like this implementation! I even think we could perhaps make ext4 always > > > behave this way to not increase size of the test matrix. Or do you see any > > > downside to this option? > > > > > > > Thanks for your suggestion. Indeed, I don't find any side effect on this > > option both in theory and in the actual use tests on ext4, I added a new > > option was just from the safe point of view and let user could disable it if > > they don't want it. I also prefer to make ext4 always behave this way.:) > > > > I would like to keep the JBD2_CYCLE_RECORD flag(ocfs2 also use jbd2, I don't > > want to disturb it until it needs), remove EXT4_MOUNT2_JOURNAL_CYCLE_RECORD > > and always set JBD2_CYCLE_RECORD on ext4 in patch 2 in the next iteration. > > Yes, that makes sense. FWIW yesterday I'v spoken with Ted and he also agrees that we don't need ext4 mount option for this. Honza -- Jan Kara <jack@suse.com> SUSE Labs, CR _______________________________________________ Ocfs2-devel mailing list Ocfs2-devel@oss.oracle.com https://oss.oracle.com/mailman/listinfo/ocfs2-devel ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Ocfs2-devel] [PATCH v3 1/2] jbd2: continue to record log between each mount 2023-03-17 11:25 ` Jan Kara via Ocfs2-devel @ 2023-03-18 2:25 ` Zhang Yi via Ocfs2-devel 0 siblings, 0 replies; 4+ messages in thread From: Zhang Yi via Ocfs2-devel @ 2023-03-18 2:25 UTC (permalink / raw) To: Jan Kara Cc: tytso, Zhang Yi, adilger.kernel, yukuai3, linux-ext4, ocfs2-devel On 2023/3/17 19:25, Jan Kara wrote: > On Wed 15-03-23 18:28:17, Jan Kara wrote: >> On Wed 15-03-23 20:37:32, Zhang Yi wrote: >>> On 2023/3/15 17:48, Jan Kara wrote: >>>> On Tue 14-03-23 22:05:21, Zhang Yi wrote: >>>>> From: Zhang Yi <yi.zhang@huawei.com> >>>>> >>>>> For a newly mounted file system, the journal committing thread always >>>>> record new transactions from the start of the journal area, no matter >>>>> whether the journal was clean or just has been recovered. So the logdump >>>>> code in debugfs cannot dump continuous logs between each mount, it is >>>>> disadvantageous to analysis corrupted file system image and locate the >>>>> file system inconsistency bugs. >>>>> >>>>> If we get a corrupted file system in the running products and want to >>>>> find out what has happened, besides lookup the system log, one effective >>>>> way is to backtrack the journal log. But we may not always run e2fsck >>>>> before each mount and the default fsck -a mode also cannot always >>>>> checkout all inconsistencies, so it could left over some inconsistencies >>>>> into the next mount until we detect it. Finally, transactions in the >>>>> journal may probably discontinuous and some relatively new transactions >>>>> has been covered, it becomes hard to analyse. If we could record >>>>> transactions continuously between each mount, we could acquire more >>>>> useful info from the journal. Like this: >>>>> >>>>> |Previous mount checkpointed/recovered logs|Current mount logs | >>>>> |{------}{---}{--------} ... {------}| ... |{======}{========}...000000| >>>>> >>>>> And yes the journal area is limited and cannot record everything, the >>>>> problematic transaction may also be covered even if we do this, but >>>>> this is still useful for fuzzy tests and short-running products. >>>>> >>>>> This patch save the head blocknr in the superblock after flushing the >>>>> journal or unmounting the file system, let the next mount could continue >>>>> to record new transaction behind it. This change is backward compatible >>>>> because the old kernel does not care about the head blocknr of the >>>>> journal. It is also fine if we mount a clean old image without valid >>>>> head blocknr, we fail back to set it to s_first just like before. >>>>> Finally, for the case of mount an unclean file system, we could also get >>>>> the journal head easily after scanning/replaying the journal, it will >>>>> continue to record new transaction after the recovered transactions. >>>>> >>>>> Signed-off-by: Zhang Yi <yi.zhang@huawei.com> >>>> >>>> I like this implementation! I even think we could perhaps make ext4 always >>>> behave this way to not increase size of the test matrix. Or do you see any >>>> downside to this option? >>>> >>> >>> Thanks for your suggestion. Indeed, I don't find any side effect on this >>> option both in theory and in the actual use tests on ext4, I added a new >>> option was just from the safe point of view and let user could disable it if >>> they don't want it. I also prefer to make ext4 always behave this way.:) >>> >>> I would like to keep the JBD2_CYCLE_RECORD flag(ocfs2 also use jbd2, I don't >>> want to disturb it until it needs), remove EXT4_MOUNT2_JOURNAL_CYCLE_RECORD >>> and always set JBD2_CYCLE_RECORD on ext4 in patch 2 in the next iteration. >> >> Yes, that makes sense. > > FWIW yesterday I'v spoken with Ted and he also agrees that we don't need > ext4 mount option for this. > Thanks! I've removed this mount option in v4. Yi. _______________________________________________ Ocfs2-devel mailing list Ocfs2-devel@oss.oracle.com https://oss.oracle.com/mailman/listinfo/ocfs2-devel ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-03-18 2:25 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <20230314140522.3266591-1-yi.zhang@huaweicloud.com> [not found] ` <20230314140522.3266591-2-yi.zhang@huaweicloud.com> [not found] ` <20230315094826.okdarxaapjyqmlhq@quack3> 2023-03-15 12:37 ` [Ocfs2-devel] [PATCH v3 1/2] jbd2: continue to record log between each mount Zhang Yi via Ocfs2-devel 2023-03-15 17:28 ` Jan Kara via Ocfs2-devel 2023-03-17 11:25 ` Jan Kara via Ocfs2-devel 2023-03-18 2:25 ` Zhang Yi via Ocfs2-devel
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).