From: Chao Yu <yuchao0@huawei.com> To: David Laight <David.Laight@ACULAB.COM>, Jaegeuk Kim <jaegeuk@kernel.org> Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "linux-f2fs-devel@lists.sourceforge.net" <linux-f2fs-devel@lists.sourceforge.net> Subject: Re: [f2fs-dev] [PATCH] f2fs: compress: support chksum Date: Tue, 3 Nov 2020 19:51:05 +0800 [thread overview] Message-ID: <4d58b7c7-70b5-514d-fab2-27a0f16886e5@huawei.com> (raw) In-Reply-To: <aa11afd31edb42979c03d2a27ed9e850@AcuMS.aculab.com> On 2020/11/3 18:35, David Laight wrote: > From: Chao Yu >> Sent: 03 November 2020 02:37 > ... >>>> Do we need to change fsck.f2fs to recover this? >> >> However, we don't know which one is correct, compressed data or chksum value? >> if compressed data was corrupted, repairing chksum value doesn't help. >> >> Or how about adding chksum values for both raw data and compressed data. > > What errors are you trying to detect? Hi, The original intention of adding this checksum feature is for code debug purpose when I develop compress framework in f2fs and add more compress algorithms into the framework, it helps to find obvious implementation bug, however this checksum feature was not fully designed, so that I didn't upstream it at that time. One other concern is to find any mismatch between original raw data and persisted data, no matter how it becomes to mismatched and then return error code to user if it detects the mismatch. And then fsck can repair mismatched chksum in the condition one persisted chksum matchs to calculated one, and one other doesn't. Thanks, > > If there are errors in the data then 'fixing' the checksum is pointless. > (You've got garbage data - might as well not have the checksum). > > If you are worried about the implementation of the compression > algorithm then a checksum of the raw data is needed. > > If you want to try error correcting burst errors in the compressed > data then a crc of the compressed data can be used for error correction. > > OTOH the most likely error is that the file meta-data and data sector > weren't both committed to disk when the system crashed. > In which case the checksum has done its job and the file is corrupt. > fsck should probably move the file to 'lost+found' for manual checking. > > David > > - > Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK > Registration No: 1397386 (Wales) > > . >
WARNING: multiple messages have this Message-ID (diff)
From: Chao Yu <yuchao0@huawei.com> To: David Laight <David.Laight@ACULAB.COM>, Jaegeuk Kim <jaegeuk@kernel.org> Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "linux-f2fs-devel@lists.sourceforge.net" <linux-f2fs-devel@lists.sourceforge.net> Subject: Re: [f2fs-dev] [PATCH] f2fs: compress: support chksum Date: Tue, 3 Nov 2020 19:51:05 +0800 [thread overview] Message-ID: <4d58b7c7-70b5-514d-fab2-27a0f16886e5@huawei.com> (raw) In-Reply-To: <aa11afd31edb42979c03d2a27ed9e850@AcuMS.aculab.com> On 2020/11/3 18:35, David Laight wrote: > From: Chao Yu >> Sent: 03 November 2020 02:37 > ... >>>> Do we need to change fsck.f2fs to recover this? >> >> However, we don't know which one is correct, compressed data or chksum value? >> if compressed data was corrupted, repairing chksum value doesn't help. >> >> Or how about adding chksum values for both raw data and compressed data. > > What errors are you trying to detect? Hi, The original intention of adding this checksum feature is for code debug purpose when I develop compress framework in f2fs and add more compress algorithms into the framework, it helps to find obvious implementation bug, however this checksum feature was not fully designed, so that I didn't upstream it at that time. One other concern is to find any mismatch between original raw data and persisted data, no matter how it becomes to mismatched and then return error code to user if it detects the mismatch. And then fsck can repair mismatched chksum in the condition one persisted chksum matchs to calculated one, and one other doesn't. Thanks, > > If there are errors in the data then 'fixing' the checksum is pointless. > (You've got garbage data - might as well not have the checksum). > > If you are worried about the implementation of the compression > algorithm then a checksum of the raw data is needed. > > If you want to try error correcting burst errors in the compressed > data then a crc of the compressed data can be used for error correction. > > OTOH the most likely error is that the file meta-data and data sector > weren't both committed to disk when the system crashed. > In which case the checksum has done its job and the file is corrupt. > fsck should probably move the file to 'lost+found' for manual checking. > > David > > - > Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK > Registration No: 1397386 (Wales) > > . > _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
next prev parent reply other threads:[~2020-11-03 11:51 UTC|newest] Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-11-02 12:23 [PATCH] f2fs: compress: support chksum Chao Yu 2020-11-02 12:23 ` [f2fs-dev] " Chao Yu 2020-11-02 16:31 ` Jaegeuk Kim 2020-11-02 16:31 ` [f2fs-dev] " Jaegeuk Kim 2020-11-03 2:02 ` Chao Yu 2020-11-03 2:02 ` [f2fs-dev] " Chao Yu 2020-11-03 2:36 ` Chao Yu 2020-11-03 2:36 ` Chao Yu 2020-11-03 10:35 ` David Laight 2020-11-03 10:35 ` David Laight 2020-11-03 11:51 ` Chao Yu [this message] 2020-11-03 11:51 ` Chao Yu 2020-11-05 1:13 ` Chao Yu 2020-11-05 1:13 ` Chao Yu 2020-11-06 21:12 ` Jaegeuk Kim 2020-11-06 21:12 ` Jaegeuk Kim 2020-11-09 1:49 ` Chao Yu 2020-11-09 1:49 ` Chao Yu 2020-11-09 17:06 ` Jaegeuk Kim 2020-11-09 17:06 ` Jaegeuk Kim 2020-11-10 1:52 ` Chao Yu 2020-11-10 1:52 ` Chao Yu 2020-11-10 4:23 ` Jaegeuk Kim 2020-11-10 4:23 ` Jaegeuk Kim 2020-11-10 6:27 ` Chao Yu 2020-11-10 6:27 ` Chao Yu 2020-11-10 12:16 ` David Laight 2020-11-10 12:16 ` David Laight 2020-11-02 17:37 ` kernel test robot 2020-11-02 17:37 ` kernel test robot
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=4d58b7c7-70b5-514d-fab2-27a0f16886e5@huawei.com \ --to=yuchao0@huawei.com \ --cc=David.Laight@ACULAB.COM \ --cc=jaegeuk@kernel.org \ --cc=linux-f2fs-devel@lists.sourceforge.net \ --cc=linux-kernel@vger.kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.