From: Gao Xiang <hsiangkao@aol.com> To: Christoph Hellwig <hch@infradead.org>, "Theodore Y. Ts'o" <tytso@mit.edu>, Eric Biggers <ebiggers@kernel.org>, Richard Weinberger <richard@nod.at> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Jan Kara <jack@suse.cz>, Chao Yu <yuchao0@huawei.com>, Dave Chinner <david@fromorbit.com>, David Sterba <dsterba@suse.cz>, Miao Xie <miaoxie@huawei.com>, devel <devel@driverdev.osuosl.org>, Stephen Rothwell <sfr@canb.auug.org.au>, Darrick <darrick.wong@oracle.com>, Amir Goldstein <amir73il@gmail.com>, linux-erofs <linux-erofs@lists.ozlabs.org>, Al Viro <viro@zeniv.linux.org.uk>, Jaegeuk Kim <jaegeuk@kernel.org>, linux-kernel <linux-kernel@vger.kernel.org>, Li Guifu <bluce.liguifu@huawei.com>, Fang Wei <fangwei1@huawei.com>, Pavel Machek <pavel@denx.de>, linux-fsdevel <linux-fsdevel@vger.kernel.org>, Andrew Morton <akpm@linux-foundation.org>, torvalds <torvalds@linux-foundation.org> Subject: Re: [PATCH] erofs: move erofs out of staging Date: Mon, 19 Aug 2019 04:14:11 +0800 [thread overview] Message-ID: <20190818201405.GA27398@hsiangkao-HP-ZHAN-66-Pro-G1> (raw) In-Reply-To: <20190818181654.GA1617@hsiangkao-HP-ZHAN-66-Pro-G1> Hi all, On Mon, Aug 19, 2019 at 02:16:55AM +0800, Gao Xiang wrote: > Hi Hch, > > On Sun, Aug 18, 2019 at 10:47:02AM -0700, Christoph Hellwig wrote: > > On Sun, Aug 18, 2019 at 10:29:38AM -0700, Eric Biggers wrote: > > > Not sure what you're even disagreeing with, as I *do* expect new filesystems to > > > be held to a high standard, and to be written with the assumption that the > > > on-disk data may be corrupted or malicious. We just can't expect the bar to be > > > so high (e.g. no bugs) that it's never been attained by *any* filesystem even > > > after years/decades of active development. If the developers were careful, the > > > code generally looks robust, and they are willing to address such bugs as they > > > are found, realistically that's as good as we can expect to get... > > > > Well, the impression I got from Richards quick look and the reply to it is > > that there is very little attempt to validate the ondisk data structure > > and there is absolutely no priority to do so. Which is very different > > from there is a bug or two here and there. > > As my second reply to Richard, I didn't fuzz all the on-disk fields for EROFS. > and as my reply to Richard / Greg, current EROFS is used on the top of dm-verity. > > I cannot say how well EROFS will be performed on malformed images (and you can > also find the bug richard pointed out is a miswritten break->continue by myself). > > I posted the upstream EROFS post on July 4, 2019 and a month and a half later, > no one can tell me (yes, thanks for kind people reply me about their suggestion) > what we should do next (you can see these emails, I sent many times) to meet > the minimal upstream requirements and rare people can even dip into my code. > > That is all I want to say. I will work on autofuzz these days, and I want to > know how to meet your requirements on this (you can tell us your standard, > how well should we do). > > OK, you don't reply to my post once, I have no idea how to get your first reply. I have made a simple fuzzer to inject messy in inode metadata, dir data, compressed indexes and super block, https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer I am testing with some given dirs and the following script. Does it look reasonable? # !/bin/bash mkdir -p mntdir for ((i=0; i<1000; ++i)); do mkfs/mkfs.erofs -F$i testdir_fsl.fuzz.img testdir_fsl > /dev/null 2>&1 umount mntdir mount -t erofs -o loop testdir_fsl.fuzz.img mntdir for j in `find mntdir -type f`; do md5sum $j > /dev/null done done Thanks, Gao Xiang > > Thanks, > Gao Xiang >
WARNING: multiple messages have this Message-ID (diff)
From: hsiangkao@aol.com (Gao Xiang) Subject: [PATCH] erofs: move erofs out of staging Date: Mon, 19 Aug 2019 04:14:11 +0800 [thread overview] Message-ID: <20190818201405.GA27398@hsiangkao-HP-ZHAN-66-Pro-G1> (raw) In-Reply-To: <20190818181654.GA1617@hsiangkao-HP-ZHAN-66-Pro-G1> Hi all, On Mon, Aug 19, 2019@02:16:55AM +0800, Gao Xiang wrote: > Hi Hch, > > On Sun, Aug 18, 2019@10:47:02AM -0700, Christoph Hellwig wrote: > > On Sun, Aug 18, 2019@10:29:38AM -0700, Eric Biggers wrote: > > > Not sure what you're even disagreeing with, as I *do* expect new filesystems to > > > be held to a high standard, and to be written with the assumption that the > > > on-disk data may be corrupted or malicious. We just can't expect the bar to be > > > so high (e.g. no bugs) that it's never been attained by *any* filesystem even > > > after years/decades of active development. If the developers were careful, the > > > code generally looks robust, and they are willing to address such bugs as they > > > are found, realistically that's as good as we can expect to get... > > > > Well, the impression I got from Richards quick look and the reply to it is > > that there is very little attempt to validate the ondisk data structure > > and there is absolutely no priority to do so. Which is very different > > from there is a bug or two here and there. > > As my second reply to Richard, I didn't fuzz all the on-disk fields for EROFS. > and as my reply to Richard / Greg, current EROFS is used on the top of dm-verity. > > I cannot say how well EROFS will be performed on malformed images (and you can > also find the bug richard pointed out is a miswritten break->continue by myself). > > I posted the upstream EROFS post on July 4, 2019 and a month and a half later, > no one can tell me (yes, thanks for kind people reply me about their suggestion) > what we should do next (you can see these emails, I sent many times) to meet > the minimal upstream requirements and rare people can even dip into my code. > > That is all I want to say. I will work on autofuzz these days, and I want to > know how to meet your requirements on this (you can tell us your standard, > how well should we do). > > OK, you don't reply to my post once, I have no idea how to get your first reply. I have made a simple fuzzer to inject messy in inode metadata, dir data, compressed indexes and super block, https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer I am testing with some given dirs and the following script. Does it look reasonable? # !/bin/bash mkdir -p mntdir for ((i=0; i<1000; ++i)); do mkfs/mkfs.erofs -F$i testdir_fsl.fuzz.img testdir_fsl > /dev/null 2>&1 umount mntdir mount -t erofs -o loop testdir_fsl.fuzz.img mntdir for j in `find mntdir -type f`; do md5sum $j > /dev/null done done Thanks, Gao Xiang > > Thanks, > Gao Xiang >
next prev parent reply other threads:[~2019-08-18 20:14 UTC|newest] Thread overview: 170+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-08-17 8:23 [PATCH] erofs: move erofs out of staging Gao Xiang 2019-08-17 8:23 ` Gao Xiang 2019-08-17 21:19 ` Richard Weinberger 2019-08-17 21:19 ` Richard Weinberger 2019-08-17 22:07 ` Gao Xiang 2019-08-17 22:07 ` Gao Xiang 2019-08-17 23:25 ` Richard Weinberger 2019-08-17 23:25 ` Richard Weinberger 2019-08-17 23:38 ` Gao Xiang 2019-08-17 23:38 ` Gao Xiang 2019-08-18 0:04 ` Gao Xiang 2019-08-18 0:04 ` Gao Xiang 2019-08-18 0:52 ` Gao Xiang 2019-08-18 0:52 ` Gao Xiang 2019-08-18 8:16 ` Richard Weinberger 2019-08-18 8:16 ` Richard Weinberger 2019-08-18 8:45 ` Gao Xiang 2019-08-18 8:45 ` Gao Xiang 2019-08-18 9:03 ` Richard Weinberger 2019-08-18 9:03 ` Richard Weinberger 2019-08-18 9:09 ` Greg Kroah-Hartman 2019-08-18 9:09 ` Greg Kroah-Hartman 2019-08-18 9:21 ` Richard Weinberger 2019-08-18 9:21 ` Richard Weinberger 2019-08-18 10:12 ` Chao Yu 2019-08-18 10:12 ` Chao Yu 2019-08-18 15:11 ` Theodore Y. Ts'o 2019-08-18 15:11 ` Theodore Y. Ts'o 2019-08-18 15:58 ` Christoph Hellwig 2019-08-18 15:58 ` Christoph Hellwig 2019-08-18 16:16 ` Eric Biggers 2019-08-18 16:16 ` Eric Biggers 2019-08-18 16:22 ` Christoph Hellwig 2019-08-18 16:22 ` Christoph Hellwig 2019-08-18 16:33 ` Gao Xiang 2019-08-18 16:33 ` Gao Xiang 2019-08-18 17:29 ` Eric Biggers 2019-08-18 17:29 ` Eric Biggers 2019-08-18 17:47 ` Christoph Hellwig 2019-08-18 17:47 ` Christoph Hellwig 2019-08-18 18:16 ` Gao Xiang 2019-08-18 18:16 ` Gao Xiang 2019-08-18 20:14 ` Gao Xiang [this message] 2019-08-18 20:14 ` Gao Xiang 2019-08-19 7:35 ` Richard Weinberger 2019-08-19 7:35 ` Richard Weinberger 2019-08-19 8:02 ` Gao Xiang 2019-08-19 8:02 ` Gao Xiang 2019-08-19 10:34 ` [PATCH 0/6] staging: erofs: first stage of corrupted compressed images Gao Xiang 2019-08-19 10:34 ` Gao Xiang 2019-08-19 10:34 ` [PATCH 1/6] staging: erofs: some compressed cluster should be submitted for corrupted images Gao Xiang 2019-08-19 10:34 ` Gao Xiang 2019-08-19 14:36 ` Chao Yu 2019-08-19 14:36 ` Chao Yu 2019-08-19 14:36 ` Chao Yu 2019-08-19 14:39 ` Chao Yu 2019-08-19 14:39 ` Chao Yu 2019-08-19 14:39 ` Chao Yu 2019-08-19 10:34 ` [PATCH 2/6] staging: erofs: cannot set EROFS_V_Z_INITED_BIT if fill_inode_lazy fails Gao Xiang 2019-08-19 10:34 ` Gao Xiang 2019-08-19 14:43 ` Chao Yu 2019-08-19 14:43 ` Chao Yu 2019-08-19 14:43 ` Chao Yu 2019-08-19 10:34 ` [PATCH 3/6] staging: erofs: add two missing erofs_workgroup_put for corrupted images Gao Xiang 2019-08-19 10:34 ` Gao Xiang 2019-08-19 14:40 ` Chao Yu 2019-08-19 14:40 ` Chao Yu 2019-08-19 14:40 ` Chao Yu 2019-08-19 10:34 ` [PATCH 4/6] staging: erofs: avoid loop in submit chains Gao Xiang 2019-08-19 10:34 ` Gao Xiang 2019-08-19 14:50 ` Chao Yu 2019-08-19 14:50 ` Chao Yu 2019-08-19 14:50 ` Chao Yu 2019-08-19 10:34 ` [PATCH 5/6] staging: erofs: detect potential multiref due to corrupted images Gao Xiang 2019-08-19 10:34 ` Gao Xiang 2019-08-19 14:57 ` Chao Yu 2019-08-19 14:57 ` Chao Yu 2019-08-19 14:57 ` Chao Yu 2019-08-21 2:19 ` Greg Kroah-Hartman 2019-08-21 2:19 ` Greg Kroah-Hartman 2019-08-21 2:19 ` Greg Kroah-Hartman 2019-08-21 14:01 ` [PATCH v2 " Gao Xiang 2019-08-21 14:01 ` Gao Xiang 2019-08-21 14:24 ` Chao Yu 2019-08-21 14:24 ` Chao Yu 2019-08-19 10:34 ` [PATCH 6/6] staging: erofs: avoid endless loop of invalid lookback distance 0 Gao Xiang 2019-08-19 10:34 ` Gao Xiang 2019-08-19 14:58 ` Chao Yu 2019-08-19 14:58 ` Chao Yu 2019-08-19 14:58 ` Chao Yu 2019-08-19 16:09 ` [PATCH] erofs: move erofs out of staging Darrick J. Wong 2019-08-19 16:09 ` Darrick J. Wong 2019-08-19 16:09 ` Darrick J. Wong 2019-08-19 20:30 ` Gao Xiang 2019-08-19 20:30 ` Gao Xiang via Linux-erofs 2019-08-19 20:30 ` Gao Xiang 2019-08-20 0:55 ` Qu Wenruo 2019-08-20 0:55 ` Qu Wenruo 2019-08-20 0:55 ` Qu Wenruo 2019-08-20 1:55 ` Gao Xiang 2019-08-20 1:55 ` Gao Xiang 2019-08-20 1:55 ` Gao Xiang 2019-08-20 2:24 ` Chao Yu 2019-08-20 2:24 ` Chao Yu 2019-08-20 2:24 ` Chao Yu 2019-08-20 2:38 ` Qu Wenruo 2019-08-20 2:38 ` Qu Wenruo 2019-08-20 2:38 ` Qu Wenruo 2019-08-20 7:15 ` Chao Yu 2019-08-20 7:15 ` Chao Yu 2019-08-20 7:15 ` Chao Yu 2019-08-20 8:46 ` Qu Wenruo 2019-08-20 8:46 ` Qu Wenruo 2019-08-20 8:46 ` Qu Wenruo 2019-08-21 2:12 ` Chao Yu 2019-08-21 2:12 ` Chao Yu 2019-08-21 2:12 ` Chao Yu 2019-08-20 15:56 ` Theodore Y. Ts'o 2019-08-20 15:56 ` Theodore Y. Ts'o 2019-08-20 15:56 ` Theodore Y. Ts'o 2019-08-20 16:35 ` Gao Xiang 2019-08-20 16:35 ` Gao Xiang via Linux-erofs 2019-08-20 16:35 ` Gao Xiang 2019-08-21 0:51 ` Theodore Y. Ts'o 2019-08-21 0:51 ` Theodore Y. Ts'o 2019-08-21 0:51 ` Theodore Y. Ts'o 2019-08-21 1:34 ` Chao Yu 2019-08-21 1:34 ` Chao Yu 2019-08-21 1:48 ` Darrick J. Wong 2019-08-21 1:48 ` Darrick J. Wong 2019-08-21 1:48 ` Darrick J. Wong 2019-08-21 1:57 ` Chao Yu 2019-08-21 1:57 ` Chao Yu 2019-08-21 1:57 ` Chao Yu 2019-08-20 3:33 ` Miao Xie 2019-08-20 3:33 ` Miao Xie 2019-08-20 3:33 ` Miao Xie 2019-08-20 3:46 ` Gao Xiang 2019-08-20 3:46 ` Gao Xiang 2019-08-20 3:46 ` Gao Xiang 2019-08-20 6:04 ` Qu Wenruo 2019-08-20 6:04 ` Qu Wenruo 2019-08-20 6:04 ` Qu Wenruo 2019-08-20 6:22 ` Gao Xiang 2019-08-20 6:22 ` Gao Xiang 2019-08-20 6:22 ` Gao Xiang 2019-08-19 7:37 ` Richard Weinberger 2019-08-19 7:37 ` Richard Weinberger 2019-08-18 17:43 ` Theodore Y. Ts'o 2019-08-18 17:43 ` Theodore Y. Ts'o 2019-08-18 16:03 ` Gao Xiang 2019-08-18 16:03 ` Gao Xiang 2019-08-18 17:06 ` Richard Weinberger 2019-08-18 17:06 ` Richard Weinberger 2019-08-18 17:46 ` Theodore Y. Ts'o 2019-08-18 17:46 ` Theodore Y. Ts'o 2019-08-18 18:00 ` Richard Weinberger 2019-08-18 18:00 ` Richard Weinberger 2019-08-18 18:31 ` Gao Xiang 2019-08-18 18:31 ` Gao Xiang 2019-08-18 9:28 ` Gao Xiang 2019-08-18 9:28 ` Gao Xiang 2019-08-19 5:28 ` [PATCH] erofs: Use common kernel logging style Joe Perches 2019-08-19 5:28 ` Joe Perches 2019-08-19 5:52 ` Gao Xiang 2019-08-19 5:52 ` Gao Xiang 2019-08-19 5:47 ` Joe Perches 2019-08-19 5:47 ` Joe Perches 2019-08-19 6:08 ` Gao Xiang 2019-08-19 6:08 ` Gao Xiang
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190818201405.GA27398@hsiangkao-HP-ZHAN-66-Pro-G1 \ --to=hsiangkao@aol.com \ --cc=akpm@linux-foundation.org \ --cc=amir73il@gmail.com \ --cc=bluce.liguifu@huawei.com \ --cc=darrick.wong@oracle.com \ --cc=david@fromorbit.com \ --cc=devel@driverdev.osuosl.org \ --cc=dsterba@suse.cz \ --cc=ebiggers@kernel.org \ --cc=fangwei1@huawei.com \ --cc=gregkh@linuxfoundation.org \ --cc=hch@infradead.org \ --cc=jack@suse.cz \ --cc=jaegeuk@kernel.org \ --cc=linux-erofs@lists.ozlabs.org \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=miaoxie@huawei.com \ --cc=pavel@denx.de \ --cc=richard@nod.at \ --cc=sfr@canb.auug.org.au \ --cc=torvalds@linux-foundation.org \ --cc=tytso@mit.edu \ --cc=viro@zeniv.linux.org.uk \ --cc=yuchao0@huawei.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.