Re: syzbot for bcachefs?

Re: syzbot for bcachefs?

[Kent Overstreet]

On Thu, Apr 04, 2024 at 10:33:07AM +0200, Dmitry Vyukov wrote:
> On Thu, 4 Apr 2024 at 01:59, Kent Overstreet <kent.overstreet@linux.dev> wrote:
> >
> > Hi - what would it take to get syzbot going for bcachefs?
> 
> Hi Kent,
> 
> There are 2 parts:
> 
> The easy one: we need to enable it in syzbot configs. Should we
> configure it as follows?
> 
> CONFIG_BCACHEFS_FS=y
> CONFIG_BCACHEFS_QUOTA=y
> CONFIG_BCACHEFS_ERASURE_CODING=y
> CONFIG_BCACHEFS_POSIX_ACL=y
> CONFIG_BCACHEFS_DEBUG=y
> # CONFIG_BCACHEFS_TESTS is not set
> # CONFIG_BCACHEFS_LOCK_TIME_STATS is not set
> # CONFIG_BCACHEFS_NO_LATENCY_ACCT is not set
> CONFIG_BCACHEFS_SIX_OPTIMISTIC_SPIN=y
> 
> In particular, I am interested in CONFIG_BCACHEFS_DEBUG. Does it
> really add more debug checks rather than lots of console printing w/o
> checks? Some "DEBUG" kernel configs are misleading here. Checks will
> be useful on syzbot, lots of printing not so much.

CONFIG_BCACHEFS_DEBUG adds piles of assertions and no extra console
printing :)

> Second part: syzbot will need to exercise the fs in some useful way.
> Is it just a "normal" fs with files, or something special?

It's a normal - but multi device - filesystem, similar featureset to
btrfs.

> Is it mounted from an image, or an image-less fs? If it needs images,
> how can they be created?

apt install bcachefs-tools
mkfs.bcachefs <dev>
mount <dev> /mnt

...

> "find tools/testing/ Documentation/ -name "*bcachefs*"" does not give
> me anything useful.

https://bcachefs.org/ has some docs, and the tools have a man page

Anything else you'd be interested in? Thomas was interested in working
on on-disk-image error injection, is that something that would be
useful?

Ariel's also probably going to be working on adding full image creation
to mkfs, i.e. pass it a directory - is that something that would be
useful as well?

Re: syzbot for bcachefs?

[Aleksandr Nogikh]

On Fri, Apr 5, 2024 at 1:16 AM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> On Thu, Apr 04, 2024 at 10:33:07AM +0200, Dmitry Vyukov wrote:
> > On Thu, 4 Apr 2024 at 01:59, Kent Overstreet <kent.overstreet@linux.dev> wrote:
> > >
> > > Hi - what would it take to get syzbot going for bcachefs?
> >
> > Hi Kent,
> >
> > There are 2 parts:
> >
> > The easy one: we need to enable it in syzbot configs. Should we
> > configure it as follows?
> >
> > CONFIG_BCACHEFS_FS=y
> > CONFIG_BCACHEFS_QUOTA=y
> > CONFIG_BCACHEFS_ERASURE_CODING=y
> > CONFIG_BCACHEFS_POSIX_ACL=y
> > CONFIG_BCACHEFS_DEBUG=y
> > # CONFIG_BCACHEFS_TESTS is not set
> > # CONFIG_BCACHEFS_LOCK_TIME_STATS is not set
> > # CONFIG_BCACHEFS_NO_LATENCY_ACCT is not set
> > CONFIG_BCACHEFS_SIX_OPTIMISTIC_SPIN=y
> >
> > In particular, I am interested in CONFIG_BCACHEFS_DEBUG. Does it
> > really add more debug checks rather than lots of console printing w/o
> > checks? Some "DEBUG" kernel configs are misleading here. Checks will
> > be useful on syzbot, lots of printing not so much.

For the record: I've just sent
https://github.com/google/syzkaller/pull/4647 with syzbot config
updates.

>
> CONFIG_BCACHEFS_DEBUG adds piles of assertions and no extra console
> printing :)
>
> > Second part: syzbot will need to exercise the fs in some useful way.
> > Is it just a "normal" fs with files, or something special?
>
> It's a normal - but multi device - filesystem, similar featureset to
> btrfs.
>
> > Is it mounted from an image, or an image-less fs? If it needs images,
> > how can they be created?
>
> apt install bcachefs-tools
> mkfs.bcachefs <dev>
> mount <dev> /mnt
>
> ...
>
> > "find tools/testing/ Documentation/ -name "*bcachefs*"" does not give
> > me anything useful.
>
> https://bcachefs.org/ has some docs, and the tools have a man page
>
> Anything else you'd be interested in? Thomas was interested in working
> on on-disk-image error injection, is that something that would be
> useful?
>
> Ariel's also probably going to be working on adding full image creation
> to mkfs, i.e. pass it a directory - is that something that would be
> useful as well?
>

Re: syzbot for bcachefs?

[Dmitry Vyukov]

On Fri, 5 Apr 2024 at 18:20, Aleksandr Nogikh <nogikh@google.com> wrote:
>
> On Fri, Apr 5, 2024 at 1:16 AM Kent Overstreet
> <kent.overstreet@linux.dev> wrote:
> >
> > On Thu, Apr 04, 2024 at 10:33:07AM +0200, Dmitry Vyukov wrote:
> > > On Thu, 4 Apr 2024 at 01:59, Kent Overstreet <kent.overstreet@linux.dev> wrote:
> > > >
> > > > Hi - what would it take to get syzbot going for bcachefs?
> > >
> > > Hi Kent,
> > >
> > > There are 2 parts:
> > >
> > > The easy one: we need to enable it in syzbot configs. Should we
> > > configure it as follows?
> > >
> > > CONFIG_BCACHEFS_FS=y
> > > CONFIG_BCACHEFS_QUOTA=y
> > > CONFIG_BCACHEFS_ERASURE_CODING=y
> > > CONFIG_BCACHEFS_POSIX_ACL=y
> > > CONFIG_BCACHEFS_DEBUG=y
> > > # CONFIG_BCACHEFS_TESTS is not set
> > > # CONFIG_BCACHEFS_LOCK_TIME_STATS is not set
> > > # CONFIG_BCACHEFS_NO_LATENCY_ACCT is not set
> > > CONFIG_BCACHEFS_SIX_OPTIMISTIC_SPIN=y
> > >
> > > In particular, I am interested in CONFIG_BCACHEFS_DEBUG. Does it
> > > really add more debug checks rather than lots of console printing w/o
> > > checks? Some "DEBUG" kernel configs are misleading here. Checks will
> > > be useful on syzbot, lots of printing not so much.
>
> For the record: I've just sent
> https://github.com/google/syzkaller/pull/4647 with syzbot config
> updates.
>
> >
> > CONFIG_BCACHEFS_DEBUG adds piles of assertions and no extra console
> > printing :)
> >
> > > Second part: syzbot will need to exercise the fs in some useful way.
> > > Is it just a "normal" fs with files, or something special?
> >
> > It's a normal - but multi device - filesystem, similar featureset to
> > btrfs.
> >
> > > Is it mounted from an image, or an image-less fs? If it needs images,
> > > how can they be created?
> >
> > apt install bcachefs-tools
> > mkfs.bcachefs <dev>
> > mount <dev> /mnt

We've enabled the configs, but we don't have good image seeds yet.
Aleksandr tried to generate them, he is currently OOO, but maybe you
can help meanwhile.
He said that running:

$ mkfs.bcachefs test.img

gives this error (on a quite big image, I don't know how large exactly it was):

insufficient space for superblocks: start 8 end 4104 > 4096 size 2048

Why could it give such an error and how to avoid it?

Also what would be a good sets of options to generate seed images?
We try to "sample" space of different images to create seeds. To give
you an idea here is what we do for other file systems:
https://github.com/google/syzkaller/blob/af24b0505c748561efb50f1d03c824d6642f6c0b/tools/syz-imagegen/imagegen.go#L60-L516

Re: syzbot for bcachefs?

[Kent Overstreet]

On Fri, Apr 19, 2024 at 10:35:27AM +0200, Dmitry Vyukov wrote:
> On Fri, 5 Apr 2024 at 18:20, Aleksandr Nogikh <nogikh@google.com> wrote:
> >
> > On Fri, Apr 5, 2024 at 1:16 AM Kent Overstreet
> > <kent.overstreet@linux.dev> wrote:
> > >
> > > On Thu, Apr 04, 2024 at 10:33:07AM +0200, Dmitry Vyukov wrote:
> > > > On Thu, 4 Apr 2024 at 01:59, Kent Overstreet <kent.overstreet@linux.dev> wrote:
> > > > >
> > > > > Hi - what would it take to get syzbot going for bcachefs?
> > > >
> > > > Hi Kent,
> > > >
> > > > There are 2 parts:
> > > >
> > > > The easy one: we need to enable it in syzbot configs. Should we
> > > > configure it as follows?
> > > >
> > > > CONFIG_BCACHEFS_FS=y
> > > > CONFIG_BCACHEFS_QUOTA=y
> > > > CONFIG_BCACHEFS_ERASURE_CODING=y
> > > > CONFIG_BCACHEFS_POSIX_ACL=y
> > > > CONFIG_BCACHEFS_DEBUG=y
> > > > # CONFIG_BCACHEFS_TESTS is not set
> > > > # CONFIG_BCACHEFS_LOCK_TIME_STATS is not set
> > > > # CONFIG_BCACHEFS_NO_LATENCY_ACCT is not set
> > > > CONFIG_BCACHEFS_SIX_OPTIMISTIC_SPIN=y
> > > >
> > > > In particular, I am interested in CONFIG_BCACHEFS_DEBUG. Does it
> > > > really add more debug checks rather than lots of console printing w/o
> > > > checks? Some "DEBUG" kernel configs are misleading here. Checks will
> > > > be useful on syzbot, lots of printing not so much.
> >
> > For the record: I've just sent
> > https://github.com/google/syzkaller/pull/4647 with syzbot config
> > updates.
> >
> > >
> > > CONFIG_BCACHEFS_DEBUG adds piles of assertions and no extra console
> > > printing :)
> > >
> > > > Second part: syzbot will need to exercise the fs in some useful way.
> > > > Is it just a "normal" fs with files, or something special?
> > >
> > > It's a normal - but multi device - filesystem, similar featureset to
> > > btrfs.
> > >
> > > > Is it mounted from an image, or an image-less fs? If it needs images,
> > > > how can they be created?
> > >
> > > apt install bcachefs-tools
> > > mkfs.bcachefs <dev>
> > > mount <dev> /mnt
> 
> We've enabled the configs, but we don't have good image seeds yet.
> Aleksandr tried to generate them, he is currently OOO, but maybe you
> can help meanwhile.
> He said that running:
> 
> $ mkfs.bcachefs test.img
> 
> gives this error (on a quite big image, I don't know how large exactly it was):
> 
> insufficient space for superblocks: start 8 end 4104 > 4096 size 2048
> 
> Why could it give such an error and how to avoid it?

What was the precise size? I'll see if I'm able to reproduce

> Also what would be a good sets of options to generate seed images?
> We try to "sample" space of different images to create seeds. To give
> you an idea here is what we do for other file systems:
> https://github.com/google/syzkaller/blob/af24b0505c748561efb50f1d03c824d6642f6c0b/tools/syz-imagegen/imagegen.go#L60-L516

--encrypted --no_passphrase
--compression=lz4
--no-checksum

That shousd get us reasonable coverage to start with.

Re: syzbot for bcachefs?

[Aleksandr Nogikh]

On Fri, Apr 19, 2024 at 7:19 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> On Fri, Apr 19, 2024 at 10:35:27AM +0200, Dmitry Vyukov wrote:
> >
> > We've enabled the configs, but we don't have good image seeds yet.
> > Aleksandr tried to generate them, he is currently OOO, but maybe you
> > can help meanwhile.
> > He said that running:
> >
> > $ mkfs.bcachefs test.img
> >
> > gives this error (on a quite big image, I don't know how large exactly it was):
> >
> > insufficient space for superblocks: start 8 end 4104 > 4096 size 2048
> >
> > Why could it give such an error and how to avoid it?
>
> What was the precise size? I'll see if I'm able to reproduce

I think the problem was that I was trying to save the generated files
to a 9p filesystem (so that I could easily take them out of my VM).
But now I've set up a proper userspace image with the latest 6.9 RC
kernel and it worked fine.

>
> > Also what would be a good sets of options to generate seed images?
> > We try to "sample" space of different images to create seeds. To give
> > you an idea here is what we do for other file systems:
> > https://github.com/google/syzkaller/blob/af24b0505c748561efb50f1d03c824d6642f6c0b/tools/syz-imagegen/imagegen.go#L60-L516
>
> --encrypted --no_passphrase
> --compression=lz4
> --no-checksum

The mkfs.bcachefs tool that I installed does not know the
--no-checksum flag, so I used --data_checksum=none
--metadata_checksum=none instead. Does it mean the same thing?

I've pushed my changes to https://github.com/google/syzkaller/pull/4752

-- 
Aleksandr

>
> That shousd get us reasonable coverage to start with.

Re: syzbot for bcachefs?

[Kent Overstreet]

On Thu, May 02, 2024 at 10:28:42PM +0200, Aleksandr Nogikh wrote:
> On Fri, Apr 19, 2024 at 7:19 PM Kent Overstreet
> <kent.overstreet@linux.dev> wrote:
> >
> > On Fri, Apr 19, 2024 at 10:35:27AM +0200, Dmitry Vyukov wrote:
> > >
> > > We've enabled the configs, but we don't have good image seeds yet.
> > > Aleksandr tried to generate them, he is currently OOO, but maybe you
> > > can help meanwhile.
> > > He said that running:
> > >
> > > $ mkfs.bcachefs test.img
> > >
> > > gives this error (on a quite big image, I don't know how large exactly it was):
> > >
> > > insufficient space for superblocks: start 8 end 4104 > 4096 size 2048
> > >
> > > Why could it give such an error and how to avoid it?
> >
> > What was the precise size? I'll see if I'm able to reproduce
> 
> I think the problem was that I was trying to save the generated files
> to a 9p filesystem (so that I could easily take them out of my VM).
> But now I've set up a proper userspace image with the latest 6.9 RC
> kernel and it worked fine.
> 
> >
> > > Also what would be a good sets of options to generate seed images?
> > > We try to "sample" space of different images to create seeds. To give
> > > you an idea here is what we do for other file systems:
> > > https://github.com/google/syzkaller/blob/af24b0505c748561efb50f1d03c824d6642f6c0b/tools/syz-imagegen/imagegen.go#L60-L516
> >
> > --encrypted --no_passphrase
> > --compression=lz4
> > --no-checksum
> 
> The mkfs.bcachefs tool that I installed does not know the
> --no-checksum flag, so I used --data_checksum=none
> --metadata_checksum=none instead. Does it mean the same thing?

That's the correct flags, I did a brainfart on that one :)

Re: syzbot for bcachefs?

[Aleksandr Nogikh]

On Fri, May 3, 2024 at 5:36 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> On Thu, May 02, 2024 at 10:28:42PM +0200, Aleksandr Nogikh wrote:
> >
> > I think the problem was that I was trying to save the generated files
> > to a 9p filesystem (so that I could easily take them out of my VM).
> > But now I've set up a proper userspace image with the latest 6.9 RC
> > kernel and it worked fine.
> >
> > >
> > > > Also what would be a good sets of options to generate seed images?
> > > > We try to "sample" space of different images to create seeds. To give
> > > > you an idea here is what we do for other file systems:
> > > > https://github.com/google/syzkaller/blob/af24b0505c748561efb50f1d03c824d6642f6c0b/tools/syz-imagegen/imagegen.go#L60-L516
> > >
> > > --encrypted --no_passphrase
> > > --compression=lz4
> > > --no-checksum
> >
> > The mkfs.bcachefs tool that I installed does not know the
> > --no-checksum flag, so I used --data_checksum=none
> > --metadata_checksum=none instead. Does it mean the same thing?
>
> That's the correct flags, I did a brainfart on that one :)

Cool :)
I've deployed the changes to syzbot today and it has already reported
8 findings:
https://syzkaller.appspot.com/upstream/s/bcachefs

-- 
Aleksandr

Re: syzbot for bcachefs?

[Kent Overstreet]

On Fri, May 03, 2024 at 06:31:30PM +0200, Aleksandr Nogikh wrote:
> On Fri, May 3, 2024 at 5:36 PM Kent Overstreet
> <kent.overstreet@linux.dev> wrote:
> >
> > On Thu, May 02, 2024 at 10:28:42PM +0200, Aleksandr Nogikh wrote:
> > >
> > > I think the problem was that I was trying to save the generated files
> > > to a 9p filesystem (so that I could easily take them out of my VM).
> > > But now I've set up a proper userspace image with the latest 6.9 RC
> > > kernel and it worked fine.
> > >
> > > >
> > > > > Also what would be a good sets of options to generate seed images?
> > > > > We try to "sample" space of different images to create seeds. To give
> > > > > you an idea here is what we do for other file systems:
> > > > > https://github.com/google/syzkaller/blob/af24b0505c748561efb50f1d03c824d6642f6c0b/tools/syz-imagegen/imagegen.go#L60-L516
> > > >
> > > > --encrypted --no_passphrase
> > > > --compression=lz4
> > > > --no-checksum
> > >
> > > The mkfs.bcachefs tool that I installed does not know the
> > > --no-checksum flag, so I used --data_checksum=none
> > > --metadata_checksum=none instead. Does it mean the same thing?
> >
> > That's the correct flags, I did a brainfart on that one :)
> 
> Cool :)
> I've deployed the changes to syzbot today and it has already reported
> 8 findings:
> https://syzkaller.appspot.com/upstream/s/bcachefs

Nice, already got fixes pushed for most of them :)