syzkaller reproducers

syzkaller reproducers

[Dmitry Vyukov]

Hi Shuah,

We discussed collecting and uploading all syzkaller reproducers
somewhere. You wanted to see how they look. I've uploaded all current
reproducers here:
https://github.com/dvyukov/syzkaller-repros
Minimalistic build/run scripts are included.
+some testing mailing lists too as this can be used as a test suite
If you have any potential uses for this, you are welcome to use it.
But then we probably need to find some more official and shared place
for them than my private github.
The test programs can also be bulk updated if necessary, because all
of this is auto-generated.

Thanks

Re: syzkaller reproducers

[Dmitry Vyukov]

On Tue, Oct 8, 2019 at 1:46 PM Dmitry Vyukov <dvyukov@google.com> wrote:
>
> Hi Shuah,
>
> We discussed collecting and uploading all syzkaller reproducers
> somewhere. You wanted to see how they look. I've uploaded all current
> reproducers here:
> https://github.com/dvyukov/syzkaller-repros
> Minimalistic build/run scripts are included.
> +some testing mailing lists too as this can be used as a test suite
> If you have any potential uses for this, you are welcome to use it.
> But then we probably need to find some more official and shared place
> for them than my private github.
> The test programs can also be bulk updated if necessary, because all
> of this is auto-generated.
>
> Thanks

+more people who expressed interest in the test suite before

Re: syzkaller reproducers

[shuah]

On 10/8/19 6:16 AM, Dmitry Vyukov wrote:
> On Tue, Oct 8, 2019 at 1:46 PM Dmitry Vyukov <dvyukov@google.com> wrote:
>>
>> Hi Shuah,
>>
>> We discussed collecting and uploading all syzkaller reproducers
>> somewhere. You wanted to see how they look. I've uploaded all current
>> reproducers here:
>> https://github.com/dvyukov/syzkaller-repros
>> Minimalistic build/run scripts are included.
>> +some testing mailing lists too as this can be used as a test suite
>> If you have any potential uses for this, you are welcome to use it.
>> But then we probably need to find some more official and shared place
>> for them than my private github.
>> The test programs can also be bulk updated if necessary, because all
>> of this is auto-generated.
>>
>> Thanks
> 
> +more people who expressed interest in the test suite before
> 

Thanks for putting this together. I am going to create a repo on
kernel.org t host these and pull your private git content in.

I will work on it this week and set things up.

thanks,
-- Shuah

Re: syzkaller reproducers

[shuah]

On 10/8/19 9:00 AM, shuah wrote:
> On 10/8/19 6:16 AM, Dmitry Vyukov wrote:
>> On Tue, Oct 8, 2019 at 1:46 PM Dmitry Vyukov <dvyukov@google.com> wrote:
>>>
>>> Hi Shuah,
>>>
>>> We discussed collecting and uploading all syzkaller reproducers
>>> somewhere. You wanted to see how they look. I've uploaded all current
>>> reproducers here:
>>> https://github.com/dvyukov/syzkaller-repros
>>> Minimalistic build/run scripts are included.
>>> +some testing mailing lists too as this can be used as a test suite
>>> If you have any potential uses for this, you are welcome to use it.
>>> But then we probably need to find some more official and shared place
>>> for them than my private github.
>>> The test programs can also be bulk updated if necessary, because all
>>> of this is auto-generated.
>>>
>>> Thanks
>>
>> +more people who expressed interest in the test suite before
>>
> 
> Thanks for putting this together. I am going to create a repo on
> kernel.org t host these and pull your private git content in.
> 
> I will work on it this week and set things up.
> 

Playing with the git getting ready to host it on kernel.org git repo.
Build worked fine and I can't get the run.sh to work.

I expected it to run what is in

syzkaller-repros/bin

It doesn't seem to do that. Looks like it wants to build. Here is what
I see. What am I doing wrong? I did a build which worked. There are some
errors due to sys/cdefs.h missing.

++ for f in *
+++ pwd
++ bin=/mnt/data/lkml/syzkaller-repros/bin
+++ mktemp -d
++ dir=/tmp/tmp.S8CjM8muRj
++ cd /tmp/tmp.S8CjM8muRj
++ timeout -s KILL 3 /mnt/data/lkml/syzkaller-repros/bin
timeout: failed to run command ‘/mnt/data/lkml/syzkaller-repros/bin’: 
Permission denied
++ rm -rf /tmp/tmp.S8CjM8muRj
++ for f in *
+++ pwd
++ bin=/mnt/data/lkml/syzkaller-repros/build.sh
+++ mktemp -d
++ dir=/tmp/tmp.zAVygWdpkt
++ cd /tmp/tmp.zAVygWdpkt
++ timeout -s KILL 3 /mnt/data/lkml/syzkaller-repros/build.sh
linux/*.c
grep: linux/*.c: No such file or directory
gcc: error: linux/*.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
++ rm -rf /tmp/tmp.zAVygWdpkt
++ for f in *
+++ pwd
++ bin=/mnt/data/lkml/syzkaller-repros/LICENSE
+++ mktemp -d
++ dir=/tmp/tmp.hhcvgOc8RA
++ cd /tmp/tmp.hhcvgOc8RA
++ timeout -s KILL 3 /mnt/data/lkml/syzkaller-repros/LICENSE
timeout: failed to run command 
‘/mnt/data/lkml/syzkaller-repros/LICENSE’: Permission denied
++ rm -rf /tmp/tmp.hhcvgOc8RA
++ for f in *
+++ pwd
++ bin=/mnt/data/lkml/syzkaller-repros/linux
+++ mktemp -d
++ dir=/tmp/tmp.sM5UO7BHRB
++ cd /tmp/tmp.sM5UO7BHRB
++ timeout -s KILL 3 /mnt/data/lkml/syzkaller-repros/linux
timeout: failed to run command ‘/mnt/data/lkml/syzkaller-repros/linux’: 
Permission denied
++ rm -rf /tmp/tmp.sM5UO7BHRB
++ for f in *
+++ pwd
++ bin=/mnt/data/lkml/syzkaller-repros/README.md
+++ mktemp -d
++ dir=/tmp/tmp.4Kkrs6iLP5
++ cd /tmp/tmp.4Kkrs6iLP5
++ timeout -s KILL 3 /mnt/data/lkml/syzkaller-repros/README.md
timeout: failed to run command 
‘/mnt/data/lkml/syzkaller-repros/README.md’: Permission denied
++ rm -rf /tmp/tmp.4Kkrs6iLP5
++ for f in *
+++ pwd
++ bin=/mnt/data/lkml/syzkaller-repros/run.sh
+++ mktemp -d
++ dir=/tmp/tmp.YFx5WEOvJn
++ cd /tmp/tmp.YFx5WEOvJn
++ timeout -s KILL 3 /mnt/data/lkml/syzkaller-repros/run.sh
+ pwd
+ bin=/tmp/tmp.YFx5WEOvJn/*
+ mktemp -d
+ dir=/tmp/tmp.qD4i4g9qYR
+ cd /tmp/tmp.qD4i4g9qYR
+ timeout -s KILL 3 /tmp/tmp.YFx5WEOvJn/*
timeout: failed to run command ‘/tmp/tmp.YFx5WEOvJn/*’: No such file or 
directory
+ rm -rf /tmp/tmp.qD4i4g9qYR
++ rm -rf /tmp/tmp.YFx5WEOvJn

thanks,
-- Shuah

Re: [Automated-testing] syzkaller reproducers

[Cyril Hrubis]

Hi!
> Playing with the git getting ready to host it on kernel.org git repo.
> Build worked fine and I can't get the run.sh to work.
> 
> I expected it to run what is in
> 
> syzkaller-repros/bin
> 
> It doesn't seem to do that. Looks like it wants to build. Here is what
> I see. What am I doing wrong?

You are suposed to run the run.sh script in the bin directory.

> I did a build which worked. There are some errors due to sys/cdefs.h
> missing.

That is because you are missing 32bit gcc and devel libs.

-- 
Cyril Hrubis
chrubis@suse.cz

Re: [Automated-testing] syzkaller reproducers

[shuah]

On 10/11/19 12:02 PM, Cyril Hrubis wrote:
> Hi!
>> Playing with the git getting ready to host it on kernel.org git repo.
>> Build worked fine and I can't get the run.sh to work.
>>
>> I expected it to run what is in
>>
>> syzkaller-repros/bin
>>
>> It doesn't seem to do that. Looks like it wants to build. Here is what
>> I see. What am I doing wrong?
> 
> You are suposed to run the run.sh script in the bin directory.

Yeah that does work.

Would be helpful to have usage instructions instead of failing. :)

> 
>> I did a build which worked. There are some errors due to sys/cdefs.h
>> missing.
> 
> That is because you are missing 32bit gcc and devel libs.
> 

Yeah. I didn't think about that.

thanks,
-- Shuah

Re: [Automated-testing] syzkaller reproducers

[Cyril Hrubis]

Hi!
> > You are suposed to run the run.sh script in the bin directory.
> 
> Yeah that does work.
> 
> Would be helpful to have usage instructions instead of failing. :)

I do not think that these scripts are ever supposed to be the used in
production testing, you need much more than this to produce results
reliably. I would expect that they are supposed to be a form of very
minimal documentation.

-- 
Cyril Hrubis
chrubis@suse.cz

Re: [Automated-testing] syzkaller reproducers

[Dmitry Vyukov]

On Mon, Oct 14, 2019 at 10:54 AM Cyril Hrubis <chrubis@suse.cz> wrote:
>
> Hi!
> > > You are suposed to run the run.sh script in the bin directory.
> >
> > Yeah that does work.
> >
> > Would be helpful to have usage instructions instead of failing. :)
>
> I do not think that these scripts are ever supposed to be the used in
> production testing, you need much more than this to produce results
> reliably. I would expect that they are supposed to be a form of very
> minimal documentation.

Yes, I just added them as quick hints: some repros are 32-bits; each
needs a new dir; some external timeout is needed for each test.

Re: [Automated-testing] syzkaller reproducers

[George Kennedy]

On 10/14/2019 7:19 AM, Dmitry Vyukov wrote:
> On Mon, Oct 14, 2019 at 10:54 AM Cyril Hrubis <chrubis@suse.cz> wrote:
>> Hi!
>>>> You are suposed to run the run.sh script in the bin directory.
>>> Yeah that does work.
>>>
>>> Would be helpful to have usage instructions instead of failing. :)
>> I do not think that these scripts are ever supposed to be the used in
>> production testing, you need much more than this to produce results
>> reliably. I would expect that they are supposed to be a form of very
>> minimal documentation.
> Yes, I just added them as quick hints: some repros are 32-bits; each
> needs a new dir; some external timeout is needed for each test.
Thank you again for the collection of repro C programs!

Hitting a lot more crashes with the collection of repro C programs than 
in all the hours of running Syzkaller. Wonder why? Any idea? This is 
with the same kernel and VM that Syzkaller is run on.

George

Re: [Automated-testing] syzkaller reproducers

[Cyril Hrubis]

Hi!
> >> I do not think that these scripts are ever supposed to be the used in
> >> production testing, you need much more than this to produce results
> >> reliably. I would expect that they are supposed to be a form of very
> >> minimal documentation.
> > Yes, I just added them as quick hints: some repros are 32-bits; each
> > needs a new dir; some external timeout is needed for each test.
> Thank you again for the collection of repro C programs!
> 
> Hitting a lot more crashes with the collection of repro C programs than 
> in all the hours of running Syzkaller. Wonder why? Any idea? This is 
> with the same kernel and VM that Syzkaller is run on.

I would guess that these reproducers are product of countless hours of
fuzzing, so it's about to be expected...

-- 
Cyril Hrubis
chrubis@suse.cz

Re: [Automated-testing] syzkaller reproducers

[Dmitry Vyukov]

On Tue, Oct 15, 2019 at 3:44 PM Cyril Hrubis <chrubis@suse.cz> wrote:
>
> Hi!
> > >> I do not think that these scripts are ever supposed to be the used in
> > >> production testing, you need much more than this to produce results
> > >> reliably. I would expect that they are supposed to be a form of very
> > >> minimal documentation.
> > > Yes, I just added them as quick hints: some repros are 32-bits; each
> > > needs a new dir; some external timeout is needed for each test.
> > Thank you again for the collection of repro C programs!
> >
> > Hitting a lot more crashes with the collection of repro C programs than
> > in all the hours of running Syzkaller. Wonder why? Any idea? This is
> > with the same kernel and VM that Syzkaller is run on.
>
> I would guess that these reproducers are product of countless hours of
> fuzzing, so it's about to be expected...


Probably. Hard to say.
If you used KCOV, KCOV_ENABLE_COMPARISONS, KASAN, LOCKDEP,
FAULT_INJECTION, all other debugging configs, compat instance and some
required image/cmdline features, then the only reason for difference
that I see is indeed longer fuzzing time.

Re: [Automated-testing] syzkaller reproducers

[George Kennedy]

Hello Dmitry,

Could we get another drop of the Syzkaller C reproducers?

Wonder if we could get the drop periodically (i.e. a drop/quarter or a 
drop to match a major linux release)?

Thank you,
George

Re: [Automated-testing] syzkaller reproducers

[George Kennedy]

Hi Dmitry,

Re-sending this request.

Also, how do you track the Upstream branches with Syzkaller? Do you have 
a version of Syzkaller for each (i.e. 4.14, 4.19, etc)?

Thank you,
George

On 12/6/2019 3:06 PM, George Kennedy wrote:
> Hello Dmitry,
>
> Could we get another drop of the Syzkaller C reproducers?
>
> Wonder if we could get the drop periodically (i.e. a drop/quarter or a 
> drop to match a major linux release)?
>
> Thank you,
> George

Re: [Automated-testing] syzkaller reproducers

[Dmitry Vyukov]

Hi George,

This was still starred in my inbox, but I never got to actually do
anything with it. Thanks for pinging me. I thought that the script to
extract the repros won't work for some reason and that I will need to
fix it first. But turns out it's still working as-is (I wanted to
submit some changes that would break it, but I never go to that as
well. Good! :)).

So here is a new drop in with 692 repros:
https://github.com/dvyukov/syzkaller-repros/commit/6a06992209c328a3115c89c020f45b844b103573
Enjoy!

Yes, we have separate managers for each version, the entries in the
Instances table correspond to syz-manager one-to-one:
https://syzkaller.appspot.com/upstream
https://syzkaller.appspot.com/linux-4.19
https://syzkaller.appspot.com/android-54



On Mon, Jan 27, 2020 at 3:20 PM George Kennedy
<george.kennedy@oracle.com> wrote:
>
> Hi Dmitry,
>
> Re-sending this request.
>
> Also, how do you track the Upstream branches with Syzkaller? Do you have
> a version of Syzkaller for each (i.e. 4.14, 4.19, etc)?
>
> Thank you,
> George
>
> On 12/6/2019 3:06 PM, George Kennedy wrote:
> > Hello Dmitry,
> >
> > Could we get another drop of the Syzkaller C reproducers?
> >
> > Wonder if we could get the drop periodically (i.e. a drop/quarter or a
> > drop to match a major linux release)?
> >
> > Thank you,
> > George
>

Re: [Automated-testing] syzkaller reproducers

[George Kennedy]

On 1/27/2020 10:26 AM, Dmitry Vyukov wrote:
> Hi George,
>
> This was still starred in my inbox, but I never got to actually do
> anything with it. Thanks for pinging me. I thought that the script to
> extract the repros won't work for some reason and that I will need to
> fix it first. But turns out it's still working as-is (I wanted to
> submit some changes that would break it, but I never go to that as
> well. Good! :)).
>
> So here is a new drop in with 692 repros:
> https://github.com/dvyukov/syzkaller-repros/commit/6a06992209c328a3115c89c020f45b844b103573
> Enjoy!
>
> Yes, we have separate managers for each version, the entries in the
> Instances table correspond to syz-manager one-to-one:
> https://syzkaller.appspot.com/upstream
> https://syzkaller.appspot.com/linux-4.19
> https://syzkaller.appspot.com/android-54

Thank you Dmitry!
George
>
>
>
> On Mon, Jan 27, 2020 at 3:20 PM George Kennedy
> <george.kennedy@oracle.com> wrote:
>> Hi Dmitry,
>>
>> Re-sending this request.
>>
>> Also, how do you track the Upstream branches with Syzkaller? Do you have
>> a version of Syzkaller for each (i.e. 4.14, 4.19, etc)?
>>
>> Thank you,
>> George
>>
>> On 12/6/2019 3:06 PM, George Kennedy wrote:
>>> Hello Dmitry,
>>>
>>> Could we get another drop of the Syzkaller C reproducers?
>>>
>>> Wonder if we could get the drop periodically (i.e. a drop/quarter or a
>>> drop to match a major linux release)?
>>>
>>> Thank you,
>>> George

Re: [Automated-testing] syzkaller reproducers

[Dmitry Vyukov]

Shauh,

We've added more reproducers to:
https://github.com/dvyukov/syzkaller-repros/tree/master/linux

It makes sense to pull in them to the kernel-arts repo. Not sure
what's the most convenient way for you since it's not exactly a
traditional "patch"? Perhaps you just copy linux/*.c files and commit?

George, another throw in of 446 repros ;)



On Mon, Jan 27, 2020 at 6:08 PM George Kennedy
<george.kennedy@oracle.com> wrote:
> > Hi George,
> >
> > This was still starred in my inbox, but I never got to actually do
> > anything with it. Thanks for pinging me. I thought that the script to
> > extract the repros won't work for some reason and that I will need to
> > fix it first. But turns out it's still working as-is (I wanted to
> > submit some changes that would break it, but I never go to that as
> > well. Good! :)).
> >
> > So here is a new drop in with 692 repros:
> > https://github.com/dvyukov/syzkaller-repros/commit/6a06992209c328a3115c89c020f45b844b103573
> > Enjoy!
> >
> > Yes, we have separate managers for each version, the entries in the
> > Instances table correspond to syz-manager one-to-one:
> > https://syzkaller.appspot.com/upstream
> > https://syzkaller.appspot.com/linux-4.19
> > https://syzkaller.appspot.com/android-54
>
> Thank you Dmitry!
> George
> >
> >
> >
> > On Mon, Jan 27, 2020 at 3:20 PM George Kennedy
> > <george.kennedy@oracle.com> wrote:
> >> Hi Dmitry,
> >>
> >> Re-sending this request.
> >>
> >> Also, how do you track the Upstream branches with Syzkaller? Do you have
> >> a version of Syzkaller for each (i.e. 4.14, 4.19, etc)?
> >>
> >> Thank you,
> >> George
> >>
> >> On 12/6/2019 3:06 PM, George Kennedy wrote:
> >>> Hello Dmitry,
> >>>
> >>> Could we get another drop of the Syzkaller C reproducers?
> >>>
> >>> Wonder if we could get the drop periodically (i.e. a drop/quarter or a
> >>> drop to match a major linux release)?
> >>>
> >>> Thank you,
> >>> George
>

Re: [Automated-testing] syzkaller reproducers

[George Kennedy]

On 3/3/2020 3:45 AM, Dmitry Vyukov wrote:
> Shauh,
>
> We've added more reproducers to:
> https://github.com/dvyukov/syzkaller-repros/tree/master/linux
>
> It makes sense to pull in them to the kernel-arts repo. Not sure
> what's the most convenient way for you since it's not exactly a
> traditional "patch"? Perhaps you just copy linux/*.c files and commit?
>
> George, another throw in of 446 repros ;)

Thank you Dmitry!
George

>
>
>
> On Mon, Jan 27, 2020 at 6:08 PM George Kennedy
> <george.kennedy@oracle.com> wrote:
>>> Hi George,
>>>
>>> This was still starred in my inbox, but I never got to actually do
>>> anything with it. Thanks for pinging me. I thought that the script to
>>> extract the repros won't work for some reason and that I will need to
>>> fix it first. But turns out it's still working as-is (I wanted to
>>> submit some changes that would break it, but I never go to that as
>>> well. Good! :)).
>>>
>>> So here is a new drop in with 692 repros:
>>> https://github.com/dvyukov/syzkaller-repros/commit/6a06992209c328a3115c89c020f45b844b103573
>>> Enjoy!
>>>
>>> Yes, we have separate managers for each version, the entries in the
>>> Instances table correspond to syz-manager one-to-one:
>>> https://syzkaller.appspot.com/upstream
>>> https://syzkaller.appspot.com/linux-4.19
>>> https://syzkaller.appspot.com/android-54
>> Thank you Dmitry!
>> George
>>>
>>>
>>> On Mon, Jan 27, 2020 at 3:20 PM George Kennedy
>>> <george.kennedy@oracle.com> wrote:
>>>> Hi Dmitry,
>>>>
>>>> Re-sending this request.
>>>>
>>>> Also, how do you track the Upstream branches with Syzkaller? Do you have
>>>> a version of Syzkaller for each (i.e. 4.14, 4.19, etc)?
>>>>
>>>> Thank you,
>>>> George
>>>>
>>>> On 12/6/2019 3:06 PM, George Kennedy wrote:
>>>>> Hello Dmitry,
>>>>>
>>>>> Could we get another drop of the Syzkaller C reproducers?
>>>>>
>>>>> Wonder if we could get the drop periodically (i.e. a drop/quarter or a
>>>>> drop to match a major linux release)?
>>>>>
>>>>> Thank you,
>>>>> George

Re: [Automated-testing] syzkaller reproducers

[shuah]

On 3/3/20 1:45 AM, Dmitry Vyukov wrote:
> Shauh,
> 
> We've added more reproducers to:
> https://github.com/dvyukov/syzkaller-repros/tree/master/linux
> 
> It makes sense to pull in them to the kernel-arts repo. Not sure
> what's the most convenient way for you since it's not exactly a
> traditional "patch"? Perhaps you just copy linux/*.c files and commit?
> 

Thanks Dmitry. I will pull these in. I will pull these in.

thanks,
-- Shuah

Re: syzkaller reproducers

[George Kennedy]

On 10/8/2019 8:16 AM, Dmitry Vyukov wrote:
> On Tue, Oct 8, 2019 at 1:46 PM Dmitry Vyukov <dvyukov@google.com> wrote:
>> Hi Shuah,
>>
>> We discussed collecting and uploading all syzkaller reproducers
>> somewhere. You wanted to see how they look. I've uploaded all current
>> reproducers here:
>> https://github.com/dvyukov/syzkaller-repros
>> Minimalistic build/run scripts are included.
>> +some testing mailing lists too as this can be used as a test suite
>> If you have any potential uses for this, you are welcome to use it.
>> But then we probably need to find some more official and shared place
>> for them than my private github.
>> The test programs can also be bulk updated if necessary, because all
>> of this is auto-generated.

Thank you for putting this together Dmitry,

George

>>
>> Thanks
> +more people who expressed interest in the test suite before

Re: [Automated-testing] syzkaller reproducers

[Richard Palethorpe]

Hello,

Dmitry Vyukov <dvyukov@google.com> writes:

> Hi Shuah,
>
> We discussed collecting and uploading all syzkaller reproducers
> somewhere. You wanted to see how they look. I've uploaded all current
> reproducers here:
> https://github.com/dvyukov/syzkaller-repros
> Minimalistic build/run scripts are included.
> +some testing mailing lists too as this can be used as a test suite
> If you have any potential uses for this, you are welcome to use it.
> But then we probably need to find some more official and shared place
> for them than my private github.
> The test programs can also be bulk updated if necessary, because all
> of this is auto-generated.
>
> Thanks

I discussed this a bit with Metan. We think it would be fairly trivial
to create an LTP wrapper for these.

So we just create an LTP test which forks and execs one of these
reproducers then checks for kernel taints after the child completes or
is killed. It can take the reproducer path as an argument and we can
generate a runtest file with all the reproducers in that we are able to
compile.

I imagine a lot of these reproducers will not work on older kernels, so
this would just be a best efforts basis. We would ignore any problems
during execution unless there is a kernel error.

This should work with existing LTP runners with maybe a minor change or
two to building and configuration.

I will start experimenting with this and post the results to the LTP
mailing list.

--
Thank you,
Richard.

Re: [Automated-testing] syzkaller reproducers

[Dmitry Vyukov]

On Tue, Oct 8, 2019 at 2:37 PM Richard Palethorpe <rpalethorpe@suse.de> wrote:
>
> Hello,
>
> Dmitry Vyukov <dvyukov@google.com> writes:
>
> > Hi Shuah,
> >
> > We discussed collecting and uploading all syzkaller reproducers
> > somewhere. You wanted to see how they look. I've uploaded all current
> > reproducers here:
> > https://github.com/dvyukov/syzkaller-repros
> > Minimalistic build/run scripts are included.
> > +some testing mailing lists too as this can be used as a test suite
> > If you have any potential uses for this, you are welcome to use it.
> > But then we probably need to find some more official and shared place
> > for them than my private github.
> > The test programs can also be bulk updated if necessary, because all
> > of this is auto-generated.
> >
> > Thanks
>
> I discussed this a bit with Metan. We think it would be fairly trivial
> to create an LTP wrapper for these.
>
> So we just create an LTP test which forks and execs one of these
> reproducers then checks for kernel taints after the child completes or
> is killed. It can take the reproducer path as an argument and we can
> generate a runtest file with all the reproducers in that we are able to
> compile.
>
> I imagine a lot of these reproducers will not work on older kernels, so
> this would just be a best efforts basis. We would ignore any problems
> during execution unless there is a kernel error.
>
> This should work with existing LTP runners with maybe a minor change or
> two to building and configuration.
>
> I will start experimenting with this and post the results to the LTP
> mailing list.


Hi Richard,

Sounds great!

Yes, these are totally best effort. May also require some configs, etc.
Also note that each should be run in a clean temp dir and with a
timeout b/c some have an infinite loop.

Re: [Automated-testing] syzkaller reproducers

[Richard Palethorpe]

Hello,

Dmitry Vyukov <dvyukov@google.com> writes:

> On Tue, Oct 8, 2019 at 2:37 PM Richard Palethorpe <rpalethorpe@suse.de> wrote:
>>
>> Hello,
>>
>> Dmitry Vyukov <dvyukov@google.com> writes:
>>
>> > Hi Shuah,
>> >
>> > We discussed collecting and uploading all syzkaller reproducers
>> > somewhere. You wanted to see how they look. I've uploaded all current
>> > reproducers here:
>> > https://github.com/dvyukov/syzkaller-repros
>> > Minimalistic build/run scripts are included.
>> > +some testing mailing lists too as this can be used as a test suite
>> > If you have any potential uses for this, you are welcome to use it.
>> > But then we probably need to find some more official and shared place
>> > for them than my private github.
>> > The test programs can also be bulk updated if necessary, because all
>> > of this is auto-generated.
>> >
>> > Thanks
>>
>> I discussed this a bit with Metan. We think it would be fairly trivial
>> to create an LTP wrapper for these.
>>
>> So we just create an LTP test which forks and execs one of these
>> reproducers then checks for kernel taints after the child completes or
>> is killed. It can take the reproducer path as an argument and we can
>> generate a runtest file with all the reproducers in that we are able to
>> compile.
>>
>> I imagine a lot of these reproducers will not work on older kernels, so
>> this would just be a best efforts basis. We would ignore any problems
>> during execution unless there is a kernel error.
>>
>> This should work with existing LTP runners with maybe a minor change or
>> two to building and configuration.
>>
>> I will start experimenting with this and post the results to the LTP
>> mailing list.
>
>
> Hi Richard,
>
> Sounds great!
>
> Yes, these are totally best effort. May also require some configs, etc.
> Also note that each should be run in a clean temp dir and with a
> timeout b/c some have an infinite loop.

Thanks, fortunately the LTP lib can do that automatically.

However the default timeout is 5 minutes, but I guess this should be
more like ~3 seconds as in the run script?

--
Thank you,
Richard.

Re: [Automated-testing] syzkaller reproducers

[Dmitry Vyukov]

On Tue, Oct 8, 2019 at 3:14 PM Richard Palethorpe <rpalethorpe@suse.de> wrote:
>
> Hello,
>
> Dmitry Vyukov <dvyukov@google.com> writes:
>
> > On Tue, Oct 8, 2019 at 2:37 PM Richard Palethorpe <rpalethorpe@suse.de> wrote:
> >>
> >> Hello,
> >>
> >> Dmitry Vyukov <dvyukov@google.com> writes:
> >>
> >> > Hi Shuah,
> >> >
> >> > We discussed collecting and uploading all syzkaller reproducers
> >> > somewhere. You wanted to see how they look. I've uploaded all current
> >> > reproducers here:
> >> > https://github.com/dvyukov/syzkaller-repros
> >> > Minimalistic build/run scripts are included.
> >> > +some testing mailing lists too as this can be used as a test suite
> >> > If you have any potential uses for this, you are welcome to use it.
> >> > But then we probably need to find some more official and shared place
> >> > for them than my private github.
> >> > The test programs can also be bulk updated if necessary, because all
> >> > of this is auto-generated.
> >> >
> >> > Thanks
> >>
> >> I discussed this a bit with Metan. We think it would be fairly trivial
> >> to create an LTP wrapper for these.
> >>
> >> So we just create an LTP test which forks and execs one of these
> >> reproducers then checks for kernel taints after the child completes or
> >> is killed. It can take the reproducer path as an argument and we can
> >> generate a runtest file with all the reproducers in that we are able to
> >> compile.
> >>
> >> I imagine a lot of these reproducers will not work on older kernels, so
> >> this would just be a best efforts basis. We would ignore any problems
> >> during execution unless there is a kernel error.
> >>
> >> This should work with existing LTP runners with maybe a minor change or
> >> two to building and configuration.
> >>
> >> I will start experimenting with this and post the results to the LTP
> >> mailing list.
> >
> >
> > Hi Richard,
> >
> > Sounds great!
> >
> > Yes, these are totally best effort. May also require some configs, etc.
> > Also note that each should be run in a clean temp dir and with a
> > timeout b/c some have an infinite loop.
>
> Thanks, fortunately the LTP lib can do that automatically.
>
> However the default timeout is 5 minutes, but I guess this should be
> more like ~3 seconds as in the run script?

This really depends on the bug type and specific bug.
syzkaller runs reproducers for up to 7 minutes. Sometimes even that is
not enough to reproduce some bugs. We also detect hung tasks only
after 3-4 minutes (may produce flakes with smaller timeout).
Most reproducers can also run in parallel, that may help to partially
neutralize large timeouts.
But also if these run repeatedly/continuously, we can use smaller
timeout on each run (don't need to catch crash on every run).
Let's start with somewhere, we can tune later as we gather experience.

Re: [Automated-testing] syzkaller reproducers

[Cyril Hrubis]

Hi!
> We discussed collecting and uploading all syzkaller reproducers
> somewhere. You wanted to see how they look. I've uploaded all current
> reproducers here:
> https://github.com/dvyukov/syzkaller-repros
> Minimalistic build/run scripts are included.

What is the reason behind statically linking the reproducers?

The difference between static and dynamic binary is a bit less than 4MB,
which gives difference of several gigabytes for the whole repo. This
amount of binary data would complicate and slow down any CI
significantly...

-- 
Cyril Hrubis
chrubis@suse.cz

Re: [Automated-testing] syzkaller reproducers

[Dmitry Vyukov]

On Thu, Oct 10, 2019 at 4:27 PM Cyril Hrubis <chrubis@suse.cz> wrote:
>
> Hi!
> > We discussed collecting and uploading all syzkaller reproducers
> > somewhere. You wanted to see how they look. I've uploaded all current
> > reproducers here:
> > https://github.com/dvyukov/syzkaller-repros
> > Minimalistic build/run scripts are included.
>
> What is the reason behind statically linking the reproducers?
>
> The difference between static and dynamic binary is a bit less than 4MB,
> which gives difference of several gigabytes for the whole repo. This
> amount of binary data would complicate and slow down any CI
> significantly...

Simply being able to run programs on a different machine. Dynamic
binaries generally don't run on a different linux machine. If you can
build dynamic binaries so that they run on test machine, feel free to
drop -static.