From: Petr Mladek <pmladek@suse.com>
To: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Cc: Shuah Khan <shuah@kernel.org>, Kees Cook <keescook@chromium.org>,
Willy Tarreau <w@1wt.eu>,
linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org,
Arpitha Raghunandan <98.arpi@gmail.com>,
Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
Brendan Higgins <brendanhiggins@google.com>
Subject: Re: [PATCH 4/4] lib/test_printf.c: use deterministic sequence of random numbers
Date: Fri, 30 Oct 2020 17:26:25 +0100 [thread overview]
Message-ID: <20201030162625.GE20201@alley> (raw)
In-Reply-To: <20201025214842.5924-5-linux@rasmusvillemoes.dk>
On Sun 2020-10-25 22:48:42, Rasmus Villemoes wrote:
> The printf test suite does each test with a few different buffer sizes
> to ensure vsnprintf() behaves correctly with respect to truncation and
> size reporting. It calls vsnprintf() with a buffer size that is
> guaranteed to be big enough, a buffer size of 0 to ensure that nothing
> gets written to the buffer, but it also calls vsnprintf() with a
> buffer size chosen to guarantee the output gets truncated somewhere in
> the middle.
>
> That buffer size is chosen randomly to increase the chance of finding
> some corner case bug (for example, there used to be some %p<foo>
> extension that would fail to produce any output if there wasn't room
> enough for it all, despite the requirement of producing as much as
> there's room for). I'm not aware of that having found anything yet,
> but should it happen, it's annoying not to be able to repeat the
> test with the same sequence of truncated lengths.
>
> For demonstration purposes, if we break one of the test cases
> deliberately, we still get different buffer sizes if we don't pass the
> seed parameter:
>
> root@(none):/# modprobe test_printf
> [ 15.317783] test_printf: vsnprintf(buf, 18, "%piS|%pIS", ...) wrote '127.000.000.001|1', expected '127-000.000.001|1'
> [ 15.323182] test_printf: failed 3 out of 388 tests
> [ 15.324034] test_printf: random seed used was 0x278bb9311979cc91
> modprobe: ERROR: could not insert 'test_printf': Invalid argument
>
> root@(none):/# modprobe test_printf
> [ 13.940909] test_printf: vsnprintf(buf, 22, "%piS|%pIS", ...) wrote '127.000.000.001|127.0', expected '127-000.000.001|127.0'
> [ 13.944744] test_printf: failed 3 out of 388 tests
> [ 13.945607] test_printf: random seed used was 0x9f72eee1c9dc02e5
> modprobe: ERROR: could not insert 'test_printf': Invalid argument
>
> but to repeat a specific sequence of tests, we can do
>
> root@(none):/# modprobe test_printf seed=0x9f72eee1c9dc02e5
> [ 448.328685] test_printf: vsnprintf(buf, 22, "%piS|%pIS", ...) wrote '127.000.000.001|127.0', expected '127-000.000.001|127.0'
> [ 448.331650] test_printf: failed 3 out of 388 tests
> [ 448.332295] test_printf: random seed used was 0x9f72eee1c9dc02e5
> modprobe: ERROR: could not insert 'test_printf': Invalid argument
>
> Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Great feature!
Reviewed-by: Petr Mladek <pmladek@suse.com>
Best Regards,
Petr
next prev parent reply other threads:[~2020-10-30 16:26 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-25 21:48 [PATCH 0/4] deterministic random testing Rasmus Villemoes
2020-10-25 21:48 ` [PATCH 1/4] prandom.h: add *_state variant of prandom_u32_max Rasmus Villemoes
2020-10-30 16:00 ` Petr Mladek
2020-10-25 21:48 ` [PATCH 2/4] kselftest_module.h: unconditionally expand the KSTM_MODULE_GLOBALS() macro Rasmus Villemoes
2020-10-30 16:02 ` Petr Mladek
2020-10-25 21:48 ` [PATCH 3/4] kselftest_module.h: add struct rnd_state and seed parameter Rasmus Villemoes
2020-10-30 16:23 ` Petr Mladek
2020-10-25 21:48 ` [PATCH 4/4] lib/test_printf.c: use deterministic sequence of random numbers Rasmus Villemoes
2020-10-30 16:26 ` Petr Mladek [this message]
2020-10-26 10:59 ` [PATCH 0/4] deterministic random testing Andy Shevchenko
2020-10-30 12:58 ` Rasmus Villemoes
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201030162625.GE20201@alley \
--to=pmladek@suse.com \
--cc=98.arpi@gmail.com \
--cc=andriy.shevchenko@linux.intel.com \
--cc=brendanhiggins@google.com \
--cc=keescook@chromium.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux@rasmusvillemoes.dk \
--cc=shuah@kernel.org \
--cc=w@1wt.eu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).