Linux-kselftest Archive on lore.kernel.org
 help / color / Atom feed
From: Rasmus Villemoes <linux@rasmusvillemoes.dk>
To: Shuah Khan <shuah@kernel.org>, Kees Cook <keescook@chromium.org>
Cc: Petr Mladek <pmladek@suse.com>, Willy Tarreau <w@1wt.eu>,
	linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org,
	Arpitha Raghunandan <98.arpi@gmail.com>,
	Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
	Brendan Higgins <brendanhiggins@google.com>,
	Rasmus Villemoes <linux@rasmusvillemoes.dk>
Subject: [PATCH 4/4] lib/test_printf.c: use deterministic sequence of random numbers
Date: Sun, 25 Oct 2020 22:48:42 +0100
Message-ID: <20201025214842.5924-5-linux@rasmusvillemoes.dk> (raw)
In-Reply-To: <20201025214842.5924-1-linux@rasmusvillemoes.dk>

The printf test suite does each test with a few different buffer sizes
to ensure vsnprintf() behaves correctly with respect to truncation and
size reporting. It calls vsnprintf() with a buffer size that is
guaranteed to be big enough, a buffer size of 0 to ensure that nothing
gets written to the buffer, but it also calls vsnprintf() with a
buffer size chosen to guarantee the output gets truncated somewhere in
the middle.

That buffer size is chosen randomly to increase the chance of finding
some corner case bug (for example, there used to be some %p<foo>
extension that would fail to produce any output if there wasn't room
enough for it all, despite the requirement of producing as much as
there's room for). I'm not aware of that having found anything yet,
but should it happen, it's annoying not to be able to repeat the
test with the same sequence of truncated lengths.

For demonstration purposes, if we break one of the test cases
deliberately, we still get different buffer sizes if we don't pass the
seed parameter:

root@(none):/# modprobe test_printf
[   15.317783] test_printf: vsnprintf(buf, 18, "%piS|%pIS", ...) wrote '127.000.000.001|1', expected '127-000.000.001|1'
[   15.323182] test_printf: failed 3 out of 388 tests
[   15.324034] test_printf: random seed used was 0x278bb9311979cc91
modprobe: ERROR: could not insert 'test_printf': Invalid argument

root@(none):/# modprobe test_printf
[   13.940909] test_printf: vsnprintf(buf, 22, "%piS|%pIS", ...) wrote '127.000.000.001|127.0', expected '127-000.000.001|127.0'
[   13.944744] test_printf: failed 3 out of 388 tests
[   13.945607] test_printf: random seed used was 0x9f72eee1c9dc02e5
modprobe: ERROR: could not insert 'test_printf': Invalid argument

but to repeat a specific sequence of tests, we can do

root@(none):/# modprobe test_printf seed=0x9f72eee1c9dc02e5
[  448.328685] test_printf: vsnprintf(buf, 22, "%piS|%pIS", ...) wrote '127.000.000.001|127.0', expected '127-000.000.001|127.0'
[  448.331650] test_printf: failed 3 out of 388 tests
[  448.332295] test_printf: random seed used was 0x9f72eee1c9dc02e5
modprobe: ERROR: could not insert 'test_printf': Invalid argument

Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
---
 lib/test_printf.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/lib/test_printf.c b/lib/test_printf.c
index 1ed4a27390cb621715ab..bbea8b807d1eafe67e01 100644
--- a/lib/test_printf.c
+++ b/lib/test_printf.c
@@ -24,6 +24,7 @@
 
 #include <linux/property.h>
 
+#define KSTM_RANDOM 1
 #include "../tools/testing/selftests/kselftest_module.h"
 
 #define BUF_SIZE 256
@@ -111,8 +112,14 @@ __test(const char *expect, int elen, const char *fmt, ...)
 	 * be able to print it as expected.
 	 */
 	failed_tests += do_test(BUF_SIZE, expect, elen, fmt, ap);
-	rand = 1 + prandom_u32_max(elen+1);
-	/* Since elen < BUF_SIZE, we have 1 <= rand <= BUF_SIZE. */
+	rand = prandom_u32_range_state(&rnd_state, 1, elen + 1);
+	/*
+	 * Except for elen == 0, we have 1 <= rand <= elen < BUF_SIZE,
+	 * i.e., the output is guaranteed to be truncated somewhere in
+	 * the middle, and we're not pretending the buffer to be
+	 * larger than it really is. For the boring case of elen == 0,
+	 * rand is 1, which is of course also <= BUF_SIZE.
+	 */
 	failed_tests += do_test(rand, expect, elen, fmt, ap);
 	failed_tests += do_test(0, expect, elen, fmt, ap);
 
-- 
2.23.0


  parent reply index

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-25 21:48 [PATCH 0/4] deterministic random testing Rasmus Villemoes
2020-10-25 21:48 ` [PATCH 1/4] prandom.h: add *_state variant of prandom_u32_max Rasmus Villemoes
2020-10-30 16:00   ` Petr Mladek
2020-10-25 21:48 ` [PATCH 2/4] kselftest_module.h: unconditionally expand the KSTM_MODULE_GLOBALS() macro Rasmus Villemoes
2020-10-30 16:02   ` Petr Mladek
2020-10-25 21:48 ` [PATCH 3/4] kselftest_module.h: add struct rnd_state and seed parameter Rasmus Villemoes
2020-10-30 16:23   ` Petr Mladek
2020-10-25 21:48 ` Rasmus Villemoes [this message]
2020-10-30 16:26   ` [PATCH 4/4] lib/test_printf.c: use deterministic sequence of random numbers Petr Mladek
2020-10-26 10:59 ` [PATCH 0/4] deterministic random testing Andy Shevchenko
2020-10-30 12:58   ` Rasmus Villemoes

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201025214842.5924-5-linux@rasmusvillemoes.dk \
    --to=linux@rasmusvillemoes.dk \
    --cc=98.arpi@gmail.com \
    --cc=andriy.shevchenko@linux.intel.com \
    --cc=brendanhiggins@google.com \
    --cc=keescook@chromium.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=pmladek@suse.com \
    --cc=shuah@kernel.org \
    --cc=w@1wt.eu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-kselftest Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-kselftest/0 linux-kselftest/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-kselftest linux-kselftest/ https://lore.kernel.org/linux-kselftest \
		linux-kselftest@vger.kernel.org
	public-inbox-index linux-kselftest

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kselftest


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git