* [Linux-kernel-mentees] [PATCH] kunit: Customize KUNIT_EXCEPT/KUNIT_ASSERT Expected messages
@ 2020-08-19 19:39 Vitor Massaru Iha
2020-08-19 22:24 ` Brendan Higgins via Linux-kernel-mentees
0 siblings, 1 reply; 3+ messages in thread
From: Vitor Massaru Iha @ 2020-08-19 19:39 UTC (permalink / raw)
To: kunit-dev; +Cc: linux-kernel-mentees, brendanhiggins, linux-kselftest
In some cases, to maintain the consistency of the Expected messages
with the original runtime test, it is necessary to customize the
Expected messages on KUnit.
As an example test_overflow conversion to KUnit (I added 1,
p->s_of+1, just to fail the test).
Using KUNIT_EXPECT_EQ:
Expected _of == p->s_of+1, but
_of == 0
p->s_of+1 == 1
not ok 1 - overflow_calculation_test
ok 2 - overflow_shift_test
Using KUNIT_EXPECT_EQ_CUSTOM_MSG:
Expected 0 + 0 to not overflow (type u8)
not ok 1 - overflow_calculation_test
Which is more similar to the error message of the original test.
Signed-off-by: Vitor Massaru Iha <vitor@massaru.org>
---
include/kunit/assert.h | 8 ++-
include/kunit/test.h | 151 +++++++++++++++++++++++++++++++++++++++--
lib/kunit/assert.c | 34 ++++++----
3 files changed, 172 insertions(+), 21 deletions(-)
diff --git a/include/kunit/assert.h b/include/kunit/assert.h
index ad889b539ab3..b3c25b4420c8 100644
--- a/include/kunit/assert.h
+++ b/include/kunit/assert.h
@@ -204,6 +204,7 @@ struct kunit_binary_assert {
long long left_value;
const char *right_text;
long long right_value;
+ bool custom_msg;
};
void kunit_binary_assert_format(const struct kunit_assert *assert,
@@ -219,6 +220,7 @@ void kunit_binary_assert_format(const struct kunit_assert *assert,
* @left_val: The actual evaluated value of the expression in the left slot.
* @right_str: A string representation of the expression in the right slot.
* @right_val: The actual evaluated value of the expression in the right slot.
+ * @custom_msg_bool: Show a custom expect message instead the default message.
*
* Initializes a &struct kunit_binary_assert. Intended to be used in
* KUNIT_EXPECT_* and KUNIT_ASSERT_* macros.
@@ -229,7 +231,8 @@ void kunit_binary_assert_format(const struct kunit_assert *assert,
left_str, \
left_val, \
right_str, \
- right_val) { \
+ right_val, \
+ custom_msg_bool) { \
.assert = KUNIT_INIT_ASSERT_STRUCT(test, \
type, \
kunit_binary_assert_format), \
@@ -237,7 +240,8 @@ void kunit_binary_assert_format(const struct kunit_assert *assert,
.left_text = left_str, \
.left_value = left_val, \
.right_text = right_str, \
- .right_value = right_val \
+ .right_value = right_val, \
+ .custom_msg = custom_msg_bool, \
}
/**
diff --git a/include/kunit/test.h b/include/kunit/test.h
index 59f3144f009a..ec821a57ec5b 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -699,6 +699,7 @@ void kunit_do_assertion(struct kunit *test,
left, \
op, \
right, \
+ custom_msg_bool, \
fmt, \
...) \
do { \
@@ -715,7 +716,8 @@ do { \
#left, \
__left, \
#right, \
- __right), \
+ __right, \
+ custom_msg_bool), \
fmt, \
##__VA_ARGS__); \
} while (0)
@@ -726,6 +728,7 @@ do { \
assert_type, \
left, \
right, \
+ custom_msg_bool, \
fmt, \
...) \
KUNIT_BASE_BINARY_ASSERTION(test, \
@@ -733,6 +736,7 @@ do { \
ASSERT_CLASS_INIT, \
assert_type, \
left, ==, right, \
+ custom_msg_bool, \
fmt, \
##__VA_ARGS__)
@@ -742,6 +746,7 @@ do { \
assert_type, \
left, \
right, \
+ custom_msg_bool, \
fmt, \
...) \
KUNIT_BASE_BINARY_ASSERTION(test, \
@@ -749,6 +754,7 @@ do { \
ASSERT_CLASS_INIT, \
assert_type, \
left, !=, right, \
+ custom_msg_bool, \
fmt, \
##__VA_ARGS__)
@@ -765,6 +771,7 @@ do { \
ASSERT_CLASS_INIT, \
assert_type, \
left, <, right, \
+ custom_msg_bool, \
fmt, \
##__VA_ARGS__)
@@ -774,6 +781,7 @@ do { \
assert_type, \
left, \
right, \
+ custom_msg_bool, \
fmt, \
...) \
KUNIT_BASE_BINARY_ASSERTION(test, \
@@ -781,6 +789,7 @@ do { \
ASSERT_CLASS_INIT, \
assert_type, \
left, <=, right, \
+ custom_msg_bool, \
fmt, \
##__VA_ARGS__)
@@ -797,6 +806,7 @@ do { \
ASSERT_CLASS_INIT, \
assert_type, \
left, >, right, \
+ custom_msg_bool, \
fmt, \
##__VA_ARGS__)
@@ -813,16 +823,23 @@ do { \
ASSERT_CLASS_INIT, \
assert_type, \
left, >=, right, \
+ custom_msg_bool, \
fmt, \
##__VA_ARGS__)
-#define KUNIT_BINARY_EQ_MSG_ASSERTION(test, assert_type, left, right, fmt, ...)\
+#define KUNIT_BINARY_EQ_MSG_ASSERTION(test, \
+ assert_type, \
+ left, right, \
+ custom_msg_bool, \
+ fmt, \
+ ...) \
KUNIT_BASE_EQ_MSG_ASSERTION(test, \
kunit_binary_assert, \
KUNIT_INIT_BINARY_ASSERT_STRUCT, \
assert_type, \
left, \
right, \
+ custom_msg_bool, \
fmt, \
##__VA_ARGS__)
@@ -831,6 +848,7 @@ do { \
assert_type, \
left, \
right, \
+ false, \
NULL)
#define KUNIT_BINARY_PTR_EQ_MSG_ASSERTION(test, \
@@ -862,6 +880,18 @@ do { \
assert_type, \
left, \
right, \
+ false, \
+ fmt, \
+ ##__VA_ARGS__)
+
+#define KUNIT_BINARY_NE_CUSTOM_MSG_ASSERTION(test, assert_type, left, right, fmt, ...)\
+ KUNIT_BASE_NE_MSG_ASSERTION(test, \
+ kunit_binary_assert, \
+ KUNIT_INIT_BINARY_ASSERT_STRUCT, \
+ assert_type, \
+ left, \
+ right, \
+ true, \
fmt, \
##__VA_ARGS__)
@@ -870,6 +900,7 @@ do { \
assert_type, \
left, \
right, \
+ false, \
NULL)
#define KUNIT_BINARY_PTR_NE_MSG_ASSERTION(test, \
@@ -904,11 +935,12 @@ do { \
fmt, \
##__VA_ARGS__)
-#define KUNIT_BINARY_LT_ASSERTION(test, assert_type, left, right) \
+#define KUNIT_BINARY_LT_ASSERTION(test, assert_type, left, right, custom_msg_bool)\
KUNIT_BINARY_LT_MSG_ASSERTION(test, \
assert_type, \
left, \
right, \
+ custom_msg_bool, \
NULL)
#define KUNIT_BINARY_PTR_LT_MSG_ASSERTION(test, \
@@ -923,6 +955,7 @@ do { \
assert_type, \
left, \
right, \
+ false, \
fmt, \
##__VA_ARGS__)
@@ -933,13 +966,20 @@ do { \
right, \
NULL)
-#define KUNIT_BINARY_LE_MSG_ASSERTION(test, assert_type, left, right, fmt, ...)\
+#define KUNIT_BINARY_LE_MSG_ASSERTION(test, \
+ assert_type, \
+ left, \
+ right, \
+ custom_msg_bool, \
+ fmt, \
+ ...) \
KUNIT_BASE_LE_MSG_ASSERTION(test, \
kunit_binary_assert, \
KUNIT_INIT_BINARY_ASSERT_STRUCT, \
assert_type, \
left, \
right, \
+ custom_msg_bool, \
fmt, \
##__VA_ARGS__)
@@ -948,6 +988,7 @@ do { \
assert_type, \
left, \
right, \
+ false, \
NULL)
#define KUNIT_BINARY_PTR_LE_MSG_ASSERTION(test, \
@@ -972,13 +1013,20 @@ do { \
right, \
NULL)
-#define KUNIT_BINARY_GT_MSG_ASSERTION(test, assert_type, left, right, fmt, ...)\
+#define KUNIT_BINARY_GT_MSG_ASSERTION(test, \
+ assert_type, \
+ left, \
+ right, \
+ custom_msg_bool, \
+ fmt, \
+ ...) \
KUNIT_BASE_GT_MSG_ASSERTION(test, \
kunit_binary_assert, \
KUNIT_INIT_BINARY_ASSERT_STRUCT, \
assert_type, \
left, \
right, \
+ custom_msg_bool, \
fmt, \
##__VA_ARGS__)
@@ -1001,6 +1049,7 @@ do { \
assert_type, \
left, \
right, \
+ custom_msg_bool, \
fmt, \
##__VA_ARGS__)
@@ -1011,7 +1060,13 @@ do { \
right, \
NULL)
-#define KUNIT_BINARY_GE_MSG_ASSERTION(test, assert_type, left, right, fmt, ...)\
+#define KUNIT_BINARY_GE_MSG_ASSERTION(test, \
+ assert_type, \
+ left, \
+ right, \
+ custom_msg_bool, \
+ fmt, \
+ ...) \
KUNIT_BASE_GE_MSG_ASSERTION(test, \
kunit_binary_assert, \
KUNIT_INIT_BINARY_ASSERT_STRUCT, \
@@ -1026,12 +1081,14 @@ do { \
assert_type, \
left, \
right, \
+ false, \
NULL)
#define KUNIT_BINARY_PTR_GE_MSG_ASSERTION(test, \
assert_type, \
left, \
right, \
+ custom_msg_bool, \
fmt, \
...) \
KUNIT_BASE_GE_MSG_ASSERTION(test, \
@@ -1040,6 +1097,7 @@ do { \
assert_type, \
left, \
right, \
+ custom_msg_bool, \
fmt, \
##__VA_ARGS__)
@@ -1197,6 +1255,16 @@ do { \
KUNIT_EXPECTATION, \
left, \
right, \
+ false, \
+ fmt, \
+ ##__VA_ARGS__)
+
+#define KUNIT_EXPECT_EQ_CUSTOM_MSG(test, left, right, fmt, ...) \
+ KUNIT_BINARY_EQ_MSG_ASSERTION(test, \
+ KUNIT_EXPECTATION, \
+ left, \
+ right, \
+ true, \
fmt, \
##__VA_ARGS__)
@@ -1222,9 +1290,18 @@ do { \
KUNIT_EXPECTATION, \
left, \
right, \
+ false, \
fmt, \
##__VA_ARGS__)
+#define KUNIT_EXPECT_PTR_EQ_CUSTOM_MSG(test, left, right, fmt, ...) \
+ KUNIT_BINARY_PTR_EQ_MSG_ASSERTION(test, \
+ KUNIT_EXPECTATION, \
+ left, \
+ right, \
+ true, \
+ fmt, \
+ ##__VA_ARGS__)
/**
* KUNIT_EXPECT_NE() - An expectation that @left and @right are not equal.
* @test: The test context object.
@@ -1244,6 +1321,16 @@ do { \
KUNIT_EXPECTATION, \
left, \
right, \
+ false, \
+ fmt, \
+ ##__VA_ARGS__)
+
+#define KUNIT_EXPECT_NE_CUSTOM_MSG(test, left, right, fmt, ...) \
+ KUNIT_BINARY_NE_MSG_ASSERTION(test, \
+ KUNIT_EXPECTATION, \
+ left, \
+ right, \
+ true, \
fmt, \
##__VA_ARGS__)
@@ -1269,6 +1356,7 @@ do { \
KUNIT_EXPECTATION, \
left, \
right, \
+ false, \
fmt, \
##__VA_ARGS__)
@@ -1313,6 +1401,16 @@ do { \
KUNIT_EXPECTATION, \
left, \
right, \
+ false, \
+ fmt, \
+ ##__VA_ARGS__)
+
+#define KUNIT_EXPECT_LE_CUSTOM_MSG(test, left, right, fmt, ...) \
+ KUNIT_BINARY_LE_MSG_ASSERTION(test, \
+ KUNIT_EXPECTATION, \
+ left, \
+ right, \
+ true, \
fmt, \
##__VA_ARGS__)
@@ -1467,6 +1565,8 @@ do { \
fmt, \
##__VA_ARGS__)
+//#define KUNIT_ASSERT_EQ_TYPE(test, )
+
/**
* KUNIT_ASSERT_EQ() - Sets an assertion that @left and @right are equal.
* @test: The test context object.
@@ -1485,6 +1585,16 @@ do { \
KUNIT_ASSERTION, \
left, \
right, \
+ false, \
+ fmt, \
+ ##__VA_ARGS__)
+
+#define KUNIT_ASSERT_EQ_CUSTOM_MSG(test, left, right, fmt, ...) \
+ KUNIT_BINARY_EQ_MSG_ASSERTION(test, \
+ KUNIT_ASSERTION, \
+ left, \
+ right, \
+ true, \
fmt, \
##__VA_ARGS__)
@@ -1506,9 +1616,18 @@ do { \
KUNIT_ASSERTION, \
left, \
right, \
+ false, \
fmt, \
##__VA_ARGS__)
+#define KUNIT_ASSERT_PTR_EQ_CUSTOM_MSG(test, left, right, fmt, ...) \
+ KUNIT_BINARY_PTR_EQ_MSG_ASSERTION(test, \
+ KUNIT_ASSERTION, \
+ left, \
+ right, \
+ true, \
+ fmt, \
+ ##__VA_ARGS__)
/**
* KUNIT_ASSERT_NE() - An assertion that @left and @right are not equal.
* @test: The test context object.
@@ -1527,6 +1646,7 @@ do { \
KUNIT_ASSERTION, \
left, \
right, \
+ false, \
fmt, \
##__VA_ARGS__)
@@ -1549,6 +1669,7 @@ do { \
KUNIT_ASSERTION, \
left, \
right, \
+ false, \
fmt, \
##__VA_ARGS__)
/**
@@ -1594,6 +1715,15 @@ do { \
fmt, \
##__VA_ARGS__)
+#define KUNIT_ASSERT_LE_CUSTOM_MSG(test, left, right, fmt, ...) \
+ KUNIT_BINARY_LE_MSG_ASSERTION(test, \
+ KUNIT_ASSERTION, \
+ left, \
+ right, \
+ true, \
+ fmt, \
+ ##__VA_ARGS__)
+
/**
* KUNIT_ASSERT_GT() - An assertion that @left is greater than @right.
* @test: The test context object.
@@ -1638,6 +1768,15 @@ do { \
fmt, \
##__VA_ARGS__)
+#define KUNIT_ASSERT_GE_CUSTOM_MSG(test, left, right, fmt, ...) \
+ KUNIT_BINARY_GE_MSG_ASSERTION(test, \
+ KUNIT_ASSERTION, \
+ left, \
+ right, \
+ true, \
+ fmt, \
+ ##__VA_ARGS__)
+
/**
* KUNIT_ASSERT_STREQ() - An assertion that strings @left and @right are equal.
* @test: The test context object.
diff --git a/lib/kunit/assert.c b/lib/kunit/assert.c
index 33acdaa28a7d..202f9fdeed0e 100644
--- a/lib/kunit/assert.c
+++ b/lib/kunit/assert.c
@@ -32,8 +32,14 @@ EXPORT_SYMBOL_GPL(kunit_base_assert_format);
void kunit_assert_print_msg(const struct kunit_assert *assert,
struct string_stream *stream)
{
- if (assert->message.fmt)
- string_stream_add(stream, "\n%pV", &assert->message);
+ struct kunit_binary_assert *binary_assert = container_of(
+ assert, struct kunit_binary_assert, assert);
+ if (assert->message.fmt) {
+ if (binary_assert->custom_msg == false)
+ string_stream_add(stream, "\n" KUNIT_SUBSUBTEST_INDENT "%pV", &assert->message);
+ else
+ string_stream_add(stream, KUNIT_SUBTEST_INDENT "%pV", &assert->message);
+ }
}
EXPORT_SYMBOL_GPL(kunit_assert_print_msg);
@@ -92,17 +98,19 @@ void kunit_binary_assert_format(const struct kunit_assert *assert,
assert, struct kunit_binary_assert, assert);
kunit_base_assert_format(assert, stream);
- string_stream_add(stream,
- KUNIT_SUBTEST_INDENT "Expected %s %s %s, but\n",
- binary_assert->left_text,
- binary_assert->operation,
- binary_assert->right_text);
- string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld\n",
- binary_assert->left_text,
- binary_assert->left_value);
- string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld",
- binary_assert->right_text,
- binary_assert->right_value);
+ if (binary_assert->custom_msg == false) {
+ string_stream_add(stream,
+ KUNIT_SUBTEST_INDENT "Expected %s %s %s, but\n",
+ binary_assert->left_text,
+ binary_assert->operation,
+ binary_assert->right_text);
+ string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld\n",
+ binary_assert->left_text,
+ binary_assert->left_value);
+ string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld",
+ binary_assert->right_text,
+ binary_assert->right_value);
+ }
kunit_assert_print_msg(assert, stream);
}
EXPORT_SYMBOL_GPL(kunit_binary_assert_format);
base-commit: d43c7fb05765152d4d4a39a8ef957c4ea14d8847
--
2.26.2
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [Linux-kernel-mentees] [PATCH] kunit: Customize KUNIT_EXCEPT/KUNIT_ASSERT Expected messages
2020-08-19 19:39 [Linux-kernel-mentees] [PATCH] kunit: Customize KUNIT_EXCEPT/KUNIT_ASSERT Expected messages Vitor Massaru Iha
@ 2020-08-19 22:24 ` Brendan Higgins via Linux-kernel-mentees
2020-08-20 20:02 ` Vitor Massaru Iha
0 siblings, 1 reply; 3+ messages in thread
From: Brendan Higgins via Linux-kernel-mentees @ 2020-08-19 22:24 UTC (permalink / raw)
To: Vitor Massaru Iha
Cc: linux-kernel-mentees, open list:KERNEL SELFTEST FRAMEWORK,
KUnit Development
On Wed, Aug 19, 2020 at 12:39 PM Vitor Massaru Iha <vitor@massaru.org> wrote:
>
> In some cases, to maintain the consistency of the Expected messages
> with the original runtime test, it is necessary to customize the
> Expected messages on KUnit.
>
> As an example test_overflow conversion to KUnit (I added 1,
> p->s_of+1, just to fail the test).
>
> Using KUNIT_EXPECT_EQ:
>
> Expected _of == p->s_of+1, but
> _of == 0
> p->s_of+1 == 1
> not ok 1 - overflow_calculation_test
> ok 2 - overflow_shift_test
>
> Using KUNIT_EXPECT_EQ_CUSTOM_MSG:
>
> Expected 0 + 0 to not overflow (type u8)
> not ok 1 - overflow_calculation_test
>
> Which is more similar to the error message of the original test.
>
> Signed-off-by: Vitor Massaru Iha <vitor@massaru.org>
All in all this looks pretty reasonable to me; however, is there any
chance you could provide an example of its usage?
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [Linux-kernel-mentees] [PATCH] kunit: Customize KUNIT_EXCEPT/KUNIT_ASSERT Expected messages
2020-08-19 22:24 ` Brendan Higgins via Linux-kernel-mentees
@ 2020-08-20 20:02 ` Vitor Massaru Iha
0 siblings, 0 replies; 3+ messages in thread
From: Vitor Massaru Iha @ 2020-08-20 20:02 UTC (permalink / raw)
To: Brendan Higgins
Cc: linux-kernel-mentees, open list:KERNEL SELFTEST FRAMEWORK,
KUnit Development
On Wed, Aug 19, 2020 at 7:24 PM Brendan Higgins
<brendanhiggins@google.com> wrote:
>
> On Wed, Aug 19, 2020 at 12:39 PM Vitor Massaru Iha <vitor@massaru.org> wrote:
> >
> > In some cases, to maintain the consistency of the Expected messages
> > with the original runtime test, it is necessary to customize the
> > Expected messages on KUnit.
> >
> > As an example test_overflow conversion to KUnit (I added 1,
> > p->s_of+1, just to fail the test).
> >
> > Using KUNIT_EXPECT_EQ:
> >
> > Expected _of == p->s_of+1, but
> > _of == 0
> > p->s_of+1 == 1
> > not ok 1 - overflow_calculation_test
> > ok 2 - overflow_shift_test
> >
> > Using KUNIT_EXPECT_EQ_CUSTOM_MSG:
> >
> > Expected 0 + 0 to not overflow (type u8)
> > not ok 1 - overflow_calculation_test
> >
> > Which is more similar to the error message of the original test.
> >
> > Signed-off-by: Vitor Massaru Iha <vitor@massaru.org>
>
> All in all this looks pretty reasonable to me; however, is there any
> chance you could provide an example of its usage?
Sure I'll do that.
I am reviewing the errors reported by 0day in the overflow_kunit patch v3,
and I noticed when compiling overflow_kunit with the parameters of
0day and this patch, there were some errors related to this patch that I sent.
I'll fix it too.
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-08-20 20:03 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-19 19:39 [Linux-kernel-mentees] [PATCH] kunit: Customize KUNIT_EXCEPT/KUNIT_ASSERT Expected messages Vitor Massaru Iha
2020-08-19 22:24 ` Brendan Higgins via Linux-kernel-mentees
2020-08-20 20:02 ` Vitor Massaru Iha
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).