From: Matt Gilbride <mattgilbride@google.com>
To: Miguel Ojeda <ojeda@kernel.org>
Cc: "David Gow" <davidgow@google.com>,
"Brendan Higgins" <brendan.higgins@linux.dev>,
"Wedson Almeida Filho" <wedsonaf@gmail.com>,
"Alex Gaynor" <alex.gaynor@gmail.com>,
"Boqun Feng" <boqun.feng@gmail.com>,
"Gary Guo" <gary@garyguo.net>,
"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
"Benno Lossin" <benno.lossin@proton.me>,
"Alice Ryhl" <aliceryhl@google.com>,
"Andreas Hindborg" <nmi@metaspace.dk>,
"Philip Li" <philip.li@intel.com>,
kunit-dev@googlegroups.com, linux-kselftest@vger.kernel.org,
rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org,
patches@lists.linux.dev
Subject: Re: [PATCH 5/6] rust: support running Rust documentation tests as KUnit ones
Date: Wed, 14 Jun 2023 20:19:37 -0400 [thread overview]
Message-ID: <CABVGRkcnKX3Sb=LgUHz+E0XPxvemkuO+StAq4xxDY2xSaAt0GQ@mail.gmail.com> (raw)
In-Reply-To: <20230614180837.630180-6-ojeda@kernel.org>
> Rust has documentation tests: these are typically examples of
> usage of any item (e.g. function, struct, module...).
>
> They are very convenient because they are just written
> alongside the documentation. For instance:
>
> /// Sums two numbers.
> ///
> /// ```
> /// assert_eq!(mymod::f(10, 20), 30);
> /// ```
> pub fn f(a: i32, b: i32) -> i32 {
> a + b
> }
>
> In userspace, the tests are collected and run via `rustdoc`.
> Using the tool as-is would be useful already, since it allows
> to compile-test most tests (thus enforcing they are kept
> in sync with the code they document) and run those that do not
> depend on in-kernel APIs.
>
> However, by transforming the tests into a KUnit test suite,
> they can also be run inside the kernel. Moreover, the tests
> get to be compiled as other Rust kernel objects instead of
> targeting userspace.
>
> On top of that, the integration with KUnit means the Rust
> support gets to reuse the existing testing facilities. For
> instance, the kernel log would look like:
>
> KTAP version 1
> 1..1
> KTAP version 1
> # Subtest: rust_doctests_kernel
> 1..59
> # Doctest from line 13
> ok 1 rust_doctest_kernel_build_assert_rs_0
> # Doctest from line 56
> ok 2 rust_doctest_kernel_build_assert_rs_1
> # Doctest from line 122
> ok 3 rust_doctest_kernel_init_rs_0
> ...
> # Doctest from line 150
> ok 59 rust_doctest_kernel_types_rs_2
> # rust_doctests_kernel: pass:59 fail:0 skip:0 total:59
> # Totals: pass:59 fail:0 skip:0 total:59
> ok 1 rust_doctests_kernel
>
> Therefore, add support for running Rust documentation tests
> in KUnit. Some other notes about the current implementation
> and support follow.
>
> The transformation is performed by a couple scripts written
> as Rust hostprogs.
>
> Tests using the `?` operator are also supported as usual, e.g.:
>
> /// ```
> /// # use kernel::{spawn_work_item, workqueue};
> /// spawn_work_item!(workqueue::system(), || pr_info!("x"))?;
> /// # Ok::<(), Error>(())
> /// ```
>
> The tests are also compiled with Clippy under `CLIPPY=1`, just like
> normal code, thus also benefitting from extra linting.
>
> The names of the tests are currently automatically generated.
> This allows to reduce the burden for documentation writers,
> while keeping them fairly stable for bisection. This is an
> improvement over the `rustdoc`-generated names, which include
> the line number; but ideally we would like to get `rustdoc` to
> provide the Rust item path and a number (for multiple examples
> in a single documented Rust item).
>
> In order for developers to easily see from which original line
> a failed doctests came from, a KTAP diagnostic line is printed
> to the log. In the future, we may be able to use a proper KUnit
> facility to append this sort of information instead.
>
> A notable difference from KUnit C tests is that the Rust tests
> appear to assert using the usual `assert!` and `assert_eq!`
> macros from the Rust standard library (`core`). We provide
> a custom version that forwards the call to KUnit instead.
> Importantly, these macros do not require passing context,
> unlike the KUnit C ones (i.e. `struct kunit *`). This makes
> them easier to use, and readers of the documentation do not need
> to care about which testing framework is used. In addition, it
> may allow us to test third-party code more easily in the future.
>
> However, a current limitation is that KUnit does not support
> assertions in other tasks. Thus we presently simply print an
> error to the kernel log if an assertion actually failed. This
> should be revisited to properly fail the test, perhaps saving
> the context somewhere else, or letting KUnit handle it.
>
> Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Tested-by: Matt Gilbride <mattgilbride@google.com>
next prev parent reply other threads:[~2023-06-15 0:21 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-14 18:08 [PATCH 0/6] KUnit integration for Rust doctests Miguel Ojeda
2023-06-14 18:08 ` [PATCH 1/6] rust: init: make doctests compilable/testable Miguel Ojeda
2023-06-14 20:32 ` Alice Ryhl
2023-06-15 1:03 ` Martin Rodriguez Reboredo
2023-06-15 9:31 ` Miguel Ojeda
2023-06-15 13:33 ` Martin Rodriguez Reboredo
2023-06-15 23:47 ` Vincenzo Palazzo
2023-06-16 4:51 ` David Gow
2023-06-16 11:12 ` Björn Roy Baron
2023-06-25 10:13 ` Benno Lossin
2023-06-14 18:08 ` [PATCH 2/6] rust: str: " Miguel Ojeda
2023-06-14 20:32 ` Alice Ryhl
2023-06-15 1:04 ` Martin Rodriguez Reboredo
2023-06-15 23:47 ` Vincenzo Palazzo
2023-06-16 4:51 ` David Gow
2023-06-16 11:13 ` Björn Roy Baron
2023-06-14 18:08 ` [PATCH 3/6] rust: sync: " Miguel Ojeda
2023-06-14 20:32 ` Alice Ryhl
2023-06-15 1:04 ` Martin Rodriguez Reboredo
2023-06-16 4:51 ` David Gow
2023-06-16 11:14 ` Björn Roy Baron
2023-06-14 18:08 ` [PATCH 4/6] rust: types: " Miguel Ojeda
2023-06-14 20:32 ` Alice Ryhl
2023-06-15 1:06 ` Martin Rodriguez Reboredo
2023-06-16 4:52 ` David Gow
2023-06-16 11:14 ` Björn Roy Baron
2023-06-14 18:08 ` [PATCH 5/6] rust: support running Rust documentation tests as KUnit ones Miguel Ojeda
2023-06-14 20:38 ` Alice Ryhl
2023-06-14 22:09 ` Miguel Ojeda
2023-06-14 22:09 ` Miguel Ojeda
2023-06-15 0:19 ` Matt Gilbride [this message]
2023-06-15 3:49 ` Martin Rodriguez Reboredo
2023-06-15 9:23 ` Miguel Ojeda
2023-06-15 13:50 ` Martin Rodriguez Reboredo
2023-06-15 23:53 ` Vincenzo Palazzo
2023-06-16 4:52 ` David Gow
2023-06-19 17:16 ` Sergio González Collado
2023-06-30 18:17 ` Boqun Feng
2023-06-14 18:08 ` [PATCH 6/6] MAINTAINERS: add Rust KUnit files to the KUnit entry Miguel Ojeda
2023-06-15 3:49 ` Martin Rodriguez Reboredo
2023-06-15 23:46 ` Vincenzo Palazzo
2023-06-16 4:53 ` David Gow
2023-06-15 1:44 ` [PATCH 0/6] KUnit integration for Rust doctests Boqun Feng
2023-06-15 8:20 ` Miguel Ojeda
2023-06-16 4:44 ` David Gow
2023-06-16 4:51 ` David Gow
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CABVGRkcnKX3Sb=LgUHz+E0XPxvemkuO+StAq4xxDY2xSaAt0GQ@mail.gmail.com' \
--to=mattgilbride@google.com \
--cc=alex.gaynor@gmail.com \
--cc=aliceryhl@google.com \
--cc=benno.lossin@proton.me \
--cc=bjorn3_gh@protonmail.com \
--cc=boqun.feng@gmail.com \
--cc=brendan.higgins@linux.dev \
--cc=davidgow@google.com \
--cc=gary@garyguo.net \
--cc=kunit-dev@googlegroups.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=nmi@metaspace.dk \
--cc=ojeda@kernel.org \
--cc=patches@lists.linux.dev \
--cc=philip.li@intel.com \
--cc=rust-for-linux@vger.kernel.org \
--cc=wedsonaf@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).