From: Brendan Higgins <brendanhiggins@google.com>
To: Stephen Boyd <sboyd@kernel.org>
Cc: Frank Rowand <frowand.list@gmail.com>,
Greg KH <gregkh@linuxfoundation.org>,
Josh Poimboeuf <jpoimboe@redhat.com>,
Kees Cook <keescook@google.com>,
Kieran Bingham <kieran.bingham@ideasonboard.com>,
Luis Chamberlain <mcgrof@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Rob Herring <robh@kernel.org>, shuah <shuah@kernel.org>,
"Theodore Ts'o" <tytso@mit.edu>,
Masahiro Yamada <yamada.masahiro@socionext.com>,
devicetree <devicetree@vger.kernel.org>,
dri-devel <dri-devel@lists.freedesktop.org>,
kunit-dev@googlegroups.com,
"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
linux-fsdevel@vger.kernel.org,
linux-kbuild <linux-kbuild@vger.kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
"open list:KERNEL SELFTEST FRAMEWORK"
<linux-kselftest@vger.kernel.org>,
linux-nvdimm <linux-nvdimm@lists.01.org>,
linux-um@lists.infradead.org,
Sasha Levin <Alexander.Levin@microsoft.com>,
"Bird, Timothy" <Tim.Bird@sony.com>,
Amir Goldstein <amir73il@gmail.com>,
Dan Carpenter <dan.carpenter@oracle.com>,
Daniel Vetter <daniel@ffwll.ch>, Jeff Dike <jdike@addtoit.com>,
Joel Stanley <joel@jms.id.au>,
Julia Lawall <julia.lawall@lip6.fr>,
Kevin Hilman <khilman@baylibre.com>,
Knut Omang <knut.omang@oracle.com>,
Logan Gunthorpe <logang@deltatee.com>,
Michael Ellerman <mpe@ellerman.id.au>,
Petr Mladek <pmladek@suse.com>,
Randy Dunlap <rdunlap@infradead.org>,
Richard Weinberger <richard@nod.at>,
David Rientjes <rientjes@google.com>,
Steven Rostedt <rostedt@goodmis.org>,
wfg@linux.intel.com
Subject: Re: [PATCH v9 03/18] kunit: test: add string_stream a std::stream like string builder
Date: Mon, 15 Jul 2019 14:11:50 -0700 [thread overview]
Message-ID: <CAFd5g47481sRaez=yEJN4_ghiXZbxayk1Y04tAZpuzPLsmnhKg@mail.gmail.com> (raw)
In-Reply-To: <20190715204356.4E3F92145D@mail.kernel.org>
On Mon, Jul 15, 2019 at 1:43 PM Stephen Boyd <sboyd@kernel.org> wrote:
>
> Quoting Brendan Higgins (2019-07-12 01:17:29)
> > diff --git a/include/kunit/string-stream.h b/include/kunit/string-stream.h
> > new file mode 100644
> > index 0000000000000..0552a05781afe
> > --- /dev/null
> > +++ b/include/kunit/string-stream.h
> > @@ -0,0 +1,49 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * C++ stream style string builder used in KUnit for building messages.
> > + *
> > + * Copyright (C) 2019, Google LLC.
> > + * Author: Brendan Higgins <brendanhiggins@google.com>
> > + */
> > +
> > +#ifndef _KUNIT_STRING_STREAM_H
> > +#define _KUNIT_STRING_STREAM_H
> > +
> > +#include <linux/types.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/kref.h>
>
> What is this include for? I'd expect to see linux/list.h instead.
Sorry about that. I used to reference count this before I made it a
kunit managed resource.
> > +#include <stdarg.h>
> > +
> > +struct string_stream_fragment {
> > + struct list_head node;
> > + char *fragment;
> > +};
> > +
> > +struct string_stream {
> > + size_t length;
> > + struct list_head fragments;
> > + /* length and fragments are protected by this lock */
> > + spinlock_t lock;
> > +};
> > +
> > diff --git a/kunit/string-stream.c b/kunit/string-stream.c
> > new file mode 100644
> > index 0000000000000..0463a92dad74b
> > --- /dev/null
> > +++ b/kunit/string-stream.c
> > @@ -0,0 +1,147 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * C++ stream style string builder used in KUnit for building messages.
> > + *
> > + * Copyright (C) 2019, Google LLC.
> > + * Author: Brendan Higgins <brendanhiggins@google.com>
> > + */
> > +
> > +#include <linux/list.h>
> > +#include <linux/slab.h>
> > +#include <kunit/string-stream.h>
> > +#include <kunit/test.h>
> > +
> > +int string_stream_vadd(struct string_stream *stream,
> > + const char *fmt,
> > + va_list args)
> > +{
> > + struct string_stream_fragment *frag_container;
> > + int len;
> > + va_list args_for_counting;
> > + unsigned long flags;
> > +
> > + /* Make a copy because `vsnprintf` could change it */
> > + va_copy(args_for_counting, args);
> > +
> > + /* Need space for null byte. */
> > + len = vsnprintf(NULL, 0, fmt, args_for_counting) + 1;
> > +
> > + va_end(args_for_counting);
> > +
> > + frag_container = kmalloc(sizeof(*frag_container), GFP_KERNEL);
>
> This is confusing in that it allocates with GFP_KERNEL but then grabs a
> spinlock to add and remove from the fragment list. Is it ever going to
> be called from a place where it can't sleep? If so, the GFP_KERNEL needs
> to be changed. Otherwise, maybe a mutex would work better to protect
> access to the fragment list.
Right, using a mutex here would be fine. Sorry, I meant to filter for
my usage of them after you asked me to remove them in 01, but
evidently I forgot to do so. Sorry, will fix.
> I also wonder if it would be better to just have a big slop buffer of a
> 4K page or something so that we almost never have to allocate anything
> with a string_stream and we can just rely on a reader consuming data
> while writers are writing. That might work out better, but I don't quite
> understand the use case for the string stream.
That makes sense, but might that also waste memory since we will
almost never need that much memory?
> > + if (!frag_container)
> > + return -ENOMEM;
> > +
> > + frag_container->fragment = kmalloc(len, GFP_KERNEL);
> > + if (!frag_container->fragment) {
> > + kfree(frag_container);
> > + return -ENOMEM;
> > + }
> > +
> > + len = vsnprintf(frag_container->fragment, len, fmt, args);
> > + spin_lock_irqsave(&stream->lock, flags);
> > + stream->length += len;
> > + list_add_tail(&frag_container->node, &stream->fragments);
> > + spin_unlock_irqrestore(&stream->lock, flags);
> > +
> > + return 0;
> > +}
> > +
> [...]
> > +
> > +bool string_stream_is_empty(struct string_stream *stream)
> > +{
> > + bool is_empty;
> > + unsigned long flags;
> > +
> > + spin_lock_irqsave(&stream->lock, flags);
>
> I'm not sure what benefit grabbing the lock is having here. If the list
> isn't empty after this is called then the race isn't resolved by
> grabbing and releasing the lock. The function is returning stale data in
> that case.
Good point, I didn't realize list_empty was protected by READ_ONCE. Will fix.
> > + is_empty = list_empty(&stream->fragments);
> > + spin_unlock_irqrestore(&stream->lock, flags);
> > +
> > + return is_empty;
> > +}
> > +
> > +static int string_stream_init(struct kunit_resource *res, void *context)
> > +{
> > + struct string_stream *stream;
> > +
> > + stream = kzalloc(sizeof(*stream), GFP_KERNEL);
> > + if (!stream)
> > + return -ENOMEM;
> > +
> > + res->allocation = stream;
> > + INIT_LIST_HEAD(&stream->fragments);
> > + spin_lock_init(&stream->lock);
> > +
> > + return 0;
> > +}
> > +
> > +static void string_stream_free(struct kunit_resource *res)
> > +{
> > + struct string_stream *stream = res->allocation;
> > +
> > + string_stream_clear(stream);
> > + kfree(stream);
> > +}
> > +
> > +struct string_stream *alloc_string_stream(struct kunit *test)
> > +{
> > + struct kunit_resource *res;
> > +
> > + res = kunit_alloc_resource(test,
> > + string_stream_init,
> > + string_stream_free,
> > + NULL);
> > +
> > + if (!res)
> > + return NULL;
> > +
> > + return res->allocation;
>
> Maybe kunit_alloc_resource() should just return res->allocation, or
> NULL, so that these functions can be simplified to 'return
> kunit_alloc_resource()'? Does the caller ever care to do anything with
> struct kunit_resource anyway?
Another good point. I think originally I thought it might, but now
with the mandatory init function, the user has to provide a function
where they can do the init work. They might as well do it there. Will
fix.
next prev parent reply other threads:[~2019-07-15 21:12 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-12 8:17 [PATCH v9 00/18] kunit: introduce KUnit, the Linux kernel unit testing framework Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 01/18] kunit: test: add KUnit test runner core Brendan Higgins
2019-07-15 20:10 ` Stephen Boyd
2019-07-15 21:25 ` Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 02/18] kunit: test: add test resource management API Brendan Higgins
2019-07-15 20:24 ` Stephen Boyd
2019-07-15 20:30 ` Brendan Higgins
2019-07-15 20:51 ` Stephen Boyd
2019-07-12 8:17 ` [PATCH v9 03/18] kunit: test: add string_stream a std::stream like string builder Brendan Higgins
2019-07-15 20:43 ` Stephen Boyd
2019-07-15 21:11 ` Brendan Higgins [this message]
2019-07-15 22:04 ` Stephen Boyd
2019-07-15 22:11 ` Brendan Higgins
2019-07-15 22:43 ` Brendan Higgins
2019-07-16 15:33 ` Stephen Boyd
2019-07-16 18:55 ` Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 04/18] kunit: test: add kunit_stream a std::stream like logger Brendan Higgins
2019-07-15 22:15 ` Stephen Boyd
2019-07-16 7:57 ` Brendan Higgins
2019-07-16 8:37 ` Brendan Higgins
2019-07-16 15:30 ` Stephen Boyd
2019-07-16 17:51 ` Brendan Higgins
2019-07-16 17:50 ` Stephen Boyd
2019-07-16 18:52 ` Brendan Higgins
2019-07-18 17:50 ` Stephen Boyd
2019-07-18 19:22 ` Brendan Higgins
2019-07-19 0:08 ` Brendan Higgins
2019-07-22 18:10 ` Brendan Higgins
2019-07-22 20:03 ` Stephen Boyd
2019-07-22 22:30 ` Brendan Higgins
2019-07-22 23:54 ` Stephen Boyd
2019-07-23 0:32 ` Brendan Higgins
2019-07-24 7:31 ` Petr Mladek
2019-07-25 20:21 ` Brendan Higgins
2019-07-26 8:31 ` Petr Mladek
2019-08-01 18:55 ` Brendan Higgins
2019-08-01 18:59 ` Brendan Higgins
2019-08-01 21:14 ` Stephen Boyd
2019-08-01 21:43 ` Brendan Higgins
2019-08-12 20:41 ` Brendan Higgins
2019-08-02 7:37 ` John Ogness
2019-08-12 21:12 ` Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 05/18] kunit: test: add the concept of expectations Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 06/18] kbuild: enable building KUnit Brendan Higgins
2019-07-15 20:49 ` Stephen Boyd
2019-07-12 8:17 ` [PATCH v9 07/18] kunit: test: add initial tests Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 08/18] objtool: add kunit_try_catch_throw to the noreturn list Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 09/18] kunit: test: add support for test abort Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 10/18] kunit: test: add tests for kunit " Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 11/18] kunit: test: add the concept of assertions Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 12/18] kunit: test: add tests for KUnit managed resources Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 13/18] kunit: tool: add Python wrappers for running KUnit tests Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 14/18] kunit: defconfig: add defconfigs for building " Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 15/18] Documentation: kunit: add documentation for KUnit Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 16/18] MAINTAINERS: add entry for KUnit the unit testing framework Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 17/18] kernel/sysctl-test: Add null pointer test for sysctl.c:proc_dointvec() Brendan Higgins
2019-07-12 8:17 ` [PATCH v9 18/18] MAINTAINERS: add proc sysctl KUnit test to PROC SYSCTL section Brendan Higgins
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAFd5g47481sRaez=yEJN4_ghiXZbxayk1Y04tAZpuzPLsmnhKg@mail.gmail.com' \
--to=brendanhiggins@google.com \
--cc=Alexander.Levin@microsoft.com \
--cc=Tim.Bird@sony.com \
--cc=amir73il@gmail.com \
--cc=dan.carpenter@oracle.com \
--cc=daniel@ffwll.ch \
--cc=devicetree@vger.kernel.org \
--cc=dri-devel@lists.freedesktop.org \
--cc=frowand.list@gmail.com \
--cc=gregkh@linuxfoundation.org \
--cc=jdike@addtoit.com \
--cc=joel@jms.id.au \
--cc=jpoimboe@redhat.com \
--cc=julia.lawall@lip6.fr \
--cc=keescook@google.com \
--cc=khilman@baylibre.com \
--cc=kieran.bingham@ideasonboard.com \
--cc=knut.omang@oracle.com \
--cc=kunit-dev@googlegroups.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-nvdimm@lists.01.org \
--cc=linux-um@lists.infradead.org \
--cc=logang@deltatee.com \
--cc=mcgrof@kernel.org \
--cc=mpe@ellerman.id.au \
--cc=peterz@infradead.org \
--cc=pmladek@suse.com \
--cc=rdunlap@infradead.org \
--cc=richard@nod.at \
--cc=rientjes@google.com \
--cc=robh@kernel.org \
--cc=rostedt@goodmis.org \
--cc=sboyd@kernel.org \
--cc=shuah@kernel.org \
--cc=tytso@mit.edu \
--cc=wfg@linux.intel.com \
--cc=yamada.masahiro@socionext.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).