From: Song Liu <song@kernel.org>
To: Donald Buczek <buczek@molgen.mpg.de>
Cc: Logan Gunthorpe <logang@deltatee.com>,
open list <linux-kernel@vger.kernel.org>,
linux-raid <linux-raid@vger.kernel.org>,
Christoph Hellwig <hch@infradead.org>,
Guoqing Jiang <guoqing.jiang@linux.dev>, Xiao Ni <xni@redhat.com>,
Stephen Bates <sbates@raithlin.com>,
Martin Oliveira <Martin.Oliveira@eideticom.com>,
David Sloan <David.Sloan@eideticom.com>
Subject: Re: [PATCH v1 12/15] md/raid5-cache: Add RCU protection to conf->log accesses
Date: Sun, 22 May 2022 23:47:41 -0700 [thread overview]
Message-ID: <CAPhsuW5FKh7VKgU+=eU4_ad4btFMwU7ymMNMZ5TZVQbv82MaHg@mail.gmail.com> (raw)
In-Reply-To: <62b09487-9223-db3d-2165-789a51230060@molgen.mpg.de>
On Sun, May 22, 2022 at 12:32 AM Donald Buczek <buczek@molgen.mpg.de> wrote:
>
> On 19.05.22 21:13, Logan Gunthorpe wrote:
> > The mdadm test 21raid5cache randomly fails with NULL pointer accesses
> > conf->log when run repeatedly. conf->log was sort of protected with
> > a RCU, but most dereferences were not done with the correct functions.
> >
> > Add rcu_read_locks() and rcu_access_pointers() to the appropriate
> > places.
> >
> > Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
[...]
> > diff --git a/drivers/md/raid5-log.h b/drivers/md/raid5-log.h
> > index f26e6f4c7f9a..24b4dbd5b25c 100644
> > --- a/drivers/md/raid5-log.h
> > +++ b/drivers/md/raid5-log.h
> > @@ -58,7 +58,7 @@ static inline int log_stripe(struct stripe_head *sh, struct stripe_head_state *s
> > {
> > struct r5conf *conf = sh->raid_conf;
> >
> > - if (conf->log) {
> > + if (rcu_access_pointer(conf->log)) {
>
>
> A problem here is that `struct r5l_log` of `conf->log` is private to raid5-cache.c and gcc below version 10 (wrongly) regards the `typeof(*p) *local` declaration of __rcu_access_pointer as a dereference:
>
> CC drivers/md/raid5.o
>
> In file included from ./include/linux/rculist.h:11:0,
>
> from ./include/linux/dcache.h:8,
>
> from ./include/linux/fs.h:8,
>
> from ./include/linux/highmem.h:5,
>
> from ./include/linux/bvec.h:10,
>
> from ./include/linux/blk_types.h:10,
>
> from ./include/linux/blkdev.h:9,
>
> from drivers/md/raid5.c:38:
>
> drivers/md/raid5-log.h: In function ‘log_stripe’:
>
> ./include/linux/rcupdate.h:384:9: error: dereferencing pointer to incomplete type ‘struct r5l_log’
>
> typeof(*p) *local = (typeof(*p) *__force)READ_ONCE(p); \
>
> ^
>
> ./include/linux/rcupdate.h:495:31: note: in expansion of macro ‘__rcu_access_pointer’
>
> #define rcu_access_pointer(p) __rcu_access_pointer((p), __UNIQUE_ID(rcu), __rcu)
>
> ^~~~~~~~~~~~~~~~~~~~
>
> drivers/md/raid5-log.h:61:6: note: in expansion of macro ‘rcu_access_pointer’
>
> if (rcu_access_pointer(conf->log)) {
>
> ^~~~~~~~~~~~~~~~~~
>
> make[2]: *** [scripts/Makefile.build:288: drivers/md/raid5.o] Error 1
>
> make[1]: *** [scripts/Makefile.build:550: drivers/md] Error 2
>
> make: *** [Makefile:1834: drivers] Error 2
This is annoying.. And there are a few other cases in raid5-log.h and
raid5.c.
Maybe we should move the definition of r5l_log to raid5-log.h?
Thanks,
Song
>
>
> See https://godbolt.org/z/TPP8MdKbc to test compiler versions with this construct.
>
> Best
>
> Donald
>
>
> > if (!test_bit(STRIPE_R5C_CACHING, &sh->state)) {
> > /* writing out phase */
> > if (s->waiting_extra_page)
> > @@ -79,7 +79,7 @@ static inline void log_stripe_write_finished(struct stripe_head *sh)
> > {
> > struct r5conf *conf = sh->raid_conf;
> >
> > - if (conf->log)
> > + if (rcu_access_pointer(conf->log))
> > r5l_stripe_write_finished(sh);
> > else if (raid5_has_ppl(conf))
> > ppl_stripe_write_finished(sh);
> > @@ -87,7 +87,7 @@ static inline void log_stripe_write_finished(struct stripe_head *sh)
> >
> > static inline void log_write_stripe_run(struct r5conf *conf)
> > {
> > - if (conf->log)
> > + if (rcu_access_pointer(conf->log))
> > r5l_write_stripe_run(conf);
> > else if (raid5_has_ppl(conf))
> > ppl_write_stripe_run(conf);
> > @@ -95,7 +95,7 @@ static inline void log_write_stripe_run(struct r5conf *conf)
> >
> > static inline void log_flush_stripe_to_raid(struct r5conf *conf)
> > {
> > - if (conf->log)
> > + if (rcu_access_pointer(conf->log))
> > r5l_flush_stripe_to_raid(conf);
> > else if (raid5_has_ppl(conf))
> > ppl_write_stripe_run(conf);
> > @@ -105,7 +105,7 @@ static inline int log_handle_flush_request(struct r5conf *conf, struct bio *bio)
> > {
> > int ret = -ENODEV;
> >
> > - if (conf->log)
> > + if (rcu_access_pointer(conf->log))
> > ret = r5l_handle_flush_request(conf, bio);
> > else if (raid5_has_ppl(conf))
> > ret = ppl_handle_flush_request(bio);
> > @@ -115,7 +115,7 @@ static inline int log_handle_flush_request(struct r5conf *conf, struct bio *bio)
> >
> > static inline void log_quiesce(struct r5conf *conf, int quiesce)
> > {
> > - if (conf->log)
> > + if (rcu_access_pointer(conf->log))
> > r5l_quiesce(conf, quiesce);
> > else if (raid5_has_ppl(conf))
> > ppl_quiesce(conf, quiesce);
> > @@ -123,7 +123,7 @@ static inline void log_quiesce(struct r5conf *conf, int quiesce)
> >
> > static inline void log_exit(struct r5conf *conf)
> > {
> > - if (conf->log)
> > + if (rcu_access_pointer(conf->log))
> > r5l_exit_log(conf);
> > else if (raid5_has_ppl(conf))
> > ppl_exit_log(conf);
> > diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
[...]
next prev parent reply other threads:[~2022-05-23 7:29 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-19 19:12 [PATCH v1 00/15] Bug fixes for mdadm tests Logan Gunthorpe
2022-05-19 19:12 ` [PATCH v1 01/15] md/raid5-log: Drop extern decorators for function prototypes Logan Gunthorpe
2022-05-21 11:36 ` Christoph Hellwig
2022-05-19 19:12 ` [PATCH v1 02/15] md/raid5-cache: Refactor r5c_is_writeback() to take a struct r5conf Logan Gunthorpe
2022-05-21 11:37 ` Christoph Hellwig
2022-05-19 19:12 ` [PATCH v1 03/15] md/raid5-cache: Refactor r5l_start() " Logan Gunthorpe
2022-05-21 11:37 ` Christoph Hellwig
2022-05-19 19:13 ` [PATCH v1 04/15] md/raid5-cache: Refactor r5l_flush_stripe_to_raid() " Logan Gunthorpe
2022-05-21 11:38 ` Christoph Hellwig
2022-05-19 19:13 ` [PATCH v1 05/15] md/raid5-cache: Refactor r5l_wake_reclaim() " Logan Gunthorpe
2022-05-21 11:38 ` Christoph Hellwig
2022-05-19 19:13 ` [PATCH v1 06/15] md/raid5-cache: Refactor remaining functions to take a r5conf Logan Gunthorpe
2022-05-21 11:40 ` Christoph Hellwig
2022-05-19 19:13 ` [PATCH v1 07/15] md/raid5-ppl: Drop unused argument from ppl_handle_flush_request() Logan Gunthorpe
2022-05-21 11:41 ` Christoph Hellwig
2022-05-19 19:13 ` [PATCH v1 08/15] md/raid5-cache: Pass the log through to r5c_finish_cache_stripe() Logan Gunthorpe
2022-05-21 11:42 ` Christoph Hellwig
2022-05-19 19:13 ` [PATCH v1 09/15] md/raid5-cache: Don't pass conf to r5c_calculate_new_cp() Logan Gunthorpe
2022-05-21 11:42 ` Christoph Hellwig
2022-05-19 19:13 ` [PATCH v1 10/15] md/raid5-cache: Take struct r5l_log in r5c_log_required_to_flush_cache() Logan Gunthorpe
2022-05-21 11:43 ` Christoph Hellwig
2022-05-19 19:13 ` [PATCH v1 11/15] md/raid5: Ensure array is suspended for calls to log_exit() Logan Gunthorpe
2022-05-21 11:44 ` Christoph Hellwig
2022-05-19 19:13 ` [PATCH v1 12/15] md/raid5-cache: Add RCU protection to conf->log accesses Logan Gunthorpe
2022-05-21 11:50 ` Christoph Hellwig
2022-05-22 7:31 ` Donald Buczek
2022-05-23 6:47 ` Song Liu [this message]
2022-05-23 18:15 ` Song Liu
2022-05-24 16:14 ` Logan Gunthorpe
2022-05-24 15:59 ` Logan Gunthorpe
2022-05-24 18:13 ` Song Liu
2022-05-22 7:32 ` Donald Buczek
2022-05-24 15:55 ` Logan Gunthorpe
2022-05-19 19:13 ` [PATCH v1 13/15] md/raid5-cache: Annotate pslot with __rcu notation Logan Gunthorpe
2022-05-21 11:51 ` Christoph Hellwig
2022-05-19 19:13 ` [PATCH v1 14/15] md: Ensure resync is reported after it starts Logan Gunthorpe
2022-05-21 11:51 ` Christoph Hellwig
2022-05-24 15:45 ` Logan Gunthorpe
2022-05-19 19:13 ` [PATCH v1 15/15] md: Notify sysfs sync_completed in md_reap_sync_thread() Logan Gunthorpe
2022-05-21 11:52 ` Christoph Hellwig
2022-05-23 6:28 ` [PATCH v1 00/15] Bug fixes for mdadm tests Song Liu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAPhsuW5FKh7VKgU+=eU4_ad4btFMwU7ymMNMZ5TZVQbv82MaHg@mail.gmail.com' \
--to=song@kernel.org \
--cc=David.Sloan@eideticom.com \
--cc=Martin.Oliveira@eideticom.com \
--cc=buczek@molgen.mpg.de \
--cc=guoqing.jiang@linux.dev \
--cc=hch@infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-raid@vger.kernel.org \
--cc=logang@deltatee.com \
--cc=sbates@raithlin.com \
--cc=xni@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).