[PATCH v4 0/2] Ensure that a request queue is dissociated from the cgroup controller

[PATCH v4 0/2] Ensure that a request queue is dissociated from the cgroup controller

[Bart Van Assche]

Hello Jens,

Several block drivers call alloc_disk() followed by put_disk() if something
fails before device_add_disk() is called without calling blk_cleanup_queue().
Make sure that also for this scenario a request queue is dissociated from the
cgroup controller. This patch avoids that loading the parport_pc, paride and
pf drivers trigger a kernel crash. Since this patch series fixes a regression,
please consider these patches for kernel v4.19.

Thanks,

Bart.

Changes between v3 and v4:
- Added "Cc: stable" tags.

Changes between v2 and v3:
- Avoid code duplication by introducing a new helper function.

Changes between v1 and v2:
- Fixed the build for CONFIG_BLK_CGROUP=n.

Bart Van Assche (2):
  block: Introduce blk_exit_queue()
  block: Ensure that a request queue is dissociated from the cgroup
    controller

 block/blk-core.c  | 54 ++++++++++++++++++++++++++---------------------
 block/blk-sysfs.c | 25 ++++++++++++++++++++++
 block/blk.h       |  1 +
 3 files changed, 56 insertions(+), 24 deletions(-)

-- 
2.18.0

[PATCH v4 1/2] block: Introduce blk_exit_queue()

[Bart Van Assche]

This patch does not change any functionality.

Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Ming Lei <ming.lei@redhat.com>
Cc: Omar Sandoval <osandov@fb.com>
Cc: Johannes Thumshirn <jthumshirn@suse.de>
Cc: Alexandru Moise <00moses.alexander00@gmail.com>
Cc: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: <stable@vger.kernel.org>
---
 block/blk-core.c | 54 +++++++++++++++++++++++++++---------------------
 block/blk.h      |  1 +
 2 files changed, 31 insertions(+), 24 deletions(-)

diff --git a/block/blk-core.c b/block/blk-core.c
index 7c65e297a4f1..5fbfa8484c95 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -719,6 +719,35 @@ void blk_set_queue_dying(struct request_queue *q)
 }
 EXPORT_SYMBOL_GPL(blk_set_queue_dying);
 
+/* Unconfigure the I/O scheduler and dissociate from the cgroup controller. */
+void blk_exit_queue(struct request_queue *q)
+{
+	/*
+	 * Since the I/O scheduler exit code may access cgroup information,
+	 * perform I/O scheduler exit before disassociating from the block
+	 * cgroup controller.
+	 */
+	if (q->elevator) {
+		ioc_clear_queue(q);
+		elevator_exit(q, q->elevator);
+		q->elevator = NULL;
+	}
+
+	/*
+	 * Remove all references to @q from the block cgroup controller before
+	 * restoring @q->queue_lock to avoid that restoring this pointer causes
+	 * e.g. blkcg_print_blkgs() to crash.
+	 */
+	blkcg_exit_queue(q);
+
+	/*
+	 * Since the cgroup code may dereference the @q->backing_dev_info
+	 * pointer, only decrease its reference count after having removed the
+	 * association with the block cgroup controller.
+	 */
+	bdi_put(q->backing_dev_info);
+}
+
 /**
  * blk_cleanup_queue - shutdown a request queue
  * @q: request queue to shutdown
@@ -788,30 +817,7 @@ void blk_cleanup_queue(struct request_queue *q)
 	 */
 	WARN_ON_ONCE(q->kobj.state_in_sysfs);
 
-	/*
-	 * Since the I/O scheduler exit code may access cgroup information,
-	 * perform I/O scheduler exit before disassociating from the block
-	 * cgroup controller.
-	 */
-	if (q->elevator) {
-		ioc_clear_queue(q);
-		elevator_exit(q, q->elevator);
-		q->elevator = NULL;
-	}
-
-	/*
-	 * Remove all references to @q from the block cgroup controller before
-	 * restoring @q->queue_lock to avoid that restoring this pointer causes
-	 * e.g. blkcg_print_blkgs() to crash.
-	 */
-	blkcg_exit_queue(q);
-
-	/*
-	 * Since the cgroup code may dereference the @q->backing_dev_info
-	 * pointer, only decrease its reference count after having removed the
-	 * association with the block cgroup controller.
-	 */
-	bdi_put(q->backing_dev_info);
+	blk_exit_queue(q);
 
 	if (q->mq_ops)
 		blk_mq_free_queue(q);
diff --git a/block/blk.h b/block/blk.h
index 6adae8f94279..aeb8026f4d7b 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -130,6 +130,7 @@ void blk_free_flush_queue(struct blk_flush_queue *q);
 int blk_init_rl(struct request_list *rl, struct request_queue *q,
 		gfp_t gfp_mask);
 void blk_exit_rl(struct request_queue *q, struct request_list *rl);
+void blk_exit_queue(struct request_queue *q);
 void blk_rq_bio_prep(struct request_queue *q, struct request *rq,
 			struct bio *bio);
 void blk_queue_bypass_start(struct request_queue *q);
-- 
2.18.0

[PATCH v4 2/2] block: Ensure that a request queue is dissociated from the cgroup controller

[Bart Van Assche]

Several block drivers call alloc_disk() followed by put_disk() if
something fails before device_add_disk() is called without calling
blk_cleanup_queue(). Make sure that also for this scenario a request
queue is dissociated from the cgroup controller. This patch avoids
that loading the parport_pc, paride and pf drivers triggers the
following kernel crash:

BUG: KASAN: null-ptr-deref in pi_init+0x42e/0x580 [paride]
Read of size 4 at addr 0000000000000008 by task modprobe/744
Call Trace:
dump_stack+0x9a/0xeb
kasan_report+0x139/0x350
pi_init+0x42e/0x580 [paride]
pf_init+0x2bb/0x1000 [pf]
do_one_initcall+0x8e/0x405
do_init_module+0xd9/0x2f2
load_module+0x3ab4/0x4700
SYSC_finit_module+0x176/0x1a0
do_syscall_64+0xee/0x2b0
entry_SYSCALL_64_after_hwframe+0x42/0xb7

Reported-by: Alexandru Moise <00moses.alexander00@gmail.com>
Fixes: a063057d7c73 ("block: Fix a race between request queue removal and the block cgroup controller") # v4.17
Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com>
Tested-by: Alexandru Moise <00moses.alexander00@gmail.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Ming Lei <ming.lei@redhat.com>
Cc: Johannes Thumshirn <jthumshirn@suse.de>
Cc: Alexandru Moise <00moses.alexander00@gmail.com>
Cc: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: <stable@vger.kernel.org>
---
 block/blk-sysfs.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index ca1984ecbdeb..26275d9babcb 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -802,6 +802,31 @@ static void __blk_release_queue(struct work_struct *work)
 		blk_stat_remove_callback(q, q->poll_cb);
 	blk_stat_free_callback(q->poll_cb);
 
+	if (!blk_queue_dead(q)) {
+		/*
+		 * Last reference was dropped without having called
+		 * blk_cleanup_queue().
+		 */
+		WARN_ONCE(blk_queue_init_done(q),
+			  "request queue %p has been registered but blk_cleanup_queue() has not been called for that queue\n",
+			  q);
+		blk_exit_queue(q);
+	}
+
+#ifdef CONFIG_BLK_CGROUP
+	{
+		struct blkcg_gq *blkg;
+
+		rcu_read_lock();
+		blkg = blkg_lookup(&blkcg_root, q);
+		rcu_read_unlock();
+
+		WARN(blkg,
+		     "request queue %p is being released but it has not yet been removed from the blkcg controller\n",
+		     q);
+	}
+#endif
+
 	blk_free_queue_stats(q->stats);
 
 	blk_exit_rl(q, &q->root_rl);
-- 
2.18.0

Re: [PATCH v4 0/2] Ensure that a request queue is dissociated from the cgroup controller

[Bart Van Assche]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-7", Size: 752 bytes --]

On Mon, 2018-07-30 at 14:10 -0700, Bart Van Assche wrote:
+AD4- Several block drivers call alloc+AF8-disk() followed by put+AF8-disk(=
) if something
+AD4- fails before device+AF8-add+AF8-disk() is called without calling blk+=
AF8-cleanup+AF8-queue().
+AD4- Make sure that also for this scenario a request queue is dissociated =
from the
+AD4- cgroup controller. This patch avoids that loading the parport+AF8-pc,=
 paride and
+AD4- pf drivers trigger a kernel crash. Since this patch series fixes a re=
gression,
+AD4- please consider these patches for kernel v4.19.

+AFs- ... +AF0-

Can anyone help with reviewing this patch series? This patch series is seve=
ral
months old and fixes a regression that was reported by a Linux user.

Thanks,

Bart.

Re: [PATCH v4 1/2] block: Introduce blk_exit_queue()

[Johannes Thumshirn]

Looks good,
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
-- 
Johannes Thumshirn                                          Storage
jthumshirn@suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N�rnberg
GF: Felix Imend�rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N�rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

Re: [PATCH v4 2/2] block: Ensure that a request queue is dissociated from the cgroup controller

[Johannes Thumshirn]

>From my limited insight into this:
Looks good,
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
-- 
Johannes Thumshirn                                          Storage
jthumshirn@suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N�rnberg
GF: Felix Imend�rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N�rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

Re: [PATCH v4 2/2] block: Ensure that a request queue is dissociated from the cgroup controller

[Jens Axboe]

On 7/30/18 3:10 PM, Bart Van Assche wrote:
> Several block drivers call alloc_disk() followed by put_disk() if
> something fails before device_add_disk() is called without calling
> blk_cleanup_queue(). Make sure that also for this scenario a request
> queue is dissociated from the cgroup controller. This patch avoids
> that loading the parport_pc, paride and pf drivers triggers the
> following kernel crash:
> 
> BUG: KASAN: null-ptr-deref in pi_init+0x42e/0x580 [paride]
> Read of size 4 at addr 0000000000000008 by task modprobe/744
> Call Trace:
> dump_stack+0x9a/0xeb
> kasan_report+0x139/0x350
> pi_init+0x42e/0x580 [paride]
> pf_init+0x2bb/0x1000 [pf]
> do_one_initcall+0x8e/0x405
> do_init_module+0xd9/0x2f2
> load_module+0x3ab4/0x4700
> SYSC_finit_module+0x176/0x1a0
> do_syscall_64+0xee/0x2b0
> entry_SYSCALL_64_after_hwframe+0x42/0xb7
> 
> Reported-by: Alexandru Moise <00moses.alexander00@gmail.com>
> Fixes: a063057d7c73 ("block: Fix a race between request queue removal and the block cgroup controller") # v4.17
> Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com>
> Tested-by: Alexandru Moise <00moses.alexander00@gmail.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Ming Lei <ming.lei@redhat.com>
> Cc: Johannes Thumshirn <jthumshirn@suse.de>
> Cc: Alexandru Moise <00moses.alexander00@gmail.com>
> Cc: Joseph Qi <joseph.qi@linux.alibaba.com>
> Cc: <stable@vger.kernel.org>
> ---
>  block/blk-sysfs.c | 25 +++++++++++++++++++++++++
>  1 file changed, 25 insertions(+)
> 
> diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
> index ca1984ecbdeb..26275d9babcb 100644
> --- a/block/blk-sysfs.c
> +++ b/block/blk-sysfs.c
> @@ -802,6 +802,31 @@ static void __blk_release_queue(struct work_struct *work)
>  		blk_stat_remove_callback(q, q->poll_cb);
>  	blk_stat_free_callback(q->poll_cb);
>  
> +	if (!blk_queue_dead(q)) {
> +		/*
> +		 * Last reference was dropped without having called
> +		 * blk_cleanup_queue().
> +		 */
> +		WARN_ONCE(blk_queue_init_done(q),
> +			  "request queue %p has been registered but blk_cleanup_queue() has not been called for that queue\n",
> +			  q);
> +		blk_exit_queue(q);
> +	}
> +
> +#ifdef CONFIG_BLK_CGROUP
> +	{
> +		struct blkcg_gq *blkg;
> +
> +		rcu_read_lock();
> +		blkg = blkg_lookup(&blkcg_root, q);
> +		rcu_read_unlock();
> +
> +		WARN(blkg,
> +		     "request queue %p is being released but it has not yet been removed from the blkcg controller\n",
> +		     q);
> +	}
> +#endif

This last hunk should go in the cgroup code.

-- 
Jens Axboe

Re: [PATCH v4 2/2] block: Ensure that a request queue is dissociated from the cgroup controller

[Bart Van Assche]

On Wed, 2018-08-08 at 08:41 -0600, Jens Axboe wrote:
> On 7/30/18 3:10 PM, Bart Van Assche wrote:
> > +#ifdef CONFIG_BLK_CGROUP
> > +	{
> > +		struct blkcg_gq *blkg;
> > +
> > +		rcu_read_lock();
> > +		blkg = blkg_lookup(&blkcg_root, q);
> > +		rcu_read_unlock();
> > +
> > +		WARN(blkg,
> > +		     "request queue %p is being released but it has=
 not yet been removed from the blkcg controller\n",
> > +		     q);
> > +	}
> > +#endif
>=20
> This last hunk should go in the cgroup code.

Hello Jens,

How about leaving out the #ifdef CONFIG_BLK_CGROUP / #endif=
? Would that be
sufficient? I just noticed that blkg_lookup() returns NULL anyway if cg=
roup support
is disabled in the kernel config.

Thanks,

Bart.

Re: [PATCH v4 2/2] block: Ensure that a request queue is dissociated from the cgroup controller

[Bart Van Assche]

On Wed, 2018-08-08 at 08:41 -0600, Jens Axboe wrote:
> On 7/30/18 3:10 PM, Bart Van Assche wrote:
> > +#ifdef CONFIG_BLK_CGROUP
> > +	{
> > +		struct blkcg_gq *blkg;
> > +
> > +		rcu_read_lock();
> > +		blkg = blkg_lookup(&blkcg_root, q);
> > +		rcu_read_unlock();
> > +
> > +		WARN(blkg,
> > +		     "request queue %p is being released but it has not yet been removed from the blkcg controller\n",
> > +		     q);
> > +	}
> > +#endif
> 
> This last hunk should go in the cgroup code.

Hello Jens,

How about leaving out the #ifdef CONFIG_BLK_CGROUP / #endif? Would that be
sufficient? I just noticed that blkg_lookup() returns NULL anyway if cgroup support
is disabled in the kernel config.

Thanks,

Bart.

Re: [PATCH v4 2/2] block: Ensure that a request queue is dissociated from the cgroup controller

[Jens Axboe]

On 8/8/18 9:04 AM, Bart Van Assche wrote:
> On Wed, 2018-08-08 at 08:41 -0600, Jens Axboe wrote:
>> On 7/30/18 3:10 PM, Bart Van Assche wrote:
>>> +#ifdef CONFIG_BLK_CGROUP
>>> +	{
>>> +		struct blkcg_gq *blkg;
>>> +
>>> +		rcu_read_lock();
>>> +		blkg = blkg_lookup(&blkcg_root, q);
>>> +		rcu_read_unlock();
>>> +
>>> +		WARN(blkg,
>>> +		     "request queue %p is being released but it has not yet been removed from the blkcg controller\n",
>>> +		     q);
>>> +	}
>>> +#endif
>>
>> This last hunk should go in the cgroup code.
> 
> Hello Jens,
> 
> How about leaving out the #ifdef CONFIG_BLK_CGROUP / #endif? Would that be
> sufficient? I just noticed that blkg_lookup() returns NULL anyway if cgroup support
> is disabled in the kernel config.

That's even better.

-- 
Jens Axboe