Re: [PATCH] block: make sure big bio is splitted into at most 256 bvecs

From: Kent Overstreet <kent.overstreet@gmail.com>
To: Shaohua Li <shli@fb.com>, Jeff Moyer <jmoyer@redhat.com>
Cc: Ming Lei <ming.lei@canonical.com>, Jens Axboe <axboe@fb.com>,
	linux-kernel@vger.kernel.org, linux-block@vger.kernel.org,
	Christoph Hellwig <hch@infradead.org>,
	Eric Wheeler <bcache@lists.ewheeler.net>,
	Sebastian Roesner <sroesner-kernelorg@roesner-online.de>,
	"4.2+" <stable@vger.kernel.org>
Subject: Re: [PATCH] block: make sure big bio is splitted into at most 256 bvecs
Date: Tue, 5 Apr 2016 16:27:33 -0800	[thread overview]
Message-ID: <20160406002733.GB31161@kmo-pixel> (raw)
In-Reply-To: <20160405182720.GA2375685@devbig084.prn1.facebook.com>

On Tue, Apr 05, 2016 at 11:27:21AM -0700, Shaohua Li wrote:
> On Wed, Apr 06, 2016 at 01:44:06AM +0800, Ming Lei wrote:
> > After arbitrary bio size is supported, the incoming bio may
> > be very big. We have to split the bio into small bios so that
> > each holds at most BIO_MAX_PAGES bvecs for safety reason, such
> > as bio_clone().
> > 
> > This patch fixes the following kernel crash:
> > 
> > > [  172.660142] BUG: unable to handle kernel NULL pointer dereference at
> > > 0000000000000028
> > > [  172.660229] IP: [<ffffffff811e53b4>] bio_trim+0xf/0x2a
> > > [  172.660289] PGD 7faf3e067 PUD 7f9279067 PMD 0
> > > [  172.660399] Oops: 0000 [#1] SMP
> > > [...]
> > > [  172.664780] Call Trace:
> > > [  172.664813]  [<ffffffffa007f3be>] ? raid1_make_request+0x2e8/0xad7 [raid1]
> > > [  172.664846]  [<ffffffff811f07da>] ? blk_queue_split+0x377/0x3d4
> > > [  172.664880]  [<ffffffffa005fb5f>] ? md_make_request+0xf6/0x1e9 [md_mod]
> > > [  172.664912]  [<ffffffff811eb860>] ? generic_make_request+0xb5/0x155
> > > [  172.664947]  [<ffffffffa0445c89>] ? prio_io+0x85/0x95 [bcache]
> > > [  172.664981]  [<ffffffffa0448252>] ? register_cache_set+0x355/0x8d0 [bcache]
> > > [  172.665016]  [<ffffffffa04497d3>] ? register_bcache+0x1006/0x1174 [bcache]
> > 
> > Fixes: 54efd50(block: make generic_make_request handle arbitrarily sized bios)
> 
> this bug is introduced by d2be537c3ba
> > Reported-by: Sebastian Roesner <sroesner-kernelorg@roesner-online.de>
> > Reported-by: Eric Wheeler <bcache@lists.ewheeler.net>
> > Cc: stable@vger.kernel.org (4.2+)
> > Cc: Shaohua Li <shli@fb.com>
> > Signed-off-by: Ming Lei <ming.lei@canonical.com>
> > ---
> > I can reproduce the issue and verify the fix by the following approach:
> > 	- create one raid1 over two virtio-blk 
> > 	- build bcache device over the above raid1 and another cache device.
> > 	- set cache mode as writeback
> > 	- run random write over ext4 on the bcache device
> > 	- then the crash can be triggered
> 
> can you explain why this is better than my original patch?

Shaohua, what was your original patch? I'm sorry, I know I saw it at one point
but I can't remember what it was.

I didn't see Jeff's patch that introduced this bug until your email just now.
Argh.

Jeff, "block: bump BLK_DEF_MAX_SECTORS to 2560" doesn't make much sense as far as
I can tell without changing the BIO_MAX_PAGES too - that's probably why you
weren't seeing much performance increase from that patch...

But BLK_DEF_MAX_SECTORS should not have been enforcing the BIO_MAX_PAGES limit
so that patch was not at fault.