* [RFC PATCH 0/1] nbd: fix crash when unmaping nbd device with fs still mounted
@ 2017-03-20 22:58 Ming Lin
2017-03-20 22:58 ` [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() Ming Lin
0 siblings, 1 reply; 6+ messages in thread
From: Ming Lin @ 2017-03-20 22:58 UTC (permalink / raw)
To: nbd-general, Josef Bacik, Ratna Manoj Bolla
Cc: linux-block, linux-kernel, jianshu.ljs, xiongwei.jiang,
james.liu, Markus Pargmann
Hi all,
I run into a BUG_ON(!buffer_mapped(bh)) crash with below script.
$ rbd-nbd map mypool/myimg
$ mkfs.ext4 /dev/nbd0
$ mount /dev/nbd0 /mnt/
$ rbd-nbd unmap /dev/nbd0
$ umount /mnt
[ 1248.870131] kernel BUG at /home/mlin/linux/fs/buffer.c:3103!
[ 1248.871214] invalid opcode: 0000 [#1] SMP
[ 1248.879468] CPU: 0 PID: 2450 Comm: umount Tainted: G E 4.11.0-rc2+ #2
[ 1248.896579] Call Trace:
[ 1248.897056] __sync_dirty_buffer+0x6e/0xe0
[ 1248.897870] ext4_commit_super+0x1eb/0x290 [ext4]
[ 1248.898795] ext4_put_super+0x2fa/0x3c0 [ext4]
[ 1248.899662] generic_shutdown_super+0x6f/0x100
[ 1248.900525] kill_block_super+0x27/0x70
[ 1248.901257] deactivate_locked_super+0x43/0x70
[ 1248.902112] deactivate_super+0x46/0x60
[ 1248.902869] cleanup_mnt+0x3f/0x80
[ 1248.903526] __cleanup_mnt+0x12/0x20
[ 1248.904218] task_work_run+0x83/0xb0
[ 1248.904941] exit_to_usermode_loop+0x59/0x7b
[ 1248.905769] do_syscall_64+0x165/0x180
[ 1248.907603] entry_SYSCALL64_slow_path+0x25/0x25
Last year, Ratna posted a patch to fix it.
https://lkml.org/lkml/2016/4/20/257
Ratna's script to reproduce the bug.
$ qemu-img create -f qcow2 f.img 1G
$ mkfs.ext4 f.img
$ qemu-nbd -c /dev/nbd0 f.img
$ mount /dev/nbd0 dir
$ killall -KILL qemu-nbd
$ sleep 1
$ ls dir
$ umount dir
I ported Rantna's patch to 4.11-rc2 and confirmed that it fixes the crash.
Jan Kara had some comments about this bug:
http://www.kernelhub.org/?p=2&msg=361407
I hope to fix this bug in the upstream kernel first and then back port it to
our production system.
Please see "PATCH 1/1" for detail.
Thanks,
Ming
^ permalink raw reply [flat|nested] 6+ messages in thread
* [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device()
2017-03-20 22:58 [RFC PATCH 0/1] nbd: fix crash when unmaping nbd device with fs still mounted Ming Lin
@ 2017-03-20 22:58 ` Ming Lin
2017-03-22 20:48 ` [Nbd] " Ming Lin
2017-03-23 17:51 ` Josef Bacik
0 siblings, 2 replies; 6+ messages in thread
From: Ming Lin @ 2017-03-20 22:58 UTC (permalink / raw)
To: nbd-general, Josef Bacik, Ratna Manoj Bolla
Cc: linux-block, linux-kernel, jianshu.ljs, xiongwei.jiang,
james.liu, Markus Pargmann
From: Ratna Manoj Bolla <manoj.br@gmail.com>
When a filesystem is mounted on a nbd device and on a disconnect, because
of kill_bdev(), and resetting bdev size to zero, buffer_head mappings are
getting destroyed under mounted filesystem.
After a bdev size reset(i.e bdev->bd_inode->i_size = 0) on a disconnect,
followed by a sys_umount(),
generic_shutdown_super()->...
->__sync_blockdev()->...
-blkdev_writepages()->...
->do_invalidatepage()->...
-discard_buffer() is discarding superblock buffer_head assumed
to be in mapped state by ext4_commit_super().
[mlin: ported to 4.11-rc2]
Signed-off-by: Ratna Manoj Bolla <manoj.br@gmail.com
---
drivers/block/nbd.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index cb4ccfc..a6a3643 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -125,7 +125,8 @@ static const char *nbdcmd_to_ascii(int cmd)
static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev)
{
- bd_set_size(bdev, 0);
+ if (bdev->bd_openers <= 1)
+ bd_set_size(bdev, 0);
set_capacity(nbd->disk, 0);
kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE);
@@ -603,6 +604,8 @@ static void nbd_reset(struct nbd_device *nbd)
static void nbd_bdev_reset(struct block_device *bdev)
{
+ if (bdev->bd_openers > 1)
+ return;
set_device_ro(bdev, false);
bdev->bd_inode->i_size = 0;
if (max_part > 0) {
@@ -666,7 +669,8 @@ static int nbd_clear_sock(struct nbd_device *nbd, struct block_device *bdev)
{
sock_shutdown(nbd);
nbd_clear_que(nbd);
- kill_bdev(bdev);
+
+ __invalidate_device(bdev, true);
nbd_bdev_reset(bdev);
/*
* We want to give the run thread a chance to wait for everybody
--
1.8.3.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [Nbd] [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device()
2017-03-20 22:58 ` [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() Ming Lin
@ 2017-03-22 20:48 ` Ming Lin
2017-03-22 21:43 ` Josef Bacik
2017-03-23 17:51 ` Josef Bacik
1 sibling, 1 reply; 6+ messages in thread
From: Ming Lin @ 2017-03-22 20:48 UTC (permalink / raw)
To: Josef Bacik
Cc: nbd-general, Ratna Manoj Bolla, lkml, linux-block, jianshu.ljs,
LIU, Fei, Markus Pargmann, xiongwei.jiang
On Mon, Mar 20, 2017 at 3:58 PM, Ming Lin <mlin@kernel.org> wrote:
> From: Ratna Manoj Bolla <manoj.br@gmail.com>
>
> When a filesystem is mounted on a nbd device and on a disconnect, because
> of kill_bdev(), and resetting bdev size to zero, buffer_head mappings are
> getting destroyed under mounted filesystem.
>
> After a bdev size reset(i.e bdev->bd_inode->i_size = 0) on a disconnect,
> followed by a sys_umount(),
> generic_shutdown_super()->...
> ->__sync_blockdev()->...
> -blkdev_writepages()->...
> ->do_invalidatepage()->...
> -discard_buffer() is discarding superblock buffer_head assumed
> to be in mapped state by ext4_commit_super().
>
> [mlin: ported to 4.11-rc2]
> Signed-off-by: Ratna Manoj Bolla <manoj.br@gmail.com
> ---
> drivers/block/nbd.c | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index cb4ccfc..a6a3643 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -125,7 +125,8 @@ static const char *nbdcmd_to_ascii(int cmd)
>
> static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev)
> {
> - bd_set_size(bdev, 0);
> + if (bdev->bd_openers <= 1)
> + bd_set_size(bdev, 0);
> set_capacity(nbd->disk, 0);
> kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE);
>
> @@ -603,6 +604,8 @@ static void nbd_reset(struct nbd_device *nbd)
>
> static void nbd_bdev_reset(struct block_device *bdev)
> {
> + if (bdev->bd_openers > 1)
> + return;
> set_device_ro(bdev, false);
> bdev->bd_inode->i_size = 0;
> if (max_part > 0) {
> @@ -666,7 +669,8 @@ static int nbd_clear_sock(struct nbd_device *nbd, struct block_device *bdev)
> {
> sock_shutdown(nbd);
> nbd_clear_que(nbd);
> - kill_bdev(bdev);
> +
> + __invalidate_device(bdev, true);
> nbd_bdev_reset(bdev);
> /*
> * We want to give the run thread a chance to wait for everybody
> --
> 1.8.3.1
Hi Josef,
Any comments?
Thanks,
Ming
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Nbd] [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device()
2017-03-22 20:48 ` [Nbd] " Ming Lin
@ 2017-03-22 21:43 ` Josef Bacik
0 siblings, 0 replies; 6+ messages in thread
From: Josef Bacik @ 2017-03-22 21:43 UTC (permalink / raw)
To: Ming Lin
Cc: nbd-general, Ratna Manoj Bolla, lkml, linux-block, jianshu.ljs,
LIU, Fei, Markus Pargmann, xiongwei.jiang
Hey sorry I just got back from LSF, I’ll look at this in the morning. Thanks,
Josef
On 3/22/17, 4:48 PM, "Ming Lin" <mlin@kernel.org> wrote:
On Mon, Mar 20, 2017 at 3:58 PM, Ming Lin <mlin@kernel.org> wrote:
> From: Ratna Manoj Bolla <manoj.br@gmail.com>
>
> When a filesystem is mounted on a nbd device and on a disconnect, because
> of kill_bdev(), and resetting bdev size to zero, buffer_head mappings are
> getting destroyed under mounted filesystem.
>
> After a bdev size reset(i.e bdev->bd_inode->i_size = 0) on a disconnect,
> followed by a sys_umount(),
> generic_shutdown_super()->...
> ->__sync_blockdev()->...
> -blkdev_writepages()->...
> ->do_invalidatepage()->...
> -discard_buffer() is discarding superblock buffer_head assumed
> to be in mapped state by ext4_commit_super().
>
> [mlin: ported to 4.11-rc2]
> Signed-off-by: Ratna Manoj Bolla <manoj.br@gmail.com
> ---
> drivers/block/nbd.c | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index cb4ccfc..a6a3643 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -125,7 +125,8 @@ static const char *nbdcmd_to_ascii(int cmd)
>
> static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev)
> {
> - bd_set_size(bdev, 0);
> + if (bdev->bd_openers <= 1)
> + bd_set_size(bdev, 0);
> set_capacity(nbd->disk, 0);
> kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE);
>
> @@ -603,6 +604,8 @@ static void nbd_reset(struct nbd_device *nbd)
>
> static void nbd_bdev_reset(struct block_device *bdev)
> {
> + if (bdev->bd_openers > 1)
> + return;
> set_device_ro(bdev, false);
> bdev->bd_inode->i_size = 0;
> if (max_part > 0) {
> @@ -666,7 +669,8 @@ static int nbd_clear_sock(struct nbd_device *nbd, struct block_device *bdev)
> {
> sock_shutdown(nbd);
> nbd_clear_que(nbd);
> - kill_bdev(bdev);
> +
> + __invalidate_device(bdev, true);
> nbd_bdev_reset(bdev);
> /*
> * We want to give the run thread a chance to wait for everybody
> --
> 1.8.3.1
Hi Josef,
Any comments?
Thanks,
Ming
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device()
2017-03-20 22:58 ` [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() Ming Lin
2017-03-22 20:48 ` [Nbd] " Ming Lin
@ 2017-03-23 17:51 ` Josef Bacik
2017-03-23 20:58 ` Ming Lin
1 sibling, 1 reply; 6+ messages in thread
From: Josef Bacik @ 2017-03-23 17:51 UTC (permalink / raw)
To: Ming Lin, nbd-general, Ratna Manoj Bolla
Cc: linux-block, linux-kernel, jianshu.ljs, xiongwei.jiang, james.liu
Yeah I think this is ok, I’ll throw it on my queue for fixes for this cycle. Thanks,
Josef
On 3/20/17, 6:58 PM, "Ming Lin" <mlin@kernel.org> wrote:
From: Ratna Manoj Bolla <manoj.br@gmail.com>
When a filesystem is mounted on a nbd device and on a disconnect, because
of kill_bdev(), and resetting bdev size to zero, buffer_head mappings are
getting destroyed under mounted filesystem.
After a bdev size reset(i.e bdev->bd_inode->i_size = 0) on a disconnect,
followed by a sys_umount(),
generic_shutdown_super()->...
->__sync_blockdev()->...
-blkdev_writepages()->...
->do_invalidatepage()->...
-discard_buffer() is discarding superblock buffer_head assumed
to be in mapped state by ext4_commit_super().
[mlin: ported to 4.11-rc2]
Signed-off-by: Ratna Manoj Bolla <manoj.br@gmail.com
---
drivers/block/nbd.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index cb4ccfc..a6a3643 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -125,7 +125,8 @@ static const char *nbdcmd_to_ascii(int cmd)
static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev)
{
- bd_set_size(bdev, 0);
+ if (bdev->bd_openers <= 1)
+ bd_set_size(bdev, 0);
set_capacity(nbd->disk, 0);
kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE);
@@ -603,6 +604,8 @@ static void nbd_reset(struct nbd_device *nbd)
static void nbd_bdev_reset(struct block_device *bdev)
{
+ if (bdev->bd_openers > 1)
+ return;
set_device_ro(bdev, false);
bdev->bd_inode->i_size = 0;
if (max_part > 0) {
@@ -666,7 +669,8 @@ static int nbd_clear_sock(struct nbd_device *nbd, struct block_device *bdev)
{
sock_shutdown(nbd);
nbd_clear_que(nbd);
- kill_bdev(bdev);
+
+ __invalidate_device(bdev, true);
nbd_bdev_reset(bdev);
/*
* We want to give the run thread a chance to wait for everybody
--
1.8.3.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device()
2017-03-23 17:51 ` Josef Bacik
@ 2017-03-23 20:58 ` Ming Lin
0 siblings, 0 replies; 6+ messages in thread
From: Ming Lin @ 2017-03-23 20:58 UTC (permalink / raw)
To: Josef Bacik
Cc: nbd-general, Ratna Manoj Bolla, linux-block, linux-kernel,
jianshu.ljs, xiongwei.jiang, james.liu
On Thu, Mar 23, 2017 at 10:51 AM, Josef Bacik <jbacik@fb.com> wrote:
> Yeah I think this is ok, I’ll throw it on my queue for fixes for this cycle. Thanks,
Great. Thanks.
>
> Josef
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-03-23 20:59 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-20 22:58 [RFC PATCH 0/1] nbd: fix crash when unmaping nbd device with fs still mounted Ming Lin
2017-03-20 22:58 ` [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() Ming Lin
2017-03-22 20:48 ` [Nbd] " Ming Lin
2017-03-22 21:43 ` Josef Bacik
2017-03-23 17:51 ` Josef Bacik
2017-03-23 20:58 ` Ming Lin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).