[PATCH] pmem: fix the crash when unbind namespaces

[PATCH] pmem: fix the crash when unbind namespaces

[sumiyawang]

From: sumiyawang <sumiyawang@tencent.com>

kernel will crash when use after free the ioremap space,
which is triggered by ndctl destroy-namespace while some IO operations
exist.
The sequence of pmem driver release chain should be changed:
freeze the queue and wait io finished first, then iounmap. 

[47202.018374] BUG: unable to handle page fault for address: ffffc90080200000
[47202.018822] CPU: 36 PID: 9606 Comm: systemd-udevd 
[47202.020921] Call Trace:
[47202.020969]  ? pmem_do_bvec+0xf9/0x3a0
[47202.021036]  ? xas_alloc+0x55/0xd0
[47202.021094]  pmem_rw_page+0x4b/0x80
[47202.021159]  bdev_read_page+0x86/0xb0
[47202.021222]  do_mpage_readpage+0x5d4/0x7a0
[47202.021297]  ? lru_cache_add+0xe/0x10
[47202.021359]  mpage_readpages+0xf9/0x1c0
[47202.021428]  ? bd_link_disk_holder+0x1a0/0x1a0
[47202.021506]  blkdev_readpages+0x1d/0x20
[47202.021574]  read_pages+0x67/0x1a0

ndctl Call Trace in vmcore:
PID: 23473  TASK: ffff88c4fbbe8000  CPU: 1   COMMAND: "ndctl"
__schedule
schedule
blk_mq_freeze_queue_wait
blk_freeze_queue
blk_cleanup_queue
pmem_release_queue
devm_action_release
release_nodes
devres_release_all
device_release_driver_internal
device_driver_detach
unbind_store

At that time, the ioremap space is released

Signed-off-by: sumiyawang <sumiyawang@tencent.com>
Reviewed-by: yongduan <yongduan@tencent.com>
---
 drivers/nvdimm/pmem.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
index 1e0615b..72de88f 100644
--- a/drivers/nvdimm/pmem.c
+++ b/drivers/nvdimm/pmem.c
@@ -450,11 +450,11 @@ static int pmem_attach_disk(struct device *dev,
 		pmem->pfn_flags |= PFN_MAP;
 		bb_range = pmem->pgmap.range;
 	} else {
+		addr = devm_memremap(dev, pmem->phys_addr,
+				pmem->size, ARCH_MEMREMAP_PMEM);
 		if (devm_add_action_or_reset(dev, pmem_release_queue,
 					&pmem->pgmap))
 			return -ENOMEM;
-		addr = devm_memremap(dev, pmem->phys_addr,
-				pmem->size, ARCH_MEMREMAP_PMEM);
 		bb_range.start =  res->start;
 		bb_range.end = res->end;
 	}
-- 
2.7.4

Re: [PATCH] pmem: fix the crash when unbind namespaces

[Dan Williams]

On Sun, Aug 22, 2021 at 4:49 AM <sumiyawang@gmail.com> wrote:
>
> From: sumiyawang <sumiyawang@tencent.com>
>
> kernel will crash when use after free the ioremap space,
> which is triggered by ndctl destroy-namespace while some IO operations
> exist.
> The sequence of pmem driver release chain should be changed:
> freeze the queue and wait io finished first, then iounmap.
>
> [47202.018374] BUG: unable to handle page fault for address: ffffc90080200000
> [47202.018822] CPU: 36 PID: 9606 Comm: systemd-udevd
> [47202.020921] Call Trace:
> [47202.020969]  ? pmem_do_bvec+0xf9/0x3a0
> [47202.021036]  ? xas_alloc+0x55/0xd0
> [47202.021094]  pmem_rw_page+0x4b/0x80
> [47202.021159]  bdev_read_page+0x86/0xb0
> [47202.021222]  do_mpage_readpage+0x5d4/0x7a0
> [47202.021297]  ? lru_cache_add+0xe/0x10
> [47202.021359]  mpage_readpages+0xf9/0x1c0
> [47202.021428]  ? bd_link_disk_holder+0x1a0/0x1a0
> [47202.021506]  blkdev_readpages+0x1d/0x20
> [47202.021574]  read_pages+0x67/0x1a0
>
> ndctl Call Trace in vmcore:
> PID: 23473  TASK: ffff88c4fbbe8000  CPU: 1   COMMAND: "ndctl"
> __schedule
> schedule
> blk_mq_freeze_queue_wait
> blk_freeze_queue
> blk_cleanup_queue
> pmem_release_queue
> devm_action_release
> release_nodes
> devres_release_all
> device_release_driver_internal
> device_driver_detach
> unbind_store
>
> At that time, the ioremap space is released

Thanks, looks good. I reworded the changelog a bit and added a "Cc:
stable" as well as a "Fixes: 50f44ee7248a ("mm/devm_memremap_pages:
fix final page put race")" tag. I'll send this up during the v5.15
merge window.

https://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm.git/commit/?h=libnvdimm-fixes&id=32b2397c

Re: [PATCH] pmem: fix the crash when unbind namespaces

[Dan Williams]

On Sun, Aug 22, 2021 at 4:49 AM <sumiyawang@gmail.com> wrote:
>
> From: sumiyawang <sumiyawang@tencent.com>
>
> kernel will crash when use after free the ioremap space,
> which is triggered by ndctl destroy-namespace while some IO operations
> exist.
> The sequence of pmem driver release chain should be changed:
> freeze the queue and wait io finished first, then iounmap.
>
> [47202.018374] BUG: unable to handle page fault for address: ffffc90080200000
> [47202.018822] CPU: 36 PID: 9606 Comm: systemd-udevd
> [47202.020921] Call Trace:
> [47202.020969]  ? pmem_do_bvec+0xf9/0x3a0
> [47202.021036]  ? xas_alloc+0x55/0xd0
> [47202.021094]  pmem_rw_page+0x4b/0x80
> [47202.021159]  bdev_read_page+0x86/0xb0
> [47202.021222]  do_mpage_readpage+0x5d4/0x7a0
> [47202.021297]  ? lru_cache_add+0xe/0x10
> [47202.021359]  mpage_readpages+0xf9/0x1c0
> [47202.021428]  ? bd_link_disk_holder+0x1a0/0x1a0
> [47202.021506]  blkdev_readpages+0x1d/0x20
> [47202.021574]  read_pages+0x67/0x1a0
>
> ndctl Call Trace in vmcore:
> PID: 23473  TASK: ffff88c4fbbe8000  CPU: 1   COMMAND: "ndctl"
> __schedule
> schedule
> blk_mq_freeze_queue_wait
> blk_freeze_queue
> blk_cleanup_queue
> pmem_release_queue
> devm_action_release
> release_nodes
> devres_release_all
> device_release_driver_internal
> device_driver_detach
> unbind_store
>
> At that time, the ioremap space is released

Thanks, looks good. I reworded the changelog a bit and added a "Cc:
stable" as well as a "Fixes: 50f44ee7248a ("mm/devm_memremap_pages:
fix final page put race")" tag. I'll send this up during the v5.15
merge window.

https://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm.git/commit/?h=libnvdimm-fixes&id=32b2397c