* switch the block layer to use kmap_local_page v2
@ 2021-06-15 13:24 Christoph Hellwig
2021-06-15 13:24 ` [PATCH 01/18] mm: add a kunmap_local_dirty helper Christoph Hellwig
` (18 more replies)
0 siblings, 19 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
Hi all,
this series switches the core block layer code and all users of the
existing bvec kmap helpers to use kmap_local_page. Drivers that
currently use open coded kmap_atomic calls will converted in a follow
on series.
To do so a new kunmap variant is added that calls
flush_kernel_dcache_page. I'm not entirely sure where to call
flush_dcache_page vs flush_kernel_dcache_page, so I've tried to follow
the documentation here, but additional feedback would be welcome.
Changes since v1:
- add more/better comments
- add a new kunmap_local_dirty helper to feal with
flush(_kernel)_dcache_page
Diffstat:
arch/mips/include/asm/mach-rc32434/rb.h | 2 -
block/bio-integrity.c | 14 +++-----
block/bio.c | 37 ++++++----------------
block/blk-map.c | 2 -
block/bounce.c | 39 +++++-------------------
block/t10-pi.c | 16 +++------
drivers/block/ps3disk.c | 19 +----------
drivers/block/rbd.c | 15 +--------
drivers/md/dm-writecache.c | 5 +--
include/linux/bio.h | 42 -------------------------
include/linux/bvec.h | 52 ++++++++++++++++++++++++++++++--
include/linux/highmem-internal.h | 7 ++++
include/linux/highmem.h | 10 ++++--
13 files changed, 102 insertions(+), 158 deletions(-)
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PATCH 01/18] mm: add a kunmap_local_dirty helper
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-18 3:01 ` Ira Weiny
2021-06-15 13:24 ` [PATCH 02/18] mm: use kunmap_local_dirty in memcpy_to_page Christoph Hellwig
` (17 subsequent siblings)
18 siblings, 1 reply; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
Add a helper that calls flush_kernel_dcache_page before unmapping the
local mapping. flush_kernel_dcache_page is required for all pages
potentially mapped into userspace that were written to using kmap*,
so having a helper that does the right thing can be very convenient.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
include/linux/highmem-internal.h | 7 +++++++
include/linux/highmem.h | 4 ++++
2 files changed, 11 insertions(+)
diff --git a/include/linux/highmem-internal.h b/include/linux/highmem-internal.h
index 7902c7d8b55f..bd37706db147 100644
--- a/include/linux/highmem-internal.h
+++ b/include/linux/highmem-internal.h
@@ -224,4 +224,11 @@ do { \
__kunmap_local(__addr); \
} while (0)
+#define kunmap_local_dirty(__page, __addr) \
+do { \
+ if (!PageSlab(__page)) \
+ flush_kernel_dcache_page(__page); \
+ kunmap_local(__addr); \
+} while (0)
+
#endif
diff --git a/include/linux/highmem.h b/include/linux/highmem.h
index 832b49b50c7b..65f548db4f2d 100644
--- a/include/linux/highmem.h
+++ b/include/linux/highmem.h
@@ -93,6 +93,10 @@ static inline void kmap_flush_unused(void);
* On HIGHMEM enabled systems mapping a highmem page has the side effect of
* disabling migration in order to keep the virtual address stable across
* preemption. No caller of kmap_local_page() can rely on this side effect.
+ *
+ * If data is written to the returned kernel mapping, the callers needs to
+ * unmap the mapping using kunmap_local_dirty(), else kunmap_local() should
+ * be used.
*/
static inline void *kmap_local_page(struct page *page);
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 02/18] mm: use kunmap_local_dirty in memcpy_to_page
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
2021-06-15 13:24 ` [PATCH 01/18] mm: add a kunmap_local_dirty helper Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 03/18] mm: use kmap_local_page in memzero_page Christoph Hellwig
` (16 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
memcpy_to_page can write to potentially mapped page cache pages, so
use kunmap_local_dirty to make sure flush_kernel_dcache_pages is
called.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
include/linux/highmem.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/highmem.h b/include/linux/highmem.h
index 65f548db4f2d..d0497c0daf80 100644
--- a/include/linux/highmem.h
+++ b/include/linux/highmem.h
@@ -333,7 +333,7 @@ static inline void memcpy_to_page(struct page *page, size_t offset,
VM_BUG_ON(offset + len > PAGE_SIZE);
memcpy(to + offset, from, len);
- kunmap_local(to);
+ kunmap_local_dirty(page, to);
}
static inline void memzero_page(struct page *page, size_t offset, size_t len)
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 03/18] mm: use kmap_local_page in memzero_page
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
2021-06-15 13:24 ` [PATCH 01/18] mm: add a kunmap_local_dirty helper Christoph Hellwig
2021-06-15 13:24 ` [PATCH 02/18] mm: use kunmap_local_dirty in memcpy_to_page Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 04/18] MIPS: don't include <linux/genhd.h> in <asm/mach-rc32434/rb.h> Christoph Hellwig
` (15 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Chaitanya Kulkarni,
Mike Snitzer, Geoff Levand, linuxppc-dev, linux-mips,
Dongsheng Yang, linux-kernel, James E.J. Bottomley, dm-devel,
Ilya Dryomov, Ira Weiny, ceph-devel
No need for kmap_atomic here.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
---
include/linux/highmem.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/linux/highmem.h b/include/linux/highmem.h
index d0497c0daf80..d3df0e2db44f 100644
--- a/include/linux/highmem.h
+++ b/include/linux/highmem.h
@@ -338,9 +338,9 @@ static inline void memcpy_to_page(struct page *page, size_t offset,
static inline void memzero_page(struct page *page, size_t offset, size_t len)
{
- char *addr = kmap_atomic(page);
+ char *addr = kmap_local_page(page);
memset(addr + offset, 0, len);
- kunmap_atomic(addr);
+ kunmap_local_dirty(page, addr);
}
#endif /* _LINUX_HIGHMEM_H */
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 04/18] MIPS: don't include <linux/genhd.h> in <asm/mach-rc32434/rb.h>
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (2 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 03/18] mm: use kmap_local_page in memzero_page Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 05/18] bvec: fix the include guards for bvec.h Christoph Hellwig
` (14 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Chaitanya Kulkarni,
Mike Snitzer, Geoff Levand, linuxppc-dev, linux-mips,
Dongsheng Yang, linux-kernel, James E.J. Bottomley, dm-devel,
Ilya Dryomov, Ira Weiny, ceph-devel
There is no need to include genhd.h from a random arch header, and not
doing so prevents the possibility for nasty include loops.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
---
arch/mips/include/asm/mach-rc32434/rb.h | 2 --
1 file changed, 2 deletions(-)
diff --git a/arch/mips/include/asm/mach-rc32434/rb.h b/arch/mips/include/asm/mach-rc32434/rb.h
index d502673a4f6c..34d179ca020b 100644
--- a/arch/mips/include/asm/mach-rc32434/rb.h
+++ b/arch/mips/include/asm/mach-rc32434/rb.h
@@ -7,8 +7,6 @@
#ifndef __ASM_RC32434_RB_H
#define __ASM_RC32434_RB_H
-#include <linux/genhd.h>
-
#define REGBASE 0x18000000
#define IDT434_REG_BASE ((volatile void *) KSEG1ADDR(REGBASE))
#define UART0BASE 0x58000
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 05/18] bvec: fix the include guards for bvec.h
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (3 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 04/18] MIPS: don't include <linux/genhd.h> in <asm/mach-rc32434/rb.h> Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 06/18] bvec: add a bvec_kmap_local helper Christoph Hellwig
` (13 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Bart Van Assche,
Mike Snitzer, Geoff Levand, linuxppc-dev, Chaitanya Kulkarni,
linux-mips, Dongsheng Yang, linux-kernel, James E.J. Bottomley,
dm-devel, Ilya Dryomov, Ira Weiny, ceph-devel
Fix the include guards to match the file naming.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
---
include/linux/bvec.h | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index ff832e698efb..883faf5f1523 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -4,8 +4,8 @@
*
* Copyright (C) 2001 Ming Lei <ming.lei@canonical.com>
*/
-#ifndef __LINUX_BVEC_ITER_H
-#define __LINUX_BVEC_ITER_H
+#ifndef __LINUX_BVEC_H
+#define __LINUX_BVEC_H
#include <linux/bug.h>
#include <linux/errno.h>
@@ -183,4 +183,4 @@ static inline void bvec_advance(const struct bio_vec *bvec,
}
}
-#endif /* __LINUX_BVEC_ITER_H */
+#endif /* __LINUX_BVEC_H */
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 06/18] bvec: add a bvec_kmap_local helper
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (4 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 05/18] bvec: fix the include guards for bvec.h Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-16 16:52 ` [dm-devel] " Bart Van Assche
2021-06-15 13:24 ` [PATCH 07/18] bvec: add memcpy_{from, to}_bvec and memzero_bvec helper Christoph Hellwig
` (12 subsequent siblings)
18 siblings, 1 reply; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Chaitanya Kulkarni,
Mike Snitzer, Geoff Levand, linuxppc-dev, linux-mips,
Dongsheng Yang, linux-kernel, James E.J. Bottomley, dm-devel,
Ilya Dryomov, Ira Weiny, ceph-devel
Add a helper to call kmap_local_page on a bvec. There is no need for
an unmap helper given that kunmap_local accept any address in the mapped
page.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
---
include/linux/bvec.h | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index 883faf5f1523..f8710af18eef 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -7,6 +7,7 @@
#ifndef __LINUX_BVEC_H
#define __LINUX_BVEC_H
+#include <linux/highmem.h>
#include <linux/bug.h>
#include <linux/errno.h>
#include <linux/limits.h>
@@ -183,4 +184,16 @@ static inline void bvec_advance(const struct bio_vec *bvec,
}
}
+/**
+ * bvec_kmap_local - map a bvec into the kernel virtual address space
+ * @bvec: bvec to map
+ *
+ * Must be called on single-page bvecs only. Call kunmap_local on the returned
+ * address to unmap.
+ */
+static inline void *bvec_kmap_local(struct bio_vec *bvec)
+{
+ return kmap_local_page(bvec->bv_page) + bvec->bv_offset;
+}
+
#endif /* __LINUX_BVEC_H */
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 07/18] bvec: add memcpy_{from, to}_bvec and memzero_bvec helper
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (5 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 06/18] bvec: add a bvec_kmap_local helper Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 08/18] block: use memzero_page in zero_fill_bio Christoph Hellwig
` (11 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Chaitanya Kulkarni,
Mike Snitzer, Geoff Levand, linuxppc-dev, linux-mips,
Dongsheng Yang, linux-kernel, James E.J. Bottomley, dm-devel,
Ilya Dryomov, Ira Weiny, ceph-devel
Add helpers to perform common memory operation on a bvec.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
---
include/linux/bvec.h | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index f8710af18eef..f9fa43b940ff 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -196,4 +196,37 @@ static inline void *bvec_kmap_local(struct bio_vec *bvec)
return kmap_local_page(bvec->bv_page) + bvec->bv_offset;
}
+/**
+ * memcpy_from_bvec - copy data from a bvec
+ * @bvec: bvec to copy from
+ *
+ * Must be called on single-page bvecs only.
+ */
+static inline void memcpy_from_bvec(char *to, struct bio_vec *bvec)
+{
+ memcpy_from_page(to, bvec->bv_page, bvec->bv_offset, bvec->bv_len);
+}
+
+/**
+ * memcpy_to_bvec - copy data to a bvec
+ * @bvec: bvec to copy to
+ *
+ * Must be called on single-page bvecs only.
+ */
+static inline void memcpy_to_bvec(struct bio_vec *bvec, const char *from)
+{
+ memcpy_to_page(bvec->bv_page, bvec->bv_offset, from, bvec->bv_len);
+}
+
+/**
+ * memzero_bvec - zero all data in a bvec
+ * @bvec: bvec to zero
+ *
+ * Must be called on single-page bvecs only.
+ */
+static inline void memzero_bvec(struct bio_vec *bvec)
+{
+ memzero_page(bvec->bv_page, bvec->bv_offset, bvec->bv_len);
+}
+
#endif /* __LINUX_BVEC_H */
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 08/18] block: use memzero_page in zero_fill_bio
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (6 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 07/18] bvec: add memcpy_{from, to}_bvec and memzero_bvec helper Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 09/18] rbd: use memzero_bvec Christoph Hellwig
` (10 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Chaitanya Kulkarni,
Mike Snitzer, Geoff Levand, linuxppc-dev, linux-mips,
Dongsheng Yang, linux-kernel, James E.J. Bottomley, dm-devel,
Ilya Dryomov, Ira Weiny, ceph-devel
Use memzero_bvec to zero each segment in the bio instead of manually
mapping and zeroing the data.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
---
block/bio.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index 44205dfb6b60..1d7abdb83a39 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -495,16 +495,11 @@ EXPORT_SYMBOL(bio_kmalloc);
void zero_fill_bio(struct bio *bio)
{
- unsigned long flags;
struct bio_vec bv;
struct bvec_iter iter;
- bio_for_each_segment(bv, bio, iter) {
- char *data = bvec_kmap_irq(&bv, &flags);
- memset(data, 0, bv.bv_len);
- flush_dcache_page(bv.bv_page);
- bvec_kunmap_irq(data, &flags);
- }
+ bio_for_each_segment(bv, bio, iter)
+ memzero_bvec(&bv);
}
EXPORT_SYMBOL(zero_fill_bio);
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 09/18] rbd: use memzero_bvec
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (7 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 08/18] block: use memzero_page in zero_fill_bio Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 10/18] dm-writecache: use bvec_kmap_local instead of bvec_kmap_irq Christoph Hellwig
` (9 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
Use memzero_bvec instead of reimplementing it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
---
drivers/block/rbd.c | 15 ++-------------
1 file changed, 2 insertions(+), 13 deletions(-)
diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c
index 531d390902dd..501a10bfb1fe 100644
--- a/drivers/block/rbd.c
+++ b/drivers/block/rbd.c
@@ -1219,24 +1219,13 @@ static void rbd_dev_mapping_clear(struct rbd_device *rbd_dev)
rbd_dev->mapping.size = 0;
}
-static void zero_bvec(struct bio_vec *bv)
-{
- void *buf;
- unsigned long flags;
-
- buf = bvec_kmap_irq(bv, &flags);
- memset(buf, 0, bv->bv_len);
- flush_dcache_page(bv->bv_page);
- bvec_kunmap_irq(buf, &flags);
-}
-
static void zero_bios(struct ceph_bio_iter *bio_pos, u32 off, u32 bytes)
{
struct ceph_bio_iter it = *bio_pos;
ceph_bio_iter_advance(&it, off);
ceph_bio_iter_advance_step(&it, bytes, ({
- zero_bvec(&bv);
+ memzero_bvec(&bv);
}));
}
@@ -1246,7 +1235,7 @@ static void zero_bvecs(struct ceph_bvec_iter *bvec_pos, u32 off, u32 bytes)
ceph_bvec_iter_advance(&it, off);
ceph_bvec_iter_advance_step(&it, bytes, ({
- zero_bvec(&bv);
+ memzero_bvec(&bv);
}));
}
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 10/18] dm-writecache: use bvec_kmap_local instead of bvec_kmap_irq
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (8 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 09/18] rbd: use memzero_bvec Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 11/18] ps3disk: use memcpy_{from,to}_bvec Christoph Hellwig
` (8 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
There is no need to disable interrupts in bio_copy_block, and the local
only mappings helps to avoid any sort of problems with stray writes
into the bio data.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
---
drivers/md/dm-writecache.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/drivers/md/dm-writecache.c b/drivers/md/dm-writecache.c
index aecc246ade26..93ca454eaca9 100644
--- a/drivers/md/dm-writecache.c
+++ b/drivers/md/dm-writecache.c
@@ -1205,14 +1205,13 @@ static void memcpy_flushcache_optimized(void *dest, void *source, size_t size)
static void bio_copy_block(struct dm_writecache *wc, struct bio *bio, void *data)
{
void *buf;
- unsigned long flags;
unsigned size;
int rw = bio_data_dir(bio);
unsigned remaining_size = wc->block_size;
do {
struct bio_vec bv = bio_iter_iovec(bio, bio->bi_iter);
- buf = bvec_kmap_irq(&bv, &flags);
+ buf = bvec_kmap_local(&bv);
size = bv.bv_len;
if (unlikely(size > remaining_size))
size = remaining_size;
@@ -1230,7 +1229,7 @@ static void bio_copy_block(struct dm_writecache *wc, struct bio *bio, void *data
memcpy_flushcache_optimized(data, buf, size);
}
- bvec_kunmap_irq(buf, &flags);
+ kunmap_local(buf);
data = (char *)data + size;
remaining_size -= size;
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 11/18] ps3disk: use memcpy_{from,to}_bvec
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (9 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 10/18] dm-writecache: use bvec_kmap_local instead of bvec_kmap_irq Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-16 16:36 ` Geoff Levand
2021-06-15 13:24 ` [PATCH 12/18] block: remove bvec_kmap_irq and bvec_kunmap_irq Christoph Hellwig
` (7 subsequent siblings)
18 siblings, 1 reply; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
Use the bvec helpers instead of open coding the copy.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/block/ps3disk.c | 19 +++----------------
1 file changed, 3 insertions(+), 16 deletions(-)
diff --git a/drivers/block/ps3disk.c b/drivers/block/ps3disk.c
index f374ea2c67ce..b7d4c3efd7a8 100644
--- a/drivers/block/ps3disk.c
+++ b/drivers/block/ps3disk.c
@@ -83,26 +83,13 @@ static void ps3disk_scatter_gather(struct ps3_storage_device *dev,
unsigned int offset = 0;
struct req_iterator iter;
struct bio_vec bvec;
- unsigned int i = 0;
- size_t size;
- void *buf;
rq_for_each_segment(bvec, req, iter) {
- unsigned long flags;
- dev_dbg(&dev->sbd.core, "%s:%u: bio %u: %u sectors from %llu\n",
- __func__, __LINE__, i, bio_sectors(iter.bio),
- iter.bio->bi_iter.bi_sector);
-
- size = bvec.bv_len;
- buf = bvec_kmap_irq(&bvec, &flags);
if (gather)
- memcpy(dev->bounce_buf+offset, buf, size);
+ memcpy_from_bvec(dev->bounce_buf + offset, &bvec);
else
- memcpy(buf, dev->bounce_buf+offset, size);
- offset += size;
- flush_kernel_dcache_page(bvec.bv_page);
- bvec_kunmap_irq(buf, &flags);
- i++;
+ memcpy_to_bvec(&bvec, dev->bounce_buf + offset);
+ offset += bvec.bv_len;
}
}
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 12/18] block: remove bvec_kmap_irq and bvec_kunmap_irq
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (10 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 11/18] ps3disk: use memcpy_{from,to}_bvec Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 13/18] block: rewrite bio_copy_data_iter to use bvec_kmap_local and memcpy_to_bvec Christoph Hellwig
` (6 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
These two helpers are entirely unused now.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
include/linux/bio.h | 42 ------------------------------------------
1 file changed, 42 deletions(-)
diff --git a/include/linux/bio.h b/include/linux/bio.h
index d2b98efb5cc5..8070f3f77c14 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -5,7 +5,6 @@
#ifndef __LINUX_BIO_H
#define __LINUX_BIO_H
-#include <linux/highmem.h>
#include <linux/mempool.h>
#include <linux/ioprio.h>
/* struct bio, bio_vec and BIO_* flags are defined in blk_types.h */
@@ -519,47 +518,6 @@ static inline void bio_clone_blkg_association(struct bio *dst,
struct bio *src) { }
#endif /* CONFIG_BLK_CGROUP */
-#ifdef CONFIG_HIGHMEM
-/*
- * remember never ever reenable interrupts between a bvec_kmap_irq and
- * bvec_kunmap_irq!
- */
-static inline char *bvec_kmap_irq(struct bio_vec *bvec, unsigned long *flags)
-{
- unsigned long addr;
-
- /*
- * might not be a highmem page, but the preempt/irq count
- * balancing is a lot nicer this way
- */
- local_irq_save(*flags);
- addr = (unsigned long) kmap_atomic(bvec->bv_page);
-
- BUG_ON(addr & ~PAGE_MASK);
-
- return (char *) addr + bvec->bv_offset;
-}
-
-static inline void bvec_kunmap_irq(char *buffer, unsigned long *flags)
-{
- unsigned long ptr = (unsigned long) buffer & PAGE_MASK;
-
- kunmap_atomic((void *) ptr);
- local_irq_restore(*flags);
-}
-
-#else
-static inline char *bvec_kmap_irq(struct bio_vec *bvec, unsigned long *flags)
-{
- return page_address(bvec->bv_page) + bvec->bv_offset;
-}
-
-static inline void bvec_kunmap_irq(char *buffer, unsigned long *flags)
-{
- *flags = 0;
-}
-#endif
-
/*
* BIO list management for use by remapping drivers (e.g. DM or MD) and loop.
*
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 13/18] block: rewrite bio_copy_data_iter to use bvec_kmap_local and memcpy_to_bvec
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (11 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 12/18] block: remove bvec_kmap_irq and bvec_kunmap_irq Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 14/18] block: use memcpy_to_bvec in copy_to_high_bio_irq Christoph Hellwig
` (5 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
Use the proper helpers instead of open coding the copy.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/bio.c | 28 ++++++++--------------------
1 file changed, 8 insertions(+), 20 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index 1d7abdb83a39..c14d2e66c084 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1186,27 +1186,15 @@ EXPORT_SYMBOL(bio_advance);
void bio_copy_data_iter(struct bio *dst, struct bvec_iter *dst_iter,
struct bio *src, struct bvec_iter *src_iter)
{
- struct bio_vec src_bv, dst_bv;
- void *src_p, *dst_p;
- unsigned bytes;
-
while (src_iter->bi_size && dst_iter->bi_size) {
- src_bv = bio_iter_iovec(src, *src_iter);
- dst_bv = bio_iter_iovec(dst, *dst_iter);
-
- bytes = min(src_bv.bv_len, dst_bv.bv_len);
-
- src_p = kmap_atomic(src_bv.bv_page);
- dst_p = kmap_atomic(dst_bv.bv_page);
-
- memcpy(dst_p + dst_bv.bv_offset,
- src_p + src_bv.bv_offset,
- bytes);
-
- kunmap_atomic(dst_p);
- kunmap_atomic(src_p);
-
- flush_dcache_page(dst_bv.bv_page);
+ struct bio_vec src_bv = bio_iter_iovec(src, *src_iter);
+ struct bio_vec dst_bv = bio_iter_iovec(dst, *dst_iter);
+ unsigned int bytes = min(src_bv.bv_len, dst_bv.bv_len);
+ void *src_buf;
+
+ src_buf = bvec_kmap_local(&src_bv);
+ memcpy_to_bvec(&dst_bv, src_buf);
+ kunmap_local(src_buf);
bio_advance_iter_single(src, src_iter, bytes);
bio_advance_iter_single(dst, dst_iter, bytes);
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 14/18] block: use memcpy_to_bvec in copy_to_high_bio_irq
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (12 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 13/18] block: rewrite bio_copy_data_iter to use bvec_kmap_local and memcpy_to_bvec Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 15/18] block: use memcpy_from_bvec in bio_copy_kern_endio_read Christoph Hellwig
` (4 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
Use memcpy_to_bvec instead of opencoding the logic.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/bounce.c | 20 ++------------------
1 file changed, 2 insertions(+), 18 deletions(-)
diff --git a/block/bounce.c b/block/bounce.c
index 94081e013c58..7e9e666c04dc 100644
--- a/block/bounce.c
+++ b/block/bounce.c
@@ -67,18 +67,6 @@ static __init int init_emergency_pool(void)
__initcall(init_emergency_pool);
-/*
- * highmem version, map in to vec
- */
-static void bounce_copy_vec(struct bio_vec *to, unsigned char *vfrom)
-{
- unsigned char *vto;
-
- vto = kmap_atomic(to->bv_page);
- memcpy(vto + to->bv_offset, vfrom, to->bv_len);
- kunmap_atomic(vto);
-}
-
/*
* Simple bounce buffer support for highmem pages. Depending on the
* queue gfp mask set, *to may or may not be a highmem page. kmap it
@@ -86,7 +74,6 @@ static void bounce_copy_vec(struct bio_vec *to, unsigned char *vfrom)
*/
static void copy_to_high_bio_irq(struct bio *to, struct bio *from)
{
- unsigned char *vfrom;
struct bio_vec tovec, fromvec;
struct bvec_iter iter;
/*
@@ -104,11 +91,8 @@ static void copy_to_high_bio_irq(struct bio *to, struct bio *from)
* been modified by the block layer, so use the original
* copy, bounce_copy_vec already uses tovec->bv_len
*/
- vfrom = page_address(fromvec.bv_page) +
- tovec.bv_offset;
-
- bounce_copy_vec(&tovec, vfrom);
- flush_dcache_page(tovec.bv_page);
+ memcpy_to_bvec(&tovec, page_address(fromvec.bv_page) +
+ tovec.bv_offset);
}
bio_advance_iter(from, &from_iter, tovec.bv_len);
}
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 15/18] block: use memcpy_from_bvec in bio_copy_kern_endio_read
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (13 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 14/18] block: use memcpy_to_bvec in copy_to_high_bio_irq Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 16/18] block: use memcpy_from_bvec in __blk_queue_bounce Christoph Hellwig
` (3 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
Use memcpy_from_bvec instead of open coding the logic.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/blk-map.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/block/blk-map.c b/block/blk-map.c
index 3743158ddaeb..d1448aaad980 100644
--- a/block/blk-map.c
+++ b/block/blk-map.c
@@ -400,7 +400,7 @@ static void bio_copy_kern_endio_read(struct bio *bio)
struct bvec_iter_all iter_all;
bio_for_each_segment_all(bvec, bio, iter_all) {
- memcpy(p, page_address(bvec->bv_page), bvec->bv_len);
+ memcpy_from_bvec(p, bvec);
p += bvec->bv_len;
}
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 16/18] block: use memcpy_from_bvec in __blk_queue_bounce
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (14 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 15/18] block: use memcpy_from_bvec in bio_copy_kern_endio_read Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 17/18] block: use bvec_kmap_local in t10_pi_type1_{prepare, complete} Christoph Hellwig
` (2 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
Rewrite the actual bounce buffering loop in __blk_queue_bounce to that
the memcpy_to_bvec helper can be used to perform the data copies.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/bounce.c | 19 +++++++------------
1 file changed, 7 insertions(+), 12 deletions(-)
diff --git a/block/bounce.c b/block/bounce.c
index 7e9e666c04dc..05fc7148489d 100644
--- a/block/bounce.c
+++ b/block/bounce.c
@@ -239,24 +239,19 @@ void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)
* because the 'bio' is single-page bvec.
*/
for (i = 0, to = bio->bi_io_vec; i < bio->bi_vcnt; to++, i++) {
- struct page *page = to->bv_page;
+ struct page *bounce_page;
- if (!PageHighMem(page))
+ if (!PageHighMem(to->bv_page))
continue;
- to->bv_page = mempool_alloc(&page_pool, GFP_NOIO);
- inc_zone_page_state(to->bv_page, NR_BOUNCE);
+ bounce_page = mempool_alloc(&page_pool, GFP_NOIO);
+ inc_zone_page_state(bounce_page, NR_BOUNCE);
if (rw == WRITE) {
- char *vto, *vfrom;
-
- flush_dcache_page(page);
-
- vto = page_address(to->bv_page) + to->bv_offset;
- vfrom = kmap_atomic(page) + to->bv_offset;
- memcpy(vto, vfrom, to->bv_len);
- kunmap_atomic(vfrom);
+ flush_dcache_page(to->bv_page);
+ memcpy_from_bvec(page_address(bounce_page), to);
}
+ to->bv_page = bounce_page;
}
trace_block_bio_bounce(*bio_orig);
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 17/18] block: use bvec_kmap_local in t10_pi_type1_{prepare, complete}
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (15 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 16/18] block: use memcpy_from_bvec in __blk_queue_bounce Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-15 13:24 ` [PATCH 18/18] block: use bvec_kmap_local in bio_integrity_process Christoph Hellwig
2021-06-16 16:03 ` switch the block layer to use kmap_local_page v2 Martin K. Petersen
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
Using local kmaps slightly reduces the chances to stray writes, and
the bvec interface cleans up the code a little bit.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/t10-pi.c | 16 ++++++----------
1 file changed, 6 insertions(+), 10 deletions(-)
diff --git a/block/t10-pi.c b/block/t10-pi.c
index d910534b3a41..00c203b2a921 100644
--- a/block/t10-pi.c
+++ b/block/t10-pi.c
@@ -147,11 +147,10 @@ static void t10_pi_type1_prepare(struct request *rq)
break;
bip_for_each_vec(iv, bip, iter) {
- void *p, *pmap;
unsigned int j;
+ void *p;
- pmap = kmap_atomic(iv.bv_page);
- p = pmap + iv.bv_offset;
+ p = bvec_kmap_local(&iv);
for (j = 0; j < iv.bv_len; j += tuple_sz) {
struct t10_pi_tuple *pi = p;
@@ -161,8 +160,7 @@ static void t10_pi_type1_prepare(struct request *rq)
ref_tag++;
p += tuple_sz;
}
-
- kunmap_atomic(pmap);
+ kunmap_local(p);
}
bip->bip_flags |= BIP_MAPPED_INTEGRITY;
@@ -195,11 +193,10 @@ static void t10_pi_type1_complete(struct request *rq, unsigned int nr_bytes)
struct bvec_iter iter;
bip_for_each_vec(iv, bip, iter) {
- void *p, *pmap;
unsigned int j;
+ void *p;
- pmap = kmap_atomic(iv.bv_page);
- p = pmap + iv.bv_offset;
+ p = bvec_kmap_local(&iv);
for (j = 0; j < iv.bv_len && intervals; j += tuple_sz) {
struct t10_pi_tuple *pi = p;
@@ -210,8 +207,7 @@ static void t10_pi_type1_complete(struct request *rq, unsigned int nr_bytes)
intervals--;
p += tuple_sz;
}
-
- kunmap_atomic(pmap);
+ kunmap_local(p);
}
}
}
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 18/18] block: use bvec_kmap_local in bio_integrity_process
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (16 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 17/18] block: use bvec_kmap_local in t10_pi_type1_{prepare, complete} Christoph Hellwig
@ 2021-06-15 13:24 ` Christoph Hellwig
2021-06-16 16:03 ` switch the block layer to use kmap_local_page v2 Martin K. Petersen
18 siblings, 0 replies; 26+ messages in thread
From: Christoph Hellwig @ 2021-06-15 13:24 UTC (permalink / raw)
To: Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, linux-mips, Dongsheng Yang,
linux-kernel, James E.J. Bottomley, dm-devel, Ilya Dryomov,
Ira Weiny, ceph-devel
Using local kmaps slightly reduces the chances to stray writes, and
the bvec interface cleans up the code a little bit.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/bio-integrity.c | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/block/bio-integrity.c b/block/bio-integrity.c
index 4b4eb8964a6f..8f54d49dc500 100644
--- a/block/bio-integrity.c
+++ b/block/bio-integrity.c
@@ -172,18 +172,16 @@ static blk_status_t bio_integrity_process(struct bio *bio,
iter.prot_buf = prot_buf;
__bio_for_each_segment(bv, bio, bviter, *proc_iter) {
- void *kaddr = kmap_atomic(bv.bv_page);
+ void *kaddr = bvec_kmap_local(&bv);
- iter.data_buf = kaddr + bv.bv_offset;
+ iter.data_buf = kaddr;
iter.data_size = bv.bv_len;
-
ret = proc_fn(&iter);
- if (ret) {
- kunmap_atomic(kaddr);
- return ret;
- }
+ kunmap_local(kaddr);
+
+ if (ret)
+ break;
- kunmap_atomic(kaddr);
}
return ret;
}
--
2.30.2
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: switch the block layer to use kmap_local_page v2
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
` (17 preceding siblings ...)
2021-06-15 13:24 ` [PATCH 18/18] block: use bvec_kmap_local in bio_integrity_process Christoph Hellwig
@ 2021-06-16 16:03 ` Martin K. Petersen
18 siblings, 0 replies; 26+ messages in thread
From: Martin K. Petersen @ 2021-06-16 16:03 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Jens Axboe, linux-block, Thomas Bogendoerfer, Mike Snitzer,
Geoff Levand, linuxppc-dev, ceph-devel, linux-mips,
Dongsheng Yang, linux-kernel, James E.J. Bottomley, dm-devel,
linux-arch, Thomas Gleixner, Ira Weiny, Ilya Dryomov
Christoph,
> this series switches the core block layer code and all users of the
> existing bvec kmap helpers to use kmap_local_page. Drivers that
> currently use open coded kmap_atomic calls will converted in a follow
> on series.
Looks OK to me.
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 11/18] ps3disk: use memcpy_{from,to}_bvec
2021-06-15 13:24 ` [PATCH 11/18] ps3disk: use memcpy_{from,to}_bvec Christoph Hellwig
@ 2021-06-16 16:36 ` Geoff Levand
0 siblings, 0 replies; 26+ messages in thread
From: Geoff Levand @ 2021-06-16 16:36 UTC (permalink / raw)
To: Christoph Hellwig, Jens Axboe, Thomas Gleixner
Cc: linux-arch, linux-block, Thomas Bogendoerfer, Mike Snitzer,
linuxppc-dev, linux-mips, Dongsheng Yang, linux-kernel,
James E.J. Bottomley, dm-devel, Geert Uytterhoeven, Ilya Dryomov,
Ira Weiny, ceph-devel
Hi Christoph,
On 6/15/21 6:24 AM, Christoph Hellwig wrote:
> Use the bvec helpers instead of open coding the copy.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> drivers/block/ps3disk.c | 19 +++----------------
> 1 file changed, 3 insertions(+), 16 deletions(-)
I tested your patch set applied to v5.13-rc6 on PS3 and it seemed to be
working OK.
I did some rsync's, some dd's, some fsck's, etc. If you have anything
you could suggest that you think would exercise your changes I could
try that also.
Tested-by: Geoff Levand <geoff@infradead.org>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [dm-devel] [PATCH 06/18] bvec: add a bvec_kmap_local helper
2021-06-15 13:24 ` [PATCH 06/18] bvec: add a bvec_kmap_local helper Christoph Hellwig
@ 2021-06-16 16:52 ` Bart Van Assche
0 siblings, 0 replies; 26+ messages in thread
From: Bart Van Assche @ 2021-06-16 16:52 UTC (permalink / raw)
To: Christoph Hellwig, Jens Axboe, Thomas Gleixner
Cc: linux-arch, James E.J. Bottomley, Thomas Bogendoerfer,
Chaitanya Kulkarni, Mike Snitzer, Geoff Levand, Ira Weiny,
linux-mips, Dongsheng Yang, linux-kernel, linux-block, dm-devel,
Ilya Dryomov, linuxppc-dev, ceph-devel
On 6/15/21 6:24 AM, Christoph Hellwig wrote:
> +/**
> + * bvec_kmap_local - map a bvec into the kernel virtual address space
> + * @bvec: bvec to map
> + *
> + * Must be called on single-page bvecs only. Call kunmap_local on the returned
> + * address to unmap.
> + */
> +static inline void *bvec_kmap_local(struct bio_vec *bvec)
> +{
> + return kmap_local_page(bvec->bv_page) + bvec->bv_offset;
> +}
Hi Christoph,
Would it be appropriate to add WARN_ON_ONCE(bvec->bv_offset >=
PAGE_SIZE) in this function?
Thanks,
Bart.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 01/18] mm: add a kunmap_local_dirty helper
2021-06-15 13:24 ` [PATCH 01/18] mm: add a kunmap_local_dirty helper Christoph Hellwig
@ 2021-06-18 3:01 ` Ira Weiny
2021-06-18 3:37 ` Herbert Xu
0 siblings, 1 reply; 26+ messages in thread
From: Ira Weiny @ 2021-06-18 3:01 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Jens Axboe, linux-block, Thomas Bogendoerfer, Herbert Xu,
Mike Snitzer, David S. Miller, Geoff Levand, ceph-devel,
linux-mips, Dongsheng Yang, linux-kernel, James E.J. Bottomley,
dm-devel, linux-arch, Thomas Gleixner, linuxppc-dev,
Ilya Dryomov
On Tue, Jun 15, 2021 at 03:24:39PM +0200, Christoph Hellwig wrote:
> Add a helper that calls flush_kernel_dcache_page before unmapping the
> local mapping. flush_kernel_dcache_page is required for all pages
> potentially mapped into userspace that were written to using kmap*,
> so having a helper that does the right thing can be very convenient.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> include/linux/highmem-internal.h | 7 +++++++
> include/linux/highmem.h | 4 ++++
> 2 files changed, 11 insertions(+)
>
> diff --git a/include/linux/highmem-internal.h b/include/linux/highmem-internal.h
> index 7902c7d8b55f..bd37706db147 100644
> --- a/include/linux/highmem-internal.h
> +++ b/include/linux/highmem-internal.h
> @@ -224,4 +224,11 @@ do { \
> __kunmap_local(__addr); \
> } while (0)
>
> +#define kunmap_local_dirty(__page, __addr) \
I think having to store the page and addr to return to kunmap_local_dirty() is
going to be a pain in some code paths. Not a show stopper but see below...
> +do { \
> + if (!PageSlab(__page)) \
Was there some clarification why the page can't be a Slab page? Or is this
just an optimization?
> + flush_kernel_dcache_page(__page); \
Is this required on 32bit systems? Why is kunmap_flush_on_unmap() not
sufficient on 64bit systems? The normal kunmap_local() path does that.
I'm sorry but I did not see a conclusion to my query on V1. Herbert implied the
he just copied from the crypto code.[1] I'm concerned that this _dirty() call
is just going to confuse the users of kmap even more. So why can't we get to
the bottom of why flush_kernel_dcache_page() needs so much logic around it
before complicating the general kernel users.
I would like to see it go away if possible.
Ira
[1] https://lore.kernel.org/lkml/20210615050258.GA5208@gondor.apana.org.au/
> + kunmap_local(__addr); \
> +} while (0)
> +
> #endif
> diff --git a/include/linux/highmem.h b/include/linux/highmem.h
> index 832b49b50c7b..65f548db4f2d 100644
> --- a/include/linux/highmem.h
> +++ b/include/linux/highmem.h
> @@ -93,6 +93,10 @@ static inline void kmap_flush_unused(void);
> * On HIGHMEM enabled systems mapping a highmem page has the side effect of
> * disabling migration in order to keep the virtual address stable across
> * preemption. No caller of kmap_local_page() can rely on this side effect.
> + *
> + * If data is written to the returned kernel mapping, the callers needs to
> + * unmap the mapping using kunmap_local_dirty(), else kunmap_local() should
> + * be used.
> */
> static inline void *kmap_local_page(struct page *page);
>
> --
> 2.30.2
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 01/18] mm: add a kunmap_local_dirty helper
2021-06-18 3:01 ` Ira Weiny
@ 2021-06-18 3:37 ` Herbert Xu
2021-06-18 18:12 ` Ira Weiny
0 siblings, 1 reply; 26+ messages in thread
From: Herbert Xu @ 2021-06-18 3:37 UTC (permalink / raw)
To: Ira Weiny
Cc: Jens Axboe, linux-block, Thomas Bogendoerfer, Mike Snitzer,
David S. Miller, Geoff Levand, Christoph Lameter, ceph-devel,
linux-mips, Dongsheng Yang, linux-kernel, James E.J. Bottomley,
dm-devel, linux-arch, Thomas Gleixner, Ilya Dryomov,
linuxppc-dev, Christoph Hellwig
On Thu, Jun 17, 2021 at 08:01:57PM -0700, Ira Weiny wrote:
>
> > + flush_kernel_dcache_page(__page); \
>
> Is this required on 32bit systems? Why is kunmap_flush_on_unmap() not
> sufficient on 64bit systems? The normal kunmap_local() path does that.
>
> I'm sorry but I did not see a conclusion to my query on V1. Herbert implied the
> he just copied from the crypto code.[1] I'm concerned that this _dirty() call
> is just going to confuse the users of kmap even more. So why can't we get to
> the bottom of why flush_kernel_dcache_page() needs so much logic around it
> before complicating the general kernel users.
>
> I would like to see it go away if possible.
This thread may be related:
https://lwn.net/Articles/240249/
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 01/18] mm: add a kunmap_local_dirty helper
2021-06-18 3:37 ` Herbert Xu
@ 2021-06-18 18:12 ` Ira Weiny
2021-06-24 6:32 ` [PATCH] crypto: scatterwalk - Remove obsolete PageSlab check Herbert Xu
0 siblings, 1 reply; 26+ messages in thread
From: Ira Weiny @ 2021-06-18 18:12 UTC (permalink / raw)
To: Herbert Xu
Cc: Jens Axboe, linux-block, Thomas Bogendoerfer, Mike Snitzer,
David S. Miller, Geoff Levand, Christoph Lameter, ceph-devel,
linux-mips, Dongsheng Yang, linux-kernel, James E.J. Bottomley,
dm-devel, linux-arch, Thomas Gleixner, Ilya Dryomov,
linuxppc-dev, Christoph Hellwig
On Fri, Jun 18, 2021 at 11:37:28AM +0800, Herbert Xu wrote:
> On Thu, Jun 17, 2021 at 08:01:57PM -0700, Ira Weiny wrote:
> >
> > > + flush_kernel_dcache_page(__page); \
> >
> > Is this required on 32bit systems? Why is kunmap_flush_on_unmap() not
> > sufficient on 64bit systems? The normal kunmap_local() path does that.
> >
> > I'm sorry but I did not see a conclusion to my query on V1. Herbert implied the
> > he just copied from the crypto code.[1] I'm concerned that this _dirty() call
> > is just going to confuse the users of kmap even more. So why can't we get to
> > the bottom of why flush_kernel_dcache_page() needs so much logic around it
> > before complicating the general kernel users.
> >
> > I would like to see it go away if possible.
>
> This thread may be related:
>
> https://lwn.net/Articles/240249/
Interesting! Thanks!
Digging around a bit more I found:
https://lore.kernel.org/patchwork/patch/439637/
Auditing all the flush_dcache_page() arch code reveals that the mapping field
is either unused, or is checked for NULL. Furthermore, all the implementations
call page_mapping_file() which further limits the page to not be a swap page.
All flush_kernel_dcache_page() implementations appears to operate the same way
in all arch's which define that call.
So I'm confident now that additional !PageSlab(__page) checks are not needed
and this patch is unnecessary. Christoph, can we leave this out of the kmap
API and just fold the flush_kernel_dcache_page() calls back into the bvec code?
Unfortunately, I'm not convinced this can be handled completely by
kunmap_local() nor the mem*_page() calls because there is a difference between
flush_dcache_page() and flush_kernel_dcache_page() in most archs... [parisc
being an exception which falls back to flush_kernel_dcache_page()]...
It seems like the generic unmap path _should_ be able to determine which call
to make based on the page but I'd have to look at that more.
Ira
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PATCH] crypto: scatterwalk - Remove obsolete PageSlab check
2021-06-18 18:12 ` Ira Weiny
@ 2021-06-24 6:32 ` Herbert Xu
0 siblings, 0 replies; 26+ messages in thread
From: Herbert Xu @ 2021-06-24 6:32 UTC (permalink / raw)
To: Ira Weiny
Cc: Jens Axboe, linux-block, Thomas Bogendoerfer, Mike Snitzer,
David S. Miller, Geoff Levand, Christoph Lameter, ceph-devel,
linux-mips, Dongsheng Yang, linux-kernel, James E.J. Bottomley,
dm-devel, Linux Crypto Mailing List, linux-arch, Thomas Gleixner,
Ilya Dryomov, linuxppc-dev, Christoph Hellwig
On Fri, Jun 18, 2021 at 11:12:58AM -0700, Ira Weiny wrote:
>
> Interesting! Thanks!
>
> Digging around a bit more I found:
>
> https://lore.kernel.org/patchwork/patch/439637/
Nice find. So we can at least get rid of the PageSlab call from
the Crypto API.
---8<---
As it is now legal to call flush_dcache_page on slab pages we
no longer need to do the check in the Crypto API.
Reported-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
diff --git a/include/crypto/scatterwalk.h b/include/crypto/scatterwalk.h
index c837d0775474..7af08174a721 100644
--- a/include/crypto/scatterwalk.h
+++ b/include/crypto/scatterwalk.h
@@ -81,12 +81,7 @@ static inline void scatterwalk_pagedone(struct scatter_walk *walk, int out,
struct page *page;
page = sg_page(walk->sg) + ((walk->offset - 1) >> PAGE_SHIFT);
- /* Test ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE first as
- * PageSlab cannot be optimised away per se due to
- * use of volatile pointer.
- */
- if (ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE && !PageSlab(page))
- flush_dcache_page(page);
+ flush_dcache_page(page);
}
if (more && walk->offset >= walk->sg->offset + walk->sg->length)
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply related [flat|nested] 26+ messages in thread
end of thread, other threads:[~2021-06-24 6:33 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-15 13:24 switch the block layer to use kmap_local_page v2 Christoph Hellwig
2021-06-15 13:24 ` [PATCH 01/18] mm: add a kunmap_local_dirty helper Christoph Hellwig
2021-06-18 3:01 ` Ira Weiny
2021-06-18 3:37 ` Herbert Xu
2021-06-18 18:12 ` Ira Weiny
2021-06-24 6:32 ` [PATCH] crypto: scatterwalk - Remove obsolete PageSlab check Herbert Xu
2021-06-15 13:24 ` [PATCH 02/18] mm: use kunmap_local_dirty in memcpy_to_page Christoph Hellwig
2021-06-15 13:24 ` [PATCH 03/18] mm: use kmap_local_page in memzero_page Christoph Hellwig
2021-06-15 13:24 ` [PATCH 04/18] MIPS: don't include <linux/genhd.h> in <asm/mach-rc32434/rb.h> Christoph Hellwig
2021-06-15 13:24 ` [PATCH 05/18] bvec: fix the include guards for bvec.h Christoph Hellwig
2021-06-15 13:24 ` [PATCH 06/18] bvec: add a bvec_kmap_local helper Christoph Hellwig
2021-06-16 16:52 ` [dm-devel] " Bart Van Assche
2021-06-15 13:24 ` [PATCH 07/18] bvec: add memcpy_{from, to}_bvec and memzero_bvec helper Christoph Hellwig
2021-06-15 13:24 ` [PATCH 08/18] block: use memzero_page in zero_fill_bio Christoph Hellwig
2021-06-15 13:24 ` [PATCH 09/18] rbd: use memzero_bvec Christoph Hellwig
2021-06-15 13:24 ` [PATCH 10/18] dm-writecache: use bvec_kmap_local instead of bvec_kmap_irq Christoph Hellwig
2021-06-15 13:24 ` [PATCH 11/18] ps3disk: use memcpy_{from,to}_bvec Christoph Hellwig
2021-06-16 16:36 ` Geoff Levand
2021-06-15 13:24 ` [PATCH 12/18] block: remove bvec_kmap_irq and bvec_kunmap_irq Christoph Hellwig
2021-06-15 13:24 ` [PATCH 13/18] block: rewrite bio_copy_data_iter to use bvec_kmap_local and memcpy_to_bvec Christoph Hellwig
2021-06-15 13:24 ` [PATCH 14/18] block: use memcpy_to_bvec in copy_to_high_bio_irq Christoph Hellwig
2021-06-15 13:24 ` [PATCH 15/18] block: use memcpy_from_bvec in bio_copy_kern_endio_read Christoph Hellwig
2021-06-15 13:24 ` [PATCH 16/18] block: use memcpy_from_bvec in __blk_queue_bounce Christoph Hellwig
2021-06-15 13:24 ` [PATCH 17/18] block: use bvec_kmap_local in t10_pi_type1_{prepare, complete} Christoph Hellwig
2021-06-15 13:24 ` [PATCH 18/18] block: use bvec_kmap_local in bio_integrity_process Christoph Hellwig
2021-06-16 16:03 ` switch the block layer to use kmap_local_page v2 Martin K. Petersen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).