* [RESEND PATCH V5 00/33] block: support multipage bvec
@ 2018-05-25 3:45 Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 01/33] block: rename bio_for_each_segment* with bio_for_each_page* Ming Lei
` (34 more replies)
0 siblings, 35 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:45 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
Hi,
This patchset brings multipage bvec into block layer:
1) what is multipage bvec?
Multipage bvecs means that one 'struct bio_bvec' can hold multiple pages
which are physically contiguous instead of one single page used in linux
kernel for long time.
2) why is multipage bvec introduced?
Kent proposed the idea[1] first.
As system's RAM becomes much bigger than before, and huge page, transparent
huge page and memory compaction are widely used, it is a bit easy now
to see physically contiguous pages from fs in I/O. On the other hand, from
block layer's view, it isn't necessary to store intermediate pages into bvec,
and it is enough to just store the physicallly contiguous 'segment' in each
io vector.
Also huge pages are being brought to filesystem and swap [2][6], we can
do IO on a hugepage each time[3], which requires that one bio can transfer
at least one huge page one time. Turns out it isn't flexiable to change
BIO_MAX_PAGES simply[3][5]. Multipage bvec can fit in this case very well.
As we saw, if CONFIG_THP_SWAP is enabled, BIO_MAX_PAGES can be configured
as much bigger, such as 512, which requires at least two 4K pages for holding
the bvec table.
With multipage bvec:
- Inside block layer, both bio splitting and sg map can become more
efficient than before by just traversing the physically contiguous
'segment' instead of each page.
- segment handling in block layer can be improved much in future since it
should be quite easy to convert multipage bvec into segment easily. For
example, we might just store segment in each bvec directly in future.
- bio size can be increased and it should improve some high-bandwidth IO
case in theory[4].
- there is opportunity in future to improve memory footprint of bvecs.
3) how is multipage bvec implemented in this patchset?
The 1st 3 patches are prepare patches for multipage bvec, and one big
change is to rename bio_for_each_segment*() as bio_for_each_page*().
The patches of 4~31 implement multipage bvec in block layer:
- put all tricks into bvec/bio/rq iterators, and as far as
drivers and fs use these standard iterators, they are happy
with multipage bvec
- use multipage bvec to split bio and map sg
- introduce bio_for_each_segment*() for iterating bio segment by
segment
- make current bio_for_each_page*() to itereate page by page and
make sure current uses won't be broken
The patch 32 redefines BIO_MAX_PAGES as 256.
The patch 33 documents usages of bio iterator helpers.
These patches can be found in the following git tree:
gitweb: https://github.com/ming1/linux/commits/v4.17-rc-blk-next-mp_bvec_v5
git: https://github.com/ming1/linux.git #v4.17-rc-blk-next-mp_bvec_v5
Lots of test(blktest, xfstests, ltp io, ...) have been run with this patchset,
and not see regression.
Thanks Christoph for reviewing the early version and providing very good
suggestions, such as: introduce bio_init_with_vec_table(), remove another
unnecessary helpers for cleanup and so on.
Any comments are welcome!
V5:
- remove some of prepare patches, which have been merged already
- add bio_clone_seg_bioset() to fix DM's bio clone, which
is introduced by 18a25da84354c6b (dm: ensure bio submission follows
a depth-first tree walk)
- rebase on the latest block for-v4.18
V4:
- rename bio_for_each_segment*() as bio_for_each_page*(), rename
bio_segments() as bio_pages(), rename rq_for_each_segment() as
rq_for_each_pages(), because these helpers never return real
segment, and they always return single page bvec
- introducing segment_for_each_page_all()
- introduce new bio_for_each_segment*()/rq_for_each_segment()/bio_segments()
for returning real multipage segment
- rewrite segment_last_page()
- rename bvec iterator helper as suggested by Christoph
- replace comment with applying bio helpers as suggested by Christoph
- document usage of bio iterator helpers
- redefine BIO_MAX_PAGES as 256 to make the biggest bvec table
accommodated in 4K page
- move bio_alloc_pages() into bcache as suggested by Christoph
V3:
- rebase on v4.13-rc3 with for-next of block tree
- run more xfstests: xfs/ext4 over NVMe, Sata, DM(linear),
MD(raid1), and not see regressions triggered
- add Reviewed-by on some btrfs patches
- remove two MD patches because both are merged to linus tree
already
V2:
- bvec table direct access in raid has been cleaned, so NO_MP
flag is dropped
- rebase on recent Neil Brown's change on bio and bounce code
- reorganize the patchset
V1:
- against v4.10-rc1 and some cleanup in V0 are in -linus already
- handle queue_virt_boundary() in mp bvec change and make NVMe happy
- further BTRFS cleanup
- remove QUEUE_FLAG_SPLIT_MP
- rename for two new helpers of bio_for_each_segment_all()
- fix bounce convertion
- address comments in V0
[1], http://marc.info/?l=linux-kernel&m=141680246629547&w=2
[2], https://patchwork.kernel.org/patch/9451523/
[3], http://marc.info/?t=147735447100001&r=1&w=2
[4], http://marc.info/?l=linux-mm&m=147745525801433&w=2
[5], http://marc.info/?t=149569484500007&r=1&w=2
[6], http://marc.info/?t=149820215300004&r=1&w=2
Ming Lei (33):
block: rename bio_for_each_segment* with bio_for_each_page*
block: rename rq_for_each_segment as rq_for_each_page
block: rename bio_segments() with bio_pages()
block: introduce multipage page bvec helpers
block: introduce bio_for_each_segment()
block: use bio_for_each_segment() to compute segments count
block: use bio_for_each_segment() to map sg
block: introduce segment_last_page()
fs/buffer.c: use bvec iterator to truncate the bio
btrfs: use segment_last_page to get bio's last page
block: implement bio_pages_all() via bio_for_each_page_all()
block: introduce bio_segments()
block: introduce rq_for_each_segment()
block: loop: pass segments to iov_iter
block: introduce bio_clone_seg_bioset()
dm: clone bio via bio_clone_seg_bioset
block: bio: introduce bio_for_each_page_all2 and
bio_for_each_segment_all
block: deal with dirtying pages for multipage bvec
block: convert to bio_for_each_page_all2()
md/dm/bcache: conver to bio_for_each_page_all2 and
bio_for_each_segment
fs: conver to bio_for_each_page_all2
btrfs: conver to bio_for_each_page_all2
ext4: conver to bio_for_each_page_all2
f2fs: conver to bio_for_each_page_all2
xfs: conver to bio_for_each_page_all2
exofs: conver to bio_for_each_page_all2
gfs2: conver to bio_for_each_page_all2
block: kill bio_for_each_page_all()
block: rename bio_for_each_page_all2 as bio_for_each_page_all
block: enable multipage bvecs
block: bio: pass segments to bio if bio_add_page() is bypassed
block: always define BIO_MAX_PAGES as 256
block: document usage of bio iterator helpers
Documentation/block/biodoc.txt | 6 +-
Documentation/block/biovecs.txt | 36 ++++++-
arch/m68k/emu/nfblock.c | 2 +-
arch/xtensa/platforms/iss/simdisk.c | 2 +-
block/bio-integrity.c | 2 +-
block/bio.c | 186 +++++++++++++++++++++++++++++-------
block/blk-core.c | 2 +-
block/blk-merge.c | 158 +++++++++++++++++++++++-------
block/blk-zoned.c | 5 +-
block/bounce.c | 10 +-
drivers/block/aoe/aoecmd.c | 4 +-
drivers/block/brd.c | 2 +-
drivers/block/drbd/drbd_main.c | 4 +-
drivers/block/drbd/drbd_receiver.c | 2 +-
drivers/block/drbd/drbd_worker.c | 2 +-
drivers/block/floppy.c | 4 +-
drivers/block/loop.c | 10 +-
drivers/block/nbd.c | 4 +-
drivers/block/null_blk.c | 4 +-
drivers/block/ps3disk.c | 4 +-
drivers/block/ps3vram.c | 2 +-
drivers/block/rsxx/dma.c | 2 +-
drivers/block/zram/zram_drv.c | 2 +-
drivers/md/bcache/btree.c | 3 +-
drivers/md/bcache/debug.c | 2 +-
drivers/md/bcache/request.c | 2 +-
drivers/md/dm-crypt.c | 3 +-
drivers/md/dm-integrity.c | 4 +-
drivers/md/dm-log-writes.c | 4 +-
drivers/md/dm.c | 6 +-
drivers/md/raid1.c | 3 +-
drivers/md/raid5.c | 2 +-
drivers/nvdimm/blk.c | 2 +-
drivers/nvdimm/btt.c | 2 +-
drivers/nvdimm/pmem.c | 2 +-
drivers/s390/block/dasd_diag.c | 4 +-
drivers/s390/block/dasd_eckd.c | 16 ++--
drivers/s390/block/dasd_fba.c | 6 +-
drivers/s390/block/dcssblk.c | 2 +-
drivers/s390/block/scm_blk.c | 2 +-
drivers/s390/block/xpram.c | 2 +-
drivers/target/target_core_pscsi.c | 2 +-
fs/block_dev.c | 6 +-
fs/btrfs/check-integrity.c | 6 +-
fs/btrfs/compression.c | 8 +-
fs/btrfs/disk-io.c | 3 +-
fs/btrfs/extent_io.c | 14 ++-
fs/btrfs/file-item.c | 4 +-
fs/btrfs/inode.c | 12 ++-
fs/btrfs/raid56.c | 5 +-
fs/buffer.c | 5 +-
fs/crypto/bio.c | 3 +-
fs/direct-io.c | 4 +-
fs/exofs/ore.c | 3 +-
fs/exofs/ore_raid.c | 3 +-
fs/ext4/page-io.c | 3 +-
fs/ext4/readpage.c | 3 +-
fs/f2fs/data.c | 9 +-
fs/gfs2/lops.c | 6 +-
fs/gfs2/meta_io.c | 3 +-
fs/iomap.c | 3 +-
fs/mpage.c | 3 +-
fs/xfs/xfs_aops.c | 5 +-
include/linux/bio.h | 98 +++++++++++++++----
include/linux/blkdev.h | 6 +-
include/linux/bvec.h | 134 ++++++++++++++++++++++++--
include/linux/ceph/messenger.h | 2 +-
67 files changed, 671 insertions(+), 204 deletions(-)
--
2.9.5
^ permalink raw reply [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 01/33] block: rename bio_for_each_segment* with bio_for_each_page*
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
@ 2018-05-25 3:45 ` Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 02/33] block: rename rq_for_each_segment as rq_for_each_page Ming Lei
` (33 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:45 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
It is a tree-wide mechanical replacement since both bio_for_each_segment()
and bio_for_each_segment_all() never returns real segment at all, and
both just return one page per bvec and deceive us for long time, so fix
their names.
This is a pre-patch for supporting multipage bvec. Once multipage bvec
is in, each bvec will store a real multipage segment, so people won't be
confused with these wrong names.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
Documentation/block/biovecs.txt | 4 ++--
arch/m68k/emu/nfblock.c | 2 +-
arch/xtensa/platforms/iss/simdisk.c | 2 +-
block/bio-integrity.c | 2 +-
block/bio.c | 24 ++++++++++++------------
block/blk-merge.c | 6 +++---
block/blk-zoned.c | 4 ++--
block/bounce.c | 8 ++++----
drivers/block/aoe/aoecmd.c | 4 ++--
drivers/block/brd.c | 2 +-
drivers/block/drbd/drbd_main.c | 4 ++--
drivers/block/drbd/drbd_receiver.c | 2 +-
drivers/block/drbd/drbd_worker.c | 2 +-
drivers/block/nbd.c | 2 +-
drivers/block/null_blk.c | 2 +-
drivers/block/ps3vram.c | 2 +-
drivers/block/rsxx/dma.c | 2 +-
drivers/block/zram/zram_drv.c | 2 +-
drivers/md/bcache/btree.c | 2 +-
drivers/md/bcache/debug.c | 2 +-
drivers/md/bcache/request.c | 2 +-
drivers/md/bcache/util.c | 2 +-
drivers/md/dm-crypt.c | 2 +-
drivers/md/dm-integrity.c | 4 ++--
drivers/md/dm-log-writes.c | 2 +-
drivers/md/dm.c | 2 +-
drivers/md/raid1.c | 2 +-
drivers/md/raid5.c | 2 +-
drivers/nvdimm/blk.c | 2 +-
drivers/nvdimm/btt.c | 2 +-
drivers/nvdimm/pmem.c | 2 +-
drivers/s390/block/dcssblk.c | 2 +-
drivers/s390/block/xpram.c | 2 +-
fs/block_dev.c | 4 ++--
fs/btrfs/check-integrity.c | 4 ++--
fs/btrfs/compression.c | 2 +-
fs/btrfs/disk-io.c | 2 +-
fs/btrfs/extent_io.c | 6 +++---
fs/btrfs/file-item.c | 4 ++--
fs/btrfs/inode.c | 8 ++++----
fs/btrfs/raid56.c | 4 ++--
fs/crypto/bio.c | 2 +-
fs/direct-io.c | 2 +-
fs/exofs/ore.c | 2 +-
fs/exofs/ore_raid.c | 2 +-
fs/ext4/page-io.c | 2 +-
fs/ext4/readpage.c | 2 +-
fs/f2fs/data.c | 6 +++---
fs/gfs2/lops.c | 2 +-
fs/gfs2/meta_io.c | 2 +-
fs/iomap.c | 2 +-
fs/mpage.c | 2 +-
fs/xfs/xfs_aops.c | 2 +-
include/linux/bio.h | 10 +++++-----
include/linux/blkdev.h | 2 +-
include/linux/ceph/messenger.h | 2 +-
56 files changed, 92 insertions(+), 92 deletions(-)
diff --git a/Documentation/block/biovecs.txt b/Documentation/block/biovecs.txt
index 25689584e6e0..b4d238b8d9fc 100644
--- a/Documentation/block/biovecs.txt
+++ b/Documentation/block/biovecs.txt
@@ -28,10 +28,10 @@ normal code doesn't have to deal with bi_bvec_done.
constructed from the raw biovecs but taking into account bi_bvec_done and
bi_size.
- bio_for_each_segment() has been updated to take a bvec_iter argument
+ bio_for_each_page() has been updated to take a bvec_iter argument
instead of an integer (that corresponded to bi_idx); for a lot of code the
conversion just required changing the types of the arguments to
- bio_for_each_segment().
+ bio_for_each_page().
* Advancing a bvec_iter is done with bio_advance_iter(); bio_advance() is a
wrapper around bio_advance_iter() that operates on bio->bi_iter, and also
diff --git a/arch/m68k/emu/nfblock.c b/arch/m68k/emu/nfblock.c
index e9110b9b8bcd..8b226eac9289 100644
--- a/arch/m68k/emu/nfblock.c
+++ b/arch/m68k/emu/nfblock.c
@@ -69,7 +69,7 @@ static blk_qc_t nfhd_make_request(struct request_queue *queue, struct bio *bio)
dir = bio_data_dir(bio);
shift = dev->bshift;
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
len = bvec.bv_len;
len >>= 9;
nfhd_read_write(dev->id, 0, dir, sec >> shift, len >> shift,
diff --git a/arch/xtensa/platforms/iss/simdisk.c b/arch/xtensa/platforms/iss/simdisk.c
index 026211e7ab09..1455883089d4 100644
--- a/arch/xtensa/platforms/iss/simdisk.c
+++ b/arch/xtensa/platforms/iss/simdisk.c
@@ -108,7 +108,7 @@ static blk_qc_t simdisk_make_request(struct request_queue *q, struct bio *bio)
struct bvec_iter iter;
sector_t sector = bio->bi_iter.bi_sector;
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
char *buffer = kmap_atomic(bvec.bv_page) + bvec.bv_offset;
unsigned len = bvec.bv_len >> SECTOR_SHIFT;
diff --git a/block/bio-integrity.c b/block/bio-integrity.c
index add7c7c85335..738496d75385 100644
--- a/block/bio-integrity.c
+++ b/block/bio-integrity.c
@@ -204,7 +204,7 @@ static blk_status_t bio_integrity_process(struct bio *bio,
iter.seed = proc_iter->bi_sector;
iter.prot_buf = prot_buf;
- __bio_for_each_segment(bv, bio, bviter, *proc_iter) {
+ __bio_for_each_page(bv, bio, bviter, *proc_iter) {
void *kaddr = kmap_atomic(bv.bv_page);
iter.data_buf = kaddr + bv.bv_offset;
diff --git a/block/bio.c b/block/bio.c
index 0a4df92cd689..5495dc30d080 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -536,7 +536,7 @@ void zero_fill_bio_iter(struct bio *bio, struct bvec_iter start)
struct bio_vec bv;
struct bvec_iter iter;
- __bio_for_each_segment(bv, bio, iter, start) {
+ __bio_for_each_page(bv, bio, iter, start) {
char *data = bvec_kmap_irq(&bv, &flags);
memset(data, 0, bv.bv_len);
flush_dcache_page(bv.bv_page);
@@ -700,7 +700,7 @@ struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
bio->bi_io_vec[bio->bi_vcnt++] = bio_src->bi_io_vec[0];
break;
default:
- bio_for_each_segment(bv, bio_src, iter)
+ bio_for_each_page(bv, bio_src, iter)
bio->bi_io_vec[bio->bi_vcnt++] = bv;
break;
}
@@ -1092,7 +1092,7 @@ static int bio_copy_from_iter(struct bio *bio, struct iov_iter *iter)
int i;
struct bio_vec *bvec;
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
ssize_t ret;
ret = copy_page_from_iter(bvec->bv_page,
@@ -1123,7 +1123,7 @@ static int bio_copy_to_iter(struct bio *bio, struct iov_iter iter)
int i;
struct bio_vec *bvec;
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
ssize_t ret;
ret = copy_page_to_iter(bvec->bv_page,
@@ -1146,7 +1146,7 @@ void bio_free_pages(struct bio *bio)
struct bio_vec *bvec;
int i;
- bio_for_each_segment_all(bvec, bio, i)
+ bio_for_each_page_all(bvec, bio, i)
__free_page(bvec->bv_page);
}
EXPORT_SYMBOL(bio_free_pages);
@@ -1385,7 +1385,7 @@ struct bio *bio_map_user_iov(struct request_queue *q,
return bio;
out_unmap:
- bio_for_each_segment_all(bvec, bio, j) {
+ bio_for_each_page_all(bvec, bio, j) {
put_page(bvec->bv_page);
}
bio_put(bio);
@@ -1400,7 +1400,7 @@ static void __bio_unmap_user(struct bio *bio)
/*
* make sure we dirty pages we wrote to
*/
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
if (bio_data_dir(bio) == READ)
set_page_dirty_lock(bvec->bv_page);
@@ -1493,7 +1493,7 @@ static void bio_copy_kern_endio_read(struct bio *bio)
struct bio_vec *bvec;
int i;
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
memcpy(p, page_address(bvec->bv_page), bvec->bv_len);
p += bvec->bv_len;
}
@@ -1603,7 +1603,7 @@ void bio_set_pages_dirty(struct bio *bio)
struct bio_vec *bvec;
int i;
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
if (page && !PageCompound(page))
@@ -1617,7 +1617,7 @@ static void bio_release_pages(struct bio *bio)
struct bio_vec *bvec;
int i;
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
if (page)
@@ -1671,7 +1671,7 @@ void bio_check_pages_dirty(struct bio *bio)
int nr_clean_pages = 0;
int i;
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
if (PageDirty(page) || PageCompound(page)) {
@@ -1730,7 +1730,7 @@ void bio_flush_dcache_pages(struct bio *bi)
struct bio_vec bvec;
struct bvec_iter iter;
- bio_for_each_segment(bvec, bi, iter)
+ bio_for_each_page(bvec, bi, iter)
flush_dcache_page(bvec.bv_page);
}
EXPORT_SYMBOL(bio_flush_dcache_pages);
diff --git a/block/blk-merge.c b/block/blk-merge.c
index 5573d0fbec53..fc2aa21b7959 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -110,7 +110,7 @@ static struct bio *blk_bio_segment_split(struct request_queue *q,
struct bio *new = NULL;
const unsigned max_sectors = get_max_io_size(q, bio);
- bio_for_each_segment(bv, bio, iter) {
+ bio_for_each_page(bv, bio, iter) {
/*
* If the queue doesn't support SG gaps and adding this
* offset would create a gap, disallow it.
@@ -245,7 +245,7 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q,
seg_size = 0;
nr_phys_segs = 0;
for_each_bio(bio) {
- bio_for_each_segment(bv, bio, iter) {
+ bio_for_each_page(bv, bio, iter) {
/*
* If SG merging is disabled, each bio vector is
* a segment
@@ -412,7 +412,7 @@ static int __blk_bios_map_sg(struct request_queue *q, struct bio *bio,
int cluster = blk_queue_cluster(q), nsegs = 0;
for_each_bio(bio)
- bio_for_each_segment(bvec, bio, iter)
+ bio_for_each_page(bvec, bio, iter)
__blk_segment_map_sg(q, &bvec, sglist, &bvprv, sg,
&nsegs, &cluster);
diff --git a/block/blk-zoned.c b/block/blk-zoned.c
index 08e84ef2bc05..77f3cecfaa7d 100644
--- a/block/blk-zoned.c
+++ b/block/blk-zoned.c
@@ -190,7 +190,7 @@ int blkdev_report_zones(struct block_device *bdev,
n = 0;
nz = 0;
nr_rep = 0;
- bio_for_each_segment_all(bv, bio, i) {
+ bio_for_each_page_all(bv, bio, i) {
if (!bv->bv_page)
break;
@@ -223,7 +223,7 @@ int blkdev_report_zones(struct block_device *bdev,
*nr_zones = nz;
out:
- bio_for_each_segment_all(bv, bio, i)
+ bio_for_each_page_all(bv, bio, i)
__free_page(bv->bv_page);
bio_put(bio);
diff --git a/block/bounce.c b/block/bounce.c
index fea9c8146d82..f4ee4b81f7a2 100644
--- a/block/bounce.c
+++ b/block/bounce.c
@@ -119,7 +119,7 @@ static void copy_to_high_bio_irq(struct bio *to, struct bio *from)
*/
struct bvec_iter from_iter = BVEC_ITER_ALL_INIT;
- bio_for_each_segment(tovec, to, iter) {
+ bio_for_each_page(tovec, to, iter) {
fromvec = bio_iter_iovec(from, from_iter);
if (tovec.bv_page != fromvec.bv_page) {
/*
@@ -147,7 +147,7 @@ static void bounce_end_io(struct bio *bio, mempool_t *pool)
/*
* free up bounce indirect pages used
*/
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
orig_vec = bio_iter_iovec(bio_orig, orig_iter);
if (bvec->bv_page != orig_vec.bv_page) {
dec_zone_page_state(bvec->bv_page, NR_BOUNCE);
@@ -204,7 +204,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
int sectors = 0;
bool passthrough = bio_is_passthrough(*bio_orig);
- bio_for_each_segment(from, *bio_orig, iter) {
+ bio_for_each_page(from, *bio_orig, iter) {
if (i++ < BIO_MAX_PAGES)
sectors += from.bv_len >> 9;
if (page_to_pfn(from.bv_page) > q->limits.bounce_pfn)
@@ -222,7 +222,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
bio = bio_clone_bioset(*bio_orig, GFP_NOIO, passthrough ? NULL :
bounce_bio_set);
- bio_for_each_segment_all(to, bio, i) {
+ bio_for_each_page_all(to, bio, i) {
struct page *page = to->bv_page;
if (page_to_pfn(page) <= q->limits.bounce_pfn)
diff --git a/drivers/block/aoe/aoecmd.c b/drivers/block/aoe/aoecmd.c
index 096882e54095..ae930cd00579 100644
--- a/drivers/block/aoe/aoecmd.c
+++ b/drivers/block/aoe/aoecmd.c
@@ -299,7 +299,7 @@ skb_fillup(struct sk_buff *skb, struct bio *bio, struct bvec_iter iter)
int frag = 0;
struct bio_vec bv;
- __bio_for_each_segment(bv, bio, iter, iter)
+ __bio_for_each_page(bv, bio, iter, iter)
skb_fill_page_desc(skb, frag++, bv.bv_page,
bv.bv_offset, bv.bv_len);
}
@@ -1031,7 +1031,7 @@ bvcpy(struct sk_buff *skb, struct bio *bio, struct bvec_iter iter, long cnt)
iter.bi_size = cnt;
- __bio_for_each_segment(bv, bio, iter, iter) {
+ __bio_for_each_page(bv, bio, iter, iter) {
char *p = kmap_atomic(bv.bv_page) + bv.bv_offset;
skb_copy_bits(skb, soff, p, bv.bv_len);
kunmap_atomic(p);
diff --git a/drivers/block/brd.c b/drivers/block/brd.c
index 39c5b90cc187..ff9359f63a3a 100644
--- a/drivers/block/brd.c
+++ b/drivers/block/brd.c
@@ -291,7 +291,7 @@ static blk_qc_t brd_make_request(struct request_queue *q, struct bio *bio)
if (bio_end_sector(bio) > get_capacity(bio->bi_disk))
goto io_error;
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
unsigned int len = bvec.bv_len;
int err;
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 185f1ef00a7c..3d8c2ce1a3c7 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -1601,7 +1601,7 @@ static int _drbd_send_bio(struct drbd_peer_device *peer_device, struct bio *bio)
struct bvec_iter iter;
/* hint all but last page with MSG_MORE */
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
int err;
err = _drbd_no_send_page(peer_device, bvec.bv_page,
@@ -1623,7 +1623,7 @@ static int _drbd_send_zc_bio(struct drbd_peer_device *peer_device, struct bio *b
struct bvec_iter iter;
/* hint all but last page with MSG_MORE */
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
int err;
err = _drbd_send_page(peer_device, bvec.bv_page,
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index c72dee0ef083..1ad8b9258ab5 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -1919,7 +1919,7 @@ static int recv_dless_read(struct drbd_peer_device *peer_device, struct drbd_req
bio = req->master_bio;
D_ASSERT(peer_device->device, sector == bio->bi_iter.bi_sector);
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
void *mapped = kmap(bvec.bv_page) + bvec.bv_offset;
expect = min_t(int, data_size, bvec.bv_len);
err = drbd_recv_all_warn(peer_device->connection, mapped, expect);
diff --git a/drivers/block/drbd/drbd_worker.c b/drivers/block/drbd/drbd_worker.c
index 1476cb3439f4..61e1217920a1 100644
--- a/drivers/block/drbd/drbd_worker.c
+++ b/drivers/block/drbd/drbd_worker.c
@@ -337,7 +337,7 @@ void drbd_csum_bio(struct crypto_ahash *tfm, struct bio *bio, void *digest)
sg_init_table(&sg, 1);
crypto_ahash_init(req);
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
sg_set_page(&sg, bvec.bv_page, bvec.bv_len, bvec.bv_offset);
ahash_request_set_crypt(req, &sg, NULL, sg.length);
crypto_ahash_update(req);
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index abc0a815354f..52f683bb2b9a 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -498,7 +498,7 @@ static int nbd_send_cmd(struct nbd_device *nbd, struct nbd_cmd *cmd, int index)
struct bvec_iter iter;
struct bio_vec bvec;
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
bool is_last = !next && bio_iter_last(bvec, iter);
int flags = is_last ? 0 : MSG_MORE;
diff --git a/drivers/block/null_blk.c b/drivers/block/null_blk.c
index a76553293a31..506c74501114 100644
--- a/drivers/block/null_blk.c
+++ b/drivers/block/null_blk.c
@@ -1171,7 +1171,7 @@ static int null_handle_bio(struct nullb_cmd *cmd)
}
spin_lock_irq(&nullb->lock);
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
len = bvec.bv_len;
err = null_transfer(nullb, bvec.bv_page, len, bvec.bv_offset,
op_is_write(bio_op(bio)), sector,
diff --git a/drivers/block/ps3vram.c b/drivers/block/ps3vram.c
index 6a55959cbf78..c2c5eeefa620 100644
--- a/drivers/block/ps3vram.c
+++ b/drivers/block/ps3vram.c
@@ -557,7 +557,7 @@ static struct bio *ps3vram_do_bio(struct ps3_system_bus_device *dev,
struct bvec_iter iter;
struct bio *next;
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
/* PS3 is ppc64, so we don't handle highmem */
char *ptr = page_address(bvec.bv_page) + bvec.bv_offset;
size_t len = bvec.bv_len, retlen;
diff --git a/drivers/block/rsxx/dma.c b/drivers/block/rsxx/dma.c
index beaccf197a5a..fec10b58a960 100644
--- a/drivers/block/rsxx/dma.c
+++ b/drivers/block/rsxx/dma.c
@@ -723,7 +723,7 @@ blk_status_t rsxx_dma_queue_bio(struct rsxx_cardinfo *card,
bv_len -= RSXX_HW_BLK_SIZE;
}
} else {
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
bv_len = bvec.bv_len;
bv_off = bvec.bv_offset;
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 0f3fadd71230..cf5c9a712959 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -1197,7 +1197,7 @@ static void __zram_make_request(struct zram *zram, struct bio *bio)
break;
}
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
struct bio_vec bv = bvec;
unsigned int unwritten = bvec.bv_len;
diff --git a/drivers/md/bcache/btree.c b/drivers/md/bcache/btree.c
index 17936b2dc7d6..a9d82911c3d2 100644
--- a/drivers/md/bcache/btree.c
+++ b/drivers/md/bcache/btree.c
@@ -424,7 +424,7 @@ static void do_btree_node_write(struct btree *b)
struct bio_vec *bv;
void *base = (void *) ((unsigned long) i & ~(PAGE_SIZE - 1));
- bio_for_each_segment_all(bv, b->bio, j)
+ bio_for_each_page_all(bv, b->bio, j)
memcpy(page_address(bv->bv_page),
base + j * PAGE_SIZE, PAGE_SIZE);
diff --git a/drivers/md/bcache/debug.c b/drivers/md/bcache/debug.c
index 4e63c6f6c04d..b0951a668675 100644
--- a/drivers/md/bcache/debug.c
+++ b/drivers/md/bcache/debug.c
@@ -121,7 +121,7 @@ void bch_data_verify(struct cached_dev *dc, struct bio *bio)
submit_bio_wait(check);
citer.bi_size = UINT_MAX;
- bio_for_each_segment(bv, bio, iter) {
+ bio_for_each_page(bv, bio, iter) {
void *p1 = kmap_atomic(bv.bv_page);
void *p2;
diff --git a/drivers/md/bcache/request.c b/drivers/md/bcache/request.c
index 8e3e8655ed63..68a5c613fb93 100644
--- a/drivers/md/bcache/request.c
+++ b/drivers/md/bcache/request.c
@@ -43,7 +43,7 @@ static void bio_csum(struct bio *bio, struct bkey *k)
struct bvec_iter iter;
uint64_t csum = 0;
- bio_for_each_segment(bv, bio, iter) {
+ bio_for_each_page(bv, bio, iter) {
void *d = kmap(bv.bv_page) + bv.bv_offset;
csum = bch_crc64_update(csum, d, bv.bv_len);
kunmap(bv.bv_page);
diff --git a/drivers/md/bcache/util.c b/drivers/md/bcache/util.c
index 74febd5230df..77230973a110 100644
--- a/drivers/md/bcache/util.c
+++ b/drivers/md/bcache/util.c
@@ -303,7 +303,7 @@ int bch_bio_alloc_pages(struct bio *bio, gfp_t gfp_mask)
int i;
struct bio_vec *bv;
- bio_for_each_segment_all(bv, bio, i) {
+ bio_for_each_page_all(bv, bio, i) {
bv->bv_page = alloc_page(gfp_mask);
if (!bv->bv_page) {
while (--bv >= bio->bi_io_vec)
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 44ff473dab3e..74737ae0ef11 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -1451,7 +1451,7 @@ static void crypt_free_buffer_pages(struct crypt_config *cc, struct bio *clone)
unsigned int i;
struct bio_vec *bv;
- bio_for_each_segment_all(bv, clone, i) {
+ bio_for_each_page_all(bv, clone, i) {
BUG_ON(!bv->bv_page);
mempool_free(bv->bv_page, cc->page_pool);
}
diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c
index 77d9fe58dae2..c0de39c60fa8 100644
--- a/drivers/md/dm-integrity.c
+++ b/drivers/md/dm-integrity.c
@@ -1256,7 +1256,7 @@ static void integrity_metadata(struct work_struct *w)
if (!checksums)
checksums = checksums_onstack;
- __bio_for_each_segment(bv, bio, iter, dio->orig_bi_iter) {
+ __bio_for_each_page(bv, bio, iter, dio->orig_bi_iter) {
unsigned pos;
char *mem, *checksums_ptr;
@@ -1376,7 +1376,7 @@ static int dm_integrity_map(struct dm_target *ti, struct bio *bio)
if (ic->sectors_per_block > 1) {
struct bvec_iter iter;
struct bio_vec bv;
- bio_for_each_segment(bv, bio, iter) {
+ bio_for_each_page(bv, bio, iter) {
if (unlikely(bv.bv_len & ((ic->sectors_per_block << SECTOR_SHIFT) - 1))) {
DMERR("Bio vector (%u,%u) is not aligned on %u-sector boundary",
bv.bv_offset, bv.bv_len, ic->sectors_per_block);
diff --git a/drivers/md/dm-log-writes.c b/drivers/md/dm-log-writes.c
index c90c7c08a77f..ab31ba9c3b37 100644
--- a/drivers/md/dm-log-writes.c
+++ b/drivers/md/dm-log-writes.c
@@ -732,7 +732,7 @@ static int log_writes_map(struct dm_target *ti, struct bio *bio)
* can't just hold onto the page until some later point, we have to
* manually copy the contents.
*/
- bio_for_each_segment(bv, bio, iter) {
+ bio_for_each_page(bv, bio, iter) {
struct page *page;
void *src, *dst;
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index 4ea404dbcf0b..f1db181e082e 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -1156,7 +1156,7 @@ void dm_remap_zone_report(struct dm_target *ti, struct bio *bio, sector_t start)
* Remap the start sector of the reported zones. For sequential zones,
* also remap the write pointer position.
*/
- bio_for_each_segment(bvec, report_bio, iter) {
+ bio_for_each_page(bvec, report_bio, iter) {
addr = kmap_atomic(bvec.bv_page);
/* Remember the report header in the first page */
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index e9e3308cb0a7..e318a0c19eb0 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -2123,7 +2123,7 @@ static void process_checks(struct r1bio *r1_bio)
/* Now we can 'fixup' the error value */
sbio->bi_status = 0;
- bio_for_each_segment_all(bi, sbio, j)
+ bio_for_each_page_all(bi, sbio, j)
page_len[j] = bi->bv_len;
if (!status) {
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index be117d0a65a8..2ce47a567801 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -1247,7 +1247,7 @@ async_copy_data(int frombio, struct bio *bio, struct page **page,
flags |= ASYNC_TX_FENCE;
init_async_submit(&submit, flags, tx, NULL, NULL, NULL);
- bio_for_each_segment(bvl, bio, iter) {
+ bio_for_each_page(bvl, bio, iter) {
int len = bvl.bv_len;
int clen;
int b_offset = 0;
diff --git a/drivers/nvdimm/blk.c b/drivers/nvdimm/blk.c
index 62e9cb167aad..7cae5b7d1c45 100644
--- a/drivers/nvdimm/blk.c
+++ b/drivers/nvdimm/blk.c
@@ -187,7 +187,7 @@ static blk_qc_t nd_blk_make_request(struct request_queue *q, struct bio *bio)
nsblk = q->queuedata;
rw = bio_data_dir(bio);
do_acct = nd_iostat_start(bio, &start);
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
unsigned int len = bvec.bv_len;
BUG_ON(len > PAGE_SIZE);
diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c
index 85de8053aa34..1613b591b695 100644
--- a/drivers/nvdimm/btt.c
+++ b/drivers/nvdimm/btt.c
@@ -1452,7 +1452,7 @@ static blk_qc_t btt_make_request(struct request_queue *q, struct bio *bio)
return BLK_QC_T_NONE;
do_acct = nd_iostat_start(bio, &start);
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
unsigned int len = bvec.bv_len;
if (len > PAGE_SIZE || len < btt->sector_size ||
diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
index 9d714926ecf5..89bb8aaa7ff5 100644
--- a/drivers/nvdimm/pmem.c
+++ b/drivers/nvdimm/pmem.c
@@ -183,7 +183,7 @@ static blk_qc_t pmem_make_request(struct request_queue *q, struct bio *bio)
nvdimm_flush(nd_region);
do_acct = nd_iostat_start(bio, &start);
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
rc = pmem_do_bvec(pmem, bvec.bv_page, bvec.bv_len,
bvec.bv_offset, op_is_write(bio_op(bio)),
iter.bi_sector);
diff --git a/drivers/s390/block/dcssblk.c b/drivers/s390/block/dcssblk.c
index 0a312e450207..c2df0b87352a 100644
--- a/drivers/s390/block/dcssblk.c
+++ b/drivers/s390/block/dcssblk.c
@@ -884,7 +884,7 @@ dcssblk_make_request(struct request_queue *q, struct bio *bio)
}
index = (bio->bi_iter.bi_sector >> 3);
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
page_addr = (unsigned long)
page_address(bvec.bv_page) + bvec.bv_offset;
source_addr = dev_info->start + (index<<12) + bytes_done;
diff --git a/drivers/s390/block/xpram.c b/drivers/s390/block/xpram.c
index 3df5d68d09f0..e930952a0d4b 100644
--- a/drivers/s390/block/xpram.c
+++ b/drivers/s390/block/xpram.c
@@ -203,7 +203,7 @@ static blk_qc_t xpram_make_request(struct request_queue *q, struct bio *bio)
if ((bio->bi_iter.bi_sector >> 3) > 0xffffffffU - xdev->offset)
goto fail;
index = (bio->bi_iter.bi_sector >> 3) + xdev->offset;
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
page_addr = (unsigned long)
kmap(bvec.bv_page) + bvec.bv_offset;
bytes = bvec.bv_len;
diff --git a/fs/block_dev.c b/fs/block_dev.c
index 7ec920e27065..65498a34efa9 100644
--- a/fs/block_dev.c
+++ b/fs/block_dev.c
@@ -242,7 +242,7 @@ __blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter,
}
__set_current_state(TASK_RUNNING);
- bio_for_each_segment_all(bvec, &bio, i) {
+ bio_for_each_page_all(bvec, &bio, i) {
if (should_dirty && !PageCompound(bvec->bv_page))
set_page_dirty_lock(bvec->bv_page);
put_page(bvec->bv_page);
@@ -310,7 +310,7 @@ static void blkdev_bio_end_io(struct bio *bio)
struct bio_vec *bvec;
int i;
- bio_for_each_segment_all(bvec, bio, i)
+ bio_for_each_page_all(bvec, bio, i)
put_page(bvec->bv_page);
bio_put(bio);
}
diff --git a/fs/btrfs/check-integrity.c b/fs/btrfs/check-integrity.c
index dc062b195c46..e5f7df09683f 100644
--- a/fs/btrfs/check-integrity.c
+++ b/fs/btrfs/check-integrity.c
@@ -2817,7 +2817,7 @@ static void __btrfsic_submit_bio(struct bio *bio)
goto leave;
cur_bytenr = dev_bytenr;
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
BUG_ON(bvec.bv_len != PAGE_SIZE);
mapped_datav[i] = kmap(bvec.bv_page);
i++;
@@ -2832,7 +2832,7 @@ static void __btrfsic_submit_bio(struct bio *bio)
mapped_datav, segs,
bio, &bio_is_patched,
NULL, bio->bi_opf);
- bio_for_each_segment(bvec, bio, iter)
+ bio_for_each_page(bvec, bio, iter)
kunmap(bvec.bv_page);
kfree(mapped_datav);
} else if (NULL != dev_state && (bio->bi_opf & REQ_PREFLUSH)) {
diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c
index 1061575a7d25..4ea718bdfb41 100644
--- a/fs/btrfs/compression.c
+++ b/fs/btrfs/compression.c
@@ -172,7 +172,7 @@ static void end_compressed_bio_read(struct bio *bio)
* checked so the end_io handlers know about it
*/
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_segment_all(bvec, cb->orig_bio, i)
+ bio_for_each_page_all(bvec, cb->orig_bio, i)
SetPageChecked(bvec->bv_page);
bio_endio(cb->orig_bio);
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 60caa68c3618..c6dc8a636413 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -831,7 +831,7 @@ static blk_status_t btree_csum_one_bio(struct bio *bio)
int i, ret = 0;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
root = BTRFS_I(bvec->bv_page->mapping->host)->root;
ret = csum_dirty_buffer(root->fs_info, bvec->bv_page);
if (ret)
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index e99b329002cf..4f314d87ce4d 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -2458,7 +2458,7 @@ static void end_bio_extent_writepage(struct bio *bio)
int i;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
struct inode *inode = page->mapping->host;
struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
@@ -2529,7 +2529,7 @@ static void end_bio_extent_readpage(struct bio *bio)
int i;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
struct inode *inode = page->mapping->host;
struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
@@ -3682,7 +3682,7 @@ static void end_bio_extent_buffer_writepage(struct bio *bio)
int i, done;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
eb = (struct extent_buffer *)page->private;
diff --git a/fs/btrfs/file-item.c b/fs/btrfs/file-item.c
index f9dd6d1836a3..e322592d0b69 100644
--- a/fs/btrfs/file-item.c
+++ b/fs/btrfs/file-item.c
@@ -209,7 +209,7 @@ static blk_status_t __btrfs_lookup_bio_sums(struct inode *inode, struct bio *bio
if (dio)
offset = logical_offset;
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
page_bytes_left = bvec.bv_len;
if (count)
goto next;
@@ -451,7 +451,7 @@ blk_status_t btrfs_csum_one_bio(struct inode *inode, struct bio *bio,
sums->bytenr = (u64)bio->bi_iter.bi_sector << 9;
index = 0;
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
if (!contig)
offset = page_offset(bvec.bv_page) + bvec.bv_offset;
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index d241285a0d2a..f78155e3a4dc 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -7894,7 +7894,7 @@ static void btrfs_retry_endio_nocsum(struct bio *bio)
done->uptodate = 1;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_segment_all(bvec, bio, i)
+ bio_for_each_page_all(bvec, bio, i)
clean_io_failure(BTRFS_I(inode)->root->fs_info, failure_tree,
io_tree, done->start, bvec->bv_page,
btrfs_ino(BTRFS_I(inode)), 0);
@@ -7924,7 +7924,7 @@ static blk_status_t __btrfs_correct_data_nocsum(struct inode *inode,
done.inode = inode;
io_bio->bio.bi_iter = io_bio->iter;
- bio_for_each_segment(bvec, &io_bio->bio, iter) {
+ bio_for_each_page(bvec, &io_bio->bio, iter) {
nr_sectors = BTRFS_BYTES_TO_BLKS(fs_info, bvec.bv_len);
pgoff = bvec.bv_offset;
@@ -7986,7 +7986,7 @@ static void btrfs_retry_endio(struct bio *bio)
failure_tree = &BTRFS_I(inode)->io_failure_tree;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
ret = __readpage_endio_check(inode, io_bio, i, bvec->bv_page,
bvec->bv_offset, done->start,
bvec->bv_len);
@@ -8031,7 +8031,7 @@ static blk_status_t __btrfs_subio_endio_read(struct inode *inode,
done.inode = inode;
io_bio->bio.bi_iter = io_bio->iter;
- bio_for_each_segment(bvec, &io_bio->bio, iter) {
+ bio_for_each_page(bvec, &io_bio->bio, iter) {
nr_sectors = BTRFS_BYTES_TO_BLKS(fs_info, bvec.bv_len);
pgoff = bvec.bv_offset;
diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c
index 9abd950e7f78..ab9d80f79ffe 100644
--- a/fs/btrfs/raid56.c
+++ b/fs/btrfs/raid56.c
@@ -1161,7 +1161,7 @@ static void index_rbio_pages(struct btrfs_raid_bio *rbio)
if (bio_flagged(bio, BIO_CLONED))
bio->bi_iter = btrfs_io_bio(bio)->iter;
- bio_for_each_segment(bvec, bio, iter) {
+ bio_for_each_page(bvec, bio, iter) {
rbio->bio_pages[page_index + i] = bvec.bv_page;
i++;
}
@@ -1448,7 +1448,7 @@ static void set_bio_pages_uptodate(struct bio *bio)
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_segment_all(bvec, bio, i)
+ bio_for_each_page_all(bvec, bio, i)
SetPageUptodate(bvec->bv_page);
}
diff --git a/fs/crypto/bio.c b/fs/crypto/bio.c
index 0d5e6a569d58..2dda77c3a89a 100644
--- a/fs/crypto/bio.c
+++ b/fs/crypto/bio.c
@@ -38,7 +38,7 @@ static void completion_pages(struct work_struct *work)
struct bio_vec *bv;
int i;
- bio_for_each_segment_all(bv, bio, i) {
+ bio_for_each_page_all(bv, bio, i) {
struct page *page = bv->bv_page;
int ret = fscrypt_decrypt_page(page->mapping->host, page,
PAGE_SIZE, 0, page->index);
diff --git a/fs/direct-io.c b/fs/direct-io.c
index 093fb54cd316..bbf25b0de9f8 100644
--- a/fs/direct-io.c
+++ b/fs/direct-io.c
@@ -551,7 +551,7 @@ static blk_status_t dio_bio_complete(struct dio *dio, struct bio *bio)
if (dio->is_async && dio->op == REQ_OP_READ && dio->should_dirty) {
bio_check_pages_dirty(bio); /* transfers ownership */
} else {
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
if (dio->op == REQ_OP_READ && !PageCompound(page) &&
diff --git a/fs/exofs/ore.c b/fs/exofs/ore.c
index ddbf87246898..fe81fd6fe553 100644
--- a/fs/exofs/ore.c
+++ b/fs/exofs/ore.c
@@ -407,7 +407,7 @@ static void _clear_bio(struct bio *bio)
struct bio_vec *bv;
unsigned i;
- bio_for_each_segment_all(bv, bio, i) {
+ bio_for_each_page_all(bv, bio, i) {
unsigned this_count = bv->bv_len;
if (likely(PAGE_SIZE == this_count))
diff --git a/fs/exofs/ore_raid.c b/fs/exofs/ore_raid.c
index 27cbdb697649..2c3346cd1b29 100644
--- a/fs/exofs/ore_raid.c
+++ b/fs/exofs/ore_raid.c
@@ -437,7 +437,7 @@ static void _mark_read4write_pages_uptodate(struct ore_io_state *ios, int ret)
if (!bio)
continue;
- bio_for_each_segment_all(bv, bio, i) {
+ bio_for_each_page_all(bv, bio, i) {
struct page *page = bv->bv_page;
SetPageUptodate(page);
diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c
index db7590178dfc..52f2937f5603 100644
--- a/fs/ext4/page-io.c
+++ b/fs/ext4/page-io.c
@@ -64,7 +64,7 @@ static void ext4_finish_bio(struct bio *bio)
int i;
struct bio_vec *bvec;
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
#ifdef CONFIG_EXT4_FS_ENCRYPTION
struct page *data_page = NULL;
diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c
index 9ffa6fad18db..572b6296f709 100644
--- a/fs/ext4/readpage.c
+++ b/fs/ext4/readpage.c
@@ -81,7 +81,7 @@ static void mpage_end_io(struct bio *bio)
return;
}
}
- bio_for_each_segment_all(bv, bio, i) {
+ bio_for_each_page_all(bv, bio, i) {
struct page *page = bv->bv_page;
if (!bio->bi_status) {
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 02237d4d91f5..89da84b0f0bd 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -71,7 +71,7 @@ static void f2fs_read_end_io(struct bio *bio)
}
}
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
if (!bio->bi_status) {
@@ -92,7 +92,7 @@ static void f2fs_write_end_io(struct bio *bio)
struct bio_vec *bvec;
int i;
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
enum count_type type = WB_DATA_TYPE(page);
@@ -274,7 +274,7 @@ static bool __has_merged_page(struct f2fs_bio_info *io,
if (!inode && !ino)
return true;
- bio_for_each_segment_all(bvec, io->bio, i) {
+ bio_for_each_page_all(bvec, io->bio, i) {
if (bvec->bv_page->mapping)
target = bvec->bv_page;
diff --git a/fs/gfs2/lops.c b/fs/gfs2/lops.c
index 4d6567990baf..a31b9b028957 100644
--- a/fs/gfs2/lops.c
+++ b/fs/gfs2/lops.c
@@ -214,7 +214,7 @@ static void gfs2_end_log_write(struct bio *bio)
wake_up(&sdp->sd_logd_waitq);
}
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
page = bvec->bv_page;
if (page_has_buffers(page))
gfs2_end_log_write_bh(sdp, bvec, bio->bi_status);
diff --git a/fs/gfs2/meta_io.c b/fs/gfs2/meta_io.c
index 52de1036d9f9..1d720352310a 100644
--- a/fs/gfs2/meta_io.c
+++ b/fs/gfs2/meta_io.c
@@ -191,7 +191,7 @@ static void gfs2_meta_read_endio(struct bio *bio)
struct bio_vec *bvec;
int i;
- bio_for_each_segment_all(bvec, bio, i) {
+ bio_for_each_page_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
struct buffer_head *bh = page_buffers(page);
unsigned int len = bvec->bv_len;
diff --git a/fs/iomap.c b/fs/iomap.c
index afd163586aa0..42b0b1697b3d 100644
--- a/fs/iomap.c
+++ b/fs/iomap.c
@@ -818,7 +818,7 @@ static void iomap_dio_bio_end_io(struct bio *bio)
struct bio_vec *bvec;
int i;
- bio_for_each_segment_all(bvec, bio, i)
+ bio_for_each_page_all(bvec, bio, i)
put_page(bvec->bv_page);
bio_put(bio);
}
diff --git a/fs/mpage.c b/fs/mpage.c
index b7e7f570733a..1cf322c4d6f8 100644
--- a/fs/mpage.c
+++ b/fs/mpage.c
@@ -49,7 +49,7 @@ static void mpage_end_io(struct bio *bio)
struct bio_vec *bv;
int i;
- bio_for_each_segment_all(bv, bio, i) {
+ bio_for_each_page_all(bv, bio, i) {
struct page *page = bv->bv_page;
page_endio(page, op_is_write(bio_op(bio)),
blk_status_to_errno(bio->bi_status));
diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
index 0ab824f574ed..13e2c167aec3 100644
--- a/fs/xfs/xfs_aops.c
+++ b/fs/xfs/xfs_aops.c
@@ -180,7 +180,7 @@ xfs_destroy_ioend(
next = bio->bi_private;
/* walk each page on bio, ending page IO on them */
- bio_for_each_segment_all(bvec, bio, i)
+ bio_for_each_page_all(bvec, bio, i)
xfs_finish_page_writeback(inode, bvec, error);
bio_put(bio);
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 98b175cc00d5..63b988043eff 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -157,7 +157,7 @@ static inline void *bio_data(struct bio *bio)
* drivers should _never_ use the all version - the bio may have been split
* before it got to the driver and the driver won't own all of it
*/
-#define bio_for_each_segment_all(bvl, bio, i) \
+#define bio_for_each_page_all(bvl, bio, i) \
for (i = 0, bvl = (bio)->bi_io_vec; i < (bio)->bi_vcnt; i++, bvl++)
static inline void bio_advance_iter(struct bio *bio, struct bvec_iter *iter,
@@ -188,14 +188,14 @@ static inline bool bio_rewind_iter(struct bio *bio, struct bvec_iter *iter,
return bvec_iter_rewind(bio->bi_io_vec, iter, bytes);
}
-#define __bio_for_each_segment(bvl, bio, iter, start) \
+#define __bio_for_each_page(bvl, bio, iter, start) \
for (iter = (start); \
(iter).bi_size && \
((bvl = bio_iter_iovec((bio), (iter))), 1); \
bio_advance_iter((bio), &(iter), (bvl).bv_len))
-#define bio_for_each_segment(bvl, bio, iter) \
- __bio_for_each_segment(bvl, bio, iter, (bio)->bi_iter)
+#define bio_for_each_page(bvl, bio, iter) \
+ __bio_for_each_page(bvl, bio, iter, (bio)->bi_iter)
#define bio_iter_last(bvec, iter) ((iter).bi_size == (bvec).bv_len)
@@ -221,7 +221,7 @@ static inline unsigned bio_segments(struct bio *bio)
break;
}
- bio_for_each_segment(bv, bio, iter)
+ bio_for_each_page(bv, bio, iter)
segs++;
return segs;
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index f3999719f828..dfa750fd7a41 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -951,7 +951,7 @@ struct req_iterator {
#define rq_for_each_segment(bvl, _rq, _iter) \
__rq_for_each_bio(_iter.bio, _rq) \
- bio_for_each_segment(bvl, _iter.bio, _iter.iter)
+ bio_for_each_page(bvl, _iter.bio, _iter.iter)
#define rq_iter_last(bvec, _iter) \
(_iter.bio->bi_next == NULL && \
diff --git a/include/linux/ceph/messenger.h b/include/linux/ceph/messenger.h
index c7dfcb8a1fb2..0f6d9dc28ce1 100644
--- a/include/linux/ceph/messenger.h
+++ b/include/linux/ceph/messenger.h
@@ -135,7 +135,7 @@ struct ceph_bio_iter {
\
__cur_iter = (it)->iter; \
__cur_iter.bi_size = __cur_n; \
- __bio_for_each_segment(bv, (it)->bio, __cur_iter, __cur_iter) \
+ __bio_for_each_page(bv, (it)->bio, __cur_iter, __cur_iter) \
(void)(BVEC_STEP); \
}))
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 02/33] block: rename rq_for_each_segment as rq_for_each_page
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 01/33] block: rename bio_for_each_segment* with bio_for_each_page* Ming Lei
@ 2018-05-25 3:45 ` Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 03/33] block: rename bio_segments() with bio_pages() Ming Lei
` (32 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:45 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
rq_for_each_segment() still deceives us since this helper only returns
one page in each bvec, so fixes its name.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
Documentation/block/biodoc.txt | 6 +++---
block/blk-core.c | 2 +-
drivers/block/floppy.c | 4 ++--
drivers/block/loop.c | 12 ++++++------
drivers/block/nbd.c | 2 +-
drivers/block/null_blk.c | 2 +-
drivers/block/ps3disk.c | 4 ++--
drivers/s390/block/dasd_diag.c | 4 ++--
drivers/s390/block/dasd_eckd.c | 16 ++++++++--------
drivers/s390/block/dasd_fba.c | 6 +++---
drivers/s390/block/scm_blk.c | 2 +-
include/linux/blkdev.h | 4 ++--
12 files changed, 32 insertions(+), 32 deletions(-)
diff --git a/Documentation/block/biodoc.txt b/Documentation/block/biodoc.txt
index 86927029a52d..3aeca60e526a 100644
--- a/Documentation/block/biodoc.txt
+++ b/Documentation/block/biodoc.txt
@@ -458,7 +458,7 @@ With this multipage bio design:
- A linked list of bios is used as before for unrelated merges (*) - this
avoids reallocs and makes independent completions easier to handle.
- Code that traverses the req list can find all the segments of a bio
- by using rq_for_each_segment. This handles the fact that a request
+ by using rq_for_each_page. This handles the fact that a request
has multiple bios, each of which can have multiple segments.
- Drivers which can't process a large bio in one shot can use the bi_iter
field to keep track of the next bio_vec entry to process.
@@ -640,13 +640,13 @@ in lvm or md.
3.2.1 Traversing segments and completion units in a request
-The macro rq_for_each_segment() should be used for traversing the bios
+The macro rq_for_each_page() should be used for traversing the bios
in the request list (drivers should avoid directly trying to do it
themselves). Using these helpers should also make it easier to cope
with block changes in the future.
struct req_iterator iter;
- rq_for_each_segment(bio_vec, rq, iter)
+ rq_for_each_page(bio_vec, rq, iter)
/* bio_vec is now current segment */
I/O completion callbacks are per-bio rather than per-segment, so drivers
diff --git a/block/blk-core.c b/block/blk-core.c
index 43370faee935..6548b9170ae5 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -3423,7 +3423,7 @@ void rq_flush_dcache_pages(struct request *rq)
struct req_iterator iter;
struct bio_vec bvec;
- rq_for_each_segment(bvec, rq, iter)
+ rq_for_each_page(bvec, rq, iter)
flush_dcache_page(bvec.bv_page);
}
EXPORT_SYMBOL_GPL(rq_flush_dcache_pages);
diff --git a/drivers/block/floppy.c b/drivers/block/floppy.c
index 8ec7235fc93b..922cc9d0120a 100644
--- a/drivers/block/floppy.c
+++ b/drivers/block/floppy.c
@@ -2382,7 +2382,7 @@ static int buffer_chain_size(void)
base = bio_data(current_req->bio);
size = 0;
- rq_for_each_segment(bv, current_req, iter) {
+ rq_for_each_page(bv, current_req, iter) {
if (page_address(bv.bv_page) + bv.bv_offset != base + size)
break;
@@ -2446,7 +2446,7 @@ static void copy_buffer(int ssize, int max_sector, int max_sector_2)
size = blk_rq_cur_bytes(current_req);
- rq_for_each_segment(bv, current_req, iter) {
+ rq_for_each_page(bv, current_req, iter) {
if (!remaining)
break;
diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index 5f0df2efc26c..d04ba3f0c5de 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -290,7 +290,7 @@ static int lo_write_simple(struct loop_device *lo, struct request *rq,
struct req_iterator iter;
int ret = 0;
- rq_for_each_segment(bvec, rq, iter) {
+ rq_for_each_page(bvec, rq, iter) {
ret = lo_write_bvec(lo->lo_backing_file, &bvec, &pos);
if (ret < 0)
break;
@@ -317,7 +317,7 @@ static int lo_write_transfer(struct loop_device *lo, struct request *rq,
if (unlikely(!page))
return -ENOMEM;
- rq_for_each_segment(bvec, rq, iter) {
+ rq_for_each_page(bvec, rq, iter) {
ret = lo_do_transfer(lo, WRITE, page, 0, bvec.bv_page,
bvec.bv_offset, bvec.bv_len, pos >> 9);
if (unlikely(ret))
@@ -343,7 +343,7 @@ static int lo_read_simple(struct loop_device *lo, struct request *rq,
struct iov_iter i;
ssize_t len;
- rq_for_each_segment(bvec, rq, iter) {
+ rq_for_each_page(bvec, rq, iter) {
iov_iter_bvec(&i, ITER_BVEC, &bvec, 1, bvec.bv_len);
len = vfs_iter_read(lo->lo_backing_file, &i, &pos, 0);
if (len < 0)
@@ -378,7 +378,7 @@ static int lo_read_transfer(struct loop_device *lo, struct request *rq,
if (unlikely(!page))
return -ENOMEM;
- rq_for_each_segment(bvec, rq, iter) {
+ rq_for_each_page(bvec, rq, iter) {
loff_t offset = pos;
b.bv_page = page;
@@ -530,10 +530,10 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd,
/*
* The bios of the request may be started from the middle of
* the 'bvec' because of bio splitting, so we can't directly
- * copy bio->bi_iov_vec to new bvec. The rq_for_each_segment
+ * copy bio->bi_iov_vec to new bvec. The rq_for_each_page
* API will take care of all details for us.
*/
- rq_for_each_segment(tmp, rq, iter) {
+ rq_for_each_page(tmp, rq, iter) {
*bvec = tmp;
bvec++;
}
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 52f683bb2b9a..939a8012d25f 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -600,7 +600,7 @@ static struct nbd_cmd *nbd_read_stat(struct nbd_device *nbd, int index)
struct req_iterator iter;
struct bio_vec bvec;
- rq_for_each_segment(bvec, req, iter) {
+ rq_for_each_page(bvec, req, iter) {
iov_iter_bvec(&to, ITER_BVEC | READ,
&bvec, 1, bvec.bv_len);
result = sock_xmit(nbd, index, 0, &to, MSG_WAITALL, NULL);
diff --git a/drivers/block/null_blk.c b/drivers/block/null_blk.c
index 506c74501114..6e95955576e2 100644
--- a/drivers/block/null_blk.c
+++ b/drivers/block/null_blk.c
@@ -1136,7 +1136,7 @@ static int null_handle_rq(struct nullb_cmd *cmd)
}
spin_lock_irq(&nullb->lock);
- rq_for_each_segment(bvec, rq, iter) {
+ rq_for_each_page(bvec, rq, iter) {
len = bvec.bv_len;
err = null_transfer(nullb, bvec.bv_page, len, bvec.bv_offset,
op_is_write(req_op(rq)), sector,
diff --git a/drivers/block/ps3disk.c b/drivers/block/ps3disk.c
index afe1508d82c6..8d816bee18ac 100644
--- a/drivers/block/ps3disk.c
+++ b/drivers/block/ps3disk.c
@@ -99,7 +99,7 @@ static void ps3disk_scatter_gather(struct ps3_storage_device *dev,
size_t size;
void *buf;
- rq_for_each_segment(bvec, req, iter) {
+ rq_for_each_page(bvec, req, iter) {
unsigned long flags;
dev_dbg(&dev->sbd.core, "%s:%u: bio %u: %u sectors from %lu\n",
__func__, __LINE__, i, bio_sectors(iter.bio),
@@ -132,7 +132,7 @@ static int ps3disk_submit_request_sg(struct ps3_storage_device *dev,
struct bio_vec bv;
struct req_iterator iter;
- rq_for_each_segment(bv, req, iter)
+ rq_for_each_page(bv, req, iter)
n++;
dev_dbg(&dev->sbd.core,
"%s:%u: %s req has %u bvecs for %u sectors\n",
diff --git a/drivers/s390/block/dasd_diag.c b/drivers/s390/block/dasd_diag.c
index 131f1989f6f3..02f154056153 100644
--- a/drivers/s390/block/dasd_diag.c
+++ b/drivers/s390/block/dasd_diag.c
@@ -524,7 +524,7 @@ static struct dasd_ccw_req *dasd_diag_build_cp(struct dasd_device *memdev,
(blk_rq_pos(req) + blk_rq_sectors(req) - 1) >> block->s2b_shift;
/* Check struct bio and count the number of blocks for the request. */
count = 0;
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
if (bv.bv_len & (blksize - 1))
/* Fba can only do full blocks. */
return ERR_PTR(-EINVAL);
@@ -544,7 +544,7 @@ static struct dasd_ccw_req *dasd_diag_build_cp(struct dasd_device *memdev,
dreq->block_count = count;
dbio = dreq->bio;
recid = first_rec;
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
dst = page_address(bv.bv_page) + bv.bv_offset;
for (off = 0; off < bv.bv_len; off += blksize) {
memset(dbio, 0, sizeof (struct dasd_diag_bio));
diff --git a/drivers/s390/block/dasd_eckd.c b/drivers/s390/block/dasd_eckd.c
index be208e7adcb4..7941ab50ed77 100644
--- a/drivers/s390/block/dasd_eckd.c
+++ b/drivers/s390/block/dasd_eckd.c
@@ -3065,7 +3065,7 @@ static struct dasd_ccw_req *dasd_eckd_build_cp_cmd_single(
/* Check struct bio and count the number of blocks for the request. */
count = 0;
cidaw = 0;
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
if (bv.bv_len & (blksize - 1))
/* Eckd can only do full blocks. */
return ERR_PTR(-EINVAL);
@@ -3140,7 +3140,7 @@ static struct dasd_ccw_req *dasd_eckd_build_cp_cmd_single(
locate_record(ccw++, LO_data++, first_trk, first_offs + 1,
last_rec - recid + 1, cmd, basedev, blksize);
}
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
dst = page_address(bv.bv_page) + bv.bv_offset;
if (dasd_page_cache) {
char *copy = kmem_cache_alloc(dasd_page_cache,
@@ -3299,7 +3299,7 @@ static struct dasd_ccw_req *dasd_eckd_build_cp_cmd_track(
len_to_track_end = 0;
idaw_dst = NULL;
idaw_len = 0;
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
dst = page_address(bv.bv_page) + bv.bv_offset;
seg_len = bv.bv_len;
while (seg_len) {
@@ -3587,7 +3587,7 @@ static struct dasd_ccw_req *dasd_eckd_build_cp_tpm_track(
*/
trkcount = last_trk - first_trk + 1;
ctidaw = 0;
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
++ctidaw;
}
if (rq_data_dir(req) == WRITE)
@@ -3636,7 +3636,7 @@ static struct dasd_ccw_req *dasd_eckd_build_cp_tpm_track(
if (rq_data_dir(req) == WRITE) {
new_track = 1;
recid = first_rec;
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
dst = page_address(bv.bv_page) + bv.bv_offset;
seg_len = bv.bv_len;
while (seg_len) {
@@ -3669,7 +3669,7 @@ static struct dasd_ccw_req *dasd_eckd_build_cp_tpm_track(
}
}
} else {
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
dst = page_address(bv.bv_page) + bv.bv_offset;
last_tidaw = itcw_add_tidaw(itcw, 0x00,
dst, bv.bv_len);
@@ -3897,7 +3897,7 @@ static struct dasd_ccw_req *dasd_eckd_build_cp_raw(struct dasd_device *startdev,
for (sectors = 0; sectors < start_padding_sectors; sectors += 8)
idaws = idal_create_words(idaws, rawpadpage, PAGE_SIZE);
}
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
dst = page_address(bv.bv_page) + bv.bv_offset;
seg_len = bv.bv_len;
if (cmd == DASD_ECKD_CCW_READ_TRACK)
@@ -3958,7 +3958,7 @@ dasd_eckd_free_cp(struct dasd_ccw_req *cqr, struct request *req)
ccw++;
if (private->uses_cdl == 0 || recid > 2*blk_per_trk)
ccw++;
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
dst = page_address(bv.bv_page) + bv.bv_offset;
for (off = 0; off < bv.bv_len; off += blksize) {
/* Skip locate record. */
diff --git a/drivers/s390/block/dasd_fba.c b/drivers/s390/block/dasd_fba.c
index a6b132f7e869..b1d86cda3784 100644
--- a/drivers/s390/block/dasd_fba.c
+++ b/drivers/s390/block/dasd_fba.c
@@ -465,7 +465,7 @@ static struct dasd_ccw_req *dasd_fba_build_cp_regular(
/* Check struct bio and count the number of blocks for the request. */
count = 0;
cidaw = 0;
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
if (bv.bv_len & (blksize - 1))
/* Fba can only do full blocks. */
return ERR_PTR(-EINVAL);
@@ -506,7 +506,7 @@ static struct dasd_ccw_req *dasd_fba_build_cp_regular(
locate_record(ccw++, LO_data++, rq_data_dir(req), 0, count);
}
recid = first_rec;
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
dst = page_address(bv.bv_page) + bv.bv_offset;
if (dasd_page_cache) {
char *copy = kmem_cache_alloc(dasd_page_cache,
@@ -588,7 +588,7 @@ dasd_fba_free_cp(struct dasd_ccw_req *cqr, struct request *req)
ccw++;
if (private->rdc_data.mode.bits.data_chain != 0)
ccw++;
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
dst = page_address(bv.bv_page) + bv.bv_offset;
for (off = 0; off < bv.bv_len; off += blksize) {
/* Skip locate record. */
diff --git a/drivers/s390/block/scm_blk.c b/drivers/s390/block/scm_blk.c
index b1fcb76dd272..68c0007f3ec0 100644
--- a/drivers/s390/block/scm_blk.c
+++ b/drivers/s390/block/scm_blk.c
@@ -198,7 +198,7 @@ static int scm_request_prepare(struct scm_request *scmrq)
msb->flags |= MSB_FLAG_IDA;
msb->data_addr = (u64) aidaw;
- rq_for_each_segment(bv, req, iter) {
+ rq_for_each_page(bv, req, iter) {
WARN_ON(bv.bv_offset);
msb->blk_count += bv.bv_len >> 12;
aidaw->data_addr = (u64) page_address(bv.bv_page);
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index dfa750fd7a41..1e8e9b430008 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -942,14 +942,14 @@ struct req_iterator {
struct bio *bio;
};
-/* This should not be used directly - use rq_for_each_segment */
+/* This should not be used directly - use rq_for_each_page */
#define for_each_bio(_bio) \
for (; _bio; _bio = _bio->bi_next)
#define __rq_for_each_bio(_bio, rq) \
if ((rq->bio)) \
for (_bio = (rq)->bio; _bio; _bio = _bio->bi_next)
-#define rq_for_each_segment(bvl, _rq, _iter) \
+#define rq_for_each_page(bvl, _rq, _iter) \
__rq_for_each_bio(_iter.bio, _rq) \
bio_for_each_page(bvl, _iter.bio, _iter.iter)
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 03/33] block: rename bio_segments() with bio_pages()
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 01/33] block: rename bio_for_each_segment* with bio_for_each_page* Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 02/33] block: rename rq_for_each_segment as rq_for_each_page Ming Lei
@ 2018-05-25 3:45 ` Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 04/33] block: introduce multipage page bvec helpers Ming Lei
` (31 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:45 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
bio_segments() never returns count of actual segment, just like
original bio_for_each_segment(), so rename it as bio_pages().
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
block/bio.c | 2 +-
block/blk-merge.c | 2 +-
drivers/block/loop.c | 4 ++--
drivers/md/dm-log-writes.c | 2 +-
drivers/target/target_core_pscsi.c | 2 +-
fs/btrfs/check-integrity.c | 2 +-
fs/btrfs/inode.c | 2 +-
include/linux/bio.h | 2 +-
8 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index 5495dc30d080..d0debb22ee34 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -682,7 +682,7 @@ struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
* __bio_clone_fast() anyways.
*/
- bio = bio_alloc_bioset(gfp_mask, bio_segments(bio_src), bs);
+ bio = bio_alloc_bioset(gfp_mask, bio_pages(bio_src), bs);
if (!bio)
return NULL;
bio->bi_disk = bio_src->bi_disk;
diff --git a/block/blk-merge.c b/block/blk-merge.c
index fc2aa21b7959..545609fc4905 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -302,7 +302,7 @@ void blk_recount_segments(struct request_queue *q, struct bio *bio)
/* estimate segment number by bi_vcnt for non-cloned bio */
if (bio_flagged(bio, BIO_CLONED))
- seg_cnt = bio_segments(bio);
+ seg_cnt = bio_pages(bio);
else
seg_cnt = bio->bi_vcnt;
diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index d04ba3f0c5de..8d7d5581ca9c 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -521,7 +521,7 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd,
struct bio_vec tmp;
__rq_for_each_bio(bio, rq)
- segments += bio_segments(bio);
+ segments += bio_pages(bio);
bvec = kmalloc(sizeof(struct bio_vec) * segments, GFP_NOIO);
if (!bvec)
return -EIO;
@@ -547,7 +547,7 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd,
*/
offset = bio->bi_iter.bi_bvec_done;
bvec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
- segments = bio_segments(bio);
+ segments = bio_pages(bio);
}
atomic_set(&cmd->ref, 2);
diff --git a/drivers/md/dm-log-writes.c b/drivers/md/dm-log-writes.c
index ab31ba9c3b37..e5d455245ed9 100644
--- a/drivers/md/dm-log-writes.c
+++ b/drivers/md/dm-log-writes.c
@@ -680,7 +680,7 @@ static int log_writes_map(struct dm_target *ti, struct bio *bio)
if (discard_bio)
alloc_size = sizeof(struct pending_block);
else
- alloc_size = sizeof(struct pending_block) + sizeof(struct bio_vec) * bio_segments(bio);
+ alloc_size = sizeof(struct pending_block) + sizeof(struct bio_vec) * bio_pages(bio);
block = kzalloc(alloc_size, GFP_NOIO);
if (!block) {
diff --git a/drivers/target/target_core_pscsi.c b/drivers/target/target_core_pscsi.c
index 668934ea74cb..01116cf9d634 100644
--- a/drivers/target/target_core_pscsi.c
+++ b/drivers/target/target_core_pscsi.c
@@ -915,7 +915,7 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
rc = bio_add_pc_page(pdv->pdv_sd->request_queue,
bio, page, bytes, off);
pr_debug("PSCSI: bio->bi_vcnt: %d nr_vecs: %d\n",
- bio_segments(bio), nr_vecs);
+ bio_pages(bio), nr_vecs);
if (rc != bytes) {
pr_debug("PSCSI: Reached bio->bi_vcnt max:"
" %d i: %d bio: %p, allocating another"
diff --git a/fs/btrfs/check-integrity.c b/fs/btrfs/check-integrity.c
index e5f7df09683f..55c4db22cfb9 100644
--- a/fs/btrfs/check-integrity.c
+++ b/fs/btrfs/check-integrity.c
@@ -2800,7 +2800,7 @@ static void __btrfsic_submit_bio(struct bio *bio)
struct bvec_iter iter;
int bio_is_patched;
char **mapped_datav;
- unsigned int segs = bio_segments(bio);
+ unsigned int segs = bio_pages(bio);
dev_bytenr = 512 * bio->bi_iter.bi_sector;
bio_is_patched = 0;
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index f78155e3a4dc..9d816dc725c4 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -7844,7 +7844,7 @@ static blk_status_t dio_read_error(struct inode *inode, struct bio *failed_bio,
return BLK_STS_IOERR;
}
- segs = bio_segments(failed_bio);
+ segs = bio_pages(failed_bio);
bio_get_first_bvec(failed_bio, &bvec);
if (segs > 1 ||
(bvec.bv_len > btrfs_inode_sectorsize(inode)))
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 63b988043eff..7f92af1299ad 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -199,7 +199,7 @@ static inline bool bio_rewind_iter(struct bio *bio, struct bvec_iter *iter,
#define bio_iter_last(bvec, iter) ((iter).bi_size == (bvec).bv_len)
-static inline unsigned bio_segments(struct bio *bio)
+static inline unsigned bio_pages(struct bio *bio)
{
unsigned segs = 0;
struct bio_vec bv;
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 04/33] block: introduce multipage page bvec helpers
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (2 preceding siblings ...)
2018-05-25 3:45 ` [RESEND PATCH V5 03/33] block: rename bio_segments() with bio_pages() Ming Lei
@ 2018-05-25 3:45 ` Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 05/33] block: introduce bio_for_each_segment() Ming Lei
` (30 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:45 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
This patch introduces helpers of 'bvec_iter_segment_*' for multipage
bvec(segment) support.
The introduced interfaces treate one bvec as real multipage segment,
for example, .bv_len is the total length of the multipage segment.
The existed helpers of bvec_iter_* are interfaces for supporting current
bvec iterator which is thought as singlepage only by drivers, fs, dm and
etc. These helpers will build singlepage bvec in flight, so users of
current bio/bvec iterator still can work well and needn't change even
though we store real multipage segment into bvec table.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
include/linux/bvec.h | 63 +++++++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 60 insertions(+), 3 deletions(-)
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index fe7a22dd133b..2433c73fa5ea 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -23,6 +23,44 @@
#include <linux/kernel.h>
#include <linux/bug.h>
#include <linux/errno.h>
+#include <linux/mm.h>
+
+/*
+ * What is multipage bvecs(segment)?
+ *
+ * - bvec stored in bio->bi_io_vec is always multipage(mp) style
+ *
+ * - bvec(struct bio_vec) represents one physically contiguous I/O
+ * buffer, now the buffer may include more than one pages since
+ * multipage(mp) bvec is supported, and all these pages represented
+ * by one bvec is physically contiguous. Before mp support, at most
+ * one page can be included in one bvec, we call it singlepage(sp)
+ * bvec.
+ *
+ * - .bv_page of th bvec represents the 1st page in the mp segment
+ *
+ * - .bv_offset of the bvec represents offset of the buffer in the bvec
+ *
+ * The effect on the current drivers/filesystem/dm/bcache/...:
+ *
+ * - almost everyone supposes that one bvec only includes one single
+ * page, so we keep the sp interface not changed, for example,
+ * bio_for_each_page() still returns bvec with single page
+ *
+ * - bio_for_each_page_all() will be changed to return singlepage
+ * bvec too
+ *
+ * - during iterating, iterator variable(struct bvec_iter) is always
+ * updated in multipage bvec style and that means bvec_iter_advance()
+ * is kept not changed
+ *
+ * - returned(copied) singlepage bvec is generated in flight by bvec
+ * helpers from the stored multipage bvec(segment)
+ *
+ * - In case that some components(such as iov_iter) need to support
+ * multipage segment, we introduce new helpers(bvec_iter_segment_*) for
+ * them.
+ */
/*
* was unsigned short, but we might as well be ready for > 64kB I/O pages
@@ -52,16 +90,35 @@ struct bvec_iter {
*/
#define __bvec_iter_bvec(bvec, iter) (&(bvec)[(iter).bi_idx])
-#define bvec_iter_page(bvec, iter) \
+#define bvec_iter_segment_page(bvec, iter) \
(__bvec_iter_bvec((bvec), (iter))->bv_page)
-#define bvec_iter_len(bvec, iter) \
+#define bvec_iter_segment_len(bvec, iter) \
min((iter).bi_size, \
__bvec_iter_bvec((bvec), (iter))->bv_len - (iter).bi_bvec_done)
-#define bvec_iter_offset(bvec, iter) \
+#define bvec_iter_segment_offset(bvec, iter) \
(__bvec_iter_bvec((bvec), (iter))->bv_offset + (iter).bi_bvec_done)
+#define bvec_iter_page_idx_in_seg(bvec, iter) \
+ (bvec_iter_segment_offset((bvec), (iter)) / PAGE_SIZE)
+
+/*
+ * <page, offset,length> of singlepage(sp) segment.
+ *
+ * This helpers will be implemented for building sp bvec in flight.
+ */
+#define bvec_iter_offset(bvec, iter) \
+ (bvec_iter_segment_offset((bvec), (iter)) % PAGE_SIZE)
+
+#define bvec_iter_len(bvec, iter) \
+ min_t(unsigned, bvec_iter_segment_len((bvec), (iter)), \
+ (PAGE_SIZE - (bvec_iter_offset((bvec), (iter)))))
+
+#define bvec_iter_page(bvec, iter) \
+ nth_page(bvec_iter_segment_page((bvec), (iter)), \
+ bvec_iter_page_idx_in_seg((bvec), (iter)))
+
#define bvec_iter_bvec(bvec, iter) \
((struct bio_vec) { \
.bv_page = bvec_iter_page((bvec), (iter)), \
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 05/33] block: introduce bio_for_each_segment()
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (3 preceding siblings ...)
2018-05-25 3:45 ` [RESEND PATCH V5 04/33] block: introduce multipage page bvec helpers Ming Lei
@ 2018-05-25 3:45 ` Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 06/33] block: use bio_for_each_segment() to compute segments count Ming Lei
` (29 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:45 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
This helper is used to iterate multipage bvec for bio spliting/merge,
and it is required in bio_clone_bioset() too, so introduce it.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
include/linux/bio.h | 34 +++++++++++++++++++++++++++++++---
include/linux/bvec.h | 36 ++++++++++++++++++++++++++++++++----
2 files changed, 63 insertions(+), 7 deletions(-)
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 7f92af1299ad..3d3795b9a353 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -80,6 +80,9 @@
#define bio_data_dir(bio) \
(op_is_write(bio_op(bio)) ? WRITE : READ)
+#define bio_iter_seg_iovec(bio, iter) \
+ bvec_iter_segment_bvec((bio)->bi_io_vec, (iter))
+
/*
* Check whether this bio carries any data or not. A NULL bio is allowed.
*/
@@ -160,8 +163,8 @@ static inline void *bio_data(struct bio *bio)
#define bio_for_each_page_all(bvl, bio, i) \
for (i = 0, bvl = (bio)->bi_io_vec; i < (bio)->bi_vcnt; i++, bvl++)
-static inline void bio_advance_iter(struct bio *bio, struct bvec_iter *iter,
- unsigned bytes)
+static inline void __bio_advance_iter(struct bio *bio, struct bvec_iter *iter,
+ unsigned bytes, bool seg)
{
iter->bi_sector += bytes >> 9;
@@ -169,11 +172,26 @@ static inline void bio_advance_iter(struct bio *bio, struct bvec_iter *iter,
iter->bi_size -= bytes;
iter->bi_done += bytes;
} else {
- bvec_iter_advance(bio->bi_io_vec, iter, bytes);
+ if (!seg)
+ bvec_iter_advance(bio->bi_io_vec, iter, bytes);
+ else
+ bvec_iter_seg_advance(bio->bi_io_vec, iter, bytes);
/* TODO: It is reasonable to complete bio with error here. */
}
}
+static inline void bio_advance_iter(struct bio *bio, struct bvec_iter *iter,
+ unsigned bytes)
+{
+ __bio_advance_iter(bio, iter, bytes, false);
+}
+
+static inline void bio_advance_seg_iter(struct bio *bio, struct bvec_iter *iter,
+ unsigned bytes)
+{
+ __bio_advance_iter(bio, iter, bytes, true);
+}
+
static inline bool bio_rewind_iter(struct bio *bio, struct bvec_iter *iter,
unsigned int bytes)
{
@@ -197,6 +215,16 @@ static inline bool bio_rewind_iter(struct bio *bio, struct bvec_iter *iter,
#define bio_for_each_page(bvl, bio, iter) \
__bio_for_each_page(bvl, bio, iter, (bio)->bi_iter)
+#define __bio_for_each_segment(bvl, bio, iter, start) \
+ for (iter = (start); \
+ (iter).bi_size && \
+ ((bvl = bio_iter_seg_iovec((bio), (iter))), 1); \
+ bio_advance_seg_iter((bio), &(iter), (bvl).bv_len))
+
+/* returns one real segment(multipage bvec) each time */
+#define bio_for_each_segment(bvl, bio, iter) \
+ __bio_for_each_segment(bvl, bio, iter, (bio)->bi_iter)
+
#define bio_iter_last(bvec, iter) ((iter).bi_size == (bvec).bv_len)
static inline unsigned bio_pages(struct bio *bio)
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index 2433c73fa5ea..84c395feed49 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -126,8 +126,16 @@ struct bvec_iter {
.bv_offset = bvec_iter_offset((bvec), (iter)), \
})
-static inline bool bvec_iter_advance(const struct bio_vec *bv,
- struct bvec_iter *iter, unsigned bytes)
+#define bvec_iter_segment_bvec(bvec, iter) \
+((struct bio_vec) { \
+ .bv_page = bvec_iter_segment_page((bvec), (iter)), \
+ .bv_len = bvec_iter_segment_len((bvec), (iter)), \
+ .bv_offset = bvec_iter_segment_offset((bvec), (iter)), \
+})
+
+static inline bool __bvec_iter_advance(const struct bio_vec *bv,
+ struct bvec_iter *iter,
+ unsigned bytes, bool segment)
{
if (WARN_ONCE(bytes > iter->bi_size,
"Attempted to advance past end of bvec iter\n")) {
@@ -136,8 +144,14 @@ static inline bool bvec_iter_advance(const struct bio_vec *bv,
}
while (bytes) {
- unsigned iter_len = bvec_iter_len(bv, *iter);
- unsigned len = min(bytes, iter_len);
+ unsigned len;
+
+ if (segment)
+ len = bvec_iter_segment_len(bv, *iter);
+ else
+ len = bvec_iter_len(bv, *iter);
+
+ len = min(bytes, len);
bytes -= len;
iter->bi_size -= len;
@@ -176,6 +190,20 @@ static inline bool bvec_iter_rewind(const struct bio_vec *bv,
return true;
}
+static inline bool bvec_iter_advance(const struct bio_vec *bv,
+ struct bvec_iter *iter,
+ unsigned bytes)
+{
+ return __bvec_iter_advance(bv, iter, bytes, false);
+}
+
+static inline bool bvec_iter_seg_advance(const struct bio_vec *bv,
+ struct bvec_iter *iter,
+ unsigned bytes)
+{
+ return __bvec_iter_advance(bv, iter, bytes, true);
+}
+
#define for_each_bvec(bvl, bio_vec, iter, start) \
for (iter = (start); \
(iter).bi_size && \
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 06/33] block: use bio_for_each_segment() to compute segments count
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (4 preceding siblings ...)
2018-05-25 3:45 ` [RESEND PATCH V5 05/33] block: introduce bio_for_each_segment() Ming Lei
@ 2018-05-25 3:45 ` Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 07/33] block: use bio_for_each_segment() to map sg Ming Lei
` (28 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:45 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
Firstly it is more efficient to use bio_for_each_segment() in both
blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how many
segments there are in the bio.
Secondaly once bio_for_each_segment() is used, the bvec may need to
be splitted because its length can be very longer than max segment size,
so we have to split the big bvec into several segments.
Thirdly during splitting multipage bvec into segments, max segment number
may be reached, then the bio need to be splitted when this happens.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
block/blk-merge.c | 90 ++++++++++++++++++++++++++++++++++++++++++++++---------
1 file changed, 76 insertions(+), 14 deletions(-)
diff --git a/block/blk-merge.c b/block/blk-merge.c
index 545609fc4905..d157b752d965 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -97,6 +97,62 @@ static inline unsigned get_max_io_size(struct request_queue *q,
return sectors;
}
+/*
+ * Split the bvec @bv into segments, and update all kinds of
+ * variables.
+ */
+static bool bvec_split_segs(struct request_queue *q, struct bio_vec *bv,
+ unsigned *nsegs, unsigned *last_seg_size,
+ unsigned *front_seg_size, unsigned *sectors)
+{
+ bool need_split = false;
+ unsigned len = bv->bv_len;
+ unsigned total_len = 0;
+ unsigned new_nsegs = 0, seg_size = 0;
+
+ if ((*nsegs >= queue_max_segments(q)) || !len)
+ return need_split;
+
+ /*
+ * Multipage bvec may be too big to hold in one segment,
+ * so the current bvec has to be splitted as multiple
+ * segments.
+ */
+ while (new_nsegs + *nsegs < queue_max_segments(q)) {
+ seg_size = min(queue_max_segment_size(q), len);
+
+ new_nsegs++;
+ total_len += seg_size;
+ len -= seg_size;
+
+ if ((queue_virt_boundary(q) && ((bv->bv_offset +
+ total_len) & queue_virt_boundary(q))) || !len)
+ break;
+ }
+
+ /* split in the middle of the bvec */
+ if (len)
+ need_split = true;
+
+ /* update front segment size */
+ if (!*nsegs) {
+ unsigned first_seg_size = seg_size;
+
+ if (new_nsegs > 1)
+ first_seg_size = queue_max_segment_size(q);
+ if (*front_seg_size < first_seg_size)
+ *front_seg_size = first_seg_size;
+ }
+
+ /* update other varibles */
+ *last_seg_size = seg_size;
+ *nsegs += new_nsegs;
+ if (sectors)
+ *sectors += total_len >> 9;
+
+ return need_split;
+}
+
static struct bio *blk_bio_segment_split(struct request_queue *q,
struct bio *bio,
struct bio_set *bs,
@@ -110,7 +166,7 @@ static struct bio *blk_bio_segment_split(struct request_queue *q,
struct bio *new = NULL;
const unsigned max_sectors = get_max_io_size(q, bio);
- bio_for_each_page(bv, bio, iter) {
+ bio_for_each_segment(bv, bio, iter) {
/*
* If the queue doesn't support SG gaps and adding this
* offset would create a gap, disallow it.
@@ -125,8 +181,12 @@ static struct bio *blk_bio_segment_split(struct request_queue *q,
*/
if (nsegs < queue_max_segments(q) &&
sectors < max_sectors) {
- nsegs++;
- sectors = max_sectors;
+ /* split in the middle of bvec */
+ bv.bv_len = (max_sectors - sectors) << 9;
+ bvec_split_segs(q, &bv, &nsegs,
+ &seg_size,
+ &front_seg_size,
+ §ors);
}
goto split;
}
@@ -153,11 +213,12 @@ static struct bio *blk_bio_segment_split(struct request_queue *q,
if (nsegs == 1 && seg_size > front_seg_size)
front_seg_size = seg_size;
- nsegs++;
bvprv = bv;
bvprvp = &bvprv;
- seg_size = bv.bv_len;
- sectors += bv.bv_len >> 9;
+
+ if (bvec_split_segs(q, &bv, &nsegs, &seg_size,
+ &front_seg_size, §ors))
+ goto split;
}
@@ -225,6 +286,7 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q,
struct bio_vec bv, bvprv = { NULL };
int cluster, prev = 0;
unsigned int seg_size, nr_phys_segs;
+ unsigned front_seg_size = bio->bi_seg_front_size;
struct bio *fbio, *bbio;
struct bvec_iter iter;
@@ -245,7 +307,7 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q,
seg_size = 0;
nr_phys_segs = 0;
for_each_bio(bio) {
- bio_for_each_page(bv, bio, iter) {
+ bio_for_each_segment(bv, bio, iter) {
/*
* If SG merging is disabled, each bio vector is
* a segment
@@ -267,20 +329,20 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q,
continue;
}
new_segment:
- if (nr_phys_segs == 1 && seg_size >
- fbio->bi_seg_front_size)
- fbio->bi_seg_front_size = seg_size;
+ if (nr_phys_segs == 1 && seg_size > front_seg_size)
+ front_seg_size = seg_size;
- nr_phys_segs++;
bvprv = bv;
prev = 1;
- seg_size = bv.bv_len;
+ bvec_split_segs(q, &bv, &nr_phys_segs, &seg_size,
+ &front_seg_size, NULL);
}
bbio = bio;
}
- if (nr_phys_segs == 1 && seg_size > fbio->bi_seg_front_size)
- fbio->bi_seg_front_size = seg_size;
+ if (nr_phys_segs == 1 && seg_size > front_seg_size)
+ front_seg_size = seg_size;
+ fbio->bi_seg_front_size = front_seg_size;
if (seg_size > bbio->bi_seg_back_size)
bbio->bi_seg_back_size = seg_size;
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 07/33] block: use bio_for_each_segment() to map sg
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (5 preceding siblings ...)
2018-05-25 3:45 ` [RESEND PATCH V5 06/33] block: use bio_for_each_segment() to compute segments count Ming Lei
@ 2018-05-25 3:45 ` Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 08/33] block: introduce segment_last_page() Ming Lei
` (27 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:45 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
It is more efficient to use bio_for_each_segment() to map sg, meantime
we have to consider splitting multipage bvec as done in blk_bio_segment_split().
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
block/blk-merge.c | 72 +++++++++++++++++++++++++++++++++++++++----------------
1 file changed, 52 insertions(+), 20 deletions(-)
diff --git a/block/blk-merge.c b/block/blk-merge.c
index d157b752d965..9fc96c9f6061 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -414,6 +414,56 @@ static int blk_phys_contig_segment(struct request_queue *q, struct bio *bio,
return 0;
}
+static struct scatterlist *blk_next_sg(struct scatterlist **sg,
+ struct scatterlist *sglist)
+{
+ if (!*sg)
+ return sglist;
+ else {
+ /*
+ * If the driver previously mapped a shorter
+ * list, we could see a termination bit
+ * prematurely unless it fully inits the sg
+ * table on each mapping. We KNOW that there
+ * must be more entries here or the driver
+ * would be buggy, so force clear the
+ * termination bit to avoid doing a full
+ * sg_init_table() in drivers for each command.
+ */
+ sg_unmark_end(*sg);
+ return sg_next(*sg);
+ }
+}
+
+static unsigned blk_bvec_map_sg(struct request_queue *q,
+ struct bio_vec *bvec, struct scatterlist *sglist,
+ struct scatterlist **sg)
+{
+ unsigned nbytes = bvec->bv_len;
+ unsigned nsegs = 0, total = 0;
+
+ while (nbytes > 0) {
+ unsigned seg_size;
+ struct page *pg;
+ unsigned offset, idx;
+
+ *sg = blk_next_sg(sg, sglist);
+
+ seg_size = min(nbytes, queue_max_segment_size(q));
+ offset = (total + bvec->bv_offset) % PAGE_SIZE;
+ idx = (total + bvec->bv_offset) / PAGE_SIZE;
+ pg = nth_page(bvec->bv_page, idx);
+
+ sg_set_page(*sg, pg, seg_size, offset);
+
+ total += seg_size;
+ nbytes -= seg_size;
+ nsegs++;
+ }
+
+ return nsegs;
+}
+
static inline void
__blk_segment_map_sg(struct request_queue *q, struct bio_vec *bvec,
struct scatterlist *sglist, struct bio_vec *bvprv,
@@ -434,25 +484,7 @@ __blk_segment_map_sg(struct request_queue *q, struct bio_vec *bvec,
(*sg)->length += nbytes;
} else {
new_segment:
- if (!*sg)
- *sg = sglist;
- else {
- /*
- * If the driver previously mapped a shorter
- * list, we could see a termination bit
- * prematurely unless it fully inits the sg
- * table on each mapping. We KNOW that there
- * must be more entries here or the driver
- * would be buggy, so force clear the
- * termination bit to avoid doing a full
- * sg_init_table() in drivers for each command.
- */
- sg_unmark_end(*sg);
- *sg = sg_next(*sg);
- }
-
- sg_set_page(*sg, bvec->bv_page, nbytes, bvec->bv_offset);
- (*nsegs)++;
+ (*nsegs) += blk_bvec_map_sg(q, bvec, sglist, sg);
}
*bvprv = *bvec;
}
@@ -474,7 +506,7 @@ static int __blk_bios_map_sg(struct request_queue *q, struct bio *bio,
int cluster = blk_queue_cluster(q), nsegs = 0;
for_each_bio(bio)
- bio_for_each_page(bvec, bio, iter)
+ bio_for_each_segment(bvec, bio, iter)
__blk_segment_map_sg(q, &bvec, sglist, &bvprv, sg,
&nsegs, &cluster);
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 08/33] block: introduce segment_last_page()
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (6 preceding siblings ...)
2018-05-25 3:45 ` [RESEND PATCH V5 07/33] block: use bio_for_each_segment() to map sg Ming Lei
@ 2018-05-25 3:45 ` Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 09/33] fs/buffer.c: use bvec iterator to truncate the bio Ming Lei
` (26 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:45 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
BTRFS and guard_bio_eod() need to get the last page from one segment, so
introduce this helper to make them happy.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
include/linux/bvec.h | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index 84c395feed49..217afcd83a15 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -219,4 +219,26 @@ static inline bool bvec_iter_seg_advance(const struct bio_vec *bv,
.bi_bvec_done = 0, \
}
+/* get the last page from the multipage bvec and store it in @pg */
+static inline void segment_last_page(const struct bio_vec *seg,
+ struct bio_vec *pg)
+{
+ unsigned total = seg->bv_offset + seg->bv_len;
+ unsigned last_page = total / PAGE_SIZE;
+
+ if (last_page * PAGE_SIZE == total)
+ last_page--;
+
+ pg->bv_page = nth_page(seg->bv_page, last_page);
+
+ /* the whole segment is inside the last page */
+ if (seg->bv_offset >= last_page * PAGE_SIZE) {
+ pg->bv_offset = seg->bv_offset % PAGE_SIZE;
+ pg->bv_len = seg->bv_len;
+ } else {
+ pg->bv_offset = 0;
+ pg->bv_len = total - last_page * PAGE_SIZE;
+ }
+}
+
#endif /* __LINUX_BVEC_ITER_H */
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 09/33] fs/buffer.c: use bvec iterator to truncate the bio
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (7 preceding siblings ...)
2018-05-25 3:45 ` [RESEND PATCH V5 08/33] block: introduce segment_last_page() Ming Lei
@ 2018-05-25 3:45 ` Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 10/33] btrfs: use segment_last_page to get bio's last page Ming Lei
` (25 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:45 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
Once multipage bvec is enabled, the last bvec may include more than one
page, this patch use segment_last_page() to truncate the bio.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
fs/buffer.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/fs/buffer.c b/fs/buffer.c
index 249b83fafe48..da4b19f7b67b 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -3021,7 +3021,10 @@ void guard_bio_eod(int op, struct bio *bio)
/* ..and clear the end of the buffer for reads */
if (op == REQ_OP_READ) {
- zero_user(bvec->bv_page, bvec->bv_offset + bvec->bv_len,
+ struct bio_vec bv;
+
+ segment_last_page(bvec, &bv);
+ zero_user(bv.bv_page, bv.bv_offset + bv.bv_len,
truncated_bytes);
}
}
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 10/33] btrfs: use segment_last_page to get bio's last page
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (8 preceding siblings ...)
2018-05-25 3:45 ` [RESEND PATCH V5 09/33] fs/buffer.c: use bvec iterator to truncate the bio Ming Lei
@ 2018-05-25 3:45 ` Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 11/33] block: implement bio_pages_all() via bio_for_each_page_all() Ming Lei
` (24 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:45 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei, Chris Mason, Josef Bacik,
David Sterba, linux-btrfs
Preparing for supporting multipage bvec.
Cc: Chris Mason <clm@fb.com>
Cc: Josef Bacik <jbacik@fb.com>
Cc: David Sterba <dsterba@suse.com>
Cc: linux-btrfs@vger.kernel.org
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
fs/btrfs/compression.c | 5 ++++-
fs/btrfs/extent_io.c | 5 +++--
2 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c
index 4ea718bdfb41..be6b09dfd6a7 100644
--- a/fs/btrfs/compression.c
+++ b/fs/btrfs/compression.c
@@ -407,8 +407,11 @@ blk_status_t btrfs_submit_compressed_write(struct inode *inode, u64 start,
static u64 bio_end_offset(struct bio *bio)
{
struct bio_vec *last = bio_last_bvec_all(bio);
+ struct bio_vec bv;
- return page_offset(last->bv_page) + last->bv_len + last->bv_offset;
+ segment_last_page(last, &bv);
+
+ return page_offset(bv.bv_page) + bv.bv_len + bv.bv_offset;
}
static noinline int add_ra_bio_pages(struct inode *inode,
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 4f314d87ce4d..3c9c91a1e3e9 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -2731,11 +2731,12 @@ static int __must_check submit_one_bio(struct bio *bio, int mirror_num,
{
blk_status_t ret = 0;
struct bio_vec *bvec = bio_last_bvec_all(bio);
- struct page *page = bvec->bv_page;
+ struct bio_vec bv;
struct extent_io_tree *tree = bio->bi_private;
u64 start;
- start = page_offset(page) + bvec->bv_offset;
+ segment_last_page(bvec, &bv);
+ start = page_offset(bv.bv_page) + bv.bv_offset;
bio->bi_private = NULL;
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 11/33] block: implement bio_pages_all() via bio_for_each_page_all()
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (9 preceding siblings ...)
2018-05-25 3:45 ` [RESEND PATCH V5 10/33] btrfs: use segment_last_page to get bio's last page Ming Lei
@ 2018-05-25 3:45 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 12/33] block: introduce bio_segments() Ming Lei
` (23 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:45 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
As multipage bvec will be enabled soon, bio->bi_vcnt isn't same with
page count in the bio any more, so use bio_for_each_page_all() to
compute the number.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
include/linux/bio.h | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 3d3795b9a353..08af9272687f 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -334,8 +334,14 @@ static inline void bio_get_last_bvec(struct bio *bio, struct bio_vec *bv)
static inline unsigned bio_pages_all(struct bio *bio)
{
+ unsigned i;
+ struct bio_vec *bv;
+
WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED));
- return bio->bi_vcnt;
+
+ bio_for_each_page_all(bv, bio, i)
+ ;
+ return i;
}
static inline struct bio_vec *bio_first_bvec_all(struct bio *bio)
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 12/33] block: introduce bio_segments()
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (10 preceding siblings ...)
2018-05-25 3:45 ` [RESEND PATCH V5 11/33] block: implement bio_pages_all() via bio_for_each_page_all() Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 4:42 ` Kent Overstreet
2018-05-25 3:46 ` [RESEND PATCH V5 13/33] block: introduce rq_for_each_segment() Ming Lei
` (22 subsequent siblings)
34 siblings, 1 reply; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
There are still cases in which we need to use bio_segments() for get the
number of segment, so introduce it.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
include/linux/bio.h | 25 ++++++++++++++++++++-----
1 file changed, 20 insertions(+), 5 deletions(-)
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 08af9272687f..b24c00f99c9c 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -227,9 +227,9 @@ static inline bool bio_rewind_iter(struct bio *bio, struct bvec_iter *iter,
#define bio_iter_last(bvec, iter) ((iter).bi_size == (bvec).bv_len)
-static inline unsigned bio_pages(struct bio *bio)
+static inline unsigned __bio_elements(struct bio *bio, bool seg)
{
- unsigned segs = 0;
+ unsigned elems = 0;
struct bio_vec bv;
struct bvec_iter iter;
@@ -249,10 +249,25 @@ static inline unsigned bio_pages(struct bio *bio)
break;
}
- bio_for_each_page(bv, bio, iter)
- segs++;
+ if (!seg) {
+ bio_for_each_page(bv, bio, iter)
+ elems++;
+ } else {
+ bio_for_each_segment(bv, bio, iter)
+ elems++;
+ }
+
+ return elems;
+}
+
+static inline unsigned bio_pages(struct bio *bio)
+{
+ return __bio_elements(bio, false);
+}
- return segs;
+static inline unsigned bio_segments(struct bio *bio)
+{
+ return __bio_elements(bio, true);
}
/*
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 13/33] block: introduce rq_for_each_segment()
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (11 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 12/33] block: introduce bio_segments() Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 14/33] block: loop: pass segments to iov_iter Ming Lei
` (21 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
There are still cases in which rq_for_each_segment() is required, for
example, loop.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
include/linux/blkdev.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 1e8e9b430008..0b15bc625bd7 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -953,6 +953,10 @@ struct req_iterator {
__rq_for_each_bio(_iter.bio, _rq) \
bio_for_each_page(bvl, _iter.bio, _iter.iter)
+#define rq_for_each_segment(bvl, _rq, _iter) \
+ __rq_for_each_bio(_iter.bio, _rq) \
+ bio_for_each_segment(bvl, _iter.bio, _iter.iter)
+
#define rq_iter_last(bvec, _iter) \
(_iter.bio->bi_next == NULL && \
bio_iter_last(bvec, _iter.iter))
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 14/33] block: loop: pass segments to iov_iter
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (12 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 13/33] block: introduce rq_for_each_segment() Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 15/33] block: introduce bio_clone_seg_bioset() Ming Lei
` (20 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
iov_iter is implemented with bvec itererator, so it is safe to pass
segment to it, and this way is much more efficient than passing one
page in each bvec.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
drivers/block/loop.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index 8d7d5581ca9c..e709c0380566 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -521,7 +521,7 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd,
struct bio_vec tmp;
__rq_for_each_bio(bio, rq)
- segments += bio_pages(bio);
+ segments += bio_segments(bio);
bvec = kmalloc(sizeof(struct bio_vec) * segments, GFP_NOIO);
if (!bvec)
return -EIO;
@@ -533,7 +533,7 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd,
* copy bio->bi_iov_vec to new bvec. The rq_for_each_page
* API will take care of all details for us.
*/
- rq_for_each_page(tmp, rq, iter) {
+ rq_for_each_segment(tmp, rq, iter) {
*bvec = tmp;
bvec++;
}
@@ -547,7 +547,7 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd,
*/
offset = bio->bi_iter.bi_bvec_done;
bvec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
- segments = bio_pages(bio);
+ segments = bio_segments(bio);
}
atomic_set(&cmd->ref, 2);
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 15/33] block: introduce bio_clone_seg_bioset()
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (13 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 14/33] block: loop: pass segments to iov_iter Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 16/33] dm: clone bio via bio_clone_seg_bioset Ming Lei
` (19 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
There is one use case(DM) which requires to clone bio segment by
segement, so introduce this API.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
block/bio.c | 56 +++++++++++++++++++++++++++++++++++++++--------------
include/linux/bio.h | 1 +
2 files changed, 43 insertions(+), 14 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index d0debb22ee34..63d4fe85f42e 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -644,21 +644,13 @@ struct bio *bio_clone_fast(struct bio *bio, gfp_t gfp_mask, struct bio_set *bs)
}
EXPORT_SYMBOL(bio_clone_fast);
-/**
- * bio_clone_bioset - clone a bio
- * @bio_src: bio to clone
- * @gfp_mask: allocation priority
- * @bs: bio_set to allocate from
- *
- * Clone bio. Caller will own the returned bio, but not the actual data it
- * points to. Reference count of returned bio will be one.
- */
-struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
- struct bio_set *bs)
+static struct bio *__bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
+ struct bio_set *bs, bool page)
{
struct bvec_iter iter;
struct bio_vec bv;
struct bio *bio;
+ int nr_vecs = page ? bio_pages(bio_src) : bio_segments(bio_src);
/*
* Pre immutable biovecs, __bio_clone() used to just do a memcpy from
@@ -682,7 +674,7 @@ struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
* __bio_clone_fast() anyways.
*/
- bio = bio_alloc_bioset(gfp_mask, bio_pages(bio_src), bs);
+ bio = bio_alloc_bioset(gfp_mask, nr_vecs, bs);
if (!bio)
return NULL;
bio->bi_disk = bio_src->bi_disk;
@@ -700,8 +692,13 @@ struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
bio->bi_io_vec[bio->bi_vcnt++] = bio_src->bi_io_vec[0];
break;
default:
- bio_for_each_page(bv, bio_src, iter)
- bio->bi_io_vec[bio->bi_vcnt++] = bv;
+ if (page) {
+ bio_for_each_page(bv, bio_src, iter)
+ bio->bi_io_vec[bio->bi_vcnt++] = bv;
+ } else {
+ bio_for_each_segment(bv, bio_src, iter)
+ bio->bi_io_vec[bio->bi_vcnt++] = bv;
+ }
break;
}
@@ -719,9 +716,40 @@ struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
return bio;
}
+
+/**
+ * bio_clone_bioset - clone a bio
+ * @bio_src: bio to clone
+ * @gfp_mask: allocation priority
+ * @bs: bio_set to allocate from
+ *
+ * Clone bio. Caller will own the returned bio, but not the actual data it
+ * points to. Reference count of returned bio will be one.
+ */
+struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask,
+ struct bio_set *bs)
+{
+ return __bio_clone_bioset(bio_src, gfp_mask, bs, true);
+}
EXPORT_SYMBOL(bio_clone_bioset);
/**
+ * bio_clone_seg_bioset - clone a bio segment by segment
+ * @bio_src: bio to clone
+ * @gfp_mask: allocation priority
+ * @bs: bio_set to allocate from
+ *
+ * Clone bio. Caller will own the returned bio, but not the actual data it
+ * points to. Reference count of returned bio will be one.
+ */
+struct bio *bio_clone_seg_bioset(struct bio *bio_src, gfp_t gfp_mask,
+ struct bio_set *bs)
+{
+ return __bio_clone_bioset(bio_src, gfp_mask, bs, false);
+}
+EXPORT_SYMBOL(bio_clone_seg_bioset);
+
+/**
* bio_add_pc_page - attempt to add page to bio
* @q: the target queue
* @bio: destination bio
diff --git a/include/linux/bio.h b/include/linux/bio.h
index b24c00f99c9c..61a04c131641 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -475,6 +475,7 @@ extern void bio_put(struct bio *);
extern void __bio_clone_fast(struct bio *, struct bio *);
extern struct bio *bio_clone_fast(struct bio *, gfp_t, struct bio_set *);
extern struct bio *bio_clone_bioset(struct bio *, gfp_t, struct bio_set *bs);
+extern struct bio *bio_clone_seg_bioset(struct bio *, gfp_t, struct bio_set *bs);
extern struct bio_set fs_bio_set;
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 16/33] dm: clone bio via bio_clone_seg_bioset
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (14 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 15/33] block: introduce bio_clone_seg_bioset() Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 17/33] block: bio: introduce bio_for_each_page_all2 and bio_for_each_segment_all Ming Lei
` (18 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
The incoming bio will become very big after multipage bvec is enabled,
so we can't clone bio page by page.
This patch uses the introduced bio_clone_seg_bioset(), so the incoming
bio can be cloned successfully. This way is safe because device mapping
won't modify the bio vector on the cloned multipage bio.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
drivers/md/dm.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index f1db181e082e..425e99e20f5c 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -1581,8 +1581,8 @@ static blk_qc_t __split_and_process_bio(struct mapped_device *md,
* the usage of io->orig_bio in dm_remap_zone_report()
* won't be affected by this reassignment.
*/
- struct bio *b = bio_clone_bioset(bio, GFP_NOIO,
- md->queue->bio_split);
+ struct bio *b = bio_clone_seg_bioset(bio,
+ GFP_NOIO, md->queue->bio_split);
ci.io->orig_bio = b;
bio_advance(bio, (bio_sectors(bio) - ci.sector_count) << 9);
bio_chain(b, bio);
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 17/33] block: bio: introduce bio_for_each_page_all2 and bio_for_each_segment_all
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (15 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 16/33] dm: clone bio via bio_clone_seg_bioset Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 18/33] block: deal with dirtying pages for multipage bvec Ming Lei
` (17 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
This patch introduces bio_for_each_page_all2(), which is for replacing
bio_for_each_page_all() in case that the returned bvec has to be single
page bvec.
Given the interface type has to be changed for passing one local iterator
variable of 'bvec_iter_all', and doing all changes in one single patch
isn't realistic, so use the name of bio_for_each_page_all2() temporarily
for conversion, and once all bio_for_each_page_all() is converted, the
original name of bio_for_each_page_all() will be recovered finally.
This patch introduce bio_for_each_segment_all too, which is used for
updating bvec table directly, and users should be carful about this
helper since it returns real multipage segment now.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
include/linux/bio.h | 18 ++++++++++++++++++
include/linux/bvec.h | 6 ++++++
2 files changed, 24 insertions(+)
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 61a04c131641..75baad77d9a8 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -225,6 +225,24 @@ static inline bool bio_rewind_iter(struct bio *bio, struct bvec_iter *iter,
#define bio_for_each_segment(bvl, bio, iter) \
__bio_for_each_segment(bvl, bio, iter, (bio)->bi_iter)
+#define bio_for_each_segment_all(bvl, bio, i) \
+ bio_for_each_page_all((bvl), (bio), (i))
+
+/*
+ * This helper returns singlepage bvec to caller, and the sp bvec is
+ * generated in-flight from multipage bvec stored in bvec table. So we
+ * can _not_ change the bvec stored in bio->bi_io_vec[] via this helper.
+ *
+ * If bvec need to be updated in the table, please use
+ * bio_for_each_segment_all() and make sure it is correctly used since
+ * bvec may points to one multipage bvec.
+ */
+#define bio_for_each_page_all2(bvl, bio, i, bi) \
+ for ((bi).iter = BVEC_ITER_ALL_INIT, i = 0, bvl = &(bi).bv; \
+ (bi).iter.bi_idx < (bio)->bi_vcnt && \
+ (((bi).bv = bio_iter_iovec((bio), (bi).iter)), 1); \
+ bio_advance_iter((bio), &(bi).iter, (bi).bv.bv_len), i++)
+
#define bio_iter_last(bvec, iter) ((iter).bi_size == (bvec).bv_len)
static inline unsigned __bio_elements(struct bio *bio, bool seg)
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index 217afcd83a15..2deee87b823e 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -84,6 +84,12 @@ struct bvec_iter {
current bvec */
};
+/* this iter is only for implementing bio_for_each_page_all2() */
+struct bvec_iter_all {
+ struct bvec_iter iter;
+ struct bio_vec bv; /* in-flight singlepage bvec */
+};
+
/*
* various member access, note that bio_data should of course not be used
* on highmem page vectors
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 18/33] block: deal with dirtying pages for multipage bvec
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (16 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 17/33] block: bio: introduce bio_for_each_page_all2 and bio_for_each_segment_all Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 19/33] block: convert to bio_for_each_page_all2() Ming Lei
` (16 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
In bio_check_pages_dirty(), bvec->bv_page is used as flag for marking
if the page has been dirtied & released, and if no, it will be dirtied
in deferred workqueue.
With multipage bvec, we can't do that any more, so change the logic into
checking all pages in one mp bvec, and only release all these pages if all
are dirtied, otherwise dirty them all in deferred wrokqueue.
This patch introduces segment_for_each_page_all() to deal with the case
a bit easier.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
block/bio.c | 45 +++++++++++++++++++++++++++++++++++++--------
include/linux/bvec.h | 7 +++++++
2 files changed, 44 insertions(+), 8 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index 63d4fe85f42e..a200c42e55dc 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1630,8 +1630,9 @@ void bio_set_pages_dirty(struct bio *bio)
{
struct bio_vec *bvec;
int i;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
if (page && !PageCompound(page))
@@ -1640,16 +1641,26 @@ void bio_set_pages_dirty(struct bio *bio)
}
EXPORT_SYMBOL_GPL(bio_set_pages_dirty);
+static inline void release_mp_bvec_pages(struct bio_vec *bvec)
+{
+ struct bio_vec bv;
+ struct bvec_iter iter;
+
+ segment_for_each_page_all(bv, bvec, iter)
+ put_page(bv.bv_page);
+}
+
static void bio_release_pages(struct bio *bio)
{
struct bio_vec *bvec;
int i;
- bio_for_each_page_all(bvec, bio, i) {
+ /* iterate each mp bvec */
+ bio_for_each_segment_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
if (page)
- put_page(page);
+ release_mp_bvec_pages(bvec);
}
}
@@ -1693,20 +1704,38 @@ static void bio_dirty_fn(struct work_struct *work)
}
}
+static inline void check_mp_bvec_pages(struct bio_vec *bvec,
+ int *nr_dirty, int *nr_pages)
+{
+ struct bio_vec bv;
+ struct bvec_iter iter;
+
+ segment_for_each_page_all(bv, bvec, iter) {
+ struct page *page = bv.bv_page;
+
+ if (PageDirty(page) || PageCompound(page))
+ (*nr_dirty)++;
+ (*nr_pages)++;
+ }
+}
+
void bio_check_pages_dirty(struct bio *bio)
{
struct bio_vec *bvec;
int nr_clean_pages = 0;
int i;
- bio_for_each_page_all(bvec, bio, i) {
- struct page *page = bvec->bv_page;
+ bio_for_each_segment_all(bvec, bio, i) {
+ int nr_dirty = 0, nr_pages = 0;
+
+ check_mp_bvec_pages(bvec, &nr_dirty, &nr_pages);
- if (PageDirty(page) || PageCompound(page)) {
- put_page(page);
+ /* release all pages in the mp bvec if all are dirtied */
+ if (nr_dirty == nr_pages) {
+ release_mp_bvec_pages(bvec);
bvec->bv_page = NULL;
} else {
- nr_clean_pages++;
+ nr_clean_pages += nr_pages;
}
}
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index 2deee87b823e..893e8fef0dd0 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -225,6 +225,13 @@ static inline bool bvec_iter_seg_advance(const struct bio_vec *bv,
.bi_bvec_done = 0, \
}
+#define segment_for_each_page_all(pg_bvl, seg_bvec, iter) \
+ for (iter = BVEC_ITER_ALL_INIT, \
+ (iter).bi_size = (seg_bvec)->bv_len - (iter).bi_bvec_done;\
+ (iter).bi_size && \
+ ((pg_bvl = bvec_iter_bvec((seg_bvec), (iter))), 1); \
+ bvec_iter_advance((seg_bvec), &(iter), (pg_bvl).bv_len))
+
/* get the last page from the multipage bvec and store it in @pg */
static inline void segment_last_page(const struct bio_vec *seg,
struct bio_vec *pg)
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 19/33] block: convert to bio_for_each_page_all2()
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (17 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 18/33] block: deal with dirtying pages for multipage bvec Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 20/33] md/dm/bcache: conver to bio_for_each_page_all2 and bio_for_each_segment Ming Lei
` (15 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
We have to convert to bio_for_each_page_all2() for iterating page by
page.
bio_for_each_page_all() can't be used any more after multipage bvec is
enabled.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
block/bio.c | 18 ++++++++++++------
block/blk-zoned.c | 5 +++--
block/bounce.c | 6 ++++--
include/linux/bio.h | 3 ++-
4 files changed, 21 insertions(+), 11 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index a200c42e55dc..a14c854b9111 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1119,8 +1119,9 @@ static int bio_copy_from_iter(struct bio *bio, struct iov_iter *iter)
{
int i;
struct bio_vec *bvec;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
ssize_t ret;
ret = copy_page_from_iter(bvec->bv_page,
@@ -1150,8 +1151,9 @@ static int bio_copy_to_iter(struct bio *bio, struct iov_iter iter)
{
int i;
struct bio_vec *bvec;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
ssize_t ret;
ret = copy_page_to_iter(bvec->bv_page,
@@ -1173,8 +1175,9 @@ void bio_free_pages(struct bio *bio)
{
struct bio_vec *bvec;
int i;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bvec, bio, i)
+ bio_for_each_page_all2(bvec, bio, i, bia)
__free_page(bvec->bv_page);
}
EXPORT_SYMBOL(bio_free_pages);
@@ -1340,6 +1343,7 @@ struct bio *bio_map_user_iov(struct request_queue *q,
struct bio *bio;
int ret;
struct bio_vec *bvec;
+ struct bvec_iter_all bia;
if (!iov_iter_count(iter))
return ERR_PTR(-EINVAL);
@@ -1413,7 +1417,7 @@ struct bio *bio_map_user_iov(struct request_queue *q,
return bio;
out_unmap:
- bio_for_each_page_all(bvec, bio, j) {
+ bio_for_each_page_all2(bvec, bio, j, bia) {
put_page(bvec->bv_page);
}
bio_put(bio);
@@ -1424,11 +1428,12 @@ static void __bio_unmap_user(struct bio *bio)
{
struct bio_vec *bvec;
int i;
+ struct bvec_iter_all bia;
/*
* make sure we dirty pages we wrote to
*/
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
if (bio_data_dir(bio) == READ)
set_page_dirty_lock(bvec->bv_page);
@@ -1520,8 +1525,9 @@ static void bio_copy_kern_endio_read(struct bio *bio)
char *p = bio->bi_private;
struct bio_vec *bvec;
int i;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
memcpy(p, page_address(bvec->bv_page), bvec->bv_len);
p += bvec->bv_len;
}
diff --git a/block/blk-zoned.c b/block/blk-zoned.c
index 77f3cecfaa7d..a76053d6fd6c 100644
--- a/block/blk-zoned.c
+++ b/block/blk-zoned.c
@@ -123,6 +123,7 @@ int blkdev_report_zones(struct block_device *bdev,
unsigned int ofst;
void *addr;
int ret;
+ struct bvec_iter_all bia;
if (!q)
return -ENXIO;
@@ -190,7 +191,7 @@ int blkdev_report_zones(struct block_device *bdev,
n = 0;
nz = 0;
nr_rep = 0;
- bio_for_each_page_all(bv, bio, i) {
+ bio_for_each_page_all2(bv, bio, i, bia) {
if (!bv->bv_page)
break;
@@ -223,7 +224,7 @@ int blkdev_report_zones(struct block_device *bdev,
*nr_zones = nz;
out:
- bio_for_each_page_all(bv, bio, i)
+ bio_for_each_page_all2(bv, bio, i, bia)
__free_page(bv->bv_page);
bio_put(bio);
diff --git a/block/bounce.c b/block/bounce.c
index f4ee4b81f7a2..8b14683f4061 100644
--- a/block/bounce.c
+++ b/block/bounce.c
@@ -143,11 +143,12 @@ static void bounce_end_io(struct bio *bio, mempool_t *pool)
struct bio_vec *bvec, orig_vec;
int i;
struct bvec_iter orig_iter = bio_orig->bi_iter;
+ struct bvec_iter_all bia;
/*
* free up bounce indirect pages used
*/
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
orig_vec = bio_iter_iovec(bio_orig, orig_iter);
if (bvec->bv_page != orig_vec.bv_page) {
dec_zone_page_state(bvec->bv_page, NR_BOUNCE);
@@ -203,6 +204,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
bool bounce = false;
int sectors = 0;
bool passthrough = bio_is_passthrough(*bio_orig);
+ struct bvec_iter_all bia;
bio_for_each_page(from, *bio_orig, iter) {
if (i++ < BIO_MAX_PAGES)
@@ -222,7 +224,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
bio = bio_clone_bioset(*bio_orig, GFP_NOIO, passthrough ? NULL :
bounce_bio_set);
- bio_for_each_page_all(to, bio, i) {
+ bio_for_each_page_all2(to, bio, i, bia) {
struct page *page = to->bv_page;
if (page_to_pfn(page) <= q->limits.bounce_pfn)
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 75baad77d9a8..5ae2bc876295 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -369,10 +369,11 @@ static inline unsigned bio_pages_all(struct bio *bio)
{
unsigned i;
struct bio_vec *bv;
+ struct bvec_iter_all bia;
WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all(bv, bio, i)
+ bio_for_each_page_all2(bv, bio, i, bia)
;
return i;
}
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 20/33] md/dm/bcache: conver to bio_for_each_page_all2 and bio_for_each_segment
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (18 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 19/33] block: convert to bio_for_each_page_all2() Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 21/33] fs: conver to bio_for_each_page_all2 Ming Lei
` (14 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
In bch_bio_alloc_pages(), bio_for_each_segment() is fine because this
helper can only be used on a freshly new bio.
For other cases, we conver to bio_for_each_page_all2() since they needn't
to update bvec table.
bio_for_each_page_all() can't be used any more after multipage bvec is
enabled, so we have to convert to bio_for_each_page_all2().
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
drivers/md/bcache/btree.c | 3 ++-
drivers/md/bcache/util.c | 2 +-
drivers/md/dm-crypt.c | 3 ++-
drivers/md/raid1.c | 3 ++-
4 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/drivers/md/bcache/btree.c b/drivers/md/bcache/btree.c
index a9d82911c3d2..498f6b032b4c 100644
--- a/drivers/md/bcache/btree.c
+++ b/drivers/md/bcache/btree.c
@@ -423,8 +423,9 @@ static void do_btree_node_write(struct btree *b)
int j;
struct bio_vec *bv;
void *base = (void *) ((unsigned long) i & ~(PAGE_SIZE - 1));
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bv, b->bio, j)
+ bio_for_each_page_all2(bv, b->bio, j, bia)
memcpy(page_address(bv->bv_page),
base + j * PAGE_SIZE, PAGE_SIZE);
diff --git a/drivers/md/bcache/util.c b/drivers/md/bcache/util.c
index 77230973a110..74febd5230df 100644
--- a/drivers/md/bcache/util.c
+++ b/drivers/md/bcache/util.c
@@ -303,7 +303,7 @@ int bch_bio_alloc_pages(struct bio *bio, gfp_t gfp_mask)
int i;
struct bio_vec *bv;
- bio_for_each_page_all(bv, bio, i) {
+ bio_for_each_segment_all(bv, bio, i) {
bv->bv_page = alloc_page(gfp_mask);
if (!bv->bv_page) {
while (--bv >= bio->bi_io_vec)
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 74737ae0ef11..8fdc8349fd72 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -1450,8 +1450,9 @@ static void crypt_free_buffer_pages(struct crypt_config *cc, struct bio *clone)
{
unsigned int i;
struct bio_vec *bv;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bv, clone, i) {
+ bio_for_each_page_all2(bv, clone, i, bia) {
BUG_ON(!bv->bv_page);
mempool_free(bv->bv_page, cc->page_pool);
}
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index e318a0c19eb0..8b2b071619a2 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -2117,13 +2117,14 @@ static void process_checks(struct r1bio *r1_bio)
struct page **spages = get_resync_pages(sbio)->pages;
struct bio_vec *bi;
int page_len[RESYNC_PAGES] = { 0 };
+ struct bvec_iter_all bia;
if (sbio->bi_end_io != end_sync_read)
continue;
/* Now we can 'fixup' the error value */
sbio->bi_status = 0;
- bio_for_each_page_all(bi, sbio, j)
+ bio_for_each_page_all2(bi, sbio, j, bia)
page_len[j] = bi->bv_len;
if (!status) {
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 21/33] fs: conver to bio_for_each_page_all2
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (19 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 20/33] md/dm/bcache: conver to bio_for_each_page_all2 and bio_for_each_segment Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 22/33] btrfs: " Ming Lei
` (13 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
bio_for_each_page_all() can't be used any more after multipage bvec is
enabled, so we have to convert to bio_for_each_page_all2().
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
fs/block_dev.c | 6 ++++--
fs/crypto/bio.c | 3 ++-
fs/direct-io.c | 4 +++-
fs/iomap.c | 3 ++-
fs/mpage.c | 3 ++-
5 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/fs/block_dev.c b/fs/block_dev.c
index 65498a34efa9..f581fc0a6142 100644
--- a/fs/block_dev.c
+++ b/fs/block_dev.c
@@ -197,6 +197,7 @@ __blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter,
ssize_t ret;
blk_qc_t qc;
int i;
+ struct bvec_iter_all bia;
if ((pos | iov_iter_alignment(iter)) &
(bdev_logical_block_size(bdev) - 1))
@@ -242,7 +243,7 @@ __blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter,
}
__set_current_state(TASK_RUNNING);
- bio_for_each_page_all(bvec, &bio, i) {
+ bio_for_each_page_all2(bvec, &bio, i, bia) {
if (should_dirty && !PageCompound(bvec->bv_page))
set_page_dirty_lock(bvec->bv_page);
put_page(bvec->bv_page);
@@ -309,8 +310,9 @@ static void blkdev_bio_end_io(struct bio *bio)
} else {
struct bio_vec *bvec;
int i;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bvec, bio, i)
+ bio_for_each_page_all2(bvec, bio, i, bia)
put_page(bvec->bv_page);
bio_put(bio);
}
diff --git a/fs/crypto/bio.c b/fs/crypto/bio.c
index 2dda77c3a89a..743c3ecb7f97 100644
--- a/fs/crypto/bio.c
+++ b/fs/crypto/bio.c
@@ -37,8 +37,9 @@ static void completion_pages(struct work_struct *work)
struct bio *bio = ctx->r.bio;
struct bio_vec *bv;
int i;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bv, bio, i) {
+ bio_for_each_page_all2(bv, bio, i, bia) {
struct page *page = bv->bv_page;
int ret = fscrypt_decrypt_page(page->mapping->host, page,
PAGE_SIZE, 0, page->index);
diff --git a/fs/direct-io.c b/fs/direct-io.c
index bbf25b0de9f8..e6a6dd560da2 100644
--- a/fs/direct-io.c
+++ b/fs/direct-io.c
@@ -551,7 +551,9 @@ static blk_status_t dio_bio_complete(struct dio *dio, struct bio *bio)
if (dio->is_async && dio->op == REQ_OP_READ && dio->should_dirty) {
bio_check_pages_dirty(bio); /* transfers ownership */
} else {
- bio_for_each_page_all(bvec, bio, i) {
+ struct bvec_iter_all bia;
+
+ bio_for_each_page_all2(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
if (dio->op == REQ_OP_READ && !PageCompound(page) &&
diff --git a/fs/iomap.c b/fs/iomap.c
index 42b0b1697b3d..31d19f8f0aac 100644
--- a/fs/iomap.c
+++ b/fs/iomap.c
@@ -817,8 +817,9 @@ static void iomap_dio_bio_end_io(struct bio *bio)
} else {
struct bio_vec *bvec;
int i;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bvec, bio, i)
+ bio_for_each_page_all2(bvec, bio, i, bia)
put_page(bvec->bv_page);
bio_put(bio);
}
diff --git a/fs/mpage.c b/fs/mpage.c
index 1cf322c4d6f8..f2da0f9ec0f2 100644
--- a/fs/mpage.c
+++ b/fs/mpage.c
@@ -48,8 +48,9 @@ static void mpage_end_io(struct bio *bio)
{
struct bio_vec *bv;
int i;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bv, bio, i) {
+ bio_for_each_page_all2(bv, bio, i, bia) {
struct page *page = bv->bv_page;
page_endio(page, op_is_write(bio_op(bio)),
blk_status_to_errno(bio->bi_status));
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 22/33] btrfs: conver to bio_for_each_page_all2
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (20 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 21/33] fs: conver to bio_for_each_page_all2 Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 23/33] ext4: " Ming Lei
` (12 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
bio_for_each_page_all() can't be used any more after multipage bvec is
enabled, so we have to convert to bio_for_each_page_all2().
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
fs/btrfs/compression.c | 3 ++-
fs/btrfs/disk-io.c | 3 ++-
fs/btrfs/extent_io.c | 9 ++++++---
fs/btrfs/inode.c | 6 ++++--
fs/btrfs/raid56.c | 3 ++-
5 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c
index be6b09dfd6a7..4cfe38feae3b 100644
--- a/fs/btrfs/compression.c
+++ b/fs/btrfs/compression.c
@@ -166,13 +166,14 @@ static void end_compressed_bio_read(struct bio *bio)
} else {
int i;
struct bio_vec *bvec;
+ struct bvec_iter_all bia;
/*
* we have verified the checksum already, set page
* checked so the end_io handlers know about it
*/
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all(bvec, cb->orig_bio, i)
+ bio_for_each_page_all2(bvec, cb->orig_bio, i, bia)
SetPageChecked(bvec->bv_page);
bio_endio(cb->orig_bio);
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index c6dc8a636413..ef78fd71c2f7 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -829,9 +829,10 @@ static blk_status_t btree_csum_one_bio(struct bio *bio)
struct bio_vec *bvec;
struct btrfs_root *root;
int i, ret = 0;
+ struct bvec_iter_all bia;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
root = BTRFS_I(bvec->bv_page->mapping->host)->root;
ret = csum_dirty_buffer(root->fs_info, bvec->bv_page);
if (ret)
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 3c9c91a1e3e9..383db7a7e5a4 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -2456,9 +2456,10 @@ static void end_bio_extent_writepage(struct bio *bio)
u64 start;
u64 end;
int i;
+ struct bvec_iter_all bia;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
struct inode *inode = page->mapping->host;
struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
@@ -2527,9 +2528,10 @@ static void end_bio_extent_readpage(struct bio *bio)
int mirror;
int ret;
int i;
+ struct bvec_iter_all bia;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
struct inode *inode = page->mapping->host;
struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
@@ -3681,9 +3683,10 @@ static void end_bio_extent_buffer_writepage(struct bio *bio)
struct bio_vec *bvec;
struct extent_buffer *eb;
int i, done;
+ struct bvec_iter_all bia;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
eb = (struct extent_buffer *)page->private;
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 9d816dc725c4..8a73b26915bc 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -7883,6 +7883,7 @@ static void btrfs_retry_endio_nocsum(struct bio *bio)
struct bio_vec *bvec;
struct extent_io_tree *io_tree, *failure_tree;
int i;
+ struct bvec_iter_all bia;
if (bio->bi_status)
goto end;
@@ -7894,7 +7895,7 @@ static void btrfs_retry_endio_nocsum(struct bio *bio)
done->uptodate = 1;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all(bvec, bio, i)
+ bio_for_each_page_all2(bvec, bio, i, bia)
clean_io_failure(BTRFS_I(inode)->root->fs_info, failure_tree,
io_tree, done->start, bvec->bv_page,
btrfs_ino(BTRFS_I(inode)), 0);
@@ -7973,6 +7974,7 @@ static void btrfs_retry_endio(struct bio *bio)
int uptodate;
int ret;
int i;
+ struct bvec_iter_all bia;
if (bio->bi_status)
goto end;
@@ -7986,7 +7988,7 @@ static void btrfs_retry_endio(struct bio *bio)
failure_tree = &BTRFS_I(inode)->io_failure_tree;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
ret = __readpage_endio_check(inode, io_bio, i, bvec->bv_page,
bvec->bv_offset, done->start,
bvec->bv_len);
diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c
index ab9d80f79ffe..955fa4dbecee 100644
--- a/fs/btrfs/raid56.c
+++ b/fs/btrfs/raid56.c
@@ -1445,10 +1445,11 @@ static void set_bio_pages_uptodate(struct bio *bio)
{
struct bio_vec *bvec;
int i;
+ struct bvec_iter_all bia;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all(bvec, bio, i)
+ bio_for_each_page_all2(bvec, bio, i, bia)
SetPageUptodate(bvec->bv_page);
}
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 23/33] ext4: conver to bio_for_each_page_all2
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (21 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 22/33] btrfs: " Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 24/33] f2fs: " Ming Lei
` (11 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
bio_for_each_page_all() can't be used any more after multipage bvec is
enabled, so we have to convert to bio_for_each_page_all2().
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
fs/ext4/page-io.c | 3 ++-
fs/ext4/readpage.c | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c
index 52f2937f5603..b56a733f33c0 100644
--- a/fs/ext4/page-io.c
+++ b/fs/ext4/page-io.c
@@ -63,8 +63,9 @@ static void ext4_finish_bio(struct bio *bio)
{
int i;
struct bio_vec *bvec;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
#ifdef CONFIG_EXT4_FS_ENCRYPTION
struct page *data_page = NULL;
diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c
index 572b6296f709..c46b5ff68fa8 100644
--- a/fs/ext4/readpage.c
+++ b/fs/ext4/readpage.c
@@ -72,6 +72,7 @@ static void mpage_end_io(struct bio *bio)
{
struct bio_vec *bv;
int i;
+ struct bvec_iter_all bia;
if (ext4_bio_encrypted(bio)) {
if (bio->bi_status) {
@@ -81,7 +82,7 @@ static void mpage_end_io(struct bio *bio)
return;
}
}
- bio_for_each_page_all(bv, bio, i) {
+ bio_for_each_page_all2(bv, bio, i, bia) {
struct page *page = bv->bv_page;
if (!bio->bi_status) {
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 24/33] f2fs: conver to bio_for_each_page_all2
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (22 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 23/33] ext4: " Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-28 7:29 ` Chao Yu
2018-05-25 3:46 ` [RESEND PATCH V5 25/33] xfs: " Ming Lei
` (10 subsequent siblings)
34 siblings, 1 reply; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
bio_for_each_page_all() can't be used any more after multipage bvec is
enabled, so we have to convert to bio_for_each_page_all2().
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
fs/f2fs/data.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 89da84b0f0bd..924284e2f358 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -54,6 +54,7 @@ static void f2fs_read_end_io(struct bio *bio)
{
struct bio_vec *bvec;
int i;
+ struct bvec_iter_all bia;
#ifdef CONFIG_F2FS_FAULT_INJECTION
if (time_to_inject(F2FS_P_SB(bio_first_page_all(bio)), FAULT_IO)) {
@@ -71,7 +72,7 @@ static void f2fs_read_end_io(struct bio *bio)
}
}
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
if (!bio->bi_status) {
@@ -91,8 +92,9 @@ static void f2fs_write_end_io(struct bio *bio)
struct f2fs_sb_info *sbi = bio->bi_private;
struct bio_vec *bvec;
int i;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
enum count_type type = WB_DATA_TYPE(page);
@@ -267,6 +269,7 @@ static bool __has_merged_page(struct f2fs_bio_info *io,
struct bio_vec *bvec;
struct page *target;
int i;
+ struct bvec_iter_all bia;
if (!io->bio)
return false;
@@ -274,7 +277,7 @@ static bool __has_merged_page(struct f2fs_bio_info *io,
if (!inode && !ino)
return true;
- bio_for_each_page_all(bvec, io->bio, i) {
+ bio_for_each_page_all2(bvec, io->bio, i, bia) {
if (bvec->bv_page->mapping)
target = bvec->bv_page;
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 25/33] xfs: conver to bio_for_each_page_all2
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (23 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 24/33] f2fs: " Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 26/33] exofs: " Ming Lei
` (9 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
bio_for_each_page_all() can't be used any more after multipage bvec is
enabled, so we have to convert to bio_for_each_page_all2().
Given bvec can't be changed under bio_for_each_page_all2(), this patch
marks the bvec parameter as 'const' for xfs_finish_page_writeback().
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
fs/xfs/xfs_aops.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
index 13e2c167aec3..b5077eb4df51 100644
--- a/fs/xfs/xfs_aops.c
+++ b/fs/xfs/xfs_aops.c
@@ -107,7 +107,7 @@ xfs_find_daxdev_for_inode(
static void
xfs_finish_page_writeback(
struct inode *inode,
- struct bio_vec *bvec,
+ const struct bio_vec *bvec,
int error)
{
struct buffer_head *head = page_buffers(bvec->bv_page), *bh = head;
@@ -169,6 +169,7 @@ xfs_destroy_ioend(
for (bio = &ioend->io_inline_bio; bio; bio = next) {
struct bio_vec *bvec;
int i;
+ struct bvec_iter_all bia;
/*
* For the last bio, bi_private points to the ioend, so we
@@ -180,7 +181,7 @@ xfs_destroy_ioend(
next = bio->bi_private;
/* walk each page on bio, ending page IO on them */
- bio_for_each_page_all(bvec, bio, i)
+ bio_for_each_page_all2(bvec, bio, i, bia)
xfs_finish_page_writeback(inode, bvec, error);
bio_put(bio);
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 26/33] exofs: conver to bio_for_each_page_all2
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (24 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 25/33] xfs: " Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 27/33] gfs2: " Ming Lei
` (8 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
bio_for_each_page_all() can't be used any more after multipage bvec is
enabled, so we have to convert to bio_for_each_page_all2().
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
fs/exofs/ore.c | 3 ++-
fs/exofs/ore_raid.c | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/fs/exofs/ore.c b/fs/exofs/ore.c
index fe81fd6fe553..2a7e93f21695 100644
--- a/fs/exofs/ore.c
+++ b/fs/exofs/ore.c
@@ -406,8 +406,9 @@ static void _clear_bio(struct bio *bio)
{
struct bio_vec *bv;
unsigned i;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bv, bio, i) {
+ bio_for_each_page_all2(bv, bio, i, bia) {
unsigned this_count = bv->bv_len;
if (likely(PAGE_SIZE == this_count))
diff --git a/fs/exofs/ore_raid.c b/fs/exofs/ore_raid.c
index 2c3346cd1b29..bb0cc314a987 100644
--- a/fs/exofs/ore_raid.c
+++ b/fs/exofs/ore_raid.c
@@ -433,11 +433,12 @@ static void _mark_read4write_pages_uptodate(struct ore_io_state *ios, int ret)
/* loop on all devices all pages */
for (d = 0; d < ios->numdevs; d++) {
struct bio *bio = ios->per_dev[d].bio;
+ struct bvec_iter_all bia;
if (!bio)
continue;
- bio_for_each_page_all(bv, bio, i) {
+ bio_for_each_page_all2(bv, bio, i, bia) {
struct page *page = bv->bv_page;
SetPageUptodate(page);
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 27/33] gfs2: conver to bio_for_each_page_all2
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (25 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 26/33] exofs: " Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 28/33] block: kill bio_for_each_page_all() Ming Lei
` (7 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
bio_for_each_page_all() can't be used any more after multipage bvec is
enabled, so we have to convert to bio_for_each_page_all2().
Given bvec can't be changed inside bio_for_each_page_all2(), this patch
marks the bvec parameter as 'const' for gfs2_end_log_write_bh().
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
fs/gfs2/lops.c | 6 ++++--
fs/gfs2/meta_io.c | 3 ++-
2 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/fs/gfs2/lops.c b/fs/gfs2/lops.c
index a31b9b028957..0284cd66089c 100644
--- a/fs/gfs2/lops.c
+++ b/fs/gfs2/lops.c
@@ -168,7 +168,8 @@ u64 gfs2_log_bmap(struct gfs2_sbd *sdp)
* that is pinned in the pagecache.
*/
-static void gfs2_end_log_write_bh(struct gfs2_sbd *sdp, struct bio_vec *bvec,
+static void gfs2_end_log_write_bh(struct gfs2_sbd *sdp,
+ const struct bio_vec *bvec,
blk_status_t error)
{
struct buffer_head *bh, *next;
@@ -207,6 +208,7 @@ static void gfs2_end_log_write(struct bio *bio)
struct bio_vec *bvec;
struct page *page;
int i;
+ struct bvec_iter_all bia;
if (bio->bi_status) {
fs_err(sdp, "Error %d writing to journal, jid=%u\n",
@@ -214,7 +216,7 @@ static void gfs2_end_log_write(struct bio *bio)
wake_up(&sdp->sd_logd_waitq);
}
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
page = bvec->bv_page;
if (page_has_buffers(page))
gfs2_end_log_write_bh(sdp, bvec, bio->bi_status);
diff --git a/fs/gfs2/meta_io.c b/fs/gfs2/meta_io.c
index 1d720352310a..a945c9fa1dc6 100644
--- a/fs/gfs2/meta_io.c
+++ b/fs/gfs2/meta_io.c
@@ -190,8 +190,9 @@ static void gfs2_meta_read_endio(struct bio *bio)
{
struct bio_vec *bvec;
int i;
+ struct bvec_iter_all bia;
- bio_for_each_page_all(bvec, bio, i) {
+ bio_for_each_page_all2(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
struct buffer_head *bh = page_buffers(page);
unsigned int len = bvec->bv_len;
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 28/33] block: kill bio_for_each_page_all()
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (26 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 27/33] gfs2: " Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 29/33] block: rename bio_for_each_page_all2 as bio_for_each_page_all Ming Lei
` (6 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
No one uses it any more, so kill it and we can reuse this helper
name.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
include/linux/bio.h | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 5ae2bc876295..c5e692d43f23 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -159,8 +159,10 @@ static inline void *bio_data(struct bio *bio)
/*
* drivers should _never_ use the all version - the bio may have been split
* before it got to the driver and the driver won't own all of it
+ *
+ * This helper iterates bio segment by segment.
*/
-#define bio_for_each_page_all(bvl, bio, i) \
+#define bio_for_each_segment_all(bvl, bio, i) \
for (i = 0, bvl = (bio)->bi_io_vec; i < (bio)->bi_vcnt; i++, bvl++)
static inline void __bio_advance_iter(struct bio *bio, struct bvec_iter *iter,
@@ -225,9 +227,6 @@ static inline bool bio_rewind_iter(struct bio *bio, struct bvec_iter *iter,
#define bio_for_each_segment(bvl, bio, iter) \
__bio_for_each_segment(bvl, bio, iter, (bio)->bi_iter)
-#define bio_for_each_segment_all(bvl, bio, i) \
- bio_for_each_page_all((bvl), (bio), (i))
-
/*
* This helper returns singlepage bvec to caller, and the sp bvec is
* generated in-flight from multipage bvec stored in bvec table. So we
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 29/33] block: rename bio_for_each_page_all2 as bio_for_each_page_all
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (27 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 28/33] block: kill bio_for_each_page_all() Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 30/33] block: enable multipage bvecs Ming Lei
` (5 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
Now bio_for_each_page_all() is gone, we can reuse the name to iterate
bio page by page, which is done via bio_for_each_page_all2() now.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
block/bio.c | 14 +++++++-------
block/blk-zoned.c | 4 ++--
block/bounce.c | 4 ++--
drivers/md/bcache/btree.c | 2 +-
drivers/md/dm-crypt.c | 2 +-
drivers/md/raid1.c | 2 +-
fs/block_dev.c | 4 ++--
fs/btrfs/compression.c | 2 +-
fs/btrfs/disk-io.c | 2 +-
fs/btrfs/extent_io.c | 6 +++---
fs/btrfs/inode.c | 4 ++--
fs/btrfs/raid56.c | 2 +-
fs/crypto/bio.c | 2 +-
fs/direct-io.c | 2 +-
fs/exofs/ore.c | 2 +-
fs/exofs/ore_raid.c | 2 +-
fs/ext4/page-io.c | 2 +-
fs/ext4/readpage.c | 2 +-
fs/f2fs/data.c | 6 +++---
fs/gfs2/lops.c | 2 +-
fs/gfs2/meta_io.c | 2 +-
fs/iomap.c | 2 +-
fs/mpage.c | 2 +-
fs/xfs/xfs_aops.c | 2 +-
include/linux/bio.h | 4 ++--
include/linux/bvec.h | 2 +-
26 files changed, 41 insertions(+), 41 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index a14c854b9111..c160c143cc1b 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1121,7 +1121,7 @@ static int bio_copy_from_iter(struct bio *bio, struct iov_iter *iter)
struct bio_vec *bvec;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
ssize_t ret;
ret = copy_page_from_iter(bvec->bv_page,
@@ -1153,7 +1153,7 @@ static int bio_copy_to_iter(struct bio *bio, struct iov_iter iter)
struct bio_vec *bvec;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
ssize_t ret;
ret = copy_page_to_iter(bvec->bv_page,
@@ -1177,7 +1177,7 @@ void bio_free_pages(struct bio *bio)
int i;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bvec, bio, i, bia)
+ bio_for_each_page_all(bvec, bio, i, bia)
__free_page(bvec->bv_page);
}
EXPORT_SYMBOL(bio_free_pages);
@@ -1417,7 +1417,7 @@ struct bio *bio_map_user_iov(struct request_queue *q,
return bio;
out_unmap:
- bio_for_each_page_all2(bvec, bio, j, bia) {
+ bio_for_each_page_all(bvec, bio, j, bia) {
put_page(bvec->bv_page);
}
bio_put(bio);
@@ -1433,7 +1433,7 @@ static void __bio_unmap_user(struct bio *bio)
/*
* make sure we dirty pages we wrote to
*/
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
if (bio_data_dir(bio) == READ)
set_page_dirty_lock(bvec->bv_page);
@@ -1527,7 +1527,7 @@ static void bio_copy_kern_endio_read(struct bio *bio)
int i;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
memcpy(p, page_address(bvec->bv_page), bvec->bv_len);
p += bvec->bv_len;
}
@@ -1638,7 +1638,7 @@ void bio_set_pages_dirty(struct bio *bio)
int i;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
if (page && !PageCompound(page))
diff --git a/block/blk-zoned.c b/block/blk-zoned.c
index a76053d6fd6c..b7c182b0d805 100644
--- a/block/blk-zoned.c
+++ b/block/blk-zoned.c
@@ -191,7 +191,7 @@ int blkdev_report_zones(struct block_device *bdev,
n = 0;
nz = 0;
nr_rep = 0;
- bio_for_each_page_all2(bv, bio, i, bia) {
+ bio_for_each_page_all(bv, bio, i, bia) {
if (!bv->bv_page)
break;
@@ -224,7 +224,7 @@ int blkdev_report_zones(struct block_device *bdev,
*nr_zones = nz;
out:
- bio_for_each_page_all2(bv, bio, i, bia)
+ bio_for_each_page_all(bv, bio, i, bia)
__free_page(bv->bv_page);
bio_put(bio);
diff --git a/block/bounce.c b/block/bounce.c
index 8b14683f4061..da9fcf1a07d7 100644
--- a/block/bounce.c
+++ b/block/bounce.c
@@ -148,7 +148,7 @@ static void bounce_end_io(struct bio *bio, mempool_t *pool)
/*
* free up bounce indirect pages used
*/
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
orig_vec = bio_iter_iovec(bio_orig, orig_iter);
if (bvec->bv_page != orig_vec.bv_page) {
dec_zone_page_state(bvec->bv_page, NR_BOUNCE);
@@ -224,7 +224,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
bio = bio_clone_bioset(*bio_orig, GFP_NOIO, passthrough ? NULL :
bounce_bio_set);
- bio_for_each_page_all2(to, bio, i, bia) {
+ bio_for_each_page_all(to, bio, i, bia) {
struct page *page = to->bv_page;
if (page_to_pfn(page) <= q->limits.bounce_pfn)
diff --git a/drivers/md/bcache/btree.c b/drivers/md/bcache/btree.c
index 498f6b032b4c..da8a434dc3a0 100644
--- a/drivers/md/bcache/btree.c
+++ b/drivers/md/bcache/btree.c
@@ -425,7 +425,7 @@ static void do_btree_node_write(struct btree *b)
void *base = (void *) ((unsigned long) i & ~(PAGE_SIZE - 1));
struct bvec_iter_all bia;
- bio_for_each_page_all2(bv, b->bio, j, bia)
+ bio_for_each_page_all(bv, b->bio, j, bia)
memcpy(page_address(bv->bv_page),
base + j * PAGE_SIZE, PAGE_SIZE);
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 8fdc8349fd72..016b8c5fcaf6 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -1452,7 +1452,7 @@ static void crypt_free_buffer_pages(struct crypt_config *cc, struct bio *clone)
struct bio_vec *bv;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bv, clone, i, bia) {
+ bio_for_each_page_all(bv, clone, i, bia) {
BUG_ON(!bv->bv_page);
mempool_free(bv->bv_page, cc->page_pool);
}
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 8b2b071619a2..74737cf08cab 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -2124,7 +2124,7 @@ static void process_checks(struct r1bio *r1_bio)
/* Now we can 'fixup' the error value */
sbio->bi_status = 0;
- bio_for_each_page_all2(bi, sbio, j, bia)
+ bio_for_each_page_all(bi, sbio, j, bia)
page_len[j] = bi->bv_len;
if (!status) {
diff --git a/fs/block_dev.c b/fs/block_dev.c
index f581fc0a6142..03422da6bfff 100644
--- a/fs/block_dev.c
+++ b/fs/block_dev.c
@@ -243,7 +243,7 @@ __blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter,
}
__set_current_state(TASK_RUNNING);
- bio_for_each_page_all2(bvec, &bio, i, bia) {
+ bio_for_each_page_all(bvec, &bio, i, bia) {
if (should_dirty && !PageCompound(bvec->bv_page))
set_page_dirty_lock(bvec->bv_page);
put_page(bvec->bv_page);
@@ -312,7 +312,7 @@ static void blkdev_bio_end_io(struct bio *bio)
int i;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bvec, bio, i, bia)
+ bio_for_each_page_all(bvec, bio, i, bia)
put_page(bvec->bv_page);
bio_put(bio);
}
diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c
index 4cfe38feae3b..3a7a14db80b8 100644
--- a/fs/btrfs/compression.c
+++ b/fs/btrfs/compression.c
@@ -173,7 +173,7 @@ static void end_compressed_bio_read(struct bio *bio)
* checked so the end_io handlers know about it
*/
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all2(bvec, cb->orig_bio, i, bia)
+ bio_for_each_page_all(bvec, cb->orig_bio, i, bia)
SetPageChecked(bvec->bv_page);
bio_endio(cb->orig_bio);
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index ef78fd71c2f7..e3dbdbf4ea6b 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -832,7 +832,7 @@ static blk_status_t btree_csum_one_bio(struct bio *bio)
struct bvec_iter_all bia;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
root = BTRFS_I(bvec->bv_page->mapping->host)->root;
ret = csum_dirty_buffer(root->fs_info, bvec->bv_page);
if (ret)
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 383db7a7e5a4..3a1a1b4a4c55 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -2459,7 +2459,7 @@ static void end_bio_extent_writepage(struct bio *bio)
struct bvec_iter_all bia;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
struct inode *inode = page->mapping->host;
struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
@@ -2531,7 +2531,7 @@ static void end_bio_extent_readpage(struct bio *bio)
struct bvec_iter_all bia;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
struct inode *inode = page->mapping->host;
struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
@@ -3686,7 +3686,7 @@ static void end_bio_extent_buffer_writepage(struct bio *bio)
struct bvec_iter_all bia;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
eb = (struct extent_buffer *)page->private;
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 8a73b26915bc..812ca7ac5108 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -7895,7 +7895,7 @@ static void btrfs_retry_endio_nocsum(struct bio *bio)
done->uptodate = 1;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all2(bvec, bio, i, bia)
+ bio_for_each_page_all(bvec, bio, i, bia)
clean_io_failure(BTRFS_I(inode)->root->fs_info, failure_tree,
io_tree, done->start, bvec->bv_page,
btrfs_ino(BTRFS_I(inode)), 0);
@@ -7988,7 +7988,7 @@ static void btrfs_retry_endio(struct bio *bio)
failure_tree = &BTRFS_I(inode)->io_failure_tree;
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
ret = __readpage_endio_check(inode, io_bio, i, bvec->bv_page,
bvec->bv_offset, done->start,
bvec->bv_len);
diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c
index 955fa4dbecee..90f16fbfc378 100644
--- a/fs/btrfs/raid56.c
+++ b/fs/btrfs/raid56.c
@@ -1449,7 +1449,7 @@ static void set_bio_pages_uptodate(struct bio *bio)
ASSERT(!bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all2(bvec, bio, i, bia)
+ bio_for_each_page_all(bvec, bio, i, bia)
SetPageUptodate(bvec->bv_page);
}
diff --git a/fs/crypto/bio.c b/fs/crypto/bio.c
index 743c3ecb7f97..b1bf8913945b 100644
--- a/fs/crypto/bio.c
+++ b/fs/crypto/bio.c
@@ -39,7 +39,7 @@ static void completion_pages(struct work_struct *work)
int i;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bv, bio, i, bia) {
+ bio_for_each_page_all(bv, bio, i, bia) {
struct page *page = bv->bv_page;
int ret = fscrypt_decrypt_page(page->mapping->host, page,
PAGE_SIZE, 0, page->index);
diff --git a/fs/direct-io.c b/fs/direct-io.c
index e6a6dd560da2..b57c3aa26cab 100644
--- a/fs/direct-io.c
+++ b/fs/direct-io.c
@@ -553,7 +553,7 @@ static blk_status_t dio_bio_complete(struct dio *dio, struct bio *bio)
} else {
struct bvec_iter_all bia;
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
if (dio->op == REQ_OP_READ && !PageCompound(page) &&
diff --git a/fs/exofs/ore.c b/fs/exofs/ore.c
index 2a7e93f21695..e4b51885cf49 100644
--- a/fs/exofs/ore.c
+++ b/fs/exofs/ore.c
@@ -408,7 +408,7 @@ static void _clear_bio(struct bio *bio)
unsigned i;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bv, bio, i, bia) {
+ bio_for_each_page_all(bv, bio, i, bia) {
unsigned this_count = bv->bv_len;
if (likely(PAGE_SIZE == this_count))
diff --git a/fs/exofs/ore_raid.c b/fs/exofs/ore_raid.c
index bb0cc314a987..d14e070bfab1 100644
--- a/fs/exofs/ore_raid.c
+++ b/fs/exofs/ore_raid.c
@@ -438,7 +438,7 @@ static void _mark_read4write_pages_uptodate(struct ore_io_state *ios, int ret)
if (!bio)
continue;
- bio_for_each_page_all2(bv, bio, i, bia) {
+ bio_for_each_page_all(bv, bio, i, bia) {
struct page *page = bv->bv_page;
SetPageUptodate(page);
diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c
index b56a733f33c0..a960ac073818 100644
--- a/fs/ext4/page-io.c
+++ b/fs/ext4/page-io.c
@@ -65,7 +65,7 @@ static void ext4_finish_bio(struct bio *bio)
struct bio_vec *bvec;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
#ifdef CONFIG_EXT4_FS_ENCRYPTION
struct page *data_page = NULL;
diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c
index c46b5ff68fa8..0bb78365417b 100644
--- a/fs/ext4/readpage.c
+++ b/fs/ext4/readpage.c
@@ -82,7 +82,7 @@ static void mpage_end_io(struct bio *bio)
return;
}
}
- bio_for_each_page_all2(bv, bio, i, bia) {
+ bio_for_each_page_all(bv, bio, i, bia) {
struct page *page = bv->bv_page;
if (!bio->bi_status) {
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 924284e2f358..a3200dee15a4 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -72,7 +72,7 @@ static void f2fs_read_end_io(struct bio *bio)
}
}
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
if (!bio->bi_status) {
@@ -94,7 +94,7 @@ static void f2fs_write_end_io(struct bio *bio)
int i;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
enum count_type type = WB_DATA_TYPE(page);
@@ -277,7 +277,7 @@ static bool __has_merged_page(struct f2fs_bio_info *io,
if (!inode && !ino)
return true;
- bio_for_each_page_all2(bvec, io->bio, i, bia) {
+ bio_for_each_page_all(bvec, io->bio, i, bia) {
if (bvec->bv_page->mapping)
target = bvec->bv_page;
diff --git a/fs/gfs2/lops.c b/fs/gfs2/lops.c
index 0284cd66089c..036475303fed 100644
--- a/fs/gfs2/lops.c
+++ b/fs/gfs2/lops.c
@@ -216,7 +216,7 @@ static void gfs2_end_log_write(struct bio *bio)
wake_up(&sdp->sd_logd_waitq);
}
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
page = bvec->bv_page;
if (page_has_buffers(page))
gfs2_end_log_write_bh(sdp, bvec, bio->bi_status);
diff --git a/fs/gfs2/meta_io.c b/fs/gfs2/meta_io.c
index a945c9fa1dc6..829affd24efa 100644
--- a/fs/gfs2/meta_io.c
+++ b/fs/gfs2/meta_io.c
@@ -192,7 +192,7 @@ static void gfs2_meta_read_endio(struct bio *bio)
int i;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bvec, bio, i, bia) {
+ bio_for_each_page_all(bvec, bio, i, bia) {
struct page *page = bvec->bv_page;
struct buffer_head *bh = page_buffers(page);
unsigned int len = bvec->bv_len;
diff --git a/fs/iomap.c b/fs/iomap.c
index 31d19f8f0aac..39a49a1cd8d8 100644
--- a/fs/iomap.c
+++ b/fs/iomap.c
@@ -819,7 +819,7 @@ static void iomap_dio_bio_end_io(struct bio *bio)
int i;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bvec, bio, i, bia)
+ bio_for_each_page_all(bvec, bio, i, bia)
put_page(bvec->bv_page);
bio_put(bio);
}
diff --git a/fs/mpage.c b/fs/mpage.c
index f2da0f9ec0f2..6360217f4f97 100644
--- a/fs/mpage.c
+++ b/fs/mpage.c
@@ -50,7 +50,7 @@ static void mpage_end_io(struct bio *bio)
int i;
struct bvec_iter_all bia;
- bio_for_each_page_all2(bv, bio, i, bia) {
+ bio_for_each_page_all(bv, bio, i, bia) {
struct page *page = bv->bv_page;
page_endio(page, op_is_write(bio_op(bio)),
blk_status_to_errno(bio->bi_status));
diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
index b5077eb4df51..f3ae373d3092 100644
--- a/fs/xfs/xfs_aops.c
+++ b/fs/xfs/xfs_aops.c
@@ -181,7 +181,7 @@ xfs_destroy_ioend(
next = bio->bi_private;
/* walk each page on bio, ending page IO on them */
- bio_for_each_page_all2(bvec, bio, i, bia)
+ bio_for_each_page_all(bvec, bio, i, bia)
xfs_finish_page_writeback(inode, bvec, error);
bio_put(bio);
diff --git a/include/linux/bio.h b/include/linux/bio.h
index c5e692d43f23..fc8a8238805e 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -236,7 +236,7 @@ static inline bool bio_rewind_iter(struct bio *bio, struct bvec_iter *iter,
* bio_for_each_segment_all() and make sure it is correctly used since
* bvec may points to one multipage bvec.
*/
-#define bio_for_each_page_all2(bvl, bio, i, bi) \
+#define bio_for_each_page_all(bvl, bio, i, bi) \
for ((bi).iter = BVEC_ITER_ALL_INIT, i = 0, bvl = &(bi).bv; \
(bi).iter.bi_idx < (bio)->bi_vcnt && \
(((bi).bv = bio_iter_iovec((bio), (bi).iter)), 1); \
@@ -372,7 +372,7 @@ static inline unsigned bio_pages_all(struct bio *bio)
WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED));
- bio_for_each_page_all2(bv, bio, i, bia)
+ bio_for_each_page_all(bv, bio, i, bia)
;
return i;
}
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index 893e8fef0dd0..2811a4cd0706 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -84,7 +84,7 @@ struct bvec_iter {
current bvec */
};
-/* this iter is only for implementing bio_for_each_page_all2() */
+/* this iter is only for implementing bio_for_each_page_all() */
struct bvec_iter_all {
struct bvec_iter iter;
struct bio_vec bv; /* in-flight singlepage bvec */
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 30/33] block: enable multipage bvecs
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (28 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 29/33] block: rename bio_for_each_page_all2 as bio_for_each_page_all Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 31/33] block: bio: pass segments to bio if bio_add_page() is bypassed Ming Lei
` (4 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
This patch pulls the trigger for multipage bvecs.
Now any request queue which supports queue cluster will see multipage
bvecs.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
block/bio.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/block/bio.c b/block/bio.c
index c160c143cc1b..bc3992f52fe8 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -876,6 +876,11 @@ int bio_add_page(struct bio *bio, struct page *page,
* a consecutive offset. Optimize this special case.
*/
if (bio->bi_vcnt > 0) {
+ struct request_queue *q = NULL;
+
+ if (bio->bi_disk)
+ q = bio->bi_disk->queue;
+
bv = &bio->bi_io_vec[bio->bi_vcnt - 1];
if (page == bv->bv_page &&
@@ -883,6 +888,14 @@ int bio_add_page(struct bio *bio, struct page *page,
bv->bv_len += len;
goto done;
}
+
+ /* disable multipage bvec too if cluster isn't enabled */
+ if (q && blk_queue_cluster(q) &&
+ (bvec_to_phys(bv) + bv->bv_len ==
+ page_to_phys(page) + offset)) {
+ bv->bv_len += len;
+ goto done;
+ }
}
if (bio->bi_vcnt >= bio->bi_max_vecs)
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 31/33] block: bio: pass segments to bio if bio_add_page() is bypassed
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (29 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 30/33] block: enable multipage bvecs Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 32/33] block: always define BIO_MAX_PAGES as 256 Ming Lei
` (3 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
Under some situations, such as block direct I/O, we can't use
bio_add_page() for merging pages into multipage bvec, so
a new function is implemented for converting page array into one
segment array, then these cases can benefit from multipage bvec
too.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
block/bio.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++------
1 file changed, 48 insertions(+), 6 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index bc3992f52fe8..b7d9089cb28f 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -913,6 +913,41 @@ int bio_add_page(struct bio *bio, struct page *page,
}
EXPORT_SYMBOL(bio_add_page);
+static unsigned convert_to_segs(struct bio* bio, struct page **pages,
+ unsigned char *page_cnt,
+ unsigned nr_pages)
+{
+
+ unsigned idx;
+ unsigned nr_seg = 0;
+ struct request_queue *q = NULL;
+
+ if (bio->bi_disk)
+ q = bio->bi_disk->queue;
+
+ if (!q || !blk_queue_cluster(q)) {
+ memset(page_cnt, 0, nr_pages);
+ return nr_pages;
+ }
+
+ page_cnt[nr_seg] = 0;
+ for (idx = 1; idx < nr_pages; idx++) {
+ struct page *pg_s = pages[nr_seg];
+ struct page *pg = pages[idx];
+
+ if (page_to_pfn(pg_s) + page_cnt[nr_seg] + 1 ==
+ page_to_pfn(pg)) {
+ page_cnt[nr_seg]++;
+ } else {
+ page_cnt[++nr_seg] = 0;
+ if (nr_seg < idx)
+ pages[nr_seg] = pg;
+ }
+ }
+
+ return nr_seg + 1;
+}
+
/**
* bio_iov_iter_get_pages - pin user or kernel pages and add them to a bio
* @bio: bio to add pages to
@@ -928,6 +963,8 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)
struct page **pages = (struct page **)bv;
size_t offset, diff;
ssize_t size;
+ unsigned short nr_segs;
+ unsigned char page_cnt[nr_pages]; /* at most 256 pages */
size = iov_iter_get_pages(iter, pages, LONG_MAX, nr_pages, &offset);
if (unlikely(size <= 0))
@@ -943,13 +980,18 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)
* need to be reflected here as well.
*/
bio->bi_iter.bi_size += size;
- bio->bi_vcnt += nr_pages;
-
diff = (nr_pages * PAGE_SIZE - offset) - size;
- while (nr_pages--) {
- bv[nr_pages].bv_page = pages[nr_pages];
- bv[nr_pages].bv_len = PAGE_SIZE;
- bv[nr_pages].bv_offset = 0;
+
+ /* convert into segments */
+ nr_segs = convert_to_segs(bio, pages, page_cnt, nr_pages);
+ bio->bi_vcnt += nr_segs;
+
+ while (nr_segs--) {
+ unsigned cnt = (unsigned)page_cnt[nr_segs] + 1;
+
+ bv[nr_segs].bv_page = pages[nr_segs];
+ bv[nr_segs].bv_len = PAGE_SIZE * cnt;
+ bv[nr_segs].bv_offset = 0;
}
bv[0].bv_offset += offset;
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 32/33] block: always define BIO_MAX_PAGES as 256
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (30 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 31/33] block: bio: pass segments to bio if bio_add_page() is bypassed Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 33/33] block: document usage of bio iterator helpers Ming Lei
` (2 subsequent siblings)
34 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
Now multipage bvec can cover CONFIG_THP_SWAP, so we don't need to
increase BIO_MAX_PAGES for it.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
include/linux/bio.h | 8 --------
1 file changed, 8 deletions(-)
diff --git a/include/linux/bio.h b/include/linux/bio.h
index fc8a8238805e..839dddf81d09 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -38,15 +38,7 @@
#define BIO_BUG_ON
#endif
-#ifdef CONFIG_THP_SWAP
-#if HPAGE_PMD_NR > 256
-#define BIO_MAX_PAGES HPAGE_PMD_NR
-#else
#define BIO_MAX_PAGES 256
-#endif
-#else
-#define BIO_MAX_PAGES 256
-#endif
#define bio_prio(bio) (bio)->bi_ioprio
#define bio_set_prio(bio, prio) ((bio)->bi_ioprio = prio)
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [RESEND PATCH V5 33/33] block: document usage of bio iterator helpers
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (31 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 32/33] block: always define BIO_MAX_PAGES as 256 Ming Lei
@ 2018-05-25 3:46 ` Ming Lei
2018-05-25 16:16 ` Randy Dunlap
2018-05-25 4:53 ` [RESEND PATCH V5 00/33] block: support multipage bvec Kent Overstreet
2018-06-01 14:09 ` David Sterba
34 siblings, 1 reply; 48+ messages in thread
From: Ming Lei @ 2018-05-25 3:46 UTC (permalink / raw)
To: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana, Ming Lei
Now multipage bvec is supported, and some helpers may return page by
page, and some may return segment by segment, this patch documents the
usage for helping us use them correctly.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
Documentation/block/biovecs.txt | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)
diff --git a/Documentation/block/biovecs.txt b/Documentation/block/biovecs.txt
index b4d238b8d9fc..32a6643caeca 100644
--- a/Documentation/block/biovecs.txt
+++ b/Documentation/block/biovecs.txt
@@ -117,3 +117,35 @@ Other implications:
size limitations and the limitations of the underlying devices. Thus
there's no need to define ->merge_bvec_fn() callbacks for individual block
drivers.
+
+Usage of helpers:
+=================
+
+* The following helpers which name has suffix of "_all" can only be used on
+non-BIO_CLONED bio, and ususally they are used by filesystem code, and driver
+shouldn't use them becasue bio may have been splitted before they got to the
+driver:
+
+ bio_for_each_segment_all()
+ bio_for_each_page_all()
+ bio_pages_all()
+ bio_first_bvec_all()
+ bio_first_page_all()
+ bio_last_bvec_all()
+ segment_for_each_page_all()
+
+* The following helpers iterate bio page by page, and the local variable of
+'struct bio_vec' or the reference records single page io vector during the
+itearation:
+
+ bio_for_each_page()
+ bio_for_each_page_all()
+ segment_for_each_page_all()
+
+* The following helpers iterate bio segment by segment, and each segment may
+include multiple physically contiguous pages, and the local variable of
+'struct bio_vec' or the reference records multi page io vector during the
+itearation:
+
+ bio_for_each_segment()
+ bio_for_each_segment_all()
--
2.9.5
^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 12/33] block: introduce bio_segments()
2018-05-25 3:46 ` [RESEND PATCH V5 12/33] block: introduce bio_segments() Ming Lei
@ 2018-05-25 4:42 ` Kent Overstreet
0 siblings, 0 replies; 48+ messages in thread
From: Kent Overstreet @ 2018-05-25 4:42 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, Christoph Hellwig, Alexander Viro, David Sterba,
Huang Ying, linux-kernel, linux-block, linux-fsdevel, linux-mm,
Theodore Ts'o, Darrick J . Wong, Coly Li, Filipe Manana
On Fri, May 25, 2018 at 11:46:00AM +0800, Ming Lei wrote:
> There are still cases in which we need to use bio_segments() for get the
> number of segment, so introduce it.
>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
> ---
> include/linux/bio.h | 25 ++++++++++++++++++++-----
> 1 file changed, 20 insertions(+), 5 deletions(-)
>
> diff --git a/include/linux/bio.h b/include/linux/bio.h
> index 08af9272687f..b24c00f99c9c 100644
> --- a/include/linux/bio.h
> +++ b/include/linux/bio.h
> @@ -227,9 +227,9 @@ static inline bool bio_rewind_iter(struct bio *bio, struct bvec_iter *iter,
>
> #define bio_iter_last(bvec, iter) ((iter).bi_size == (bvec).bv_len)
>
> -static inline unsigned bio_pages(struct bio *bio)
> +static inline unsigned __bio_elements(struct bio *bio, bool seg)
This is a rather silly helper function, there isn't any actual code that's
shared, everything's behind an if () statement. Just open code it in bio_pages()
and bio_segments()
> {
> - unsigned segs = 0;
> + unsigned elems = 0;
> struct bio_vec bv;
> struct bvec_iter iter;
>
> @@ -249,10 +249,25 @@ static inline unsigned bio_pages(struct bio *bio)
> break;
> }
>
> - bio_for_each_page(bv, bio, iter)
> - segs++;
> + if (!seg) {
> + bio_for_each_page(bv, bio, iter)
> + elems++;
> + } else {
> + bio_for_each_segment(bv, bio, iter)
> + elems++;
> + }
> +
> + return elems;
> +}
> +
> +static inline unsigned bio_pages(struct bio *bio)
> +{
> + return __bio_elements(bio, false);
> +}
>
> - return segs;
> +static inline unsigned bio_segments(struct bio *bio)
> +{
> + return __bio_elements(bio, true);
> }
>
> /*
> --
> 2.9.5
>
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 00/33] block: support multipage bvec
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (32 preceding siblings ...)
2018-05-25 3:46 ` [RESEND PATCH V5 33/33] block: document usage of bio iterator helpers Ming Lei
@ 2018-05-25 4:53 ` Kent Overstreet
2018-05-25 16:30 ` Jens Axboe
2018-06-01 14:09 ` David Sterba
34 siblings, 1 reply; 48+ messages in thread
From: Kent Overstreet @ 2018-05-25 4:53 UTC (permalink / raw)
To: Ming Lei, Jens Axboe
Cc: Christoph Hellwig, Alexander Viro, David Sterba, Huang Ying,
linux-kernel, linux-block, linux-fsdevel, linux-mm,
Theodore Ts'o, Darrick J . Wong, Coly Li, Filipe Manana
On Fri, May 25, 2018 at 11:45:48AM +0800, Ming Lei wrote:
> Hi,
>
> This patchset brings multipage bvec into block layer:
patch series looks sane to me. goddamn that's a lot of renaming.
Things are going to get interesting when we start sticking compound pages in the
page cache, there'll be some interesting questions of semantics to deal with
then but I think getting this will only help w.r.t. plumbing that through and
not dealing with 4k pages unnecessarily - but I think even if we were to decide
that merging in bio_add_page() is not the way to go when the upper layers are
passing compound pages around already, this patch series helps because
regardless at some point everything under generic_make_request() is going to
have to deal with segments that are more than one page, and this patch series
makes that happen. So incremental progress.
Jens, any objections to getting this in?
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 33/33] block: document usage of bio iterator helpers
2018-05-25 3:46 ` [RESEND PATCH V5 33/33] block: document usage of bio iterator helpers Ming Lei
@ 2018-05-25 16:16 ` Randy Dunlap
0 siblings, 0 replies; 48+ messages in thread
From: Randy Dunlap @ 2018-05-25 16:16 UTC (permalink / raw)
To: Ming Lei, Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana
On 05/24/2018 08:46 PM, Ming Lei wrote:
> Now multipage bvec is supported, and some helpers may return page by
> page, and some may return segment by segment, this patch documents the
> usage for helping us use them correctly.
>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
> ---
> Documentation/block/biovecs.txt | 32 ++++++++++++++++++++++++++++++++
> 1 file changed, 32 insertions(+)
>
> diff --git a/Documentation/block/biovecs.txt b/Documentation/block/biovecs.txt
> index b4d238b8d9fc..32a6643caeca 100644
> --- a/Documentation/block/biovecs.txt
> +++ b/Documentation/block/biovecs.txt
> @@ -117,3 +117,35 @@ Other implications:
> size limitations and the limitations of the underlying devices. Thus
> there's no need to define ->merge_bvec_fn() callbacks for individual block
> drivers.
> +
> +Usage of helpers:
> +=================
> +
> +* The following helpers which name has suffix of "_all" can only be used on
* The following helpers, whose names have the suffix "_all", can only be used on
> +non-BIO_CLONED bio, and ususally they are used by filesystem code, and driver
usually
> +shouldn't use them becasue bio may have been splitted before they got to the
because split
> +driver:
> +
> + bio_for_each_segment_all()
> + bio_for_each_page_all()
> + bio_pages_all()
> + bio_first_bvec_all()
> + bio_first_page_all()
> + bio_last_bvec_all()
> + segment_for_each_page_all()
> +
> +* The following helpers iterate bio page by page, and the local variable of
> +'struct bio_vec' or the reference records single page io vector during the
> +itearation:
iteration:
> +
> + bio_for_each_page()
> + bio_for_each_page_all()
> + segment_for_each_page_all()
> +
> +* The following helpers iterate bio segment by segment, and each segment may
> +include multiple physically contiguous pages, and the local variable of
> +'struct bio_vec' or the reference records multi page io vector during the
> +itearation:
iteration:
> +
> + bio_for_each_segment()
> + bio_for_each_segment_all()
>
--
~Randy
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 00/33] block: support multipage bvec
2018-05-25 4:53 ` [RESEND PATCH V5 00/33] block: support multipage bvec Kent Overstreet
@ 2018-05-25 16:30 ` Jens Axboe
2018-05-27 7:23 ` Ming Lei
0 siblings, 1 reply; 48+ messages in thread
From: Jens Axboe @ 2018-05-25 16:30 UTC (permalink / raw)
To: Kent Overstreet, Ming Lei
Cc: Christoph Hellwig, Alexander Viro, David Sterba, Huang Ying,
linux-kernel, linux-block, linux-fsdevel, linux-mm,
Theodore Ts'o, Darrick J . Wong, Coly Li, Filipe Manana
On 5/24/18 10:53 PM, Kent Overstreet wrote:
> On Fri, May 25, 2018 at 11:45:48AM +0800, Ming Lei wrote:
>> Hi,
>>
>> This patchset brings multipage bvec into block layer:
>
> patch series looks sane to me. goddamn that's a lot of renaming.
Indeed... I actually objected to some of the segment -> page renaming,
but it's still in there. The foo2() temporary functions also concern me,
we all know there's nothing more permanent than a temporary fixup.
> Things are going to get interesting when we start sticking compound pages in the
> page cache, there'll be some interesting questions of semantics to deal with
> then but I think getting this will only help w.r.t. plumbing that through and
> not dealing with 4k pages unnecessarily - but I think even if we were to decide
> that merging in bio_add_page() is not the way to go when the upper layers are
> passing compound pages around already, this patch series helps because
> regardless at some point everything under generic_make_request() is going to
> have to deal with segments that are more than one page, and this patch series
> makes that happen. So incremental progress.
>
> Jens, any objections to getting this in?
I like most of it, but I'd much rather get this way earlier in the series.
We're basically just one week away from the merge window, it needs more simmer
and testing time than that. On top of that, it hasn't received much review
yet.
So as far as I'm concerned, we can kick off the 4.19 block branch with
iterated versions of this patchset.
--
Jens Axboe
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 00/33] block: support multipage bvec
2018-05-25 16:30 ` Jens Axboe
@ 2018-05-27 7:23 ` Ming Lei
2018-05-28 1:44 ` Jens Axboe
0 siblings, 1 reply; 48+ messages in thread
From: Ming Lei @ 2018-05-27 7:23 UTC (permalink / raw)
To: Jens Axboe
Cc: Kent Overstreet, Christoph Hellwig, Alexander Viro, David Sterba,
Huang Ying, linux-kernel, linux-block, linux-fsdevel, linux-mm,
Theodore Ts'o, Darrick J . Wong, Coly Li, Filipe Manana
On Fri, May 25, 2018 at 10:30:46AM -0600, Jens Axboe wrote:
> On 5/24/18 10:53 PM, Kent Overstreet wrote:
> > On Fri, May 25, 2018 at 11:45:48AM +0800, Ming Lei wrote:
> >> Hi,
> >>
> >> This patchset brings multipage bvec into block layer:
> >
> > patch series looks sane to me. goddamn that's a lot of renaming.
>
> Indeed... I actually objected to some of the segment -> page renaming,
> but it's still in there. The foo2() temporary functions also concern me,
> we all know there's nothing more permanent than a temporary fixup.
Jens, I remember I explained the renaming story to you in lsfmm a bit:
1) the current naming of segment is actually wrong, since every segment
only stores one single-page vector
2) the most important part is that once multipage bvec is introduced,
if the old _segment naming is still kept, it can be very confusing,
especially no good name is left for the helpers of dealing with real
segment.
For the foo2() temporary change, that is only for avoiding tree-wide
change in one single tree, with this way, we can change sub-system one
by one, but if you think it is good to do tree-wide conversion in one
patch, I am fine to do it in next version.
>
> > Things are going to get interesting when we start sticking compound pages in the
> > page cache, there'll be some interesting questions of semantics to deal with
> > then but I think getting this will only help w.r.t. plumbing that through and
> > not dealing with 4k pages unnecessarily - but I think even if we were to decide
> > that merging in bio_add_page() is not the way to go when the upper layers are
> > passing compound pages around already, this patch series helps because
> > regardless at some point everything under generic_make_request() is going to
> > have to deal with segments that are more than one page, and this patch series
> > makes that happen. So incremental progress.
> >
> > Jens, any objections to getting this in?
>
> I like most of it, but I'd much rather get this way earlier in the series.
> We're basically just one week away from the merge window, it needs more simmer
> and testing time than that. On top of that, it hasn't received much review
> yet.
>
> So as far as I'm concerned, we can kick off the 4.19 block branch with
> iterated versions of this patchset.
OK, I will post out again once v4.19 is started.
Thanks,
Ming
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 00/33] block: support multipage bvec
2018-05-27 7:23 ` Ming Lei
@ 2018-05-28 1:44 ` Jens Axboe
2018-05-28 2:30 ` Ming Lei
2018-06-01 8:35 ` Christoph Hellwig
0 siblings, 2 replies; 48+ messages in thread
From: Jens Axboe @ 2018-05-28 1:44 UTC (permalink / raw)
To: Ming Lei
Cc: Kent Overstreet, Christoph Hellwig, Alexander Viro, David Sterba,
Huang Ying, linux-kernel, linux-block, linux-fsdevel, linux-mm,
Theodore Ts'o, Darrick J . Wong, Coly Li, Filipe Manana
On 5/27/18 1:23 AM, Ming Lei wrote:
> On Fri, May 25, 2018 at 10:30:46AM -0600, Jens Axboe wrote:
>> On 5/24/18 10:53 PM, Kent Overstreet wrote:
>>> On Fri, May 25, 2018 at 11:45:48AM +0800, Ming Lei wrote:
>>>> Hi,
>>>>
>>>> This patchset brings multipage bvec into block layer:
>>>
>>> patch series looks sane to me. goddamn that's a lot of renaming.
>>
>> Indeed... I actually objected to some of the segment -> page
>> renaming, but it's still in there. The foo2() temporary functions
>> also concern me, we all know there's nothing more permanent than a
>> temporary fixup.
>
> Jens, I remember I explained the renaming story to you in lsfmm a bit:
>
> 1) the current naming of segment is actually wrong, since every segment
> only stores one single-page vector
>
> 2) the most important part is that once multipage bvec is introduced,
> if the old _segment naming is still kept, it can be very confusing,
> especially no good name is left for the helpers of dealing with real
> segment.
Yes, we discussed exactly this, which is why I'm surprised you went
ahead with the same approach. I told you I don't like tree wide renames,
if they can be avoided. I'd rather suffer some pain wrt page vs segments
naming, and then later do a rename (if it bothers us) once the dust has
settled on the interesting part of the changes.
I'm very well away of our current naming and what it signifies. With
#1, you are really splitting hairs, imho. Find a decent name for
multiple segment. Chunk?
> For the foo2() temporary change, that is only for avoiding tree-wide
> change in one single tree, with this way, we can change sub-system one
> by one, but if you think it is good to do tree-wide conversion in one
> patch, I am fine to do it in next version.
It's still a painful middle step.
>>> Things are going to get interesting when we start sticking compound
>>> pages in the page cache, there'll be some interesting questions of
>>> semantics to deal with then but I think getting this will only help
>>> w.r.t. plumbing that through and not dealing with 4k pages
>>> unnecessarily - but I think even if we were to decide that merging
>>> in bio_add_page() is not the way to go when the upper layers are
>>> passing compound pages around already, this patch series helps
>>> because regardless at some point everything under
>>> generic_make_request() is going to have to deal with segments that
>>> are more than one page, and this patch series makes that happen. So
>>> incremental progress.
>>>
>>> Jens, any objections to getting this in?
>>
>> I like most of it, but I'd much rather get this way earlier in the
>> series. We're basically just one week away from the merge window, it
>> needs more simmer and testing time than that. On top of that, it
>> hasn't received much review yet.
>>
>> So as far as I'm concerned, we can kick off the 4.19 block branch
>> with iterated versions of this patchset.
>
> OK, I will post out again once v4.19 is started.
Sounds good.
--
Jens Axboe
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 00/33] block: support multipage bvec
2018-05-28 1:44 ` Jens Axboe
@ 2018-05-28 2:30 ` Ming Lei
2018-06-01 8:43 ` Christoph Hellwig
2018-06-01 8:35 ` Christoph Hellwig
1 sibling, 1 reply; 48+ messages in thread
From: Ming Lei @ 2018-05-28 2:30 UTC (permalink / raw)
To: Jens Axboe
Cc: Kent Overstreet, Christoph Hellwig, Alexander Viro, David Sterba,
Huang Ying, linux-kernel, linux-block, linux-fsdevel, linux-mm,
Theodore Ts'o, Darrick J . Wong, Coly Li, Filipe Manana
On Sun, May 27, 2018 at 07:44:52PM -0600, Jens Axboe wrote:
> On 5/27/18 1:23 AM, Ming Lei wrote:
> > On Fri, May 25, 2018 at 10:30:46AM -0600, Jens Axboe wrote:
> >> On 5/24/18 10:53 PM, Kent Overstreet wrote:
> >>> On Fri, May 25, 2018 at 11:45:48AM +0800, Ming Lei wrote:
> >>>> Hi,
> >>>>
> >>>> This patchset brings multipage bvec into block layer:
> >>>
> >>> patch series looks sane to me. goddamn that's a lot of renaming.
> >>
> >> Indeed... I actually objected to some of the segment -> page
> >> renaming, but it's still in there. The foo2() temporary functions
> >> also concern me, we all know there's nothing more permanent than a
> >> temporary fixup.
> >
> > Jens, I remember I explained the renaming story to you in lsfmm a bit:
> >
> > 1) the current naming of segment is actually wrong, since every segment
> > only stores one single-page vector
> >
> > 2) the most important part is that once multipage bvec is introduced,
> > if the old _segment naming is still kept, it can be very confusing,
> > especially no good name is left for the helpers of dealing with real
> > segment.
>
> Yes, we discussed exactly this, which is why I'm surprised you went
> ahead with the same approach. I told you I don't like tree wide renames,
Maybe I misunderstood your point, that isn't strange given my poor
english, :-)
> if they can be avoided. I'd rather suffer some pain wrt page vs segments
> naming, and then later do a rename (if it bothers us) once the dust has
> settled on the interesting part of the changes.
>
> I'm very well away of our current naming and what it signifies. With
> #1, you are really splitting hairs, imho. Find a decent name for
> multiple segment. Chunk?
OK, will try _chunk in next post.
>
> > For the foo2() temporary change, that is only for avoiding tree-wide
> > change in one single tree, with this way, we can change sub-system one
> > by one, but if you think it is good to do tree-wide conversion in one
> > patch, I am fine to do it in next version.
>
> It's still a painful middle step.
I hate the conversion too, but looks it can't be avoided since
bio_for_each_segment_all() has to be changed.
Could you share us what your favorite approach is for this conversion?
Thanks,
Ming
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 24/33] f2fs: conver to bio_for_each_page_all2
2018-05-25 3:46 ` [RESEND PATCH V5 24/33] f2fs: " Ming Lei
@ 2018-05-28 7:29 ` Chao Yu
0 siblings, 0 replies; 48+ messages in thread
From: Chao Yu @ 2018-05-28 7:29 UTC (permalink / raw)
To: Ming Lei, Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet
Cc: David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana
On 2018/5/25 11:46, Ming Lei wrote:
> bio_for_each_page_all() can't be used any more after multipage bvec is
> enabled, so we have to convert to bio_for_each_page_all2().
>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
Acked-by: Chao Yu <yuchao0@huawei.com>
Thanks,
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 00/33] block: support multipage bvec
2018-05-28 1:44 ` Jens Axboe
2018-05-28 2:30 ` Ming Lei
@ 2018-06-01 8:35 ` Christoph Hellwig
2018-06-02 3:29 ` Ming Lei
1 sibling, 1 reply; 48+ messages in thread
From: Christoph Hellwig @ 2018-06-01 8:35 UTC (permalink / raw)
To: Jens Axboe
Cc: Ming Lei, Kent Overstreet, Christoph Hellwig, Alexander Viro,
David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana
On Sun, May 27, 2018 at 07:44:52PM -0600, Jens Axboe wrote:
> Yes, we discussed exactly this, which is why I'm surprised you went
> ahead with the same approach. I told you I don't like tree wide renames,
> if they can be avoided. I'd rather suffer some pain wrt page vs segments
> naming, and then later do a rename (if it bothers us) once the dust has
> settled on the interesting part of the changes.
>
> I'm very well away of our current naming and what it signifies. With
> #1, you are really splitting hairs, imho. Find a decent name for
> multiple segment. Chunk?
vec?
bio_for_each_segment (page)
bio_for_each_vec (whole bvec)
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 00/33] block: support multipage bvec
2018-05-28 2:30 ` Ming Lei
@ 2018-06-01 8:43 ` Christoph Hellwig
2018-06-01 23:55 ` Ming Lei
0 siblings, 1 reply; 48+ messages in thread
From: Christoph Hellwig @ 2018-06-01 8:43 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, Kent Overstreet, Christoph Hellwig, Alexander Viro,
David Sterba, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana
On Mon, May 28, 2018 at 10:30:43AM +0800, Ming Lei wrote:
> I hate the conversion too, but looks it can't be avoided since
> bio_for_each_segment_all() has to be changed.
I guess you mean what is currently bio_for_each_page_all in your
tree which now takes a bvec_iter_all agument? We could just
try to hide the bia variable initially under an unlike to be used
name, and then slowly move to the proper bio_for_each_page_all
API unhiding it with the rename.
But I think your current bio_for_each_segment_all should just
go away. All three users really should be using better abstractions.
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 00/33] block: support multipage bvec
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
` (33 preceding siblings ...)
2018-05-25 4:53 ` [RESEND PATCH V5 00/33] block: support multipage bvec Kent Overstreet
@ 2018-06-01 14:09 ` David Sterba
2018-06-08 12:50 ` Ming Lei
34 siblings, 1 reply; 48+ messages in thread
From: David Sterba @ 2018-06-01 14:09 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, Christoph Hellwig, Alexander Viro, Kent Overstreet,
Huang Ying, linux-kernel, linux-block, linux-fsdevel, linux-mm,
Theodore Ts'o, Darrick J . Wong, Coly Li, Filipe Manana
On Fri, May 25, 2018 at 11:45:48AM +0800, Ming Lei wrote:
> fs/btrfs/check-integrity.c | 6 +-
> fs/btrfs/compression.c | 8 +-
> fs/btrfs/disk-io.c | 3 +-
> fs/btrfs/extent_io.c | 14 ++-
> fs/btrfs/file-item.c | 4 +-
> fs/btrfs/inode.c | 12 ++-
> fs/btrfs/raid56.c | 5 +-
For the btrfs bits,
Acked-by: David Sterba <dsterba@suse.com>
but that's from the bio API user perspective only, I'll leave the design
and implementation questions to others.
I've let the patchset through fstests, no problems. One thing that caught
my eye was use of the 'struct bvec_iter_all' in random functions. As
this structure is a compound of 2 others and is 40 bytes in size, I was
curious how this increased stack consumption.
Measured with -fstack-usage before and after patch 22/33 "btrfs: conver to
bio_for_each_page_all2"
-disk-io.c:btree_csum_one_bio 48 static
+disk-io.c:btree_csum_one_bio 80 static
-extent_io.c:end_bio_extent_buffer_writepage 56 static
+extent_io.c:end_bio_extent_buffer_writepage 80 static
-extent_io.c:end_bio_extent_readpage 176 dynamic,bounded
+extent_io.c:end_bio_extent_readpage 240 dynamic,bounded
-extent_io.c:end_bio_extent_writepage 56 static
+extent_io.c:end_bio_extent_writepage 120 static
-inode.c:btrfs_retry_endio 96 dynamic,bounded
+inode.c:btrfs_retry_endio 144 dynamic,bounded
-inode.c:btrfs_retry_endio_nocsum 72 dynamic,bounded
+inode.c:btrfs_retry_endio_nocsum 104 dynamic,bounded
-raid56.c:set_bio_pages_uptodate 8 static
+raid56.c:set_bio_pages_uptodate 40 static
It's not that bad, but still quite a lot just to iterate a list of bios. I
think it's worth mentioning as it affects several other filesystems and
should be possibly optimized in the future.
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 00/33] block: support multipage bvec
2018-06-01 8:43 ` Christoph Hellwig
@ 2018-06-01 23:55 ` Ming Lei
0 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-06-01 23:55 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Jens Axboe, Kent Overstreet, Alexander Viro, David Sterba,
Huang Ying, linux-kernel, linux-block, linux-fsdevel, linux-mm,
Theodore Ts'o, Darrick J . Wong, Coly Li, Filipe Manana
On Fri, Jun 01, 2018 at 01:43:02AM -0700, Christoph Hellwig wrote:
> On Mon, May 28, 2018 at 10:30:43AM +0800, Ming Lei wrote:
> > I hate the conversion too, but looks it can't be avoided since
> > bio_for_each_segment_all() has to be changed.
>
> I guess you mean what is currently bio_for_each_page_all in your
> tree which now takes a bvec_iter_all agument? We could just
> try to hide the bia variable initially under an unlike to be used
> name, and then slowly move to the proper bio_for_each_page_all
> API unhiding it with the rename.
I tried that way at the beginning, it will cause gcc warning, since
the variable will be defined in the middle of one function, and even
worse it might break nested iterator case.
Thanks,
Ming
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 00/33] block: support multipage bvec
2018-06-01 8:35 ` Christoph Hellwig
@ 2018-06-02 3:29 ` Ming Lei
0 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-06-02 3:29 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Jens Axboe, Kent Overstreet, Alexander Viro, David Sterba,
Huang Ying, linux-kernel, linux-block, linux-fsdevel, linux-mm,
Theodore Ts'o, Darrick J . Wong, Coly Li, Filipe Manana
On Fri, Jun 01, 2018 at 01:35:57AM -0700, Christoph Hellwig wrote:
> On Sun, May 27, 2018 at 07:44:52PM -0600, Jens Axboe wrote:
> > Yes, we discussed exactly this, which is why I'm surprised you went
> > ahead with the same approach. I told you I don't like tree wide renames,
> > if they can be avoided. I'd rather suffer some pain wrt page vs segments
> > naming, and then later do a rename (if it bothers us) once the dust has
> > settled on the interesting part of the changes.
> >
> > I'm very well away of our current naming and what it signifies. With
> > #1, you are really splitting hairs, imho. Find a decent name for
> > multiple segment. Chunk?
>
> vec?
>
> bio_for_each_segment (page)
> bio_for_each_vec (whole bvec)
IMO, either vec or chunk should be fine, but one thing is that
there isn't obvious difference between segment and vec/chunk,
especially the difference is much less obvious than between page
and vec/chunk/segment.
That is why I tried to suggest to introduce bio_for_each_page_*
before.
Thanks,
Ming
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [RESEND PATCH V5 00/33] block: support multipage bvec
2018-06-01 14:09 ` David Sterba
@ 2018-06-08 12:50 ` Ming Lei
0 siblings, 0 replies; 48+ messages in thread
From: Ming Lei @ 2018-06-08 12:50 UTC (permalink / raw)
To: dsterba, Jens Axboe, Christoph Hellwig, Alexander Viro,
Kent Overstreet, Huang Ying, linux-kernel, linux-block,
linux-fsdevel, linux-mm, Theodore Ts'o, Darrick J . Wong,
Coly Li, Filipe Manana
On Fri, Jun 01, 2018 at 04:09:54PM +0200, David Sterba wrote:
> On Fri, May 25, 2018 at 11:45:48AM +0800, Ming Lei wrote:
> > fs/btrfs/check-integrity.c | 6 +-
> > fs/btrfs/compression.c | 8 +-
> > fs/btrfs/disk-io.c | 3 +-
> > fs/btrfs/extent_io.c | 14 ++-
> > fs/btrfs/file-item.c | 4 +-
> > fs/btrfs/inode.c | 12 ++-
> > fs/btrfs/raid56.c | 5 +-
>
> For the btrfs bits,
> Acked-by: David Sterba <dsterba@suse.com>
>
> but that's from the bio API user perspective only, I'll leave the design
> and implementation questions to others.
>
> I've let the patchset through fstests, no problems. One thing that caught
Thanks for your test!
> my eye was use of the 'struct bvec_iter_all' in random functions. As
> this structure is a compound of 2 others and is 40 bytes in size, I was
> curious how this increased stack consumption.
>
> Measured with -fstack-usage before and after patch 22/33 "btrfs: conver to
> bio_for_each_page_all2"
>
> -disk-io.c:btree_csum_one_bio 48 static
> +disk-io.c:btree_csum_one_bio 80 static
> -extent_io.c:end_bio_extent_buffer_writepage 56 static
> +extent_io.c:end_bio_extent_buffer_writepage 80 static
> -extent_io.c:end_bio_extent_readpage 176 dynamic,bounded
> +extent_io.c:end_bio_extent_readpage 240 dynamic,bounded
> -extent_io.c:end_bio_extent_writepage 56 static
> +extent_io.c:end_bio_extent_writepage 120 static
> -inode.c:btrfs_retry_endio 96 dynamic,bounded
> +inode.c:btrfs_retry_endio 144 dynamic,bounded
> -inode.c:btrfs_retry_endio_nocsum 72 dynamic,bounded
> +inode.c:btrfs_retry_endio_nocsum 104 dynamic,bounded
> -raid56.c:set_bio_pages_uptodate 8 static
> +raid56.c:set_bio_pages_uptodate 40 static
>
> It's not that bad, but still quite a lot just to iterate a list of bios. I
> think it's worth mentioning as it affects several other filesystems and
> should be possibly optimized in the future.
OK.
We could decrease the affect by using a lightweight iterator for
bio_for_each_page_all2(), will do it in V6.
Thanks,
Ming
^ permalink raw reply [flat|nested] 48+ messages in thread
end of thread, other threads:[~2018-06-08 12:50 UTC | newest]
Thread overview: 48+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-05-25 3:45 [RESEND PATCH V5 00/33] block: support multipage bvec Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 01/33] block: rename bio_for_each_segment* with bio_for_each_page* Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 02/33] block: rename rq_for_each_segment as rq_for_each_page Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 03/33] block: rename bio_segments() with bio_pages() Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 04/33] block: introduce multipage page bvec helpers Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 05/33] block: introduce bio_for_each_segment() Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 06/33] block: use bio_for_each_segment() to compute segments count Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 07/33] block: use bio_for_each_segment() to map sg Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 08/33] block: introduce segment_last_page() Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 09/33] fs/buffer.c: use bvec iterator to truncate the bio Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 10/33] btrfs: use segment_last_page to get bio's last page Ming Lei
2018-05-25 3:45 ` [RESEND PATCH V5 11/33] block: implement bio_pages_all() via bio_for_each_page_all() Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 12/33] block: introduce bio_segments() Ming Lei
2018-05-25 4:42 ` Kent Overstreet
2018-05-25 3:46 ` [RESEND PATCH V5 13/33] block: introduce rq_for_each_segment() Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 14/33] block: loop: pass segments to iov_iter Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 15/33] block: introduce bio_clone_seg_bioset() Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 16/33] dm: clone bio via bio_clone_seg_bioset Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 17/33] block: bio: introduce bio_for_each_page_all2 and bio_for_each_segment_all Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 18/33] block: deal with dirtying pages for multipage bvec Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 19/33] block: convert to bio_for_each_page_all2() Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 20/33] md/dm/bcache: conver to bio_for_each_page_all2 and bio_for_each_segment Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 21/33] fs: conver to bio_for_each_page_all2 Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 22/33] btrfs: " Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 23/33] ext4: " Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 24/33] f2fs: " Ming Lei
2018-05-28 7:29 ` Chao Yu
2018-05-25 3:46 ` [RESEND PATCH V5 25/33] xfs: " Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 26/33] exofs: " Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 27/33] gfs2: " Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 28/33] block: kill bio_for_each_page_all() Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 29/33] block: rename bio_for_each_page_all2 as bio_for_each_page_all Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 30/33] block: enable multipage bvecs Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 31/33] block: bio: pass segments to bio if bio_add_page() is bypassed Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 32/33] block: always define BIO_MAX_PAGES as 256 Ming Lei
2018-05-25 3:46 ` [RESEND PATCH V5 33/33] block: document usage of bio iterator helpers Ming Lei
2018-05-25 16:16 ` Randy Dunlap
2018-05-25 4:53 ` [RESEND PATCH V5 00/33] block: support multipage bvec Kent Overstreet
2018-05-25 16:30 ` Jens Axboe
2018-05-27 7:23 ` Ming Lei
2018-05-28 1:44 ` Jens Axboe
2018-05-28 2:30 ` Ming Lei
2018-06-01 8:43 ` Christoph Hellwig
2018-06-01 23:55 ` Ming Lei
2018-06-01 8:35 ` Christoph Hellwig
2018-06-02 3:29 ` Ming Lei
2018-06-01 14:09 ` David Sterba
2018-06-08 12:50 ` Ming Lei
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).