From: Jack Wang <jinpuwang@gmail.com> To: linux-block@vger.kernel.org, linux-rdma@vger.kernel.org Cc: axboe@kernel.dk, hch@infradead.org, sagi@grimberg.me, bvanassche@acm.org, leon@kernel.org, dledford@redhat.com, danil.kipnis@cloud.ionos.com, jinpu.wang@cloud.ionos.com, rpenyaev@suse.de Subject: [PATCH v5 21/25] rnbd: server: functionality for IO submission to file or block dev Date: Fri, 20 Dec 2019 16:51:05 +0100 Message-ID: <20191220155109.8959-22-jinpuwang@gmail.com> (raw) In-Reply-To: <20191220155109.8959-1-jinpuwang@gmail.com> From: Jack Wang <jinpu.wang@cloud.ionos.com> This provides helper functions for IO submission to file or block dev. Signed-off-by: Danil Kipnis <danil.kipnis@cloud.ionos.com> Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com> --- drivers/block/rnbd/rnbd-srv-dev.c | 162 ++++++++++++++++++++++++++++++ drivers/block/rnbd/rnbd-srv-dev.h | 130 ++++++++++++++++++++++++ 2 files changed, 292 insertions(+) create mode 100644 drivers/block/rnbd/rnbd-srv-dev.c create mode 100644 drivers/block/rnbd/rnbd-srv-dev.h diff --git a/drivers/block/rnbd/rnbd-srv-dev.c b/drivers/block/rnbd/rnbd-srv-dev.c new file mode 100644 index 000000000000..1f876d7fa685 --- /dev/null +++ b/drivers/block/rnbd/rnbd-srv-dev.c @@ -0,0 +1,162 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * InfiniBand Network Block Driver + * + * Copyright (c) 2014 - 2017 ProfitBricks GmbH. All rights reserved. + * Authors: Fabian Holler <mail@fholler.de> + * Jack Wang <jinpu.wang@profitbricks.com> + * Kleber Souza <kleber.souza@profitbricks.com> + * Danil Kipnis <danil.kipnis@profitbricks.com> + * Roman Penyaev <roman.penyaev@profitbricks.com> + * Milind Dumbare <Milind.dumbare@gmail.com> + * + * Copyright (c) 2017 - 2018 ProfitBricks GmbH. All rights reserved. + * Authors: Danil Kipnis <danil.kipnis@profitbricks.com> + * Roman Penyaev <roman.penyaev@profitbricks.com> + * + * Copyright (c) 2018 - 2019 1&1 IONOS Cloud GmbH. All rights reserved. + * Authors: Roman Penyaev <roman.penyaev@profitbricks.com> + * Jack Wang <jinpu.wang@cloud.ionos.com> + * Danil Kipnis <danil.kipnis@cloud.ionos.com> + */ + +/* Copyright (c) 2019 1&1 IONOS SE. All rights reserved. + * Authors: Jack Wang <jinpu.wang@cloud.ionos.com> + * Danil Kipnis <danil.kipnis@cloud.ionos.com> + * Guoqing Jiang <guoqing.jiang@cloud.ionos.com> + * Lutz Pogrell <lutz.pogrell@cloud.ionos.com> + */ +#undef pr_fmt +#define pr_fmt(fmt) KBUILD_MODNAME " L" __stringify(__LINE__) ": " fmt + +#include "rnbd-srv-dev.h" +#include "rnbd-log.h" + +struct rnbd_dev *rnbd_dev_open(const char *path, fmode_t flags, + struct bio_set *bs, rnbd_dev_io_fn io_cb) +{ + struct rnbd_dev *dev; + int ret; + + dev = kzalloc(sizeof(*dev), GFP_KERNEL); + if (!dev) + return ERR_PTR(-ENOMEM); + + dev->blk_open_flags = flags; + dev->bdev = blkdev_get_by_path(path, flags, THIS_MODULE); + ret = PTR_ERR_OR_ZERO(dev->bdev); + if (ret) + goto err; + + dev->blk_open_flags = flags; + dev->io_cb = io_cb; + bdevname(dev->bdev, dev->name); + dev->ibd_bio_set = bs; + + return dev; + +err: + kfree(dev); + return ERR_PTR(ret); +} + +void rnbd_dev_close(struct rnbd_dev *dev) +{ + blkdev_put(dev->bdev, dev->blk_open_flags); + kfree(dev); +} + +static void rnbd_dev_bi_end_io(struct bio *bio) +{ + struct rnbd_dev_blk_io *io = bio->bi_private; + + io->dev->io_cb(io->priv, blk_status_to_errno(bio->bi_status)); + bio_put(bio); +} + +/** + * rnbd_bio_map_kern - map kernel address into bio + * @q: the struct request_queue for the bio + * @data: pointer to buffer to map + * @bs: bio_set to use. + * @len: length in bytes + * @gfp_mask: allocation flags for bio allocation + * + * Map the kernel address into a bio suitable for io to a block + * device. Returns an error pointer in case of error. + */ +static struct bio *rnbd_bio_map_kern(struct request_queue *q, void *data, + struct bio_set *bs, + unsigned int len, gfp_t gfp_mask) +{ + unsigned long kaddr = (unsigned long)data; + unsigned long end = (kaddr + len + PAGE_SIZE - 1) >> PAGE_SHIFT; + unsigned long start = kaddr >> PAGE_SHIFT; + const int nr_pages = end - start; + int offset, i; + struct bio *bio; + + bio = bio_alloc_bioset(gfp_mask, nr_pages, bs); + if (!bio) + return ERR_PTR(-ENOMEM); + + offset = offset_in_page(kaddr); + for (i = 0; i < nr_pages; i++) { + unsigned int bytes = PAGE_SIZE - offset; + + if (len <= 0) + break; + + if (bytes > len) + bytes = len; + + if (bio_add_pc_page(q, bio, virt_to_page(data), bytes, + offset) < bytes) { + /* we don't support partial mappings */ + bio_put(bio); + return ERR_PTR(-EINVAL); + } + + data += bytes; + len -= bytes; + offset = 0; + } + + bio->bi_end_io = bio_put; + return bio; +} + +int rnbd_dev_submit_io(struct rnbd_dev *dev, sector_t sector, void *data, + size_t len, u32 bi_size, enum rnbd_io_flags flags, + short prio, void *priv) +{ + struct request_queue *q = bdev_get_queue(dev->bdev); + struct rnbd_dev_blk_io *io; + struct bio *bio; + + /* check if the buffer is suitable for bdev */ + if (WARN_ON(!blk_rq_aligned(q, (unsigned long)data, len))) + return -EINVAL; + + /* Generate bio with pages pointing to the rdma buffer */ + bio = rnbd_bio_map_kern(q, data, dev->ibd_bio_set, len, GFP_KERNEL); + if (IS_ERR(bio)) + return PTR_ERR(bio); + + io = container_of(bio, struct rnbd_dev_blk_io, bio); + + io->dev = dev; + io->priv = priv; + + bio->bi_end_io = rnbd_dev_bi_end_io; + bio->bi_private = io; + bio->bi_opf = rnbd_to_bio_flags(flags); + bio->bi_iter.bi_sector = sector; + bio->bi_iter.bi_size = bi_size; + bio_set_prio(bio, prio); + bio_set_dev(bio, dev->bdev); + + submit_bio(bio); + + return 0; +} diff --git a/drivers/block/rnbd/rnbd-srv-dev.h b/drivers/block/rnbd/rnbd-srv-dev.h new file mode 100644 index 000000000000..724a2a650e2d --- /dev/null +++ b/drivers/block/rnbd/rnbd-srv-dev.h @@ -0,0 +1,130 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * InfiniBand Network Block Driver + * + * Copyright (c) 2014 - 2017 ProfitBricks GmbH. All rights reserved. + * Authors: Fabian Holler <mail@fholler.de> + * Jack Wang <jinpu.wang@profitbricks.com> + * Kleber Souza <kleber.souza@profitbricks.com> + * Danil Kipnis <danil.kipnis@profitbricks.com> + * Roman Penyaev <roman.penyaev@profitbricks.com> + * Milind Dumbare <Milind.dumbare@gmail.com> + * + * Copyright (c) 2017 - 2018 ProfitBricks GmbH. All rights reserved. + * Authors: Danil Kipnis <danil.kipnis@profitbricks.com> + * Roman Penyaev <roman.penyaev@profitbricks.com> + * + * Copyright (c) 2018 - 2019 1&1 IONOS Cloud GmbH. All rights reserved. + * Authors: Roman Penyaev <roman.penyaev@profitbricks.com> + * Jack Wang <jinpu.wang@cloud.ionos.com> + * Danil Kipnis <danil.kipnis@cloud.ionos.com> + */ + +/* Copyright (c) 2019 1&1 IONOS SE. All rights reserved. + * Authors: Jack Wang <jinpu.wang@cloud.ionos.com> + * Danil Kipnis <danil.kipnis@cloud.ionos.com> + * Guoqing Jiang <guoqing.jiang@cloud.ionos.com> + * Lutz Pogrell <lutz.pogrell@cloud.ionos.com> + */ +#ifndef RNBD_SRV_DEV_H +#define RNBD_SRV_DEV_H + +#include <linux/fs.h> +#include "rnbd-proto.h" + +typedef void rnbd_dev_io_fn(void *priv, int error); + +struct rnbd_dev { + struct block_device *bdev; + struct bio_set *ibd_bio_set; + fmode_t blk_open_flags; + char name[BDEVNAME_SIZE]; + rnbd_dev_io_fn *io_cb; +}; + +struct rnbd_dev_blk_io { + struct rnbd_dev *dev; + void *priv; + /* have to be last member for front_pad usage of bioset_init */ + struct bio bio; +}; + +/** + * rnbd_dev_open() - Open a device + * @flags: open flags + * @bs: bio_set to use during block io, + * @io_cb: is called when I/O finished + */ +struct rnbd_dev *rnbd_dev_open(const char *path, fmode_t flags, + struct bio_set *bs, rnbd_dev_io_fn io_cb); + +/** + * rnbd_dev_close() - Close a device + */ +void rnbd_dev_close(struct rnbd_dev *dev); + +static inline int rnbd_dev_get_logical_bsize(const struct rnbd_dev *dev) +{ + return bdev_logical_block_size(dev->bdev); +} + +static inline int rnbd_dev_get_phys_bsize(const struct rnbd_dev *dev) +{ + return bdev_physical_block_size(dev->bdev); +} + +static inline int rnbd_dev_get_max_segs(const struct rnbd_dev *dev) +{ + return queue_max_segments(bdev_get_queue(dev->bdev)); +} + +static inline int rnbd_dev_get_max_hw_sects(const struct rnbd_dev *dev) +{ + return queue_max_hw_sectors(bdev_get_queue(dev->bdev)); +} + +static inline int +rnbd_dev_get_max_write_same_sects(const struct rnbd_dev *dev) +{ + return bdev_write_same(dev->bdev); +} + +static inline int rnbd_dev_get_secure_discard(const struct rnbd_dev *dev) +{ + return blk_queue_secure_erase(bdev_get_queue(dev->bdev)); +} + +static inline int rnbd_dev_get_max_discard_sects(const struct rnbd_dev *dev) +{ + if (!blk_queue_discard(bdev_get_queue(dev->bdev))) + return 0; + + return blk_queue_get_max_sectors(bdev_get_queue(dev->bdev), + REQ_OP_DISCARD); +} + +static inline int rnbd_dev_get_discard_granularity(const struct rnbd_dev *dev) +{ + return bdev_get_queue(dev->bdev)->limits.discard_granularity; +} + +static inline int rnbd_dev_get_discard_alignment(const struct rnbd_dev *dev) +{ + return bdev_get_queue(dev->bdev)->limits.discard_alignment; +} + +/** + * rnbd_dev_submit_io() - Submit an I/O to the disk + * @dev: device to that the I/O is submitted + * @sector: address to read/write data to + * @data: I/O data to write or buffer to read I/O date into + * @len: length of @data + * @bi_size: Amount of data that will be read/written + * @prio: IO priority + * @priv: private data passed to @io_fn + */ +int rnbd_dev_submit_io(struct rnbd_dev *dev, sector_t sector, void *data, + size_t len, u32 bi_size, enum rnbd_io_flags flags, + short prio, void *priv); + +#endif /* RNBD_SRV_DEV_H */ -- 2.17.1
next prev parent reply index Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-12-20 15:50 [PATCH v5 00/25] RTRS (former IBTRS) rdma transport library and the corresponding RNBD (former IBNBD) rdma network block device Jack Wang 2019-12-20 15:50 ` [PATCH v5 01/25] sysfs: export sysfs_remove_file_self() Jack Wang 2019-12-20 16:26 ` Jinpu Wang 2019-12-20 15:50 ` [PATCH v5 02/25] rtrs: public interface header to establish RDMA connections Jack Wang 2019-12-21 10:15 ` Leon Romanovsky 2019-12-21 14:27 ` Danil Kipnis 2019-12-22 7:36 ` Leon Romanovsky 2019-12-23 7:38 ` Jinpu Wang 2019-12-23 8:04 ` Leon Romanovsky 2019-12-23 10:31 ` Jinpu Wang 2019-12-20 15:50 ` [PATCH v5 03/25] rtrs: private headers with rtrs protocol structs and helpers Jack Wang 2019-12-20 15:50 ` [PATCH v5 04/25] rtrs: core: lib functions shared between client and server modules Jack Wang 2019-12-20 15:50 ` [PATCH v5 05/25] rtrs: client: private header with client structs and functions Jack Wang 2019-12-20 15:50 ` [PATCH v5 06/25] rtrs: client: main functionality Jack Wang 2019-12-20 15:50 ` [PATCH v5 07/25] rtrs: client: statistics functions Jack Wang 2019-12-20 15:50 ` [PATCH v5 08/25] rtrs: client: sysfs interface functions Jack Wang 2019-12-20 15:50 ` [PATCH v5 09/25] rtrs: server: private header with server structs and functions Jack Wang 2019-12-20 15:50 ` [PATCH v5 10/25] rtrs: server: main functionality Jack Wang 2019-12-20 15:50 ` [PATCH v5 11/25] rtrs: server: statistics functions Jack Wang 2019-12-20 15:50 ` [PATCH v5 12/25] rtrs: server: sysfs interface functions Jack Wang 2019-12-20 15:50 ` [PATCH v5 13/25] rtrs: include client and server modules into kernel compilation Jack Wang 2019-12-20 15:50 ` [PATCH v5 14/25] rtrs: a bit of documentation Jack Wang 2019-12-20 15:50 ` [PATCH v5 15/25] rnbd: private headers with rnbd protocol structs and helpers Jack Wang 2019-12-20 15:51 ` [PATCH v5 16/25] rnbd: client: private header with client structs and functions Jack Wang 2019-12-20 15:51 ` [PATCH v5 17/25] rnbd: client: main functionality Jack Wang 2019-12-20 15:51 ` [PATCH v5 18/25] rnbd: client: sysfs interface functions Jack Wang 2019-12-20 15:51 ` [PATCH v5 19/25] rnbd: server: private header with server structs and functions Jack Wang 2019-12-20 15:51 ` [PATCH v5 20/25] rnbd: server: main functionality Jack Wang 2019-12-20 15:51 ` Jack Wang [this message] 2019-12-20 15:51 ` [PATCH v5 22/25] rnbd: server: sysfs interface functions Jack Wang 2019-12-23 8:14 ` Leon Romanovsky 2019-12-23 8:33 ` Jinpu Wang 2019-12-20 15:51 ` [PATCH v5 23/25] rnbd: include client and server modules into kernel compilation Jack Wang 2019-12-20 15:51 ` [PATCH v5 24/25] rnbd: a bit of documentation Jack Wang 2019-12-20 15:51 ` [PATCH v5 25/25] MAINTAINERS: Add maintainers for RNBD/RTRS modules Jack Wang 2019-12-22 9:55 ` Gal Pressman 2019-12-23 7:20 ` Jinpu Wang 2019-12-21 10:17 ` [PATCH v5 00/25] RTRS (former IBTRS) rdma transport library and the corresponding RNBD (former IBNBD) rdma network block device Leon Romanovsky 2020-01-02 18:18 ` Jason Gunthorpe 2020-01-03 12:39 ` Jinpu Wang 2020-01-03 16:28 ` Bart Van Assche 2020-01-06 17:07 ` Jinpu Wang 2020-01-07 10:56 ` Jinpu Wang 2020-01-16 16:41 ` Bart Van Assche 2020-01-16 16:46 ` Jinpu Wang
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191220155109.8959-22-jinpuwang@gmail.com \ --to=jinpuwang@gmail.com \ --cc=axboe@kernel.dk \ --cc=bvanassche@acm.org \ --cc=danil.kipnis@cloud.ionos.com \ --cc=dledford@redhat.com \ --cc=hch@infradead.org \ --cc=jinpu.wang@cloud.ionos.com \ --cc=leon@kernel.org \ --cc=linux-block@vger.kernel.org \ --cc=linux-rdma@vger.kernel.org \ --cc=rpenyaev@suse.de \ --cc=sagi@grimberg.me \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-Block Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-block/0 linux-block/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-block linux-block/ https://lore.kernel.org/linux-block \ linux-block@vger.kernel.org public-inbox-index linux-block Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-block AGPL code for this site: git clone https://public-inbox.org/public-inbox.git