From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23339C8258D for ; Tue, 24 Nov 2020 13:30:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 55DBC204EC for ; Tue, 24 Nov 2020 13:30:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="KNQXXXOs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 55DBC204EC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5A5566B0119; Tue, 24 Nov 2020 08:29:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 506866B011B; Tue, 24 Nov 2020 08:29:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 30C616B011D; Tue, 24 Nov 2020 08:29:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0121.hostedemail.com [216.40.44.121]) by kanga.kvack.org (Postfix) with ESMTP id 090216B0119 for ; Tue, 24 Nov 2020 08:29:22 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id BB1C38249980 for ; Tue, 24 Nov 2020 13:29:21 +0000 (UTC) X-FDA: 77519393322.25.fruit43_5b17b3c2736e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id A9A501804E3AF for ; Tue, 24 Nov 2020 13:29:19 +0000 (UTC) X-HE-Tag: fruit43_5b17b3c2736e X-Filterd-Recvd-Size: 34427 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Tue, 24 Nov 2020 13:29:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=5Ieqq2Esw2uH5DQ55aOZnqozMhcSVzcFskydpUrvjJE=; b=KNQXXXOsTK2/JcBFY0rAA5ZlcX 0rxJfId/S/351sZwDEqvhFA5c390U1uD4rylUhYaRKFg6CHczRQ6dUEDPhs0M7k/Ew/fVtt/Ov3Un niAnpMHWjhkbP2ser95j7h5EO/ke/sf/VNnHk/BA61UsjvvFwXXFK3SXoKswP+8eqJNkECNtH2s+3 dW14guawsNUlVQtDHx4od51EcFJ04Di1WK123CesbbjdLVyDBRRoNvDjgxWoz2zoAtO4eQNpozYzv +QPQTB8IQ6tJu6EdUxjALmqoj+dwd+Q7jxcE+yPo/Eu3D7cyiB4M1I2Gc9ofwlW8yUpSWCagJHjxu LaUf7O+A==; Received: from [2001:4bb8:180:5443:c70:4a89:bc61:3] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1khYNO-0006gl-Gq; Tue, 24 Nov 2020 13:28:59 +0000 From: Christoph Hellwig To: Jens Axboe Cc: Tejun Heo , Josef Bacik , Konrad Rzeszutek Wilk , Coly Li , Mike Snitzer , Greg Kroah-Hartman , Jan Kara , Johannes Thumshirn , dm-devel@redhat.com, Richard Weinberger , Jan Kara , linux-block@vger.kernel.org, xen-devel@lists.xenproject.org, linux-bcache@vger.kernel.org, linux-mtd@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 38/45] block: switch partition lookup to use struct block_device Date: Tue, 24 Nov 2020 14:27:44 +0100 Message-Id: <20201124132751.3747337-39-hch@lst.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201124132751.3747337-1-hch@lst.de> References: <20201124132751.3747337-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use struct block_device to lookup partitions on a disk. This removes all usage of struct hd_struct from the I/O path, and this allows removing the percpu refcount in struct hd_struct. Signed-off-by: Christoph Hellwig --- block/bio.c | 4 +- block/blk-core.c | 66 ++++++++++++++---------------- block/blk-flush.c | 2 +- block/blk-mq.c | 9 ++-- block/blk-mq.h | 7 ++-- block/blk.h | 4 +- block/genhd.c | 56 +++++++++++++------------ block/partitions/core.c | 7 +--- drivers/block/drbd/drbd_receiver.c | 2 +- drivers/block/drbd/drbd_worker.c | 2 +- drivers/block/zram/zram_drv.c | 2 +- drivers/md/bcache/request.c | 4 +- drivers/md/dm.c | 4 +- drivers/md/md.c | 4 +- drivers/nvme/target/admin-cmd.c | 20 ++++----- fs/ext4/super.c | 18 +++----- fs/ext4/sysfs.c | 10 +---- fs/f2fs/f2fs.h | 2 +- fs/f2fs/super.c | 6 +-- include/linux/blkdev.h | 8 ++-- include/linux/genhd.h | 4 +- include/linux/part_stat.h | 17 ++++---- 22 files changed, 120 insertions(+), 138 deletions(-) diff --git a/block/bio.c b/block/bio.c index 669bb47a31988e..ebb18136b86f2f 100644 --- a/block/bio.c +++ b/block/bio.c @@ -608,12 +608,12 @@ void bio_truncate(struct bio *bio, unsigned new_siz= e) void guard_bio_eod(struct bio *bio) { sector_t maxsector; - struct hd_struct *part; + struct block_device *part; =20 rcu_read_lock(); part =3D __disk_get_part(bio->bi_disk, bio->bi_partno); if (part) - maxsector =3D bdev_nr_sectors(part->bdev); + maxsector =3D bdev_nr_sectors(part); else=09 maxsector =3D get_capacity(bio->bi_disk); rcu_read_unlock(); diff --git a/block/blk-core.c b/block/blk-core.c index 9ea70275fc1cfe..cee568389b7e11 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -666,10 +666,9 @@ static int __init setup_fail_make_request(char *str) } __setup("fail_make_request=3D", setup_fail_make_request); =20 -static bool should_fail_request(struct hd_struct *part, unsigned int byt= es) +static bool should_fail_request(struct block_device *part, unsigned int = bytes) { - return part->bdev->bd_make_it_fail && - should_fail(&fail_make_request, bytes); + return part->bd_make_it_fail && should_fail(&fail_make_request, bytes); } =20 static int __init fail_make_request_debugfs(void) @@ -684,7 +683,7 @@ late_initcall(fail_make_request_debugfs); =20 #else /* CONFIG_FAIL_MAKE_REQUEST */ =20 -static inline bool should_fail_request(struct hd_struct *part, +static inline bool should_fail_request(struct block_device *part, unsigned int bytes) { return false; @@ -692,11 +691,11 @@ static inline bool should_fail_request(struct hd_st= ruct *part, =20 #endif /* CONFIG_FAIL_MAKE_REQUEST */ =20 -static inline bool bio_check_ro(struct bio *bio, struct hd_struct *part) +static inline bool bio_check_ro(struct bio *bio, struct block_device *pa= rt) { const int op =3D bio_op(bio); =20 - if (part->bdev->bd_read_only && op_is_write(op)) { + if (part->bd_read_only && op_is_write(op)) { char b[BDEVNAME_SIZE]; =20 if (op_is_flush(bio->bi_opf) && !bio_sectors(bio)) @@ -704,7 +703,7 @@ static inline bool bio_check_ro(struct bio *bio, stru= ct hd_struct *part) =20 WARN_ONCE(1, "Trying to write to read-only block-device %s (partno %d)\n", - bio_devname(bio, b), part->partno); + bio_devname(bio, b), part->bd_partno); /* Older lvm-tools actually trigger this */ return false; } @@ -714,8 +713,7 @@ static inline bool bio_check_ro(struct bio *bio, stru= ct hd_struct *part) =20 static noinline int should_fail_bio(struct bio *bio) { - if (should_fail_request(bio->bi_disk->part0->bd_part, - bio->bi_iter.bi_size)) + if (should_fail_request(bio->bi_disk->part0, bio->bi_iter.bi_size)) return -EIO; return 0; } @@ -744,7 +742,7 @@ static inline int bio_check_eod(struct bio *bio, sect= or_t maxsector) */ static inline int blk_partition_remap(struct bio *bio) { - struct hd_struct *p; + struct block_device *p; int ret =3D -EIO; =20 rcu_read_lock(); @@ -757,12 +755,12 @@ static inline int blk_partition_remap(struct bio *b= io) goto out; =20 if (bio_sectors(bio)) { - if (bio_check_eod(bio, bdev_nr_sectors(p->bdev))) + if (bio_check_eod(bio, bdev_nr_sectors(p))) goto out; - bio->bi_iter.bi_sector +=3D p->bdev->bd_start_sect; - trace_block_bio_remap(bio->bi_disk->queue, bio, part_devt(p), + bio->bi_iter.bi_sector +=3D p->bd_start_sect; + trace_block_bio_remap(bio->bi_disk->queue, bio, p->bd_dev, bio->bi_iter.bi_sector - - p->bdev->bd_start_sect); + p->bd_start_sect); } bio->bi_partno =3D 0; ret =3D 0; @@ -832,7 +830,7 @@ static noinline_for_stack bool submit_bio_checks(stru= ct bio *bio) if (unlikely(blk_partition_remap(bio))) goto end_io; } else { - if (unlikely(bio_check_ro(bio, bio->bi_disk->part0->bd_part))) + if (unlikely(bio_check_ro(bio, bio->bi_disk->part0))) goto end_io; if (unlikely(bio_check_eod(bio, get_capacity(bio->bi_disk)))) goto end_io; @@ -1204,7 +1202,7 @@ blk_status_t blk_insert_cloned_request(struct reque= st_queue *q, struct request * return ret; =20 if (rq->rq_disk && - should_fail_request(rq->rq_disk->part0->bd_part, blk_rq_bytes(rq))) + should_fail_request(rq->rq_disk->part0, blk_rq_bytes(rq))) return BLK_STS_IOERR; =20 if (blk_crypto_insert_cloned_request(rq)) @@ -1263,17 +1261,18 @@ unsigned int blk_rq_err_bytes(const struct reques= t *rq) } EXPORT_SYMBOL_GPL(blk_rq_err_bytes); =20 -static void update_io_ticks(struct hd_struct *part, unsigned long now, b= ool end) +static void update_io_ticks(struct block_device *part, unsigned long now= , + bool end) { unsigned long stamp; again: - stamp =3D READ_ONCE(part->bdev->bd_stamp); + stamp =3D READ_ONCE(part->bd_stamp); if (unlikely(stamp !=3D now)) { - if (likely(cmpxchg(&part->bdev->bd_stamp, stamp, now) =3D=3D stamp)) + if (likely(cmpxchg(&part->bd_stamp, stamp, now) =3D=3D stamp)) __part_stat_add(part, io_ticks, end ? now - stamp : 1); } - if (part->partno) { - part =3D part_to_disk(part)->part0->bd_part; + if (part->bd_partno) { + part =3D bdev_whole(part); goto again; } } @@ -1282,11 +1281,9 @@ static void blk_account_io_completion(struct reque= st *req, unsigned int bytes) { if (req->part && blk_do_io_stat(req)) { const int sgrp =3D op_stat_group(req_op(req)); - struct hd_struct *part; =20 part_stat_lock(); - part =3D req->part; - part_stat_add(part, sectors[sgrp], bytes >> 9); + part_stat_add(req->part, sectors[sgrp], bytes >> 9); part_stat_unlock(); } } @@ -1301,14 +1298,11 @@ void blk_account_io_done(struct request *req, u64= now) if (req->part && blk_do_io_stat(req) && !(req->rq_flags & RQF_FLUSH_SEQ)) { const int sgrp =3D op_stat_group(req_op(req)); - struct hd_struct *part; =20 part_stat_lock(); - part =3D req->part; - - update_io_ticks(part, jiffies, true); - part_stat_inc(part, ios[sgrp]); - part_stat_add(part, nsecs[sgrp], now - req->start_time_ns); + update_io_ticks(req->part, jiffies, true); + part_stat_inc(req->part, ios[sgrp]); + part_stat_add(req->part, nsecs[sgrp], now - req->start_time_ns); part_stat_unlock(); } } @@ -1325,7 +1319,7 @@ void blk_account_io_start(struct request *rq) part_stat_unlock(); } =20 -static unsigned long __part_start_io_acct(struct hd_struct *part, +static unsigned long __part_start_io_acct(struct block_device *part, unsigned int sectors, unsigned int op) { const int sgrp =3D op_stat_group(op); @@ -1341,7 +1335,7 @@ static unsigned long __part_start_io_acct(struct hd= _struct *part, return now; } =20 -unsigned long part_start_io_acct(struct gendisk *disk, struct hd_struct = **part, +unsigned long part_start_io_acct(struct gendisk *disk, struct block_devi= ce **part, struct bio *bio) { *part =3D disk_map_sector_rcu(disk, bio->bi_iter.bi_sector); @@ -1353,11 +1347,11 @@ EXPORT_SYMBOL_GPL(part_start_io_acct); unsigned long disk_start_io_acct(struct gendisk *disk, unsigned int sect= ors, unsigned int op) { - return __part_start_io_acct(disk->part0->bd_part, sectors, op); + return __part_start_io_acct(disk->part0, sectors, op); } EXPORT_SYMBOL(disk_start_io_acct); =20 -static void __part_end_io_acct(struct hd_struct *part, unsigned int op, +static void __part_end_io_acct(struct block_device *part, unsigned int o= p, unsigned long start_time) { const int sgrp =3D op_stat_group(op); @@ -1371,7 +1365,7 @@ static void __part_end_io_acct(struct hd_struct *pa= rt, unsigned int op, part_stat_unlock(); } =20 -void part_end_io_acct(struct hd_struct *part, struct bio *bio, +void part_end_io_acct(struct block_device *part, struct bio *bio, unsigned long start_time) { __part_end_io_acct(part, bio_op(bio), start_time); @@ -1381,7 +1375,7 @@ EXPORT_SYMBOL_GPL(part_end_io_acct); void disk_end_io_acct(struct gendisk *disk, unsigned int op, unsigned long start_time) { - __part_end_io_acct(disk->part0->bd_part, op, start_time); + __part_end_io_acct(disk->part0, op, start_time); } EXPORT_SYMBOL(disk_end_io_acct); =20 diff --git a/block/blk-flush.c b/block/blk-flush.c index fcd0a60574dff8..9507dcdd58814c 100644 --- a/block/blk-flush.c +++ b/block/blk-flush.c @@ -139,7 +139,7 @@ static void blk_flush_queue_rq(struct request *rq, bo= ol add_front) =20 static void blk_account_io_flush(struct request *rq) { - struct hd_struct *part =3D rq->rq_disk->part0->bd_part; + struct block_device *part =3D rq->rq_disk->part0; =20 part_stat_lock(); part_stat_inc(part, ios[STAT_FLUSH]); diff --git a/block/blk-mq.c b/block/blk-mq.c index 55bcee5dc0320c..a2593748fa5342 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -95,7 +95,7 @@ static void blk_mq_hctx_clear_pending(struct blk_mq_hw_= ctx *hctx, } =20 struct mq_inflight { - struct hd_struct *part; + struct block_device *part; unsigned int inflight[2]; }; =20 @@ -111,7 +111,8 @@ static bool blk_mq_check_inflight(struct blk_mq_hw_ct= x *hctx, return true; } =20 -unsigned int blk_mq_in_flight(struct request_queue *q, struct hd_struct = *part) +unsigned int blk_mq_in_flight(struct request_queue *q, + struct block_device *part) { struct mq_inflight mi =3D { .part =3D part }; =20 @@ -120,8 +121,8 @@ unsigned int blk_mq_in_flight(struct request_queue *q= , struct hd_struct *part) return mi.inflight[0] + mi.inflight[1]; } =20 -void blk_mq_in_flight_rw(struct request_queue *q, struct hd_struct *part= , - unsigned int inflight[2]) +void blk_mq_in_flight_rw(struct request_queue *q, struct block_device *p= art, + unsigned int inflight[2]) { struct mq_inflight mi =3D { .part =3D part }; =20 diff --git a/block/blk-mq.h b/block/blk-mq.h index a52703c98b7736..c696515766c780 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -182,9 +182,10 @@ static inline bool blk_mq_hw_queue_mapped(struct blk= _mq_hw_ctx *hctx) return hctx->nr_ctx && hctx->tags; } =20 -unsigned int blk_mq_in_flight(struct request_queue *q, struct hd_struct = *part); -void blk_mq_in_flight_rw(struct request_queue *q, struct hd_struct *part= , - unsigned int inflight[2]); +unsigned int blk_mq_in_flight(struct request_queue *q, + struct block_device *part); +void blk_mq_in_flight_rw(struct request_queue *q, struct block_device *p= art, + unsigned int inflight[2]); =20 static inline void blk_mq_put_dispatch_budget(struct request_queue *q) { diff --git a/block/blk.h b/block/blk.h index 32ac41f7557fcc..d5bf8f3a078186 100644 --- a/block/blk.h +++ b/block/blk.h @@ -215,7 +215,7 @@ static inline void elevator_exit(struct request_queue= *q, __elevator_exit(q, e); } =20 -struct hd_struct *__disk_get_part(struct gendisk *disk, int partno); +struct block_device *__disk_get_part(struct gendisk *disk, int partno); =20 ssize_t part_size_show(struct device *dev, struct device_attribute *attr= , char *buf); @@ -348,7 +348,7 @@ void blk_queue_free_zone_bitmaps(struct request_queue= *q); static inline void blk_queue_free_zone_bitmaps(struct request_queue *q) = {} #endif =20 -struct hd_struct *disk_map_sector_rcu(struct gendisk *disk, sector_t sec= tor); +struct block_device *disk_map_sector_rcu(struct gendisk *disk, sector_t = sector); =20 int blk_alloc_devt(struct hd_struct *part, dev_t *devt); void blk_free_devt(dev_t devt); diff --git a/block/genhd.c b/block/genhd.c index 3bbf6d3a69ec63..197120c0c60f23 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -116,7 +116,7 @@ static void part_stat_read_all(struct hd_struct *part= , struct disk_stats *stat) } } =20 -static unsigned int part_in_flight(struct hd_struct *part) +static unsigned int part_in_flight(struct block_device *part) { unsigned int inflight =3D 0; int cpu; @@ -131,7 +131,8 @@ static unsigned int part_in_flight(struct hd_struct *= part) return inflight; } =20 -static void part_in_flight_rw(struct hd_struct *part, unsigned int infli= ght[2]) +static void part_in_flight_rw(struct block_device *part, + unsigned int inflight[2]) { int cpu; =20 @@ -147,7 +148,7 @@ static void part_in_flight_rw(struct hd_struct *part,= unsigned int inflight[2]) inflight[1] =3D 0; } =20 -struct hd_struct *__disk_get_part(struct gendisk *disk, int partno) +struct block_device *__disk_get_part(struct gendisk *disk, int partno) { struct disk_part_tbl *ptbl =3D rcu_dereference(disk->part_tbl); =20 @@ -172,15 +173,18 @@ struct hd_struct *__disk_get_part(struct gendisk *d= isk, int partno) */ struct hd_struct *disk_get_part(struct gendisk *disk, int partno) { - struct hd_struct *part; + struct block_device *part; =20 rcu_read_lock(); part =3D __disk_get_part(disk, partno); - if (part) - get_device(part_to_dev(part)); - rcu_read_unlock(); + if (!part) { + rcu_read_unlock(); + return NULL; + } =20 - return part; + get_device(part_to_dev(part->bd_part)); + rcu_read_unlock(); + return part->bd_part; } =20 /** @@ -254,19 +258,19 @@ struct hd_struct *disk_part_iter_next(struct disk_p= art_iter *piter) =20 /* iterate to the next partition */ for (; piter->idx !=3D end; piter->idx +=3D inc) { - struct hd_struct *part; + struct block_device *part; =20 part =3D rcu_dereference(ptbl->part[piter->idx]); if (!part) continue; - if (!bdev_nr_sectors(part->bdev) && + if (!bdev_nr_sectors(part) && !(piter->flags & DISK_PITER_INCL_EMPTY) && !(piter->flags & DISK_PITER_INCL_EMPTY_PART0 && piter->idx =3D=3D 0)) continue; =20 - get_device(part_to_dev(part)); - piter->part =3D part; + get_device(part_to_dev(part->bd_part)); + piter->part =3D part->bd_part; piter->idx +=3D inc; break; } @@ -293,10 +297,10 @@ void disk_part_iter_exit(struct disk_part_iter *pit= er) } EXPORT_SYMBOL_GPL(disk_part_iter_exit); =20 -static inline int sector_in_part(struct hd_struct *part, sector_t sector= ) +static inline int sector_in_part(struct block_device *part, sector_t sec= tor) { - return part->bdev->bd_start_sect <=3D sector && - sector < part->bdev->bd_start_sect + bdev_nr_sectors(part->bdev); + return part->bd_start_sect <=3D sector && + sector < part->bd_start_sect + bdev_nr_sectors(part); } =20 /** @@ -314,10 +318,10 @@ static inline int sector_in_part(struct hd_struct *= part, sector_t sector) * Found partition on success, part0 is returned if no partition matches * or the matched partition is being deleted. */ -struct hd_struct *disk_map_sector_rcu(struct gendisk *disk, sector_t sec= tor) +struct block_device *disk_map_sector_rcu(struct gendisk *disk, sector_t = sector) { struct disk_part_tbl *ptbl; - struct hd_struct *part; + struct block_device *part; int i; =20 rcu_read_lock(); @@ -336,7 +340,7 @@ struct hd_struct *disk_map_sector_rcu(struct gendisk = *disk, sector_t sector) } } =20 - part =3D disk->part0->bd_part; + part =3D disk->part0; out_unlock: rcu_read_unlock(); return part; @@ -866,7 +870,7 @@ void del_gendisk(struct gendisk *disk) kobject_put(disk->part0->bd_holder_dir); kobject_put(disk->slave_dir); =20 - part_stat_set_all(disk->part0->bd_part, 0); + part_stat_set_all(disk->part0, 0); disk->part0->bd_stamp =3D 0; if (!sysfs_deprecated) sysfs_remove_link(block_depr, dev_name(disk_to_dev(disk))); @@ -1173,9 +1177,9 @@ ssize_t part_stat_show(struct device *dev, =20 part_stat_read_all(p, &stat); if (queue_is_mq(q)) - inflight =3D blk_mq_in_flight(q, p); + inflight =3D blk_mq_in_flight(q, p->bdev); else - inflight =3D part_in_flight(p); + inflight =3D part_in_flight(p->bdev); =20 return sprintf(buf, "%8lu %8lu %8llu %8u " @@ -1215,9 +1219,9 @@ ssize_t part_inflight_show(struct device *dev, stru= ct device_attribute *attr, unsigned int inflight[2]; =20 if (queue_is_mq(q)) - blk_mq_in_flight_rw(q, p, inflight); + blk_mq_in_flight_rw(q, p->bdev, inflight); else - part_in_flight_rw(p, inflight); + part_in_flight_rw(p->bdev, inflight); =20 return sprintf(buf, "%8u %8u\n", inflight[0], inflight[1]); } @@ -1490,9 +1494,9 @@ static int diskstats_show(struct seq_file *seqf, vo= id *v) while ((hd =3D disk_part_iter_next(&piter))) { part_stat_read_all(hd, &stat); if (queue_is_mq(gp->queue)) - inflight =3D blk_mq_in_flight(gp->queue, hd); + inflight =3D blk_mq_in_flight(gp->queue, hd->bdev); else - inflight =3D part_in_flight(hd); + inflight =3D part_in_flight(hd->bdev); =20 seq_printf(seqf, "%4d %7d %s " "%lu %lu %lu %u " @@ -1610,7 +1614,7 @@ struct gendisk *__alloc_disk_node(int minors, int n= ode_id) goto out_bdput; =20 ptbl =3D rcu_dereference_protected(disk->part_tbl, 1); - rcu_assign_pointer(ptbl->part[0], disk->part0->bd_part); + rcu_assign_pointer(ptbl->part[0], disk->part0); =20 disk->minors =3D minors; rand_initialize_disk(disk); diff --git a/block/partitions/core.c b/block/partitions/core.c index 4b0352cb29e132..8beab9e7727e27 100644 --- a/block/partitions/core.c +++ b/block/partitions/core.c @@ -298,12 +298,9 @@ void delete_partition(struct hd_struct *part) struct disk_part_tbl *ptbl =3D rcu_dereference_protected(disk->part_tbl, 1); =20 - /* - * ->part_tbl is referenced in this part's release handler, so - * we have to hold the disk device - */ rcu_assign_pointer(ptbl->part[part->partno], NULL); rcu_assign_pointer(ptbl->last_lookup, NULL); + kobject_put(part->bdev->bd_holder_dir); device_del(part_to_dev(part)); =20 @@ -421,7 +418,7 @@ static struct hd_struct *add_partition(struct gendisk= *disk, int partno, =20 /* everything is up and running, commence */ bdev_add(bdev, devt); - rcu_assign_pointer(ptbl->part[partno], p); + rcu_assign_pointer(ptbl->part[partno], bdev); =20 /* suppress uevent if the disk suppresses it */ if (!dev_get_uevent_suppress(ddev)) diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd= _receiver.c index 9e5c2fdfda3629..09c86ef3f0fd93 100644 --- a/drivers/block/drbd/drbd_receiver.c +++ b/drivers/block/drbd/drbd_receiver.c @@ -2802,7 +2802,7 @@ bool drbd_rs_c_min_rate_throttle(struct drbd_device= *device) if (c_min_rate =3D=3D 0) return false; =20 - curr_events =3D (int)part_stat_read_accum(disk->part0->bd_part, sectors= ) - + curr_events =3D (int)part_stat_read_accum(disk->part0, sectors) - atomic_read(&device->rs_sect_ev); =20 if (atomic_read(&device->ap_actlog_cnt) diff --git a/drivers/block/drbd/drbd_worker.c b/drivers/block/drbd/drbd_w= orker.c index 343f56b86bb766..02044ab7f767d5 100644 --- a/drivers/block/drbd/drbd_worker.c +++ b/drivers/block/drbd/drbd_worker.c @@ -1679,7 +1679,7 @@ void drbd_rs_controller_reset(struct drbd_device *d= evice) atomic_set(&device->rs_sect_ev, 0); device->rs_in_flight =3D 0; device->rs_last_events =3D - (int)part_stat_read_accum(disk->part0->bd_part, sectors); + (int)part_stat_read_accum(disk->part0, sectors); =20 /* Updating the RCU protected object in place is necessary since this function gets called from atomic context. diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.= c index 153858734cd47d..01757f9578dcb8 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1687,7 +1687,7 @@ static void zram_reset_device(struct zram *zram) zram->disksize =3D 0; =20 set_capacity_and_notify(zram->disk, 0); - part_stat_set_all(zram->disk->part0->bd_part, 0); + part_stat_set_all(zram->disk->part0, 0); =20 up_write(&zram->init_lock); /* I/O operation under all of CPU are done so let's free */ diff --git a/drivers/md/bcache/request.c b/drivers/md/bcache/request.c index afac8d07c1bd00..85b1f2a9b72d68 100644 --- a/drivers/md/bcache/request.c +++ b/drivers/md/bcache/request.c @@ -475,7 +475,7 @@ struct search { unsigned int read_dirty_data:1; unsigned int cache_missed:1; =20 - struct hd_struct *part; + struct block_device *part; unsigned long start_time; =20 struct btree_op op; @@ -1073,7 +1073,7 @@ struct detached_dev_io_private { unsigned long start_time; bio_end_io_t *bi_end_io; void *bi_private; - struct hd_struct *part; + struct block_device *part; }; =20 static void detached_dev_end_io(struct bio *bio) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 1b2db4d530ea71..176adcff56b380 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1607,7 +1607,7 @@ static blk_qc_t __split_and_process_bio(struct mapp= ed_device *md, * (by eliminating DM's splitting and just using bio_split) */ part_stat_lock(); - __dm_part_stat_sub(dm_disk(md)->part0->bd_part, + __dm_part_stat_sub(dm_disk(md)->part0, sectors[op_stat_group(bio_op(bio))], ci.sector_count); part_stat_unlock(); =20 @@ -2242,7 +2242,7 @@ EXPORT_SYMBOL_GPL(dm_put); static bool md_in_flight_bios(struct mapped_device *md) { int cpu; - struct hd_struct *part =3D dm_disk(md)->part0->bd_part; + struct block_device *part =3D dm_disk(md)->part0; long sum =3D 0; =20 for_each_possible_cpu(cpu) { diff --git a/drivers/md/md.c b/drivers/md/md.c index 3696c2d77a4dd7..0065736f05b428 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -464,7 +464,7 @@ struct md_io { bio_end_io_t *orig_bi_end_io; void *orig_bi_private; unsigned long start_time; - struct hd_struct *part; + struct block_device *part; }; =20 static void md_end_io(struct bio *bio) @@ -8441,7 +8441,7 @@ static int is_mddev_idle(struct mddev *mddev, int i= nit) rcu_read_lock(); rdev_for_each_rcu(rdev, mddev) { struct gendisk *disk =3D rdev->bdev->bd_disk; - curr_events =3D (int)part_stat_read_accum(disk->part0->bd_part, sector= s) - + curr_events =3D (int)part_stat_read_accum(disk->part0, sectors) - atomic_read(&disk->sync_io); /* sync IO will cause sync_io to increase before the disk_stats * as sync_io is counted when a request starts, and diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-= cmd.c index dca34489a1dc9e..8d90235e4fcc5a 100644 --- a/drivers/nvme/target/admin-cmd.c +++ b/drivers/nvme/target/admin-cmd.c @@ -89,12 +89,12 @@ static u16 nvmet_get_smart_log_nsid(struct nvmet_req = *req, if (!ns->bdev) goto out; =20 - host_reads =3D part_stat_read(ns->bdev->bd_part, ios[READ]); - data_units_read =3D DIV_ROUND_UP(part_stat_read(ns->bdev->bd_part, - sectors[READ]), 1000); - host_writes =3D part_stat_read(ns->bdev->bd_part, ios[WRITE]); - data_units_written =3D DIV_ROUND_UP(part_stat_read(ns->bdev->bd_part, - sectors[WRITE]), 1000); + host_reads =3D part_stat_read(ns->bdev, ios[READ]); + data_units_read =3D + DIV_ROUND_UP(part_stat_read(ns->bdev, sectors[READ]), 1000); + host_writes =3D part_stat_read(ns->bdev, ios[WRITE]); + data_units_written =3D + DIV_ROUND_UP(part_stat_read(ns->bdev, sectors[WRITE]), 1000); =20 put_unaligned_le64(host_reads, &slog->host_reads[0]); put_unaligned_le64(data_units_read, &slog->data_units_read[0]); @@ -120,12 +120,12 @@ static u16 nvmet_get_smart_log_all(struct nvmet_req= *req, /* we don't have the right data for file backed ns */ if (!ns->bdev) continue; - host_reads +=3D part_stat_read(ns->bdev->bd_part, ios[READ]); + host_reads +=3D part_stat_read(ns->bdev, ios[READ]); data_units_read +=3D DIV_ROUND_UP( - part_stat_read(ns->bdev->bd_part, sectors[READ]), 1000); - host_writes +=3D part_stat_read(ns->bdev->bd_part, ios[WRITE]); + part_stat_read(ns->bdev, sectors[READ]), 1000); + host_writes +=3D part_stat_read(ns->bdev, ios[WRITE]); data_units_written +=3D DIV_ROUND_UP( - part_stat_read(ns->bdev->bd_part, sectors[WRITE]), 1000); + part_stat_read(ns->bdev, sectors[WRITE]), 1000); } =20 put_unaligned_le64(host_reads, &slog->host_reads[0]); diff --git a/fs/ext4/super.c b/fs/ext4/super.c index 6633b20224d509..c303a0ff0b1701 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -4048,9 +4048,8 @@ static int ext4_fill_super(struct super_block *sb, = void *data, int silent) sbi->s_sb =3D sb; sbi->s_inode_readahead_blks =3D EXT4_DEF_INODE_READAHEAD_BLKS; sbi->s_sb_block =3D sb_block; - if (sb->s_bdev->bd_part) - sbi->s_sectors_written_start =3D - part_stat_read(sb->s_bdev->bd_part, sectors[STAT_WRITE]); + sbi->s_sectors_written_start =3D + part_stat_read(sb->s_bdev, sectors[STAT_WRITE]); =20 /* Cleanup superblock name */ strreplace(sb->s_id, '/', '!'); @@ -5509,15 +5508,10 @@ static int ext4_commit_super(struct super_block *= sb, int sync) */ if (!(sb->s_flags & SB_RDONLY)) ext4_update_tstamp(es, s_wtime); - if (sb->s_bdev->bd_part) - es->s_kbytes_written =3D - cpu_to_le64(EXT4_SB(sb)->s_kbytes_written + - ((part_stat_read(sb->s_bdev->bd_part, - sectors[STAT_WRITE]) - - EXT4_SB(sb)->s_sectors_written_start) >> 1)); - else - es->s_kbytes_written =3D - cpu_to_le64(EXT4_SB(sb)->s_kbytes_written); + es->s_kbytes_written =3D + cpu_to_le64(EXT4_SB(sb)->s_kbytes_written + + ((part_stat_read(sb->s_bdev, sectors[STAT_WRITE]) - + EXT4_SB(sb)->s_sectors_written_start) >> 1)); if (percpu_counter_initialized(&EXT4_SB(sb)->s_freeclusters_counter)) ext4_free_blocks_count_set(es, EXT4_C2B(EXT4_SB(sb), percpu_counter_sum_positive( diff --git a/fs/ext4/sysfs.c b/fs/ext4/sysfs.c index 4e27fe6ed3ae6a..075aa3a19ff5f1 100644 --- a/fs/ext4/sysfs.c +++ b/fs/ext4/sysfs.c @@ -62,11 +62,8 @@ static ssize_t session_write_kbytes_show(struct ext4_s= b_info *sbi, char *buf) { struct super_block *sb =3D sbi->s_buddy_cache->i_sb; =20 - if (!sb->s_bdev->bd_part) - return snprintf(buf, PAGE_SIZE, "0\n"); return snprintf(buf, PAGE_SIZE, "%lu\n", - (part_stat_read(sb->s_bdev->bd_part, - sectors[STAT_WRITE]) - + (part_stat_read(sb->s_bdev, sectors[STAT_WRITE]) - sbi->s_sectors_written_start) >> 1); } =20 @@ -74,12 +71,9 @@ static ssize_t lifetime_write_kbytes_show(struct ext4_= sb_info *sbi, char *buf) { struct super_block *sb =3D sbi->s_buddy_cache->i_sb; =20 - if (!sb->s_bdev->bd_part) - return snprintf(buf, PAGE_SIZE, "0\n"); return snprintf(buf, PAGE_SIZE, "%llu\n", (unsigned long long)(sbi->s_kbytes_written + - ((part_stat_read(sb->s_bdev->bd_part, - sectors[STAT_WRITE]) - + ((part_stat_read(sb->s_bdev, sectors[STAT_WRITE]) - EXT4_SB(sb)->s_sectors_written_start) >> 1))); } =20 diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index cb700d79729680..49681a8d2b14a5 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -1675,7 +1675,7 @@ static inline bool f2fs_is_multi_device(struct f2fs= _sb_info *sbi) * and the return value is in kbytes. s is of struct f2fs_sb_info. */ #define BD_PART_WRITTEN(s) \ -(((u64)part_stat_read((s)->sb->s_bdev->bd_part, sectors[STAT_WRITE]) - = \ + (((u64)part_stat_read((s)->sb->s_bdev, sectors[STAT_WRITE]) - \ (s)->sectors_written_start) >> 1) =20 static inline void f2fs_update_time(struct f2fs_sb_info *sbi, int type) diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index d4e7fab352bacb..af9f449da64bac 100644 --- a/fs/f2fs/super.c +++ b/fs/f2fs/super.c @@ -3700,10 +3700,8 @@ static int f2fs_fill_super(struct super_block *sb,= void *data, int silent) } =20 /* For write statistics */ - if (sb->s_bdev->bd_part) - sbi->sectors_written_start =3D - (u64)part_stat_read(sb->s_bdev->bd_part, - sectors[STAT_WRITE]); + sbi->sectors_written_start =3D + (u64)part_stat_read(sb->s_bdev, sectors[STAT_WRITE]); =20 /* Read accumulated write IO statistics if exists */ seg_i =3D CURSEG_I(sbi, CURSEG_HOT_NODE); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 8fc0b266610f7f..0e83989b9678c3 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -191,7 +191,7 @@ struct request { }; =20 struct gendisk *rq_disk; - struct hd_struct *part; + struct block_device *part; #ifdef CONFIG_BLK_RQ_ALLOC_TIME /* Time that the first bio started allocating this request. */ u64 alloc_time_ns; @@ -1943,9 +1943,9 @@ unsigned long disk_start_io_acct(struct gendisk *di= sk, unsigned int sectors, void disk_end_io_acct(struct gendisk *disk, unsigned int op, unsigned long start_time); =20 -unsigned long part_start_io_acct(struct gendisk *disk, struct hd_struct = **part, - struct bio *bio); -void part_end_io_acct(struct hd_struct *part, struct bio *bio, +unsigned long part_start_io_acct(struct gendisk *disk, + struct block_device **part, struct bio *bio); +void part_end_io_acct(struct block_device *part, struct bio *bio, unsigned long start_time); =20 /** diff --git a/include/linux/genhd.h b/include/linux/genhd.h index 6e16c264439bdb..77443a1031e373 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -131,8 +131,8 @@ enum { struct disk_part_tbl { struct rcu_head rcu_head; int len; - struct hd_struct __rcu *last_lookup; - struct hd_struct __rcu *part[]; + struct block_device __rcu *last_lookup; + struct block_device __rcu *part[]; }; =20 struct disk_events; diff --git a/include/linux/part_stat.h b/include/linux/part_stat.h index 680de036691ef9..d2558121d48c00 100644 --- a/include/linux/part_stat.h +++ b/include/linux/part_stat.h @@ -25,26 +25,26 @@ struct disk_stats { #define part_stat_unlock() preempt_enable() =20 #define part_stat_get_cpu(part, field, cpu) \ - (per_cpu_ptr((part)->bdev->bd_stats, (cpu))->field) + (per_cpu_ptr((part)->bd_stats, (cpu))->field) =20 #define part_stat_get(part, field) \ part_stat_get_cpu(part, field, smp_processor_id()) =20 #define part_stat_read(part, field) \ ({ \ - typeof((part)->bdev->bd_stats->field) res =3D 0; \ + typeof((part)->bd_stats->field) res =3D 0; \ unsigned int _cpu; \ for_each_possible_cpu(_cpu) \ - res +=3D per_cpu_ptr((part)->bdev->bd_stats, _cpu)->field; \ + res +=3D per_cpu_ptr((part)->bd_stats, _cpu)->field; \ res; \ }) =20 -static inline void part_stat_set_all(struct hd_struct *part, int value) +static inline void part_stat_set_all(struct block_device *part, int valu= e) { int i; =20 for_each_possible_cpu(i) - memset(per_cpu_ptr(part->bdev->bd_stats, i), value, + memset(per_cpu_ptr(part->bd_stats, i), value, sizeof(struct disk_stats)); } =20 @@ -54,13 +54,12 @@ static inline void part_stat_set_all(struct hd_struct= *part, int value) part_stat_read(part, field[STAT_DISCARD])) =20 #define __part_stat_add(part, field, addnd) \ - __this_cpu_add((part)->bdev->bd_stats->field, addnd) + __this_cpu_add((part)->bd_stats->field, addnd) =20 #define part_stat_add(part, field, addnd) do { \ __part_stat_add((part), field, addnd); \ - if ((part)->partno) \ - __part_stat_add(part_to_disk((part))->part0->bd_part, \ - field, addnd); \ + if ((part)->bd_partno) \ + __part_stat_add(bdev_whole(part), field, addnd); \ } while (0) =20 #define part_stat_dec(part, field) \ --=20 2.29.2