From: Ming Lei <ming.lei@redhat.com> To: Jens Axboe <axboe@kernel.dk> Cc: linux-block@vger.kernel.org, Hannes Reinecke <hare@suse.com>, Keith Busch <keith.busch@intel.com>, linux-nvme@lists.infradead.org, Sagi Grimberg <sagi@grimberg.me>, Ming Lei <ming.lei@redhat.com>, Dongli Zhang <dongli.zhang@oracle.com>, James Smart <james.smart@broadcom.com>, Bart Van Assche <bart.vanassche@wdc.com>, linux-scsi@vger.kernel.org, "Martin K . Petersen" <martin.petersen@oracle.com>, Christoph Hellwig <hch@lst.de>, "James E . J . Bottomley" <jejb@linux.vnet.ibm.com>, jianchao wang <jianchao.w.wang@oracle.com> Subject: [PATCH V6 5/9] blk-mq: split blk_mq_alloc_and_init_hctx into two parts Date: Wed, 17 Apr 2019 11:44:06 +0800 [thread overview] Message-ID: <20190417034410.31957-6-ming.lei@redhat.com> (raw) In-Reply-To: <20190417034410.31957-1-ming.lei@redhat.com> Split blk_mq_alloc_and_init_hctx into two parts, and one is blk_mq_alloc_hctx() which is for allocating all hctx resources, another is blk_mq_init_hctx() which is for initializing hctx, and serves as counter-part of blk_mq_exit_hctx(). Cc: Dongli Zhang <dongli.zhang@oracle.com> Cc: James Smart <james.smart@broadcom.com> Cc: Bart Van Assche <bart.vanassche@wdc.com> Cc: linux-scsi@vger.kernel.org, Cc: Martin K . Petersen <martin.petersen@oracle.com>, Cc: Christoph Hellwig <hch@lst.de>, Cc: James E . J . Bottomley <jejb@linux.vnet.ibm.com>, Cc: jianchao wang <jianchao.w.wang@oracle.com> Reviewed-by: Hannes Reinecke <hare@suse.com> Signed-off-by: Ming Lei <ming.lei@redhat.com> --- block/blk-mq.c | 76 +++++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 49 insertions(+), 27 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 4c6fbbc6791b..eeebba6ec0f7 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2304,10 +2304,38 @@ static int blk_mq_hw_ctx_size(struct blk_mq_tag_set *tag_set) return hw_ctx_size; } +static int blk_mq_init_hctx(struct request_queue *q, + struct blk_mq_tag_set *set, + struct blk_mq_hw_ctx *hctx, + unsigned hctx_idx) +{ + hctx->queue_num = hctx_idx; + + cpuhp_state_add_instance_nocalls(CPUHP_BLK_MQ_DEAD, &hctx->cpuhp_dead); + + hctx->tags = set->tags[hctx_idx]; + + if (set->ops->init_hctx && + set->ops->init_hctx(hctx, set->driver_data, hctx_idx)) + goto unregister_cpu_notifier; + + if (blk_mq_init_request(set, hctx->fq->flush_rq, hctx_idx, + hctx->numa_node)) + goto exit_hctx; + return 0; + + exit_hctx: + if (set->ops->exit_hctx) + set->ops->exit_hctx(hctx, hctx_idx); + unregister_cpu_notifier: + blk_mq_remove_cpuhp(hctx); + return -1; +} + static struct blk_mq_hw_ctx * -__blk_mq_alloc_and_init_hctx(struct request_queue *q, - struct blk_mq_tag_set *set, - unsigned hctx_idx, int node) +blk_mq_alloc_hctx(struct request_queue *q, + struct blk_mq_tag_set *set, + unsigned hctx_idx, int node) { struct blk_mq_hw_ctx *hctx; @@ -2324,8 +2352,6 @@ __blk_mq_alloc_and_init_hctx(struct request_queue *q, atomic_set(&hctx->nr_active, 0); hctx->numa_node = node; - hctx->queue_num = hctx_idx; - if (node == NUMA_NO_NODE) hctx->numa_node = set->numa_node; node = hctx->numa_node; @@ -2336,10 +2362,6 @@ __blk_mq_alloc_and_init_hctx(struct request_queue *q, hctx->queue = q; hctx->flags = set->flags & ~BLK_MQ_F_TAG_SHARED; - cpuhp_state_add_instance_nocalls(CPUHP_BLK_MQ_DEAD, &hctx->cpuhp_dead); - - hctx->tags = set->tags[hctx_idx]; - /* * Allocate space for all possible cpus to avoid allocation at * runtime @@ -2347,29 +2369,21 @@ __blk_mq_alloc_and_init_hctx(struct request_queue *q, hctx->ctxs = kmalloc_array_node(nr_cpu_ids, sizeof(void *), GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY, node); if (!hctx->ctxs) - goto unregister_cpu_notifier; + goto free_cpumask; if (sbitmap_init_node(&hctx->ctx_map, nr_cpu_ids, ilog2(8), GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY, node)) goto free_ctxs; - hctx->nr_ctx = 0; spin_lock_init(&hctx->dispatch_wait_lock); init_waitqueue_func_entry(&hctx->dispatch_wait, blk_mq_dispatch_wake); INIT_LIST_HEAD(&hctx->dispatch_wait.entry); - if (set->ops->init_hctx && - set->ops->init_hctx(hctx, set->driver_data, hctx_idx)) - goto free_bitmap; - hctx->fq = blk_alloc_flush_queue(q, hctx->numa_node, set->cmd_size, GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY); if (!hctx->fq) - goto exit_hctx; - - if (blk_mq_init_request(set, hctx->fq->flush_rq, hctx_idx, node)) - goto free_fq; + goto free_bitmap; if (hctx->flags & BLK_MQ_F_BLOCKING) init_srcu_struct(hctx->srcu); @@ -2377,17 +2391,11 @@ __blk_mq_alloc_and_init_hctx(struct request_queue *q, return hctx; - free_fq: - blk_free_flush_queue(hctx->fq); - exit_hctx: - if (set->ops->exit_hctx) - set->ops->exit_hctx(hctx, hctx_idx); free_bitmap: sbitmap_free(&hctx->ctx_map); free_ctxs: kfree(hctx->ctxs); - unregister_cpu_notifier: - blk_mq_remove_cpuhp(hctx); + free_cpumask: free_cpumask_var(hctx->cpumask); free_hctx: kfree(hctx); @@ -2742,7 +2750,21 @@ static struct blk_mq_hw_ctx *blk_mq_alloc_and_init_hctx( struct blk_mq_tag_set *set, struct request_queue *q, int hctx_idx, int node) { - return __blk_mq_alloc_and_init_hctx(q, set, hctx_idx, node); + struct blk_mq_hw_ctx *hctx; + + hctx = blk_mq_alloc_hctx(q, set, hctx_idx, node); + if (!hctx) + goto fail; + + if (blk_mq_init_hctx(q, set, hctx, hctx_idx)) + goto free_hctx; + + return hctx; + + free_hctx: + kobject_put(&hctx->kobj); + fail: + return NULL; } static void blk_mq_realloc_hw_ctxs(struct blk_mq_tag_set *set, -- 2.9.5
WARNING: multiple messages have this Message-ID (diff)
From: ming.lei@redhat.com (Ming Lei) Subject: [PATCH V6 5/9] blk-mq: split blk_mq_alloc_and_init_hctx into two parts Date: Wed, 17 Apr 2019 11:44:06 +0800 [thread overview] Message-ID: <20190417034410.31957-6-ming.lei@redhat.com> (raw) In-Reply-To: <20190417034410.31957-1-ming.lei@redhat.com> Split blk_mq_alloc_and_init_hctx into two parts, and one is blk_mq_alloc_hctx() which is for allocating all hctx resources, another is blk_mq_init_hctx() which is for initializing hctx, and serves as counter-part of blk_mq_exit_hctx(). Cc: Dongli Zhang <dongli.zhang at oracle.com> Cc: James Smart <james.smart at broadcom.com> Cc: Bart Van Assche <bart.vanassche at wdc.com> Cc: linux-scsi at vger.kernel.org, Cc: Martin K . Petersen <martin.petersen at oracle.com>, Cc: Christoph Hellwig <hch at lst.de>, Cc: James E . J . Bottomley <jejb at linux.vnet.ibm.com>, Cc: jianchao wang <jianchao.w.wang at oracle.com> Reviewed-by: Hannes Reinecke <hare at suse.com> Signed-off-by: Ming Lei <ming.lei at redhat.com> --- block/blk-mq.c | 76 +++++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 49 insertions(+), 27 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 4c6fbbc6791b..eeebba6ec0f7 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2304,10 +2304,38 @@ static int blk_mq_hw_ctx_size(struct blk_mq_tag_set *tag_set) return hw_ctx_size; } +static int blk_mq_init_hctx(struct request_queue *q, + struct blk_mq_tag_set *set, + struct blk_mq_hw_ctx *hctx, + unsigned hctx_idx) +{ + hctx->queue_num = hctx_idx; + + cpuhp_state_add_instance_nocalls(CPUHP_BLK_MQ_DEAD, &hctx->cpuhp_dead); + + hctx->tags = set->tags[hctx_idx]; + + if (set->ops->init_hctx && + set->ops->init_hctx(hctx, set->driver_data, hctx_idx)) + goto unregister_cpu_notifier; + + if (blk_mq_init_request(set, hctx->fq->flush_rq, hctx_idx, + hctx->numa_node)) + goto exit_hctx; + return 0; + + exit_hctx: + if (set->ops->exit_hctx) + set->ops->exit_hctx(hctx, hctx_idx); + unregister_cpu_notifier: + blk_mq_remove_cpuhp(hctx); + return -1; +} + static struct blk_mq_hw_ctx * -__blk_mq_alloc_and_init_hctx(struct request_queue *q, - struct blk_mq_tag_set *set, - unsigned hctx_idx, int node) +blk_mq_alloc_hctx(struct request_queue *q, + struct blk_mq_tag_set *set, + unsigned hctx_idx, int node) { struct blk_mq_hw_ctx *hctx; @@ -2324,8 +2352,6 @@ __blk_mq_alloc_and_init_hctx(struct request_queue *q, atomic_set(&hctx->nr_active, 0); hctx->numa_node = node; - hctx->queue_num = hctx_idx; - if (node == NUMA_NO_NODE) hctx->numa_node = set->numa_node; node = hctx->numa_node; @@ -2336,10 +2362,6 @@ __blk_mq_alloc_and_init_hctx(struct request_queue *q, hctx->queue = q; hctx->flags = set->flags & ~BLK_MQ_F_TAG_SHARED; - cpuhp_state_add_instance_nocalls(CPUHP_BLK_MQ_DEAD, &hctx->cpuhp_dead); - - hctx->tags = set->tags[hctx_idx]; - /* * Allocate space for all possible cpus to avoid allocation at * runtime @@ -2347,29 +2369,21 @@ __blk_mq_alloc_and_init_hctx(struct request_queue *q, hctx->ctxs = kmalloc_array_node(nr_cpu_ids, sizeof(void *), GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY, node); if (!hctx->ctxs) - goto unregister_cpu_notifier; + goto free_cpumask; if (sbitmap_init_node(&hctx->ctx_map, nr_cpu_ids, ilog2(8), GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY, node)) goto free_ctxs; - hctx->nr_ctx = 0; spin_lock_init(&hctx->dispatch_wait_lock); init_waitqueue_func_entry(&hctx->dispatch_wait, blk_mq_dispatch_wake); INIT_LIST_HEAD(&hctx->dispatch_wait.entry); - if (set->ops->init_hctx && - set->ops->init_hctx(hctx, set->driver_data, hctx_idx)) - goto free_bitmap; - hctx->fq = blk_alloc_flush_queue(q, hctx->numa_node, set->cmd_size, GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY); if (!hctx->fq) - goto exit_hctx; - - if (blk_mq_init_request(set, hctx->fq->flush_rq, hctx_idx, node)) - goto free_fq; + goto free_bitmap; if (hctx->flags & BLK_MQ_F_BLOCKING) init_srcu_struct(hctx->srcu); @@ -2377,17 +2391,11 @@ __blk_mq_alloc_and_init_hctx(struct request_queue *q, return hctx; - free_fq: - blk_free_flush_queue(hctx->fq); - exit_hctx: - if (set->ops->exit_hctx) - set->ops->exit_hctx(hctx, hctx_idx); free_bitmap: sbitmap_free(&hctx->ctx_map); free_ctxs: kfree(hctx->ctxs); - unregister_cpu_notifier: - blk_mq_remove_cpuhp(hctx); + free_cpumask: free_cpumask_var(hctx->cpumask); free_hctx: kfree(hctx); @@ -2742,7 +2750,21 @@ static struct blk_mq_hw_ctx *blk_mq_alloc_and_init_hctx( struct blk_mq_tag_set *set, struct request_queue *q, int hctx_idx, int node) { - return __blk_mq_alloc_and_init_hctx(q, set, hctx_idx, node); + struct blk_mq_hw_ctx *hctx; + + hctx = blk_mq_alloc_hctx(q, set, hctx_idx, node); + if (!hctx) + goto fail; + + if (blk_mq_init_hctx(q, set, hctx, hctx_idx)) + goto free_hctx; + + return hctx; + + free_hctx: + kobject_put(&hctx->kobj); + fail: + return NULL; } static void blk_mq_realloc_hw_ctxs(struct blk_mq_tag_set *set, -- 2.9.5
next prev parent reply other threads:[~2019-04-17 3:45 UTC|newest] Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-04-17 3:44 [PATCH V6 0/9] blk-mq: fix races related with freeing queue Ming Lei 2019-04-17 3:44 ` Ming Lei 2019-04-17 3:44 ` [PATCH V6 1/9] blk-mq: grab .q_usage_counter when queuing request from plug code path Ming Lei 2019-04-17 3:44 ` Ming Lei 2019-04-17 3:44 ` [PATCH V6 2/9] blk-mq: move cancel of requeue_work into blk_mq_release Ming Lei 2019-04-17 3:44 ` Ming Lei 2019-04-17 12:00 ` Hannes Reinecke 2019-04-17 12:00 ` Hannes Reinecke 2019-04-17 3:44 ` [PATCH V6 3/9] blk-mq: free hw queue's resource in hctx's release handler Ming Lei 2019-04-17 3:44 ` Ming Lei 2019-04-17 12:02 ` Hannes Reinecke 2019-04-17 12:02 ` Hannes Reinecke 2019-04-17 3:44 ` [PATCH V6 4/9] blk-mq: move all hctx alloction & initialization into __blk_mq_alloc_and_init_hctx Ming Lei 2019-04-17 3:44 ` Ming Lei 2019-04-17 12:03 ` Hannes Reinecke 2019-04-17 12:03 ` Hannes Reinecke 2019-04-17 3:44 ` Ming Lei [this message] 2019-04-17 3:44 ` [PATCH V6 5/9] blk-mq: split blk_mq_alloc_and_init_hctx into two parts Ming Lei 2019-04-17 3:44 ` [PATCH V6 6/9] blk-mq: always free hctx after request queue is freed Ming Lei 2019-04-17 3:44 ` Ming Lei 2019-04-17 12:08 ` Hannes Reinecke 2019-04-17 12:08 ` Hannes Reinecke 2019-04-17 12:59 ` Ming Lei 2019-04-17 12:59 ` Ming Lei 2019-04-22 3:30 ` Ming Lei 2019-04-22 3:30 ` Ming Lei 2019-04-23 11:19 ` Hannes Reinecke 2019-04-23 11:19 ` Hannes Reinecke 2019-04-23 13:30 ` Ming Lei 2019-04-23 13:30 ` Ming Lei 2019-04-23 14:07 ` Hannes Reinecke 2019-04-23 14:07 ` Hannes Reinecke 2019-04-24 1:12 ` Ming Lei 2019-04-24 1:12 ` Ming Lei 2019-04-24 1:45 ` Ming Lei 2019-04-24 1:45 ` Ming Lei 2019-04-24 5:55 ` Hannes Reinecke 2019-04-24 5:55 ` Hannes Reinecke 2019-04-17 3:44 ` [PATCH V6 7/9] blk-mq: move cancel of hctx->run_work into blk_mq_hw_sysfs_release Ming Lei 2019-04-17 3:44 ` Ming Lei 2019-04-17 3:44 ` [PATCH V6 8/9] block: don't drain in-progress dispatch in blk_cleanup_queue() Ming Lei 2019-04-17 3:44 ` Ming Lei 2019-04-17 3:44 ` [PATCH V6 9/9] nvme: hold request queue's refcount in ns's whole lifetime Ming Lei 2019-04-17 3:44 ` Ming Lei 2019-04-17 12:10 ` Hannes Reinecke 2019-04-17 12:10 ` Hannes Reinecke 2019-04-17 15:55 ` Keith Busch 2019-04-17 15:55 ` Keith Busch 2019-04-17 17:22 ` [PATCH V6 0/9] blk-mq: fix races related with freeing queue James Smart 2019-04-17 17:22 ` James Smart
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190417034410.31957-6-ming.lei@redhat.com \ --to=ming.lei@redhat.com \ --cc=axboe@kernel.dk \ --cc=bart.vanassche@wdc.com \ --cc=dongli.zhang@oracle.com \ --cc=hare@suse.com \ --cc=hch@lst.de \ --cc=james.smart@broadcom.com \ --cc=jejb@linux.vnet.ibm.com \ --cc=jianchao.w.wang@oracle.com \ --cc=keith.busch@intel.com \ --cc=linux-block@vger.kernel.org \ --cc=linux-nvme@lists.infradead.org \ --cc=linux-scsi@vger.kernel.org \ --cc=martin.petersen@oracle.com \ --cc=sagi@grimberg.me \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.