All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jens Axboe <axboe@kernel.dk>
To: John Garry <john.garry@huawei.com>,
	"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>
Subject: Re: [PATCH 6/9] nvme: add support for batched completion of polled IO
Date: Wed, 13 Oct 2021 09:07:05 -0600	[thread overview]
Message-ID: <228a30bd-c931-d7c3-52dc-5ef4e98de2f4@kernel.dk> (raw)
In-Reply-To: <659e549a-db56-ecae-35a3-2f6203dc3a28@huawei.com>

On 10/13/21 3:09 AM, John Garry wrote:
> On 12/10/2021 19:17, Jens Axboe wrote:
>> Signed-off-by: Jens Axboe<axboe@kernel.dk>
>> ---
>>   drivers/nvme/host/pci.c | 69 +++++++++++++++++++++++++++++++++++++----
>>   1 file changed, 63 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
>> index 4ad63bb9f415..4713da708cd4 100644
>> --- a/drivers/nvme/host/pci.c
>> +++ b/drivers/nvme/host/pci.c
>> @@ -959,7 +959,7 @@ static blk_status_t nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
>>   	return ret;
>>   }
>>   
>> -static void nvme_pci_complete_rq(struct request *req)
>> +static void nvme_pci_unmap_rq(struct request *req)
>>   {
>>   	struct nvme_iod *iod = blk_mq_rq_to_pdu(req);
>>   	struct nvme_dev *dev = iod->nvmeq->dev;
>> @@ -969,9 +969,34 @@ static void nvme_pci_complete_rq(struct request *req)
>>   			       rq_integrity_vec(req)->bv_len, rq_data_dir(req));
>>   	if (blk_rq_nr_phys_segments(req))
>>   		nvme_unmap_data(dev, req);
>> +}
>> +
>> +static void nvme_pci_complete_rq(struct request *req)
>> +{
>> +	nvme_pci_unmap_rq(req);
>>   	nvme_complete_rq(req);
>>   }
>>   
>> +static void nvme_pci_complete_batch(struct io_batch *ib)
>> +{
>> +	struct request *req;
>> +
>> +	req = ib->req_list;
>> +	while (req) {
>> +		nvme_pci_unmap_rq(req);
> 
> This will do the DMA SG unmap per request. Often this is a performance 
> bottle neck when we have an IOMMU enabled in strict mode. So since we 
> complete in batches, could we combine all the SGs in the batch to do one 
> big DMA unmap SG, and not one-by-one?

It is indeed, I actually have a patch for persistent maps as well. But even
without that, it would make sense to handle these unmaps a bit smarter. That
requires some iommu work though which I'm not that interested in right now,
could be done on top of this one for someone motivated enough.

-- 
Jens Axboe


  reply	other threads:[~2021-10-13 15:07 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-12 18:17 [PATCHSET 0/9] Batched completions Jens Axboe
2021-10-12 18:17 ` [PATCH 1/9] block: add a struct io_batch argument to fops->iopoll() Jens Axboe
2021-10-12 18:25   ` Bart Van Assche
2021-10-12 18:28     ` Jens Axboe
2021-10-12 18:17 ` [PATCH 2/9] sbitmap: add helper to clear a batch of tags Jens Axboe
2021-10-12 18:29   ` Bart Van Assche
2021-10-12 18:34     ` Jens Axboe
2021-10-12 18:17 ` [PATCH 3/9] sbitmap: test bit before calling test_and_set_bit() Jens Axboe
2021-10-12 18:17 ` [PATCH 4/9] block: add support for blk_mq_end_request_batch() Jens Axboe
2021-10-12 18:32   ` Bart Van Assche
2021-10-12 18:55     ` Jens Axboe
2021-10-12 18:17 ` [PATCH 5/9] nvme: move the fast path nvme error and disposition helpers Jens Axboe
2021-10-13  6:57   ` Christoph Hellwig
2021-10-13  6:57     ` Christoph Hellwig
2021-10-13 14:41     ` Jens Axboe
2021-10-13 15:11       ` Christoph Hellwig
2021-10-12 18:17 ` [PATCH 6/9] nvme: add support for batched completion of polled IO Jens Axboe
2021-10-13  7:08   ` Christoph Hellwig
2021-10-13 15:10     ` Jens Axboe
2021-10-13 15:16       ` Christoph Hellwig
2021-10-13 15:42         ` Jens Axboe
2021-10-13 15:49           ` Jens Axboe
2021-10-13 15:50           ` Christoph Hellwig
2021-10-13 16:04             ` Jens Axboe
2021-10-13 16:13               ` Christoph Hellwig
2021-10-13 16:33                 ` Jens Axboe
2021-10-13 16:45                   ` Jens Axboe
2021-10-13  9:09   ` John Garry
2021-10-13 15:07     ` Jens Axboe [this message]
2021-10-12 18:17 ` [PATCH 7/9] block: assign batch completion handler in blk_poll() Jens Axboe
2021-10-12 18:17 ` [PATCH 8/9] io_uring: utilize the io_batch infrastructure for more efficient polled IO Jens Axboe
2021-10-12 18:17 ` [PATCH 9/9] nvme: wire up completion batching for the IRQ path Jens Axboe
2021-10-13  7:12   ` Christoph Hellwig
2021-10-13 15:04     ` Jens Axboe
2021-10-13 16:54 [PATCHSET v2 0/9] Batched completions Jens Axboe
2021-10-13 16:54 ` [PATCH 6/9] nvme: add support for batched completion of polled IO Jens Axboe
2021-10-14  7:43   ` Christoph Hellwig
2021-10-14 15:30     ` Jens Axboe
2021-10-14 15:34       ` Jens Axboe
2021-10-14 16:07       ` Christoph Hellwig
2021-10-14 16:11         ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=228a30bd-c931-d7c3-52dc-5ef4e98de2f4@kernel.dk \
    --to=axboe@kernel.dk \
    --cc=john.garry@huawei.com \
    --cc=linux-block@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.