* Re: [PATCH v6 0/4] Add MMC software queue support
2019-11-22 9:50 ` Arnd Bergmann
@ 2019-11-22 13:19 ` Linus Walleij
2019-11-22 13:49 ` Arnd Bergmann
2019-11-26 7:40 ` Paolo Valente
2019-11-26 11:17 ` Hannes Reinecke
2 siblings, 1 reply; 17+ messages in thread
From: Linus Walleij @ 2019-11-22 13:19 UTC (permalink / raw)
To: Arnd Bergmann
Cc: (Exiting) Baolin Wang, Baolin Wang, Adrian Hunter, Ulf Hansson,
Asutosh Das, Orson Zhai, Lyra Zhang, Vincent Guittot, linux-mmc,
linux-kernel, Hannes Reinecke, linux-block, Paolo Valente
On Fri, Nov 22, 2019 at 10:50 AM Arnd Bergmann <arnd@arndb.de> wrote:
> I suppose to make the submission non-blocking, all operations that
> currently block in the submission path may have to be changed first.
>
> For the case of a partition switch (same for retune), I suppose
> something like this can be done:
>
> - in queue_rq() check whether a partition switch is needed. If not,
> submit the current rq
> - if a partition switch is needed, submit the partition switch cmd
> instead, and return busy status
> - when the completion arrives for the partition switch, call back into
> blk_mq to have it call queue_rq again.
>
> Or possibly even (this might not be possible without signifcant
> restructuring):
>
> - when preparing a request that would require a partition switch,
> insert another meta-request to switch the partition ahead of it.
>
> I do realize that this is a significant departure from how it was done
> in the past, but it seems cleaner that way to me.
This partition business really need a proper overhaul.
I outlined the work elsewhere but the problem is that the
eMMC "partitions" such as boot partitions and the usecase-defined
"general" partition (notice SD cards do not have this problem)
are badly integrated with the Linux partition manager.
Instead of mapping these partitions 1:1 to the Linux
partitions they are separate block devices with their own
block queue while still having a name that suggest they
are just a partition of the device. Which they are. The
only thing peculiar with them is that the firmware in the
card are aware of them, I think the partitions that are
not primary may trade update correctness for speed,
such that e.g. boot partitions may have extra redundant
pages in the device so that they never become corrupted.
But card vendors would have to comment.
This has peculiar side effects yielding weird user experiences
such that
dd if=/dev/mmcblk0 of=my-mmc-backup.img
will actually NOT make a backup of the whole device,
only the primary partition.
This should be fixed. My preferred solution would be to just
catenate the logical blocks for these partitions beyond those
of the primary partition, stash these offsets away somewhere
and when they are accessed, insert special partition switch
commands into the block scheduler just like you said.
Right now the MMC core is trying to coordinate the uses of
different partitions by arbitrating different requests from
typically 4 different block devices instead which isn't very
good to say the least.
Also each block device eats memory
and it should really just be one block device.
Yours,
Linus Walleij
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v6 0/4] Add MMC software queue support
2019-11-22 13:19 ` Linus Walleij
@ 2019-11-22 13:49 ` Arnd Bergmann
0 siblings, 0 replies; 17+ messages in thread
From: Arnd Bergmann @ 2019-11-22 13:49 UTC (permalink / raw)
To: Linus Walleij
Cc: (Exiting) Baolin Wang, Baolin Wang, Adrian Hunter, Ulf Hansson,
Asutosh Das, Orson Zhai, Lyra Zhang, Vincent Guittot, linux-mmc,
linux-kernel, Hannes Reinecke, linux-block, Paolo Valente
On Fri, Nov 22, 2019 at 2:20 PM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> On Fri, Nov 22, 2019 at 10:50 AM Arnd Bergmann <arnd@arndb.de> wrote:
>
> > I suppose to make the submission non-blocking, all operations that
> > currently block in the submission path may have to be changed first.
> >
> > For the case of a partition switch (same for retune), I suppose
> > something like this can be done:
> >
> > - in queue_rq() check whether a partition switch is needed. If not,
> > submit the current rq
> > - if a partition switch is needed, submit the partition switch cmd
> > instead, and return busy status
> > - when the completion arrives for the partition switch, call back into
> > blk_mq to have it call queue_rq again.
> >
> > Or possibly even (this might not be possible without signifcant
> > restructuring):
> >
> > - when preparing a request that would require a partition switch,
> > insert another meta-request to switch the partition ahead of it.
> >
> > I do realize that this is a significant departure from how it was done
> > in the past, but it seems cleaner that way to me.
>
> This partition business really need a proper overhaul.
>
> I outlined the work elsewhere but the problem is that the
> eMMC "partitions" such as boot partitions and the usecase-defined
> "general" partition (notice SD cards do not have this problem)
> are badly integrated with the Linux partition manager.
I think that's a totally orthogonal problem though: we may
well be able to come up with a different way of representing
the extra partitions to user space or as separate block devices,
but in the end, this looks exactly the same to mm
->queue_rq() callback. If we have send a cmd to one partition
and want to send the next cmd to another partition, we first
have to send the partition switch cmd.
Arnd
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v6 0/4] Add MMC software queue support
2019-11-22 9:50 ` Arnd Bergmann
2019-11-22 13:19 ` Linus Walleij
@ 2019-11-26 7:40 ` Paolo Valente
2019-11-26 9:54 ` Arnd Bergmann
2019-11-26 11:17 ` Hannes Reinecke
2 siblings, 1 reply; 17+ messages in thread
From: Paolo Valente @ 2019-11-26 7:40 UTC (permalink / raw)
To: Arnd Bergmann
Cc: (Exiting) Baolin Wang, Baolin Wang, Adrian Hunter, Ulf Hansson,
asutoshd, Orson Zhai, Lyra Zhang, Linus Walleij, Vincent Guittot,
linux-mmc, linux-kernel, Hannes Reinecke, linux-block
> Il giorno 22 nov 2019, alle ore 10:50, Arnd Bergmann <arnd@arndb.de> ha scritto:
>
> (adding Paolo as well, maybe he has some more insights)
>
> On Mon, Nov 18, 2019 at 11:04 AM (Exiting) Baolin Wang
> <baolin.wang@linaro.org> wrote:
>> On Tue, 12 Nov 2019 at 16:48, Baolin Wang <baolin.wang7@gmail.com> wrote:
>>> On Tue, Nov 12, 2019 at 12:59 AM Arnd Bergmann <arnd@arndb.de> wrote:
>>>> On Mon, Nov 11, 2019 at 1:58 PM Baolin Wang <baolin.wang@linaro.org> wrote:
>>>>>> - With that change in place calling a blocking __mmc_claim_host() is
>>>>>> still a problem, so there should still be a nonblocking mmc_try_claim_host()
>>>>>> for the submission path, leading to a BLK_STS_DEV_RESOURCE (?)
>>>>>> return code from mmc_mq_queue_rq(). Basically mmc_mq_queue_rq()
>>>>>> should always return right away, either after having queued the next I/O
>>>>>> or with an error, but not waiting for the device in any way.
>>>>>
>>>>> Actually not only the mmc_claim_host() will block the MMC request
>>>>> processing, in this routine, the mmc_blk_part_switch() and
>>>>> mmc_retune() can also block the request processing. Moreover the part
>>>>> switching and tuning should be sync operations, and we can not move
>>>>> them to a work or a thread.
>>>>
>>>> Ok, I see.
>>>>
>>>> Those would also cause requests to be sent to the device or the host
>>>> controller, right? Maybe we can treat them as "a non-IO request
>>>
>>> Right.
>>>
>>>> has successfully been queued to the device" events, returning
>>>> busy from the mmc_mq_queue_rq() function and then running
>>>> the queue again when they complete?
>>>
>>> Yes, seems reasonable to me.
>>>
>>>>
>>>>>> - For the packed requests, there is apparently a very simple way to implement
>>>>>> that without a software queue: mmc_mq_queue_rq() is allowed to look at
>>>>>> and dequeue all requests that are currently part of the request_queue,
>>>>>> so it should take out as many as it wants to submit at once and send
>>>>>> them all down to the driver together, avoiding the need for any further
>>>>>> round-trips to blk_mq or maintaining a queue in mmc.
>>>>>
>>>>> You mean we can dispatch a request directly from
>>>>> elevator->type->ops.dispatch_request()? but we still need some helper
>>>>> functions to check if these requests can be packed (the package
>>>>> condition), and need to invent new APIs to start a packed request (or
>>>>> using cqe interfaces, which means we still need to implement some cqe
>>>>> callbacks).
>>>>
>>>> I don't know how the dispatch_request() function fits in there,
>>>> what Hannes told me is that in ->queue_rq() you can always
>>>> look at the following requests that are already queued up
>>>> and take the next ones off the list. Looking at bd->last
>>>> tells you if there are additional requests. If there are, you can
>>>> look at the next one from blk_mq_hw_ctx (not sure how, but
>>>> should not be hard to find)
>>>>
>>>> I also see that there is a commit_rqs() callback that may
>>>> go along with queue_rq(), implementing that one could make
>>>> this easier as well.
>>>
>>> Yes, we can use queue_rq()/commit_rqs() and bd->last (now bd->last may
>>> can not work well, see [1]), but like we talked before, for packed
>>> request, we still need some new interfaces (for example, a interface
>>> used to start a packed request, and a interface used to complete a
>>> packed request), but at last we got a consensus that we should re-use
>>> the CQE interfaces instead of new invention.
>>>
>>> [1] https://lore.kernel.org/patchwork/patch/1102897/
>>>
>>>>
>>>>>> - The DMA management (bounce buffer, map, unmap) that is currently
>>>>>> done in mmc_blk_mq_issue_rq() should ideally be done in the
>>>>>> init_request()/exit_request() (?) callbacks from mmc_mq_ops so this
>>>>>> can be done asynchronously, out of the critical timing path for the
>>>>>> submission. With this, there won't be any need for a software queue.
>>>>>
>>>>> This is not true, now the blk-mq will allocate some static request
>>>>> objects (usually the static requests number should be the same with
>>>>> the hardware queue depth) saved in struct blk_mq_tags. So the
>>>>> init_request() is used to initialize the static requests when
>>>>> allocating them, and call exit_request to free the static requests
>>>>> when freeing the 'struct blk_mq_tags', such as the queue is dead. So
>>>>> we can not move the DMA management into the init_request/exit_request.
>>>>
>>>> Ok, I must have misremembered which callback that is then, but I guess
>>>> there is some other place to do it.
>>>
>>> I checked the 'struct blk_mq_ops', and I did not find a ops can be
>>> used to do DMA management. And I also checked UFS driver, it also did
>>> the DMA mapping in the queue_rq() (scsi_queue_rq() --->
>>> ufshcd_queuecommand() ---> ufshcd_map_sg()). Maybe I missed something?
>>>
>>> Moreover like I said above, for the packed request, we still need
>>> implement something (like the software queue) based on the CQE
>>> interfaces to help to handle packed requests.
>>
>> After some investigation and offline discussion with you, I still have
>> some concerns about your suggestion.
>>
>> 1) Now blk-mq have not supplied some ops to prepare a request, which is
>> used to do some DMA management asynchronously. But yes, we can
>> introduce new ops for blk-mq. But there are still some remaining
>> preparation in mmc_mq_queue_rq(), like mmc part switch. For software
>> queue, we can prepare a request totally after issuing one.
>
> I suppose to make the submission non-blocking, all operations that
> currently block in the submission path may have to be changed first.
>
> For the case of a partition switch (same for retune), I suppose
> something like this can be done:
>
> - in queue_rq() check whether a partition switch is needed. If not,
> submit the current rq
> - if a partition switch is needed, submit the partition switch cmd
> instead, and return busy status
> - when the completion arrives for the partition switch, call back into
> blk_mq to have it call queue_rq again.
>
> Or possibly even (this might not be possible without signifcant
> restructuring):
>
> - when preparing a request that would require a partition switch,
> insert another meta-request to switch the partition ahead of it.
>
> I do realize that this is a significant departure from how it was done
> in the past, but it seems cleaner that way to me.
>
>> 2) I wonder if it is appropriate that using the irq threaded context
>> to dispatch next request, actually we will still introduce a context
>> switch here. Now we will complete a request in the hard irq handler
>> and kick the softirq to do time-consuming operations, like DMA
>> unmapping , and will start next request in the hard irq handler
>> without context switch. Moreover if we remove the BLK_MQ_F_BLOCKING in
>> future like you suggested, then we can remove all context switch. And
>> I think we can dispatch next request in the softirq context (actually
>> the CQE already did).
>
> I hope Hannes (or someone else) can comment here, as I don't
> know exactly what his objection to kicking off the next cmd in the
> hardirq was.
>
> I think generally, deferring all slow operations to an irqthread
> rather than a softirq is a good idea, but I share your concern that
> this can introduce an unnecessary latency between the the
> the IRQ is signaled and the time the following cmd is sent to the
> hardware.
>
> Doing everything in a single (irqthread) context is clearly simpler,
> so this would need to be measured carefully to avoid unnecessary
> complexity, but I think don't see anything stopping us from having
> the fast-path where the low-level driver first checks for any possible
> error conditions in hardirq context and the fires off a prepared cmd
> right away whenever it can before triggering the irqthread that does
> everything else. I think this has to be a per-driver optimization, so
> the common case would just have an irqthread.
>
>> 3) For packed request support, I did not see an example that block
>> driver can dispatch a request from the IO scheduler in queue_rq() and
>> no APIs supported from blk-mq. And we do not know where can dispatch a
>> request in queue_rq(), from IO scheduler? from ctx? or from
>> hctx->dispatch list? and if this request can not be passed to host
>> now, how to do it? Seems lots of complicated things.
>
> The only way I can see is the ->last flag, so if blk_mq submits multiple
> requests in a row to queue_rq() with this flag cleared and calls
> ->commit_rqs() after the last one. This seems to be what the scsi
> disk driver and the nvme driver rely on, and we should be able to use
> it the same way for packed cmds, by checking each time in queue_rq()
> whether requests can/should be combined and reporting busy otherwise
> (after preparing a combined mmc cmd).
> blk_mq will then call commit_rqs, which should do the actual submission
> to the hardware driver.
>
> Now as you point out, the *big* problem with this is that we never
> get multiple requests together in practice, i.e. the last flag is almost
> always set, and any optimization around it has no effect.
>
> This is where I'm a bit lost in the code as well, but it seems that
> this is part of the current bfq design that only sends one request down
> the driver stack at a time, and this would have to change first before
> we can rely on this for packing requests.
>
> Paolo, can you comment on why this is currently done, or if it can
> be changed? It seems to me that sending multiple requests at
> once would also have a significant benefit on the per-request overhead
> on NVMe devices with with bfq.
>
Hi,
actually, "one request dispatched at a time" is not a peculiarity of
bfq. Any scheduler can provide only one request at a time, with the
current blk-mq API for I/O schedulers.
Yet, when it is time to refill an hardware queue, blk-mq pulls as many
requests as it deems appropriate from the scheduler, by invoking the
latter multiple times. See blk_mq_do_dispatch_sched() in
block/blk-mq-sched.c.
I don't know where the glitch for MMC is with respect to this scheme.
Thanks,
Paolo
>> Moreover, we still need some interfaces for the packed request
>> handling, from previous discussion, we still need something like MMC
>> software queue based on the CQE to help to handle the packed request.
>>
>> So I think I still need to introduce the MMC software queue, on the one
>> hand is that it can really improve the performance from fio data and
>> avoid a long latency, on the other hand we can expand it to support
>> packed request easily in future. Thanks.
>>
>> (Anyway I will still post the V7 to address Adrian's comments and to
>> see if we can get a consensus there).
>
>
> Arnd
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v6 0/4] Add MMC software queue support
2019-11-26 7:40 ` Paolo Valente
@ 2019-11-26 9:54 ` Arnd Bergmann
0 siblings, 0 replies; 17+ messages in thread
From: Arnd Bergmann @ 2019-11-26 9:54 UTC (permalink / raw)
To: Paolo Valente
Cc: (Exiting) Baolin Wang, Baolin Wang, Adrian Hunter, Ulf Hansson,
Asutosh Das, Orson Zhai, Lyra Zhang, Linus Walleij,
Vincent Guittot, linux-mmc, linux-kernel, Hannes Reinecke,
linux-block
On Tue, Nov 26, 2019 at 8:41 AM Paolo Valente <paolo.valente@linaro.org> wrote:
> > Il giorno 22 nov 2019, alle ore 10:50, Arnd Bergmann <arnd@arndb.de> ha scritto:
> > On Mon, Nov 18, 2019 at 11:04 AM (Exiting) Baolin Wang <baolin.wang@linaro.org> wrote:
> > Paolo, can you comment on why this is currently done, or if it can
> > be changed? It seems to me that sending multiple requests at
> > once would also have a significant benefit on the per-request overhead
> > on NVMe devices with with bfq.
> >
>
> Hi,
> actually, "one request dispatched at a time" is not a peculiarity of
> bfq. Any scheduler can provide only one request at a time, with the
> current blk-mq API for I/O schedulers.
>
> Yet, when it is time to refill an hardware queue, blk-mq pulls as many
> requests as it deems appropriate from the scheduler, by invoking the
> latter multiple times. See blk_mq_do_dispatch_sched() in
> block/blk-mq-sched.c.
>
> I don't know where the glitch for MMC is with respect to this scheme.
Right, this is what is puzzling me as well: in both blk_mq_do_dispatch_sched()
and blk_mq_do_dispatch_ctx(), we seem to always take one request from
the scheduler and dispatch it to the device, regardless of the driver or
the scheduler, so there should only ever be one request in the local list.
Yet, both the blk_mq_dispatch_rq_list() function and the NVMe driver
appear to be written based on the idea that there are multiple entries
in this list. The one place that I see putting multiple requests on the
local list before dispatching them is the end of
blk_mq_sched_dispatch_requests():
if (!list_empty(&rq_list)) {
...
}
} else if (has_sched_dispatch) {
blk_mq_do_dispatch_sched(hctx);
} else if (hctx->dispatch_busy) {
/* dequeue request one by one from sw queue if queue is busy */
blk_mq_do_dispatch_ctx(hctx);
} else {
-> blk_mq_flush_busy_ctxs(hctx, &rq_list); <----
blk_mq_dispatch_rq_list(q, &rq_list, false);
}
So as you said, if we use an elevator (has_sched_dispatch == true),
we only get one request, but without an elevator, we get into this
optimized path.
Could we perhaps change the ops.dispatch_request() function to pass
down the list as in https://paste.ubuntu.com/p/MfSRwKqFCs/ ?
Arnd
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v6 0/4] Add MMC software queue support
2019-11-22 9:50 ` Arnd Bergmann
2019-11-22 13:19 ` Linus Walleij
2019-11-26 7:40 ` Paolo Valente
@ 2019-11-26 11:17 ` Hannes Reinecke
2019-11-27 7:49 ` Baolin Wang
2019-11-27 9:00 ` Christoph Hellwig
2 siblings, 2 replies; 17+ messages in thread
From: Hannes Reinecke @ 2019-11-26 11:17 UTC (permalink / raw)
To: Arnd Bergmann, (Exiting) Baolin Wang
Cc: Baolin Wang, Adrian Hunter, Ulf Hansson, asutoshd, Orson Zhai,
Lyra Zhang, Linus Walleij, Vincent Guittot, linux-mmc,
linux-kernel, Hannes Reinecke, linux-block, Paolo Valente
On 11/22/19 10:50 AM, Arnd Bergmann wrote:
> (adding Paolo as well, maybe he has some more insights)
>
> On Mon, Nov 18, 2019 at 11:04 AM (Exiting) Baolin Wang
> <baolin.wang@linaro.org> wrote:
>> On Tue, 12 Nov 2019 at 16:48, Baolin Wang <baolin.wang7@gmail.com> wrote:
>>> On Tue, Nov 12, 2019 at 12:59 AM Arnd Bergmann <arnd@arndb.de> wrote:
>>>> On Mon, Nov 11, 2019 at 1:58 PM Baolin Wang <baolin.wang@linaro.org> wrote:
>>>>>> - With that change in place calling a blocking __mmc_claim_host() is
>>>>>> still a problem, so there should still be a nonblocking mmc_try_claim_host()
>>>>>> for the submission path, leading to a BLK_STS_DEV_RESOURCE (?)
>>>>>> return code from mmc_mq_queue_rq(). Basically mmc_mq_queue_rq()
>>>>>> should always return right away, either after having queued the next I/O
>>>>>> or with an error, but not waiting for the device in any way.
>>>>>
>>>>> Actually not only the mmc_claim_host() will block the MMC request
>>>>> processing, in this routine, the mmc_blk_part_switch() and
>>>>> mmc_retune() can also block the request processing. Moreover the part
>>>>> switching and tuning should be sync operations, and we can not move
>>>>> them to a work or a thread.
>>>>
>>>> Ok, I see.
>>>>
>>>> Those would also cause requests to be sent to the device or the host
>>>> controller, right? Maybe we can treat them as "a non-IO request
>>>
>>> Right.
>>>
>>>> has successfully been queued to the device" events, returning
>>>> busy from the mmc_mq_queue_rq() function and then running
>>>> the queue again when they complete?
>>>
>>> Yes, seems reasonable to me.
>>>
>>>>
>>>>>> - For the packed requests, there is apparently a very simple way to implement
>>>>>> that without a software queue: mmc_mq_queue_rq() is allowed to look at
>>>>>> and dequeue all requests that are currently part of the request_queue,
>>>>>> so it should take out as many as it wants to submit at once and send
>>>>>> them all down to the driver together, avoiding the need for any further
>>>>>> round-trips to blk_mq or maintaining a queue in mmc.
>>>>>
>>>>> You mean we can dispatch a request directly from
>>>>> elevator->type->ops.dispatch_request()? but we still need some helper
>>>>> functions to check if these requests can be packed (the package
>>>>> condition), and need to invent new APIs to start a packed request (or
>>>>> using cqe interfaces, which means we still need to implement some cqe
>>>>> callbacks).
>>>>
>>>> I don't know how the dispatch_request() function fits in there,
>>>> what Hannes told me is that in ->queue_rq() you can always
>>>> look at the following requests that are already queued up
>>>> and take the next ones off the list. Looking at bd->last
>>>> tells you if there are additional requests. If there are, you can
>>>> look at the next one from blk_mq_hw_ctx (not sure how, but
>>>> should not be hard to find)
>>>>
>>>> I also see that there is a commit_rqs() callback that may
>>>> go along with queue_rq(), implementing that one could make
>>>> this easier as well.
>>>
>>> Yes, we can use queue_rq()/commit_rqs() and bd->last (now bd->last may
>>> can not work well, see [1]), but like we talked before, for packed
>>> request, we still need some new interfaces (for example, a interface
>>> used to start a packed request, and a interface used to complete a
>>> packed request), but at last we got a consensus that we should re-use
>>> the CQE interfaces instead of new invention.
>>>
>>> [1] https://lore.kernel.org/patchwork/patch/1102897/
>>>
>>>>
>>>>>> - The DMA management (bounce buffer, map, unmap) that is currently
>>>>>> done in mmc_blk_mq_issue_rq() should ideally be done in the
>>>>>> init_request()/exit_request() (?) callbacks from mmc_mq_ops so this
>>>>>> can be done asynchronously, out of the critical timing path for the
>>>>>> submission. With this, there won't be any need for a software queue.
>>>>>
>>>>> This is not true, now the blk-mq will allocate some static request
>>>>> objects (usually the static requests number should be the same with
>>>>> the hardware queue depth) saved in struct blk_mq_tags. So the
>>>>> init_request() is used to initialize the static requests when
>>>>> allocating them, and call exit_request to free the static requests
>>>>> when freeing the 'struct blk_mq_tags', such as the queue is dead. So
>>>>> we can not move the DMA management into the init_request/exit_request.
>>>>
>>>> Ok, I must have misremembered which callback that is then, but I guess
>>>> there is some other place to do it.
>>>
>>> I checked the 'struct blk_mq_ops', and I did not find a ops can be
>>> used to do DMA management. And I also checked UFS driver, it also did
>>> the DMA mapping in the queue_rq() (scsi_queue_rq() --->
>>> ufshcd_queuecommand() ---> ufshcd_map_sg()). Maybe I missed something?
>>>
>>> Moreover like I said above, for the packed request, we still need
>>> implement something (like the software queue) based on the CQE
>>> interfaces to help to handle packed requests.
>>
>> After some investigation and offline discussion with you, I still have
>> some concerns about your suggestion.
>>
>> 1) Now blk-mq have not supplied some ops to prepare a request, which is
>> used to do some DMA management asynchronously. But yes, we can
>> introduce new ops for blk-mq. But there are still some remaining
>> preparation in mmc_mq_queue_rq(), like mmc part switch. For software
>> queue, we can prepare a request totally after issuing one.
>
> I suppose to make the submission non-blocking, all operations that
> currently block in the submission path may have to be changed first.
>
> For the case of a partition switch (same for retune), I suppose
> something like this can be done:
>
> - in queue_rq() check whether a partition switch is needed. If not,
> submit the current rq
> - if a partition switch is needed, submit the partition switch cmd
> instead, and return busy status
> - when the completion arrives for the partition switch, call back into
> blk_mq to have it call queue_rq again.
>
> Or possibly even (this might not be possible without signifcant
> restructuring):
>
> - when preparing a request that would require a partition switch,
> insert another meta-request to switch the partition ahead of it.
>
> I do realize that this is a significant departure from how it was done
> in the past, but it seems cleaner that way to me.
>
I would be treating the partition issue separate from the queued/batched
submission.
Aligning with the 'traditional' linux way for partition handling is
definitely the way to go IMO; otherwise you'll end up having to worry
about resource allocation between distinct queues (like you have to do
now), and will be having a hard time trying to map it properly to the
underlying hardware abstraction in blk-mq.
For starters I would keep a partition marker in the driver instance, and
calculate the parition for each incoming request. If the partition is
different you'll have to insert a partition switch request before
submitting the actual one.
To do this efficiently it would be good to know if:
a) How invasive is the partition switch? Would it be feasible to eg add
a partition switch for every command? This might sound daft now, but if
we get request batching going it might not the _that_ expensive after all...
b) Can the request switch command batched together with the normal
command? IE is is possible to have them both send in one go?
If so it would make life _so_ much easier; we could submit both command
at the same time, and won't have to worry about handling internal
completions ...
>> 2) I wonder if it is appropriate that using the irq threaded context
>> to dispatch next request, actually we will still introduce a context
>> switch here. Now we will complete a request in the hard irq handler
>> and kick the softirq to do time-consuming operations, like DMA
>> unmapping , and will start next request in the hard irq handler
>> without context switch. Moreover if we remove the BLK_MQ_F_BLOCKING in
>> future like you suggested, then we can remove all context switch. And
>> I think we can dispatch next request in the softirq context (actually
>> the CQE already did).
>
> I hope Hannes (or someone else) can comment here, as I don't
> know exactly what his objection to kicking off the next cmd in the
> hardirq was.
>
The point being that you'll have to have a context switch anyway
(between hardirq and softirq), and you'll be able to better handle
recovery as the hardirq handler is pretty generic and the won't be any
chance of that one becoming stuck.
And, of course, modern software design :-)
> I think generally, deferring all slow operations to an irqthread
> rather than a softirq is a good idea, but I share your concern that
> this can introduce an unnecessary latency between the the
> the IRQ is signaled and the time the following cmd is sent to the
> hardware.
> > Doing everything in a single (irqthread) context is clearly simpler,
> so this would need to be measured carefully to avoid unnecessary
> complexity, but I think don't see anything stopping us from having
> the fast-path where the low-level driver first checks for any possible
> error conditions in hardirq context and the fires off a prepared cmd
> right away whenever it can before triggering the irqthread that does
> everything else. I think this has to be a per-driver optimization, so
> the common case would just have an irqthread.
>
>> 3) For packed request support, I did not see an example that block
>> driver can dispatch a request from the IO scheduler in queue_rq() and
>> no APIs supported from blk-mq. And we do not know where can dispatch a
>> request in queue_rq(), from IO scheduler? from ctx? or from
>> hctx->dispatch list? and if this request can not be passed to host
>> now, how to do it? Seems lots of complicated things.
>
> The only way I can see is the ->last flag, so if blk_mq submits multiple
> requests in a row to queue_rq() with this flag cleared and calls
> ->commit_rqs() after the last one. This seems to be what the scsi
> disk driver and the nvme driver rely on, and we should be able to use
> it the same way for packed cmds, by checking each time in queue_rq()
> whether requests can/should be combined and reporting busy otherwise
> (after preparing a combined mmc cmd).
> blk_mq will then call commit_rqs, which should do the actual submission
> to the hardware driver.
>
The ->last flag really depends on the submission thread, eg if something
in the upper layers is using on-stack plugging.
In my experience this is done only in some specific use-cases resp.
filesystems, so this is not something you can rely on to make any decisions.
> Now as you point out, the *big* problem with this is that we never
> get multiple requests together in practice, i.e. the last flag is almost
> always set, and any optimization around it has no effect.
>
See above. I don't think the using the ->last flag is the way to go here.
What you really need to do here is to inject some 'pushback' into the
block layer so that is has a _chance_ of assembling more requests.
But the actual design really needs to take hardware features into account.
As mentioned above, initially I would concentrate on getting the
partitioning working with a single request queue; once that is done we
can look at request batching proper.
And for that you could have a look at the S/390 DASD driver (right,
Arndt?), which has a very similar concept.
Cheers,
Hannes
--
Dr. Hannes Reinecke Teamlead Storage & Networking
hare@suse.de +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), GF: Felix Imendörffer
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v6 0/4] Add MMC software queue support
2019-11-26 11:17 ` Hannes Reinecke
@ 2019-11-27 7:49 ` Baolin Wang
2019-11-27 9:00 ` Christoph Hellwig
1 sibling, 0 replies; 17+ messages in thread
From: Baolin Wang @ 2019-11-27 7:49 UTC (permalink / raw)
To: Hannes Reinecke
Cc: Arnd Bergmann, (Exiting) Baolin Wang, Adrian Hunter, Ulf Hansson,
asutoshd, Orson Zhai, Lyra Zhang, Linus Walleij, Vincent Guittot,
linux-mmc, linux-kernel, Hannes Reinecke, linux-block,
Paolo Valente
On Tue, Nov 26, 2019 at 7:17 PM Hannes Reinecke <hare@suse.de> wrote:
>
> On 11/22/19 10:50 AM, Arnd Bergmann wrote:
> > (adding Paolo as well, maybe he has some more insights)
> >
> > On Mon, Nov 18, 2019 at 11:04 AM (Exiting) Baolin Wang
> > <baolin.wang@linaro.org> wrote:
> >> On Tue, 12 Nov 2019 at 16:48, Baolin Wang <baolin.wang7@gmail.com> wrote:
> >>> On Tue, Nov 12, 2019 at 12:59 AM Arnd Bergmann <arnd@arndb.de> wrote:
> >>>> On Mon, Nov 11, 2019 at 1:58 PM Baolin Wang <baolin.wang@linaro.org> wrote:
> >>>>>> - With that change in place calling a blocking __mmc_claim_host() is
> >>>>>> still a problem, so there should still be a nonblocking mmc_try_claim_host()
> >>>>>> for the submission path, leading to a BLK_STS_DEV_RESOURCE (?)
> >>>>>> return code from mmc_mq_queue_rq(). Basically mmc_mq_queue_rq()
> >>>>>> should always return right away, either after having queued the next I/O
> >>>>>> or with an error, but not waiting for the device in any way.
> >>>>>
> >>>>> Actually not only the mmc_claim_host() will block the MMC request
> >>>>> processing, in this routine, the mmc_blk_part_switch() and
> >>>>> mmc_retune() can also block the request processing. Moreover the part
> >>>>> switching and tuning should be sync operations, and we can not move
> >>>>> them to a work or a thread.
> >>>>
> >>>> Ok, I see.
> >>>>
> >>>> Those would also cause requests to be sent to the device or the host
> >>>> controller, right? Maybe we can treat them as "a non-IO request
> >>>
> >>> Right.
> >>>
> >>>> has successfully been queued to the device" events, returning
> >>>> busy from the mmc_mq_queue_rq() function and then running
> >>>> the queue again when they complete?
> >>>
> >>> Yes, seems reasonable to me.
> >>>
> >>>>
> >>>>>> - For the packed requests, there is apparently a very simple way to implement
> >>>>>> that without a software queue: mmc_mq_queue_rq() is allowed to look at
> >>>>>> and dequeue all requests that are currently part of the request_queue,
> >>>>>> so it should take out as many as it wants to submit at once and send
> >>>>>> them all down to the driver together, avoiding the need for any further
> >>>>>> round-trips to blk_mq or maintaining a queue in mmc.
> >>>>>
> >>>>> You mean we can dispatch a request directly from
> >>>>> elevator->type->ops.dispatch_request()? but we still need some helper
> >>>>> functions to check if these requests can be packed (the package
> >>>>> condition), and need to invent new APIs to start a packed request (or
> >>>>> using cqe interfaces, which means we still need to implement some cqe
> >>>>> callbacks).
> >>>>
> >>>> I don't know how the dispatch_request() function fits in there,
> >>>> what Hannes told me is that in ->queue_rq() you can always
> >>>> look at the following requests that are already queued up
> >>>> and take the next ones off the list. Looking at bd->last
> >>>> tells you if there are additional requests. If there are, you can
> >>>> look at the next one from blk_mq_hw_ctx (not sure how, but
> >>>> should not be hard to find)
> >>>>
> >>>> I also see that there is a commit_rqs() callback that may
> >>>> go along with queue_rq(), implementing that one could make
> >>>> this easier as well.
> >>>
> >>> Yes, we can use queue_rq()/commit_rqs() and bd->last (now bd->last may
> >>> can not work well, see [1]), but like we talked before, for packed
> >>> request, we still need some new interfaces (for example, a interface
> >>> used to start a packed request, and a interface used to complete a
> >>> packed request), but at last we got a consensus that we should re-use
> >>> the CQE interfaces instead of new invention.
> >>>
> >>> [1] https://lore.kernel.org/patchwork/patch/1102897/
> >>>
> >>>>
> >>>>>> - The DMA management (bounce buffer, map, unmap) that is currently
> >>>>>> done in mmc_blk_mq_issue_rq() should ideally be done in the
> >>>>>> init_request()/exit_request() (?) callbacks from mmc_mq_ops so this
> >>>>>> can be done asynchronously, out of the critical timing path for the
> >>>>>> submission. With this, there won't be any need for a software queue.
> >>>>>
> >>>>> This is not true, now the blk-mq will allocate some static request
> >>>>> objects (usually the static requests number should be the same with
> >>>>> the hardware queue depth) saved in struct blk_mq_tags. So the
> >>>>> init_request() is used to initialize the static requests when
> >>>>> allocating them, and call exit_request to free the static requests
> >>>>> when freeing the 'struct blk_mq_tags', such as the queue is dead. So
> >>>>> we can not move the DMA management into the init_request/exit_request.
> >>>>
> >>>> Ok, I must have misremembered which callback that is then, but I guess
> >>>> there is some other place to do it.
> >>>
> >>> I checked the 'struct blk_mq_ops', and I did not find a ops can be
> >>> used to do DMA management. And I also checked UFS driver, it also did
> >>> the DMA mapping in the queue_rq() (scsi_queue_rq() --->
> >>> ufshcd_queuecommand() ---> ufshcd_map_sg()). Maybe I missed something?
> >>>
> >>> Moreover like I said above, for the packed request, we still need
> >>> implement something (like the software queue) based on the CQE
> >>> interfaces to help to handle packed requests.
> >>
> >> After some investigation and offline discussion with you, I still have
> >> some concerns about your suggestion.
> >>
> >> 1) Now blk-mq have not supplied some ops to prepare a request, which is
> >> used to do some DMA management asynchronously. But yes, we can
> >> introduce new ops for blk-mq. But there are still some remaining
> >> preparation in mmc_mq_queue_rq(), like mmc part switch. For software
> >> queue, we can prepare a request totally after issuing one.
> >
> > I suppose to make the submission non-blocking, all operations that
> > currently block in the submission path may have to be changed first.
> >
> > For the case of a partition switch (same for retune), I suppose
> > something like this can be done:
> >
> > - in queue_rq() check whether a partition switch is needed. If not,
> > submit the current rq
> > - if a partition switch is needed, submit the partition switch cmd
> > instead, and return busy status
> > - when the completion arrives for the partition switch, call back into
> > blk_mq to have it call queue_rq again.
> >
> > Or possibly even (this might not be possible without signifcant
> > restructuring):
> >
> > - when preparing a request that would require a partition switch,
> > insert another meta-request to switch the partition ahead of it.
> >
> > I do realize that this is a significant departure from how it was done
> > in the past, but it seems cleaner that way to me.
> >
> I would be treating the partition issue separate from the queued/batched
> submission.
>
> Aligning with the 'traditional' linux way for partition handling is
> definitely the way to go IMO; otherwise you'll end up having to worry
> about resource allocation between distinct queues (like you have to do
> now), and will be having a hard time trying to map it properly to the
> underlying hardware abstraction in blk-mq.
>
> For starters I would keep a partition marker in the driver instance, and
> calculate the parition for each incoming request. If the partition is
> different you'll have to insert a partition switch request before
> submitting the actual one.
>
> To do this efficiently it would be good to know if:
> a) How invasive is the partition switch? Would it be feasible to eg add
> a partition switch for every command? This might sound daft now, but if
> we get request batching going it might not the _that_ expensive after all...
This is expensive I think, now not all SD host controllers or cards
can handle batching request, only for those which can support ADMA3 or
packed command.
> b) Can the request switch command batched together with the normal
> command? IE is is possible to have them both send in one go?
I do not think that the switch command can be batched together with
the normal command. We must wait for the completion of the switch
command before sending the normal command. It should be the SYNC
command in MMC stack.
I think the first method suggested by Arnd can work.
> If so it would make life _so_ much easier; we could submit both command
> at the same time, and won't have to worry about handling internal
> completions ...
>
> >> 2) I wonder if it is appropriate that using the irq threaded context
> >> to dispatch next request, actually we will still introduce a context
> >> switch here. Now we will complete a request in the hard irq handler
> >> and kick the softirq to do time-consuming operations, like DMA
> >> unmapping , and will start next request in the hard irq handler
> >> without context switch. Moreover if we remove the BLK_MQ_F_BLOCKING in
> >> future like you suggested, then we can remove all context switch. And
> >> I think we can dispatch next request in the softirq context (actually
> >> the CQE already did).
> >
> > I hope Hannes (or someone else) can comment here, as I don't
> > know exactly what his objection to kicking off the next cmd in the
> > hardirq was.
> >
> The point being that you'll have to have a context switch anyway
> (between hardirq and softirq), and you'll be able to better handle
> recovery as the hardirq handler is pretty generic and the won't be any
> chance of that one becoming stuck.
> And, of course, modern software design :-)
>
> > I think generally, deferring all slow operations to an irqthread
> > rather than a softirq is a good idea, but I share your concern that
> > this can introduce an unnecessary latency between the the
> > the IRQ is signaled and the time the following cmd is sent to the
> > hardware.
> > > Doing everything in a single (irqthread) context is clearly simpler,
> > so this would need to be measured carefully to avoid unnecessary
> > complexity, but I think don't see anything stopping us from having
> > the fast-path where the low-level driver first checks for any possible
> > error conditions in hardirq context and the fires off a prepared cmd
> > right away whenever it can before triggering the irqthread that does
> > everything else. I think this has to be a per-driver optimization, so
> > the common case would just have an irqthread.
> >
> >> 3) For packed request support, I did not see an example that block
> >> driver can dispatch a request from the IO scheduler in queue_rq() and
> >> no APIs supported from blk-mq. And we do not know where can dispatch a
> >> request in queue_rq(), from IO scheduler? from ctx? or from
> >> hctx->dispatch list? and if this request can not be passed to host
> >> now, how to do it? Seems lots of complicated things.
> >
> > The only way I can see is the ->last flag, so if blk_mq submits multiple
> > requests in a row to queue_rq() with this flag cleared and calls
> > ->commit_rqs() after the last one. This seems to be what the scsi
> > disk driver and the nvme driver rely on, and we should be able to use
> > it the same way for packed cmds, by checking each time in queue_rq()
> > whether requests can/should be combined and reporting busy otherwise
> > (after preparing a combined mmc cmd).
> > blk_mq will then call commit_rqs, which should do the actual submission
> > to the hardware driver.
> >
> The ->last flag really depends on the submission thread, eg if something
> in the upper layers is using on-stack plugging.
> In my experience this is done only in some specific use-cases resp.
> filesystems, so this is not something you can rely on to make any decisions.
The on-stack plugging is enabled in my case, and from the commit
message of introducing this structure, the ->last flag is used to
indicate the last request in the chain, but seems work abnormally.
"Since we have the notion of a 'last' request in a chain, we can use
this to have the hardware optimize the issuing of requests. Add
a list_head parameter to queue_rq that the driver can use to
temporarily store hw commands for issue when 'last' is true. If we
are doing a chain of requests, pass in a NULL list for the first
request to force issue of that immediately, then batch the remainder
for deferred issue until the last request has been sent."
>
> > Now as you point out, the *big* problem with this is that we never
> > get multiple requests together in practice, i.e. the last flag is almost
> > always set, and any optimization around it has no effect.
> >
> See above. I don't think the using the ->last flag is the way to go here.
> What you really need to do here is to inject some 'pushback' into the
> block layer so that is has a _chance_ of assembling more requests.
How about Arnd's suggestion? looks reasonable to me that we can have a
new API to handle batching request.
https://paste.ubuntu.com/p/MfSRwKqFCs/
>
> But the actual design really needs to take hardware features into account.
>
> As mentioned above, initially I would concentrate on getting the
> partitioning working with a single request queue; once that is done we
> can look at request batching proper.
OK.
>
> And for that you could have a look at the S/390 DASD driver (right,
> Arndt?), which has a very similar concept.
Yes, I've looked at the dasd.c driver, and it used a list to link
requests from blk-mq and handled them with a tasklet, but if this will
cause a long latency if we linked more requests into the list and
dispatched them to the controller slowly?
Thanks for your input.
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [PATCH v6 0/4] Add MMC software queue support
2019-11-26 11:17 ` Hannes Reinecke
2019-11-27 7:49 ` Baolin Wang
@ 2019-11-27 9:00 ` Christoph Hellwig
2019-11-27 12:01 ` Arnd Bergmann
2019-11-28 12:15 ` Martin K. Petersen
1 sibling, 2 replies; 17+ messages in thread
From: Christoph Hellwig @ 2019-11-27 9:00 UTC (permalink / raw)
To: Hannes Reinecke
Cc: Arnd Bergmann, (Exiting) Baolin Wang, Baolin Wang, Adrian Hunter,
Ulf Hansson, asutoshd, Orson Zhai, Lyra Zhang, Linus Walleij,
Vincent Guittot, linux-mmc, linux-kernel, Hannes Reinecke,
linux-block, Paolo Valente
On Tue, Nov 26, 2019 at 12:17:15PM +0100, Hannes Reinecke wrote:
> Aligning with the 'traditional' linux way for partition handling is
> definitely the way to go IMO; otherwise you'll end up having to worry
> about resource allocation between distinct queues (like you have to do
> now), and will be having a hard time trying to map it properly to the
> underlying hardware abstraction in blk-mq.
Sorry, but this is complete bullshit. Except for the very unfortunate
name MMC partitions have nothing to do with partitions. They are a
concept roughly equivalent to SCSI logical units and nvme namespace,
just with a pretty idiotic design decision that only allows I/O to one
of them at a time. The block layer way to deal with them is to use
a shared tagset for multiple request queues, which doesn't use up a
whole lot of resources. The only hard part is the draining when
switching between partitions, and there is no really nice way to
deal with that. If requests are batched enough we could just drain
and switch every time an other partition access comes in. Especially
so if people only use partitions for boot partitions and other rarely
used areas. If that doesn't work out we'll just have to reject other
partition access and then use a timer and/or counter to eventually
switch and provide basic fairness.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v6 0/4] Add MMC software queue support
2019-11-27 9:00 ` Christoph Hellwig
@ 2019-11-27 12:01 ` Arnd Bergmann
2019-12-10 15:17 ` Ulf Hansson
2019-11-28 12:15 ` Martin K. Petersen
1 sibling, 1 reply; 17+ messages in thread
From: Arnd Bergmann @ 2019-11-27 12:01 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Hannes Reinecke, (Exiting) Baolin Wang, Baolin Wang,
Adrian Hunter, Ulf Hansson, Asutosh Das, Orson Zhai, Lyra Zhang,
Linus Walleij, Vincent Guittot, linux-mmc, linux-kernel,
Hannes Reinecke, linux-block, Paolo Valente
On Wed, Nov 27, 2019 at 10:00 AM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Tue, Nov 26, 2019 at 12:17:15PM +0100, Hannes Reinecke wrote:
> If requests are batched enough we could just drain
> and switch every time an other partition access comes in. Especially
> so if people only use partitions for boot partitions and other rarely
> used areas.
We only support a single user partition plus up to two boot partitions that
are accessed rarely, I don't think there is any reason to optimize switching
between them.
The only change that I think we need here is to change the partition switch
from something that is done synchronously during ->queue_rq() to
something that fits better into normal scheme of sending a cmd to
the device, returning BLK_STS_RESOURCE from ->queue_rq.
Possibly this could even be turned into a standard struct request that is
added between two normal requests for different partitions at some
point, if this simplifies the logic (I suspect it won't, but it may be worth
a try).
Arnd
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [PATCH v6 0/4] Add MMC software queue support
2019-11-27 12:01 ` Arnd Bergmann
@ 2019-12-10 15:17 ` Ulf Hansson
0 siblings, 0 replies; 17+ messages in thread
From: Ulf Hansson @ 2019-12-10 15:17 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Christoph Hellwig, Hannes Reinecke, (Exiting) Baolin Wang,
Baolin Wang, Adrian Hunter, Asutosh Das, Orson Zhai, Lyra Zhang,
Linus Walleij, Vincent Guittot, linux-mmc, linux-kernel,
Hannes Reinecke, linux-block, Paolo Valente
On Wed, 27 Nov 2019 at 13:01, Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Wed, Nov 27, 2019 at 10:00 AM Christoph Hellwig <hch@infradead.org> wrote:
> >
> > On Tue, Nov 26, 2019 at 12:17:15PM +0100, Hannes Reinecke wrote:
> > If requests are batched enough we could just drain
> > and switch every time an other partition access comes in. Especially
> > so if people only use partitions for boot partitions and other rarely
> > used areas.
>
> We only support a single user partition plus up to two boot partitions that
> are accessed rarely, I don't think there is any reason to optimize switching
> between them.
I agree. However, let me just add some more information to this.
There are more partitions, like the RPMB for example. In regards to
partition switching, after serving a request to the RPMB partition, we
always switch back to the main user area. I think that is sufficient.
Also note that requests for the RPMB partitions are managed via
REQ_OP_DRV_IN|OUT.
>
> The only change that I think we need here is to change the partition switch
> from something that is done synchronously during ->queue_rq() to
> something that fits better into normal scheme of sending a cmd to
> the device, returning BLK_STS_RESOURCE from ->queue_rq.
You want to translate them to be managed similar to REQ_OP_DRV_IN|OUT, no?
I am just trying to understand what this would help us with, but I
don't get it, sorry.
I realize that I am joining the show a bit late, apologize for that.
But it seems like you are forgetting about re-tuning, urgent bkops,
card detect, SDIO combo cards, etc.
For example, re-tuning may be required because of a CRC error on the
previously sent transfer. Thus re-tuning must be done before serving
the next request.
Likewise, when the device signals urgent bkops status, we must not
serve any new request until the card has notified us that it is ready
with it's internal housekeeping operations.
> Possibly this could even be turned into a standard struct request that is
> added between two normal requests for different partitions at some
> point, if this simplifies the logic (I suspect it won't, but it may be worth
> a try).
Doing so, means re-tuning, bkops, etc, also needs to be managed in the
same way. Is this really the way to go?
Kind regards
Uffe
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v6 0/4] Add MMC software queue support
2019-11-27 9:00 ` Christoph Hellwig
2019-11-27 12:01 ` Arnd Bergmann
@ 2019-11-28 12:15 ` Martin K. Petersen
2019-11-28 15:54 ` Christoph Hellwig
1 sibling, 1 reply; 17+ messages in thread
From: Martin K. Petersen @ 2019-11-28 12:15 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Hannes Reinecke, Arnd Bergmann, (Exiting) Baolin Wang,
Baolin Wang, Adrian Hunter, Ulf Hansson, asutoshd, Orson Zhai,
Lyra Zhang, Linus Walleij, Vincent Guittot, linux-mmc,
linux-kernel, Hannes Reinecke, linux-block, Paolo Valente
Christoph,
> equivalent to SCSI logical units and nvme namespace, just with a
> pretty idiotic design decision that only allows I/O to one of them at
> a time. The block layer way to deal with them is to use a shared
> tagset for multiple request queues, which doesn't use up a whole lot
> of resources. The only hard part is the draining when switching
> between partitions, and there is no really nice way to deal with that.
> If requests are batched enough we could just drain and switch every
> time an other partition access comes in.
This mirrors single_lun in SCSI closely. I was hoping we could
eventually get rid of that travesty but if MMC needs something similar,
maybe it would be good to move that plumbing to block?
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v6 0/4] Add MMC software queue support
2019-11-28 12:15 ` Martin K. Petersen
@ 2019-11-28 15:54 ` Christoph Hellwig
0 siblings, 0 replies; 17+ messages in thread
From: Christoph Hellwig @ 2019-11-28 15:54 UTC (permalink / raw)
To: Martin K. Petersen
Cc: Christoph Hellwig, Hannes Reinecke, Arnd Bergmann,
(Exiting) Baolin Wang, Baolin Wang, Adrian Hunter, Ulf Hansson,
asutoshd, Orson Zhai, Lyra Zhang, Linus Walleij, Vincent Guittot,
linux-mmc, linux-kernel, Hannes Reinecke, linux-block,
Paolo Valente
On Thu, Nov 28, 2019 at 07:15:09AM -0500, Martin K. Petersen wrote:
> This mirrors single_lun in SCSI closely. I was hoping we could
> eventually get rid of that travesty but if MMC needs something similar,
> maybe it would be good to move that plumbing to block?
Oh, I totally forgot about single_lun. Given that it is only set
for ancient CD changers I'm not even sure that code works properly
anymore. Having common code that is exercised regularly by mmc would
certainly be better than the current SCSI code, but I'm not sure how
well this is going to work out.
^ permalink raw reply [flat|nested] 17+ messages in thread