* [PATCH] nvme: set dma alignment to qword
@ 2020-05-21 2:22 Keith Busch
2020-05-21 6:05 ` Sagi Grimberg
2020-05-26 14:53 ` Christoph Hellwig
0 siblings, 2 replies; 8+ messages in thread
From: Keith Busch @ 2020-05-21 2:22 UTC (permalink / raw)
To: linux-nvme, hch, sagi; +Cc: Keith Busch
The default dma alignment mask is 511, which is much larger than any nvme
controller requires. NVMe controllers accept qword aligned DMA addresses,
so set the request_queue constraints to that. This can help avoid bounce
buffers on user passthrough commands.
Signed-off-by: Keith Busch <kbusch@kernel.org>
---
drivers/nvme/host/core.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 805d289e6cd9..ba860efd250d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
}
blk_queue_virt_boundary(q, ctrl->page_size - 1);
+ blk_queue_dma_alignment(q, 7);
if (ctrl->vwc & NVME_CTRL_VWC_PRESENT)
vwc = true;
blk_queue_write_cache(q, vwc, vwc);
--
2.24.1
_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH] nvme: set dma alignment to qword
2020-05-21 2:22 [PATCH] nvme: set dma alignment to qword Keith Busch
@ 2020-05-21 6:05 ` Sagi Grimberg
2020-05-21 15:28 ` Keith Busch
2020-05-26 14:53 ` Christoph Hellwig
1 sibling, 1 reply; 8+ messages in thread
From: Sagi Grimberg @ 2020-05-21 6:05 UTC (permalink / raw)
To: Keith Busch, linux-nvme, hch
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 805d289e6cd9..ba860efd250d 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
> blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
> }
> blk_queue_virt_boundary(q, ctrl->page_size - 1);
> + blk_queue_dma_alignment(q, 7);
Shouldn't be an issue for rdma/tcp, think that it should also be OK for
FC as well but not sure.
_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] nvme: set dma alignment to qword
2020-05-21 6:05 ` Sagi Grimberg
@ 2020-05-21 15:28 ` Keith Busch
2020-05-21 15:29 ` Christoph Hellwig
0 siblings, 1 reply; 8+ messages in thread
From: Keith Busch @ 2020-05-21 15:28 UTC (permalink / raw)
To: Sagi Grimberg; +Cc: hch, linux-nvme
On Wed, May 20, 2020 at 11:05:51PM -0700, Sagi Grimberg wrote:
>
> > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> > index 805d289e6cd9..ba860efd250d 100644
> > --- a/drivers/nvme/host/core.c
> > +++ b/drivers/nvme/host/core.c
> > @@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
> > blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
> > }
> > blk_queue_virt_boundary(q, ctrl->page_size - 1);
> > + blk_queue_dma_alignment(q, 7);
>
> Shouldn't be an issue for rdma/tcp, think that it should also be OK for
> FC as well but not sure.
Okay, just to be safe, I'll make this a nvme_ctrl_ops callback.
_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] nvme: set dma alignment to qword
2020-05-21 15:28 ` Keith Busch
@ 2020-05-21 15:29 ` Christoph Hellwig
2020-05-21 15:41 ` Keith Busch
0 siblings, 1 reply; 8+ messages in thread
From: Christoph Hellwig @ 2020-05-21 15:29 UTC (permalink / raw)
To: Keith Busch; +Cc: Sagi Grimberg, linux-nvme, hch
On Thu, May 21, 2020 at 08:28:20AM -0700, Keith Busch wrote:
> On Wed, May 20, 2020 at 11:05:51PM -0700, Sagi Grimberg wrote:
> >
> > > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> > > index 805d289e6cd9..ba860efd250d 100644
> > > --- a/drivers/nvme/host/core.c
> > > +++ b/drivers/nvme/host/core.c
> > > @@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
> > > blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
> > > }
> > > blk_queue_virt_boundary(q, ctrl->page_size - 1);
> > > + blk_queue_dma_alignment(q, 7);
> >
> > Shouldn't be an issue for rdma/tcp, think that it should also be OK for
> > FC as well but not sure.
>
> Okay, just to be safe, I'll make this a nvme_ctrl_ops callback.
No callbacks please, especially without users. If FC needs an override
we should just add a field to struct nvme_ctrl.
_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] nvme: set dma alignment to qword
2020-05-21 15:29 ` Christoph Hellwig
@ 2020-05-21 15:41 ` Keith Busch
2020-05-21 15:42 ` Christoph Hellwig
0 siblings, 1 reply; 8+ messages in thread
From: Keith Busch @ 2020-05-21 15:41 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Sagi Grimberg, linux-nvme
On Thu, May 21, 2020 at 05:29:29PM +0200, Christoph Hellwig wrote:
> On Thu, May 21, 2020 at 08:28:20AM -0700, Keith Busch wrote:
> > On Wed, May 20, 2020 at 11:05:51PM -0700, Sagi Grimberg wrote:
> > >
> > > > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> > > > index 805d289e6cd9..ba860efd250d 100644
> > > > --- a/drivers/nvme/host/core.c
> > > > +++ b/drivers/nvme/host/core.c
> > > > @@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
> > > > blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
> > > > }
> > > > blk_queue_virt_boundary(q, ctrl->page_size - 1);
> > > > + blk_queue_dma_alignment(q, 7);
> > >
> > > Shouldn't be an issue for rdma/tcp, think that it should also be OK for
> > > FC as well but not sure.
> >
> > Okay, just to be safe, I'll make this a nvme_ctrl_ops callback.
>
> No callbacks please, especially without users. If FC needs an override
> we should just add a field to struct nvme_ctrl.
Well, pci would be a user. But if you're okay with having the driver
default to this alignment, would you consider taking this original patch,
or do you want confirmation from FC developers on whether or not this
is safe?
_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] nvme: set dma alignment to qword
2020-05-21 15:41 ` Keith Busch
@ 2020-05-21 15:42 ` Christoph Hellwig
2020-05-21 19:46 ` James Smart
0 siblings, 1 reply; 8+ messages in thread
From: Christoph Hellwig @ 2020-05-21 15:42 UTC (permalink / raw)
To: Keith Busch; +Cc: Christoph Hellwig, linux-nvme, Sagi Grimberg
On Thu, May 21, 2020 at 08:41:51AM -0700, Keith Busch wrote:
> On Thu, May 21, 2020 at 05:29:29PM +0200, Christoph Hellwig wrote:
> > On Thu, May 21, 2020 at 08:28:20AM -0700, Keith Busch wrote:
> > > On Wed, May 20, 2020 at 11:05:51PM -0700, Sagi Grimberg wrote:
> > > >
> > > > > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> > > > > index 805d289e6cd9..ba860efd250d 100644
> > > > > --- a/drivers/nvme/host/core.c
> > > > > +++ b/drivers/nvme/host/core.c
> > > > > @@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
> > > > > blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
> > > > > }
> > > > > blk_queue_virt_boundary(q, ctrl->page_size - 1);
> > > > > + blk_queue_dma_alignment(q, 7);
> > > >
> > > > Shouldn't be an issue for rdma/tcp, think that it should also be OK for
> > > > FC as well but not sure.
> > >
> > > Okay, just to be safe, I'll make this a nvme_ctrl_ops callback.
> >
> > No callbacks please, especially without users. If FC needs an override
> > we should just add a field to struct nvme_ctrl.
>
> Well, pci would be a user. But if you're okay with having the driver
> default to this alignment, would you consider taking this original patch,
> or do you want confirmation from FC developers on whether or not this
> is safe?
Lets wait for feedback from the FC folks.
_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] nvme: set dma alignment to qword
2020-05-21 15:42 ` Christoph Hellwig
@ 2020-05-21 19:46 ` James Smart
0 siblings, 0 replies; 8+ messages in thread
From: James Smart @ 2020-05-21 19:46 UTC (permalink / raw)
To: Christoph Hellwig, Keith Busch; +Cc: Sagi Grimberg, linux-nvme
On 5/21/2020 8:42 AM, Christoph Hellwig wrote:
> On Thu, May 21, 2020 at 08:41:51AM -0700, Keith Busch wrote:
>> On Thu, May 21, 2020 at 05:29:29PM +0200, Christoph Hellwig wrote:
>>> On Thu, May 21, 2020 at 08:28:20AM -0700, Keith Busch wrote:
>>>> On Wed, May 20, 2020 at 11:05:51PM -0700, Sagi Grimberg wrote:
>>>>>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>>>>>> index 805d289e6cd9..ba860efd250d 100644
>>>>>> --- a/drivers/nvme/host/core.c
>>>>>> +++ b/drivers/nvme/host/core.c
>>>>>> @@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
>>>>>> blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
>>>>>> }
>>>>>> blk_queue_virt_boundary(q, ctrl->page_size - 1);
>>>>>> + blk_queue_dma_alignment(q, 7);
>>>>> Shouldn't be an issue for rdma/tcp, think that it should also be OK for
>>>>> FC as well but not sure.
>>>> Okay, just to be safe, I'll make this a nvme_ctrl_ops callback.
>>> No callbacks please, especially without users. If FC needs an override
>>> we should just add a field to struct nvme_ctrl.
>> Well, pci would be a user. But if you're okay with having the driver
>> default to this alignment, would you consider taking this original patch,
>> or do you want confirmation from FC developers on whether or not this
>> is safe?
> Lets wait for feedback from the FC folks.
>
> _______________________________________________
> linux-nvme mailing list
> linux-nvme@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme
It won't bother the FC devices as they are typically byte-based for
payload dma requirements.
-- james
_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] nvme: set dma alignment to qword
2020-05-21 2:22 [PATCH] nvme: set dma alignment to qword Keith Busch
2020-05-21 6:05 ` Sagi Grimberg
@ 2020-05-26 14:53 ` Christoph Hellwig
1 sibling, 0 replies; 8+ messages in thread
From: Christoph Hellwig @ 2020-05-26 14:53 UTC (permalink / raw)
To: Keith Busch; +Cc: hch, linux-nvme, sagi
Thanks,
applied to nvme-5.8.
_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2020-05-26 14:53 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-21 2:22 [PATCH] nvme: set dma alignment to qword Keith Busch
2020-05-21 6:05 ` Sagi Grimberg
2020-05-21 15:28 ` Keith Busch
2020-05-21 15:29 ` Christoph Hellwig
2020-05-21 15:41 ` Keith Busch
2020-05-21 15:42 ` Christoph Hellwig
2020-05-21 19:46 ` James Smart
2020-05-26 14:53 ` Christoph Hellwig
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).