All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Dillow <dillowda-1Heg1YXhbW8@public.gmane.org>
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Cc: Roland Dreier <rdreier-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org>,
	Eli Cohen <eli-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
Subject: IB: increase DMA max_segment_size on Mellanox hardware
Date: Sun, 16 Jan 2011 21:09:44 -0500	[thread overview]
Message-ID: <1295230184.2574.26.camel@obelisk.thedillows.org> (raw)

By default, each device is assumed to be able only handle 64 KB chunks
during DMA. By giving the segment size a larger value, the block layer
will coalesce more S/G entries together for SRP, allowing larger
requests with the same sg_tablesize setting.

This ups the value on Mellanox hardware to 2 GB, though there is no HW
limit.

Signed-off-by: David Dillow <dillowda-1Heg1YXhbW8@public.gmane.org>
--
It could be 3 GB or even (4 GB - 1), since it's an unsigned int, but we
should rarely see a 2 GB request anyways.

I think it is probably safe to do something similar for the qib and
ipath drivers, and perhaps the iWarp cards for iSER.

This patch also reduces the iterations in SRP's mapping code for some
upcoming patches I have.

diff --git a/drivers/infiniband/hw/mthca/mthca_main.c b/drivers/infiniband/hw/mthca/mthca_main.c
index 5eee666..deb79d4 100644
--- a/drivers/infiniband/hw/mthca/mthca_main.c
+++ b/drivers/infiniband/hw/mthca/mthca_main.c
@@ -1043,6 +1043,9 @@ static int __mthca_init_one(struct pci_dev *pdev, int hca_type)
 		}
 	}
 
+	/* not a HW limit, but no way to say infinity */
+	dma_set_max_seg_size(&pdev->dev, 2 * 1024 * 1024 * 1024);
+
 	mdev = (struct mthca_dev *) ib_alloc_device(sizeof *mdev);
 	if (!mdev) {
 		dev_err(&pdev->dev, "Device struct alloc failed, "
diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c
index 782f11d..699852e 100644
--- a/drivers/net/mlx4/main.c
+++ b/drivers/net/mlx4/main.c
@@ -1109,6 +1109,9 @@ static int __mlx4_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
 		}
 	}
 
+	/* not a HW limit, but no way to say infinity */
+	dma_set_max_seg_size(&pdev->dev, 2 * 1024 * 1024 * 1024);
+
 	priv = kzalloc(sizeof *priv, GFP_KERNEL);
 	if (!priv) {
 		dev_err(&pdev->dev, "Device struct alloc failed, "


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

             reply	other threads:[~2011-01-17  2:09 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-01-17  2:09 David Dillow [this message]
     [not found] ` <1295230184.2574.26.camel-1q1vX8mYZiGLUyTwlgNVppKKF0rrzTr+@public.gmane.org>
2011-01-17 11:16   ` IB: increase DMA max_segment_size on Mellanox hardware Or Gerlitz
     [not found]     ` <4D342510.80701-smomgflXvOZWk0Htik3J/w@public.gmane.org>
2011-01-18  2:46       ` David Dillow
     [not found]         ` <1295318783.3051.81.camel-1q1vX8mYZiGLUyTwlgNVppKKF0rrzTr+@public.gmane.org>
2011-01-18  5:22           ` Jason Gunthorpe
2011-01-18  7:16           ` Or Gerlitz
     [not found]             ` <4D353E5E.2020500-smomgflXvOZWk0Htik3J/w@public.gmane.org>
2011-01-18 12:25               ` David Dillow
     [not found]                 ` <1295353524.3051.121.camel-1q1vX8mYZiGLUyTwlgNVppKKF0rrzTr+@public.gmane.org>
2011-01-18 18:01                   ` Jason Gunthorpe
2011-01-20 10:14                   ` Or Gerlitz
     [not found]                     ` <4D380AF2.4010905-smomgflXvOZWk0Htik3J/w@public.gmane.org>
2011-01-20 12:15                       ` David Dillow
     [not found]                         ` <1295525737.22825.24.camel-1q1vX8mYZiGLUyTwlgNVppKKF0rrzTr+@public.gmane.org>
2011-01-24 15:18                           ` Or Gerlitz
     [not found]                             ` <4D3D9845.4050803-smomgflXvOZWk0Htik3J/w@public.gmane.org>
2011-01-24 19:36                               ` David Dillow
     [not found]                                 ` <1295897819.29946.17.camel-FqX9LgGZnHWDB2HL1qBt2PIbXMQ5te18@public.gmane.org>
2011-02-17  1:06                                   ` David Dillow
     [not found]                                     ` <1297904810.8833.44.camel-FqX9LgGZnHWDB2HL1qBt2PIbXMQ5te18@public.gmane.org>
2011-03-16 19:17                                       ` Or Gerlitz
2011-01-20 10:18           ` Or Gerlitz
     [not found]             ` <4D380BDC.5080002-smomgflXvOZWk0Htik3J/w@public.gmane.org>
2011-01-20 12:25               ` David Dillow
2011-03-22  8:01   ` Or Gerlitz
     [not found]     ` <4D885748.6060600-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2011-03-22 12:51       ` David Dillow

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1295230184.2574.26.camel@obelisk.thedillows.org \
    --to=dillowda-1heg1yxhbw8@public.gmane.org \
    --cc=eli-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org \
    --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=rdreier-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.