All of lore.kernel.org
 help / color / mirror / Atom feed
From: Israel Rukshin <israelr@mellanox.com>
To: Linux-nvme <linux-nvme@lists.infradead.org>,
	Sagi Grimberg <sagi@grimberg.me>, Christoph Hellwig <hch@lst.de>,
	James Smart <jsmart2021@gmail.com>,
	Keith Busch <kbusch@kernel.org>
Cc: Israel Rukshin <israelr@mellanox.com>, Max Gurtovoy <maxg@mellanox.com>
Subject: [PATCH 0/3] nvme: Avoid preallocating big SGL for data
Date: Sun, 24 Nov 2019 18:38:29 +0200	[thread overview]
Message-ID: <1574613512-5943-1-git-send-email-israelr@mellanox.com> (raw)

When nvme allocates a tagset it preallocates a big buffer for the IO SGL
based on SG_CHUNK_SIZE.

Modern DMA engines are often capable of dealing with very big segments so
the SG_CHUNK_SIZE is often too big. SG_CHUNK_SIZE results in a static 4KB
SGL allocation per command.

If a controller has lots of deep queues, preallocation for the sg list can
consume substantial amounts of memory. Switch to runtime allocation for SGL
for lists longer than 2 entries. This is the approach used by NVMe PCI so
it should be reasonable for NVMeOF as well. Runtime SGL allocation has
always been the case for the legacy I/O path so this is nothing new.

The preallocated small SGL depends on SG_CHAIN so if the ARCH doesn't
support SG_CHAIN, use only runtime allocation for the SGL.

Israel Rukshin (3):
  nvme-rdma: Avoid preallocating big SGL for data
  nvme-fc: Avoid preallocating big SGL for data
  nvmet-loop: Avoid preallocating big SGL for data

 drivers/nvme/host/fc.c     |  8 ++++----
 drivers/nvme/host/nvme.h   |  6 ++++++
 drivers/nvme/host/rdma.c   | 10 +++++-----
 drivers/nvme/target/loop.c |  8 ++++----
 4 files changed, 19 insertions(+), 13 deletions(-)

-- 
1.8.3.1


_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

             reply	other threads:[~2019-11-24 16:39 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-24 16:38 Israel Rukshin [this message]
2019-11-24 16:38 ` [PATCH 1/3] nvme-rdma: Avoid preallocating big SGL for data Israel Rukshin
2019-11-26 16:53   ` Christoph Hellwig
2019-11-24 16:38 ` [PATCH 2/3] nvme-fc: " Israel Rukshin
2019-11-25 17:04   ` James Smart
2019-11-24 16:38 ` [PATCH 3/3] nvmet-loop: " Israel Rukshin
2019-11-25  2:24   ` Chaitanya Kulkarni
2019-11-26 16:53   ` Christoph Hellwig
2019-11-26 17:40   ` Keith Busch

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1574613512-5943-1-git-send-email-israelr@mellanox.com \
    --to=israelr@mellanox.com \
    --cc=hch@lst.de \
    --cc=jsmart2021@gmail.com \
    --cc=kbusch@kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=maxg@mellanox.com \
    --cc=sagi@grimberg.me \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.