Linux-RDMA Archive on lore.kernel.org
 help / color / Atom feed
From: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Leon Romanovsky <leon@kernel.org>, Christoph Hellwig <hch@lst.de>,
	Doug Ledford <dledford@redhat.com>,
	linux-rdma@vger.kernel.org, intel-gfx@lists.freedesktop.org,
	Roland Scheidegger <sroland@vmware.com>,
	dri-devel@lists.freedesktop.org, David Airlie <airlied@linux.ie>,
	VMware Graphics <linux-graphics-maintainer@vmware.com>,
	Maor Gottlieb <maorg@nvidia.com>,
	Maor Gottlieb <maorg@mellanox.com>
Subject: Re: [Intel-gfx] [PATCH rdma-next v3 1/2] lib/scatterlist: Add support in dynamic allocation of SG table from pages
Date: Fri, 25 Sep 2020 13:29:49 +0100
Message-ID: <c5956163-1769-ee40-e4ed-45532d8c4e19@linux.intel.com> (raw)
In-Reply-To: <20200925115833.GZ9475@nvidia.com>


On 25/09/2020 12:58, Jason Gunthorpe wrote:
> On Fri, Sep 25, 2020 at 12:41:29PM +0100, Tvrtko Ursulin wrote:
>>
>> On 25/09/2020 08:13, Leon Romanovsky wrote:
>>> On Thu, Sep 24, 2020 at 09:21:20AM +0100, Tvrtko Ursulin wrote:
>>>>
>>>> On 22/09/2020 09:39, Leon Romanovsky wrote:
>>>>> From: Maor Gottlieb <maorg@mellanox.com>
>>>>>
>>>>> Extend __sg_alloc_table_from_pages to support dynamic allocation of
>>>>> SG table from pages. It should be used by drivers that can't supply
>>>>> all the pages at one time.
>>>>>
>>>>> This function returns the last populated SGE in the table. Users should
>>>>> pass it as an argument to the function from the second call and forward.
>>>>> As before, nents will be equal to the number of populated SGEs (chunks).
>>>>
>>>> So it's appending and growing the "list", did I get that right? Sounds handy
>>>> indeed. Some comments/questions below.
>>>
>>> Yes, we (RDMA) use this function to chain contiguous pages.
>>
>> I will eveluate if i915 could start using it. We have some loops which build
>> page by page and coalesce.
> 
> Christoph H doesn't like it, but if there are enough cases we should
> really have a pin_user_pages_to_sg() rather than open code this all
> over the place.
> 
> With THP the chance of getting a coalescing SG is much higher, and
> everything is more efficient with larger SGEs.

Right, I was actually referring to i915 sites where we build sg tables 
out of shmem and plain kernel pages. In those areas we have some open 
coded coalescing loops (see for instance our shmem_get_pages). Plus a 
local "trim" to discard the unused entries, since we allocate 
pessimistically not knowing how coalescing will pan out. This kind of 
core function which appends pages could replace some of that. Maybe it 
would be slightly less efficient but I will pencil in to at least 
evaluate it.

Otherwise I do agree that coalescing is a win and in the past I have 
measured savings in a few MiB range just for struct scatterlist storage.

Regards,

Tvrtko

  reply index

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-22  8:39 [PATCH rdma-next v3 0/2] Dynamicaly allocate SG table from the pages Leon Romanovsky
2020-09-22  8:39 ` [PATCH rdma-next v3 1/2] lib/scatterlist: Add support in dynamic allocation of SG table from pages Leon Romanovsky
2020-09-23  5:42   ` Christoph Hellwig
     [not found]   ` <118a03ef-d160-e202-81cc-16c9c39359fc@linux.intel.com>
2020-09-25  7:13     ` [Intel-gfx] " Leon Romanovsky
2020-09-25 11:55       ` Jason Gunthorpe
2020-09-25 12:18         ` Maor Gottlieb
2020-09-25 12:33           ` Tvrtko Ursulin
2020-09-25 13:39             ` Maor Gottlieb
2020-09-25 13:54               ` Tvrtko Ursulin
     [not found]       ` <adff5752-582c-2065-89e2-924ef732911a@linux.intel.com>
2020-09-25 11:58         ` Jason Gunthorpe
2020-09-25 12:29           ` Tvrtko Ursulin [this message]
2020-09-25 12:34             ` Jason Gunthorpe
2020-09-25 12:13         ` Maor Gottlieb
2020-09-22  8:39 ` [PATCH rdma-next v3 2/2] RDMA/umem: Move to allocate " Leon Romanovsky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=c5956163-1769-ee40-e4ed-45532d8c4e19@linux.intel.com \
    --to=tvrtko.ursulin@linux.intel.com \
    --cc=airlied@linux.ie \
    --cc=dledford@redhat.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=hch@lst.de \
    --cc=intel-gfx@lists.freedesktop.org \
    --cc=jgg@nvidia.com \
    --cc=leon@kernel.org \
    --cc=linux-graphics-maintainer@vmware.com \
    --cc=linux-rdma@vger.kernel.org \
    --cc=maorg@mellanox.com \
    --cc=maorg@nvidia.com \
    --cc=sroland@vmware.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-RDMA Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-rdma/0 linux-rdma/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-rdma linux-rdma/ https://lore.kernel.org/linux-rdma \
		linux-rdma@vger.kernel.org
	public-inbox-index linux-rdma

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-rdma


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git