Linux-RDMA Archive on lore.kernel.org
 help / color / Atom feed
From: Maor Gottlieb <maorg@nvidia.com>
To: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
	Leon Romanovsky <leon@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>,
	Doug Ledford <dledford@redhat.com>,
	"Jason Gunthorpe" <jgg@nvidia.com>, <linux-rdma@vger.kernel.org>,
	<intel-gfx@lists.freedesktop.org>,
	Roland Scheidegger <sroland@vmware.com>,
	<dri-devel@lists.freedesktop.org>,
	David Airlie <airlied@linux.ie>,
	"VMware Graphics" <linux-graphics-maintainer@vmware.com>,
	Maor Gottlieb <maorg@mellanox.com>
Subject: Re: [Intel-gfx] [PATCH rdma-next v3 1/2] lib/scatterlist: Add support in dynamic allocation of SG table from pages
Date: Fri, 25 Sep 2020 15:13:27 +0300
Message-ID: <a8cca9f0-58ec-654d-939e-8568d17d4c60@nvidia.com> (raw)
In-Reply-To: <adff5752-582c-2065-89e2-924ef732911a@linux.intel.com>


On 9/25/2020 2:41 PM, Tvrtko Ursulin wrote:
>
> On 25/09/2020 08:13, Leon Romanovsky wrote:
>> On Thu, Sep 24, 2020 at 09:21:20AM +0100, Tvrtko Ursulin wrote:
>>>
>>> On 22/09/2020 09:39, Leon Romanovsky wrote:
>>>> From: Maor Gottlieb <maorg@mellanox.com>
>>>>
>>>> Extend __sg_alloc_table_from_pages to support dynamic allocation of
>>>> SG table from pages. It should be used by drivers that can't supply
>>>> all the pages at one time.
>>>>
>>>> This function returns the last populated SGE in the table. Users 
>>>> should
>>>> pass it as an argument to the function from the second call and 
>>>> forward.
>>>> As before, nents will be equal to the number of populated SGEs 
>>>> (chunks).
>>>
>>> So it's appending and growing the "list", did I get that right? 
>>> Sounds handy
>>> indeed. Some comments/questions below.
>>
>> Yes, we (RDMA) use this function to chain contiguous pages.
>
> I will eveluate if i915 could start using it. We have some loops which 
> build page by page and coalesce.
>
> [snip]
>
>>>>        if (unlikely(ret))
>>>> diff --git a/tools/testing/scatterlist/main.c 
>>>> b/tools/testing/scatterlist/main.c
>>>> index 0a1464181226..4899359a31ac 100644
>>>> --- a/tools/testing/scatterlist/main.c
>>>> +++ b/tools/testing/scatterlist/main.c
>>>> @@ -55,14 +55,13 @@ int main(void)
>>>>        for (i = 0, test = tests; test->expected_segments; test++, 
>>>> i++) {
>>>>            struct page *pages[MAX_PAGES];
>>>>            struct sg_table st;
>>>> -        int ret;
>>>> +        struct scatterlist *sg;
>>>>
>>>>            set_pages(pages, test->pfn, test->num_pages);
>>>>
>>>> -        ret = __sg_alloc_table_from_pages(&st, pages, 
>>>> test->num_pages,
>>>> -                          0, test->size, test->max_seg,
>>>> -                          GFP_KERNEL);
>>>> -        assert(ret == test->alloc_ret);
>>>> +        sg = __sg_alloc_table_from_pages(&st, pages, 
>>>> test->num_pages, 0,
>>>> +                test->size, test->max_seg, NULL, 0, GFP_KERNEL);
>>>> +        assert(PTR_ERR_OR_ZERO(sg) == test->alloc_ret);
>>>
>>> Some test coverage for relatively complex code would be very 
>>> welcomed. Since
>>> the testing framework is already there, even if it bit-rotted a bit, 
>>> but
>>> shouldn't be hard to fix.
>>>
>>> A few tests to check append/grow works as expected, in terms of how 
>>> the end
>>> table looks like given the initial state and some different page 
>>> patterns
>>> added to it. And both crossing and not crossing into sg chaining 
>>> scenarios.
>>
>> This function is basic for all RDMA devices and we are pretty confident
>> that the old and new flows are tested thoroughly.
>>
>> We will add proper test in next kernel cycle.
>
> Patch seems to be adding a requirement that all callers of 
> (__)sg_alloc_table_from_pages pass in zeroed struct sg_table, which 
> wasn't the case so far.
>
> Have you audited all the callers and/or fixed them? There seems to be 
> quite a few. Gut feel says problem would probably be better solved in 
> lib/scatterlist.c and not by making all the callers memset. Should be 
> possible if you make sure you only read st->nents if prev was passed in?
>
> I've fixed the unit test and with this change the existing tests do 
> pass. But without zeroing it does fail on the very first, single page, 
> test scenario.
>
> You can pull the unit test hacks from 
> git://people.freedesktop.org/~tursulin/drm-intel sgtest.
>
> Regards,
>
> Tvrtko

Thanks for finding this issue.  In the regular flow, 
__sg_alloc_table_from_pages memset the sg_table struct, but currently 
the code access this struct before. Will be fixed internally in scatterlist.


  parent reply index

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-22  8:39 [PATCH rdma-next v3 0/2] Dynamicaly allocate SG table from the pages Leon Romanovsky
2020-09-22  8:39 ` [PATCH rdma-next v3 1/2] lib/scatterlist: Add support in dynamic allocation of SG table from pages Leon Romanovsky
2020-09-23  5:42   ` Christoph Hellwig
     [not found]   ` <118a03ef-d160-e202-81cc-16c9c39359fc@linux.intel.com>
2020-09-25  7:13     ` [Intel-gfx] " Leon Romanovsky
2020-09-25 11:55       ` Jason Gunthorpe
2020-09-25 12:18         ` Maor Gottlieb
2020-09-25 12:33           ` Tvrtko Ursulin
2020-09-25 13:39             ` Maor Gottlieb
2020-09-25 13:54               ` Tvrtko Ursulin
     [not found]       ` <adff5752-582c-2065-89e2-924ef732911a@linux.intel.com>
2020-09-25 11:58         ` Jason Gunthorpe
2020-09-25 12:29           ` Tvrtko Ursulin
2020-09-25 12:34             ` Jason Gunthorpe
2020-09-25 12:13         ` Maor Gottlieb [this message]
2020-09-22  8:39 ` [PATCH rdma-next v3 2/2] RDMA/umem: Move to allocate " Leon Romanovsky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a8cca9f0-58ec-654d-939e-8568d17d4c60@nvidia.com \
    --to=maorg@nvidia.com \
    --cc=airlied@linux.ie \
    --cc=dledford@redhat.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=hch@lst.de \
    --cc=intel-gfx@lists.freedesktop.org \
    --cc=jgg@nvidia.com \
    --cc=leon@kernel.org \
    --cc=linux-graphics-maintainer@vmware.com \
    --cc=linux-rdma@vger.kernel.org \
    --cc=maorg@mellanox.com \
    --cc=sroland@vmware.com \
    --cc=tvrtko.ursulin@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-RDMA Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-rdma/0 linux-rdma/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-rdma linux-rdma/ https://lore.kernel.org/linux-rdma \
		linux-rdma@vger.kernel.org
	public-inbox-index linux-rdma

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-rdma


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git