From: Jason Gunthorpe <jgg@nvidia.com>
To: Zhu Yanjun <zyjzyj2000@gmail.com>
Cc: Leon Romanovsky <leon@kernel.org>,
Doug Ledford <dledford@redhat.com>,
RDMA mailing list <linux-rdma@vger.kernel.org>,
maorg@nvidia.com
Subject: Re: Fwd: [PATCH 1/1] RDMA/umem: add back hugepage sg list
Date: Fri, 12 Mar 2021 09:02:43 -0400 [thread overview]
Message-ID: <20210312130243.GU2356281@nvidia.com> (raw)
In-Reply-To: <CAD=hENcb4DAs7hhPLqgqAhLr8xYqxkMhGp=99bfgq1SZN57QHA@mail.gmail.com>
On Fri, Mar 12, 2021 at 04:04:35PM +0800, Zhu Yanjun wrote:
> On Fri, Mar 12, 2021 at 8:25 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
> >
> > On Thu, Mar 11, 2021 at 06:41:43PM +0800, Zhu Yanjun wrote:
> > > On Mon, Mar 8, 2021 at 8:16 PM Jason Gunthorpe <jgg@nvidia.com> wrote:
> > > >
> > > > On Mon, Mar 08, 2021 at 06:13:52PM +0800, Zhu Yanjun wrote:
> > > >
> > > > > And I delved into the source code of __sg_alloc_table_from_pages. I
> > > > > found that this function is related with ib_dma_max_seg_size. So
> > > > > when ib_dma_max_seg_size is set to UINT_MAX, the sg dma address is
> > > > > 4K (one page). When ib_dma_max_seg_size is set to SZ_2M, the sg dma
> > > > > address is 2M now.
> > > >
> > > > That seems like a bug, you should fix it
> > >
> > > Hi, Jason && Leon
> > >
> > > I compared the function __sg_alloc_table_from_pages with ib_umem_add_sg_table.
> > > In __sg_alloc_table_from_pages:
> > >
> > > "
> > > 449 if (prv) {
> > > 450 unsigned long paddr = (page_to_pfn(sg_page(prv))
> > > * PAGE_SIZE +
> > > 451 prv->offset + prv->length) /
> > > 452 PAGE_SIZE;
> > > 453
> > > 454 if (WARN_ON(offset))
> > > 455 return ERR_PTR(-EINVAL);
> > > 456
> > > 457 /* Merge contiguous pages into the last SG */
> > > 458 prv_len = prv->length;
> > > 459 while (n_pages && page_to_pfn(pages[0]) == paddr) {
> > > 460 if (prv->length + PAGE_SIZE > max_segment)
> > > 461 break;
> > > 462 prv->length += PAGE_SIZE;
> > > 463 paddr++;
> > > 464 pages++;
> > > 465 n_pages--;
> > > 466 }
> > > 467 if (!n_pages)
> > > 468 goto out;
> > > 469 }
> > >
> > > "
> > > if prv->length + PAGE_SIZE > max_segment, then set another sg.
> > > In the commit "RDMA/umem: Move to allocate SG table from pages",
> > > max_segment is dma_get_max_seg_size.
> > > Normally it is UINT_MAX. So in my host, prv->length + PAGE_SIZE is
> > > usually less than max_segment
> > > since length is unsigned int.
> >
> > I don't understand what you are trying to say
> >
> > 460 if (prv->length + PAGE_SIZE > max_segment)
> >
> > max_segment should be a very big number and "prv->length + PAGE_SIZE" should
> > always be < max_segment so it should always be increasing the size of
> > prv->length and 'rpv' here is the sgl.
>
> Sure.
>
> When max_segment is UINT_MAX, prv->length is UINT_MAX - PAGE_SIZE.
> It is big. That is, segment size is UINT_MAX - PAGE_SIZE.
>
> But from the function ib_umem_add_sg_table, the prv->length is SZ_2M.
> That is, segment size if SZ_2M.
>
> It is the difference between the 2 functions.
I still have no idea what you are trying to say. Why would prv->length
be 'UINT - PAGE_SIZE'? That sounds like max_segment
Jason
next prev parent reply other threads:[~2021-03-12 13:03 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20210307221034.568606-1-yanjun.zhu@intel.com>
2021-03-07 14:28 ` Fwd: [PATCH 1/1] RDMA/umem: add back hugepage sg list Zhu Yanjun
2021-03-07 17:20 ` Leon Romanovsky
2021-03-07 17:22 ` Leon Romanovsky
2021-03-08 2:44 ` Zhu Yanjun
2021-03-08 10:13 ` Zhu Yanjun
2021-03-08 12:16 ` Jason Gunthorpe
2021-03-11 10:41 ` Zhu Yanjun
2021-03-12 0:25 ` Jason Gunthorpe
2021-03-12 8:04 ` Zhu Yanjun
2021-03-12 8:05 ` Zhu Yanjun
2021-03-12 13:02 ` Jason Gunthorpe [this message]
2021-03-12 13:42 ` Zhu Yanjun
2021-03-12 13:49 ` Zhu Yanjun
2021-03-12 14:01 ` Jason Gunthorpe
2021-03-13 3:02 ` Zhu Yanjun
2021-03-19 13:00 ` Jason Gunthorpe
2021-03-19 13:33 ` Zhu Yanjun
2021-03-19 13:48 ` Jason Gunthorpe
2021-03-20 3:38 ` Zhu Yanjun
2021-03-20 3:49 ` Zhu Yanjun
2021-03-20 20:38 ` Jason Gunthorpe
2021-03-21 8:06 ` Zhu Yanjun
2021-03-21 12:07 ` Jason Gunthorpe
2021-03-21 12:54 ` Zhu Yanjun
2021-03-21 13:03 ` Jason Gunthorpe
2021-03-21 14:38 ` Zhu Yanjun
2021-03-21 15:52 ` Jason Gunthorpe
2021-03-22 5:07 ` Zhu Yanjun
2021-03-08 0:44 ` Jason Gunthorpe
2021-03-08 2:47 ` Zhu Yanjun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210312130243.GU2356281@nvidia.com \
--to=jgg@nvidia.com \
--cc=dledford@redhat.com \
--cc=leon@kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=maorg@nvidia.com \
--cc=zyjzyj2000@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.