Linux-RDMA Archive on lore.kernel.org
 help / color / Atom feed
From: Jason Gunthorpe <jgg@nvidia.com>
To: Maor Gottlieb <maorg@nvidia.com>,
	Tvrtko Ursulin <tvrtko.ursulin@intel.com>,
	Christoph Hellwig <hch@lst.de>
Cc: Gal Pressman <galpress@amazon.com>,
	Bob Pearson <rpearsonhpe@gmail.com>,
	Leon Romanovsky <leonro@nvidia.com>, <linux-rdma@vger.kernel.org>
Subject: Re: dynamic-sg patch has broken rdma_rxe
Date: Thu, 15 Oct 2020 21:31:27 -0300
Message-ID: <20201016003127.GD6219@nvidia.com> (raw)
In-Reply-To: <8cf4796d-4dcb-ef5a-83ac-e11134eac99b@nvidia.com>

On Thu, Oct 15, 2020 at 03:21:34PM +0300, Maor Gottlieb wrote:
> 
> On 10/15/2020 2:23 PM, Gal Pressman wrote:
> > On 15/10/2020 10:44, Maor Gottlieb wrote:
> > > On 10/15/2020 1:51 AM, Jason Gunthorpe wrote:
> > > > On Tue, Oct 13, 2020 at 09:33:14AM -0500, Bob Pearson wrote:
> > > > > Jason,
> > > > > 
> > > > > Just pulled for-next and now hit the following warning.
> > > > > Register user space memory is not longer working.
> > > > > I am trying to debug this but if you have any idea where to look let me know.
> > > > The offset_in_page is wrong, but it is protecting some other logic..
> > > > 
> > > > Maor? Leon? Can you sort it out tomorrow?
> > > Leon and I investigated it. This check existed before my series to protect the
> > > alloc_table_from_pages logic. It's still relevant.
> > > This patch that broke it:  54816d3e69d1 ("RDMA: Explicitly pass in the
> > > dma_device to ib_register_device"), and according to below link it was
> > > expected.  The safest approach is to set the max_segment_size back the 2GB in
> > > all drivers. What do you think?
> > > 
> > > https://lore.kernel.org/linux-rdma/20200923072111.GA31828@infradead.org/
> > FWIW, EFA is broken as well (same call trace) so it's not just software drivers.
> 
> This is true to all drivers that call to ib_umem_get and set UINT_MAX  as
> max_segment_size.
> Jason,  maybe instead of set UINT_MAX as max_segment_size, need to set
> SCATTERLIST_MAX_SEGMENT which does the required alignment.

SCATTERLIST_MAX_SEGMENT is almost never used, however there are lots
of places passing UINT_MAX or similar as the max_segsize for DMA.

The only place that does use it looks goofy to me:

	dma_set_max_seg_size(dev->dev, min_t(unsigned int, U32_MAX & PAGE_MASK,
					     SCATTERLIST_MAX_SEGMENT));

The seg_size should reflect the HW capability, not be mixed in with
knowledge about SGL internals. If the SGL can't build up to the HW
limit then it is fine to internally silently reduce it.

So I think we need to fix the scatterlist code, like below, and
just remove SCATTERLIST_MAX_SEGMENT completely.

It fixes things? Are you OK with this Christoph?

I need to get this fixed for the merge window PR I want to send on
Friday.

diff --git a/lib/scatterlist.c b/lib/scatterlist.c
index e102fdfaa75be7..d158033834cdbc 100644
--- a/lib/scatterlist.c
+++ b/lib/scatterlist.c
@@ -435,7 +435,9 @@ struct scatterlist *__sg_alloc_table_from_pages(struct sg_table *sgt,
 	unsigned int added_nents = 0;
 	struct scatterlist *s = prv;
 
-	if (WARN_ON(!max_segment || offset_in_page(max_segment)))
+	/* Avoid overflow when computing sg_len + PAGE_SIZE */
+	max_segment = max_segment & PAGE_MASK;
+	if (WARN_ON(max_segment < PAGE_SIZE))
 		return ERR_PTR(-EINVAL);
 
 	if (IS_ENABLED(CONFIG_ARCH_NO_SG_CHAIN) && prv)

  reply index

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-13 14:33 Bob Pearson
2020-10-13 16:34 ` Bob Pearson
2020-10-14 22:51 ` Jason Gunthorpe
2020-10-15  7:44   ` Maor Gottlieb
2020-10-15 11:23     ` Gal Pressman
2020-10-15 12:21       ` Maor Gottlieb
2020-10-16  0:31         ` Jason Gunthorpe [this message]
2020-10-16  7:13           ` Christoph Hellwig
     [not found]           ` <796ca31aed8f469c957cb850385b9d09@intel.com>
2020-10-16 11:58             ` Jason Gunthorpe
2020-10-19  9:50               ` Tvrtko Ursulin
2020-10-19 12:12                 ` Jason Gunthorpe
2020-10-19 12:29                   ` Tvrtko Ursulin
2020-10-19 12:48                     ` Jason Gunthorpe
2020-10-20 11:37                       ` Tvrtko Ursulin
2020-10-20 11:47                         ` Jason Gunthorpe
2020-10-20 12:31                           ` Tvrtko Ursulin
2020-10-20 12:56                             ` Jason Gunthorpe
2020-10-20 13:09                               ` Tvrtko Ursulin
2020-10-20 13:32                                 ` Jason Gunthorpe
2020-10-15 15:35     ` Bob Pearson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201016003127.GD6219@nvidia.com \
    --to=jgg@nvidia.com \
    --cc=galpress@amazon.com \
    --cc=hch@lst.de \
    --cc=leonro@nvidia.com \
    --cc=linux-rdma@vger.kernel.org \
    --cc=maorg@nvidia.com \
    --cc=rpearsonhpe@gmail.com \
    --cc=tvrtko.ursulin@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-RDMA Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-rdma/0 linux-rdma/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-rdma linux-rdma/ https://lore.kernel.org/linux-rdma \
		linux-rdma@vger.kernel.org
	public-inbox-index linux-rdma

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-rdma


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git