From: Sagi Grimberg <sagi@grimberg.me> To: Bart Van Assche <bart.vanassche@sandisk.com>, Doug Ledford <dledford@redhat.com> Cc: linux-rdma@vger.kernel.org, Israel Rukshin <israelr@mellanox.com>, Max Gurtovoy <maxg@mellanox.com>, Leon Romanovsky <leonro@mellanox.com>, Mark Bloch <markb@mellanox.com>, Yuval Shaia <yuval.shaia@oracle.com>, "# 4 . 7+" <stable@vger.kernel.org> Subject: Re: [PATCH v2 1/8] IB/SRP: Avoid using IB_MR_TYPE_SG_GAPS Date: Wed, 15 Feb 2017 17:38:06 +0200 [thread overview] Message-ID: <cebcaeae-94a6-de82-cfc8-ce055b273836@grimberg.me> (raw) In-Reply-To: <20170214185636.29250-2-bart.vanassche@sandisk.com> > Tests have shown that the following error message is reported when > using SG-GAPS registration with an mlx5 adapter: > > scsi host1: ib_srp: failed RECV status WR flushed (5) for CQE ffff880bd4270eb0 > 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 > 00000000 0f007806 2500002a ad9fafd1 > scsi host1: ib_srp: reconnect succeeded > mlx5_0:dump_cqe:262:(pid 7369): dump error cqe > 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 > 00000000 0f007806 25000032 00105dd0 > scsi host1: ib_srp: failed FAST REG status memory management operation error (6) for CQE ffff880b92860138 > > Hence avoid using SG-GAPS memory registrations. Additionally, > always configure the blk_queue_virt_boundary() to avoid to trigger > a mapping failure when using adapters that support SG-GAPS (e.g. > mlx5). Hi Guys, Sorry for addressing this late, but has this failure been investigated? Max, Israel, what does this error syndrome map to? Looking at mlx5_ib_sg_to_klms, I think the mr->length is incorrectly incremented. Does the following change fix the problem? -- diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c index 8f608debe141..c21c9eee37f6 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -1832,7 +1832,7 @@ mlx5_ib_sg_to_klms(struct mlx5_ib_mr *mr, klms[i].va = cpu_to_be64(sg_dma_address(sg) + sg_offset); klms[i].bcount = cpu_to_be32(sg_dma_len(sg) - sg_offset); klms[i].key = cpu_to_be32(lkey); - mr->ibmr.length += sg_dma_len(sg); + mr->ibmr.length += sg_dma_len(sg) - sg_offset; sg_offset = 0; }
WARNING: multiple messages have this Message-ID (diff)
From: Sagi Grimberg <sagi@grimberg.me> To: Bart Van Assche <bart.vanassche@sandisk.com>, Doug Ledford <dledford@redhat.com> Cc: linux-rdma@vger.kernel.org, Israel Rukshin <israelr@mellanox.com>, Max Gurtovoy <maxg@mellanox.com>, Leon Romanovsky <leonro@mellanox.com>, Mark Bloch <markb@mellanox.com>, Yuval Shaia <yuval.shaia@oracle.com>, "# 4 . 7+" <stable@vger.kernel.org> Subject: Re: [PATCH v2 1/8] IB/SRP: Avoid using IB_MR_TYPE_SG_GAPS Date: Wed, 15 Feb 2017 17:38:06 +0200 [thread overview] Message-ID: <cebcaeae-94a6-de82-cfc8-ce055b273836@grimberg.me> (raw) In-Reply-To: <20170214185636.29250-2-bart.vanassche@sandisk.com> > Tests have shown that the following error message is reported when > using SG-GAPS registration with an mlx5 adapter: > > scsi host1: ib_srp: failed RECV status WR flushed (5) for CQE ffff880bd4270eb0 > 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 > 00000000 0f007806 2500002a ad9fafd1 > scsi host1: ib_srp: reconnect succeeded > mlx5_0:dump_cqe:262:(pid 7369): dump error cqe > 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 00000000 > 00000000 0f007806 25000032 00105dd0 > scsi host1: ib_srp: failed FAST REG status memory management operation error (6) for CQE ffff880b92860138 > > Hence avoid using SG-GAPS memory registrations. Additionally, > always configure the blk_queue_virt_boundary() to avoid to trigger > a mapping failure when using adapters that support SG-GAPS (e.g. > mlx5). Hi Guys, Sorry for addressing this late, but has this failure been investigated? Max, Israel, what does this error syndrome map to? Looking at mlx5_ib_sg_to_klms, I think the mr->length is incorrectly incremented. Does the following change fix the problem? -- diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c index 8f608debe141..c21c9eee37f6 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -1832,7 +1832,7 @@ mlx5_ib_sg_to_klms(struct mlx5_ib_mr *mr, klms[i].va = cpu_to_be64(sg_dma_address(sg) + sg_offset); klms[i].bcount = cpu_to_be32(sg_dma_len(sg) - sg_offset); klms[i].key = cpu_to_be32(lkey); - mr->ibmr.length += sg_dma_len(sg); + mr->ibmr.length += sg_dma_len(sg) - sg_offset; sg_offset = 0; } --
next prev parent reply other threads:[~2017-02-15 15:38 UTC|newest] Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-02-14 18:56 [PATCH v2 0/8] IB/srp bug fixes Bart Van Assche [not found] ` <20170214185636.29250-1-bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org> 2017-02-14 18:56 ` [PATCH v2 1/8] IB/SRP: Avoid using IB_MR_TYPE_SG_GAPS Bart Van Assche 2017-02-14 18:56 ` Bart Van Assche [not found] ` <20170214185636.29250-2-bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org> 2017-02-15 0:22 ` Bart Van Assche 2017-02-15 0:22 ` Bart Van Assche 2017-02-15 7:14 ` Leon Romanovsky 2017-02-15 7:14 ` Leon Romanovsky 2017-02-15 8:19 ` Leon Romanovsky 2017-02-15 8:19 ` Leon Romanovsky [not found] ` <20170215081945.GP6989-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org> 2017-02-15 13:33 ` Laurence Oberman 2017-02-15 13:33 ` Laurence Oberman 2017-02-15 13:43 ` Laurence Oberman 2017-02-15 13:47 ` Leon Romanovsky 2017-02-15 13:47 ` Leon Romanovsky 2017-02-15 14:53 ` Laurence Oberman [not found] ` <242820990.31706010.1487170436012.JavaMail.zimbra-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2017-02-15 15:18 ` Leon Romanovsky 2017-02-15 15:18 ` Leon Romanovsky 2017-02-15 15:42 ` Sagi Grimberg 2017-02-15 15:38 ` Sagi Grimberg [this message] 2017-02-15 15:38 ` Sagi Grimberg [not found] ` <cebcaeae-94a6-de82-cfc8-ce055b273836-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org> 2017-02-15 15:42 ` Laurence Oberman 2017-02-15 15:42 ` Laurence Oberman 2017-02-15 16:18 ` Max Gurtovoy 2017-02-15 16:18 ` Max Gurtovoy 2017-02-15 16:27 ` Sagi Grimberg [not found] ` <0514bb01-95cf-c10a-b883-494f149845f3-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> 2017-02-15 16:30 ` Leon Romanovsky 2017-02-15 16:30 ` Leon Romanovsky 2017-02-15 16:37 ` Laurence Oberman 2017-02-15 16:37 ` Laurence Oberman 2017-02-15 16:55 ` Sagi Grimberg 2017-02-15 23:49 ` Bart Van Assche 2017-02-16 6:14 ` Leon Romanovsky 2017-02-16 6:14 ` Leon Romanovsky 2017-02-16 9:11 ` Max Gurtovoy 2017-02-16 9:11 ` Max Gurtovoy 2017-02-14 18:56 ` [PATCH v2 2/8] IB/srp: Avoid that duplicate responses trigger a kernel bug Bart Van Assche 2017-02-14 18:56 ` Bart Van Assche 2017-02-15 7:22 ` Leon Romanovsky 2017-02-14 18:56 ` [PATCH v2 3/8] IB/srp: Fix race conditions related to task management Bart Van Assche 2017-02-14 18:56 ` Bart Van Assche 2017-02-14 18:56 ` [PATCH v2 4/8] IB/srp: Document locking conventions Bart Van Assche 2017-02-14 18:56 ` [PATCH v2 5/8] IB/srp: Make a diagnostic message more informative Bart Van Assche 2017-02-14 18:56 ` [PATCH v2 6/8] IB/srp: Improve an error path Bart Van Assche 2017-02-14 18:56 ` [PATCH v2 7/8] IB/core: Add support for draining IB_POLL_DIRECT completion queues Bart Van Assche [not found] ` <20170214185636.29250-8-bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org> 2017-02-14 19:00 ` Steve Wise 2017-02-15 7:21 ` Leon Romanovsky 2017-02-14 18:56 ` [PATCH v2 8/8] IB/srp: Drain the send queue before destroying a QP Bart Van Assche 2017-02-19 14:21 ` [PATCH v2 0/8] IB/srp bug fixes Doug Ledford [not found] <017955b3-8fd5-40da-8bd5-023bc2f23fb4@email.android.com> [not found] ` <017955b3-8fd5-40da-8bd5-023bc2f23fb4-2ueSQiBKiTY7tOexoI0I+QC/G2K4zDHf@public.gmane.org> 2017-02-15 15:31 ` [PATCH v2 1/8] IB/SRP: Avoid using IB_MR_TYPE_SG_GAPS Bart Van Assche 2017-02-15 15:31 ` Bart Van Assche [not found] ` <1487172663.2990.5.camel-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org> 2017-02-15 15:34 ` Laurence Oberman 2017-02-15 15:34 ` Laurence Oberman
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=cebcaeae-94a6-de82-cfc8-ce055b273836@grimberg.me \ --to=sagi@grimberg.me \ --cc=bart.vanassche@sandisk.com \ --cc=dledford@redhat.com \ --cc=israelr@mellanox.com \ --cc=leonro@mellanox.com \ --cc=linux-rdma@vger.kernel.org \ --cc=markb@mellanox.com \ --cc=maxg@mellanox.com \ --cc=stable@vger.kernel.org \ --cc=yuval.shaia@oracle.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.