From: Tom Talpey <tom@talpey.com>
To: chucklever@gmail.com
Cc: linux-rdma <linux-rdma@vger.kernel.org>,
Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH v1 2/8] xprtrdma: Do not post Receives after disconnect
Date: Wed, 31 Mar 2021 17:22:56 -0400 [thread overview]
Message-ID: <b5286b3f-439a-ef7c-cd7c-6cebfbccb02d@talpey.com> (raw)
In-Reply-To: <CAFMMQGvortADqgmAzskZKcnyHDzsTEW0FtR501wpP+deUM57FA@mail.gmail.com>
On 3/31/2021 4:31 PM, Chuck Lever wrote:
> On Wed, Mar 31, 2021 at 4:01 PM Tom Talpey <tom@talpey.com> wrote:
>>
>> On 3/31/2021 3:36 PM, Chuck Lever wrote:
>>> Currently the Receive completion handler refreshes the Receive Queue
>>> whenever a successful Receive completion occurs.
>>>
>>> On disconnect, xprtrdma drains the Receive Queue. The first few
>>> Receive completions after a disconnect are typically successful,
>>> until the first flushed Receive.
snip
>> Is it not possible to mark the WRs as
>> being part of a batch, and allowing them to flush? You could borrow a
>> bit in the completion cookie, and check it when the CQE pops out. Maybe.
>
> It's not an issue with batching, it's an issue with posting Receives from the
> Receive completion handler. I'd think that any of the ULPs that post Receives
> in their completion handler would have the same issue.
>
> The purpose of the QP drain in rpcrdma_xprt_disconnect() is to ensure there
> are no more WRs in flight so that the hardware resources can be safely
> destroyed. If the Receive completion handler continues to post Receive WRs
> after the drain sentinel has been posted, leaks and crashes become possible.
Well, why not do an atomic_set() of a flag just before posting the
sentinel, and check it with atomic_get() before any other RQ post?
Tom.
next prev parent reply other threads:[~2021-03-31 21:23 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-31 19:35 [PATCH v1 0/8] xprtrdma Receive Queue fixes Chuck Lever
2021-03-31 19:36 ` [PATCH v1 1/8] xprtrdma: Avoid Receive Queue wrapping Chuck Lever
2021-03-31 20:05 ` Tom Talpey
2021-03-31 19:36 ` [PATCH v1 2/8] xprtrdma: Do not post Receives after disconnect Chuck Lever
2021-03-31 19:59 ` Tom Talpey
2021-03-31 20:31 ` Chuck Lever
2021-03-31 21:22 ` Tom Talpey [this message]
2021-04-01 16:56 ` Chuck Lever
2021-03-31 19:36 ` [PATCH v1 3/8] xprtrdma: Put flushed Receives on free list instead of destroying them Chuck Lever
2021-03-31 20:02 ` Tom Talpey
2021-03-31 19:36 ` [PATCH v1 4/8] xprtrdma: Improve locking around rpcrdma_rep destruction Chuck Lever
2021-03-31 19:36 ` [PATCH v1 5/8] xprtrdma: Improve commentary around rpcrdma_reps_unmap() Chuck Lever
2021-03-31 19:36 ` [PATCH v1 6/8] xprtrdma: Improve locking around rpcrdma_rep creation Chuck Lever
2021-03-31 19:36 ` [PATCH v1 7/8] xprtrdma: Fix cwnd update ordering Chuck Lever
2021-03-31 19:36 ` [PATCH v1 8/8] xprtrdma: Delete rpcrdma_recv_buffer_put() Chuck Lever
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b5286b3f-439a-ef7c-cd7c-6cebfbccb02d@talpey.com \
--to=tom@talpey.com \
--cc=chucklever@gmail.com \
--cc=linux-nfs@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).