From: 858585 jemmy <jemmy858585@gmail.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: dledford@redhat.com, akpm@linux-foundation.org,
qing.huang@oracle.com, Leon Romanovsky <leon@kernel.org>,
artemyko@mellanox.com, dan.j.williams@intel.com,
linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org,
adido@mellanox.com, Gal Shachaf <galsha@mellanox.com>,
Aviad Yehezkel <aviadye@mellanox.com>,
Lidong Chen <lidongchen@tencent.com>
Subject: Re: [PATCH] IB/umem: use tgid instead of pid in ib_umem structure
Date: Tue, 8 May 2018 16:32:51 +0800 [thread overview]
Message-ID: <CAOGPPbd3BgLqyjXPwcpoYOH2N_kN2j+SkeOhSie4FriB9TRosg@mail.gmail.com> (raw)
In-Reply-To: <20180508063006.aicgwalirnkjmeuf@ziepe.ca>
On Tue, May 8, 2018 at 2:30 PM, Jason Gunthorpe <jgg@ziepe.ca> wrote:
> On Mon, May 07, 2018 at 09:38:53AM +0800, 858585 jemmy wrote:
>> On Sat, May 5, 2018 at 2:23 AM, Jason Gunthorpe <jgg@ziepe.ca> wrote:
>> > On Fri, May 04, 2018 at 04:51:15PM +0800, 858585 jemmy wrote:
>> >> On Fri, May 4, 2018 at 11:14 AM, 858585 jemmy <jemmy858585@gmail.com> wrote:
>> >> > On Thu, May 3, 2018 at 11:33 PM, Jason Gunthorpe <jgg@ziepe.ca> wrote:
>> >> >> On Thu, May 03, 2018 at 10:04:34PM +0800, Lidong Chen wrote:
>> >> >>> The userspace may invoke ibv_reg_mr and ibv_dereg_mr by different threads.
>> >> >>> If when ibv_dereg_mr invoke and the thread which invoked ibv_reg_mr has
>> >> >>> exited, get_pid_task will return NULL, ib_umem_release does not decrease
>> >> >>> mm->pinned_vm. This patch fixes it by use tgid.
>> >> >>>
>> >> >>> Signed-off-by: Lidong Chen <lidongchen@tencent.com>
>> >> >>> drivers/infiniband/core/umem.c | 12 ++++++------
>> >> >>> include/rdma/ib_umem.h | 2 +-
>> >> >>> 2 files changed, 7 insertions(+), 7 deletions(-)
>> >> >>
>> >> >> Why are we even using a struct pid for this? Does anyone know?
>> >> >
>> >> > commit 87773dd56d5405ac28119fcfadacefd35877c18f add pid in ib_umem structure.
>> >> >
>> >> > and the comment has such information:
>> >> > Later a different process with a different mm_struct than the one that
>> >> > allocated the ib_umem struct
>> >> > ends up releasing it which results in decrementing the new processes
>> >> > mm->pinned_vm count past
>> >> > zero and wrapping.
>> >>
>> >> I think a different process should not have the permission to release ib_umem.
>> >> so maybe the reason is not a different process?
>> >> can ib_umem_release be invoked in interrupt context?
>> >
>> > We plan to restore fork support and add some way to share MRs between
>> > processes, so we must consider having a different process release the
>> > umem than acquired it.
>>
>> If restore fork support, what is the expected behavior?
>> If parent process pinned_vm is x, what is the child process pinned_vm
>> value after fork? It reset to zero now.
>> If the parent process call ibv_dereg_mr after fork, should the child
>> process decrease pinned_vm?
>> If the child process call ibv_dereg_mr after fork, should the parent
>> process decrease pinned_vm?
>
> If I recall the purpose of accessing the MM during de-register is to
> undo the pinned pages change (pinned_vm) that register performed.
>
> So, the semantic is simple, during deregister we must access excatly
> the same MM that was used during register and undo the change to
> pinned_vm.
>
> The approach should be to find the most reliably way to hold a
> reference to the MM that was used during register.
>
> Apparently we can't just hold a ref on the mm (according to mm_get's
> comment at least)
>
> tgid is clearly a better indirect reference to the mm than pid (pid is
> so obviously wrong)
>
> But I am wondering why not just hold struct task here instead of tgid?
> Isn't task->mm going to be more reliably than tgid->task->mm ??
I think get_task_struct(current->group_leader) is also work.
But I find ib_ucontext structure already have a tgid field, so I think this not
necessary to ib_umem have tgid again. we can use ib_ucontext->tgid.
I will send a v2 patch.
>
> Jason
prev parent reply other threads:[~2018-05-08 8:32 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-05-03 14:04 [PATCH] IB/umem: use tgid instead of pid in ib_umem structure Lidong Chen
2018-05-03 15:33 ` Jason Gunthorpe
2018-05-03 18:12 ` Leon Romanovsky
2018-05-03 18:26 ` Jason Gunthorpe
2018-05-03 18:43 ` Leon Romanovsky
2018-05-03 22:01 ` Jason Gunthorpe
2018-05-04 8:32 ` 858585 jemmy
2018-05-04 13:39 ` Leon Romanovsky
2018-05-04 15:14 ` lidongchen(陈立东)
2018-05-04 2:41 ` 858585 jemmy
2018-05-04 3:14 ` 858585 jemmy
2018-05-04 8:51 ` 858585 jemmy
2018-05-04 18:23 ` Jason Gunthorpe
2018-05-07 1:38 ` 858585 jemmy
2018-05-08 6:30 ` Jason Gunthorpe
2018-05-08 8:32 ` 858585 jemmy [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAOGPPbd3BgLqyjXPwcpoYOH2N_kN2j+SkeOhSie4FriB9TRosg@mail.gmail.com \
--to=jemmy858585@gmail.com \
--cc=adido@mellanox.com \
--cc=akpm@linux-foundation.org \
--cc=artemyko@mellanox.com \
--cc=aviadye@mellanox.com \
--cc=dan.j.williams@intel.com \
--cc=dledford@redhat.com \
--cc=galsha@mellanox.com \
--cc=jgg@ziepe.ca \
--cc=leon@kernel.org \
--cc=lidongchen@tencent.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=qing.huang@oracle.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).