From: Wenbin Zeng <wenbin.zeng@gmail.com>
To: "J. Bruce Fields" <bfields@fieldses.org>
Cc: viro@zeniv.linux.org.uk, davem@davemloft.net, jlayton@kernel.org,
trond.myklebust@hammerspace.com, anna.schumaker@netapp.com,
wenbinzeng@tencent.com, dsahern@gmail.com,
nicolas.dichtel@6wind.com, willy@infradead.org,
edumazet@google.com, jakub.kicinski@netronome.com,
tyhicks@canonical.com, chuck.lever@oracle.com, neilb@suse.com,
linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org, linux-nfs@vger.kernel.org
Subject: Re: [PATCH 0/3] auth_gss: netns refcount leaks when use-gss-proxy==1
Date: Fri, 10 May 2019 13:09:46 +0800 [thread overview]
Message-ID: <20190510050946.GA17994@bridge> (raw)
In-Reply-To: <20190509205218.GA23548@fieldses.org>
On Thu, May 09, 2019 at 04:52:18PM -0400, J. Bruce Fields wrote:
> Thanks for figuring this out!
>
> I guess I'll take these patches (with the one fix in your response to
> Al) through the nfsd tree, unless someone tells me otherwise. (The
> original bug was introduced through nfsd.)
Thank you, Bruce.
I am submitting v2 with that fix right away.
>
> How serious are the consequences of the leak? I'm wondering if it's
> worth a stable cc or not.
Though the leak only happens with _privileged_ docker containers that have
gssproxy service enabled and use-gss-proxy set to 1, the consequences
can be ugly, the killed/stopped containers not only leave struct net
unfreed, also possibly leave behind veth devices linked to the netns, in
environments that containers are frequently killed/stopped, it is quite
ugly.
>
> --b.
>
> On Wed, May 01, 2019 at 02:42:22PM +0800, Wenbin Zeng wrote:
> > This patch series fixes an auth_gss bug that results in netns refcount leaks when use-gss-proxy is set to 1.
> >
> > The problem was found in privileged docker containers with gssproxy service enabled and /proc/net/rpc/use-gss-proxy set to 1, the corresponding struct net->count ends up at 2 after container gets killed, the consequence is that the struct net cannot be freed.
> >
> > It turns out that write_gssp() called gssp_rpc_create() to create a rpc client, this increases net->count by 2; rpcsec_gss_exit_net() is supposed to decrease net->count but it never gets called because its call-path is:
> > net->count==0 -> cleanup_net -> ops_exit_list -> rpcsec_gss_exit_net
> > Before rpcsec_gss_exit_net() gets called, net->count cannot reach 0, this is a deadlock situation.
> >
> > To fix the problem, we must break the deadlock, rpcsec_gss_exit_net() should move out of the put() path and find another chance to get called, I think nsfs_evict() is a good place to go, when netns inode gets evicted we call rpcsec_gss_exit_net() to free the rpc client, this requires a new callback i.e. evict to be added in struct proc_ns_operations, and add netns_evict() as one of netns_operations as well.
> >
> > Wenbin Zeng (3):
> > nsfs: add evict callback into struct proc_ns_operations
> > netns: add netns_evict into netns_operations
> > auth_gss: fix deadlock that blocks rpcsec_gss_exit_net when
> > use-gss-proxy==1
> >
> > fs/nsfs.c | 2 ++
> > include/linux/proc_ns.h | 1 +
> > include/net/net_namespace.h | 1 +
> > net/core/net_namespace.c | 12 ++++++++++++
> > net/sunrpc/auth_gss/auth_gss.c | 9 ++++++---
> > 5 files changed, 22 insertions(+), 3 deletions(-)
> >
> > --
> > 1.8.3.1
next prev parent reply other threads:[~2019-05-10 5:09 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-01 6:42 [PATCH 0/3] auth_gss: netns refcount leaks when use-gss-proxy==1 Wenbin Zeng
2019-05-01 6:42 ` [PATCH 1/3] nsfs: add evict callback into struct proc_ns_operations Wenbin Zeng
2019-05-02 3:04 ` Al Viro
2019-05-04 16:08 ` Wenbin Zeng
2019-05-01 6:42 ` [PATCH 2/3] netns: add netns_evict into netns_operations Wenbin Zeng
2019-05-04 4:10 ` David Miller
2019-05-01 6:42 ` [PATCH 3/3] auth_gss: fix deadlock that blocks rpcsec_gss_exit_net when use-gss-proxy==1 Wenbin Zeng
2019-05-09 20:52 ` [PATCH 0/3] auth_gss: netns refcount leaks " J. Bruce Fields
2019-05-10 5:09 ` Wenbin Zeng [this message]
2019-05-10 6:36 ` [PATCH v2 " Wenbin Zeng
2019-05-10 6:36 ` [PATCH v2 1/3] nsfs: add evict callback into struct proc_ns_operations Wenbin Zeng
2019-05-10 6:36 ` [PATCH v2 2/3] netns: add netns_evict into netns_operations Wenbin Zeng
2019-05-10 22:13 ` David Miller
2019-05-10 6:36 ` [PATCH v2 3/3] auth_gss: fix deadlock that blocks rpcsec_gss_exit_net when use-gss-proxy==1 Wenbin Zeng
2019-05-15 1:03 ` [PATCH v2 0/3] auth_gss: netns refcount leaks " J. Bruce Fields
2019-06-12 8:37 ` Wenbin Zeng
2019-06-12 15:52 ` J. Bruce Fields
2021-09-07 14:48 ` wanghai (M)
2021-09-08 20:51 ` J. Bruce Fields
2021-09-09 2:52 ` wanghai (M)
2021-09-09 19:52 ` J. Bruce Fields
2019-06-12 12:09 ` [PATCH v3 " Wenbin Zeng
2019-06-12 12:09 ` [PATCH v3 1/3] nsfs: add evict callback into struct proc_ns_operations Wenbin Zeng
2019-06-12 12:09 ` [PATCH v3 2/3] netns: add netns_evict into netns_operations Wenbin Zeng
2019-06-12 12:09 ` [PATCH v3 3/3] auth_gss: fix deadlock that blocks rpcsec_gss_exit_net when use-gss-proxy==1 Wenbin Zeng
2019-08-01 19:53 ` [PATCH v3 0/3] auth_gss: netns refcount leaks " J. Bruce Fields
2021-08-28 11:26 ` wanghai (M)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190510050946.GA17994@bridge \
--to=wenbin.zeng@gmail.com \
--cc=anna.schumaker@netapp.com \
--cc=bfields@fieldses.org \
--cc=chuck.lever@oracle.com \
--cc=davem@davemloft.net \
--cc=dsahern@gmail.com \
--cc=edumazet@google.com \
--cc=jakub.kicinski@netronome.com \
--cc=jlayton@kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nfs@vger.kernel.org \
--cc=neilb@suse.com \
--cc=netdev@vger.kernel.org \
--cc=nicolas.dichtel@6wind.com \
--cc=trond.myklebust@hammerspace.com \
--cc=tyhicks@canonical.com \
--cc=viro@zeniv.linux.org.uk \
--cc=wenbinzeng@tencent.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).