Linux-NFS Archive on lore.kernel.org
 help / color / Atom feed
From: Wenbin Zeng <wenbin.zeng@gmail.com>
To: "J. Bruce Fields" <bfields@fieldses.org>
Cc: viro@zeniv.linux.org.uk, davem@davemloft.net, jlayton@kernel.org,
	trond.myklebust@hammerspace.com, anna.schumaker@netapp.com,
	wenbinzeng@tencent.com, dsahern@gmail.com,
	nicolas.dichtel@6wind.com, willy@infradead.org,
	edumazet@google.com, jakub.kicinski@netronome.com,
	tyhicks@canonical.com, chuck.lever@oracle.com, neilb@suse.com,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
	netdev@vger.kernel.org, linux-nfs@vger.kernel.org
Subject: Re: [PATCH 0/3] auth_gss: netns refcount leaks when use-gss-proxy==1
Date: Fri, 10 May 2019 13:09:46 +0800
Message-ID: <20190510050946.GA17994@bridge> (raw)
In-Reply-To: <20190509205218.GA23548@fieldses.org>

On Thu, May 09, 2019 at 04:52:18PM -0400, J. Bruce Fields wrote:
> Thanks for figuring this out!
> 
> I guess I'll take these patches (with the one fix in your response to
> Al) through the nfsd tree, unless someone tells me otherwise.  (The
> original bug was introduced through nfsd.)

Thank you, Bruce.
I am submitting v2 with that fix right away.

> 
> How serious are the consequences of the leak?  I'm wondering if it's
> worth a stable cc or not.

Though the leak only happens with _privileged_ docker containers that have
gssproxy service enabled and use-gss-proxy set to 1, the consequences
can be ugly, the killed/stopped containers not only leave struct net
unfreed, also possibly leave behind veth devices linked to the netns, in
environments that containers are frequently killed/stopped, it is quite
ugly.

> 
> --b.
> 
> On Wed, May 01, 2019 at 02:42:22PM +0800, Wenbin Zeng wrote:
> > This patch series fixes an auth_gss bug that results in netns refcount leaks when use-gss-proxy is set to 1.
> > 
> > The problem was found in privileged docker containers with gssproxy service enabled and /proc/net/rpc/use-gss-proxy set to 1, the corresponding struct net->count ends up at 2 after container gets killed, the consequence is that the struct net cannot be freed.
> > 
> > It turns out that write_gssp() called gssp_rpc_create() to create a rpc client, this increases net->count by 2; rpcsec_gss_exit_net() is supposed to decrease net->count but it never gets called because its call-path is:
> > 	net->count==0 -> cleanup_net -> ops_exit_list -> rpcsec_gss_exit_net
> > Before rpcsec_gss_exit_net() gets called, net->count cannot reach 0, this is a deadlock situation.
> > 
> > To fix the problem, we must break the deadlock, rpcsec_gss_exit_net() should move out of the put() path and find another chance to get called, I think nsfs_evict() is a good place to go, when netns inode gets evicted we call rpcsec_gss_exit_net() to free the rpc client, this requires a new callback i.e. evict to be added in struct proc_ns_operations, and add netns_evict() as one of netns_operations as well.
> > 
> > Wenbin Zeng (3):
> >   nsfs: add evict callback into struct proc_ns_operations
> >   netns: add netns_evict into netns_operations
> >   auth_gss: fix deadlock that blocks rpcsec_gss_exit_net when
> >     use-gss-proxy==1
> > 
> >  fs/nsfs.c                      |  2 ++
> >  include/linux/proc_ns.h        |  1 +
> >  include/net/net_namespace.h    |  1 +
> >  net/core/net_namespace.c       | 12 ++++++++++++
> >  net/sunrpc/auth_gss/auth_gss.c |  9 ++++++---
> >  5 files changed, 22 insertions(+), 3 deletions(-)
> > 
> > -- 
> > 1.8.3.1

  reply index

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-01  6:42 Wenbin Zeng
2019-05-01  6:42 ` [PATCH 1/3] nsfs: add evict callback into struct proc_ns_operations Wenbin Zeng
2019-05-02  3:04   ` Al Viro
2019-05-04 16:08     ` Wenbin Zeng
2019-05-01  6:42 ` [PATCH 2/3] netns: add netns_evict into netns_operations Wenbin Zeng
2019-05-04  4:10   ` David Miller
2019-05-01  6:42 ` [PATCH 3/3] auth_gss: fix deadlock that blocks rpcsec_gss_exit_net when use-gss-proxy==1 Wenbin Zeng
2019-05-09 20:52 ` [PATCH 0/3] auth_gss: netns refcount leaks " J. Bruce Fields
2019-05-10  5:09   ` Wenbin Zeng [this message]
2019-05-10  6:36 ` [PATCH v2 " Wenbin Zeng
2019-05-10  6:36   ` [PATCH v2 1/3] nsfs: add evict callback into struct proc_ns_operations Wenbin Zeng
2019-05-10  6:36   ` [PATCH v2 2/3] netns: add netns_evict into netns_operations Wenbin Zeng
2019-05-10 22:13     ` David Miller
2019-05-10  6:36   ` [PATCH v2 3/3] auth_gss: fix deadlock that blocks rpcsec_gss_exit_net when use-gss-proxy==1 Wenbin Zeng
2019-05-15  1:03   ` [PATCH v2 0/3] auth_gss: netns refcount leaks " J. Bruce Fields
2019-06-12  8:37     ` Wenbin Zeng
2019-06-12 15:52       ` J. Bruce Fields
2019-06-12 12:09 ` [PATCH v3 " Wenbin Zeng
2019-06-12 12:09   ` [PATCH v3 1/3] nsfs: add evict callback into struct proc_ns_operations Wenbin Zeng
2019-06-12 12:09   ` [PATCH v3 2/3] netns: add netns_evict into netns_operations Wenbin Zeng
2019-06-12 12:09   ` [PATCH v3 3/3] auth_gss: fix deadlock that blocks rpcsec_gss_exit_net when use-gss-proxy==1 Wenbin Zeng
2019-08-01 19:53   ` [PATCH v3 0/3] auth_gss: netns refcount leaks " J. Bruce Fields

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190510050946.GA17994@bridge \
    --to=wenbin.zeng@gmail.com \
    --cc=anna.schumaker@netapp.com \
    --cc=bfields@fieldses.org \
    --cc=chuck.lever@oracle.com \
    --cc=davem@davemloft.net \
    --cc=dsahern@gmail.com \
    --cc=edumazet@google.com \
    --cc=jakub.kicinski@netronome.com \
    --cc=jlayton@kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=neilb@suse.com \
    --cc=netdev@vger.kernel.org \
    --cc=nicolas.dichtel@6wind.com \
    --cc=trond.myklebust@hammerspace.com \
    --cc=tyhicks@canonical.com \
    --cc=viro@zeniv.linux.org.uk \
    --cc=wenbinzeng@tencent.com \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-NFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-nfs/0 linux-nfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-nfs linux-nfs/ https://lore.kernel.org/linux-nfs \
		linux-nfs@vger.kernel.org linux-nfs@archiver.kernel.org
	public-inbox-index linux-nfs


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-nfs


AGPL code for this site: git clone https://public-inbox.org/ public-inbox