linux-nfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "J. Bruce Fields" <bfields@fieldses.org>
To: Frank Sorenson <sorenson@redhat.com>
Cc: Roberto Bergantinos Corpas <rbergant@redhat.com>,
	chuck.lever@oracle.com, trond.myklebust@hammerspace.com,
	linux-nfs@vger.kernel.org
Subject: Re: [PATCH] sunrpc: expiry_time should be seconds not timeval
Date: Fri, 24 Jan 2020 16:23:06 -0500	[thread overview]
Message-ID: <20200124212306.GD26874@fieldses.org> (raw)
In-Reply-To: <438af54f-e1a8-467a-4ef1-821b67b7bb6c@redhat.com>

On Fri, Jan 24, 2020 at 10:53:19AM -0600, Frank Sorenson wrote:
> On 1/24/20 4:11 AM, Roberto Bergantinos Corpas wrote:
> > When upcalling gssproxy, cache_head.expiry_time is set as a
> > timeval, not seconds since boot. As such, RPC cache expiry
> > logic will not clean expired objects created under
> > auth.rpcsec.context cache.

Looks like expiration times have worked this way since 2010's
c5b29f885afe "sunrpc: use seconds since boot in expiry cache".
gss_proxy_save_rsc was added in 2012 with 030d794bf498 "SUNRPC: Use
gssproxy upcall for server RPCGSS authentication", so it's the gssproxy
code that introduced the bug.  That's a while for this to lurk, but it
sounds like it required a bit of an extreme case to make it obvious.

Applying with a stable cc, Frank's Tested-by and a note on the above.
Thanks, everyone!

--b.

> > 
> > This has proven to cause kernel memory leaks on field.
> > 
> > Signed-off-by: Roberto Bergantinos Corpas <rbergant@redhat.com>
> > ---
> >  net/sunrpc/auth_gss/svcauth_gss.c | 4 ++++
> >  1 file changed, 4 insertions(+)
> > 
> > diff --git a/net/sunrpc/auth_gss/svcauth_gss.c b/net/sunrpc/auth_gss/svcauth_gss.c
> > index 8be2f209982b..725cf5b5ae40 100644
> > --- a/net/sunrpc/auth_gss/svcauth_gss.c
> > +++ b/net/sunrpc/auth_gss/svcauth_gss.c
> > @@ -1211,6 +1211,7 @@ static int gss_proxy_save_rsc(struct cache_detail *cd,
> >  		dprintk("RPC:       No creds found!\n");
> >  		goto out;
> >  	} else {
> > +		struct timespec boot;
> >  
> >  		/* steal creds */
> >  		rsci.cred = ud->creds;
> > @@ -1231,6 +1232,9 @@ static int gss_proxy_save_rsc(struct cache_detail *cd,
> >  						&expiry, GFP_KERNEL);
> >  		if (status)
> >  			goto out;
> > +
> > +		getboottime(&boot);
> > +		expiry -= boot.tv_sec;
> >  	}
> >  
> >  	rsci.h.expiry_time = expiry;
> > 
> 
> The accumulating  become apparent when the client uses kerberos tickets
> with very short (10 seconds or fewer) lifetimes and renewable lifetimes:
> 
> mount server:/exports /mnt/tmp -overs=4,sec=krb5p
> life="2s"
> rlife="3s"
> while true ; do
> 	while true ; do
> 		kinit -l $life -R >/dev/null 2>&1 && break
> 		echo 'PASSWORD' | kinit -l $life -r $rlife \
> 			>/dev/null 2>&1 && break
> 	done
> 	timeout -k 1 2 touch /mnt/tmp/foo
> 	echo -n .
> done
> 
> Due to the entry expiration occurring 50 years in the future, the
> customer had accumulated in excess of 400,000 entries in the cache over
> about a month with just 6 nfs clients.  The entries, with all the
> accompanying structs which had been allocated consumed over 2 GiB from
> various slab caches.
> 
> A flush of the cache cleans everything out, however they will again
> accumulate afterward.
> 
> This patch fixes the expiration issue.
> 
> Tested-By: Frank Sorenson <sorenson@redhat.com>
> 
> 
> Frank
> --
> Frank Sorenson
> sorenson@redhat.com
> Principal Software Maintenance Engineer
> Global Support Services - filesystems
> Red Hat
> 
> 
> 
> 
> 
> 
> 

  reply	other threads:[~2020-01-24 21:23 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-24 10:11 [PATCH] sunrpc: expiry_time should be seconds not timeval Roberto Bergantinos Corpas
2020-01-24 16:53 ` Frank Sorenson
2020-01-24 21:23   ` J. Bruce Fields [this message]
2020-02-04 10:32 Roberto Bergantinos Corpas

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200124212306.GD26874@fieldses.org \
    --to=bfields@fieldses.org \
    --cc=chuck.lever@oracle.com \
    --cc=linux-nfs@vger.kernel.org \
    --cc=rbergant@redhat.com \
    --cc=sorenson@redhat.com \
    --cc=trond.myklebust@hammerspace.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).