What is a reasonable minimum lease time?

What is a reasonable minimum lease time?

[Frank Filz]

We have an issue with the Ganesha server with very short (2 second) 
lease times. Ganesha uses a 1 second granularity for lease management, 
and considers a time since last renewed equal to the lease time as too 
long. The result is that the lease period may be short close to 2 
seconds depending on when within a given second things actually happened 
(so a last renew at 0.99 with a subsequent renew at 2.01 which is just 
over one second looks like 2 seconds to Ganesha and thus is >= the 2 
second lease time and not good enough. A simple change would be to 
change the >= to a >, which gives one more second, but it still could 
result in the lease time being almost 1 second too short which is 
significant with a 2 second lease time. But if the minimum reasonable 
lease time is more like 5 or 10 seconds, that 1 second becomes less 
significant.

The bigger fix would be to use a finer grained time, but that adds 
complexity, but if people really want to run with 2 second lease times 
and it makes any kind of sense, we would need to make that change.

Thanks


Frank

Re: What is a reasonable minimum lease time?

[Mkrtchyan, Tigran]

Hi Frank,

IMHO, very short leases can trigger a massive state recovery on network hiccups.

Our server offers 90 seconds to the client. The client usually renew lease (sequence) once in a minute.
During high IO periods lease is not required and when idle, then once in a minute is sufficient to
keep the mount alive. In a worst case, when a client got a lock and dispersal, a competing lock/open
will block only for 90s. Whatever number you have, it should be 


BTW, IETF mailing list probably a better place for this question.

Regards,
   Tigran.


----- Original Message -----
> From: "Frank Filz" <ffilz@redhat.com>
> To: "linux-nfs" <linux-nfs@vger.kernel.org>
> Sent: Wednesday, December 12, 2018 10:14:59 PM
> Subject: What is a reasonable minimum lease time?

> We have an issue with the Ganesha server with very short (2 second)
> lease times. Ganesha uses a 1 second granularity for lease management,
> and considers a time since last renewed equal to the lease time as too
> long. The result is that the lease period may be short close to 2
> seconds depending on when within a given second things actually happened
> (so a last renew at 0.99 with a subsequent renew at 2.01 which is just
> over one second looks like 2 seconds to Ganesha and thus is >= the 2
> second lease time and not good enough. A simple change would be to
> change the >= to a >, which gives one more second, but it still could
> result in the lease time being almost 1 second too short which is
> significant with a 2 second lease time. But if the minimum reasonable
> lease time is more like 5 or 10 seconds, that 1 second becomes less
> significant.
> 
> The bigger fix would be to use a finer grained time, but that adds
> complexity, but if people really want to run with 2 second lease times
> and it makes any kind of sense, we would need to make that change.
> 
> Thanks
> 
> 
> Frank

Re: What is a reasonable minimum lease time?

[Trond Myklebust]

On Wed, 2018-12-12 at 23:11 +0100, Mkrtchyan, Tigran wrote:
> Hi Frank,
> 
> IMHO, very short leases can trigger a massive state recovery on
> network hiccups.
> 
> Our server offers 90 seconds to the client. The client usually renew
> lease (sequence) once in a minute.
> During high IO periods lease is not required and when idle, then once
> in a minute is sufficient to
> keep the mount alive. In a worst case, when a client got a lock and
> dispersal, a competing lock/open
> will block only for 90s. Whatever number you have, it should be 
> 

Given that TCP SYN timeouts generally follow an exponential law, and
can easily reach > 20s, then I'd argue a 2s lease is on the extremely
short end of tolerable. You probably want to do something a lot larger
if you want lock recovery after a server crash to be reliable.

> 
> BTW, IETF mailing list probably a better place for this question.
> 
> Regards,
>    Tigran.
> 
> 
> ----- Original Message -----
> > From: "Frank Filz" <ffilz@redhat.com>
> > To: "linux-nfs" <linux-nfs@vger.kernel.org>
> > Sent: Wednesday, December 12, 2018 10:14:59 PM
> > Subject: What is a reasonable minimum lease time?
> > We have an issue with the Ganesha server with very short (2 second)
> > lease times. Ganesha uses a 1 second granularity for lease
> > management,
> > and considers a time since last renewed equal to the lease time as
> > too
> > long. The result is that the lease period may be short close to 2
> > seconds depending on when within a given second things actually
> > happened
> > (so a last renew at 0.99 with a subsequent renew at 2.01 which is
> > just
> > over one second looks like 2 seconds to Ganesha and thus is >= the
> > 2
> > second lease time and not good enough. A simple change would be to
> > change the >= to a >, which gives one more second, but it still
> > could
> > result in the lease time being almost 1 second too short which is
> > significant with a 2 second lease time. But if the minimum
> > reasonable
> > lease time is more like 5 or 10 seconds, that 1 second becomes less
> > significant.
> > 
> > The bigger fix would be to use a finer grained time, but that adds
> > complexity, but if people really want to run with 2 second lease
> > times
> > and it makes any kind of sense, we would need to make that change.
> > 
> > Thanks
> > 
> > 
> > Frank
-- 
Trond Myklebust
CTO, Hammerspace Inc
4300 El Camino Real, Suite 105
Los Altos, CA 94022
www.hammer.space