[PATCH] Update atime from future.

[PATCH] Update atime from future.

[yangsheng]

Relatime should update the inode atime if it is more than a day in the
future.  The original problem seen was a tarball that had a bad atime,
but could also happen if someone fat-fingers a "touch".  The future
atime will never be fixed.  Before the relatime patch, the future atime
would be updated back to the current time on the next access.

Only update the atime if it is more than one day in the future.  That
avoids thrashing the atime if the clocks on clients of a network fs are
only slightly out of sync, but still allows fixing bogus atimes.

Signed-off-by: yangsheng <sickamd@gmail.com>
Reviewed-by: adilger@dilger.ca
---
 fs/inode.c | 10 +++++++++-
 1 files changed, 9 insertions(+), 1 deletions(-)

diff --git a/fs/inode.c b/fs/inode.c
index 14084b7..5c4379a 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -1465,6 +1465,7 @@ sector_t bmap(struct inode *inode, sector_t block)
 }
 EXPORT_SYMBOL(bmap);
 
+#define RELATIME_MARGIN (24*60*60)
 /*
  * With relative atime, only update atime if the previous atime is
  * earlier than either the ctime or mtime or if at least a day has
@@ -1488,10 +1489,17 @@ static int relatime_need_update(struct vfsmount *mnt, struct inode *inode,
 		return 1;
 
 	/*
+	 * Is the previous atime value in future? If yes,
+	 * update atime:
+	 */
+	if ((long)(now.tv_sec - inode->i_atime.tv_sec) < -RELATIME_MARGIN)
+		return 1;
+
+	/*
 	 * Is the previous atime value older than a day? If yes,
 	 * update atime:
 	 */
-	if ((long)(now.tv_sec - inode->i_atime.tv_sec) >= 24*60*60)
+	if ((long)(now.tv_sec - inode->i_atime.tv_sec) >= RELATIME_MARGIN)
 		return 1;
 	/*
 	 * Good, we can skip the atime update:
-- 
1.7.11.7

Re: [PATCH] Update atime from future.

[Greg KH]

On Tue, Dec 04, 2012 at 01:56:39AM +0800, yangsheng wrote:
> Relatime should update the inode atime if it is more than a day in the
> future.  The original problem seen was a tarball that had a bad atime,
> but could also happen if someone fat-fingers a "touch".  The future
> atime will never be fixed.  Before the relatime patch, the future atime
> would be updated back to the current time on the next access.
> 
> Only update the atime if it is more than one day in the future.  That
> avoids thrashing the atime if the clocks on clients of a network fs are
> only slightly out of sync, but still allows fixing bogus atimes.
> 
> Signed-off-by: yangsheng <sickamd@gmail.com>
> Reviewed-by: adilger@dilger.ca
> ---
>  fs/inode.c | 10 +++++++++-
>  1 files changed, 9 insertions(+), 1 deletions(-)


<formletter>

This is not the correct way to submit patches for inclusion in the
stable kernel tree.  Please read Documentation/stable_kernel_rules.txt
for how to do this properly.

</formletter>

Re: [PATCH] Update atime from future.

[Zach Brown]

On Tue, Dec 04, 2012 at 01:56:39AM +0800, yangsheng wrote:
> Relatime should update the inode atime if it is more than a day in the
> future.  The original problem seen was a tarball that had a bad atime,
> but could also happen if someone fat-fingers a "touch".  The future
> atime will never be fixed.  Before the relatime patch, the future atime
> would be updated back to the current time on the next access.

I guess.

>  	/*
> +	 * Is the previous atime value in future? If yes,
> +	 * update atime:
> +	 */
> +	if ((long)(now.tv_sec - inode->i_atime.tv_sec) < -RELATIME_MARGIN)
> +		return 1;

But this is confusing to read.  "If atime is less than a negative day in
the past.. wait, what?"

It seems like we should combine the two RELATIME_MARGIN tests.

	/*
	 * Update atime if it's older than a day or more than a day
	 * in the future, which we assume is corrupt.
	 */
	if (abs(inode->i_atime.tv_sec - now.tv_sec)) >= RELATIME_MARGIN)
		return 1;

(I don't know if you'd still need the (long) cast in there, given the
type tests in abs()).

- z

Re: [PATCH] Update atime from future.

[Dave Chinner]

On Tue, Dec 04, 2012 at 01:56:39AM +0800, yangsheng wrote:
> Relatime should update the inode atime if it is more than a day in the
> future.  The original problem seen was a tarball that had a bad atime,
> but could also happen if someone fat-fingers a "touch".  The future
> atime will never be fixed.  Before the relatime patch, the future atime
> would be updated back to the current time on the next access.

So if someone accidentally changes time back a few days, access
times go backwards for everything? That doesn't sound right to me -
it's going to seriously screw up backups and other scanners that use
atime to determine "newly accessed files"....

IMO, if you fat-finger a manual atime update or use atimes direct
from tarballs, then that's your problem as a user and not the
responsibility of the kernel to magically fix for you....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH] Update atime from future.

[Andreas Dilger]

On 2012-12-04, at 13:24, Dave Chinner <david@fromorbit.com> wrote:
> On Tue, Dec 04, 2012 at 01:56:39AM +0800, yangsheng wrote:
>> Relatime should update the inode atime if it is more than a day in the
>> future.  The original problem seen was a tarball that had a bad atime,
>> but could also happen if someone fat-fingers a "touch".  The future
>> atime will never be fixed.  Before the relatime patch, the future atime
>> would be updated back to the current time on the next access.
> 
> So if someone accidentally changes time back a few days, access
> times go backwards for everything?

And they will go forward again if the files are accessed again in the future. 

> That doesn't sound right to me -
> it's going to seriously screw up backups and other scanners that use
> atime to determine "newly accessed files"....

I think it is equally as easy to say that if someone accidentally sets the time too far in the future (e.g. 2102 instead of 2012) for a while, then the atimes will be even more broken since the kernel will never update them again even after the clock is fixed. 

The point is that the behaviour before the relatime patch was that the kernel updated the atime to the current time as the kernel knows about it, it didn't make any decision about "the past" or "the future".

Relatime is about reducing the frequency of atime updates, not about deciding that one timestamp is more correct than another. 

I'd be ok to change the margin for what constitutes a "future" time (a few days or a week?), but if atime updates happen once a day for times in the past  I don't think it is incorrect to update the atime if it is more than a day in the future. 

Cheers, Andreas

> IMO, if you fat-finger a manual atime update or use atimes direct
> from tarballs, then that's your problem as a user and not the
> responsibility of the kernel to magically fix for you....
> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

Re: [PATCH] Update atime from future.

[Dave Chinner]

On Tue, Dec 04, 2012 at 05:22:32PM -0700, Andreas Dilger wrote:
> The point is that the behaviour before the relatime patch was that
> the kernel updated the atime to the current time as the kernel
> knows about it, it didn't make any decision about "the past" or
> "the future".
> 
> Relatime is about reducing the frequency of atime updates, not
> about deciding that one timestamp is more correct than another. 

That makes sense.

Indeed, that's what the commit message should say rather than
drawing arbitrary lines in the sand about what is a valid atime
without further justification.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

[PATCH] Update atime from future.

[yangsheng]

If atime has been wrong set to future, then it cannot
be updated back to current time.

Signed-off-by: sickamd@gmail.com
Reviewed-by: adilger@dilger.ca
---
 fs/inode.c |   10 +++++++++-
 1 files changed, 9 insertions(+), 1 deletions(-)

diff --git a/fs/inode.c b/fs/inode.c
index da85e56..d92779f 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -1446,6 +1446,8 @@ sector_t bmap(struct inode *inode, sector_t block)
 }
 EXPORT_SYMBOL(bmap);
 
+#define RELATIME_MARGIN (24 * 60 * 60)
+
 /*
  * With relative atime, only update atime if the previous atime is
  * earlier than either the ctime or mtime or if at least a day has
@@ -1469,10 +1471,16 @@ static int relatime_need_update(struct vfsmount *mnt, struct inode *inode,
 		return 1;
 
 	/*
+	 * Is the previous atime value in future? If yes,
+	 * update atime:
+	 */
+	if ((long)(now.tv_sec - inode->i_atime.tv_sec) < -RELATIME_MARGIN)
+		return 1;
+	/*
 	 * Is the previous atime value older than a day? If yes,
 	 * update atime:
 	 */
-	if ((long)(now.tv_sec - inode->i_atime.tv_sec) >= 24*60*60)
+	if ((long)(now.tv_sec - inode->i_atime.tv_sec) >= RELATIME_MARGIN)
 		return 1;
 	/*
 	 * Good, we can skip the atime update:
-- 
1.7.2.3

[PATCH] Update atime from future.

[yangsheng]

Signed-off-by: sickamd@gmail.com
---
 fs/inode.c |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)

diff --git a/fs/inode.c b/fs/inode.c
index da85e56..6c8effd 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -1469,7 +1469,13 @@ static int relatime_need_update(struct vfsmount *mnt, struct inode *inode,
 		return 1;
 
 	/*
-	 * Is the previous atime value older than a day? If yes,
+	 * Is the previous atime value in future? If yes,
+	 * update atime:
+	 */
+	if ((long)(now.tv_sec - inode->i_atime.tv_sec) < 0)
+		return 1;
+	/*
+	 * Is the previous atime value old than a day? If yes,
 	 * update atime:
 	 */
 	if ((long)(now.tv_sec - inode->i_atime.tv_sec) >= 24*60*60)
-- 
1.7.2.3

Re: [PATCH] Update atime from future.

[Andrew Morton]

On Wed, 29 Dec 2010 21:58:41 +0800 yangsheng <sickamd@gmail.com> wrote:

> Signed-off-by: sickamd@gmail.com
> ---
>  fs/inode.c |    8 +++++++-
>  1 files changed, 7 insertions(+), 1 deletions(-)
> 
> diff --git a/fs/inode.c b/fs/inode.c
> index da85e56..6c8effd 100644
> --- a/fs/inode.c
> +++ b/fs/inode.c
> @@ -1469,7 +1469,13 @@ static int relatime_need_update(struct vfsmount *mnt, struct inode *inode,
>  		return 1;
>  
>  	/*
> -	 * Is the previous atime value older than a day? If yes,
> +	 * Is the previous atime value in future? If yes,
> +	 * update atime:
> +	 */
> +	if ((long)(now.tv_sec - inode->i_atime.tv_sec) < 0)
> +		return 1;
> +	/*
> +	 * Is the previous atime value old than a day? If yes,
>  	 * update atime:
>  	 */
>  	if ((long)(now.tv_sec - inode->i_atime.tv_sec) >= 24*60*60)

Why do you believe this change is needed?  Did you observe some problem
which it fixes?  If so, please fully describe that problem.

Re: [PATCH] Update atime from future.

[YangSheng]

On 01/03/2011 06:17 PM, Andrew Morton wrote:
> On Wed, 29 Dec 2010 21:58:41 +0800 yangsheng<sickamd@gmail.com>  wrote:
>
>    
>> Signed-off-by: sickamd@gmail.com
>> ---
>>   fs/inode.c |    8 +++++++-
>>   1 files changed, 7 insertions(+), 1 deletions(-)
>>
>> diff --git a/fs/inode.c b/fs/inode.c
>> index da85e56..6c8effd 100644
>> --- a/fs/inode.c
>> +++ b/fs/inode.c
>> @@ -1469,7 +1469,13 @@ static int relatime_need_update(struct vfsmount *mnt, struct inode *inode,
>>   		return 1;
>>
>>   	/*
>> -	 * Is the previous atime value older than a day? If yes,
>> +	 * Is the previous atime value in future? If yes,
>> +	 * update atime:
>> +	 */
>> +	if ((long)(now.tv_sec - inode->i_atime.tv_sec)<  0)
>> +		return 1;
>> +	/*
>> +	 * Is the previous atime value old than a day? If yes,
>>   	 * update atime:
>>   	 */
>>   	if ((long)(now.tv_sec - inode->i_atime.tv_sec)>= 24*60*60)
>>      
> Why do you believe this change is needed?  Did you observe some problem
> which it fixes?  If so, please fully describe that problem.
>    
If atime has been set to future(maybe cause by some accident system time 
adjust or wrong set by touch). It cannot be update to reflect fact 
access time before system time running over one day.

Thanks

Re: [PATCH] Update atime from future.

[Rogier Wolff]

On Mon, Jan 03, 2011 at 02:17:08AM -0800, Andrew Morton wrote:
> On Wed, 29 Dec 2010 21:58:41 +0800 yangsheng <sickamd@gmail.com> wrote:
> > -	 * Is the previous atime value older than a day? If yes,
> > +	 * Is the previous atime value old than a day? If yes,

You introduced a typo. 

	Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
**    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. 
Does it sit on the couch all day? Is it unemployed? Please be specific! 
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ

Re: [PATCH] Update atime from future.

[Steven Whitehouse]

Hi,

On Wed, 2010-12-29 at 21:58 +0800, yangsheng wrote:
> Signed-off-by: sickamd@gmail.com
> ---
>  fs/inode.c |    8 +++++++-
>  1 files changed, 7 insertions(+), 1 deletions(-)
> 
> diff --git a/fs/inode.c b/fs/inode.c
> index da85e56..6c8effd 100644
> --- a/fs/inode.c
> +++ b/fs/inode.c
> @@ -1469,7 +1469,13 @@ static int relatime_need_update(struct vfsmount *mnt, struct inode *inode,
>  		return 1;
>  
>  	/*
> -	 * Is the previous atime value older than a day? If yes,
> +	 * Is the previous atime value in future? If yes,
> +	 * update atime:
> +	 */
> +	if ((long)(now.tv_sec - inode->i_atime.tv_sec) < 0)
> +		return 1;
> +	/*
> +	 * Is the previous atime value old than a day? If yes,
>  	 * update atime:
>  	 */
>  	if ((long)(now.tv_sec - inode->i_atime.tv_sec) >= 24*60*60)

I don't think this is a good plan for cluster filesystems, since if the
times on the nodes are not exactly synchronised (we do highly recommend
people run ntp or similar) then this might lead to excessive atime
updating. The current behaviour is to ignore atimes which are in the
future for exactly this reason,

Steve.

Re: [PATCH] Update atime from future.

[Andreas Dilger]

On 2011-01-03, at 3:27, Steven Whitehouse <swhiteho@redhat.com> wrote:
> On Wed, 2010-12-29 at 21:58 +0800, yangsheng wrote:
>> Signed-off-by: sickamd@gmail.com
>> ---
>> fs/inode.c |    8 +++++++-
>> 1 files changed, 7 insertions(+), 1 deletions(-)
>> 
>> diff --git a/fs/inode.c b/fs/inode.c
>> index da85e56..6c8effd 100644
>> --- a/fs/inode.c
>> +++ b/fs/inode.c
>> @@ -1469,7 +1469,13 @@ static int relatime_need_update(struct vfsmount *mnt, struct inode *inode,
>>        return 1;
>> 
>>    /*
>> -     * Is the previous atime value older than a day? If yes,
>> +     * Is the previous atime value in future? If yes,
>> +     * update atime:
>> +     */
>> +    if ((long)(now.tv_sec - inode->i_atime.tv_sec) < 0)
>> +        return 1;
>> +    /*
>> +     * Is the previous atime value old than a day? If yes,
>>     * update atime:
>>     */
>>    if ((long)(now.tv_sec - inode->i_atime.tv_sec) >= 24*60*60)
> 
> I don't think this is a good plan for cluster filesystems, since if the
> times on the nodes are not exactly synchronised (we do highly recommend
> people run ntp or similar) then this might lead to excessive atime
> updating. The current behaviour is to ignore atimes which are in the
> future for exactly this reason,

The problem that is seen is if a tarball has stored a bad atime, or someone fat-fingers a "touch" then the future atime will never be fixed. Before the relatime patch, the future atime would be updated back to the current time on the next access. One if our regression tests for Lustre caught this. 

I wouldn't mind changing the relatime check so that it only updates the atime if it is more than one day in the future. That will avoid thrashing atime if the clocks are only slightly out of sync, but still allow fixing completely bogus atimes. 

Cheers, Andreas

Re: [PATCH] Update atime from future.

[Steven Whitehouse]

Hi,

On Mon, 2011-01-03 at 09:27 -0700, Andreas Dilger wrote:
> On 2011-01-03, at 3:27, Steven Whitehouse <swhiteho@redhat.com> wrote:
> > On Wed, 2010-12-29 at 21:58 +0800, yangsheng wrote:
> >> Signed-off-by: sickamd@gmail.com
> >> ---
> >> fs/inode.c |    8 +++++++-
> >> 1 files changed, 7 insertions(+), 1 deletions(-)
> >> 
> >> diff --git a/fs/inode.c b/fs/inode.c
> >> index da85e56..6c8effd 100644
> >> --- a/fs/inode.c
> >> +++ b/fs/inode.c
> >> @@ -1469,7 +1469,13 @@ static int relatime_need_update(struct vfsmount *mnt, struct inode *inode,
> >>        return 1;
> >> 
> >>    /*
> >> -     * Is the previous atime value older than a day? If yes,
> >> +     * Is the previous atime value in future? If yes,
> >> +     * update atime:
> >> +     */
> >> +    if ((long)(now.tv_sec - inode->i_atime.tv_sec) < 0)
> >> +        return 1;
> >> +    /*
> >> +     * Is the previous atime value old than a day? If yes,
> >>     * update atime:
> >>     */
> >>    if ((long)(now.tv_sec - inode->i_atime.tv_sec) >= 24*60*60)
> > 
> > I don't think this is a good plan for cluster filesystems, since if the
> > times on the nodes are not exactly synchronised (we do highly recommend
> > people run ntp or similar) then this might lead to excessive atime
> > updating. The current behaviour is to ignore atimes which are in the
> > future for exactly this reason,
> 
> The problem that is seen is if a tarball has stored a bad atime, or someone fat-fingers a "touch" then the future atime will never be fixed. Before the relatime patch, the future atime would be updated back to the current time on the next access. One if our regression tests for Lustre caught this. 
> 
> I wouldn't mind changing the relatime check so that it only updates the atime if it is more than one day in the future. That will avoid thrashing atime if the clocks are only slightly out of sync, but still allow fixing completely bogus atimes. 
> 
> Cheers, Andreas

That sounds like a good solution to me,

Steve.

Re: [PATCH] Update atime from future.

[YangSheng]

On 01/03/2011 06:27 PM, Steven Whitehouse wrote:
> Hi,
>
> On Wed, 2010-12-29 at 21:58 +0800, yangsheng wrote:
>    
>> Signed-off-by: sickamd@gmail.com
>> ---
>>   fs/inode.c |    8 +++++++-
>>   1 files changed, 7 insertions(+), 1 deletions(-)
>>
>> diff --git a/fs/inode.c b/fs/inode.c
>> index da85e56..6c8effd 100644
>> --- a/fs/inode.c
>> +++ b/fs/inode.c
>> @@ -1469,7 +1469,13 @@ static int relatime_need_update(struct vfsmount *mnt, struct inode *inode,
>>   		return 1;
>>
>>   	/*
>> -	 * Is the previous atime value older than a day? If yes,
>> +	 * Is the previous atime value in future? If yes,
>> +	 * update atime:
>> +	 */
>> +	if ((long)(now.tv_sec - inode->i_atime.tv_sec)<  0)
>> +		return 1;
>> +	/*
>> +	 * Is the previous atime value old than a day? If yes,
>>   	 * update atime:
>>   	 */
>>   	if ((long)(now.tv_sec - inode->i_atime.tv_sec)>= 24*60*60)
>>      
> I don't think this is a good plan for cluster filesystems, since if the
> times on the nodes are not exactly synchronised (we do highly recommend
> people run ntp or similar) then this might lead to excessive atime
> updating. The current behaviour is to ignore atimes which are in the
> future for exactly this reason,
>    
I agreed in theory.  Anyway, a two-way update may cause shake in some 
case. Like a cluster environment with time gap between cluster members. 
But future atime also is a trouble things i think. Of course, I hope a 
clever patch to fix them all.

Thanks
yangsheng

Re: [PATCH] Update atime from future.

[Pavel Machek]

	return 1;
> > +	/*
> > +	 * Is the previous atime value old than a day? If yes,
> >  	 * update atime:
> >  	 */
> >  	if ((long)(now.tv_sec - inode->i_atime.tv_sec) >= 24*60*60)
> 
> I don't think this is a good plan for cluster filesystems, since if the
> times on the nodes are not exactly synchronised (we do highly recommend
> people run ntp or similar) then this might lead to excessive atime
> updating. The current behaviour is to ignore atimes which are in the
> future for exactly this reason,

Well, would these "update storms" really be a problem?

AFAICT they should be fairly non-frequent, and worst thing that can
happen is that you'll do as many updates as different time settings,
settling for the lowest value...?
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [PATCH] Update atime from future.

[Steven Whitehouse]

Hi,

On Tue, 2011-01-11 at 14:33 +0100, Pavel Machek wrote:
> 	return 1;
> > > +	/*
> > > +	 * Is the previous atime value old than a day? If yes,
> > >  	 * update atime:
> > >  	 */
> > >  	if ((long)(now.tv_sec - inode->i_atime.tv_sec) >= 24*60*60)
> > 
> > I don't think this is a good plan for cluster filesystems, since if the
> > times on the nodes are not exactly synchronised (we do highly recommend
> > people run ntp or similar) then this might lead to excessive atime
> > updating. The current behaviour is to ignore atimes which are in the
> > future for exactly this reason,
> 
> Well, would these "update storms" really be a problem?
> 
> AFAICT they should be fairly non-frequent, and worst thing that can
> happen is that you'll do as many updates as different time settings,
> settling for the lowest value...?
> 									Pavel

Sorry for the delay in replying. It has been a problem in the past,
certainly. I think it is best to be cautious in this case, since that
way we can be sure it won't be a problem. The chosen solution looks ok
to me,

Steve.