ReiserFS patch for updating ctimes of renamed files

ReiserFS patch for updating ctimes of renamed files

[Alex Adriaanse]

Hi,

I ran into some trouble trying to do incremental backups with GNU tar
(using --listed-incremental) where renaming a file in between backups would
cause the file to disappear upon restoration.  When investigating the issue
I discovered that this doesn't happen on ext2, ext3, and tmpfs filesystems
but only on ReiserFS filesystems.  I also noticed that for example ext3
updates the affected file's ctime upon rename whereas ReiserFS doesn't, so
I'm thinking this causes tar to believe that the file existed before the
first backup was taking under the new name, and as a result it doesn't back
it up during the second backup.  So I believe ReiserFS needs to update
ctimes for renamed files in order for incremental GNU tar backups to work
reliably.

I made some changes to the reiserfs_rename function that I *think* should
fix the problem.  However, I don't know much about ReiserFS's internals, and
I haven't been able to test them out to see if things work now since I can't
afford to deal with potential FS corruption with my current Linux box.

I included a patch below against the 2.4.22 kernel with my changes.  Would
somebody mind taking a look at this to see if I did things right here (and
perhaps wouldn't mind testing it out either)?  If it works then I (and I'm
sure others who've experienced the same problem) would like to see the
changes applied to the next 2.4.x (and 2.6.x?) release.

Thanks a lot.

Alex

--- fs/reiserfs/namei.c.orig    Mon Aug 25 06:44:43 2003
+++ fs/reiserfs/namei.c Sun Oct 12 00:39:05 2003
@@ -1207,6 +1207,8 @@
     journal_mark_dirty (&th, old_dir->i_sb, old_de.de_bh);
     old_dir->i_ctime = old_dir->i_mtime = CURRENT_TIME;
     new_dir->i_ctime = new_dir->i_mtime = CURRENT_TIME;
+    old_inode->i_ctime = CURRENT_TIME;
+    reiserfs_update_sd (&th, old_inode);

     if (new_dentry_inode) {
        // adjust link number of the victim

Re: ReiserFS patch for updating ctimes of renamed files

[jw schultz]

On Sun, Oct 12, 2003 at 01:05:19AM -0500, Alex Adriaanse wrote:
> Hi,
> 
> I ran into some trouble trying to do incremental backups with GNU tar
> (using --listed-incremental) where renaming a file in between backups would
> cause the file to disappear upon restoration.  When investigating the issue
> I discovered that this doesn't happen on ext2, ext3, and tmpfs filesystems
> but only on ReiserFS filesystems.  I also noticed that for example ext3
> updates the affected file's ctime upon rename whereas ReiserFS doesn't, so
> I'm thinking this causes tar to believe that the file existed before the
> first backup was taking under the new name, and as a result it doesn't back
> it up during the second backup.  So I believe ReiserFS needs to update
> ctimes for renamed files in order for incremental GNU tar backups to work
> reliably.
> 
> I made some changes to the reiserfs_rename function that I *think* should
> fix the problem.  However, I don't know much about ReiserFS's internals, and
> I haven't been able to test them out to see if things work now since I can't
> afford to deal with potential FS corruption with my current Linux box.
> 
> I included a patch below against the 2.4.22 kernel with my changes.  Would
> somebody mind taking a look at this to see if I did things right here (and
> perhaps wouldn't mind testing it out either)?  If it works then I (and I'm
> sure others who've experienced the same problem) would like to see the
> changes applied to the next 2.4.x (and 2.6.x?) release.

Hmm.  I'm conflicted.

rename(2) manpage:
	Any other hard links to the file (as created using
	link(2)) are unaffected.

A change to ctime would affect the other links.

stat(2) manpage:
	The field st_ctime is changed by writing or by
	setting inode information (i.e., owner, group, link
	count, mode, etc.).

I am not aware of any field in the inode structure that must
be changed by an atomic rename.  Per documentation the only
reason rename should update st_ctime is if it does a
link+unlink sequence which would alter st_nlink briefly.

On the other hand it does seem to me there ought to be some
record that something about the inode changed.  st_ctime would
be the only appropriate indicator.

rename() SUSv3:
	Some implementations mark for update the st_ctime
	field of renamed files and some do not. Applications
	which make use of the st_ctime field may behave
	differently with respect to renamed files unless
	they are designed to allow for either behavior.

So reiserfs is on this point definitely standards conformant
already.  A change could at best be seen as an enhancement.




-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt

Re: ReiserFS patch for updating ctimes of renamed files

[Hans Reiser]

Vladimir will look at this and get back to you.  Thanks kindly for this.

Hans

Alex Adriaanse wrote:

>Hi,
>
>I ran into some trouble trying to do incremental backups with GNU tar
>(using --listed-incremental) where renaming a file in between backups would
>cause the file to disappear upon restoration.  When investigating the issue
>I discovered that this doesn't happen on ext2, ext3, and tmpfs filesystems
>but only on ReiserFS filesystems.  I also noticed that for example ext3
>updates the affected file's ctime upon rename whereas ReiserFS doesn't, so
>I'm thinking this causes tar to believe that the file existed before the
>first backup was taking under the new name, and as a result it doesn't back
>it up during the second backup.  So I believe ReiserFS needs to update
>ctimes for renamed files in order for incremental GNU tar backups to work
>reliably.
>
>I made some changes to the reiserfs_rename function that I *think* should
>fix the problem.  However, I don't know much about ReiserFS's internals, and
>I haven't been able to test them out to see if things work now since I can't
>afford to deal with potential FS corruption with my current Linux box.
>
>I included a patch below against the 2.4.22 kernel with my changes.  Would
>somebody mind taking a look at this to see if I did things right here (and
>perhaps wouldn't mind testing it out either)?  If it works then I (and I'm
>sure others who've experienced the same problem) would like to see the
>changes applied to the next 2.4.x (and 2.6.x?) release.
>
>Thanks a lot.
>
>Alex
>
>--- fs/reiserfs/namei.c.orig    Mon Aug 25 06:44:43 2003
>+++ fs/reiserfs/namei.c Sun Oct 12 00:39:05 2003
>@@ -1207,6 +1207,8 @@
>     journal_mark_dirty (&th, old_dir->i_sb, old_de.de_bh);
>     old_dir->i_ctime = old_dir->i_mtime = CURRENT_TIME;
>     new_dir->i_ctime = new_dir->i_mtime = CURRENT_TIME;
>+    old_inode->i_ctime = CURRENT_TIME;
>+    reiserfs_update_sd (&th, old_inode);
>
>     if (new_dentry_inode) {
>        // adjust link number of the victim
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>
>  
>


-- 
Hans

Re: ReiserFS patch for updating ctimes of renamed files

[Hans Reiser]

jw schultz wrote:

>On Sun, Oct 12, 2003 at 01:05:19AM -0500, Alex Adriaanse wrote:
>  
>
>>Hi,
>>
>>I ran into some trouble trying to do incremental backups with GNU tar
>>(using --listed-incremental) where renaming a file in between backups would
>>cause the file to disappear upon restoration.  When investigating the issue
>>I discovered that this doesn't happen on ext2, ext3, and tmpfs filesystems
>>but only on ReiserFS filesystems.  I also noticed that for example ext3
>>updates the affected file's ctime upon rename whereas ReiserFS doesn't, so
>>I'm thinking this causes tar to believe that the file existed before the
>>first backup was taking under the new name, and as a result it doesn't back
>>it up during the second backup.  So I believe ReiserFS needs to update
>>ctimes for renamed files in order for incremental GNU tar backups to work
>>reliably.
>>
>>I made some changes to the reiserfs_rename function that I *think* should
>>fix the problem.  However, I don't know much about ReiserFS's internals, and
>>I haven't been able to test them out to see if things work now since I can't
>>afford to deal with potential FS corruption with my current Linux box.
>>
>>I included a patch below against the 2.4.22 kernel with my changes.  Would
>>somebody mind taking a look at this to see if I did things right here (and
>>perhaps wouldn't mind testing it out either)?  If it works then I (and I'm
>>sure others who've experienced the same problem) would like to see the
>>changes applied to the next 2.4.x (and 2.6.x?) release.
>>    
>>
>
>Hmm.  I'm conflicted.
>
>rename(2) manpage:
>	Any other hard links to the file (as created using
>	link(2)) are unaffected.
>
>A change to ctime would affect the other links.
>
>stat(2) manpage:
>	The field st_ctime is changed by writing or by
>	setting inode information (i.e., owner, group, link
>	count, mode, etc.).
>
>I am not aware of any field in the inode structure that must
>be changed by an atomic rename.  Per documentation the only
>reason rename should update st_ctime is if it does a
>link+unlink sequence which would alter st_nlink briefly.
>
>On the other hand it does seem to me there ought to be some
>record that something about the inode changed.  st_ctime would
>be the only appropriate indicator.
>
>rename() SUSv3:
>	Some implementations mark for update the st_ctime
>	field of renamed files and some do not. Applications
>	which make use of the st_ctime field may behave
>	differently with respect to renamed files unless
>	they are designed to allow for either behavior.
>
>So reiserfs is on this point definitely standards conformant
>already.  A change could at best be seen as an enhancement.
>
>
>
>
>  
>
thanks Mr. Schultz, you saved us a lot of work in reviewing this issue.

In theory it is cleaner and purer to do it the way we did.  In practice, 
Alex's problem seems like a real one, and I don't know how hard it is to 
change tar to do the right thing.  We'll discuss it in a small seminar 
today.

-- 
Hans

Re: ReiserFS patch for updating ctimes of renamed files

[Hans Reiser]

Alex, are you convinced by jw?  (I think I am.)  Would you be willing to 
submit a patch for tar instead?

Hans

jw schultz wrote:

>On Mon, Oct 13, 2003 at 09:49:20AM +0400, Hans Reiser wrote:
>  
>
> In theory it is cleaner and purer to do it the way we did. In practice,
>
>>Alex's problem seems like a real one, and I don't know how hard it is to 
>>change tar to do the right thing.  We'll discuss it in a small seminar 
>>today.
>>    
>>
>
>Updating ctime does seem messy and a bit irrelevant for the
>atomic rename.  You are modifying the directories not the
>fricken file. This isn't DOS!  But it would seem he does
>indeed have an issue although i'm not sure what.  I've never
>used the listed-incremental option of tar and since the
>manpage is incomplete <rant deleted> i don't know what it
>actually does.  However, i have found the use of ctime to be
>terribly unreliable for file management and given what the
>standards have to say on the issue it sounds like tar is
>being abused or has a bug.
>
>
>  
>


-- 
Hans

Re: ReiserFS patch for updating ctimes of renamed files

[Andrew Morton]

Hans Reiser <reiser@namesys.com> wrote:
>
> In theory it is cleaner and purer to do it the way we did.  In practice, 
>  Alex's problem seems like a real one, and I don't know how hard it is to 
>  change tar to do the right thing.  We'll discuss it in a small seminar 
>  today.

It would be best to make this change.  minix, ext2 and ext3 do set ctime,
so it is "the Linux standard".

btw, this code:

      old_dir->i_ctime = old_dir->i_mtime = CURRENT_TIME;
      new_dir->i_ctime = new_dir->i_mtime = CURRENT_TIME;
      old_inode->i_ctime = CURRENT_TIME;

should avoid evaluating CURRENT_TIME three times: is has some computational
cost and if an interrupt happens at the wrong time you end up with
differing times in the inode(s).

RE: ReiserFS patch for updating ctimes of renamed files

[Alex Adriaanse]

Hans,

Yes, I agree with J.W.  However, I also think that Andrew has a good point
in that the behavior across Linux filesystems (ReiserFS, ext2, ext3, minix,
etc.) should be consistent.  Either they should all update ctime during
renames, or none of them should.

Anyway, I'll try to work with the GNU tar maintainer to get this problem in
tar fixed.  It'll probably be a lot harder to fix in tar than to have
ReiserFS update ctimes since it'll require major changes in
the --listed-incremental snapshot files.  However, if you don't think it's a
good idea to make these changes to ReiserFS then we'll just work on fixing
up tar.

Thanks,

Alex

-----Original Message-----
From: linux-kernel-owner@vger.kernel.org
[mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Hans Reiser
Sent: Monday, October 13, 2003 3:46 AM
To: Alex Adriaanse
Cc: jw schultz; Linux Kernel Mailing List; vs@namesys.com
Subject: Re: ReiserFS patch for updating ctimes of renamed files


Alex, are you convinced by jw?  (I think I am.)  Would you be willing to
submit a patch for tar instead?

Hans

jw schultz wrote:

>On Mon, Oct 13, 2003 at 09:49:20AM +0400, Hans Reiser wrote:
>
>
> In theory it is cleaner and purer to do it the way we did. In practice,
>
>>Alex's problem seems like a real one, and I don't know how hard it is to
>>change tar to do the right thing.  We'll discuss it in a small seminar
>>today.
>>
>>
>
>Updating ctime does seem messy and a bit irrelevant for the
>atomic rename.  You are modifying the directories not the
>fricken file. This isn't DOS!  But it would seem he does
>indeed have an issue although i'm not sure what.  I've never
>used the listed-incremental option of tar and since the
>manpage is incomplete <rant deleted> i don't know what it
>actually does.  However, i have found the use of ctime to be
>terribly unreliable for file management and given what the
>standards have to say on the issue it sounds like tar is
>being abused or has a bug.
>
>
>
>


--
Hans


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: ReiserFS patch for updating ctimes of renamed files

[Hans Reiser]

Let's see what Andrew says after he reads J.W.'s reasoning.  I agree 
that reiserfs should do the same thing as the other filesystems in 
Linux, but J.W. seems to be right that they are doing the wrong thing.

Hans

Alex Adriaanse wrote:

>Hans,
>
>Yes, I agree with J.W.  However, I also think that Andrew has a good point
>in that the behavior across Linux filesystems (ReiserFS, ext2, ext3, minix,
>etc.) should be consistent.  Either they should all update ctime during
>renames, or none of them should.
>
>Anyway, I'll try to work with the GNU tar maintainer to get this problem in
>tar fixed.  It'll probably be a lot harder to fix in tar than to have
>ReiserFS update ctimes since it'll require major changes in
>the --listed-incremental snapshot files.  However, if you don't think it's a
>good idea to make these changes to ReiserFS then we'll just work on fixing
>up tar.
>
>Thanks,
>
>Alex
>
>-----Original Message-----
>From: linux-kernel-owner@vger.kernel.org
>[mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Hans Reiser
>Sent: Monday, October 13, 2003 3:46 AM
>To: Alex Adriaanse
>Cc: jw schultz; Linux Kernel Mailing List; vs@namesys.com
>Subject: Re: ReiserFS patch for updating ctimes of renamed files
>
>
>Alex, are you convinced by jw?  (I think I am.)  Would you be willing to
>submit a patch for tar instead?
>
>Hans
>
>jw schultz wrote:
>
>  
>
>>On Mon, Oct 13, 2003 at 09:49:20AM +0400, Hans Reiser wrote:
>>
>>
>>In theory it is cleaner and purer to do it the way we did. In practice,
>>
>>    
>>
>>>Alex's problem seems like a real one, and I don't know how hard it is to
>>>change tar to do the right thing.  We'll discuss it in a small seminar
>>>today.
>>>
>>>
>>>      
>>>
>>Updating ctime does seem messy and a bit irrelevant for the
>>atomic rename.  You are modifying the directories not the
>>fricken file. This isn't DOS!  But it would seem he does
>>indeed have an issue although i'm not sure what.  I've never
>>used the listed-incremental option of tar and since the
>>manpage is incomplete <rant deleted> i don't know what it
>>actually does.  However, i have found the use of ctime to be
>>terribly unreliable for file management and given what the
>>standards have to say on the issue it sounds like tar is
>>being abused or has a bug.
>>
>>
>>
>>
>>    
>>
>
>
>--
>Hans
>
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>
>
>  
>


-- 
Hans

Re: ReiserFS patch for updating ctimes of renamed files

[Hans Reiser]

Andrew Morton wrote:

>Hans Reiser <reiser@namesys.com> wrote:
>  
>
>>In theory it is cleaner and purer to do it the way we did.  In practice, 
>> Alex's problem seems like a real one, and I don't know how hard it is to 
>> change tar to do the right thing.  We'll discuss it in a small seminar 
>> today.
>>    
>>
>
>It would be best to make this change.  minix, ext2 and ext3 do set ctime,
>so it is "the Linux standard".
>
>
>
>
>  
>
do you think schultz's arguments about why it is wrong are correct?  
They seem well thought out to me.

-- 
Hans

Re: ReiserFS patch for updating ctimes of renamed files

[Andrew Morton]

Hans Reiser <reiser@namesys.com> wrote:
>
> do you think schultz's arguments about why it is wrong are correct?  
> They seem well thought out to me.

Well given that you've renamed the file, you do want the backup program to
pick up the "new" file.  But it'd be a pretty lame backup program which was
fooled by a missing ctime update.

Yes, John has a point but we're not going to go and change all the other
filesystems (are we?).

Re: ReiserFS patch for updating ctimes of renamed files

[Hans Reiser]

Andrew Morton wrote:

>Hans Reiser <reiser@namesys.com> wrote:
>  
>
>>do you think schultz's arguments about why it is wrong are correct?  
>>They seem well thought out to me.
>>    
>>
>
>Well given that you've renamed the file, you do want the backup program to
>pick up the "new" file.  But it'd be a pretty lame backup program which was
>fooled by a missing ctime update.
>
>Yes, John has a point but we're not going to go and change all the other
>filesystems (are we?).
>
>
>
>  
>
why not?  It is correct therefor....

-- 
Hans

Re: ReiserFS patch for updating ctimes of renamed files

[Andrew Morton]

Hans Reiser <reiser@namesys.com> wrote:
>
> >Yes, John has a point but we're not going to go and change all the other
>  >filesystems (are we?).
>  >
>  >
>  >
>  >  
>  >
>  why not?  It is correct therefor....

Because that's the way we've always done it and there have been zero
complaints about it and changing it risks breaking things.

I should think that's pretty obvious?

Re: ReiserFS patch for updating ctimes of renamed files

[jw schultz]

On Tue, Oct 14, 2003 at 10:09:40AM +0400, Hans Reiser wrote:
> Let's see what Andrew says after he reads J.W.'s reasoning.  I agree 
> that reiserfs should do the same thing as the other filesystems in 
> Linux, but J.W. seems to be right that they are doing the wrong thing.

Whoa there.  I am not saying that the other filesystems
are wrong.  All i am saying as i can see no reason why, as
defined, the ctime of a file should be updated when nothing
else in the inode structure as been updated.

Of course if i had designed it in the first place with the
filesystem semantics that we have now there might be no
rename system call.  Renames would be done by link(oldpath,
newpath); unlink(oldpath);  A sequence that would cause
ctime to change as a result of nlink changes.  A sequence
that might be appropriate in some cases even inside the
filesystem code.

If you read my first posting on this thread you will see
that i do see some, albeit questionable, value to the ctime
update as a means of spotting the fact something relating to
the inode has changed.  While not technically required it is
not necessarily wrong to update ctime.  SUSv3 even allows
for it and for some filesystems it would positively be the
right thing to do.

PS.  The name is J.W. or JW and has never been John.

> 
> Hans
> 
> Alex Adriaanse wrote:
> 
> >Hans,
> >
> >Yes, I agree with J.W.  However, I also think that Andrew has a good point
> >in that the behavior across Linux filesystems (ReiserFS, ext2, ext3, minix,
> >etc.) should be consistent.  Either they should all update ctime during
> >renames, or none of them should.
> >
> >Anyway, I'll try to work with the GNU tar maintainer to get this problem in
> >tar fixed.  It'll probably be a lot harder to fix in tar than to have
> >ReiserFS update ctimes since it'll require major changes in
> >the --listed-incremental snapshot files.  However, if you don't think it's 
> >a
> >good idea to make these changes to ReiserFS then we'll just work on fixing
> >up tar.
> >
> >Thanks,
> >
> >Alex
> >
> >-----Original Message-----
> >From: linux-kernel-owner@vger.kernel.org
> >[mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Hans Reiser
> >Sent: Monday, October 13, 2003 3:46 AM
> >To: Alex Adriaanse
> >Cc: jw schultz; Linux Kernel Mailing List; vs@namesys.com
> >Subject: Re: ReiserFS patch for updating ctimes of renamed files
> >
> >
> >Alex, are you convinced by jw?  (I think I am.)  Would you be willing to
> >submit a patch for tar instead?
> >
> >Hans
> >
> >jw schultz wrote:
> >
> > 
> >
> >>On Mon, Oct 13, 2003 at 09:49:20AM +0400, Hans Reiser wrote:
> >>
> >>
> >>In theory it is cleaner and purer to do it the way we did. In practice,
> >>
> >>   
> >>
> >>>Alex's problem seems like a real one, and I don't know how hard it is to
> >>>change tar to do the right thing.  We'll discuss it in a small seminar
> >>>today.
> >>>
> >>>
> >>>     
> >>>
> >>Updating ctime does seem messy and a bit irrelevant for the
> >>atomic rename.  You are modifying the directories not the
> >>fricken file. This isn't DOS!  But it would seem he does
> >>indeed have an issue although i'm not sure what.  I've never
> >>used the listed-incremental option of tar and since the
> >>manpage is incomplete <rant deleted> i don't know what it
> >>actually does.  However, i have found the use of ctime to be
> >>terribly unreliable for file management and given what the
> >>standards have to say on the issue it sounds like tar is
> >>being abused or has a bug.
> >>
> >>
> >>
> >>
> >>   
> >>
> >
> >
> >--
> >Hans
> >
> >
> >-
> >To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> >the body of a message to majordomo@vger.kernel.org
> >More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >Please read the FAQ at  http://www.tux.org/lkml/
> >
> >
> >
> > 
> >
> 
> 
> -- 
> Hans
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt

Re: ReiserFS patch for updating ctimes of renamed files

[jw schultz]

On Mon, Oct 13, 2003 at 11:25:27PM -0700, Andrew Morton wrote:
> Hans Reiser <reiser@namesys.com> wrote:
> >
> > do you think schultz's arguments about why it is wrong are correct?  
> > They seem well thought out to me.
> 
> Well given that you've renamed the file, you do want the backup program to
> pick up the "new" file.  But it'd be a pretty lame backup program which was
> fooled by a missing ctime update.
> 
> Yes, John has a point but we're not going to go and change all the other
> filesystems (are we?).

I don't know who John is but i sure hope we are not going to
go changing how working filesystems function.  It may be
technically correct to not update ctime but that doesn't
mean that it is incorrect to update it either.  It all
depends on the filesystem.  They aren't all the same.  We
have some that don't support symlinks or hardlinks or that
have or lack other features.

<OT>
There are change detections through timestamp (mtime) i am
concerned about.  As an rsync maintainer i worry about be
extended attributes or ACLs changing with no modifiable
timestamps being updated.  And, by the way, because you
cannot set ctime it doesn't qualify.  Then there is jfs
which i found did not update mtime when directories change
unless the block count changes, but that might have been
fixed already.
</OT>

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt

Re: ReiserFS patch for updating ctimes of renamed files

[Jamie Lokier]

jw schultz wrote:
> Of course if i had designed it in the first place with the
> filesystem semantics that we have now there might be no
> rename system call.  Renames would be done by link(oldpath,
> newpath); unlink(oldpath);  A sequence that would cause
> ctime to change as a result of nlink changes.  A sequence
> that might be appropriate in some cases even inside the
> filesystem code.

Once upon a time, that's how renames were always done.

The rename() system call was added because (a) it provides the
additional atomicity semantic, which link+unlink does not; (b) it is
useful to allow directory renames, but directory hard links are
dangerous so not allowed any more.

-- Jamie

Re: ReiserFS patch for updating ctimes of renamed files

[Alex Adriaanse]

Hi Vladimir,

I must've messed something up when testing the updates of ctime during a
rename on ext2, because when looking at the ext2 code the ctime of a
renamed file doesn't normally get updated.  Maybe I simply forgot to
test it on ext2 and only tested out tmpfs.  Oops. :)  Sorry for the
confusion.  Based on my findings below, ext2's (or minix or ufs for that
matter) behavior seems also inconsistent with some of the other FSes I
mentioned below (ext2/minix/ufs also seem to share a lot of code in
their rename functions).

When glancing through the 2.6.0-test9 source code, I verified that ext3
updates the ctime (fs/ext3/namei.c:2244, with the comment, "Like most
other Unix systems, set the ctime for inodes on a rename."), JFS updates
ctime (fs/jfs/namei.c:1231), xfs updates it (xfs/xfs_rename.c:480), etc.
I believe all these updates are made for both directories and
non-directories, so it appears that your patch which updates only a
directory's ctime needs to cover non-directories as well (at least if
you're trying to be consistent across other Linux filesystems like
ext3/jfs/xfs).

Alex

On Thu, Nov 20, 2003 at 05:46:15PM +0300, Vladimir Saveliev wrote:
> Hi
> 
> Sorry for delay with this.
> I looked over linux's renames and they seem to be doing exactly what reiserfs does: 
> do not change anything (neither ctime nor mtime) renaming not-directory. Quick test confirms that.
> Please, look at its log:
> <LOG>
> tribesman:/rescue # mount
> /dev/hda2 on / type reiserfs (rw)
> proc on /proc type proc (rw)
> devpts on /dev/pts type devpts (rw,mode=0620,gid=5)
> /dev/hda1 on /rescue type ext2 (rw)
> shmfs on /dev/shm type shm (rw)
> tribesman:/rescue # stat bigfile
>   File: `bigfile'
>   Size: 1099511627777   Blocks: 198896     IO Block: 4096   regular file
> Device: 301h/769d       Inode: 187         Links: 1
> Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
> Access: 2003-02-13 19:56:51.000000000 +0300
> Modify: 2003-02-13 19:56:10.000000000 +0300
> Change: 2003-02-13 19:56:10.000000000 +0300
> 
> tribesman:/rescue # mv bigfile tmp/
> tribesman:/rescue # stat tmp/bigfile
>   File: `tmp/bigfile'
>   Size: 1099511627777   Blocks: 198896     IO Block: 4096   regular file
> Device: 301h/769d       Inode: 187         Links: 1
> Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
> Access: 2003-02-13 19:56:51.000000000 +0300
> Modify: 2003-02-13 19:56:10.000000000 +0300
> Change: 2003-02-13 19:56:10.000000000 +0300
> </LOG>
> 
> 
> However, renaming directory reiserfs did not update its ctime and mtime. Patch to fix that is attached.
> Alex, does it cause tar to behave on reiserfs similar to ext2, ext3, etc?
> 
> Thanks,
> vs
> 
> 
> On Wednesday 19 November 2003 20:30, Hans Reiser wrote:
> > Alex Adriaanse wrote:
> > 
> > >Hi Hans & Vladimir,
> > >
> > >Is there any chance that this patch will make it into 2.4.24 (or 2.6.0 for
> > >that matter)? 
> > >
> > > I'm just curious.
> > >
> > >Thanks,
> > >
> > >Alex
> > >
> > >Alex Adriaanse wrote:
> > >>Hi Hans,
> > >>
> > >>I updated my patch to include Andrew's suggestion of eliminating extra
> > >>    
> > >>
> > >calls
> > >  
> > >
> > >>to CURRENT_TIME.  I also finally got a chance to test it out, and it seems
> > >>to work.  After applying this patch, ctime gets updated after a rename, and
> > >>GNU tar now backs things up properly.  I also could not detect any
> > >>filesystem corruption after doing some renames.
> > >>
> > >>Alex
> > >>
> > >>--- fs/reiserfs/namei.c.orig    Mon Aug 25 06:44:43 2003
> > >>+++ fs/reiserfs/namei.c Fri Oct 24 17:16:33 2003
> > >>@@ -1205,8 +1205,11 @@
> > >>
> > >>    mark_de_hidden (old_de.de_deh + old_de.de_entry_num);
> > >>    journal_mark_dirty (&th, old_dir->i_sb, old_de.de_bh);
> > >>-    old_dir->i_ctime = old_dir->i_mtime = CURRENT_TIME;
> > >>-    new_dir->i_ctime = new_dir->i_mtime = CURRENT_TIME;
> > >>+    ctime = CURRENT_TIME;
> > >>+    old_dir->i_ctime = old_dir->i_mtime = ctime;
> > >>+    new_dir->i_ctime = new_dir->i_mtime = ctime;
> > >>+    old_inode->i_ctime = ctime;
> > >>+    reiserfs_update_sd (&th, old_inode);
> > >>
> > >>    if (new_dentry_inode) {
> > >>       // adjust link number of the victim
> > >>@@ -1215,7 +1218,6 @@
> > >>       } else {
> > >>           new_dentry_inode->i_nlink--;
> > >>       }
> > >>-       ctime = CURRENT_TIME;
> > >>       new_dentry_inode->i_ctime = ctime;
> > >>       savelink = new_dentry_inode->i_nlink;
> > >>    }
> > >>
> > >>

RE: ReiserFS patch for updating ctimes of renamed files

[Alex Adriaanse]

Hi Hans,

I updated my patch to include Andrew's suggestion of eliminating extra calls
to CURRENT_TIME.  I also finally got a chance to test it out, and it seems
to work.  After applying this patch, ctime gets updated after a rename, and
GNU tar now backs things up properly.  I also could not detect any
filesystem corruption after doing some renames.

Alex

--- fs/reiserfs/namei.c.orig    Mon Aug 25 06:44:43 2003
+++ fs/reiserfs/namei.c Fri Oct 24 17:16:33 2003
@@ -1205,8 +1205,11 @@

     mark_de_hidden (old_de.de_deh + old_de.de_entry_num);
     journal_mark_dirty (&th, old_dir->i_sb, old_de.de_bh);
-    old_dir->i_ctime = old_dir->i_mtime = CURRENT_TIME;
-    new_dir->i_ctime = new_dir->i_mtime = CURRENT_TIME;
+    ctime = CURRENT_TIME;
+    old_dir->i_ctime = old_dir->i_mtime = ctime;
+    new_dir->i_ctime = new_dir->i_mtime = ctime;
+    old_inode->i_ctime = ctime;
+    reiserfs_update_sd (&th, old_inode);

     if (new_dentry_inode) {
        // adjust link number of the victim
@@ -1215,7 +1218,6 @@
        } else {
            new_dentry_inode->i_nlink--;
        }
-       ctime = CURRENT_TIME;
        new_dentry_inode->i_ctime = ctime;
        savelink = new_dentry_inode->i_nlink;
     }

RE: ReiserFS patch for updating ctimes of renamed files

[Alex Adriaanse]

Great.  Thanks a lot!

Where would I be able to download the patch that you guys will be producing
once it's through QA?  At ftp://ftp.namesys.com/pub/reiserfs-for-2.4/ or is
there some other place I can get it (e.g. BK/CVS/HTTP/FTP) before it's
posted at that link?

Thanks,

Alex

-----Original Message-----
From: linux-kernel-owner@vger.kernel.org
[mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Hans Reiser
Sent: Tuesday, October 14, 2003 3:41 AM
To: Anton Ertl
Cc: linux-kernel@vger.kernel.org; Andrew Morton; vs@thebsh.namesys.com;
jw schultz; Alex Adriaanse
Subject: Re: ReiserFS patch for updating ctimes of renamed files


I looked again at the definition of the difference between ctime and
mtime on the stat man page, and I think that updating ctime in response
to rename is as reasonable as updating it in response to changing the
number of links.

Ok, we will conform, and I will accept the kindly donated patch, along
with Andrew's optimization of our evaluation of CURRENT_TIME.  vs,
please add Andrew's suggested optimization and sent the result through
QA.  Thanks to all for your good advice.

--
Hans


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: ReiserFS patch for updating ctimes of renamed files

[Hans Reiser]

I looked again at the definition of the difference between ctime and 
mtime on the stat man page, and I think that updating ctime in response 
to rename is as reasonable as updating it in response to changing the 
number of links.

Ok, we will conform, and I will accept the kindly donated patch, along 
with Andrew's optimization of our evaluation of CURRENT_TIME.  vs, 
please add Andrew's suggested optimization and sent the result through 
QA.  Thanks to all for your good advice.

-- 
Hans

Re: ReiserFS patch for updating ctimes of renamed files

[Anton Ertl]

Hans Reiser:
>Andrew Morton wrote:
>>Yes, John has a point but we're not going to go and change all the other
>>filesystems (are we?).
>>
>why not?  It is correct therefor....

Many years ago we had the same problem with ext2: it did not change
ctime on rename, so GNU tar did not pick up the renamed files on
incremental backup.  Fortunately a few kernel versions later ext2
changed to the current behaviour (unfortunately I don't remember the
kernel version).

IIRC our sysadmin submitted a bug report for GNU tar at the time, and
got the answer that GNU tar would not change.

- anton
-- 
M. Anton Ertl                    Some things have to be seen to be believed
anton@mips.complang.tuwien.ac.at Most things have to be believed to be seen
http://www.complang.tuwien.ac.at/anton/home.html