All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
@ 2009-10-19 17:17 Steve French
  2009-10-19 18:55 ` Andreas Dilger
  0 siblings, 1 reply; 31+ messages in thread
From: Steve French @ 2009-10-19 17:17 UTC (permalink / raw)
  To: linux-fsdevel; +Cc: samba-technical

A new version of the format for Samba server related file system
extended attributes is expected in the next version of Samba so the
topic of create time and nanosecond timestamps for Linux has come up
again.  Since some file systems don't support storing create time
(birth time), and none support updating create time, or for that
matter for storing any nanosecond timestamps at all (a millisecond
seems like a much longer time today than when the stat structure was
defined), and dos attributes, Samba server stores these in extended
attributes, which is awkward on those file systems which store
(different) versions of these on disk.

In the absence of any syscalls for utimes (nanosecond granularity) and
no syscall for returning (or setting) create time (birth time), and
ext4 likely to become popular (ext4 stores nanosecond timestamps, and
create time), Samba developers (and probably other apps such as backup
applications) need a way to modify the inode's nanosecond timestamps
and get at file create time, perhaps via ioctl and/or xattr.

I think the chance of adding new syscalls for these and libc getting
modified for this is near zero, so any opinions about the best way for
Samba server to get at this information?

Mingming Cao wrote:
> 1)  Ext4 adds support for date-created timestamps. But, current stat()interface
> doesn't support the file creation time/delete time. As far as I know, it is hard
> to modify or add the necessary systemcall to export this info out to
> user programs for now. These changes would require coordination of many projects.
>2) from the linux man page utimes seems only allow to set microsecond timestamp


-- 
Thanks,

Steve

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-19 17:17 ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime Steve French
@ 2009-10-19 18:55 ` Andreas Dilger
  2009-10-19 18:58   ` Jeremy Allison
                     ` (3 more replies)
  0 siblings, 4 replies; 31+ messages in thread
From: Andreas Dilger @ 2009-10-19 18:55 UTC (permalink / raw)
  To: Steve French; +Cc: linux-fsdevel, samba-technical

On 19-Oct-09, at 11:17, Steve French wrote:
> A new version of the format for Samba server related file system
> extended attributes is expected in the next version of Samba so the
> topic of create time and nanosecond timestamps for Linux has come up
> again.  Since some file systems don't support storing create time
> (birth time), and none support updating create time, or for that
> matter for storing any nanosecond timestamps at all (a millisecond
> seems like a much longer time today than when the stat structure was
> defined), and dos attributes, Samba server stores these in extended
> attributes, which is awkward on those file systems which store
> (different) versions of these on disk.

I had proposed in the past that these file attributes be exposed to
userspace as virtual xattrs, e.g. user.cr_time returning a struct
timespec, but the data is stored internal to the filesystem in whatever
format is most efficient for it.  We only strictly need user.crtime
for this, but I wouldn't object to exporting other inode attributes in
this way (e.g. user.mtime, user.atime, etc).

I believe that nanosecond timestamps ARE supported with newer  
filesystems
using stat().  There is the utimensat() system call, which takes a  
timespec
parameter, so it looks to be suitable for setting nanosecond timestamps
from userspace.  I don't know if glibc() supports this, but I do see it
in the kernel.

As for _updating_ create time, isn't that sort of defeating the purpose
of create time?

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-19 18:55 ` Andreas Dilger
@ 2009-10-19 18:58   ` Jeremy Allison
  2009-10-19 19:37     ` Steve French
  2009-10-19 19:12   ` Zach Brown
                     ` (2 subsequent siblings)
  3 siblings, 1 reply; 31+ messages in thread
From: Jeremy Allison @ 2009-10-19 18:58 UTC (permalink / raw)
  To: Andreas Dilger; +Cc: linux-fsdevel, Steve French, samba-technical

On Mon, Oct 19, 2009 at 12:55:29PM -0600, Andreas Dilger wrote:
>
> As for _updating_ create time, isn't that sort of defeating the purpose
> of create time?

Yes, you would think so, wouldn't you. Unfortunately that's
the way Windows behaves (create time can be updated) so that's
what we have to emulate. If does indeed make no sense (it's
just another timestamp meta-data that can't be used for auditing
at that point).

Jeremy.

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-19 18:55 ` Andreas Dilger
  2009-10-19 18:58   ` Jeremy Allison
@ 2009-10-19 19:12   ` Zach Brown
  2009-10-19 20:49     ` Jamie Lokier
  2009-10-19 19:45   ` Steve French
  2009-10-20  0:41   ` Mingming
  3 siblings, 1 reply; 31+ messages in thread
From: Zach Brown @ 2009-10-19 19:12 UTC (permalink / raw)
  To: Andreas Dilger; +Cc: Steve French, linux-fsdevel, samba-technical


> As for _updating_ create time, isn't that sort of defeating the purpose
> of create time?

Maybe one would want to restore the original create times when restoring
from backups?

- z

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-19 18:58   ` Jeremy Allison
@ 2009-10-19 19:37     ` Steve French
  0 siblings, 0 replies; 31+ messages in thread
From: Steve French @ 2009-10-19 19:37 UTC (permalink / raw)
  To: Jeremy Allison; +Cc: Andreas Dilger, linux-fsdevel, samba-technical

On Mon, Oct 19, 2009 at 1:58 PM, Jeremy Allison <jra@samba.org> wrote:
> On Mon, Oct 19, 2009 at 12:55:29PM -0600, Andreas Dilger wrote:
>>
>> As for _updating_ create time, isn't that sort of defeating the purpose
>> of create time?
>
> Yes, you would think so, wouldn't you. Unfortunately that's
> the way Windows behaves (create time can be updated) so that's
> what we have to emulate. If does indeed make no sense (it's
> just another timestamp meta-data that can't be used for auditing
> at that point).

Updating create time does make it harder to use for auditing, but
I expect that 20 years ago when this was added someone was thinking
about making sure you could preserve all attributes on backup/restore
especially for a system restore after disk failure - and end up with
the same file metadata as when you started.


-- 
Thanks,

Steve

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-19 18:55 ` Andreas Dilger
  2009-10-19 18:58   ` Jeremy Allison
  2009-10-19 19:12   ` Zach Brown
@ 2009-10-19 19:45   ` Steve French
  2009-10-19 20:11     ` Andreas Dilger
  2009-10-20  0:41   ` Mingming
  3 siblings, 1 reply; 31+ messages in thread
From: Steve French @ 2009-10-19 19:45 UTC (permalink / raw)
  To: Andreas Dilger; +Cc: linux-fsdevel, samba-technical

On Mon, Oct 19, 2009 at 1:55 PM, Andreas Dilger <adilger@sun.com> wrote:
> On 19-Oct-09, at 11:17, Steve French wrote:
>>
>> A new version of the format for Samba server related file system
>> extended attributes is expected in the next version of Samba so the
>> topic of create time and nanosecond timestamps for Linux has come up
>> again.  Since some file systems don't support storing create time
>> (birth time), and none support updating create time, or for that
>> matter for storing any nanosecond timestamps at all (a millisecond
>> seems like a much longer time today than when the stat structure was
>> defined), and dos attributes, Samba server stores these in extended
>> attributes, which is awkward on those file systems which store
>> (different) versions of these on disk.
>
> I had proposed in the past that these file attributes be exposed to
> userspace as virtual xattrs, e.g. user.cr_time returning a struct
> timespec, but the data is stored internal to the filesystem in whatever
> format is most efficient for it.  We only strictly need user.crtime
> for this, but I wouldn't object to exporting other inode attributes in
> this way (e.g. user.mtime, user.atime, etc).

Yes - the fake xattr approach was the approach that I preferred
(since I could do this trivially to cifs, and for ext4 it seems equally easy),
but I wonder if this "path based" approach is slightly harder for Samba,
since for most cases Samba maps to "handle based" equivalents
(when the same operation can come in eitherpath based or
handle based - Samba mostly maps to handle based).  In cifs
I also store dos attributes (as presumably vfat and ntfs could too) which
if requested could return the same way (network backup application writers
have asked me for an xattr for this before, but I would prefer to use
a standard xattr name rather than something cifs specific).  As we
move to SMB2, all network file operations become handle based
(except open).

It may be easier for Samba as an ioctl (handle based) rather than
xattr (path based) - not sure.


-- 
Thanks,

Steve
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-19 19:45   ` Steve French
@ 2009-10-19 20:11     ` Andreas Dilger
  2009-10-19 22:24       ` Steve French
  0 siblings, 1 reply; 31+ messages in thread
From: Andreas Dilger @ 2009-10-19 20:11 UTC (permalink / raw)
  To: Steve French; +Cc: linux-fsdevel, samba-technical

On 19-Oct-09, at 13:45, Steve French wrote:
> On Mon, Oct 19, 2009 at 1:55 PM, Andreas Dilger <adilger@sun.com>  
> wrote:
>> I had proposed in the past that these file attributes be exposed to
>> userspace as virtual xattrs, e.g. user.cr_time returning a struct
>> timespec, but the data is stored internal to the filesystem in  
>> whatever
>> format is most efficient for it.  We only strictly need user.crtime
>> for this, but I wouldn't object to exporting other inode attributes  
>> in
>> this way (e.g. user.mtime, user.atime, etc).
>
> Yes - the fake xattr approach was the approach that I preferred
> (since I could do this trivially to cifs, and for ext4 it seems  
> equally easy),
> but I wonder if this "path based" approach is slightly harder for  
> Samba,
> since for most cases Samba maps to "handle based" equivalents
> (when the same operation can come in eitherpath based or
> handle based - Samba mostly maps to handle based).

Could you please elaborate what you mean by "path based" for xattrs?
They can be gotten with sys_fgetxattr() from a file handle just as
easily as from the filename.

I definitely agree with having a common attribute name for these.

As for being able to write to the "create time" attribute, I would  
prefer
that this be a filesystem mount option.  For some users (myself  
included)
I don't care whether Windows is unhappy that it can't update this  
creation
time - I'd prefer to know when a file is actually created.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-19 19:12   ` Zach Brown
@ 2009-10-19 20:49     ` Jamie Lokier
  0 siblings, 0 replies; 31+ messages in thread
From: Jamie Lokier @ 2009-10-19 20:49 UTC (permalink / raw)
  To: Zach Brown; +Cc: Andreas Dilger, Steve French, linux-fsdevel, samba-technical

Zach Brown wrote:
> > As for _updating_ create time, isn't that sort of defeating the purpose
> > of create time?
> 
> Maybe one would want to restore the original create times when restoring
> from backups?

Ideally that can be restricted to the special privilege that only
backup programs have?  I know Windows has something like that, but I'm
not sure how it's used.

-- Jamie

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-19 20:11     ` Andreas Dilger
@ 2009-10-19 22:24       ` Steve French
  2009-10-19 23:12         ` Andreas Dilger
  0 siblings, 1 reply; 31+ messages in thread
From: Steve French @ 2009-10-19 22:24 UTC (permalink / raw)
  To: Andreas Dilger; +Cc: linux-fsdevel, samba-technical

On Mon, Oct 19, 2009 at 3:11 PM, Andreas Dilger <adilger@sun.com> wrote:
> On 19-Oct-09, at 13:45, Steve French wrote:
>>
>> On Mon, Oct 19, 2009 at 1:55 PM, Andreas Dilger <adilger@sun.com> wrote:
>>>
>>> I had proposed in the past that these file attributes be exposed to
>>> userspace as virtual xattrs, e.g. user.cr_time returning a struct
>>> timespec, but the data is stored internal to the filesystem in whatever
>>> format is most efficient for it.  We only strictly need user.crtime
>>> for this, but I wouldn't object to exporting other inode attributes in
>>> this way (e.g. user.mtime, user.atime, etc).
>>
>> Yes - the fake xattr approach was the approach that I preferred
>> (since I could do this trivially to cifs, and for ext4 it seems equally
>> easy),
>> but I wonder if this "path based" approach is slightly harder for Samba,
>> since for most cases Samba maps to "handle based" equivalents
>> (when the same operation can come in eitherpath based or
>> handle based - Samba mostly maps to handle based).
>
> Could you please elaborate what you mean by "path based" for xattrs?
> They can be gotten with sys_fgetxattr() from a file handle just as
> easily as from the filename.

Although handle based interfaces are there for xattrs in userspace,
internally the xattr calls are path based, so handle based
calls get converted to path based in kernel and you could lose
some of the performance benefit - path based could be slightly
slower (due to path revalidation, and access checks) than
handle based calls.
        int (*setxattr) (struct dentry *, const char *,const void *,size_t,int);
        ssize_t (*getxattr) (struct dentry *, const char *, void *, size_t);
        ssize_t (*listxattr) (struct dentry *, char *, size_t);
        int (*removexattr) (struct dentry *, const char *);



>
> I definitely agree with having a common attribute name for these.
>
> As for being able to write to the "create time" attribute, I would prefer
> that this be a filesystem mount option.  For some users (myself included)
> I don't care whether Windows is unhappy that it can't update this creation
> time - I'd prefer to know when a file is actually created.

I agree - if create time could be overwritten - that behavior hould be
configurable (another post mentioned the alternative to mount
option - a flag for this perhaps along the lines of the a "backup intent"
flag - although somewhat different than what Windows uses that for).



-- 
Thanks,

Steve
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-19 22:24       ` Steve French
@ 2009-10-19 23:12         ` Andreas Dilger
  2009-10-20  3:31           ` Steve French
  2009-10-20 12:44           ` jim owens
  0 siblings, 2 replies; 31+ messages in thread
From: Andreas Dilger @ 2009-10-19 23:12 UTC (permalink / raw)
  To: Steve French; +Cc: linux-fsdevel, samba-technical

On 19-Oct-09, at 16:24, Steve French wrote:
> On Mon, Oct 19, 2009 at 3:11 PM, Andreas Dilger <adilger@sun.com>  
> wrote:
>> On 19-Oct-09, at 13:45, Steve French wrote:
>>> Yes - the fake xattr approach was the approach that I preferred
>>> (since I could do this trivially to cifs, and for ext4 it seems  
>>> equally
>>> easy),
>>> but I wonder if this "path based" approach is slightly harder for  
>>> Samba,
>>> since for most cases Samba maps to "handle based" equivalents
>>> (when the same operation can come in eitherpath based or
>>> handle based - Samba mostly maps to handle based).
>>
>> Could you please elaborate what you mean by "path based" for xattrs?
>> They can be gotten with sys_fgetxattr() from a file handle just as
>> easily as from the filename.
>
> Although handle based interfaces are there for xattrs in userspace,
> internally the xattr calls are path based, so handle based
> calls get converted to path based in kernel and you could lose
> some of the performance benefit - path based could be slightly
> slower (due to path revalidation, and access checks) than
> handle based calls.
>        int (*setxattr) (struct dentry *, const char *,const void  
> *,size_t,int);
>        ssize_t (*getxattr) (struct dentry *, const char *, void *,  
> size_t);
>        ssize_t (*listxattr) (struct dentry *, char *, size_t);
>        int (*removexattr) (struct dentry *, const char *);

I'm not sure I understand your point.  sys_fgetxattr() maps directly to
vfs_getxattr(), and while it still does permission checks, that doesn't
have anything to do with pathname revalidation AFAICS.

There are an equal number of permission checks in sys_fstat- 
 >vfs_getattr()
as in sys_fgetxattr->vfs_getxattr().

>> As for being able to write to the "create time" attribute, I would  
>> prefer
>> that this be a filesystem mount option.  For some users (myself  
>> included)
>> I don't care whether Windows is unhappy that it can't update this  
>> creation
>> time - I'd prefer to know when a file is actually created.
>
> I agree - if create time could be overwritten - that behavior hould be
> configurable (another post mentioned the alternative to mount
> option - a flag for this perhaps along the lines of the a "backup  
> intent"
> flag - although somewhat different than what Windows uses that for).


If this is a flag that a user can configure/select themselves, then it
is completely useless to me.  If it is a mount option and/or possibly an
additional process capability that would be more useful.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-19 18:55 ` Andreas Dilger
                     ` (2 preceding siblings ...)
  2009-10-19 19:45   ` Steve French
@ 2009-10-20  0:41   ` Mingming
  3 siblings, 0 replies; 31+ messages in thread
From: Mingming @ 2009-10-20  0:41 UTC (permalink / raw)
  To: Andreas Dilger; +Cc: Steve French, linux-fsdevel, samba-technical

On Mon, 2009-10-19 at 12:55 -0600, Andreas Dilger wrote:
> On 19-Oct-09, at 11:17, Steve French wrote:
> > A new version of the format for Samba server related file system
> > extended attributes is expected in the next version of Samba so the
> > topic of create time and nanosecond timestamps for Linux has come up
> > again.  Since some file systems don't support storing create time
> > (birth time), and none support updating create time, or for that
> > matter for storing any nanosecond timestamps at all (a millisecond
> > seems like a much longer time today than when the stat structure was
> > defined), and dos attributes, Samba server stores these in extended
> > attributes, which is awkward on those file systems which store
> > (different) versions of these on disk.
> 
> I had proposed in the past that these file attributes be exposed to
> userspace as virtual xattrs, e.g. user.cr_time returning a struct
> timespec, but the data is stored internal to the filesystem in whatever
> format is most efficient for it.  We only strictly need user.crtime
> for this, but I wouldn't object to exporting other inode attributes in
> this way (e.g. user.mtime, user.atime, etc).
> 
> I believe that nanosecond timestamps ARE supported with newer  
> filesystems
> using stat().  There is the utimensat() system call, which takes a  
> timespec
> parameter, so it looks to be suitable for setting nanosecond timestamps
> from userspace.  I don't know if glibc() supports this, but I do see it
> in the kernel.
> 

How we are going to set the nanotimestamps today? utimes(2) man page
seems indicating it only support setting time at  microseond level.

> As for _updating_ create time, isn't that sort of defeating the purpose
> of create time?
> 

Perphas the backup/restore wants to preserve the original creation
timestamp after the file being backuped?

> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-19 23:12         ` Andreas Dilger
@ 2009-10-20  3:31           ` Steve French
  2009-10-20 12:44           ` jim owens
  1 sibling, 0 replies; 31+ messages in thread
From: Steve French @ 2009-10-20  3:31 UTC (permalink / raw)
  To: Andreas Dilger; +Cc: linux-fsdevel, samba-technical

On Mon, Oct 19, 2009 at 6:12 PM, Andreas Dilger <adilger@sun.com> wrote:
> On 19-Oct-09, at 16:24, Steve French wrote:
>> some of the performance benefit - path based could be slightly
>> slower (due to path revalidation, and access checks) than
>> handle based calls.
>>       int (*setxattr) (struct dentry *, const char *,const void
>> *,size_t,int);
>>       ssize_t (*getxattr) (struct dentry *, const char *, void *, size_t);
>>       ssize_t (*listxattr) (struct dentry *, char *, size_t);
>>       int (*removexattr) (struct dentry *, const char *);
>
> I'm not sure I understand your point.  sys_fgetxattr() maps directly to
> vfs_getxattr(), and while it still does permission checks, that doesn't
> have anything to do with pathname revalidation AFAICS.
>
> There are an equal number of permission checks in sys_fstat->vfs_getattr()
> as in sys_fgetxattr->vfs_getxattr().

Good point.  Looking more carefully I think you are right - the path
revalidation
is skipped in fgetxattr so it probably isn't much slower if at all (than
a handle based ioctl call would be).

>>> As for being able to write to the "create time" attribute, I would prefer
>>> that this be a filesystem mount option.  For some users (myself included)
>>> I don't care whether Windows is unhappy that it can't update this
>>> creation
>>> time - I'd prefer to know when a file is actually created.
> If this is a flag that a user can configure/select themselves, then it
> is completely useless to me.  If it is a mount option and/or possibly an
> additional process capability that would be more useful.

Yes ... there is a precedent for a process capability for something similar
(e.g. a Windows user must have both the "backup privilege" in his
process capability and specify a flag indicating he wants to
use "backup intent" for certain operations).   There are already
33 capabilities in include/linux/capability.h,  some as narrow as this.

-- 
Thanks,

Steve
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-19 23:12         ` Andreas Dilger
  2009-10-20  3:31           ` Steve French
@ 2009-10-20 12:44           ` jim owens
  2009-10-20 20:33             ` Andreas Dilger
  1 sibling, 1 reply; 31+ messages in thread
From: jim owens @ 2009-10-20 12:44 UTC (permalink / raw)
  To: Andreas Dilger; +Cc: Steve French, linux-fsdevel, samba-technical

Andreas Dilger wrote:
> On 19-Oct-09, at 16:24, Steve French wrote:
>> On Mon, Oct 19, 2009 at 3:11 PM, Andreas Dilger <adilger@sun.com> wrote:
>>> As for being able to write to the "create time" attribute, I would 
>>> prefer
>>> that this be a filesystem mount option.  For some users (myself 
>>> included)
>>> I don't care whether Windows is unhappy that it can't update this 
>>> creation
>>> time - I'd prefer to know when a file is actually created.
>>
>> I agree - if create time could be overwritten - that behavior hould be
>> configurable (another post mentioned the alternative to mount
>> option - a flag for this perhaps along the lines of the a "backup intent"
>> flag - although somewhat different than what Windows uses that for).
> 
> If this is a flag that a user can configure/select themselves, then it
> is completely useless to me.  If it is a mount option and/or possibly an
> additional process capability that would be more useful.

Restricting the modification of create time is pointless.

A number of filesystems and operating systems besides windows
have create time and it MUST be settable on restore just like
the access and modification times.

There is no value in it needing "special privileges".  What
you are forgetting is *NO* timestamp on a filesystem can be
trusted to be correct.  All times are just fields set based
on what the system thinks the time is now (not to mention that
any disk editor can change the field).  The "now time" can be
intentionally or accidentally set wrong, or as has happened to
me, a totally crap value from a bad RTC battery. :)

I did legal records storage a long time ago, and I can
guarantee you disk timestamps are meaningless.

On the other hand, if you just want stamps for debugging
and internal purposes, then you do what I did, put versions
and times inside on-disk metadata structures the filesystem
does not expose.  As long as you remember all stamps are still
at the mercy of those who can patch your clock, your code,
and your raw disk blocks.

jim

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-20 12:44           ` jim owens
@ 2009-10-20 20:33             ` Andreas Dilger
  2009-10-20 20:49               ` Steve French
  2009-10-20 21:10               ` jim owens
  0 siblings, 2 replies; 31+ messages in thread
From: Andreas Dilger @ 2009-10-20 20:33 UTC (permalink / raw)
  To: jim owens; +Cc: Steve French, linux-fsdevel, samba-technical

On 20-Oct-09, at 06:44, jim owens wrote:
> Andreas Dilger wrote:
>> On 19-Oct-09, at 16:24, Steve French wrote:
>>> On Mon, Oct 19, 2009 at 3:11 PM, Andreas Dilger <adilger@sun.com>  
>>> wrote:
>>>> As for being able to write to the "create time" attribute, I  
>>>> would prefer
>>>> that this be a filesystem mount option.  For some users (myself  
>>>> included)
>>>> I don't care whether Windows is unhappy that it can't update this  
>>>> creation
>>>> time - I'd prefer to know when a file is actually created.
>>>
>>> I agree - if create time could be overwritten - that behavior  
>>> hould be
>>> configurable (another post mentioned the alternative to mount
>>> option - a flag for this perhaps along the lines of the a "backup  
>>> intent"
>>> flag - although somewhat different than what Windows uses that for).
>> If this is a flag that a user can configure/select themselves, then  
>> it
>> is completely useless to me.  If it is a mount option and/or  
>> possibly an
>> additional process capability that would be more useful.
>
> Restricting the modification of create time is pointless.

You contradict yourself later.  What we have NOW is creation timestamps
inside on-disk metadata structures the filesystem does not expose.
Going a step further - to expose this in a read-only manner does not
remove the usefulness of this field, but making it read-write does.

For those users/admins that want to be able to change this (due to
Samba, or whatever), great, we can give them this ability, but for
users who do NOT want regular users to be able to change the file
creation timestamp, that should be possible as well.

> A number of filesystems and operating systems besides windows
> have create time and it MUST be settable on restore just like
> the access and modification times.

It seems those other systems can't choke _too_ badly, given that
they don't have creation timestamps today and seem to work fine.
Since no Unix tools or applications depend on the presence of this
field, I would say that the default be to NOT allow this timestamp
to be changed.

> There is no value in it needing "special privileges".  What
> you are forgetting is *NO* timestamp on a filesystem can be
> trusted to be correct.  All times are just fields set based
> on what the system thinks the time is now (not to mention that
> any disk editor can change the field).  The "now time" can be
> intentionally or accidentally set wrong, or as has happened to
> me, a totally crap value from a bad RTC battery. :)

> On the other hand, if you just want stamps for debugging
> and internal purposes, then you do what I did, put versions
> and times inside on-disk metadata structures the filesystem
> does not expose.  As long as you remember all stamps are still
> at the mercy of those who can patch your clock, your code,
> and your raw disk blocks.


Sure, if you give every user write permission to the disk, then
every bit of security is meaningless, but I don't expect that
many sysadmins will do this.  For a single-user system, they
can do whatever they want in the end, so preventing regular
users from changing the timestamp is no hindrance to them in the
end either.


Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-20 20:33             ` Andreas Dilger
@ 2009-10-20 20:49               ` Steve French
  2009-10-20 20:59                 ` Sunil Mushran
  2009-10-21  0:44                 ` Andreas Dilger
  2009-10-20 21:10               ` jim owens
  1 sibling, 2 replies; 31+ messages in thread
From: Steve French @ 2009-10-20 20:49 UTC (permalink / raw)
  To: Andreas Dilger; +Cc: jim owens, linux-fsdevel, samba-technical

On Tue, Oct 20, 2009 at 3:33 PM, Andreas Dilger <adilger@sun.com> wrote:
> For those users/admins that want to be able to change this (due to
> Samba, or whatever), great, we can give them this ability, but for
> users who do NOT want regular users to be able to change the file
> creation timestamp, that should be possible as well.

So, are we ready for Mingming or one of the ext4 developers to propose
a patch for this via xattrs (I can do a similar one for cifs).
Sounds like various have said:

1) xattrs instead of ioctl
2) get of create time allowed by default, but set of create time limited

But for the second part of this - how should we set nanosecond
timestamps for the others (or in Samba's case, rounded to 100
nanoseconds for DCE time) ... a millisecond granularity for utimes is
a very long time these days.



-- 
Thanks,

Steve

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and  getting/setting nanosecond time stamps and utime
  2009-10-20 20:49               ` Steve French
@ 2009-10-20 20:59                 ` Sunil Mushran
  2009-10-20 21:11                   ` Steve French
  2009-10-21  0:44                 ` Andreas Dilger
  1 sibling, 1 reply; 31+ messages in thread
From: Sunil Mushran @ 2009-10-20 20:59 UTC (permalink / raw)
  To: Steve French; +Cc: Andreas Dilger, jim owens, linux-fsdevel, samba-technical

Steve French wrote:
> So, are we ready for Mingming or one of the ext4 developers to propose
> a patch for this via xattrs (I can do a similar one for cifs).
> Sounds like various have said:
>
> 1) xattrs instead of ioctl
> 2) get of create time allowed by default, but set of create time limited
>
> But for the second part of this - how should we set nanosecond
> timestamps for the others (or in Samba's case, rounded to 100
> nanoseconds for DCE time) ... a millisecond granularity for utimes is
> a very long time these days.

What's wrong with stat(2)? AFAIK, it supports nanoseconds.

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-20 20:33             ` Andreas Dilger
  2009-10-20 20:49               ` Steve French
@ 2009-10-20 21:10               ` jim owens
  1 sibling, 0 replies; 31+ messages in thread
From: jim owens @ 2009-10-20 21:10 UTC (permalink / raw)
  To: Andreas Dilger; +Cc: Steve French, linux-fsdevel, samba-technical

Andreas Dilger wrote:
> On 20-Oct-09, at 06:44, jim owens wrote:
>> Restricting the modification of create time is pointless.
> 
> You contradict yourself later.  What we have NOW is creation timestamps
> inside on-disk metadata structures the filesystem does not expose.
> Going a step further - to expose this in a read-only manner does not
> remove the usefulness of this field, but making it read-write does.

The apparent contradiction is that I forgot to explicitly say
there are two kinds of create time stamps in filesystems:
   - "user create time"
   - "internal filesystem create time"

My assumption was we were *adding* a "user create time" which
must be settable by things like star that archive and restore.

Otherwise we have "modification time" before "user create time"
when a user restores their files.

An "internal filesystem create time" that is not exposed or
is read-only is different.  And AFAIK most linux filesystems
do not have either create time today so unless we want to say
Samba only uses ext4, we need to think broader in scope.

> Since no Unix tools or applications depend on the presence of this
> field, I would say that the default be to NOT allow this timestamp
> to be changed.

If nobody uses the field, what is the need to protect it?
Seems you are now contradicting yourself.

jim

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-20 20:59                 ` Sunil Mushran
@ 2009-10-20 21:11                   ` Steve French
  2009-10-20 21:23                     ` Sunil Mushran
  2009-10-21 11:59                     ` Henrik Nordstrom
  0 siblings, 2 replies; 31+ messages in thread
From: Steve French @ 2009-10-20 21:11 UTC (permalink / raw)
  To: Sunil Mushran; +Cc: Andreas Dilger, jim owens, linux-fsdevel, samba-technical

On Tue, Oct 20, 2009 at 3:59 PM, Sunil Mushran <sunil.mushran@oracle.com> wrote:
> Steve French wrote:
>>
>> So, are we ready for Mingming or one of the ext4 developers to propose
>> a patch for this via xattrs (I can do a similar one for cifs).
>> Sounds like various have said:
>>
>> 1) xattrs instead of ioctl
>> 2) get of create time allowed by default, but set of create time limited
>>
>> But for the second part of this - how should we set nanosecond
>> timestamps for the others (or in Samba's case, rounded to 100
>> nanoseconds for DCE time) ... a millisecond granularity for utimes is
>> a very long time these days.
>
> What's wrong with stat(2)? AFAIK, it supports nanoseconds.

An earlier post from Mingming IIRC noted that utimes does not support
setting nanosecond timestamps (e.g. write time).

"The function utime() allows specification of time stamps with a
resolution of 1 second. ... The function utimes() is similar, but the
times argument allows a reso lution of 1 microsecond for the
timestamps. "   1 millisecond is not sufficient, and as a result Samba
has to store duplicate time stamps (in Samba specific xattrs) which we
would like to avoid wherever possible


-- 
Thanks,

Steve

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and  getting/setting nanosecond time stamps and utime
  2009-10-20 21:11                   ` Steve French
@ 2009-10-20 21:23                     ` Sunil Mushran
  2009-10-20 21:37                       ` Steve French
  2009-10-21 11:59                     ` Henrik Nordstrom
  1 sibling, 1 reply; 31+ messages in thread
From: Sunil Mushran @ 2009-10-20 21:23 UTC (permalink / raw)
  To: Steve French; +Cc: Andreas Dilger, jim owens, linux-fsdevel, samba-technical

Steve French wrote:
> On Tue, Oct 20, 2009 at 3:59 PM, Sunil Mushran <sunil.mushran@oracle.com> wrote:
>> Steve French wrote:
>>> So, are we ready for Mingming or one of the ext4 developers to propose
>>> a patch for this via xattrs (I can do a similar one for cifs).
>>> Sounds like various have said:
>>>
>>> 1) xattrs instead of ioctl
>>> 2) get of create time allowed by default, but set of create time limited
>>>
>>> But for the second part of this - how should we set nanosecond
>>> timestamps for the others (or in Samba's case, rounded to 100
>>> nanoseconds for DCE time) ... a millisecond granularity for utimes is
>>> a very long time these days.
>> What's wrong with stat(2)? AFAIK, it supports nanoseconds.
>
> An earlier post from Mingming IIRC noted that utimes does not support
> setting nanosecond timestamps (e.g. write time).
>
> "The function utime() allows specification of time stamps with a
> resolution of 1 second. ... The function utimes() is similar, but the
> times argument allows a reso lution of 1 microsecond for the
> timestamps. "   1 millisecond is not sufficient, and as a result Samba
> has to store duplicate time stamps (in Samba specific xattrs) which we
> would like to avoid wherever possible

If you are trying to update it, you can use utimensat(2).

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-20 21:23                     ` Sunil Mushran
@ 2009-10-20 21:37                       ` Steve French
  2009-10-20 21:49                         ` Sunil Mushran
  0 siblings, 1 reply; 31+ messages in thread
From: Steve French @ 2009-10-20 21:37 UTC (permalink / raw)
  To: Sunil Mushran; +Cc: jim owens, linux-fsdevel, Andreas Dilger, samba-technical

On Tue, Oct 20, 2009 at 4:23 PM, Sunil Mushran <sunil.mushran@oracle.com> wrote:
> Steve French wrote:
>>
>> On Tue, Oct 20, 2009 at 3:59 PM, Sunil Mushran <sunil.mushran@oracle.com>
>> wrote:
>>>
>>> Steve French wrote:
>>>>
>>>> So, are we ready for Mingming or one of the ext4 developers to propose
>>>> a patch for this via xattrs (I can do a similar one for cifs).
>>>> Sounds like various have said:
>>>>
>>>> 1) xattrs instead of ioctl
>>>> 2) get of create time allowed by default, but set of create time limited
>>>>
>>>> But for the second part of this - how should we set nanosecond
>>>> timestamps for the others (or in Samba's case, rounded to 100
>>>> nanoseconds for DCE time) ... a millisecond granularity for utimes is
>>>> a very long time these days.
>>>
>>> What's wrong with stat(2)? AFAIK, it supports nanoseconds.
>>
>> An earlier post from Mingming IIRC noted that utimes does not support
>> setting nanosecond timestamps (e.g. write time).
>>
>> "The function utime() allows specification of time stamps with a
>> resolution of 1 second. ... The function utimes() is similar, but the
>> times argument allows a reso lution of 1 microsecond for the
>> timestamps. "   1 millisecond is not sufficient, and as a result Samba
>> has to store duplicate time stamps (in Samba specific xattrs) which we
>> would like to avoid wherever possible
>
> If you are trying to update it, you can use utimensat(2).

Looks like that should work based on what I see in fs/utimes.c ... but
I have heard from two that think it doesn't work, so time to test it
... (unfortunately
the libc is not new enough on this system ... ugh).



-- 
Thanks,

Steve

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and  getting/setting nanosecond time stamps and utime
  2009-10-20 21:37                       ` Steve French
@ 2009-10-20 21:49                         ` Sunil Mushran
  2009-10-20 21:56                           ` Steve French
  0 siblings, 1 reply; 31+ messages in thread
From: Sunil Mushran @ 2009-10-20 21:49 UTC (permalink / raw)
  To: Steve French; +Cc: Andreas Dilger, jim owens, linux-fsdevel, samba-technical

Steve French wrote:
> On Tue, Oct 20, 2009 at 4:23 PM, Sunil Mushran <sunil.mushran@oracle.com> wrote:
>   
>> If you are trying to update it, you can use utimensat(2).
>>     
>
> Looks like that should work based on what I see in fs/utimes.c ... but
> I have heard from two that think it doesn't work, so time to test it
> ... (unfortunately
> the libc is not new enough on this system ... ugh).
>   

I just ran the following on my box. (2.6.28-15-generic #52-Ubuntu SMP)
This is on ocfs2.

$ stat DSCN1417.jpg
  File: `DSCN1417.jpg'
  Size: 60991         Blocks: 120        IO Block: 4096   regular file
Device: 807h/2055d    Inode: 9031712     Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( xxxxx/xxxxxxx)   Gid: ( xxxx/xxxxxxx)
Access: 2009-09-10 10:49:08.508507794 -0700
Modify: 2006-07-05 18:27:05.000000000 -0700
Change: 2007-05-29 15:23:16.000058283 -0700

$ touch DSCN1417.jpg

$ stat DSCN1417.jpg
  File: `DSCN1417.jpg'
  Size: 60991         Blocks: 120        IO Block: 4096   regular file
Device: 807h/2055d    Inode: 9031712     Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( xxxx/xxxxxxx)   Gid: ( xxxx/xxxxxx)
Access: 2009-10-20 14:41:51.224405489 -0700
Modify: 2009-10-20 14:41:51.224405489 -0700
Change: 2009-10-20 14:41:51.224405489 -0700


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-20 21:49                         ` Sunil Mushran
@ 2009-10-20 21:56                           ` Steve French
  2009-10-20 22:16                             ` Sunil Mushran
  0 siblings, 1 reply; 31+ messages in thread
From: Steve French @ 2009-10-20 21:56 UTC (permalink / raw)
  To: Sunil Mushran; +Cc: Andreas Dilger, jim owens, linux-fsdevel, samba-technical

On Tue, Oct 20, 2009 at 4:49 PM, Sunil Mushran <sunil.mushran@oracle.com> wrote:
> Steve French wrote:
>>
>> On Tue, Oct 20, 2009 at 4:23 PM, Sunil Mushran <sunil.mushran@oracle.com>
>> wrote:
>>
>>>
>>> If you are trying to update it, you can use utimensat(2).
>>>
>>
>> Looks like that should work based on what I see in fs/utimes.c ... but
>> I have heard from two that think it doesn't work, so time to test it
>> ... (unfortunately
>> the libc is not new enough on this system ... ugh).

strace shows that touch on my ubuntu system still calls utimesat
(touch --version displays v6.10 2008).   What version of touch are you
running?



-- 
Thanks,

Steve

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-20 21:56                           ` Steve French
@ 2009-10-20 22:16                             ` Sunil Mushran
  2009-10-21 23:45                               ` Mingming
  0 siblings, 1 reply; 31+ messages in thread
From: Sunil Mushran @ 2009-10-20 22:16 UTC (permalink / raw)
  To: Steve French; +Cc: jim owens, linux-fsdevel, Andreas Dilger, samba-technical

Steve French wrote:
> On Tue, Oct 20, 2009 at 4:49 PM, Sunil Mushran <sunil.mushran@oracle.com> wrote:
>> Steve French wrote:
>>> On Tue, Oct 20, 2009 at 4:23 PM, Sunil Mushran <sunil.mushran@oracle.com>
>>> wrote:
>>>
>>>> If you are trying to update it, you can use utimensat(2).
>>>>
>>> Looks like that should work based on what I see in fs/utimes.c ... but
>>> I have heard from two that think it doesn't work, so time to test it
>>> ... (unfortunately
>>> the libc is not new enough on this system ... ugh).
>
> strace shows that touch on my ubuntu system still calls utimesat
> (touch --version displays v6.10 2008).   What version of touch are you
> running?
$ touch --version
touch (GNU coreutils) 6.10

I am running the same and it calls utimensat(). Time to look at the source.

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-20 20:49               ` Steve French
  2009-10-20 20:59                 ` Sunil Mushran
@ 2009-10-21  0:44                 ` Andreas Dilger
  2009-10-21 23:42                   ` Mingming
  1 sibling, 1 reply; 31+ messages in thread
From: Andreas Dilger @ 2009-10-21  0:44 UTC (permalink / raw)
  To: Steve French; +Cc: jim owens, linux-fsdevel, samba-technical

[-- Attachment #1: Type: text/plain, Size: 1752 bytes --]

On 20-Oct-09, at 14:49, Steve French wrote:
> On Tue, Oct 20, 2009 at 3:33 PM, Andreas Dilger <adilger@sun.com>  
> wrote:
>> For those users/admins that want to be able to change this (due to
>> Samba, or whatever), great, we can give them this ability, but for
>> users who do NOT want regular users to be able to change the file
>> creation timestamp, that should be possible as well.
>
> So, are we ready for Mingming or one of the ext4 developers to propose
> a patch for this via xattrs (I can do a similar one for cifs).
> Sounds like various have said:
>
> 1) xattrs instead of ioctl
> 2) get of create time allowed by default, but set of create time  
> limited


A wildly untested/uncompiled patch is attached for purposes of  
discussion.
One issue that I've already come across is if we are storing a raw  
"timespec"
we will get different xattrs on disk in the fallback case where the on- 
disk
inodes are not large enough to store the crtime inside the inode.

This version of the patch will fall back to storing the timespec on disk
in the supplied size/endianness as an xattr, but it definitely needs  
to be
handled more appropriately (endian/word size).  It _would_ be nice to  
store
a full 64-bit timespec on disk, but given that the in-inode format  
only can
handle 34 bits of seconds + 30 bits of nanoseconds, it isn't clear  
whether
the "fallback" should do better than that.

The main question is whether storing a "user.crtime" as an on-disk xattr
is a desirable fallback, or if this should just fail?  Similarly, should
the lack of on-disk crtime return "0.0" as the creation time, or should
it return -ENODATA or similar?

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

[-- Attachment #2: crtime_xattr.diff --]
[-- Type: application/octet-stream, Size: 3308 bytes --]

--- ./fs/ext4/ext4.h.orig	2009-10-20 16:12:11.000000000 -0600
+++ ./fs/ext4/ext4.h	2009-10-20 16:22:23.000000000 -0600
@@ -560,6 +699,7 @@ do {									       \
 #define EXT4_MOUNT_DELALLOC		0x8000000 /* Delalloc support */
 #define EXT4_MOUNT_DATA_ERR_ABORT	0x10000000 /* Abort on file data write*/
 #define EXT4_MOUNT_BLOCK_VALIDITY	0x20000000 /* Block validity checking */
+#define EXT4_MOUNT_USER_CRTIME		0x40000000 /* Users can set crtime */
 
 #define clear_opt(o, opt)		o &= ~EXT4_MOUNT_##opt
 #define set_opt(o, opt)			o |= EXT4_MOUNT_##opt
--- ./fs/ext4/xattr_user.c.orig	2008-10-09 16:13:53.000000000 -0600
+++ ./fs/ext4/xattr_user.c	2009-10-20 16:17:23.000000000 -0600
@@ -36,8 +36,25 @@ ext4_xattr_user_get(struct inode *inode,
 {
 	if (strcmp(name, "") == 0)
 		return -EINVAL;
+
+	if (strcmp(name, "crtime") == 0 &&
+	    EXT4_FITS_IN_INODE(EXT4_I(inode), inode, i_crtime)) {
+		struct timespec *crtime = &EXT4_I(inode)->i_crtime;
+
+		if (size == 0)
+			return sizeof(*crtime);
+
+		if (size < sizeof(*crtime))
+			return -ERANGE;
+
+		memcpy(buffer, crtime, sizeof(*crtime));
+
+		return sizeof(*crtime);
+	}
+
 	if (!test_opt(inode->i_sb, XATTR_USER))
 		return -EOPNOTSUPP;
+
 	return ext4_xattr_get(inode, EXT4_XATTR_INDEX_USER, name, buffer, size);
 }
 
@@ -47,8 +64,30 @@ ext4_xattr_user_set(struct inode *inode,
 {
 	if (strcmp(name, "") == 0)
 		return -EINVAL;
+
+	/* Prevent regular users from modifying the file creation time */
+	if (strcmp(name, "crtime") == 0 &&
+	    !(capable(CAP_DAC_OVERRIDE) || test_opt(inode->i_sb, USER_CRTIME)))
+		return -EACCES;
+
+	/* If there is room in the large inode, store the create time there,
+	 * otherwise store it as a regular extended attribute. */
+	if (strcmp(name, "crtime") == 0 &&
+	    EXT4_FITS_IN_INODE(EXT4_I(inode), inode, i_crtime)) {
+		struct timespec *crtime = &EXT4_I(inode)->i_crtime;
+
+		if (size < sizeof(*crtime))
+			return -EINVAL;
+
+		memcpy(crtime, value, sizeof(*crtime));
+		ext4_dirty_inode(inode);
+
+		return sizeof(*crtime);
+	}
+
 	if (!test_opt(inode->i_sb, XATTR_USER))
 		return -EOPNOTSUPP;
+
 	return ext4_xattr_set(inode, EXT4_XATTR_INDEX_USER, name,
 			      value, size, flags);
 }
--- ./fs/ext4/super.c.orig	2009-10-20 16:11:29.000000000 -0600
+++ ./fs/ext4/super.c	2009-10-20 16:38:49.000000000 -0600
@@ -1023,6 +1049,7 @@ enum {
 	Opt_stripe, Opt_delalloc, Opt_nodelalloc,
 	Opt_block_validity, Opt_noblock_validity,
 	Opt_inode_readahead_blks, Opt_journal_ioprio,
+	Opt_user_crtime, Opt_nouser_crtime
 };
 
 static const match_table_t tokens = {
@@ -1088,6 +1116,8 @@ static const match_table_t tokens = {
 	{Opt_auto_da_alloc, "auto_da_alloc=%u"},
 	{Opt_auto_da_alloc, "auto_da_alloc"},
 	{Opt_noauto_da_alloc, "noauto_da_alloc"},
+	{Opt_user_crtime, "user_crtime" },
+	{Opt_nouser_crtime, "nouser_crtime" },
 	{Opt_err, NULL},
 };
 
@@ -1552,6 +1554,12 @@ set_qf_format:
 			else
 				set_opt(sbi->s_mount_opt,NO_AUTO_DA_ALLOC);
 			break;
+		case Opt_user_crtime:
+			set_opt(sbi->s_mount_opt, USER_CRTIME);
+			break;
+		case Opt_nouser_crtime:
+			clear_opt(sbi->s_mount_opt, USER_CRTIME);
+			break;
 		default:
 			ext4_msg(sb, KERN_ERR,
 			       "Unrecognized mount option \"%s\" "

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-20 21:11                   ` Steve French
  2009-10-20 21:23                     ` Sunil Mushran
@ 2009-10-21 11:59                     ` Henrik Nordstrom
  2009-10-21 15:36                       ` Steve French
  1 sibling, 1 reply; 31+ messages in thread
From: Henrik Nordstrom @ 2009-10-21 11:59 UTC (permalink / raw)
  To: Steve French
  Cc: Sunil Mushran, jim owens, linux-fsdevel, Andreas Dilger, samba-technical

tis 2009-10-20 klockan 16:11 -0500 skrev Steve French:

> "The function utime() allows specification of time stamps with a
> resolution of 1 second. ... The function utimes() is similar, but the
> times argument allows a reso lution of 1 microsecond for the
> timestamps. "   1 millisecond is not sufficient

But it isn't 1 millisecond, it's granulatity is 1 microsecond (1/1000
millisecond, or 1000 nanoseconds)

utime -> second (time_t), same granularity as old stat(), time() and
friends.

utimes -> microsecond (struct timeval) (1/1000000), same granularity as
gettimeofday() and friends.

utimen -> nanosecond  (struct timespec) (1/1000000000), same granularity
as current stat(), clock_gettime() and friends.

Regards
Henrik


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-21 11:59                     ` Henrik Nordstrom
@ 2009-10-21 15:36                       ` Steve French
  2009-10-21 18:56                         ` Brad Boyer
  2009-10-21 23:03                         ` Björn Jacke
  0 siblings, 2 replies; 31+ messages in thread
From: Steve French @ 2009-10-21 15:36 UTC (permalink / raw)
  To: Henrik Nordstrom
  Cc: Sunil Mushran, jim owens, linux-fsdevel, Andreas Dilger, samba-technical

On Wed, Oct 21, 2009 at 6:59 AM, Henrik Nordstrom
<henrik@henriknordstrom.net> wrote:
> utimen -> nanosecond  (struct timespec) (1/1000000000), same granularity
> as current stat(), clock_gettime() and friends.

Yes.   I have opened bug 6836 in samba bugzilla to track having smbd
use utimensat (but not sure whether they will be able to detect when
the file system can support nanosecond timestamps through - even with
use of utimensat - so there may be questions about how to do runtime
detection of whether the filesystem is one like ext4, xfs, jfs etc.
that can store such timestamps, if not they probably still have to
store 100 nanosecond "DCE time" granulariy timestamps in xattrs)

The getting (and optionally setting) create time (via xattr) is still
an open question, but am hopeful that that won't be hard to add for
ext4.
-- 
Thanks,

Steve
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-21 15:36                       ` Steve French
@ 2009-10-21 18:56                         ` Brad Boyer
  2009-10-21 23:03                         ` Björn Jacke
  1 sibling, 0 replies; 31+ messages in thread
From: Brad Boyer @ 2009-10-21 18:56 UTC (permalink / raw)
  To: Steve French
  Cc: Henrik Nordstrom, Sunil Mushran, jim owens, linux-fsdevel,
	Andreas Dilger, samba-technical

On Wed, Oct 21, 2009 at 10:36:00AM -0500, Steve French wrote:
> Yes.   I have opened bug 6836 in samba bugzilla to track having smbd
> use utimensat (but not sure whether they will be able to detect when
> the file system can support nanosecond timestamps through - even with
> use of utimensat - so there may be questions about how to do runtime
> detection of whether the filesystem is one like ext4, xfs, jfs etc.
> that can store such timestamps, if not they probably still have to
> store 100 nanosecond "DCE time" granulariy timestamps in xattrs)

I think the right way to do it would be to implement a system call
for pathconf/fpathconf and add a selector for timestamp granularity.
This obviously isn't an instant fix since it's not there already, but
I think it's the right long-term interface. We already have the
functions available in libc, but currently they don't call into the
kernel for anything. If there was a system call and an operation
somewhere in the VFS code to call into the individual drivers, then we
could export this as well as lots of other useful information. It would
also be based on a POSIX standard interface.

	Brad Boyer
	flar@allandria.com


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-21 15:36                       ` Steve French
  2009-10-21 18:56                         ` Brad Boyer
@ 2009-10-21 23:03                         ` Björn Jacke
  2009-10-22 21:50                           ` Steve French
  1 sibling, 1 reply; 31+ messages in thread
From: Björn Jacke @ 2009-10-21 23:03 UTC (permalink / raw)
  To: Steve French
  Cc: Sunil Mushran, Andreas Dilger, samba-technical, Henrik Nordstrom,
	linux-fsdevel, jim owens

[-- Attachment #1: Type: text/plain, Size: 1849 bytes --]

On 2009-10-21 at 10:36 -0500 Steve French sent off:
> On Wed, Oct 21, 2009 at 6:59 AM, Henrik Nordstrom
> <henrik@henriknordstrom.net> wrote:
> > utimen -> nanosecond  (struct timespec) (1/1000000000), same granularity
> > as current stat(), clock_gettime() and friends.
> 
> Yes.   I have opened bug 6836 in samba bugzilla to track having smbd
> use utimensat (but not sure whether they will be able to detect when
> the file system can support nanosecond timestamps through - even with
> use of utimensat - so there may be questions about how to do runtime
> detection of whether the filesystem is one like ext4, xfs, jfs etc.
> that can store such timestamps, if not they probably still have to
> store 100 nanosecond "DCE time" granulariy timestamps in xattrs)

up to these days Samba just supports the granularity the undelying filesystem
supports and I think there is no need to change this. The utimensat
documentation says: "The file's relevant timestamp shall be set to the greatest
value supported by the file system that is not greater than the specified
time." So just using this call (where available) is all Samba will have to do -
I see nothing more to care about.

Unfortunately the designers of utimensat() didn't think of setting of birth
times on systems that have it. It would have been nice if this brand new call
had covered this, too. The FreeBSD folks thought about adding a system call to
set birth time.  All they currently have is a rather hackish[1] way to modify
it timewise in one direction.

Cheers
Björn

[1] http://fuse4bsd.creo.hu/localcgi/man-cgi.cgi?utimes+2
    ... For file systems that support file birth (creation) times (such as UFS2),
    the birth time will be set to the value of the second element if the second
    element is older than the currently set birth time.


[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-21  0:44                 ` Andreas Dilger
@ 2009-10-21 23:42                   ` Mingming
  0 siblings, 0 replies; 31+ messages in thread
From: Mingming @ 2009-10-21 23:42 UTC (permalink / raw)
  To: Andreas Dilger; +Cc: Steve French, jim owens, linux-fsdevel, samba-technical

On Tue, 2009-10-20 at 18:44 -0600, Andreas Dilger wrote:
> On 20-Oct-09, at 14:49, Steve French wrote:
> > On Tue, Oct 20, 2009 at 3:33 PM, Andreas Dilger <adilger@sun.com>  
> > wrote:
> >> For those users/admins that want to be able to change this (due to
> >> Samba, or whatever), great, we can give them this ability, but for
> >> users who do NOT want regular users to be able to change the file
> >> creation timestamp, that should be possible as well.
> >
> > So, are we ready for Mingming or one of the ext4 developers to propose
> > a patch for this via xattrs (I can do a similar one for cifs).
> > Sounds like various have said:
> >
> > 1) xattrs instead of ioctl
> > 2) get of create time allowed by default, but set of create time  
> > limited
> 
> 
> A wildly untested/uncompiled patch is attached for purposes of  
> discussion.
> One issue that I've already come across is if we are storing a raw  
> "timespec"
> we will get different xattrs on disk in the fallback case where the on- 
> disk
> inodes are not large enough to store the crtime inside the inode.
> 
> This version of the patch will fall back to storing the timespec on disk
> in the supplied size/endianness as an xattr, but it definitely needs  
> to be
> handled more appropriately (endian/word size).  It _would_ be nice to  
> store
> a full 64-bit timespec on disk, but given that the in-inode format  
> only can
> handle 34 bits of seconds + 30 bits of nanoseconds, it isn't clear  
> whether
> the "fallback" should do better than that.
> 
Could we transparently convert user provided raw creation time and store
the same format as whose in-inode creation timestamp in the fallback
case? Just to be consistent. User doesn't have to know what type/size of
ext4 inode it is accessing...

> The main question is whether storing a "user.crtime" as an on-disk xattr
> is a desirable fallback, or if this should just fail?  

I think it would be nice to have this fallback, just be consistent, that
for all type of ext4 inodes there is a way to set file creation time for
allowed users(whether default 256 bytes large inode which could store
the 34+30bit creation time, or the 128 bytes smaller inode, which
doesn't have space to store the creation in the inode)


Regards,
Mingming
> Similarly, should
> the lack of on-disk crtime return "0.0" as the creation time, or should
> it return -ENODATA or similar?
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.



^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-20 22:16                             ` Sunil Mushran
@ 2009-10-21 23:45                               ` Mingming
  0 siblings, 0 replies; 31+ messages in thread
From: Mingming @ 2009-10-21 23:45 UTC (permalink / raw)
  To: Sunil Mushran
  Cc: Steve French, Andreas Dilger, jim owens, linux-fsdevel, samba-technical

On Tue, 2009-10-20 at 15:16 -0700, Sunil Mushran wrote:
> Steve French wrote:
> > On Tue, Oct 20, 2009 at 4:49 PM, Sunil Mushran <sunil.mushran@oracle.com> wrote:
> >> Steve French wrote:
> >>> On Tue, Oct 20, 2009 at 4:23 PM, Sunil Mushran <sunil.mushran@oracle.com>
> >>> wrote:
> >>>
> >>>> If you are trying to update it, you can use utimensat(2).
> >>>>
> >>> Looks like that should work based on what I see in fs/utimes.c ... but
> >>> I have heard from two that think it doesn't work, so time to test it
> >>> ... (unfortunately
> >>> the libc is not new enough on this system ... ugh).
> >
> > strace shows that touch on my ubuntu system still calls utimesat
> > (touch --version displays v6.10 2008).   What version of touch are you
> > running?
> $ touch --version
> touch (GNU coreutils) 6.10
> 
> I am running the same and it calls utimensat(). Time to look at the source.
> 

Just to confirm that I am running the same version of touch too, version
6.10, looks like it calls utimensat() and sets the nanosecond
timestamps.

Mingming
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime
  2009-10-21 23:03                         ` Björn Jacke
@ 2009-10-22 21:50                           ` Steve French
  0 siblings, 0 replies; 31+ messages in thread
From: Steve French @ 2009-10-22 21:50 UTC (permalink / raw)
  To: Steve French, Henrik Nordstrom, Sunil Mushran, jim owens, Andreas Dilger

On Wed, Oct 21, 2009 at 6:03 PM, Björn Jacke <bj@sernet.de> wrote:
> On 2009-10-21 at 10:36 -0500 Steve French sent off:
>> On Wed, Oct 21, 2009 at 6:59 AM, Henrik Nordstrom
>> <henrik@henriknordstrom.net> wrote:
>> > utimen -> nanosecond  (struct timespec) (1/1000000000), same granularity
>> > as current stat(), clock_gettime() and friends.
>>
>> Yes.   I have opened bug 6836 in samba bugzilla to track having smbd
>> use utimensat (but not sure whether they will be able to detect when
>> the file system can support nanosecond timestamps through - even with
>> use of utimensat - so there may be questions about how to do runtime
>> detection of whether the filesystem is one like ext4, xfs, jfs etc.
>> that can store such timestamps, if not they probably still have to
>> store 100 nanosecond "DCE time" granulariy timestamps in xattrs)
>
> up to these days Samba just supports the granularity the undelying filesystem
> supports and I think there is no need to change this. The utimensat
> documentation says: "The file's relevant timestamp shall be set to the greatest
> value supported by the file system that is not greater than the specified
> time." So just using this call (where available) is all Samba will have to do -
> I see nothing more to care about.

Time granularity of 1 second on ext3 is so atrociously bad, and ext3
is so common,
that I see obvious reason why Samba server would want to store 100
nanosecond time stamps (in xattrs) for this very common case.

It would be very confusing to some Windows apps to have time on files seem to
move backwards (due to rounding to 1 second granularity) - to set the
time to something and have the time when next retrieved (on average)
a 1/2 second earlier than what they just set.

-- 
Thanks,

Steve
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 31+ messages in thread

end of thread, other threads:[~2009-10-22 21:58 UTC | newest]

Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-10-19 17:17 ext4 - getting at birth time (file create time) and getting/setting nanosecond time stamps and utime Steve French
2009-10-19 18:55 ` Andreas Dilger
2009-10-19 18:58   ` Jeremy Allison
2009-10-19 19:37     ` Steve French
2009-10-19 19:12   ` Zach Brown
2009-10-19 20:49     ` Jamie Lokier
2009-10-19 19:45   ` Steve French
2009-10-19 20:11     ` Andreas Dilger
2009-10-19 22:24       ` Steve French
2009-10-19 23:12         ` Andreas Dilger
2009-10-20  3:31           ` Steve French
2009-10-20 12:44           ` jim owens
2009-10-20 20:33             ` Andreas Dilger
2009-10-20 20:49               ` Steve French
2009-10-20 20:59                 ` Sunil Mushran
2009-10-20 21:11                   ` Steve French
2009-10-20 21:23                     ` Sunil Mushran
2009-10-20 21:37                       ` Steve French
2009-10-20 21:49                         ` Sunil Mushran
2009-10-20 21:56                           ` Steve French
2009-10-20 22:16                             ` Sunil Mushran
2009-10-21 23:45                               ` Mingming
2009-10-21 11:59                     ` Henrik Nordstrom
2009-10-21 15:36                       ` Steve French
2009-10-21 18:56                         ` Brad Boyer
2009-10-21 23:03                         ` Björn Jacke
2009-10-22 21:50                           ` Steve French
2009-10-21  0:44                 ` Andreas Dilger
2009-10-21 23:42                   ` Mingming
2009-10-20 21:10               ` jim owens
2009-10-20  0:41   ` Mingming

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.