[GIT PULL] vfs: Add support for timestamp limits

[GIT PULL] vfs: Add support for timestamp limits

[Deepa Dinamani]

Hi Al, Arnd,

This is a pull request for filling in min and max timestamps for filesystems.
I've added all the acks, and dropped the adfs patch. That will be merged through
Russell's tree.

Thanks,
Deepa

The following changes since commit 5d18cb62218608a1388858880ad3ec76d6cb0d3b:

  Add linux-next specific files for 20190828 (2019-08-28 19:59:14 +1000)

are available in the Git repository at:

  https://github.com/deepa-hub/vfs limits

for you to fetch changes up to f0f216afa4c7e4dee9121fde52ccf57f76119188:

  isofs: Initialize filesystem timestamp ranges (2019-08-28 19:19:36 -0700)

----------------------------------------------------------------
Deepa Dinamani (19):
      vfs: Add file timestamp range support
      vfs: Add timestamp_truncate() api
      timestamp_truncate: Replace users of timespec64_trunc
      mount: Add mount warning for impending timestamp expiry
      utimes: Clamp the timestamps before update
      fs: Fill in max and min timestamps in superblock
      9p: Fill min and max timestamps in sb
      ext4: Initialize timestamps limits
      fs: nfs: Initialize filesystem timestamp ranges
      fs: cifs: Initialize filesystem timestamp ranges
      fs: fat: Initialize filesystem timestamp ranges
      fs: affs: Initialize filesystem timestamp ranges
      fs: sysv: Initialize filesystem timestamp ranges
      fs: ceph: Initialize filesystem timestamp ranges
      fs: orangefs: Initialize filesystem timestamp ranges
      fs: hpfs: Initialize filesystem timestamp ranges
      fs: omfs: Initialize filesystem timestamp ranges
      pstore: fs superblock limits
      isofs: Initialize filesystem timestamp ranges

 fs/9p/vfs_super.c        |  6 +++++-
 fs/affs/amigaffs.c       |  2 +-
 fs/affs/amigaffs.h       |  3 +++
 fs/affs/inode.c          |  4 ++--
 fs/affs/super.c          |  4 ++++
 fs/attr.c                | 21 ++++++++++++---------
 fs/befs/linuxvfs.c       |  2 ++
 fs/bfs/inode.c           |  2 ++
 fs/ceph/super.c          |  2 ++
 fs/cifs/cifsfs.c         | 22 ++++++++++++++++++++++
 fs/cifs/netmisc.c        | 14 +++++++-------
 fs/coda/inode.c          |  3 +++
 fs/configfs/inode.c      | 12 ++++++------
 fs/cramfs/inode.c        |  2 ++
 fs/efs/super.c           |  2 ++
 fs/ext2/super.c          |  2 ++
 fs/ext4/ext4.h           | 10 +++++++++-
 fs/ext4/super.c          | 17 +++++++++++++++--
 fs/f2fs/file.c           | 21 ++++++++++++---------
 fs/fat/inode.c           | 12 ++++++++++++
 fs/freevxfs/vxfs_super.c |  2 ++
 fs/hpfs/hpfs_fn.h        |  6 ++----
 fs/hpfs/super.c          |  2 ++
 fs/inode.c               | 33 ++++++++++++++++++++++++++++++++-
 fs/isofs/inode.c         |  7 +++++++
 fs/jffs2/fs.c            |  3 +++
 fs/jfs/super.c           |  2 ++
 fs/kernfs/inode.c        |  7 +++----
 fs/minix/inode.c         |  2 ++
 fs/namespace.c           | 33 ++++++++++++++++++++++++++++++++-
 fs/nfs/super.c           | 20 +++++++++++++++++++-
 fs/ntfs/inode.c          | 21 ++++++++++++---------
 fs/omfs/inode.c          |  4 ++++
 fs/orangefs/super.c      |  2 ++
 fs/pstore/ram.c          |  2 ++
 fs/qnx4/inode.c          |  2 ++
 fs/qnx6/inode.c          |  2 ++
 fs/reiserfs/super.c      |  3 +++
 fs/romfs/super.c         |  2 ++
 fs/squashfs/super.c      |  2 ++
 fs/super.c               |  2 ++
 fs/sysv/super.c          |  5 ++++-
 fs/ubifs/file.c          | 21 ++++++++++++---------
 fs/ufs/super.c           |  7 +++++++
 fs/utimes.c              |  6 ++----
 fs/xfs/xfs_super.c       |  2 ++
 include/linux/fs.h       |  5 +++++
 include/linux/time64.h   |  2 ++
 48 files changed, 298 insertions(+), 72 deletions(-)

Re: [GIT PULL] vfs: Add support for timestamp limits

[Arnd Bergmann]

On Thu, Aug 29, 2019 at 6:12 AM Deepa Dinamani <deepa.kernel@gmail.com> wrote:
>
> Hi Al, Arnd,
>
> This is a pull request for filling in min and max timestamps for filesystems.
> I've added all the acks, and dropped the adfs patch. That will be merged through
> Russell's tree.
>
> Thanks,
> Deepa
>
> The following changes since commit 5d18cb62218608a1388858880ad3ec76d6cb0d3b:
>
>   Add linux-next specific files for 20190828 (2019-08-28 19:59:14 +1000)
>
> are available in the Git repository at:
>
>   https://github.com/deepa-hub/vfs limits

Please rebase this branch on top of linux-5.3-rc6 and resend.
I can't pull a branch that contains linux-next.

Maybe drop the orangefs patch for now, at least until we have come
to a conclusion on that.

       Arnd

[GIT PULL RESEND] vfs: Add support for timestamp limits

[Deepa Dinamani]

[resending, rebased onto linux v5.3-rc6, and dropped orangefs patch from the series]

Hi Al, Arnd,

This is a pull request for filling in min and max timestamps for filesystems.
I've added all the acks, and dropped the adfs patch. That will be merged through
Russell's tree.

Dropped orangefs until the maintainers decide what its limits should be.

The following changes since commit a55aa89aab90fae7c815b0551b07be37db359d76:

  Linux 5.3-rc6 (2019-08-25 12:01:23 -0700)

are available in the Git repository at:

  https://github.com/deepa-hub/vfs limits

for you to fetch changes up to 5ad32b3acded06183f40806f76b030c3143017bb:

  isofs: Initialize filesystem timestamp ranges (2019-08-30 08:11:25 -0700)

----------------------------------------------------------------

- Deepa

Deepa Dinamani (18):
      vfs: Add file timestamp range support
      vfs: Add timestamp_truncate() api
      timestamp_truncate: Replace users of timespec64_trunc
      mount: Add mount warning for impending timestamp expiry
      utimes: Clamp the timestamps before update
      fs: Fill in max and min timestamps in superblock
      9p: Fill min and max timestamps in sb
      ext4: Initialize timestamps limits
      fs: nfs: Initialize filesystem timestamp ranges
      fs: cifs: Initialize filesystem timestamp ranges
      fs: fat: Initialize filesystem timestamp ranges
      fs: affs: Initialize filesystem timestamp ranges
      fs: sysv: Initialize filesystem timestamp ranges
      fs: ceph: Initialize filesystem timestamp ranges
      fs: hpfs: Initialize filesystem timestamp ranges
      fs: omfs: Initialize filesystem timestamp ranges
      pstore: fs superblock limits
      isofs: Initialize filesystem timestamp ranges

 fs/9p/vfs_super.c        |  6 +++++-
 fs/affs/amigaffs.c       |  2 +-
 fs/affs/amigaffs.h       |  3 +++
 fs/affs/inode.c          |  4 ++--
 fs/affs/super.c          |  4 ++++
 fs/attr.c                | 21 ++++++++++++---------
 fs/befs/linuxvfs.c       |  2 ++
 fs/bfs/inode.c           |  2 ++
 fs/ceph/super.c          |  2 ++
 fs/cifs/cifsfs.c         | 22 ++++++++++++++++++++++
 fs/cifs/netmisc.c        | 14 +++++++-------
 fs/coda/inode.c          |  3 +++
 fs/configfs/inode.c      | 12 ++++++------
 fs/cramfs/inode.c        |  2 ++
 fs/efs/super.c           |  2 ++
 fs/ext2/super.c          |  2 ++
 fs/ext4/ext4.h           | 10 +++++++++-
 fs/ext4/super.c          | 17 +++++++++++++++--
 fs/f2fs/file.c           | 21 ++++++++++++---------
 fs/fat/inode.c           | 12 ++++++++++++
 fs/freevxfs/vxfs_super.c |  2 ++
 fs/hpfs/hpfs_fn.h        |  6 ++----
 fs/hpfs/super.c          |  2 ++
 fs/inode.c               | 33 ++++++++++++++++++++++++++++++++-
 fs/isofs/inode.c         |  7 +++++++
 fs/jffs2/fs.c            |  3 +++
 fs/jfs/super.c           |  2 ++
 fs/kernfs/inode.c        |  7 +++----
 fs/minix/inode.c         |  2 ++
 fs/namespace.c           | 33 ++++++++++++++++++++++++++++++++-
 fs/nfs/super.c           | 20 +++++++++++++++++++-
 fs/ntfs/inode.c          | 21 ++++++++++++---------
 fs/omfs/inode.c          |  4 ++++
 fs/pstore/ram.c          |  2 ++
 fs/qnx4/inode.c          |  2 ++
 fs/qnx6/inode.c          |  2 ++
 fs/reiserfs/super.c      |  3 +++
 fs/romfs/super.c         |  2 ++
 fs/squashfs/super.c      |  2 ++
 fs/super.c               |  2 ++
 fs/sysv/super.c          |  5 ++++-
 fs/ubifs/file.c          | 21 ++++++++++++---------
 fs/ufs/super.c           |  7 +++++++
 fs/utimes.c              |  6 ++----
 fs/xfs/xfs_super.c       |  2 ++
 include/linux/fs.h       |  5 +++++
 include/linux/time64.h   |  2 ++
 47 files changed, 296 insertions(+), 72 deletions(-)

Re: [GIT PULL] vfs: Add support for timestamp limits

[Mike Marshall]

I spoke to Walt Ligon about the versioning code, and also shared
this thread with him. He isn't a fan of the versioning code.
and we think it should go. As I read through the commit messages
from when the versioning code was added, it relates to mtime
on directories. If a directory is read, and it has enough entries,
it might take several operations to collect all the entries. During
this time the directory might change. The versioning is a way to
tell that something changed between one of the operations...

   commit: 7878027e9c2 (Oct 2004)
     - added a directory version that is passed back from the server to
       the client on each successful readdir call (happens to be the
       directory's mtime encoded as an opaque uint64_t)

We will work to see if we can figure out what we need to do to Orangefs
on both the userspace side and the kernel module side to have all 64 bit
time values.

I've also read up some on the y2038 cleanup work that Arnd and Deepa
have been doing...

Thanks to Arnd and Deepa for looking so deeply into the Orangefs
userspace code...

-Mike

On Sat, Aug 31, 2019 at 6:50 PM Deepa Dinamani <deepa.kernel@gmail.com> wrote:
>
> > I think it's unclear from the orangefs source code what the intention is,
> > as there is a mixed of signed and unsigned types used for the inode
> > stamps:
> >
> > #define encode_PVFS_time encode_int64_t
> > #define encode_int64_t(pptr,x) do { \
> >     *(int64_t*) *(pptr) = cpu_to_le64(*(x)); \
> >     *(pptr) += 8; \
> > } while (0)
> > #define decode_PVFS_time decode_int64_t
> > #define decode_int64_t(pptr,x) do { \
> >     *(x) = le64_to_cpu(*(int64_t*) *(pptr)); \
> >     *(pptr) += 8; \
> > } while (0)
> >
> > This suggests that making it unsigned may have been an accident.
> >
> > Then again,  it's clearly and consistently printed as unsigned in
> > user space:
> >
> >         gossip_debug(
> >             GOSSIP_GETATTR_DEBUG, " VERSION is %llu, mtime is %llu\n",
> >             llu(s_op->attr.mtime), llu(resp_attr->mtime));
>
> I think I had noticed these two and decided maybe the intention was to
> use unsigned types.
>
> > A related issue I noticed is this:
> >
> > PVFS_time PINT_util_mktime_version(PVFS_time time)
> > {
> >     struct timeval t = {0,0};
> >     PVFS_time version = (time << 32);
> >
> >     gettimeofday(&t, NULL);
> >     version |= (PVFS_time)t.tv_usec;
> >     return version;
> > }
> > PVFS_time PINT_util_mkversion_time(PVFS_time version)
> > {
> >     return (PVFS_time)(version >> 32);
> > }
> > static PINT_sm_action getattr_verify_attribs(
> >         struct PINT_smcb *smcb, job_status_s *js_p)
> > {
> > ...
> >     resp_attr->mtime = PINT_util_mkversion_time(s_op->attr.mtime);
> > ...
> > }
> >
> > which suggests that at least for some purposes, the mtime field
> > is only an unsigned 32-bit number (1970..2106). From my readiing,
> > this affects the on-disk format, but not the protocol implemented
> > by the kernel.
> >
> > atime and ctime are apparently 64-bit, but mtime is only 32-bit
> > seconds, plus a 32-bit 'version'. I suppose the server could be
> > fixed to allow a larger range, but probably would take it out of
> > the 'version' bits, not the upper half.
>
> I had missed this part. Thanks.
>
> > To be on the safe side, I suppose the kernel can only assume
> > an unsigned 32-bit range to be available. If the server gets
> > extended beyond that, it would have to pass a feature flag.
>
> This makes sense to me also. And, as Arnd pointed out on the IRC, if
> there are negative timestamps that are already in use, this will be a
> problem for those use cases.
> I can update tha patch to use limits 0-u32_max.
>
> -Deepa

Re: [GIT PULL] vfs: Add support for timestamp limits

[Arnd Bergmann]

On Thu, Sep 5, 2019 at 6:58 PM Mike Marshall <hubcap@omnibond.com> wrote:
>
> I spoke to Walt Ligon about the versioning code, and also shared
> this thread with him. He isn't a fan of the versioning code.
> and we think it should go. As I read through the commit messages
> from when the versioning code was added, it relates to mtime
> on directories. If a directory is read, and it has enough entries,
> it might take several operations to collect all the entries. During
> this time the directory might change. The versioning is a way to
> tell that something changed between one of the operations...
>
>    commit: 7878027e9c2 (Oct 2004)
>      - added a directory version that is passed back from the server to
>        the client on each successful readdir call (happens to be the
>        directory's mtime encoded as an opaque uint64_t)
>
> We will work to see if we can figure out what we need to do to Orangefs
> on both the userspace side and the kernel module side to have all 64 bit
> time values.

Ok, sounds good. For the time being, I have applied the patch
that limits the kernel to timestamps in the 1970 to 2106 range, which
is compatible with the existing user space and will be good enough for
a while.

If you can ignore the old pre-versioning interfaces, you can decide to
encode the epoch number in the remaining 12 bits of the on-disk
representation, like

    time64 = (u32)(timeversion >> 32) + ((s64)(timeversion & 0xffc00000) << 12);

to extend it into the far-enough future (136 years times 2^12), or possibly
using

    time64 = (s32)(timeversion >> 32) + ((s64)(timeversion & 0xffc00000) << 12);

to interpret existing timestamps with the msb set as dates between 1902
and 1970, which would fix the test case that broke, but disallow dates past
2038 with unmodified kernels.

     Arnd