[PATCH] overlayfs: set ctime when setting mtime and atime

[PATCH] overlayfs: set ctime when setting mtime and atime

[Jeff Layton]

Nathan reported that he was seeing the new warning in
setattr_copy_mgtime pop when starting podman containers. Overlayfs is
trying to set the atime and mtime via notify_change without also
setting the ctime.

POSIX states that when the atime and mtime are updated via utimes() that
we must also update the ctime to the current time. The situation with
overlayfs copy-up is analogies, so add ATTR_CTIME to the bitmask.
notify_change will fill in the value.

Reported-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
The new WARN_ON_ONCE in setattr_copy_mgtime caught a bug! Fix up
overlayfs to ensure that the ctime on the upper inode is also updated
when copying up the atime and mtime.
---
 fs/overlayfs/copy_up.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
index d1761ec5866a..ada3fcc9c6d5 100644
--- a/fs/overlayfs/copy_up.c
+++ b/fs/overlayfs/copy_up.c
@@ -337,7 +337,7 @@ static int ovl_set_timestamps(struct ovl_fs *ofs, struct dentry *upperdentry,
 {
 	struct iattr attr = {
 		.ia_valid =
-		     ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET,
+		     ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET | ATTR_CTIME,
 		.ia_atime = stat->atime,
 		.ia_mtime = stat->mtime,
 	};

---
base-commit: 9cb8e7c86ac793862e7bea7904b3426942bbd7ef
change-id: 20230913-ctime-299173760dd9

Best regards,
-- 
Jeff Layton <jlayton@kernel.org>

Re: [PATCH] overlayfs: set ctime when setting mtime and atime

[Jeff Layton]

On Wed, 2023-09-13 at 09:33 -0400, Jeff Layton wrote:
> Nathan reported that he was seeing the new warning in
> setattr_copy_mgtime pop when starting podman containers. Overlayfs is
> trying to set the atime and mtime via notify_change without also
> setting the ctime.
> 
> POSIX states that when the atime and mtime are updated via utimes() that
> we must also update the ctime to the current time. The situation with
> overlayfs copy-up is analogies, so add ATTR_CTIME to the bitmask.

Bah...make that "analogous".

> notify_change will fill in the value.
> 
> Reported-by: Nathan Chancellor <nathan@kernel.org>
> Signed-off-by: Jeff Layton <jlayton@kernel.org>
> ---
> The new WARN_ON_ONCE in setattr_copy_mgtime caught a bug! Fix up
> overlayfs to ensure that the ctime on the upper inode is also updated
> when copying up the atime and mtime.
> ---
>  fs/overlayfs/copy_up.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
> index d1761ec5866a..ada3fcc9c6d5 100644
> --- a/fs/overlayfs/copy_up.c
> +++ b/fs/overlayfs/copy_up.c
> @@ -337,7 +337,7 @@ static int ovl_set_timestamps(struct ovl_fs *ofs, struct dentry *upperdentry,
>  {
>  	struct iattr attr = {
>  		.ia_valid =
> -		     ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET,
> +		     ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET | ATTR_CTIME,
>  		.ia_atime = stat->atime,
>  		.ia_mtime = stat->mtime,
>  	};
> 
> ---
> base-commit: 9cb8e7c86ac793862e7bea7904b3426942bbd7ef
> change-id: 20230913-ctime-299173760dd9
> 
> Best regards,

-- 
Jeff Layton <jlayton@kernel.org>

Re: [PATCH] overlayfs: set ctime when setting mtime and atime

[Christian Brauner]

On Wed, Sep 13, 2023 at 09:33:12AM -0400, Jeff Layton wrote:
> Nathan reported that he was seeing the new warning in
> setattr_copy_mgtime pop when starting podman containers. Overlayfs is
> trying to set the atime and mtime via notify_change without also
> setting the ctime.
> 
> POSIX states that when the atime and mtime are updated via utimes() that
> we must also update the ctime to the current time. The situation with
> overlayfs copy-up is analogies, so add ATTR_CTIME to the bitmask.
> notify_change will fill in the value.
> 
> Reported-by: Nathan Chancellor <nathan@kernel.org>
> Signed-off-by: Jeff Layton <jlayton@kernel.org>
> ---

Looks good to me,
Acked-by: Christian Brauner <brauner@kernel.org>

So we can wait for ovl to upstream this fix next and then we'll delay
sending the ctime fixes or we'll take this fixup as well. Just let me
know what you all prefer.

Re: [PATCH] overlayfs: set ctime when setting mtime and atime

[Jeff Layton]

On Wed, 2023-09-13 at 18:45 +0200, Christian Brauner wrote:
> On Wed, Sep 13, 2023 at 09:33:12AM -0400, Jeff Layton wrote:
> > Nathan reported that he was seeing the new warning in
> > setattr_copy_mgtime pop when starting podman containers. Overlayfs is
> > trying to set the atime and mtime via notify_change without also
> > setting the ctime.
> > 
> > POSIX states that when the atime and mtime are updated via utimes() that
> > we must also update the ctime to the current time. The situation with
> > overlayfs copy-up is analogies, so add ATTR_CTIME to the bitmask.
> > notify_change will fill in the value.
> > 
> > Reported-by: Nathan Chancellor <nathan@kernel.org>
> > Signed-off-by: Jeff Layton <jlayton@kernel.org>
> > ---
> 
> Looks good to me,
> Acked-by: Christian Brauner <brauner@kernel.org>
> 
> So we can wait for ovl to upstream this fix next and then we'll delay
> sending the ctime fixes or we'll take this fixup as well. Just let me
> know what you all prefer.

No preference here.
-- 
Jeff Layton <jlayton@kernel.org>

Re: [PATCH] overlayfs: set ctime when setting mtime and atime

[Amir Goldstein]

On Wed, Sep 13, 2023 at 4:33 PM Jeff Layton <jlayton@kernel.org> wrote:
>
> Nathan reported that he was seeing the new warning in
> setattr_copy_mgtime pop when starting podman containers. Overlayfs is
> trying to set the atime and mtime via notify_change without also
> setting the ctime.
>
> POSIX states that when the atime and mtime are updated via utimes() that
> we must also update the ctime to the current time. The situation with
> overlayfs copy-up is analogous, so add ATTR_CTIME to the bitmask.
> notify_change will fill in the value.
>

IDGI, if ctime always needs to be set along with ATIME / MTIME, why not
let notify_change() set the bit instead of assert and fix all the callers?
But maybe I am missing something.

Anyway, I have no objection to the ovl patch.
It's fine by me if Christian applies it to the vfs.ctime branch with my ACK.

Thanks,
Amir.

> Reported-by: Nathan Chancellor <nathan@kernel.org>
> Signed-off-by: Jeff Layton <jlayton@kernel.org>
> ---
> The new WARN_ON_ONCE in setattr_copy_mgtime caught a bug! Fix up
> overlayfs to ensure that the ctime on the upper inode is also updated
> when copying up the atime and mtime.
> ---
>  fs/overlayfs/copy_up.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
> index d1761ec5866a..ada3fcc9c6d5 100644
> --- a/fs/overlayfs/copy_up.c
> +++ b/fs/overlayfs/copy_up.c
> @@ -337,7 +337,7 @@ static int ovl_set_timestamps(struct ovl_fs *ofs, struct dentry *upperdentry,
>  {
>         struct iattr attr = {
>                 .ia_valid =
> -                    ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET,
> +                    ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET | ATTR_CTIME,
>                 .ia_atime = stat->atime,
>                 .ia_mtime = stat->mtime,
>         };
>
> ---
> base-commit: 9cb8e7c86ac793862e7bea7904b3426942bbd7ef
> change-id: 20230913-ctime-299173760dd9
>
> Best regards,
> --
> Jeff Layton <jlayton@kernel.org>
>

Re: [PATCH] overlayfs: set ctime when setting mtime and atime

[Nathan Chancellor]

On Wed, Sep 13, 2023 at 09:33:12AM -0400, Jeff Layton wrote:
> Nathan reported that he was seeing the new warning in
> setattr_copy_mgtime pop when starting podman containers. Overlayfs is
> trying to set the atime and mtime via notify_change without also
> setting the ctime.
> 
> POSIX states that when the atime and mtime are updated via utimes() that
> we must also update the ctime to the current time. The situation with
> overlayfs copy-up is analogies, so add ATTR_CTIME to the bitmask.
> notify_change will fill in the value.
> 
> Reported-by: Nathan Chancellor <nathan@kernel.org>
> Signed-off-by: Jeff Layton <jlayton@kernel.org>

I applied this patch on top of next-20230913 and I do not see the
warning I reported on any of my machines. Thanks for the quick fix!

Tested-by: Nathan Chancellor <nathan@kernel.org>

> ---
> The new WARN_ON_ONCE in setattr_copy_mgtime caught a bug! Fix up
> overlayfs to ensure that the ctime on the upper inode is also updated
> when copying up the atime and mtime.
> ---
>  fs/overlayfs/copy_up.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
> index d1761ec5866a..ada3fcc9c6d5 100644
> --- a/fs/overlayfs/copy_up.c
> +++ b/fs/overlayfs/copy_up.c
> @@ -337,7 +337,7 @@ static int ovl_set_timestamps(struct ovl_fs *ofs, struct dentry *upperdentry,
>  {
>  	struct iattr attr = {
>  		.ia_valid =
> -		     ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET,
> +		     ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET | ATTR_CTIME,
>  		.ia_atime = stat->atime,
>  		.ia_mtime = stat->mtime,
>  	};
> 
> ---
> base-commit: 9cb8e7c86ac793862e7bea7904b3426942bbd7ef
> change-id: 20230913-ctime-299173760dd9
> 
> Best regards,
> -- 
> Jeff Layton <jlayton@kernel.org>
> 

Re: [PATCH] overlayfs: set ctime when setting mtime and atime

[Christian Brauner]

On Wed, 13 Sep 2023 09:33:12 -0400, Jeff Layton wrote:
> Nathan reported that he was seeing the new warning in
> setattr_copy_mgtime pop when starting podman containers. Overlayfs is
> trying to set the atime and mtime via notify_change without also
> setting the ctime.
> 
> POSIX states that when the atime and mtime are updated via utimes() that
> we must also update the ctime to the current time. The situation with
> overlayfs copy-up is analogies, so add ATTR_CTIME to the bitmask.
> notify_change will fill in the value.
> 
> [...]

Applied to the vfs.ctime branch of the vfs/vfs.git tree.
Patches in the vfs.ctime branch should appear in linux-next soon.

Please report any outstanding bugs that were missed during review in a
new review to the original patch series allowing us to drop it.

It's encouraged to provide Acked-bys and Reviewed-bys even though the
patch has now been applied. If possible patch trailers will be updated.

Note that commit hashes shown below are subject to change due to rebase,
trailer updates or similar. If in doubt, please check the listed branch.

tree:   https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git
branch: vfs.ctime

[1/1] overlayfs: set ctime when setting mtime and atime
      https://git.kernel.org/vfs/vfs/c/f8edd3368615

Re: [PATCH] overlayfs: set ctime when setting mtime and atime

[Jeff Layton]

On Wed, 2023-09-13 at 20:36 +0300, Amir Goldstein wrote:
> On Wed, Sep 13, 2023 at 4:33 PM Jeff Layton <jlayton@kernel.org> wrote:
> > 
> > Nathan reported that he was seeing the new warning in
> > setattr_copy_mgtime pop when starting podman containers. Overlayfs is
> > trying to set the atime and mtime via notify_change without also
> > setting the ctime.
> > 
> > POSIX states that when the atime and mtime are updated via utimes() that
> > we must also update the ctime to the current time. The situation with
> > overlayfs copy-up is analogous, so add ATTR_CTIME to the bitmask.
> > notify_change will fill in the value.
> > 
> 
> IDGI, if ctime always needs to be set along with ATIME / MTIME, why not
> let notify_change() set the bit instead of assert and fix all the callers?
> But maybe I am missing something.
> 

Traditionally notify_change has always been given an explicit mask of
attrs to change by the caller. I'm a little hesitant to start putting
POSIX policy in there.

Still, that may be the better thing to do over the long haul. I think
that there are some other bugs in the notify_change callers as well: for
instance, cachefiles_adjust_size truncates files, but doesn't update the
timestamps. I'm pretty sure that's wrong.

I think if we want to change how setattr ctime updates work, we'll
probably need to do it in the context of a larger notify_change
overhaul.

> Anyway, I have no objection to the ovl patch.
> It's fine by me if Christian applies it to the vfs.ctime branch with my ACK.
> 

Many thanks!

> Thanks,
> Amir.
> 
> > Reported-by: Nathan Chancellor <nathan@kernel.org>
> > Signed-off-by: Jeff Layton <jlayton@kernel.org>
> > ---
> > The new WARN_ON_ONCE in setattr_copy_mgtime caught a bug! Fix up
> > overlayfs to ensure that the ctime on the upper inode is also updated
> > when copying up the atime and mtime.
> > ---
> >  fs/overlayfs/copy_up.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
> > index d1761ec5866a..ada3fcc9c6d5 100644
> > --- a/fs/overlayfs/copy_up.c
> > +++ b/fs/overlayfs/copy_up.c
> > @@ -337,7 +337,7 @@ static int ovl_set_timestamps(struct ovl_fs *ofs, struct dentry *upperdentry,
> >  {
> >         struct iattr attr = {
> >                 .ia_valid =
> > -                    ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET,
> > +                    ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET | ATTR_CTIME,
> >                 .ia_atime = stat->atime,
> >                 .ia_mtime = stat->mtime,
> >         };
> > 
> > ---
> > base-commit: 9cb8e7c86ac793862e7bea7904b3426942bbd7ef
> > change-id: 20230913-ctime-299173760dd9
> > 
> > Best regards,
> > --
> > Jeff Layton <jlayton@kernel.org>
> > 

-- 
Jeff Layton <jlayton@kernel.org>

Re: [PATCH] overlayfs: set ctime when setting mtime and atime

[Jeff Layton]

On Thu, 2023-09-14 at 10:39 +0200, Christian Brauner wrote:
> On Wed, 13 Sep 2023 09:33:12 -0400, Jeff Layton wrote:
> > Nathan reported that he was seeing the new warning in
> > setattr_copy_mgtime pop when starting podman containers. Overlayfs is
> > trying to set the atime and mtime via notify_change without also
> > setting the ctime.
> > 
> > POSIX states that when the atime and mtime are updated via utimes() that
> > we must also update the ctime to the current time. The situation with
> > overlayfs copy-up is analogies, so add ATTR_CTIME to the bitmask.
> > notify_change will fill in the value.
> > 
> > [...]
> 
> Applied to the vfs.ctime branch of the vfs/vfs.git tree.
> Patches in the vfs.ctime branch should appear in linux-next soon.
> 
> Please report any outstanding bugs that were missed during review in a
> new review to the original patch series allowing us to drop it.
> 
> It's encouraged to provide Acked-bys and Reviewed-bys even though the
> patch has now been applied. If possible patch trailers will be updated.
> 
> Note that commit hashes shown below are subject to change due to rebase,
> trailer updates or similar. If in doubt, please check the listed branch.
> 
> tree:   https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git
> branch: vfs.ctime
> 
> [1/1] overlayfs: set ctime when setting mtime and atime
>       https://git.kernel.org/vfs/vfs/c/f8edd3368615

Christian, are you still planning to pick up this patch? I saw that it
was dropped from linux-next. Since the mgtime patches have been reverted
for now, it may be best for this to go in via the overlayfs tree ?

Thanks,
-- 
Jeff Layton <jlayton@kernel.org>

Re: [PATCH] overlayfs: set ctime when setting mtime and atime

[Amir Goldstein]

On Fri, Sep 22, 2023 at 4:52 PM Jeff Layton <jlayton@kernel.org> wrote:
>
> On Thu, 2023-09-14 at 10:39 +0200, Christian Brauner wrote:
> > On Wed, 13 Sep 2023 09:33:12 -0400, Jeff Layton wrote:
> > > Nathan reported that he was seeing the new warning in
> > > setattr_copy_mgtime pop when starting podman containers. Overlayfs is
> > > trying to set the atime and mtime via notify_change without also
> > > setting the ctime.
> > >
> > > POSIX states that when the atime and mtime are updated via utimes() that
> > > we must also update the ctime to the current time. The situation with
> > > overlayfs copy-up is analogies, so add ATTR_CTIME to the bitmask.
> > > notify_change will fill in the value.
> > >
> > > [...]
> >
> > Applied to the vfs.ctime branch of the vfs/vfs.git tree.
> > Patches in the vfs.ctime branch should appear in linux-next soon.
> >
> > Please report any outstanding bugs that were missed during review in a
> > new review to the original patch series allowing us to drop it.
> >
> > It's encouraged to provide Acked-bys and Reviewed-bys even though the
> > patch has now been applied. If possible patch trailers will be updated.
> >
> > Note that commit hashes shown below are subject to change due to rebase,
> > trailer updates or similar. If in doubt, please check the listed branch.
> >
> > tree:   https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git
> > branch: vfs.ctime
> >
> > [1/1] overlayfs: set ctime when setting mtime and atime
> >       https://git.kernel.org/vfs/vfs/c/f8edd3368615
>
> Christian, are you still planning to pick up this patch? I saw that it
> was dropped from linux-next. Since the mgtime patches have been reverted
> for now, it may be best for this to go in via the overlayfs tree ?

I think this is a long standing overlayfs bug fix, so it should go into 6.6
and not wait for 6.7 anyway.
Also need to add CC stable (don't think we need to bother with Fixes).

I do not have any overlayfs fixes queued ATM.
Christian, if you have any pening VFS fixes, you may send it along with them
or we could just ask Linus to apply this one directly, so that any mgtime
changes that may still happen for 6.7 will already have this fluke fixed.

Thanks,
Amir.

Re: [PATCH] overlayfs: set ctime when setting mtime and atime

[Christian Brauner]

> Christian, if you have any pening VFS fixes, you may send it along with them
> or we could just ask Linus to apply this one directly, so that any mgtime
> changes that may still happen for 6.7 will already have this fluke fixed.

I'll move the fix from vfs.misc into vfs.fixes now and then send it with
the pile of other fixes to Linus this week.