git checkout behaviour when only ctime of file changes

git checkout behaviour when only ctime of file changes

[Cristian Morales Vega]

I thought you may want to take a look at
https://issues.jenkins.io/browse/JENKINS-65395.

Basically after something updates the ctime of a file, a
"git checkout" can behave differently depending on whether a
"git update-index --refresh" has been run before or not.
_Maybe_ it could make sense for "git checkout" to always behave as if
"git update-index --refresh" had been run before? No idea really.

I have seen this with git version 2.30.2.

Re: git checkout behaviour when only ctime of file changes

[Bagas Sanjaya]

On 19/04/21 16.11, Cristian Morales Vega wrote:
> Basically after something updates the ctime of a file, a
> "git checkout" can behave differently depending on whether a
> "git update-index --refresh" has been run before or not.
> _Maybe_ it could make sense for "git checkout" to always behave as if
> "git update-index --refresh" had been run before? No idea really.
I know that setting SELinux label can alter file ctime, but when
I need to write the test that simulate file ctime update, do you
know which Unix command can trigger that, beside SELinux tools?

-- 
An old man doll... just what I always wanted! - Clara

Re: git checkout behaviour when only ctime of file changes

[Cristian Morales Vega]

On Mon, 19 Apr 2021 at 12:10, Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>
> On 19/04/21 16.11, Cristian Morales Vega wrote:
> > Basically after something updates the ctime of a file, a
> > "git checkout" can behave differently depending on whether a
> > "git update-index --refresh" has been run before or not.
> > _Maybe_ it could make sense for "git checkout" to always behave as if
> > "git update-index --refresh" had been run before? No idea really.
> I know that setting SELinux label can alter file ctime, but when
> I need to write the test that simulate file ctime update, do you
> know which Unix command can trigger that, beside SELinux tools?

I am no expert on SELinux and I'm not sure I understand the question.
I know "chcon", from coreutils, can do the SELinux labeling/ctime
update. You could use that when writing a test. There is an example on
the fifth comment of JENKINS-65395.
I know "docker run --volume <host-dir>:<container-dir>:rw,z" will do
the same, at least if SELinux is enabled. Which is what triggers
JENKINS-65395.
Don't really know a lot more.

Re: git checkout behaviour when only ctime of file changes

[Junio C Hamano]

Cristian Morales Vega <christian.morales.vega@gmail.com> writes:

> I thought you may want to take a look at
> https://issues.jenkins.io/browse/JENKINS-65395.
>
> Basically after something updates the ctime of a file, a
> "git checkout" can behave differently depending on whether a
> "git update-index --refresh" has been run before or not.

Changes to ctime, not just to mtime, participate in "the path may
have been changed so need to go back to the contents to see if it
truly has" logic, and "update-index --refresh" is a mechanism to
clear that "the path may have been changed" bit.  

So if your "behave differently ... with or without refresh" happens
equally if you replace 'ctime' in your statement with 'mtime', then
everything is working as designed.  But it was unclear from your
description.

Assuming that the difference with and without refresh after mtime
change is the same as the difference with and without refresh after
ctime change, the "core.trustctime" configuration variable is for
those whose files have untrustworthy ctime (due to background
scanners and other funnies).

Re: git checkout behaviour when only ctime of file changes

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2000 bytes --]

On 2021-04-19 at 09:11:12, Cristian Morales Vega wrote:
> I thought you may want to take a look at
> https://issues.jenkins.io/browse/JENKINS-65395.
> 
> Basically after something updates the ctime of a file, a
> "git checkout" can behave differently depending on whether a
> "git update-index --refresh" has been run before or not.
> _Maybe_ it could make sense for "git checkout" to always behave as if
> "git update-index --refresh" had been run before? No idea really.

I believe the situation you're seeing is that git checkout usually
doesn't rewrite files in the working tree that are already up to date.
This makes checkout much faster in large working trees.

By default, Git does include the ctime in its computation of whether a
file is up to date.  If the ctime changes, then the file is considered
to be stale.  git checkout, without an intervening command, will
overwrite it, since it's dirty and just overwriting it is cheaper than
determining whether it is in fact up to date.

git update-index --refresh does the check about whether the file is in
fact up to date.  This isn't free; if you've modified 1000 files, or if
you've modified large files, there's going to be a cost to this, and it
can be very substantial.  As mentioned, git checkout doesn't want to
perform that cost needlessly, so it overwrites the file unconditionally.

As far as I'm aware, Git doesn't document which files it does and does
not rewrite in the working tree as part of a checkout, except as part of
certain specified options (e.g., the --overlay option).  It is therefore
free to choose any algorithm it chooses, and in this case, we've
optimized for checkout performance, not minimal modifications.

There is a config option, core.trustctime, which can be set to false if
you don't want to consider the ctime as part of whether a file is up to
date.  But Git is behaving correctly and as desired here.
-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: git checkout behaviour when only ctime of file changes

[Cristian Morales Vega]

On Tue, 20 Apr 2021 at 03:05, brian m. carlson
<sandals@crustytoothpaste.net> wrote:
>
> On 2021-04-19 at 09:11:12, Cristian Morales Vega wrote:
> > I thought you may want to take a look at
> > https://issues.jenkins.io/browse/JENKINS-65395.
> >
> > Basically after something updates the ctime of a file, a
> > "git checkout" can behave differently depending on whether a
> > "git update-index --refresh" has been run before or not.
> > _Maybe_ it could make sense for "git checkout" to always behave as if
> > "git update-index --refresh" had been run before? No idea really.
>
> I believe the situation you're seeing is that git checkout usually
> doesn't rewrite files in the working tree that are already up to date.
> This makes checkout much faster in large working trees.
>
> By default, Git does include the ctime in its computation of whether a
> file is up to date.  If the ctime changes, then the file is considered
> to be stale.  git checkout, without an intervening command, will
> overwrite it, since it's dirty and just overwriting it is cheaper than
> determining whether it is in fact up to date.

Yes, this is basically it. The "just overwriting it is cheaper than
determining whether it is in fact up to date" is the main thing. I was
thinking that maybe, if ctime has changed but mtime hasn't, it could
be faster to not overwrite the files since I _think_ it means the file
contents don't need to be checked, only the basic file permissions (I
don't think git stores any extended attributes, does it?). But I could
be completely wrong here. Otherwise "core.trustctime" seems like the
perfect option for the problem.