[RFC] Case insensitive Git attributes

[RFC] Case insensitive Git attributes

[Lars Schneider]

Hi,

Git attributes for path names are generally case sensitive. However, on 
a case insensitive file system (e.g. macOS/Windows) they appear to be
case insensitive (`*.bar` would match `foo.bar` and `foo.BAR`). That 
works great until a Git users joins the party with a case sensitive file 
system. For this Git user only files that match the exact case of the 
attribute pattern get the attributes (only `foo.bar`).

This inconsistent behavior can confuse Git users. An advanced Git user
could use a glob pattern (e.g. `*.[bB][aA][rR]) to match files in a
case insensitive way. However, this can get confusing quickly, too.

I wonder if we can do something about this. One idea could be to add an
attribute "case-sensitive" (or "caseSensitive") and set it to false 
(if desired) for all files in .gitattributes for a given repo.

### .gitattributes example ###

* case-sensitive=false
*.bar something

###

I haven't looked into the feasibility of an implementation, yet. However,
would that be an acceptable approach?

Thanks,
Lars

Re: [RFC] Case insensitive Git attributes

[Stefan Beller]

On Sun, Oct 16, 2016 at 6:04 PM, Lars Schneider
<larsxschneider@gmail.com> wrote:
> Hi,
>
> Git attributes for path names are generally case sensitive. However, on
> a case insensitive file system (e.g. macOS/Windows) they appear to be
> case insensitive (`*.bar` would match `foo.bar` and `foo.BAR`). That
> works great until a Git users joins the party with a case sensitive file
> system. For this Git user only files that match the exact case of the
> attribute pattern get the attributes (only `foo.bar`).
>
> This inconsistent behavior can confuse Git users. An advanced Git user
> could use a glob pattern (e.g. `*.[bB][aA][rR]) to match files in a
> case insensitive way. However, this can get confusing quickly, too.
>
> I wonder if we can do something about this. One idea could be to add an
> attribute "case-sensitive" (or "caseSensitive") and set it to false
> (if desired) for all files in .gitattributes for a given repo.

FYI: I am currently refactoring the attr subsystem (e.g.
https://public-inbox.org/git/20161012224109.23410-1-sbeller@google.com/
"attr: convert to new threadsafe API")

>
> ### .gitattributes example ###
>
> * case-sensitive=false

Would this modify the current file only or the whole stack of attrs?
(In just one way or the whole stack, i.e. can you add this in .git/info/exclude
and the attribute file in the home dir also behaves differently? Or rather the
other way round when the system wide attr file enables case insensitivity,
each repository local config is set automatically? both ways?)

> *.bar something
>
> ###
>
> I haven't looked into the feasibility of an implementation, yet. However,
> would that be an acceptable approach?

Conceptually I would prefer if we had a single switch that indicates a
case insensitive FS. That could be used for different purposes as well,
that are FS relevant such as checking in, checking out/renaming files
in the working tree? (does any such switch already exist for case
sensitivity?)

Thanks,
Stefan

>
> Thanks,
> Lars
>
>
>

Re: [RFC] Case insensitive Git attributes

[Torsten Bögershausen]

On 17/10/16 05:07, Stefan Beller wrote:
> On Sun, Oct 16, 2016 at 6:04 PM, Lars Schneider
> <larsxschneider@gmail.com> wrote:
>> Hi,
>>
>> Git attributes for path names are generally case sensitive. However, on
>> a case insensitive file system (e.g. macOS/Windows) they appear to be
>> case insensitive (`*.bar` would match `foo.bar` and `foo.BAR`).
This feels like a bug:

$ git diff
diff --git a/.gitattributes b/.gitattributes
index 5e98806..1419867 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -1,3 +1,4 @@
  * whitespace=!indent,trail,space
  *.[ch] whitespace=indent,trail,space
  *.sh whitespace=indent,trail,space
+*.C text

-----------
$ git -c core.ignorecase=false check-attr --all git.c
git.c: whitespace: indent,trail,space

#But running on a case insensitve FS I get:
$ git  check-attr --all git.c
git.c: text: set
git.c: whitespace: indent,trail,space


>> That
>> works great until a Git users joins the party with a case sensitive file
>> system. For this Git user only files that match the exact case of the
>> attribute pattern get the attributes (only `foo.bar`).
>>
>> This inconsistent behavior can confuse Git users. An advanced Git user
>> could use a glob pattern (e.g. `*.[bB][aA][rR]) to match files in a
>> case insensitive way. However, this can get confusing quickly, too.
>>
>> I wonder if we can do something about this. One idea could be to add an
>> attribute "case-sensitive" (or "caseSensitive") and set it to false
>> (if desired) for all files in .gitattributes for a given repo.
> FYI: I am currently refactoring the attr subsystem (e.g.
> https://public-inbox.org/git/20161012224109.23410-1-sbeller@google.com/
> "attr: convert to new threadsafe API")
>
>> ### .gitattributes example ###
>>
>> * case-sensitive=false
How about
* ignorecase=true
  ?

> Would this modify the current file only or the whole stack of attrs?
> (In just one way or the whole stack, i.e. can you add this in .git/info/exclude
> and the attribute file in the home dir also behaves differently? Or rather the
> other way round when the system wide attr file enables case insensitivity,
> each repository local config is set automatically? both ways?)
>
>> *.bar something
>>
>> ###
>>
>> I haven't looked into the feasibility of an implementation, yet. However,
>> would that be an acceptable approach?
> Conceptually I would prefer if we had a single switch that indicates a
> case insensitive FS. That could be used for different purposes as well,
> that are FS relevant such as checking in, checking out/renaming files
> in the working tree? (does any such switch already exist for case
> sensitivity?)
>
> Thanks,
> Stefan
>
>> Thanks,
>> Lars
>>
>>
>>

Re: [RFC] Case insensitive Git attributes

[Johannes Schindelin]

Hi Stefan,

On Sun, 16 Oct 2016, Stefan Beller wrote:

> Conceptually I would prefer if we had a single switch that indicates a
> case insensitive FS.

AFAIU Lars' use case is where the FS is *case sensitive*, but he still
needs the .gitattributes to be *case insensitive* because that file
originates from a developer with such a file system.

Otherwise he would simply tack onto the core.ignoreCase flag.

Ciao,
Dscho

Re: [RFC] Case insensitive Git attributes

[Duy Nguyen]

On Mon, Oct 17, 2016 at 3:57 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Stefan,
>
> On Sun, 16 Oct 2016, Stefan Beller wrote:
>
>> Conceptually I would prefer if we had a single switch that indicates a
>> case insensitive FS.
>
> AFAIU Lars' use case is where the FS is *case sensitive*, but he still
> needs the .gitattributes to be *case insensitive* because that file
> originates from a developer with such a file system.
>
> Otherwise he would simply tack onto the core.ignoreCase flag.

That sounds to me like setting core.ignoreCase to true (on all devs'
repo) would "solve" this. Yes core.ignoreCase may introduce some side
effects when used on case-sensitive filesystems, so we probably want
something in the same spirit but limited to .gitattributes and
.gitignore only.
-- 
Duy

Re: [RFC] Case insensitive Git attributes

[Johannes Schindelin]

Hi Duy,

On Mon, 17 Oct 2016, Duy Nguyen wrote:

> On Mon, Oct 17, 2016 at 3:57 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> > Hi Stefan,
> >
> > On Sun, 16 Oct 2016, Stefan Beller wrote:
> >
> >> Conceptually I would prefer if we had a single switch that indicates a
> >> case insensitive FS.
> >
> > AFAIU Lars' use case is where the FS is *case sensitive*, but he still
> > needs the .gitattributes to be *case insensitive* because that file
> > originates from a developer with such a file system.
> >
> > Otherwise he would simply tack onto the core.ignoreCase flag.
> 
> That sounds to me like setting core.ignoreCase to true (on all devs'
> repo) would "solve" this.

It is good that you quoted this verb, because it does not solve things.
Instead, it would try to use the flag for two slightly incompatible
purposes at the same time.

The first (and so far, only) purpose is to tell Git that the current file
system is case insensitive.

The new purpose you described would be to tell Git that the *user* does
not care about the file names' case, even if the file system does.

I do not think that this leads to a better situation than before. Instead,
I am convinced that it will cause new and sometimes "entertaining"
problems because you can no longer discern between those two purposes
based on core.ignoreCase, you would have to teach Git to test every single
time whether the file system is case-sensitive or not.

Needless to say, I'd rather not see that happening. Many users, including
my colleagues and myself, rely on Git being a rock solid piece of
software, and that change would make it less so.

Ciao,
Dscho

Re: [RFC] Case insensitive Git attributes

[Duy Nguyen]

On Mon, Oct 17, 2016 at 5:46 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Duy,
>
> On Mon, 17 Oct 2016, Duy Nguyen wrote:
>
>> On Mon, Oct 17, 2016 at 3:57 PM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>> > Hi Stefan,
>> >
>> > On Sun, 16 Oct 2016, Stefan Beller wrote:
>> >
>> >> Conceptually I would prefer if we had a single switch that indicates a
>> >> case insensitive FS.
>> >
>> > AFAIU Lars' use case is where the FS is *case sensitive*, but he still
>> > needs the .gitattributes to be *case insensitive* because that file
>> > originates from a developer with such a file system.
>> >
>> > Otherwise he would simply tack onto the core.ignoreCase flag.
>>
>> That sounds to me like setting core.ignoreCase to true (on all devs'
>> repo) would "solve" this.
>
> It is good that you quoted this verb, because it does not solve things.
> Instead, it would try to use the flag for two slightly incompatible
> purposes at the same time.
>
> The first (and so far, only) purpose is to tell Git that the current file
> system is case insensitive.
>
> The new purpose you described would be to tell Git that the *user* does
> not care about the file names' case, even if the file system does.
>
> I do not think that this leads to a better situation than before. Instead,
> I am convinced that it will cause new and sometimes "entertaining"
> problems because you can no longer discern between those two purposes
> based on core.ignoreCase, you would have to teach Git to test every single
> time whether the file system is case-sensitive or not.

I agree. Which is why I wrote "we probably want something in the same
spirit but limited to .gitattributes and .gitignore only". In other
words we could have core.someName that makes .gitattributes and
.gitignore patterns case-insensitive (or core-sensitive). If it's
present, it overrides core.ignoreCase. If it's not present,
core.ignoreCase decides. I'm just not sure if the new config should
cover everything involving filename's case in git. That's too big to
fit in my head.

> Needless to say, I'd rather not see that happening. Many users, including
> my colleagues and myself, rely on Git being a rock solid piece of
> software, and that change would make it less so.
-- 
Duy

Re: [RFC] Case insensitive Git attributes

[Junio C Hamano]

Duy Nguyen <pclouds@gmail.com> writes:

> I agree. Which is why I wrote "we probably want something in the same
> spirit but limited to .gitattributes and .gitignore only". In other
> words we could have core.someName that makes .gitattributes and
> .gitignore patterns case-insensitive (or core-sensitive). If it's
> present, it overrides core.ignoreCase. If it's not present,
> core.ignoreCase decides. I'm just not sure if the new config should
> cover everything involving filename's case in git. That's too big to
> fit in my head.

Once I stopped thinking about this as "filename's case", it does fit
my head ;-)

I view the proposed knob as making patterns in .gitattributes and
.gitignore case insensitive, iow, it is a lazy and useful short-hand
for (mentally) editing "*.c attr" to "*.[cC] attr" without touching
these files.

And I agree that the knob that is missing in today's Git should
default to whatever core.ignoreCase's value is, iow, on case
insensitive filesystem, attr and ignore may match case insensitively
in today's Git, but when the knob is introduced, it should allow
forcing case sensitive match there by setting it to false, just like
the knob is proposed to be used in the oppositite direction to force
case insensitive match regardless of the case insensitiveness of the
underlying filesystem.

Re: [RFC] Case insensitive Git attributes

[Johannes Schindelin]

Hi Lars,

On Sun, 16 Oct 2016, Lars Schneider wrote:

> One idea could be to add an attribute "case-sensitive" (or
> "caseSensitive") and set it to false (if desired) for all files in
> .gitattributes for a given repo.
> 
> ### .gitattributes example ###
> 
> * case-sensitive=false
> *.bar something
> 
> ###

Hrm. Maybe a better idea would be to warn when attributes match a file
name with a different case?

Ciao,
Dscho

Re: [RFC] Case insensitive Git attributes

[Dakota Hawkins]

Apologies for the delayed bump. I think because we're talking about
affecting the behavior of .gitattributes that it would be better to
have a distinct .gitattributes option, whether or not you also have a
similar config option.

Since .gitattributes is versioned and config options are not, I think
this takes it out of the realm of personal preference. It's already
too easy for somebody to screw up and not have a "required" filter
driver (e.g. git lfs) configured and damage a repo by getting
unnoticed and unfiltered content committed.

I would love a .gitconfig option I could set that would let me stop
manually ignoring case in globs for git commands, but this might
actually make things worse for people if it were included as a config
option only -- suddenly attributes could be applied to different files
for different people.

Of course, if git supported a subset of config options you could
actually version and ensure everybody else has, this wouldn't be a
problem ;)

So, I think the correct (for today) solution is to have two options.
One for .gitattributes globs and one for everything else. I realize
that this might be somewhat controversial, so I wanted to see what
everybody thought.

Re: [RFC] Case insensitive Git attributes

[Junio C Hamano]

Dakota Hawkins <dakota@dakotahawkins.com> writes:

> Apologies for the delayed bump. I think because we're talking about
> affecting the behavior of .gitattributes that it would be better to
> have a distinct .gitattributes option, whether or not you also have a
> similar config option.

As I know I am on the To: line of the message I am responding to,
let me quicly let you know that I won't be responding to this thread
for a while as I don't recall what the discussion was about.  I will
after I'll dig and find out what the thread was about but it won't
happen immediately.  Sorry about that.

Re: [RFC] Case insensitive Git attributes

[Lars Schneider]

> On 23 Jan 2017, at 19:35, Junio C Hamano <gitster@pobox.com> wrote:
> 
> Dakota Hawkins <dakota@dakotahawkins.com> writes:
> 
>> Apologies for the delayed bump. I think because we're talking about
>> affecting the behavior of .gitattributes that it would be better to
>> have a distinct .gitattributes option, whether or not you also have a
>> similar config option.
> 
> As I know I am on the To: line of the message I am responding to,
> let me quicly let you know that I won't be responding to this thread
> for a while as I don't recall what the discussion was about.  I will
> after I'll dig and find out what the thread was about but it won't
> happen immediately.  Sorry about that.


Problem:
Git attributes for path names are generally case sensitive. However, on 
a case insensitive file system (e.g. macOS/Windows) they appear to be
case insensitive (`*.bar` would match `foo.bar` and `foo.BAR`). That 
works great until a Git users joins the party with a case sensitive file 
system. For this Git user the attributes pattern only matches files with
the exact case (*.bar` would match only `foo.bar`).

Question/Proposal:
Could we introduce some flag to signal that certain attribute patterns
are always case insensitive? 

Thread:
http://public-inbox.org/git/C83BE22D-EAC8-49E2-AEE3-22D4A99AE205@gmail.com/#t

Cheers,
Lars

Re: [RFC] Case insensitive Git attributes

[Junio C Hamano]

Lars Schneider <larsxschneider@gmail.com> writes:

> Problem:
> Git attributes for path names are generally case sensitive. However, on 
> a case insensitive file system (e.g. macOS/Windows) they appear to be
> case insensitive (`*.bar` would match `foo.bar` and `foo.BAR`). That 
> works great until a Git users joins the party with a case sensitive file 
> system. For this Git user the attributes pattern only matches files with
> the exact case (*.bar` would match only `foo.bar`).
>
> Question/Proposal:
> Could we introduce some flag to signal that certain attribute patterns
> are always case insensitive? 
>
> Thread:
> http://public-inbox.org/git/C83BE22D-EAC8-49E2-AEE3-22D4A99AE205@gmail.com/#t

Thanks for a pointer.

So you are worried about the case where somebody on a case
insensitive but case preserving system would do

    $ edit file.txt
    $ edit .gitattributes
    $ git add file.txt .gitattributes

and adds "*.TXT	someattr=true" to the attributes file, which
would set someattr to true on his system for file.txt, but when the
result is checked out on a case sensitive system, it would behave
differently because "*.TXT" does not match "file.txt"?

How do other systems address it?  Your Java, Ruby, etc. sources may
refer to another file with "import" and the derivation of the file
names from class names or package names would have the same issue,
isn't it?  Do they have an option that lets you say

    Even though the import statements may say "import a.b.C", we
    know that the source tarball was prepared on a case insensitive
    system, and I want you to look for a/b/C.java and a/b/c.java and
    use what was found.

or something like that?  Same for anything that records other
filenames in the content to refer to them, like "Makefile".

My knee jerk reaction to that is that .gitattributes and .gitignore
should not be instructed to go case insensitive on case sensitive
systems.  If the system is case insensitive but case preserving,
it probably would make sense not to do case insensitive matching,
which would prevent the issue from happening in the first place.

Re: [RFC] Case insensitive Git attributes

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> So you are worried about the case where somebody on a case
> insensitive but case preserving system would do
>
>     $ edit file.txt
>     $ edit .gitattributes
>     $ git add file.txt .gitattributes
>
> and adds "*.TXT	someattr=true" to the attributes file, which
> would set someattr to true on his system for file.txt, but when the
> result is checked out on a case sensitive system, it would behave
> differently because "*.TXT" does not match "file.txt"?
>
> How do other systems address it?  Your Java, Ruby, etc. sources may
> refer to another file with "import" and the derivation of the file
> names from class names or package names would have the same issue,
> isn't it?  Do they have an option that lets you say
>
>     Even though the import statements may say "import a.b.C", we
>     know that the source tarball was prepared on a case insensitive
>     system, and I want you to look for a/b/C.java and a/b/c.java and
>     use what was found.
>
> or something like that?  Same for anything that records other
> filenames in the content to refer to them, like "Makefile".
>
> My knee jerk reaction to that is that .gitattributes and .gitignore
> should not be instructed to go case insensitive on case sensitive
> systems.  If the system is case insensitive but case preserving,
> it probably would make sense not to do case insensitive matching,
> which would prevent the issue from happening in the first place.

Sorry, but there is a slight leap in the above that makes it hard to
track my thought, so let me clarify a bit.  

In the above, I am guessing the answer to the "How do other systems
address it?" question to be "nothing".  And that leads to the
conclusion that it is better to do "nothing on case sensitive
systems, and probably become evem more strict on case insensitive
but case preserving systems", because that will give us a chance to
expose the problem earlier, hopefully even on the originating
system.

Re: [RFC] Case insensitive Git attributes

[Lars Schneider]

> On 23 Jan 2017, at 20:38, Junio C Hamano <gitster@pobox.com> wrote:
> 
> Junio C Hamano <gitster@pobox.com> writes:
> 
>> So you are worried about the case where somebody on a case
>> insensitive but case preserving system would do
>> 
>>   $ edit file.txt
>>   $ edit .gitattributes
>>   $ git add file.txt .gitattributes
>> 
>> and adds "*.TXT	someattr=true" to the attributes file, which
>> would set someattr to true on his system for file.txt, but when the
>> result is checked out on a case sensitive system, it would behave
>> differently because "*.TXT" does not match "file.txt"?

Correct!


>> How do other systems address it?  Your Java, Ruby, etc. sources may
>> refer to another file with "import" and the derivation of the file
>> names from class names or package names would have the same issue,
>> isn't it?  Do they have an option that lets you say
>> 
>>   Even though the import statements may say "import a.b.C", we
>>   know that the source tarball was prepared on a case insensitive
>>   system, and I want you to look for a/b/C.java and a/b/c.java and
>>   use what was found.
>> 
>> or something like that?  Same for anything that records other
>> filenames in the content to refer to them, like "Makefile".
>> 
>> My knee jerk reaction to that is that .gitattributes and .gitignore
>> should not be instructed to go case insensitive on case sensitive
>> systems.  If the system is case insensitive but case preserving,
>> it probably would make sense not to do case insensitive matching,
>> which would prevent the issue from happening in the first place.
> 
> Sorry, but there is a slight leap in the above that makes it hard to
> track my thought, so let me clarify a bit.  
> 
> In the above, I am guessing the answer to the "How do other systems
> address it?" question to be "nothing".  And that leads to the
> conclusion that it is better to do "nothing on case sensitive
> systems, and probably become evem more strict on case insensitive
> but case preserving systems", because that will give us a chance to
> expose the problem earlier, hopefully even on the originating
> system.

I agree: Git attributes should behave the same on all platforms independent
of the file system type. I dug a bit deeper and realized that this is actually
already the case. However, the default (?) core.ignorecase=1 config on Win/Mac
generates the behavior explained above. I wonder if 6eba621 ("attr.c: respect 
core.ignorecase when matching attribute patterns", 2011-10-11) was a good idea.

AFAIK disabling core.ignorecase entirely on Win/Mac is no solution as this would
generate other trouble.

Git users can already create case insensitive gitattributes pattern. E.g.:
*.[tT][xX][tT]

However, based on my dayjob experience no Win/Mac developer does that as it
makes the gitattributes file unreadable. Consequently, Linux developers are 
screwed. Therefore, I wonder if it would make sense to introduce a shortcut
for the case insensitive glob pattern. E.g.:

*.txt ignorecase

If Git detects the ignorecase attribute then it could generate *.[tT][xX][tT]
automatically.