possible gitattributes eol bug with new eol=crlf | lf support?

possible gitattributes eol bug with new eol=crlf | lf support?

[Robert Buck]

I start with a repository containing only four files:

  * lf.xml
  * lf.sln
  * crlf.xml
  * crlf.sln

The files whose names are prefixed by LF contain Unix LF EOL
characters. The files whose names are prefixed by CRLF contain Windows
CRLF EOL characters. Each file contains two lines, on one line 'jim',
the second line contains 'tim'.

I later add and commit a .gitattributes file containing the following rules:

  *.sln  eol=crlf
  *.xml eol=lf

I _then_ clone the repository and open each file in binary mode to find:

  * the crlf.xml file contains CRLF when it should contain only LF
<<<<<<< BUG ???
  * the crlf.sln file contains CRLF as it rightly should
  * the lf.xml file contains LF as it rightly should
  * the lf.sln file contains CRLF as it rightly should

Conversion of LF-EOL files to CRLF works fine, but conversion of CRLF
to LF fails to occur.

The doc is a little unclear if this is expected behavior, which if I
recall correctly from the email threads related to the new eol
support, this should not have occurred.

Guidance appreciated.

Re: possible gitattributes eol bug with new eol=crlf | lf support?

[Eyvind Bernhardsen]

On 10. sep. 2010, at 00.31, Robert Buck wrote:

[...]

> Conversion of LF-EOL files to CRLF works fine, but conversion of CRLF
> to LF fails to occur.
> 
> The doc is a little unclear if this is expected behavior, which if I
> recall correctly from the email threads related to the new eol
> support, this should not have occurred.

Unfortunately, this is expected behaviour: you need to "manually" remove CRLFs when you turn on eol conversion.  The simplest way to do this is "rm .git/index && git reset", then commit the modified files (ideally in the same commit that modifies .gitattributes--this is mentioned in gitattributes(5)).

To make this work as it should, git would have to notice changes to .gitattributes and check files which have had their attributes changed.  It's on my "I wish I had time to do this" list.

- Eyvind

Re: possible gitattributes eol bug with new eol=crlf | lf support?

[Robert Buck]

I don't understand the inner workings of .git/index, but is removing
that file destructive to history or anything? What are the
implications of that delete-command?

Bob

On Fri, Sep 10, 2010 at 2:25 PM, Eyvind Bernhardsen
<eyvind.bernhardsen@gmail.com> wrote:
> On 10. sep. 2010, at 00.31, Robert Buck wrote:
>
> [...]
>
>> Conversion of LF-EOL files to CRLF works fine, but conversion of CRLF
>> to LF fails to occur.
>>
>> The doc is a little unclear if this is expected behavior, which if I
>> recall correctly from the email threads related to the new eol
>> support, this should not have occurred.
>
> Unfortunately, this is expected behaviour: you need to "manually" remove CRLFs when you turn on eol conversion.  The simplest way to do this is "rm .git/index && git reset", then commit the modified files (ideally in the same commit that modifies .gitattributes--this is mentioned in gitattributes(5)).
>
> To make this work as it should, git would have to notice changes to .gitattributes and check files which have had their attributes changed.  It's on my "I wish I had time to do this" list.
>
> - Eyvind
>
>

Re: possible gitattributes eol bug with new eol=crlf | lf support?

[Eyvind Bernhardsen]

On 10. sep. 2010, at 23.27, Robert Buck wrote:

> I don't understand the inner workings of .git/index, but is removing
> that file destructive to history or anything? What are the
> implications of that delete-command?

Removing the index will lose the changes you've staged ("git add"ed) for the next commit, but your working directory won't be touched.  If you've added a file and then modified or deleted it, you would lose the version of that file that was in the index.

"git reset" then rebuilds the index identically to the HEAD commit, but without the staged changes and (importantly) the stat cache.  The point is to make git re-check every file to see if it has been modified.

Sorry, I should have mentioned the downsides.

- Eyvind

Re: possible gitattributes eol bug with new eol=crlf | lf support?

[Robert Buck]

Thanks Eyvind,

I tried it on an experimental repository and it worked. Thank you for
your recommendation.

I also found the following link at github which achieves a similar
effect. Adding this for the record in case someone else searching for
a solution in the future wanted more detail.

    http://help.github.com/dealing-with-lineendings/

The one thing I find curious about the github article is that it seems
to recommend using autocrlf=true for ALL platforms once the linefeeds
have been normalized.

Let me ask for an opinion about this... Given we have an environment
of mixed Windows, Mac, and Linux developers, that we have just
migrated from svn and over 2000 files in the repository have messed up
line-endings, what would be your recommendation for autocrlf settings?
Oh, important note, because neither cygwin nor msysgit support newer
versions of git, we do not have the flexibility of running your new
"eol" support on Windows, while Linux developers do.

So if we did this one-time normalization on all repositories, all
branches, what holistic approach (eol, autocrlf) would keep our files
sane for a mix of 1.7.2 and later, and 1.7.0.1 and earlier, Windows,
Mac, and Linux?

Thanks Eyvind.

-Bob

On Sun, Sep 12, 2010 at 7:46 AM, Eyvind Bernhardsen
<eyvind.bernhardsen@gmail.com> wrote:
> On 10. sep. 2010, at 23.27, Robert Buck wrote:
>
>> I don't understand the inner workings of .git/index, but is removing
>> that file destructive to history or anything? What are the
>> implications of that delete-command?
>
> Removing the index will lose the changes you've staged ("git add"ed) for the next commit, but your working directory won't be touched.  If you've added a file and then modified or deleted it, you would lose the version of that file that was in the index.
>
> "git reset" then rebuilds the index identically to the HEAD commit, but without the staged changes and (importantly) the stat cache.  The point is to make git re-check every file to see if it has been modified.
>
> Sorry, I should have mentioned the downsides.
>
> - Eyvind
>
>

Re: possible gitattributes eol bug with new eol=crlf | lf support?

[Eyvind Bernhardsen]

On 12. sep. 2010, at 21.58, Robert Buck wrote:

> Thanks Eyvind,
> 
> I tried it on an experimental repository and it worked. Thank you for
> your recommendation.

Great :)

> I also found the following link at github which achieves a similar
> effect. Adding this for the record in case someone else searching for
> a solution in the future wanted more detail.
> 
>    http://help.github.com/dealing-with-lineendings/
> 
> The one thing I find curious about the github article is that it seems
> to recommend using autocrlf=true for ALL platforms once the linefeeds
> have been normalized.

That article hasn't been updated for 1.7.2, but what it's saying is that line ending normalisation should be enabled for all users (or none) if you want to avoid problems.

The point of the text attribute is to allow you to enable or disable line ending normalisation without the user having to configure anything; setting the attribute "text=auto" on all files is equivalent to every user manually setting autocrlf to "true" or "input" in that repository.

[...]

> So if we did this one-time normalization on all repositories, all
> branches, what holistic approach (eol, autocrlf) would keep our files
> sane for a mix of 1.7.2 and later, and 1.7.0.1 and earlier, Windows,
> Mac, and Linux?

I would set the text attribute to auto for all files (add the line "* text=auto" to .gitattributes) to take care of users with git 1.7.2 or newer.  Users with older versions of git should set autocrlf=true (Windows) or autocrlf=input (Mac and Linux).

Once all users are upgraded to newer versions of git you can change the normalisation to target only specific files or file types, but that's not advisable while any of your users have autocrlf enabled.

- Eyvind