git blame swallows up lines in case of mixed line endings

git blame swallows up lines in case of mixed line endings

[Sokolov, Konstantin (ext)]

Hi Folks,

I encounter unexpected behavior in the following case:

file content:

line1<CR><LF>
line2<CR>
line3<CR><LF>
line4

This is what I get as console output (on Windows):

> git blame -s file.txt
7db36436 1) line1
line3436 2) line2
7db36436 3) line4

This is the real content:

> git blame -s file.txt > blame.txt

blame.txt opened in Notepad++:

7db36436 1) line1 <CR><LF>
7db36436 2) line2 <CR>
line3 <CR><LF>
7db36436 3) line4 <LF>

Admittedly, very stupid editors, such as Windows Notepad, cannot handle mixed line endings as well. But is this also the way git blame should behave?

Kind regards
Konstantin

Re: git blame swallows up lines in case of mixed line endings

[Torsten Bögershausen]

On 2015-02-19 14.48, Sokolov, Konstantin (ext) wrote:
> Hi Folks,
> 
> I encounter unexpected behavior in the following case:
> 
> file content:
> 
> line1<CR><LF>
> line2<CR>
> line3<CR><LF>
> line4
> 
> This is what I get as console output (on Windows):
> 
>> git blame -s file.txt
> 7db36436 1) line1
> line3436 2) line2
> 7db36436 3) line4
> 
> This is the real content:
> 
>> git blame -s file.txt > blame.txt
> 
> blame.txt opened in Notepad++:
> 
> 7db36436 1) line1 <CR><LF>
> 7db36436 2) line2 <CR>
> line3 <CR><LF>
> 7db36436 3) line4 <LF>
> 
> Admittedly, very stupid editors, such as Windows Notepad, cannot handle mixed line endings as well. But is this also the way git blame should behave?
> 
> Kind regards
> Konstantin
>
Git (and other tools) do not handle a single CR as a line ending.
A line ending in Git is either CRLF or LF.

A "mixed line ending" in Git sense is a mixture between CRLF and LF, 
but not a lone CR.
(And in this sense it is the expected behavior)

Are you sure you want to use it, or is it a typo ?

AW: git blame swallows up lines in case of mixed line endings

[Sokolov, Konstantin (ext)]

It's not that I want to use CR as line ending. We just had to process a repository where it was the case. We use git blame as an automatic processing step in a bigger system. So if a line does not exist for git blame but it does exist for other parts of the system (which read the same file) we run into consistency problems. 

-----Ursprüngliche Nachricht-----
Von: Torsten Bögershausen [mailto:tboegi@web.de] 
Gesendet: Samstag, 21. Februar 2015 14:46
An: Sokolov, Konstantin (ext); 'git@vger.kernel.org'
Betreff: Re: git blame swallows up lines in case of mixed line endings

On 2015-02-19 14.48, Sokolov, Konstantin (ext) wrote:
> Hi Folks,
> 
> I encounter unexpected behavior in the following case:
> 
> file content:
> 
> line1<CR><LF>
> line2<CR>
> line3<CR><LF>
> line4
> 
> This is what I get as console output (on Windows):
> 
>> git blame -s file.txt
> 7db36436 1) line1
> line3436 2) line2
> 7db36436 3) line4
> 
> This is the real content:
> 
>> git blame -s file.txt > blame.txt
> 
> blame.txt opened in Notepad++:
> 
> 7db36436 1) line1 <CR><LF>
> 7db36436 2) line2 <CR>
> line3 <CR><LF>
> 7db36436 3) line4 <LF>
> 
> Admittedly, very stupid editors, such as Windows Notepad, cannot handle mixed line endings as well. But is this also the way git blame should behave?
> 
> Kind regards
> Konstantin
>
Git (and other tools) do not handle a single CR as a line ending.
A line ending in Git is either CRLF or LF.

A "mixed line ending" in Git sense is a mixture between CRLF and LF, but not a lone CR.
(And in this sense it is the expected behavior)

Are you sure you want to use it, or is it a typo ?

Re: git blame swallows up lines in case of mixed line endings

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

> On 2015-02-19 14.48, Sokolov, Konstantin (ext) wrote:
>> 
>> I encounter unexpected behavior in the following case:
>> 
>> file content:
>> 
>> line1<CR><LF>
>> line2<CR>
>> line3<CR><LF>
>> line4

You can mark a file as <CRLF> terminated via attributes system and
have Git convert them to use <LF> as end-of-line when file contents
are stored in Git ("<LF> as end-of-line" is the representation "git
blame" uses internally).  Konstantin said "on Windows", and I guessed
initially that the lines are marked as such, but after looking at
the blame.txt output I am not sure.

That means the contents of the lines are:

    First Line:  "line1"
    Second Line: "line2" + CR + "line3"
    Third Line:  "line4"

or if CRLF conversion is not specified in Konstantin's repository:

    First Line:  "line1" + CR
    Second Line: "line2" + CR + CR + "line3" + CR
    Third Line:  "line4" + CR

Either way, that makes the observed behavior totally expected and
understandable.

>> This is what I get as console output (on Windows):
>> 
>>> git blame -s file.txt
>> 7db36436 1) line1
>> line3436 2) line2
>> 7db36436 3) line4

As printing CR moves the cursor to the beginning of line on many
terminals, it is understandable to see the above output.  After
printing the first line, it will show 

    7db36436 2) line2

and then go back to the leftmost column and then overwrite 7db3...
with "line3".

Even though we have an option to mark <CR> alone as the end of line
marker, because the blamed content can go through the textconv
filter, it may be possible to define a textconv filter for the path
via the attribute system and convert such "mixed line endings"
contents to a series of <LF>-terminated lines.  That would split the
second line in the original input into two lines and may produce
desired result.

AW: git blame swallows up lines in case of mixed line endings

[Sokolov, Konstantin (ext)]

Thank you for going into the matter. I was not aware of the textconv filter. This is definitely a decent solution. But what exactly do you mean by "Even though we have an option to mark <CR> alone as the end of line marker"? Is there really such an option in Git or did you mean that it's possible in general (by changing the implementation)?

Kind Regards
Konstantin

-----Ursprüngliche Nachricht-----
Von: Junio C Hamano [mailto:gitster@pobox.com] 
Gesendet: Montag, 23. Februar 2015 19:45
An: Torsten Bögershausen
Cc: Sokolov, Konstantin (ext); 'git@vger.kernel.org'
Betreff: Re: git blame swallows up lines in case of mixed line endings

Torsten Bögershausen <tboegi@web.de> writes:

> On 2015-02-19 14.48, Sokolov, Konstantin (ext) wrote:
>> 
>> I encounter unexpected behavior in the following case:
>> 
>> file content:
>> 
>> line1<CR><LF>
>> line2<CR>
>> line3<CR><LF>
>> line4

You can mark a file as <CRLF> terminated via attributes system and have Git convert them to use <LF> as end-of-line when file contents are stored in Git ("<LF> as end-of-line" is the representation "git blame" uses internally).  Konstantin said "on Windows", and I guessed initially that the lines are marked as such, but after looking at the blame.txt output I am not sure.

That means the contents of the lines are:

    First Line:  "line1"
    Second Line: "line2" + CR + "line3"
    Third Line:  "line4"

or if CRLF conversion is not specified in Konstantin's repository:

    First Line:  "line1" + CR
    Second Line: "line2" + CR + CR + "line3" + CR
    Third Line:  "line4" + CR

Either way, that makes the observed behavior totally expected and understandable.

>> This is what I get as console output (on Windows):
>> 
>>> git blame -s file.txt
>> 7db36436 1) line1
>> line3436 2) line2
>> 7db36436 3) line4

As printing CR moves the cursor to the beginning of line on many terminals, it is understandable to see the above output.  After printing the first line, it will show 

    7db36436 2) line2

and then go back to the leftmost column and then overwrite 7db3...
with "line3".

Even though we have an option to mark <CR> alone as the end of line marker, because the blamed content can go through the textconv filter, it may be possible to define a textconv filter for the path via the attribute system and convert such "mixed line endings"
contents to a series of <LF>-terminated lines.  That would split the second line in the original input into two lines and may produce desired result.

Re: AW: git blame swallows up lines in case of mixed line endings

[Junio C Hamano]

"Sokolov, Konstantin (ext)" <konstantin.sokolov.ext@siemens.com>
writes:

> Thank you for going into the matter. I was not aware of the textconv
> filter. This is definitely a decent solution. But what exactly do you
> mean by "Even though we have an option to mark <CR> alone as the end
> of line marker"?

s/have/do not have/; sorry for a false hope ;-)