* Ability to ignore EOL changes for certain projects @ 2019-12-18 11:10 Scott Richmond 2019-12-18 19:27 ` Torsten Bögershausen 2019-12-20 3:02 ` brian m. carlson 0 siblings, 2 replies; 9+ messages in thread From: Scott Richmond @ 2019-12-18 11:10 UTC (permalink / raw) To: git The Problem Domain In certain dev environments (Unity3D projects) there is (AFAIK) an unsolvable problem where some files are often modified with line endings that aren't the native system or not the committed line endings for that file. Secondarily, in this case line endings don't matter - Nothing in the dev environment "cares" which kind of line ending is used. The Problem Git always cares about EOL. Git has options to transparently modify EOLs when files are checked in or out. However it is not possible to tell Git to ignore EOLs in other commands: Git status shows the file modified. Merging/Pulling has to care because it can't merge with a modified working tree. Which means the user has to care - They have to either stash the EOL changes or wipe them out. Sometimes, if the user has a particular app running, it may automatically reload that file and recreate the modified EOL changes, causing an endless loop. This problem is often made unclear to the user how to solve, especially if they aren't domain experts. To be clear, in this particular dev environment, I can say with confidence that this particular issue is a major and common pain point for users. It is made worse as many users in this environment aren't programmers by trade and aren't domain experts in version control. I also believe this environment is becoming a non-trivial portion of the Git userbase and it would be worthwhile looking into resolving. Solution Request It would be fantastic if we could tell Git to stop caring about EOL changes on a per-repo basis, with the effective output being that git status shouldn't show changes for files with differing EOLs. I'm experienced with Git, though I am not expert enough to consider creating such a change myself - It is unclear to me just how complicated a change may be. However maybe I could look into it if it was made clear that this improvement is possible and has no serious side effects. Regards, Scott Richmond. Director, Producer, Programmer Brightrock Games T: 07480795661 ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Ability to ignore EOL changes for certain projects 2019-12-18 11:10 Ability to ignore EOL changes for certain projects Scott Richmond @ 2019-12-18 19:27 ` Torsten Bögershausen 2019-12-18 21:33 ` Scott Richmond 2019-12-20 3:02 ` brian m. carlson 1 sibling, 1 reply; 9+ messages in thread From: Torsten Bögershausen @ 2019-12-18 19:27 UTC (permalink / raw) To: Scott Richmond; +Cc: git On Wed, Dec 18, 2019 at 11:10:27AM +0000, Scott Richmond wrote: > The Problem Domain > In certain dev environments (Unity3D projects) there is (AFAIK) an > unsolvable problem where some files are often modified with line > endings that aren't the native system or not the committed line > endings for that file. Secondarily, in this case line endings don't > matter - Nothing in the dev environment "cares" which kind of line > ending is used. > > The Problem > Git always cares about EOL. Git has options to transparently modify > EOLs when files are checked in or out. However it is not possible to > tell Git to ignore EOLs in other commands: > Git status shows the file modified. > Merging/Pulling has to care because it can't merge with a modified > working tree. Which means the user has to care - They have to either > stash the EOL changes or wipe them out. Sometimes, if the user has a > particular app running, it may automatically reload that file and > recreate the modified EOL changes, causing an endless loop. This > problem is often made unclear to the user how to solve, especially if > they aren't domain experts. > > To be clear, in this particular dev environment, I can say with > confidence that this particular issue is a major and common pain point > for users. It is made worse as many users in this environment aren't > programmers by trade and aren't domain experts in version control. I > also believe this environment is becoming a non-trivial portion of the > Git userbase and it would be worthwhile looking into resolving. > > Solution Request > It would be fantastic if we could tell Git to stop caring about EOL > changes on a per-repo basis, with the effective output being that git > status shouldn't show changes for files with differing EOLs. > > I'm experienced with Git, though I am not expert enough to consider > creating such a change myself - It is unclear to me just how > complicated a change may be. However maybe I could look into it if it > was made clear that this improvement is possible and has no serious > side effects. Hej Scott, I think that you problem can be solved. For each repository, you can tell Git to ignore the line endings, CRLF vs LF. If you start with a fresh repo, you can do something like: echo "* text=auto" >.gitattributes git add .gitattributes git commit -m "Add .gitattributes" For existing repos, we need to take another step: echo "* text=auto" >.gitattributes git add .gitattributes git add --renormlize . git commit -m "Add .gitattributes" More information can be found e.g. here: https://git-scm.com/docs/git-add Once you done that, you can merge branches into the master branch with help of the renormalize git merge -X renormalze branch-name See even here: https://git-scm.com/docs/git-merge This is just a (too) short introduction, I hope that it helps and that you find the time to dig somewhat deeper into the documentation. Other developers have that problem as well, you are not alone. If you have a public repo, I could help with one example. > > Regards, > > Scott Richmond. > Director, Producer, Programmer > Brightrock Games > T: 07480795661 ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Ability to ignore EOL changes for certain projects 2019-12-18 19:27 ` Torsten Bögershausen @ 2019-12-18 21:33 ` Scott Richmond 2019-12-18 22:34 ` Philip Oakley 2019-12-19 3:12 ` Torsten Bögershausen 0 siblings, 2 replies; 9+ messages in thread From: Scott Richmond @ 2019-12-18 21:33 UTC (permalink / raw) To: Torsten Bögershausen; +Cc: git Hey Torsten, Thanks for the reply! Correct me if am wrong, but those steps don't tell git to "ignore" line endings. That just causes git to check all text files in and out with a specific EOL type (Either automatically chosen, or not). If an app in the dev env changes a files' EOL to something else, git will notice the change locally. Regards, Scott Richmond. Director, Producer, Programmer Brightrock Games T: 07480795661 On Wed, Dec 18, 2019 at 7:27 PM Torsten Bögershausen <tboegi@web.de> wrote: > > On Wed, Dec 18, 2019 at 11:10:27AM +0000, Scott Richmond wrote: > > The Problem Domain > > In certain dev environments (Unity3D projects) there is (AFAIK) an > > unsolvable problem where some files are often modified with line > > endings that aren't the native system or not the committed line > > endings for that file. Secondarily, in this case line endings don't > > matter - Nothing in the dev environment "cares" which kind of line > > ending is used. > > > > The Problem > > Git always cares about EOL. Git has options to transparently modify > > EOLs when files are checked in or out. However it is not possible to > > tell Git to ignore EOLs in other commands: > > Git status shows the file modified. > > Merging/Pulling has to care because it can't merge with a modified > > working tree. Which means the user has to care - They have to either > > stash the EOL changes or wipe them out. Sometimes, if the user has a > > particular app running, it may automatically reload that file and > > recreate the modified EOL changes, causing an endless loop. This > > problem is often made unclear to the user how to solve, especially if > > they aren't domain experts. > > > > To be clear, in this particular dev environment, I can say with > > confidence that this particular issue is a major and common pain point > > for users. It is made worse as many users in this environment aren't > > programmers by trade and aren't domain experts in version control. I > > also believe this environment is becoming a non-trivial portion of the > > Git userbase and it would be worthwhile looking into resolving. > > > > Solution Request > > It would be fantastic if we could tell Git to stop caring about EOL > > changes on a per-repo basis, with the effective output being that git > > status shouldn't show changes for files with differing EOLs. > > > > I'm experienced with Git, though I am not expert enough to consider > > creating such a change myself - It is unclear to me just how > > complicated a change may be. However maybe I could look into it if it > > was made clear that this improvement is possible and has no serious > > side effects. > > Hej Scott, > I think that you problem can be solved. > For each repository, you can tell Git to ignore the line endings, > CRLF vs LF. > > If you start with a fresh repo, > you can do something like: > > echo "* text=auto" >.gitattributes > git add .gitattributes > git commit -m "Add .gitattributes" > > For existing repos, we need to take another step: > > echo "* text=auto" >.gitattributes > git add .gitattributes > git add --renormlize . > git commit -m "Add .gitattributes" > > More information can be found e.g. here: > https://git-scm.com/docs/git-add > > Once you done that, you can merge branches > into the master branch with help of the renormalize > > git merge -X renormalze branch-name > > See even here: > https://git-scm.com/docs/git-merge > > > This is just a (too) short introduction, I hope that it > helps and that you find the time to dig somewhat deeper into > the documentation. > > Other developers have that problem as well, you are not alone. > > If you have a public repo, I could help with one example. > > > > > > Regards, > > > > Scott Richmond. > > Director, Producer, Programmer > > Brightrock Games > > T: 07480795661 ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Ability to ignore EOL changes for certain projects 2019-12-18 21:33 ` Scott Richmond @ 2019-12-18 22:34 ` Philip Oakley 2019-12-19 13:39 ` Scott Richmond 2019-12-19 3:12 ` Torsten Bögershausen 1 sibling, 1 reply; 9+ messages in thread From: Philip Oakley @ 2019-12-18 22:34 UTC (permalink / raw) To: Scott Richmond, Torsten Bögershausen; +Cc: git On 18/12/2019 21:33, Scott Richmond wrote: > Hey Torsten, > > Thanks for the reply! > > Correct me if am wrong, but those steps don't tell git to "ignore" > line endings. That just causes git to check all text files in and out > with a specific EOL type (Either automatically chosen, or not). If an > app in the dev env changes a files' EOL to something else, git will > notice the change locally. A broader question is "how does the dev environment handle lone `CR` characters? are they also considered as EOLs?" If I recall previous discussions, part of the issue is determining what the alternate abuse cases are, such as lone CRs, or alternate character sets to the 'pure' utf-8. You will still have the difficulty of how `identicality` is determined, which currently uses the sha1/oid value. In essence you need a way of ensuring that all checkins are always of one defined EOL, but then need git to have a broader allowance of changed EOL values (including no lone CRs that are not EOL, etc). Does the environment (on any of the OS's) change the files in the background, such that the file time stamps will indicate a modification? Philip > Regards, > > Scott Richmond. > Director, Producer, Programmer > Brightrock Games > T: 07480795661 > > On Wed, Dec 18, 2019 at 7:27 PM Torsten Bögershausen <tboegi@web.de> wrote: >> On Wed, Dec 18, 2019 at 11:10:27AM +0000, Scott Richmond wrote: >>> The Problem Domain >>> In certain dev environments (Unity3D projects) there is (AFAIK) an >>> unsolvable problem where some files are often modified with line >>> endings that aren't the native system or not the committed line >>> endings for that file. Secondarily, in this case line endings don't >>> matter - Nothing in the dev environment "cares" which kind of line >>> ending is used. >>> >>> The Problem >>> Git always cares about EOL. Git has options to transparently modify >>> EOLs when files are checked in or out. However it is not possible to >>> tell Git to ignore EOLs in other commands: >>> Git status shows the file modified. >>> Merging/Pulling has to care because it can't merge with a modified >>> working tree. Which means the user has to care - They have to either >>> stash the EOL changes or wipe them out. Sometimes, if the user has a >>> particular app running, it may automatically reload that file and >>> recreate the modified EOL changes, causing an endless loop. This >>> problem is often made unclear to the user how to solve, especially if >>> they aren't domain experts. >>> >>> To be clear, in this particular dev environment, I can say with >>> confidence that this particular issue is a major and common pain point >>> for users. It is made worse as many users in this environment aren't >>> programmers by trade and aren't domain experts in version control. I >>> also believe this environment is becoming a non-trivial portion of the >>> Git userbase and it would be worthwhile looking into resolving. >>> >>> Solution Request >>> It would be fantastic if we could tell Git to stop caring about EOL >>> changes on a per-repo basis, with the effective output being that git >>> status shouldn't show changes for files with differing EOLs. >>> >>> I'm experienced with Git, though I am not expert enough to consider >>> creating such a change myself - It is unclear to me just how >>> complicated a change may be. However maybe I could look into it if it >>> was made clear that this improvement is possible and has no serious >>> side effects. >> Hej Scott, >> I think that you problem can be solved. >> For each repository, you can tell Git to ignore the line endings, >> CRLF vs LF. >> >> If you start with a fresh repo, >> you can do something like: >> >> echo "* text=auto" >.gitattributes >> git add .gitattributes >> git commit -m "Add .gitattributes" >> >> For existing repos, we need to take another step: >> >> echo "* text=auto" >.gitattributes >> git add .gitattributes >> git add --renormlize . >> git commit -m "Add .gitattributes" >> >> More information can be found e.g. here: >> https://git-scm.com/docs/git-add >> >> Once you done that, you can merge branches >> into the master branch with help of the renormalize >> >> git merge -X renormalze branch-name >> >> See even here: >> https://git-scm.com/docs/git-merge >> >> >> This is just a (too) short introduction, I hope that it >> helps and that you find the time to dig somewhat deeper into >> the documentation. >> >> Other developers have that problem as well, you are not alone. >> >> If you have a public repo, I could help with one example. >> >> >>> Regards, >>> >>> Scott Richmond. >>> Director, Producer, Programmer >>> Brightrock Games >>> T: 07480795661 ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Ability to ignore EOL changes for certain projects 2019-12-18 22:34 ` Philip Oakley @ 2019-12-19 13:39 ` Scott Richmond 0 siblings, 0 replies; 9+ messages in thread From: Scott Richmond @ 2019-12-19 13:39 UTC (permalink / raw) To: Philip Oakley; +Cc: Torsten Bögershausen, git > If I recall previous discussions, part of the issue is determining what the alternate abuse cases are, such as lone CRs, or alternate character sets to the 'pure' utf-8. In this dev env, I would be surprised if any CR type makes a difference. But I could be wrong. If that is indeed one of the reasons why previous discussions broke down, maybe that could be a configurable option in the feature improvement. > Does the environment (on any of the OS's) change the files in the background, such that the file time stamps will indicate a modification? I would imagine the file timestamps do get updated yes. Though I'd have to do some research to confirm it. If indeed this isn't a bug (As Torsten suspects), then I suppose what I am asking for is a feature improvement to perform a further check after sha1/iod to confirm if the changes are only between CR types? Regards, Scott Richmond. Director, Producer, Programmer Brightrock Games T: 07480795661 On Wed, Dec 18, 2019 at 10:35 PM Philip Oakley <philipoakley@iee.email> wrote: > > On 18/12/2019 21:33, Scott Richmond wrote: > > Hey Torsten, > > > > Thanks for the reply! > > > > Correct me if am wrong, but those steps don't tell git to "ignore" > > line endings. That just causes git to check all text files in and out > > with a specific EOL type (Either automatically chosen, or not). If an > > app in the dev env changes a files' EOL to something else, git will > > notice the change locally. > > A broader question is "how does the dev environment handle lone `CR` > characters? are they also considered as EOLs?" > > If I recall previous discussions, part of the issue is determining what > the alternate abuse cases are, such as lone CRs, or alternate character > sets to the 'pure' utf-8. > > You will still have the difficulty of how `identicality` is determined, > which currently uses the sha1/oid value. In essence you need a way of > ensuring that all checkins are always of one defined EOL, but then need > git to have a broader allowance of changed EOL values (including no lone > CRs that are not EOL, etc). > > Does the environment (on any of the OS's) change the files in the > background, such that the file time stamps will indicate a modification? > > Philip > > > Regards, > > > > Scott Richmond. > > Director, Producer, Programmer > > Brightrock Games > > T: 07480795661 > > > > On Wed, Dec 18, 2019 at 7:27 PM Torsten Bögershausen <tboegi@web.de> wrote: > >> On Wed, Dec 18, 2019 at 11:10:27AM +0000, Scott Richmond wrote: > >>> The Problem Domain > >>> In certain dev environments (Unity3D projects) there is (AFAIK) an > >>> unsolvable problem where some files are often modified with line > >>> endings that aren't the native system or not the committed line > >>> endings for that file. Secondarily, in this case line endings don't > >>> matter - Nothing in the dev environment "cares" which kind of line > >>> ending is used. > >>> > >>> The Problem > >>> Git always cares about EOL. Git has options to transparently modify > >>> EOLs when files are checked in or out. However it is not possible to > >>> tell Git to ignore EOLs in other commands: > >>> Git status shows the file modified. > >>> Merging/Pulling has to care because it can't merge with a modified > >>> working tree. Which means the user has to care - They have to either > >>> stash the EOL changes or wipe them out. Sometimes, if the user has a > >>> particular app running, it may automatically reload that file and > >>> recreate the modified EOL changes, causing an endless loop. This > >>> problem is often made unclear to the user how to solve, especially if > >>> they aren't domain experts. > >>> > >>> To be clear, in this particular dev environment, I can say with > >>> confidence that this particular issue is a major and common pain point > >>> for users. It is made worse as many users in this environment aren't > >>> programmers by trade and aren't domain experts in version control. I > >>> also believe this environment is becoming a non-trivial portion of the > >>> Git userbase and it would be worthwhile looking into resolving. > >>> > >>> Solution Request > >>> It would be fantastic if we could tell Git to stop caring about EOL > >>> changes on a per-repo basis, with the effective output being that git > >>> status shouldn't show changes for files with differing EOLs. > >>> > >>> I'm experienced with Git, though I am not expert enough to consider > >>> creating such a change myself - It is unclear to me just how > >>> complicated a change may be. However maybe I could look into it if it > >>> was made clear that this improvement is possible and has no serious > >>> side effects. > >> Hej Scott, > >> I think that you problem can be solved. > >> For each repository, you can tell Git to ignore the line endings, > >> CRLF vs LF. > >> > >> If you start with a fresh repo, > >> you can do something like: > >> > >> echo "* text=auto" >.gitattributes > >> git add .gitattributes > >> git commit -m "Add .gitattributes" > >> > >> For existing repos, we need to take another step: > >> > >> echo "* text=auto" >.gitattributes > >> git add .gitattributes > >> git add --renormlize . > >> git commit -m "Add .gitattributes" > >> > >> More information can be found e.g. here: > >> https://git-scm.com/docs/git-add > >> > >> Once you done that, you can merge branches > >> into the master branch with help of the renormalize > >> > >> git merge -X renormalze branch-name > >> > >> See even here: > >> https://git-scm.com/docs/git-merge > >> > >> > >> This is just a (too) short introduction, I hope that it > >> helps and that you find the time to dig somewhat deeper into > >> the documentation. > >> > >> Other developers have that problem as well, you are not alone. > >> > >> If you have a public repo, I could help with one example. > >> > >> > >>> Regards, > >>> > >>> Scott Richmond. > >>> Director, Producer, Programmer > >>> Brightrock Games > >>> T: 07480795661 > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Ability to ignore EOL changes for certain projects 2019-12-18 21:33 ` Scott Richmond 2019-12-18 22:34 ` Philip Oakley @ 2019-12-19 3:12 ` Torsten Bögershausen 2019-12-19 13:25 ` Scott Richmond 1 sibling, 1 reply; 9+ messages in thread From: Torsten Bögershausen @ 2019-12-19 3:12 UTC (permalink / raw) To: Scott Richmond; +Cc: git Scott, please avoid top-posting in this list, see my answers at the end. > On Wed, Dec 18, 2019 at 7:27 PM Torsten Bögershausen <tboegi@web.de> wrote: > > > > On Wed, Dec 18, 2019 at 11:10:27AM +0000, Scott Richmond wrote: > > > The Problem Domain > > > In certain dev environments (Unity3D projects) there is (AFAIK) an > > > unsolvable problem where some files are often modified with line > > > endings that aren't the native system or not the committed line > > > endings for that file. Secondarily, in this case line endings don't > > > matter - Nothing in the dev environment "cares" which kind of line > > > ending is used. > > > > > > The Problem > > > Git always cares about EOL. Git has options to transparently modify > > > EOLs when files are checked in or out. However it is not possible to > > > tell Git to ignore EOLs in other commands: > > > Git status shows the file modified. > > > Merging/Pulling has to care because it can't merge with a modified > > > working tree. Which means the user has to care - They have to either > > > stash the EOL changes or wipe them out. Sometimes, if the user has a > > > particular app running, it may automatically reload that file and > > > recreate the modified EOL changes, causing an endless loop. This > > > problem is often made unclear to the user how to solve, especially if > > > they aren't domain experts. > > > > > > To be clear, in this particular dev environment, I can say with > > > confidence that this particular issue is a major and common pain point > > > for users. It is made worse as many users in this environment aren't > > > programmers by trade and aren't domain experts in version control. I > > > also believe this environment is becoming a non-trivial portion of the > > > Git userbase and it would be worthwhile looking into resolving. > > > > > > Solution Request > > > It would be fantastic if we could tell Git to stop caring about EOL > > > changes on a per-repo basis, with the effective output being that git > > > status shouldn't show changes for files with differing EOLs. > > > > > > I'm experienced with Git, though I am not expert enough to consider > > > creating such a change myself - It is unclear to me just how > > > complicated a change may be. However maybe I could look into it if it > > > was made clear that this improvement is possible and has no serious > > > side effects. > > > > Hej Scott, > > I think that you problem can be solved. > > For each repository, you can tell Git to ignore the line endings, > > CRLF vs LF. > > > > If you start with a fresh repo, > > you can do something like: > > > > echo "* text=auto" >.gitattributes > > git add .gitattributes > > git commit -m "Add .gitattributes" > > > > For existing repos, we need to take another step: > > > > echo "* text=auto" >.gitattributes > > git add .gitattributes > > git add --renormlize . > > git commit -m "Add .gitattributes" > > > > More information can be found e.g. here: > > https://git-scm.com/docs/git-add > > > > Once you done that, you can merge branches > > into the master branch with help of the renormalize > > > > git merge -X renormalze branch-name > > > > See even here: > > https://git-scm.com/docs/git-merge > > > > > > This is just a (too) short introduction, I hope that it > > helps and that you find the time to dig somewhat deeper into > > the documentation. > > > > Other developers have that problem as well, you are not alone. > > > > If you have a public repo, I could help with one example. > > > > > > > > > > Regards, > > > > > > Scott Richmond. > > > Director, Producer, Programmer > > > Brightrock Games > > > T: 07480795661 On Wed, Dec 18, 2019 at 09:33:32PM +0000, Scott Richmond wrote: > Hey Torsten, > > Thanks for the reply! > > Correct me if am wrong, but those steps don't tell git to "ignore" > line endings. That just causes git to check all text files in and out > with a specific EOL type (Either automatically chosen, or not). If an > app in the dev env changes a files' EOL to something else, git will > notice the change locally. > > Regards, > > Scott Richmond. > Director, Producer, Programmer > Brightrock Games > T: 07480795661 > Hej Scott, I am not sure whether I understand your question correctly. So I set up a little test, to illustrate things better. user@linux:/tmp/EOLtest $ git init Initialized empty Git repository in /tmp/EOLtest/.git/ user@linux:/tmp/EOLtest $ printf "Line1\r\nLine2\r\n" >file user@linux:/tmp/EOLtest $ git add file user@linux:/tmp/EOLtest $ git commit -m "add file with CRLF" [master (root-commit) 0e4d1df] add file with CRLF 1 file changed, 2 insertions(+) create mode 100644 file user@linux:/tmp/EOLtest $ printf "Line1\nLine2\n" >file user@linux:/tmp/EOLtest $ git status On branch master Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: file no changes added to commit (use "git add" and/or "git commit -a") user@linux:/tmp/EOLtest $ git diff diff --git a/file b/file index 4aa551d..ac2dd81 100644 --- a/file +++ b/file @@ -1,2 +1,2 @@ -Line1 -Line2 +Line1 +Line2 user@linux:/tmp/EOLtest $ echo "* text=auto" >.gitattributes user@linux:/tmp/EOLtest $ git add --renormalize . user@linux:/tmp/EOLtest $ git commit -m "Normalize the repo" [master b41136d] Normalize the repo 1 file changed, 2 insertions(+), 2 deletions(-) user@linux:/tmp/EOLtest $ user@linux:/tmp/EOLtest $ printf "Line1\r\nLine2\r\n" >file user@linux:/tmp/EOLtest $ git status On branch master Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: file user@linux:/tmp/EOLtest $ git diff warning: CRLF will be replaced by LF in file. The file will have its original line endings in your working directory ############################# No, at this point, I am surprised myself. "file" is reported as "modified", but it should not be modified, right? Is that the problem, that you have ? We can fix it, by running: user@linux:/tmp/EOLtest $ git add file user@linux:/tmp/EOLtest $ git status On branch master nothing to commit, working tree clean But I still think, that this "modified" is unexpected, and a (possible) bug in Git, thanks for reporting. Which Git version do you use ? Is this test script a description of you problem ? ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: Ability to ignore EOL changes for certain projects 2019-12-19 3:12 ` Torsten Bögershausen @ 2019-12-19 13:25 ` Scott Richmond 0 siblings, 0 replies; 9+ messages in thread From: Scott Richmond @ 2019-12-19 13:25 UTC (permalink / raw) To: Torsten Bögershausen; +Cc: git > Scott, please avoid top-posting in this list, see my answers at the end. Sure! Sorry I rarely use mail lists. > But I still think, that this "modified" is unexpected, > and a (possible) bug in Git, thanks for reporting. > > Which Git version do you use ? > Is this test script a description of you problem ? Yes this appears to be a correct repro of the kind of scenario that is common in many dev environments like ours. I'm not yet sure if this is actually a bug (I'd be happy if it is!). Reading the docs I think this is intended behaviour - All the text CRLF options relate to what Git should do on check in/out. It is correctly reporting the file change, even if it is just EOL. What Philip is describing with the sha1/iod check sounds to me what the root cause "problem" is - Any file change will show up on git. Philip appears to imply that there is no way to get git to do further introspection on the file at this time. But maybe that can be added? Re the Git version - I try to stay on the latest at all times. This issue has been present for years, for as long as I can remember. Regards, Scott Richmond. Director, Producer, Programmer Brightrock Games T: 07480795661 On Thu, Dec 19, 2019 at 3:12 AM Torsten Bögershausen <tboegi@web.de> wrote: > > Scott, please avoid top-posting in this list, see my answers at the end. > > > On Wed, Dec 18, 2019 at 7:27 PM Torsten Bögershausen <tboegi@web.de> wrote: > > > > > > On Wed, Dec 18, 2019 at 11:10:27AM +0000, Scott Richmond wrote: > > > > The Problem Domain > > > > In certain dev environments (Unity3D projects) there is (AFAIK) an > > > > unsolvable problem where some files are often modified with line > > > > endings that aren't the native system or not the committed line > > > > endings for that file. Secondarily, in this case line endings don't > > > > matter - Nothing in the dev environment "cares" which kind of line > > > > ending is used. > > > > > > > > The Problem > > > > Git always cares about EOL. Git has options to transparently modify > > > > EOLs when files are checked in or out. However it is not possible to > > > > tell Git to ignore EOLs in other commands: > > > > Git status shows the file modified. > > > > Merging/Pulling has to care because it can't merge with a modified > > > > working tree. Which means the user has to care - They have to either > > > > stash the EOL changes or wipe them out. Sometimes, if the user has a > > > > particular app running, it may automatically reload that file and > > > > recreate the modified EOL changes, causing an endless loop. This > > > > problem is often made unclear to the user how to solve, especially if > > > > they aren't domain experts. > > > > > > > > To be clear, in this particular dev environment, I can say with > > > > confidence that this particular issue is a major and common pain point > > > > for users. It is made worse as many users in this environment aren't > > > > programmers by trade and aren't domain experts in version control. I > > > > also believe this environment is becoming a non-trivial portion of the > > > > Git userbase and it would be worthwhile looking into resolving. > > > > > > > > Solution Request > > > > It would be fantastic if we could tell Git to stop caring about EOL > > > > changes on a per-repo basis, with the effective output being that git > > > > status shouldn't show changes for files with differing EOLs. > > > > > > > > I'm experienced with Git, though I am not expert enough to consider > > > > creating such a change myself - It is unclear to me just how > > > > complicated a change may be. However maybe I could look into it if it > > > > was made clear that this improvement is possible and has no serious > > > > side effects. > > > > > > Hej Scott, > > > I think that you problem can be solved. > > > For each repository, you can tell Git to ignore the line endings, > > > CRLF vs LF. > > > > > > If you start with a fresh repo, > > > you can do something like: > > > > > > echo "* text=auto" >.gitattributes > > > git add .gitattributes > > > git commit -m "Add .gitattributes" > > > > > > For existing repos, we need to take another step: > > > > > > echo "* text=auto" >.gitattributes > > > git add .gitattributes > > > git add --renormlize . > > > git commit -m "Add .gitattributes" > > > > > > More information can be found e.g. here: > > > https://git-scm.com/docs/git-add > > > > > > Once you done that, you can merge branches > > > into the master branch with help of the renormalize > > > > > > git merge -X renormalze branch-name > > > > > > See even here: > > > https://git-scm.com/docs/git-merge > > > > > > > > > This is just a (too) short introduction, I hope that it > > > helps and that you find the time to dig somewhat deeper into > > > the documentation. > > > > > > Other developers have that problem as well, you are not alone. > > > > > > If you have a public repo, I could help with one example. > > > > > > > > > > > > > > Regards, > > > > > > > > Scott Richmond. > > > > Director, Producer, Programmer > > > > Brightrock Games > > > > T: 07480795661 > On Wed, Dec 18, 2019 at 09:33:32PM +0000, Scott Richmond wrote: > > Hey Torsten, > > > > Thanks for the reply! > > > > Correct me if am wrong, but those steps don't tell git to "ignore" > > line endings. That just causes git to check all text files in and out > > with a specific EOL type (Either automatically chosen, or not). If an > > app in the dev env changes a files' EOL to something else, git will > > notice the change locally. > > > > Regards, > > > > Scott Richmond. > > Director, Producer, Programmer > > Brightrock Games > > T: 07480795661 > > > > Hej Scott, > > I am not sure whether I understand your question correctly. > So I set up a little test, to illustrate things better. > > > user@linux:/tmp/EOLtest $ git init > Initialized empty Git repository in /tmp/EOLtest/.git/ > user@linux:/tmp/EOLtest $ printf "Line1\r\nLine2\r\n" >file > user@linux:/tmp/EOLtest $ git add file > user@linux:/tmp/EOLtest $ git commit -m "add file with CRLF" > [master (root-commit) 0e4d1df] add file with CRLF > 1 file changed, 2 insertions(+) > create mode 100644 file > > user@linux:/tmp/EOLtest $ printf "Line1\nLine2\n" >file > user@linux:/tmp/EOLtest $ git status > On branch master > Changes not staged for commit: > (use "git add <file>..." to update what will be committed) > (use "git checkout -- <file>..." to discard changes in working directory) > > modified: file > > no changes added to commit (use "git add" and/or "git commit -a") > user@linux:/tmp/EOLtest $ git diff > diff --git a/file b/file > index 4aa551d..ac2dd81 100644 > --- a/file > +++ b/file > @@ -1,2 +1,2 @@ > -Line1 > -Line2 > +Line1 > +Line2 > > user@linux:/tmp/EOLtest $ echo "* text=auto" >.gitattributes > user@linux:/tmp/EOLtest $ git add --renormalize . > user@linux:/tmp/EOLtest $ git commit -m "Normalize the repo" > [master b41136d] Normalize the repo > 1 file changed, 2 insertions(+), 2 deletions(-) > user@linux:/tmp/EOLtest $ > > user@linux:/tmp/EOLtest $ printf "Line1\r\nLine2\r\n" >file > user@linux:/tmp/EOLtest $ git status > On branch master > Changes not staged for commit: > (use "git add <file>..." to update what will be committed) > (use "git checkout -- <file>..." to discard changes in working directory) > > modified: file > > user@linux:/tmp/EOLtest $ git diff > warning: CRLF will be replaced by LF in file. > The file will have its original line endings in your working directory > > ############################# > No, at this point, I am surprised myself. > "file" is reported as "modified", but it should not be modified, right? > Is that the problem, that you have ? > > > We can fix it, by running: > > user@linux:/tmp/EOLtest $ git add file > user@linux:/tmp/EOLtest $ git status > On branch master > nothing to commit, working tree clean > > But I still think, that this "modified" is unexpected, > and a (possible) bug in Git, thanks for reporting. > > Which Git version do you use ? > Is this test script a description of you problem ? > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Ability to ignore EOL changes for certain projects 2019-12-18 11:10 Ability to ignore EOL changes for certain projects Scott Richmond 2019-12-18 19:27 ` Torsten Bögershausen @ 2019-12-20 3:02 ` brian m. carlson 2019-12-20 10:58 ` Scott Richmond 1 sibling, 1 reply; 9+ messages in thread From: brian m. carlson @ 2019-12-20 3:02 UTC (permalink / raw) To: Scott Richmond; +Cc: git [-- Attachment #1: Type: text/plain, Size: 4270 bytes --] On 2019-12-18 at 11:10:27, Scott Richmond wrote: > The Problem Domain > In certain dev environments (Unity3D projects) there is (AFAIK) an > unsolvable problem where some files are often modified with line > endings that aren't the native system or not the committed line > endings for that file. Secondarily, in this case line endings don't > matter - Nothing in the dev environment "cares" which kind of line > ending is used. I'm not sure this syncs up with your statements below. Typically, programs that do editing of text files either (a) write the line endings that the file started out with or (b) write the system line endings. If you set the appropriate files with the `text` attribute in `.gitattributes` (or use `* text=auto` if appropriate), then Git will always write in the system line endings and convert internally to LF, and programs that do either (a) or (b) will work. Yet it sounds like you have development tools that don't do either (a) or (b): they write in some fixed line ending, and therefore care very much which line ending is being used. In such a case, you could write (for example) `*.yaml text eol=crlf` (or `eol=lf`) if you always want them to do that and Git will convert to those line endings and store in LF. If the problem is text editor settings, you can use a `.editorconfig` file, which is a cross-platform text editor configuration that can specify line endings. Most text editors can be configured to honor such settings, although it may require a plugin. If your problem is a shared Windows / Linux environment like WSL, you can set `core.eol` to `crlf` in the repository and things will work. If you need settings like this, you can even set them up appropriately for the system using a make target or bootstrap script so you don't need to do that by hand on each system. If you do this, then your tools will write the same line endings as are checked out, and files won't appear modified. You can see how Git itself uses this to set up files appropriately for different systems. The only case this wouldn't work is if the tools wrote some random line endings depending on an attribute other than the OS they're on, or if you had multiple tools doing different things. If that's really your problem, then yes, you'd need a new Git feature. It is of course possible to use a filter to strip out all carriage returns, but that doesn't prevent Git from showing the file as being modified. What I've proposed, of course, requires some setup work and configuration. It isn't trivial, but it does work for a lot of projects already. > Solution Request > It would be fantastic if we could tell Git to stop caring about EOL > changes on a per-repo basis, with the effective output being that git > status shouldn't show changes for files with differing EOLs. > > I'm experienced with Git, though I am not expert enough to consider > creating such a change myself - It is unclear to me just how > complicated a change may be. However maybe I could look into it if it > was made clear that this improvement is possible and has no serious > side effects. I'm not sure how such a feature would interact with how Git operates when it re-reads the index. It isn't 100% clear to me when data is filtered through various text filters such as EOL filters. All the filtering I've used is one-to-one, and therefore any modification of the file contents necessarily means that the indexed contents have changed. If Git does apply such filters when refreshing the index (such as happens before `git status`), then such a feature would be relatively easy to implement, although you'd incur a performance penalty when changing the EOL of a file, even if the file were otherwise identical. I suspect it would make the most sense as an additional value for `core.autocrlf`. If Git doesn't apply those filters, then there really isn't a way to do what you want without fundamentally changing the characteristics of how Git operates on the index, since it would still show files as modified. Maybe someone else can comment on the feasibility of this better than I can. -- brian m. carlson: Houston, Texas, US OpenPGP: https://keybase.io/bk2204 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 868 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Ability to ignore EOL changes for certain projects 2019-12-20 3:02 ` brian m. carlson @ 2019-12-20 10:58 ` Scott Richmond 0 siblings, 0 replies; 9+ messages in thread From: Scott Richmond @ 2019-12-20 10:58 UTC (permalink / raw) To: brian m. carlson, Scott Richmond, git > I'm not sure this syncs up with your statements below...[snip] Unfortunately it is the latter case where certain dev apps, or plugins within dev apps, that can occasionally write incorrect line endings. We don't have configurable control over these things unfortunately. If I widen the scope to all the scenarios you presented though - My feature request could still have value as another way to alleviate the pressure EOL changes create on a git repo. I would agree that it would be best to solve the problem at its source where possible, but it is sometimes not possible. > I'm not sure how such a feature would interact with how Git operates when it re-reads the index... AFAIK git status does not filter the results, as that would push writes to files (I think?) when the user wouldn't expect it to. So it makes sense that actions such as git status are unaffected by text/crlf options. I agree the additional cost to run a filter or similar action with git status would be a cause for concern, but maybe an optional and acceptable trade-off for the pain it alleviates elsewhere. I do not know the underlying way git works either, but I suspect it would have to be a 3rd test after sha1 & oid against the files that come out of git status. I'm guessing that is an area of git that has little configurability and therefore very difficult to make changes to. :/ Regards, Scott Richmond. Director, Producer, Programmer Brightrock Games T: 07480795661 On Fri, Dec 20, 2019 at 3:03 AM brian m. carlson <sandals@crustytoothpaste.net> wrote: > > On 2019-12-18 at 11:10:27, Scott Richmond wrote: > > The Problem Domain > > In certain dev environments (Unity3D projects) there is (AFAIK) an > > unsolvable problem where some files are often modified with line > > endings that aren't the native system or not the committed line > > endings for that file. Secondarily, in this case line endings don't > > matter - Nothing in the dev environment "cares" which kind of line > > ending is used. > > I'm not sure this syncs up with your statements below. Typically, > programs that do editing of text files either (a) write the line endings > that the file started out with or (b) write the system line endings. If > you set the appropriate files with the `text` attribute in > `.gitattributes` (or use `* text=auto` if appropriate), then Git will > always write in the system line endings and convert internally to LF, > and programs that do either (a) or (b) will work. > > Yet it sounds like you have development tools that don't do either (a) > or (b): they write in some fixed line ending, and therefore care very > much which line ending is being used. In such a case, you could > write (for example) `*.yaml text eol=crlf` (or `eol=lf`) if you always > want them to do that and Git will convert to those line endings and > store in LF. > > If the problem is text editor settings, you can use a `.editorconfig` > file, which is a cross-platform text editor configuration that can > specify line endings. Most text editors can be configured to honor such > settings, although it may require a plugin. > > If your problem is a shared Windows / Linux environment like WSL, you > can set `core.eol` to `crlf` in the repository and things will work. If > you need settings like this, you can even set them up appropriately for > the system using a make target or bootstrap script so you don't need to > do that by hand on each system. > > If you do this, then your tools will write the same line endings as are > checked out, and files won't appear modified. You can see how Git > itself uses this to set up files appropriately for different systems. > > The only case this wouldn't work is if the tools wrote some random line > endings depending on an attribute other than the OS they're on, or if > you had multiple tools doing different things. If that's really your > problem, then yes, you'd need a new Git feature. It is of course > possible to use a filter to strip out all carriage returns, but that > doesn't prevent Git from showing the file as being modified. > > What I've proposed, of course, requires some setup work and > configuration. It isn't trivial, but it does work for a lot of projects > already. > > > Solution Request > > It would be fantastic if we could tell Git to stop caring about EOL > > changes on a per-repo basis, with the effective output being that git > > status shouldn't show changes for files with differing EOLs. > > > > I'm experienced with Git, though I am not expert enough to consider > > creating such a change myself - It is unclear to me just how > > complicated a change may be. However maybe I could look into it if it > > was made clear that this improvement is possible and has no serious > > side effects. > > I'm not sure how such a feature would interact with how Git operates > when it re-reads the index. It isn't 100% clear to me when data is > filtered through various text filters such as EOL filters. All the > filtering I've used is one-to-one, and therefore any modification of the > file contents necessarily means that the indexed contents have changed. > > If Git does apply such filters when refreshing the index (such as > happens before `git status`), then such a feature would be relatively > easy to implement, although you'd incur a performance penalty when > changing the EOL of a file, even if the file were otherwise identical. > I suspect it would make the most sense as an additional value for > `core.autocrlf`. > > If Git doesn't apply those filters, then there really isn't a way to do > what you want without fundamentally changing the characteristics of how > Git operates on the index, since it would still show files as modified. > > Maybe someone else can comment on the feasibility of this better than I > can. > -- > brian m. carlson: Houston, Texas, US > OpenPGP: https://keybase.io/bk2204 ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2019-12-20 10:59 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-12-18 11:10 Ability to ignore EOL changes for certain projects Scott Richmond 2019-12-18 19:27 ` Torsten Bögershausen 2019-12-18 21:33 ` Scott Richmond 2019-12-18 22:34 ` Philip Oakley 2019-12-19 13:39 ` Scott Richmond 2019-12-19 3:12 ` Torsten Bögershausen 2019-12-19 13:25 ` Scott Richmond 2019-12-20 3:02 ` brian m. carlson 2019-12-20 10:58 ` Scott Richmond
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).