* [PATCH 0/2] enable "svn.pathnameencoding" on dcommit @ 2016-02-08 15:19 Kazutoshi Satoda 2016-02-08 15:20 ` [PATCH 1/2] git-svn: " Kazutoshi Satoda ` (2 more replies) 0 siblings, 3 replies; 25+ messages in thread From: Kazutoshi Satoda @ 2016-02-08 15:19 UTC (permalink / raw) To: git; +Cc: normalperson, alex.crezoff These are small fixes to problems I encountered using git-svn with svn.pathnameencoding configuration (cp932 in my case). The problems happen only when sending changes on non-ASCII paths. I'm sorry not coming with test scripts, but I couldn't figure out how to write tests to reproduce problems happen only with non-UTF-8 paths while the tests seems to be run on UTF-8 locale. I just ran existing tests with these fixes and there was no breakage (with UTF-8 locale). I think sending these fixes as-is is better than not sending because of my lack of ability to write such tests. I also expect that these fixes can be applied as-is because there were some small fixes without tests in the history of file perl/Git/SVN/Editor.pm. The first one is an updated fix of an very old but not applied patch. http://comments.gmane.org/gmane.comp.version-control.git/164166#o2 The second one can be seen as a separate fix, but it takes effect only after the first one. Kazutoshi SATODA (2): git-svn: enable "svn.pathnameencoding" on dcommit git-svn: apply "svn.pathnameencoding" before URL encoding perl/Git/SVN/Editor.pm | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) -- 2.7.0 ^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH 1/2] git-svn: enable "svn.pathnameencoding" on dcommit 2016-02-08 15:19 [PATCH 0/2] enable "svn.pathnameencoding" on dcommit Kazutoshi Satoda @ 2016-02-08 15:20 ` Kazutoshi Satoda 2016-02-15 0:30 ` Eric Wong 2016-02-08 15:21 ` [PATCH 2/2] git-svn: apply "svn.pathnameencoding" before URL encoding Kazutoshi Satoda 2016-02-08 22:58 ` [PATCH 0/2] enable "svn.pathnameencoding" on dcommit Eric Wong 2 siblings, 1 reply; 25+ messages in thread From: Kazutoshi Satoda @ 2016-02-08 15:20 UTC (permalink / raw) To: git; +Cc: normalperson, alex.crezoff Without the initialization of $self->{pathnameencoding}, conversion in repo_path() is always skipped as $self->{pathnameencoding} is undefined even if "svn.pathnameencoding" is configured. The lack of conversion results in mysterious failure of dcommit (e.g. "Malformed XML") which happen only when a commit involves a change on non-ASCII path. Signed-off-by: Kazutoshi SATODA <k_satoda@f2.dion.ne.jp> --- perl/Git/SVN/Editor.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/perl/Git/SVN/Editor.pm b/perl/Git/SVN/Editor.pm index c50176e..d9d9bdf 100644 --- a/perl/Git/SVN/Editor.pm +++ b/perl/Git/SVN/Editor.pm @@ -41,6 +41,7 @@ sub new { "$self->{svn_path}/" : ''; $self->{config} = $opts->{config}; $self->{mergeinfo} = $opts->{mergeinfo}; + $self->{pathnameencoding} = Git::config('svn.pathnameencoding'); return $self; } -- 2.7.0 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PATCH 1/2] git-svn: enable "svn.pathnameencoding" on dcommit 2016-02-08 15:20 ` [PATCH 1/2] git-svn: " Kazutoshi Satoda @ 2016-02-15 0:30 ` Eric Wong 0 siblings, 0 replies; 25+ messages in thread From: Eric Wong @ 2016-02-15 0:30 UTC (permalink / raw) To: Kazutoshi Satoda; +Cc: git, alex.crezoff Kazutoshi Satoda <k_satoda@f2.dion.ne.jp> wrote: > Signed-off-by: Kazutoshi SATODA <k_satoda@f2.dion.ne.jp> Signed-off-by: Eric Wong <normalperson@yhbt.net> Thanks, it took me some time to figure out the test case which I will amend: diff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh index 6a48e40..82222fd 100755 --- a/t/t9115-git-svn-dcommit-funky-renames.sh +++ b/t/t9115-git-svn-dcommit-funky-renames.sh @@ -77,11 +77,21 @@ test_expect_success 'make a commit to test rebase' ' ' test_expect_success 'git svn rebase works inside a fresh-cloned repository' ' - cd test-rebase && + ( + cd test-rebase && git svn rebase && test -e test-rebase-main && test -e test-rebase - ' + )' + +test_expect_success 'svn.pathnameencoding=cp932 new file on dcommit' ' + neq=$(printf "\201\202") && + git config svn.pathnameencoding cp932 && + echo neq >"$neq" && + git add "$neq" && + git commit -m "neq" && + git svn dcommit +' stop_httpd -- EW ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 2/2] git-svn: apply "svn.pathnameencoding" before URL encoding 2016-02-08 15:19 [PATCH 0/2] enable "svn.pathnameencoding" on dcommit Kazutoshi Satoda 2016-02-08 15:20 ` [PATCH 1/2] git-svn: " Kazutoshi Satoda @ 2016-02-08 15:21 ` Kazutoshi Satoda 2016-02-15 0:33 ` Eric Wong 2016-02-08 22:58 ` [PATCH 0/2] enable "svn.pathnameencoding" on dcommit Eric Wong 2 siblings, 1 reply; 25+ messages in thread From: Kazutoshi Satoda @ 2016-02-08 15:21 UTC (permalink / raw) To: git; +Cc: normalperson, alex.crezoff The conversion from "svn.pathnameencoding" to UTF-8 should be applied first, and then URL encoding should be applied on the resulting UTF-8 path. The reversed order of these transforms (used before this fix) makes non-UTF-8 URL which causes error from Subversion such as "Filesystem has no item: '...' path not found" when sending a rename (or a copy) from non-ASCII path. Signed-off-by: Kazutoshi SATODA <k_satoda@f2.dion.ne.jp> --- perl/Git/SVN/Editor.pm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/perl/Git/SVN/Editor.pm b/perl/Git/SVN/Editor.pm index d9d9bdf..4c4199a 100644 --- a/perl/Git/SVN/Editor.pm +++ b/perl/Git/SVN/Editor.pm @@ -144,11 +144,12 @@ sub repo_path { sub url_path { my ($self, $path) = @_; + $path = $self->repo_path($path); if ($self->{url} =~ m#^https?://#) { # characters are taken from subversion/libsvn_subr/path.c $path =~ s#([^~a-zA-Z0-9_./!$&'()*+,-])#sprintf("%%%02X",ord($1))#eg; } - $self->{url} . '/' . $self->repo_path($path); + $self->{url} . '/' . $path; } sub rmdirs { -- 2.7.0 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PATCH 2/2] git-svn: apply "svn.pathnameencoding" before URL encoding 2016-02-08 15:21 ` [PATCH 2/2] git-svn: apply "svn.pathnameencoding" before URL encoding Kazutoshi Satoda @ 2016-02-15 0:33 ` Eric Wong 0 siblings, 0 replies; 25+ messages in thread From: Eric Wong @ 2016-02-15 0:33 UTC (permalink / raw) To: Kazutoshi Satoda; +Cc: git, alex.crezoff Kazutoshi Satoda <k_satoda@f2.dion.ne.jp> wrote: > The conversion from "svn.pathnameencoding" to UTF-8 should be applied > first, and then URL encoding should be applied on the resulting UTF-8 > path. The reversed order of these transforms (used before this fix) > makes non-UTF-8 URL which causes error from Subversion such as > "Filesystem has no item: '...' path not found" when sending a rename (or > a copy) from non-ASCII path. > > Signed-off-by: Kazutoshi SATODA <k_satoda@f2.dion.ne.jp> Thanks, running full SVN tests now. Signed-off-by: Eric Wong <normalperson@yhbt.net> > --- a/perl/Git/SVN/Editor.pm > +++ b/perl/Git/SVN/Editor.pm > @@ -144,11 +144,12 @@ sub repo_path { > > sub url_path { > my ($self, $path) = @_; > + $path = $self->repo_path($path); > if ($self->{url} =~ m#^https?://#) { > # characters are taken from subversion/libsvn_subr/path.c > $path =~ s#([^~a-zA-Z0-9_./!$&'()*+,-])#sprintf("%%%02X",ord($1))#eg; > } > - $self->{url} . '/' . $self->repo_path($path); > + $self->{url} . '/' . $path; > } This is trickier to test, as it requires an https?:// URL. It always succeeds with the default file:// URL. diff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh index 82222fd..9828f05 100755 --- a/t/t9115-git-svn-dcommit-funky-renames.sh +++ b/t/t9115-git-svn-dcommit-funky-renames.sh @@ -93,6 +93,18 @@ test_expect_success 'svn.pathnameencoding=cp932 new file on dcommit' ' git svn dcommit ' +test_expect_success 'svn.pathnameencoding=cp932 rename on dcommit' ' + inf=$(printf "\201\207") && + git config svn.pathnameencoding cp932 && + echo inf >"$inf" && + git add "$inf" && + git commit -m "inf" && + git svn dcommit && + git mv "$inf" inf && + git commit -m "inf rename" && + git svn dcommit +' + stop_httpd test_done -- EW ^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PATCH 0/2] enable "svn.pathnameencoding" on dcommit 2016-02-08 15:19 [PATCH 0/2] enable "svn.pathnameencoding" on dcommit Kazutoshi Satoda 2016-02-08 15:20 ` [PATCH 1/2] git-svn: " Kazutoshi Satoda 2016-02-08 15:21 ` [PATCH 2/2] git-svn: apply "svn.pathnameencoding" before URL encoding Kazutoshi Satoda @ 2016-02-08 22:58 ` Eric Wong 2016-02-15 0:52 ` [PULL] svn pathnameencoding for git svn dcommit Eric Wong 2016-02-27 18:28 ` [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS tboegi 2 siblings, 2 replies; 25+ messages in thread From: Eric Wong @ 2016-02-08 22:58 UTC (permalink / raw) To: Kazutoshi Satoda; +Cc: git, alex.crezoff Kazutoshi Satoda <k_satoda@f2.dion.ne.jp> wrote: > I'm sorry not coming with test scripts, but I couldn't figure out how to > write tests to reproduce problems happen only with non-UTF-8 paths while > the tests seems to be run on UTF-8 locale. Thank you. I will try to work on some tests throughout the week (help appreciated!) but will be willing to push these up to Junio regardless. ^ permalink raw reply [flat|nested] 25+ messages in thread
* [PULL] svn pathnameencoding for git svn dcommit 2016-02-08 22:58 ` [PATCH 0/2] enable "svn.pathnameencoding" on dcommit Eric Wong @ 2016-02-15 0:52 ` Eric Wong 2016-02-15 21:32 ` Junio C Hamano 2016-02-16 3:29 ` Kazutoshi Satoda 2016-02-27 18:28 ` [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS tboegi 1 sibling, 2 replies; 25+ messages in thread From: Eric Wong @ 2016-02-15 0:52 UTC (permalink / raw) To: Junio C Hamano; +Cc: git, alex.crezoff, Kazutoshi Satoda I've amended tests to both commits, but the URL encoding one requires an HTTP server to test effectively. I couldn't find a test prereq for httpd, but perhaps it's good to test by default regardless in case a future SVN changes file:// behavior. I've only tested this with SVN 1.6.17 under Debian wheezy. The following changes since commit 6faf27b4ff26804a07363078b238b5cfd3dfa976: Merge branch 'tb/conversion' into next (2016-02-12 10:20:20 -0800) are available in the git repository at: git://bogomips.org/git-svn.git ks/svn-pathnameencoding for you to fetch changes up to dfee0cf8123e7f63268f05a02731ce82db136188: git-svn: apply "svn.pathnameencoding" before URL encoding (2016-02-15 00:31:21 +0000) ---------------------------------------------------------------- Kazutoshi Satoda (2): git-svn: enable "svn.pathnameencoding" on dcommit git-svn: apply "svn.pathnameencoding" before URL encoding perl/Git/SVN/Editor.pm | 4 +++- t/t9115-git-svn-dcommit-funky-renames.sh | 26 ++++++++++++++++++++++++-- 2 files changed, 27 insertions(+), 3 deletions(-) ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PULL] svn pathnameencoding for git svn dcommit 2016-02-15 0:52 ` [PULL] svn pathnameencoding for git svn dcommit Eric Wong @ 2016-02-15 21:32 ` Junio C Hamano 2016-02-16 3:29 ` Kazutoshi Satoda 1 sibling, 0 replies; 25+ messages in thread From: Junio C Hamano @ 2016-02-15 21:32 UTC (permalink / raw) To: Eric Wong; +Cc: git, alex.crezoff, Kazutoshi Satoda Eric Wong <normalperson@yhbt.net> writes: > I've amended tests to both commits, but the URL encoding one > requires an HTTP server to test effectively. > > I couldn't find a test prereq for httpd, but perhaps it's good > to test by default regardless in case a future SVN changes > file:// behavior. I've only tested this with SVN 1.6.17 under > Debian wheezy. > > The following changes since commit 6faf27b4ff26804a07363078b238b5cfd3dfa976: > > Merge branch 'tb/conversion' into next (2016-02-12 10:20:20 -0800) > > are available in the git repository at: > > git://bogomips.org/git-svn.git ks/svn-pathnameencoding > > for you to fetch changes up to dfee0cf8123e7f63268f05a02731ce82db136188: > > git-svn: apply "svn.pathnameencoding" before URL encoding (2016-02-15 00:31:21 +0000) Sorry, Eric, I cannot pull this. You made your branch on top of 'next', there are many commits that are not ready. > > ---------------------------------------------------------------- > Kazutoshi Satoda (2): > git-svn: enable "svn.pathnameencoding" on dcommit > git-svn: apply "svn.pathnameencoding" before URL encoding > > perl/Git/SVN/Editor.pm | 4 +++- > t/t9115-git-svn-dcommit-funky-renames.sh | 26 ++++++++++++++++++++++++-- > 2 files changed, 27 insertions(+), 3 deletions(-) ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PULL] svn pathnameencoding for git svn dcommit 2016-02-15 0:52 ` [PULL] svn pathnameencoding for git svn dcommit Eric Wong 2016-02-15 21:32 ` Junio C Hamano @ 2016-02-16 3:29 ` Kazutoshi Satoda 2016-02-16 6:33 ` Eric Wong 1 sibling, 1 reply; 25+ messages in thread From: Kazutoshi Satoda @ 2016-02-16 3:29 UTC (permalink / raw) To: Eric Wong, Junio C Hamano; +Cc: git, alex.crezoff On 2016/02/15 9:52 +0900, Eric Wong wrote: > I've amended tests to both commits, but the URL encoding one > requires an HTTP server to test effectively. Thank you for the tests. But, on my environment, both of them failed unexpectedly. (Windows 7 SP1, x86_64 Cygwin, LANG=ja_JP.UTF-8) For now, I don't have enough time to investigate this further. Let me just share the result. > $ ./t9115-git-svn-dcommit-funky-renames.sh -x --run='11-12' (snip) > expecting success: > neq=$(printf "\201\202") && > git config svn.pathnameencoding cp932 && > echo neq >"$neq" && > git add "$neq" && > git commit -m "neq" && > git svn dcommit > > +++ printf '\201\202' > ++ neq=$'\201\202' > ++ git config svn.pathnameencoding cp932 > ++ echo neq > ++ git add $'\201\202' > ++ git commit -m neq > On branch master > > Initial commit > > Untracked files: > svnrepo/ > "\357\202\201\357\202\202" > > nothing added to commit but untracked files present > error: last command exited with $?=1 > not ok 11 - svn.pathnameencoding=cp932 new file on dcommit > # > # neq=$(printf "\201\202") && > # git config svn.pathnameencoding cp932 && > # echo neq >"$neq" && > # git add "$neq" && > # git commit -m "neq" && > # git svn dcommit > # > > expecting success: > inf=$(printf "\201\207") && > git config svn.pathnameencoding cp932 && > echo inf >"$inf" && > git add "$inf" && > git commit -m "inf" && > git svn dcommit && > git mv "$inf" inf && > git commit -m "inf rename" && > git svn dcommit > > +++ printf '\201\207' > ++ inf=$'\201\207' > ++ git config svn.pathnameencoding cp932 > ++ echo inf > ++ git add $'\201\207' > ++ git commit -m inf > On branch master > > Initial commit > > Untracked files: > svnrepo/ > "\357\202\201\357\202\202" > "\357\202\201\357\202\207" > > nothing added to commit but untracked files present > error: last command exited with $?=1 > not ok 12 - svn.pathnameencoding=cp932 rename on dcommit > # > # inf=$(printf "\201\207") && > # git config svn.pathnameencoding cp932 && > # echo inf >"$inf" && > # git add "$inf" && > # git commit -m "inf" && > # git svn dcommit && > # git mv "$inf" inf && > # git commit -m "inf rename" && > # git svn dcommit > # Strangely, after the test failure, "git status" in the trash directory reports different status. > $ cd 'trash directory.t9115-git-svn-dcommit-funky-renames' > $ git status > On branch master > > Initial commit > > Untracked files: > (use "git add <file>..." to include in what will be committed) > > svnrepo/ > "\201\202" > "\201\207" > > nothing added to commit but untracked files present (use "git add" to track) I can manually add and commit them. > $ neq=$(printf "\201\202") > $ git add "$neq" > $ git commit -m "neq" > [master (root-commit) 3fd464f] neq > 1 file changed, 1 insertion(+) > create mode 100644 "\201\202" I can't see how "\357\202\201\357\202\202" came as output in the test. -- k_satoda ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PULL] svn pathnameencoding for git svn dcommit 2016-02-16 3:29 ` Kazutoshi Satoda @ 2016-02-16 6:33 ` Eric Wong 2016-02-16 16:19 ` Kazutoshi Satoda 0 siblings, 1 reply; 25+ messages in thread From: Eric Wong @ 2016-02-16 6:33 UTC (permalink / raw) To: Kazutoshi Satoda, Junio C Hamano; +Cc: git, alex.crezoff Junio, sorry about basing on next, I somehow thought I was going to need to depend on something there. Updated pull below if Kazutoshi can run the test effectively. Kazutoshi Satoda <k_satoda@f2.dion.ne.jp> wrote: > Thank you for the tests. But, on my environment, both of them failed > unexpectedly. (Windows 7 SP1, x86_64 Cygwin, LANG=ja_JP.UTF-8) > > For now, I don't have enough time to investigate this further. Let me > just share the result. <snip> > > Untracked files: > > svnrepo/ > > "\357\202\201\357\202\202" > > "\357\202\201\357\202\207" > > <snip> > I can't see how "\357\202\201\357\202\202" came as output in the test. I wonder if it's a shell or printf portability problem. I've repushed the branch against master and implemented the weird character use inside Perl scripts. Can you give it a try? The following changes since commit 494398473714dcbedb38b1ac79b531c7384b3bc4: Sixth batch for the 2.8 cycle (2016-02-10 14:24:14 -0800) are available in the git repository at: git://bogomips.org/git-svn.git ks/svn-pathnameencoding for you to fetch changes up to 7d7ea44ea5a1a38ee36402e6709e64c05650ba46: git-svn: apply "svn.pathnameencoding" before URL encoding (2016-02-16 06:25:01 +0000) ---------------------------------------------------------------- Kazutoshi Satoda (2): git-svn: enable "svn.pathnameencoding" on dcommit git-svn: apply "svn.pathnameencoding" before URL encoding perl/Git/SVN/Editor.pm | 4 +++- t/t9115-git-svn-dcommit-funky-renames.sh | 16 ++++++++++++++-- t/t9115/inf.perl | 12 ++++++++++++ t/t9115/neq.perl | 5 +++++ 4 files changed, 34 insertions(+), 3 deletions(-) create mode 100644 t/t9115/inf.perl create mode 100644 t/t9115/neq.perl t/t9115-git-svn-dcommit-funky-renames.sh | 16 +++------------- t/t9115/inf.perl | 12 ++++++++++++ t/t9115/neq.perl | 5 +++++ 3 files changed, 20 insertions(+), 13 deletions(-) Interdiff of test changes: diff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh index 9828f05..94698fe 100755 --- a/t/t9115-git-svn-dcommit-funky-renames.sh +++ b/t/t9115-git-svn-dcommit-funky-renames.sh @@ -84,25 +84,15 @@ test_expect_success 'git svn rebase works inside a fresh-cloned repository' ' test -e test-rebase )' -test_expect_success 'svn.pathnameencoding=cp932 new file on dcommit' ' - neq=$(printf "\201\202") && +test_expect_success PERL 'svn.pathnameencoding=cp932 new file on dcommit' ' git config svn.pathnameencoding cp932 && - echo neq >"$neq" && - git add "$neq" && + "$PERL_PATH" -w "$TEST_DIRECTORY"/t9115/neq.perl && git commit -m "neq" && git svn dcommit ' test_expect_success 'svn.pathnameencoding=cp932 rename on dcommit' ' - inf=$(printf "\201\207") && - git config svn.pathnameencoding cp932 && - echo inf >"$inf" && - git add "$inf" && - git commit -m "inf" && - git svn dcommit && - git mv "$inf" inf && - git commit -m "inf rename" && - git svn dcommit + "$PERL_PATH" -w "$TEST_DIRECTORY"/t9115/inf.perl ' stop_httpd diff --git a/t/t9115/inf.perl b/t/t9115/inf.perl new file mode 100644 index 0000000..0adcfc7 --- /dev/null +++ b/t/t9115/inf.perl @@ -0,0 +1,12 @@ +use strict; +my $path = "\201\207"; +open my $fh, '>', $path or die $!; +close $fh or die $!; +sub run { system(@_) == 0 or die join(' ', @_) . ": $?" } +run(qw(git config svn.pathnameencoding cp932)); +run(qw(git add), $path); +run(qw(git commit -m inf)); +run(qw(git svn dcommit)); +run(qw(git mv), $path, 'inf'); +run(qw(git commit -m inf-rename)); +run(qw(git svn dcommit)); diff --git a/t/t9115/neq.perl b/t/t9115/neq.perl new file mode 100644 index 0000000..91b1061 --- /dev/null +++ b/t/t9115/neq.perl @@ -0,0 +1,5 @@ +use strict; +my $path = "\201\202"; +open my $fh, '>', $path or die $!; +close $fh or die $!; +exec(qw(git add), $path); ^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PULL] svn pathnameencoding for git svn dcommit 2016-02-16 6:33 ` Eric Wong @ 2016-02-16 16:19 ` Kazutoshi Satoda 2016-02-20 23:37 ` Eric Wong 0 siblings, 1 reply; 25+ messages in thread From: Kazutoshi Satoda @ 2016-02-16 16:19 UTC (permalink / raw) To: Eric Wong, Junio C Hamano; +Cc: git, alex.crezoff [-- Attachment #1: Type: text/plain, Size: 3300 bytes --] On 2016/02/16 15:33 +0900, Eric Wong wrote: > Kazutoshi Satoda <k_satoda@f2.dion.ne.jp> wrote: >> Thank you for the tests. But, on my environment, both of them failed >> unexpectedly. (Windows 7 SP1, x86_64 Cygwin, LANG=ja_JP.UTF-8) ... >> > Untracked files: >> > svnrepo/ >> > "\357\202\201\357\202\202" >> > "\357\202\201\357\202\207" ... >> I can't see how "\357\202\201\357\202\202" came as output in the test. > > I wonder if it's a shell or printf portability problem. I've > repushed the branch against master and implemented the weird > character use inside Perl scripts. > > Can you give it a try? (Shouldn't the branch be based on maint, as these are bugfixes?) Thank you. I tried it but got similar problem: > $ ./t9115-git-svn-dcommit-funky-renames.sh -x --run='11-12' (snip) > expecting success: > git config svn.pathnameencoding cp932 && > "$PERL_PATH" -w "$TEST_DIRECTORY"/t9115/neq.perl && > git commit -m "neq" && > git svn dcommit > > ++ git config svn.pathnameencoding cp932 > ++ /usr/bin/perl -w /home/k_satoda/project/git/t/t9115/neq.perl > ++ git commit -m neq > On branch master > > Initial commit > > Untracked files: > svnrepo/ > "\357\202\201\357\202\202" > > nothing added to commit but untracked files present > error: last command exited with $?=1 > not ok 11 - svn.pathnameencoding=cp932 new file on dcommit > # > # git config svn.pathnameencoding cp932 && > # "$PERL_PATH" -w "$TEST_DIRECTORY"/t9115/neq.perl && > # git commit -m "neq" && > # git svn dcommit > # > > expecting success: > "$PERL_PATH" -w "$TEST_DIRECTORY"/t9115/inf.perl > > ++ /usr/bin/perl -w /home/k_satoda/project/git/t/t9115/inf.perl > On branch master > > Initial commit > > Untracked files: > svnrepo/ > "\357\202\201\357\202\202" > "\357\202\201\357\202\207" > > nothing added to commit but untracked files present > git commit -m inf: 256 at /home/k_satoda/project/git/t/t9115/inf.perl line 5. > error: last command exited with $?=1 > not ok 12 - svn.pathnameencoding=cp932 rename on dcommit > # > # "$PERL_PATH" -w "$TEST_DIRECTORY"/t9115/inf.perl > # I found how "\357\202\201\357\202\202" (U+F081 U+F082 in UTF-8) could come. https://cygwin.com/cygwin-ug-net/using-specialnames.html#pathnames-specialchars > Some characters are disallowed in filenames on Windows filesystems. ... ... > ... All of the above characters, except for the backslash, are converted > to special UNICODE characters in the range 0xf000 to 0xf0ff (the > "Private use area") when creating or accessing files. "U+F081 U+F082" seems the result of conversion from "0x8182" (neq in cp932) as treating each of 2 bytes as disallowed characters. And I also noticed that LANG and LC_ALL is set to "C" in test-lib.sh. Setting LC_ALL=C.UTF-8 in the test 11-12 made them pass on Cygwin. Same change made the previous version also pass. Please find the patch in the attached output of git format-patch. Could you please test with this on non-Cygwin environment? If it made no harm, please tell me what should I do to proceed this patch. Will you (Eric) please make further integration? Shall I make another series (v2) of patches? -- k_satoda [-- Attachment #2: 0001-Add-LC_ALL-C.UTF-8-in-t9115-git-svn-dcommit-funky-re.patch --] [-- Type: text/plain, Size: 1808 bytes --] >From 7b827f2d65aaa859030ba5b08055020f2bda1f0f Mon Sep 17 00:00:00 2001 From: Kazutoshi SATODA <k_satoda@f2.dion.ne.jp> Date: Wed, 17 Feb 2016 00:29:24 +0900 Subject: [PATCH] Add LC_ALL=C.UTF-8 in t9115-git-svn-dcommit-funky-renames.sh This makes the test 11-12 pass on Cygwin. --- t/t9115-git-svn-dcommit-funky-renames.sh | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh index 9828f05..59d086b 100755 --- a/t/t9115-git-svn-dcommit-funky-renames.sh +++ b/t/t9115-git-svn-dcommit-funky-renames.sh @@ -84,7 +84,16 @@ test_expect_success 'git svn rebase works inside a fresh-cloned repository' ' test -e test-rebase )' +# Without this, LC_ALL=C as set in test-lib.sh, and Cygwin converts +# non-ASCII characters in filenames unexpectedly, and causes errors. +# https://cygwin.com/cygwin-ug-net/using-specialnames.html#pathnames-specialchars +# > Some characters are disallowed in filenames on Windows filesystems. ... +# ... +# > ... All of the above characters, except for the backslash, are converted +# > to special UNICODE characters in the range 0xf000 to 0xf0ff (the +# > "Private use area") when creating or accessing files. test_expect_success 'svn.pathnameencoding=cp932 new file on dcommit' ' + export LC_ALL=C.UTF-8 && neq=$(printf "\201\202") && git config svn.pathnameencoding cp932 && echo neq >"$neq" && @@ -93,7 +102,9 @@ test_expect_success 'svn.pathnameencoding=cp932 new file on dcommit' ' git svn dcommit ' +# See the comment on the above test for setting of LC_ALL. test_expect_success 'svn.pathnameencoding=cp932 rename on dcommit' ' + export LC_ALL=C.UTF-8 && inf=$(printf "\201\207") && git config svn.pathnameencoding cp932 && echo inf >"$inf" && -- 2.7.0 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PULL] svn pathnameencoding for git svn dcommit 2016-02-16 16:19 ` Kazutoshi Satoda @ 2016-02-20 23:37 ` Eric Wong 2016-02-21 13:12 ` Kazutoshi Satoda 0 siblings, 1 reply; 25+ messages in thread From: Eric Wong @ 2016-02-20 23:37 UTC (permalink / raw) To: Kazutoshi Satoda; +Cc: Junio C Hamano, git, alex.crezoff Kazutoshi Satoda <k_satoda@f2.dion.ne.jp> wrote: > (Shouldn't the branch be based on maint, as these are bugfixes?) I'm not sure if it being previously left-out/untested feature would qualify for maint. Maybe it does, but I suppose I'll let Junio decide. > Thank you. I tried it but got similar problem: <snip> > I found how "\357\202\201\357\202\202" (U+F081 U+F082 in UTF-8) could > come. > https://cygwin.com/cygwin-ug-net/using-specialnames.html#pathnames-specialchars > > Some characters are disallowed in filenames on Windows filesystems. ... > ... > > ... All of the above characters, except for the backslash, are converted > > to special UNICODE characters in the range 0xf000 to 0xf0ff (the > > "Private use area") when creating or accessing files. > "U+F081 U+F082" seems the result of conversion from "0x8182" (neq in > cp932) as treating each of 2 bytes as disallowed characters. > > And I also noticed that LANG and LC_ALL is set to "C" in test-lib.sh. > > Setting LC_ALL=C.UTF-8 in the test 11-12 made them pass on Cygwin. > Same change made the previous version also pass. Please find the patch > in the attached output of git format-patch. Thanks. However, I also wonder what happens on machines without "C.UTF-8" support (are there still any?). > Could you please test with this on non-Cygwin environment? Works for me, at least. I've squashed your changes into the two patches already queued up. I needed to split the "export LC_ALL=C.UTF-8" statement into "LC_ALL=C.UTF-8 && export LC_ALL" for portability. > If it made no harm, please tell me what should I do to proceed this patch. > Will you (Eric) please make further integration? Shall I make another > series (v2) of patches? I've pushed out a new branch with your LC_ALL changes squashed in. However I'm unsure if there's any new portability problems with LC_ALL=C.UTF-8... Junio or anyone else: thoughts? The following changes since commit 0233b800c838ddda41db318ee396320b3c21a560: Merge branch 'maint' (2016-02-17 10:14:39 -0800) are available in the git repository at: git://bogomips.org/git-svn.git ks/svn-pathnameencoding-3 for you to fetch changes up to 980c083276ba06a9400c5b1b2558c3626bcff969: git-svn: apply "svn.pathnameencoding" before URL encoding (2016-02-20 23:30:16 +0000) ---------------------------------------------------------------- Kazutoshi Satoda (2): git-svn: enable "svn.pathnameencoding" on dcommit git-svn: apply "svn.pathnameencoding" before URL encoding perl/Git/SVN/Editor.pm | 4 +++- t/t9115-git-svn-dcommit-funky-renames.sh | 39 ++++++++++++++++++++++++++++++-- 2 files changed, 40 insertions(+), 3 deletions(-) ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PULL] svn pathnameencoding for git svn dcommit 2016-02-20 23:37 ` Eric Wong @ 2016-02-21 13:12 ` Kazutoshi Satoda 0 siblings, 0 replies; 25+ messages in thread From: Kazutoshi Satoda @ 2016-02-21 13:12 UTC (permalink / raw) To: Eric Wong; +Cc: Junio C Hamano, git, alex.crezoff On 2016/02/21 8:37 +0900, Eric Wong wrote: > Kazutoshi Satoda <k_satoda@f2.dion.ne.jp> wrote: ... >> Setting LC_ALL=C.UTF-8 in the test 11-12 made them pass on Cygwin. >> Same change made the previous version also pass. Please find the patch >> in the attached output of git format-patch. > > Thanks. However, I also wonder what happens on machines without > "C.UTF-8" support (are there still any?). > >> Could you please test with this on non-Cygwin environment? > > Works for me, at least. I've squashed your changes into the two > patches already queued up. I needed to split the > "export LC_ALL=C.UTF-8" statement into > "LC_ALL=C.UTF-8 && export LC_ALL" for portability. Thank you. >> If it made no harm, please tell me what should I do to proceed this patch. >> Will you (Eric) please make further integration? Shall I make another >> series (v2) of patches? > > I've pushed out a new branch with your LC_ALL changes squashed > in. However I'm unsure if there's any new portability problems > with LC_ALL=C.UTF-8... > > Junio or anyone else: thoughts? The test passed on my environment. I've searched use of LC_ALL values other than "C". It seems be the best to move the variable a_utf8_locale in t9129 to lib-git-svn.sh and use it also in t9115. t/Makefile:83 > $(MAKE) $(TSVN) GIT_SVN_NO_OPTIMIZE_COMMITS=0 LC_ALL=en_US.UTF-8 Here, "en_US.UTF-8" is hard-coded. I think this is at least more problematic than hard-coding "C.UTF-8". Beside hard-coding, does this take effect while test-lib.sh does LC_ALL=C ? t/t9129-git-svn-i18n-commitencoding.sh:17 > a_utf8_locale=$(locale -a | sed -n '/\.[uU][tT][fF]-*8$/{ > p > q > }') > > if test -n "$a_utf8_locale" > then > test_set_prereq UTF8 > else > say "# UTF-8 locale not available, some tests are skipped" > fi Here, a UTF-8 locale is took from "locale -a", and the test is skipped if not found. This gives "a_utf8_locale=C.utf8" on my Cygwin environment. There was a record that says the difference of ".utf8" and ".UTF-8" caused a failure. https://git.kernel.org/cgit/git/git.git/commit/?id=2de03ebe0635c93e182c3367140f999e79bdadcd t/lib-gettext.sh:17 > # is_IS.UTF-8 on Solaris and FreeBSD, is_IS.utf8 on Debian > is_IS_locale=$(locale -a 2>/dev/null | > sed -n '/^is_IS\.[uU][tT][fF]-*8$/{ > p > q > }') ... > if test -n "$is_IS_locale" && > test $GIT_INTERNAL_GETTEXT_SH_SCHEME != "fallthrough" > then > # Some of the tests need the reference Icelandic locale > test_set_prereq GETTEXT_LOCALE ... > else > say "# lib-gettext: No is_IS UTF-8 locale available" > fi (the same logic is used for is_IS.ISO8859-1.) Here, a UTF-8 locale with hard coded "is_IS" is took from "locale -a", and the test is skipped if not found. This gives "is_IS_locale=is_IS.utf8.utf8" on my Cygwin environment. -- k_satoda ^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS 2016-02-08 22:58 ` [PATCH 0/2] enable "svn.pathnameencoding" on dcommit Eric Wong 2016-02-15 0:52 ` [PULL] svn pathnameencoding for git svn dcommit Eric Wong @ 2016-02-27 18:28 ` tboegi 2016-02-28 4:59 ` Eric Wong 1 sibling, 1 reply; 25+ messages in thread From: tboegi @ 2016-02-27 18:28 UTC (permalink / raw) To: git; +Cc: normalperson, k_satoda, Torsten Bögershausen From: Torsten Bögershausen <tboegi@web.de> HFS+ under Mac OS X doesn't allow file names like "\201\207", so skip the tests using cp932. Signed-off-by: Torsten Bögershausen <tboegi@web.de> --- t/t9115-git-svn-dcommit-funky-renames.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh index 0990f8d..805a9eb 100755 --- a/t/t9115-git-svn-dcommit-funky-renames.sh +++ b/t/t9115-git-svn-dcommit-funky-renames.sh @@ -93,7 +93,7 @@ test_expect_success 'git svn rebase works inside a fresh-cloned repository' ' # > to special UNICODE characters in the range 0xf000 to 0xf0ff (the # > "Private use area") when creating or accessing files. prepare_a_utf8_locale -test_expect_success UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' +test_expect_success !UTF8_NFD_TO_NFC,UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' LC_ALL=$a_utf8_locale && export LC_ALL && neq=$(printf "\201\202") && @@ -105,10 +105,10 @@ test_expect_success UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' ' # See the comment on the above test for setting of LC_ALL. -test_expect_success 'svn.pathnameencoding=cp932 rename on dcommit' ' +test_expect_success !UTF8_NFD_TO_NFC,UTF8 'svn.pathnameencoding=cp932 rename on dcommit' ' LC_ALL=$a_utf8_locale && export LC_ALL && - inf=$(printf "\201\207") && + inf=$(printf "\201\207"o) && git config svn.pathnameencoding cp932 && echo inf >"$inf" && git add "$inf" && -- 2.7.0.303.g2c4f448.dirty ^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS 2016-02-27 18:28 ` [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS tboegi @ 2016-02-28 4:59 ` Eric Wong 2016-02-28 17:52 ` Torsten Bögershausen 0 siblings, 1 reply; 25+ messages in thread From: Eric Wong @ 2016-02-28 4:59 UTC (permalink / raw) To: tboegi; +Cc: git, k_satoda tboegi@web.de wrote: > --- a/t/t9115-git-svn-dcommit-funky-renames.sh > +++ b/t/t9115-git-svn-dcommit-funky-renames.sh > @@ -93,7 +93,7 @@ test_expect_success 'git svn rebase works inside a fresh-cloned repository' ' > # > to special UNICODE characters in the range 0xf000 to 0xf0ff (the > # > "Private use area") when creating or accessing files. > prepare_a_utf8_locale > -test_expect_success UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' > +test_expect_success !UTF8_NFD_TO_NFC,UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' Please keep lines wrapped at 80 cols or less. (I need big fonts) > @@ -105,10 +105,10 @@ test_expect_success UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' > ' > > # See the comment on the above test for setting of LC_ALL. > -test_expect_success 'svn.pathnameencoding=cp932 rename on dcommit' ' > +test_expect_success !UTF8_NFD_TO_NFC,UTF8 'svn.pathnameencoding=cp932 rename on dcommit' ' > LC_ALL=$a_utf8_locale && > export LC_ALL && > - inf=$(printf "\201\207") && > + inf=$(printf "\201\207"o) && Why the extra 'o'? > git config svn.pathnameencoding cp932 && > echo inf >"$inf" && > git add "$inf" && > -- ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS 2016-02-28 4:59 ` Eric Wong @ 2016-02-28 17:52 ` Torsten Bögershausen 2016-03-15 1:59 ` Eric Wong 0 siblings, 1 reply; 25+ messages in thread From: Torsten Bögershausen @ 2016-02-28 17:52 UTC (permalink / raw) To: Eric Wong, tboegi; +Cc: git, k_satoda On 28.02.16 05:59, Eric Wong wrote: > tboegi@web.de wrote: > Please keep lines wrapped at 80 cols or less. > (I need big fonts) OK > >> @@ -105,10 +105,10 @@ test_expect_success UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' >> ' > Why the extra 'o'? That shouldn't be there, sorry. Will send V2... ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS 2016-02-28 17:52 ` Torsten Bögershausen @ 2016-03-15 1:59 ` Eric Wong 2016-03-15 5:23 ` Torsten Bögershausen 0 siblings, 1 reply; 25+ messages in thread From: Eric Wong @ 2016-03-15 1:59 UTC (permalink / raw) To: Torsten Bögershausen; +Cc: git, k_satoda Torsten Bögershausen <tboegi@web.de> wrote: > On 28.02.16 05:59, Eric Wong wrote: > > tboegi@web.de wrote: > > Please keep lines wrapped at 80 cols or less. > > (I need big fonts) > OK > > > >> @@ -105,10 +105,10 @@ test_expect_success UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' > >> ' > > Why the extra 'o'? > That shouldn't be there, sorry. > Will send V2... I just edited locally and pushed those out to Junio: http://mid.gmane.org/20160315015726.GA25295@dcvr.yhbt.net ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS 2016-03-15 1:59 ` Eric Wong @ 2016-03-15 5:23 ` Torsten Bögershausen 2016-03-15 7:09 ` Eric Wong 0 siblings, 1 reply; 25+ messages in thread From: Torsten Bögershausen @ 2016-03-15 5:23 UTC (permalink / raw) To: Eric Wong, Torsten Bögershausen; +Cc: git, k_satoda On 03/15/2016 02:59 AM, Eric Wong wrote: > [] > I just edited locally and pushed those out to Junio: > > http://mid.gmane.org/20160315015726.GA25295@dcvr.yhbt.net > The new TC 11/12 don't pass under cygwin. Do we need cp932 ? If not, we may use the paych from here: https://github.com/tboegi/git/commit/379c01bf52464f8a50065b11af516127e9144045 Date: Tue Mar 15 05:03:18 2016 +0100 t9115: Use funcky file names that work under unicode FS Don't use funky file names, that can not be created under HFS or NTFS. diff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh index 0990f8d..d022f0d 100755 --- a/t/t9115-git-svn-dcommit-funky-renames.sh +++ b/t/t9115-git-svn-dcommit-funky-renames.sh @@ -93,11 +93,11 @@ test_expect_success 'git svn rebase works inside a fresh-cloned repository' ' # > to special UNICODE characters in the range 0xf000 to 0xf0ff (the # > "Private use area") when creating or accessing files. prepare_a_utf8_locale -test_expect_success UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' +test_expect_success UTF8 'svn.pathnameencoding=ISO8859-1 new file on dcommit' ' LC_ALL=$a_utf8_locale && export LC_ALL && - neq=$(printf "\201\202") && - git config svn.pathnameencoding cp932 && + neq=$(printf "\303\244") && + git config svn.pathnameencoding ISO8859-1 && echo neq >"$neq" && git add "$neq" && git commit -m "neq" && @@ -105,11 +105,11 @@ test_expect_success UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' ' # See the comment on the above test for setting of LC_ALL. -test_expect_success 'svn.pathnameencoding=cp932 rename on dcommit' ' +test_expect_success 'svn.pathnameencoding=ISO8859-1 rename on dcommit' ' LC_ALL=$a_utf8_locale && export LC_ALL && - inf=$(printf "\201\207") && - git config svn.pathnameencoding cp932 && + inf=$(printf "\303\226") && + git config svn.pathnameencoding ISO8859-1 && echo inf >"$inf" && git add "$inf" && git commit -m "inf" && ^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS 2016-03-15 5:23 ` Torsten Bögershausen @ 2016-03-15 7:09 ` Eric Wong 2016-03-16 17:37 ` Kazutoshi Satoda 0 siblings, 1 reply; 25+ messages in thread From: Eric Wong @ 2016-03-15 7:09 UTC (permalink / raw) To: Kazutoshi Satoda; +Cc: git, Torsten Bögershausen Torsten Bögershausen <tboegi@web.de> wrote: > On 03/15/2016 02:59 AM, Eric Wong wrote: > >[] > >I just edited locally and pushed those out to Junio: > > > >http://mid.gmane.org/20160315015726.GA25295@dcvr.yhbt.net > > > > The new TC 11/12 don't pass under cygwin. > > Do we need cp932 ? Not sure, both CP932 and ISO8859-1 work fine for me on GNU/Linux. Anyways I'm fine skipping this patch for 2.8 while we hash it out, too. Kazutoshi: can you answer? Thanks. > If not, we may use the paych from here: > https://github.com/tboegi/git/commit/379c01bf52464f8a50065b11af516127e9144045 > > Date: Tue Mar 15 05:03:18 2016 +0100 > > t9115: Use funcky file names that work under unicode FS > > Don't use funky file names, that can not be created under > HFS or NTFS. > > diff --git a/t/t9115-git-svn-dcommit-funky-renames.sh > b/t/t9115-git-svn-dcommit-funky-renames.sh > index 0990f8d..d022f0d 100755 > --- a/t/t9115-git-svn-dcommit-funky-renames.sh > +++ b/t/t9115-git-svn-dcommit-funky-renames.sh > @@ -93,11 +93,11 @@ test_expect_success 'git svn rebase works inside > a fresh-cloned repository' ' > # > to special UNICODE characters in the range 0xf000 to 0xf0ff (the > # > "Private use area") when creating or accessing files. > prepare_a_utf8_locale > -test_expect_success UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' > +test_expect_success UTF8 'svn.pathnameencoding=ISO8859-1 new file on dcommit' ' > LC_ALL=$a_utf8_locale && > export LC_ALL && > - neq=$(printf "\201\202") && > - git config svn.pathnameencoding cp932 && > + neq=$(printf "\303\244") && > + git config svn.pathnameencoding ISO8859-1 && > echo neq >"$neq" && > git add "$neq" && > git commit -m "neq" && > @@ -105,11 +105,11 @@ test_expect_success UTF8 > 'svn.pathnameencoding=cp932 new file on dcommit' ' > ' > > # See the comment on the above test for setting of LC_ALL. > -test_expect_success 'svn.pathnameencoding=cp932 rename on dcommit' ' > +test_expect_success 'svn.pathnameencoding=ISO8859-1 rename on dcommit' ' > LC_ALL=$a_utf8_locale && > export LC_ALL && > - inf=$(printf "\201\207") && > - git config svn.pathnameencoding cp932 && > + inf=$(printf "\303\226") && > + git config svn.pathnameencoding ISO8859-1 && > echo inf >"$inf" && > git add "$inf" && > git commit -m "inf" && > ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS 2016-03-15 7:09 ` Eric Wong @ 2016-03-16 17:37 ` Kazutoshi Satoda 2016-03-17 5:16 ` Torsten Bögershausen 0 siblings, 1 reply; 25+ messages in thread From: Kazutoshi Satoda @ 2016-03-16 17:37 UTC (permalink / raw) To: Eric Wong, Torsten Bögershausen; +Cc: git On 2016/03/15 16:09 +0900, Eric Wong wrote: > Torsten Bögershausen <tboegi@web.de> wrote: >> On 03/15/2016 02:59 AM, Eric Wong wrote: >> >[] >> >I just edited locally and pushed those out to Junio: >> > >> >http://mid.gmane.org/20160315015726.GA25295@dcvr.yhbt.net >> > >> >> The new TC 11/12 don't pass under cygwin. >> >> Do we need cp932 ? > > Not sure, both CP932 and ISO8859-1 work fine for me on > GNU/Linux. Anyways I'm fine skipping this patch for 2.8 > while we hash it out, too. > > Kazutoshi: can you answer? Thanks. I tried the patch. The test works (pass with my fixes, and fails without fixes) with ISO8859-1 for me on Cygnus. The change sounds good. >> If not, we may use the paych from here: >> https://github.com/tboegi/git/commit/379c01bf52464f8a50065b11af516127e9144045 >> >> Date: Tue Mar 15 05:03:18 2016 +0100 >> >> t9115: Use funcky file names that work under unicode FS "funcky" looks a typo. >> Don't use funky file names, that can not be created under >> HFS or NTFS. The file can be created on my Cygnus environment, which is under FONTS. So it looks a bit inaccurate. I think a quote from the actual error message may be useful. It will likely tell what was wrong, accurately. And also, someone may search for that message. >> - neq=$(printf "\201\202") && >> - git config svn.pathnameencoding cp932 && >> + neq=$(printf "\303\244") && >> + git config svn.pathnameencoding ISO8859-1 && The variable name "new" was for "NOT EQUAL TO" (0x8182 in cp932 = U+2260). http://unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP932.TXT Then it should be changed, too. A more abstract one may be appropriate. >> - inf=$(printf "\201\207") && >> - git config svn.pathnameencoding cp932 && >> + inf=$(printf "\303\226") && >> + git config svn.pathnameencoding ISO8859-1 && Ditto. (0x8187 in cp932 = U+221E, INFINITY) -- k_satoda ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS 2016-03-16 17:37 ` Kazutoshi Satoda @ 2016-03-17 5:16 ` Torsten Bögershausen 2016-03-17 5:35 ` Torsten Bögershausen 2016-03-18 2:14 ` Kazutoshi Satoda 0 siblings, 2 replies; 25+ messages in thread From: Torsten Bögershausen @ 2016-03-17 5:16 UTC (permalink / raw) To: Kazutoshi Satoda, Eric Wong, Torsten Bögershausen; +Cc: git On 2016-03-16 18.37, Kazutoshi Satoda wrote: > "funcky" looks a typo. > >>> Don't use funky file names, that can not be created under >>> HFS or NTFS. > > The file can be created on my Cygnus environment, which is under FONTS. > So it looks a bit inaccurate. > > I think a quote from the actual error message may be useful. It will > likely tell what was wrong, accurately. And also, someone may search for > that message. > >>> - neq=$(printf "\201\202") && >>> - git config svn.pathnameencoding cp932 && >>> + neq=$(printf "\303\244") && >>> + git config svn.pathnameencoding ISO8859-1 && > > The variable name "new" was for "NOT EQUAL TO" (0x8182 in cp932 = U+2260). > http://unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP932.TXT > Then it should be changed, too. A more abstract one may be appropriate. > >>> - inf=$(printf "\201\207") && >>> - git config svn.pathnameencoding cp932 && >>> + inf=$(printf "\303\226") && >>> + git config svn.pathnameencoding ISO8859-1 && > > Ditto. (0x8187 in cp932 = U+221E, INFINITY) > Agreed with all your comments, thanks for that. A better version is here: <https://github.com/tboegi/git/commit/7ea2fa1ffaeb1c05669a837d7fed9c60b8a0d3cb> commit 7ea2fa1ffaeb1c05669a837d7fed9c60b8a0d3cb Author: Torsten Bögershausen <tboegi@web.de> Date: Thu Mar 17 06:08:14 2016 +0100 t9115: Use prereq for funky file name Some file systems like HFS don't allow file names outside unicode. Add a precondition FS_CP932 and use it in t9115#11 and #12 diff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh index 0990f8d..5a6525c 100755 --- a/t/t9115-git-svn-dcommit-funky-renames.sh +++ b/t/t9115-git-svn-dcommit-funky-renames.sh @@ -93,7 +93,7 @@ test_expect_success 'git svn rebase works inside a fresh-cloned repository' ' # > to special UNICODE characters in the range 0xf000 to 0xf0ff (the # > "Private use area") when creating or accessing files. prepare_a_utf8_locale -test_expect_success UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' +test_expect_success UTF8,FS_CP932 'svn.pathnameencoding=cp932 new file on dcommit' ' LC_ALL=$a_utf8_locale && export LC_ALL && neq=$(printf "\201\202") && @@ -105,7 +105,7 @@ test_expect_success UTF8 'svn.pathnameencoding=cp932 new file on dcommit' ' ' # See the comment on the above test for setting of LC_ALL. -test_expect_success 'svn.pathnameencoding=cp932 rename on dcommit' ' +test_expect_success UTF8,FS_CP932 'svn.pathnameencoding=cp932 rename on dcommit' ' LC_ALL=$a_utf8_locale && export LC_ALL && inf=$(printf "\201\207") && diff --git a/t/test-lib.sh b/t/test-lib.sh index 0b47eb6..7bb1262 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1037,6 +1037,18 @@ test_lazy_prereq UTF8_NFD_TO_NFC ' esac ' +test_lazy_prereq FS_CP932 ' + # check whether FS allows filenames from cp932 + neq=$(printf "\201\202") + >"$neq" && + case "$(echo *)" in + "neq") + true ;; + *) + false ;; + esac +' + test_lazy_prereq AUTOIDENT ' sane_unset GIT_AUTHOR_NAME && sane_unset GIT_AUTHOR_EMAIL && ^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS 2016-03-17 5:16 ` Torsten Bögershausen @ 2016-03-17 5:35 ` Torsten Bögershausen 2016-03-18 2:15 ` Kazutoshi Satoda 2016-03-18 2:14 ` Kazutoshi Satoda 1 sibling, 1 reply; 25+ messages in thread From: Torsten Bögershausen @ 2016-03-17 5:35 UTC (permalink / raw) To: Torsten Bögershausen, Kazutoshi Satoda, Eric Wong; +Cc: git On 2016-03-17 06.16, Torsten Bögershausen wrote: Oh Boy, typo in the last patch and t9115#11,12 where skipped on Mac & Linux :-( The "case" should look like this: > + "$neq") > + true ;; > + *) > + false ;; > + esac And the pathch is here: <https://github.com/tboegi/git/commit/866dfc192a0d4428aebfc7242f5134899b6dafd4> Kazutoshi, could you re-test on your environment ? ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS 2016-03-17 5:35 ` Torsten Bögershausen @ 2016-03-18 2:15 ` Kazutoshi Satoda 2016-03-19 6:59 ` Torsten Bögershausen 0 siblings, 1 reply; 25+ messages in thread From: Kazutoshi Satoda @ 2016-03-18 2:15 UTC (permalink / raw) To: Torsten Bögershausen, Eric Wong; +Cc: git On 2016/03/17 14:35 +0900, Torsten Bögershausen wrote: > On 2016-03-17 06.16, Torsten Bögershausen wrote: ... > And the pathch is here: > <https://github.com/tboegi/git/commit/866dfc192a0d4428aebfc7242f5134899b6dafd4> > > Kazutoshi, could you re-test on your environment ? The test 11,12 was skipped on my environment, too: > $ ./t9115-git-svn-dcommit-funky-renames.sh > ok 1 - load repository with strange names > ... > ok 10 - git svn rebase works inside a fresh-cloned repository > ok 11 # skip svn.pathnameencoding=cp932 new file on dcommit (missing FS_CP932 of UTF8,FS_CP932) > ok 12 # skip svn.pathnameencoding=cp932 rename on dcommit (missing FS_CP932 of UTF8,FS_CP932) > # passed all 12 test(s) > 1..12 That's because the check for FS_CP932 runs under LC_ALL=C environment which affects how filenames are treated. See the comment on #11 for more details. Setting LC_ALL before the check works for me. test_lazy_prereq FS_CP932 ' # check whether FS allows filenames from cp932 + prepare_a_utf8_locale + LC_ALL=$a_utf8_locale && + export LC_ALL && neq=$(printf "\201\202") But it looks a bit strange to do "prepare_a_utf8_locale" here because it is done just before #11 now, though I think it does no harm. I also don't understand the way of the check: Why don't you just touch and test it, like this: touch "$neq" && test -e "$neq" ? -- k_satoda ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS 2016-03-18 2:15 ` Kazutoshi Satoda @ 2016-03-19 6:59 ` Torsten Bögershausen 0 siblings, 0 replies; 25+ messages in thread From: Torsten Bögershausen @ 2016-03-19 6:59 UTC (permalink / raw) To: Kazutoshi Satoda, Torsten Bögershausen, Eric Wong; +Cc: git On 2016-03-18 03.15, Kazutoshi Satoda wrote: > On 2016/03/17 14:35 +0900, Torsten Bögershausen wrote: >> On 2016-03-17 06.16, Torsten Bögershausen wrote: > ... >> And the pathch is here: >> <https://github.com/tboegi/git/commit/866dfc192a0d4428aebfc7242f5134899b6dafd4> >> >> Kazutoshi, could you re-test on your environment ? > > The test 11,12 was skipped on my environment, too: >> $ ./t9115-git-svn-dcommit-funky-renames.sh >> ok 1 - load repository with strange names >> ... >> ok 10 - git svn rebase works inside a fresh-cloned repository >> ok 11 # skip svn.pathnameencoding=cp932 new file on dcommit (missing FS_CP932 of UTF8,FS_CP932) >> ok 12 # skip svn.pathnameencoding=cp932 rename on dcommit (missing FS_CP932 of UTF8,FS_CP932) >> # passed all 12 test(s) >> 1..12 > > That's because the check for FS_CP932 runs under LC_ALL=C environment > which affects how filenames are treated. See the comment on #11 for more > details. > > Setting LC_ALL before the check works for me. > > test_lazy_prereq FS_CP932 ' > # check whether FS allows filenames from cp932 > + prepare_a_utf8_locale > + LC_ALL=$a_utf8_locale && > + export LC_ALL && > neq=$(printf "\201\202") > > But it looks a bit strange to do "prepare_a_utf8_locale" here because it > is done just before #11 now, though I think it does no harm. > > I also don't understand the way of the check: Why don't you just touch > and test it, like this: > touch "$neq" && test -e "$neq" > ? > Thanks, I will re-send a new version in a couple of days or so. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS 2016-03-17 5:16 ` Torsten Bögershausen 2016-03-17 5:35 ` Torsten Bögershausen @ 2016-03-18 2:14 ` Kazutoshi Satoda 1 sibling, 0 replies; 25+ messages in thread From: Kazutoshi Satoda @ 2016-03-18 2:14 UTC (permalink / raw) To: Torsten Bögershausen, Eric Wong; +Cc: git On 2016/03/17 14:16 +0900, Torsten Bögershausen wrote: > On 2016-03-16 18.37, Kazutoshi Satoda wrote: >> "funcky" looks a typo. >> >>>> Don't use funky file names, that can not be created under >>>> HFS or NTFS. >> >> The file can be created on my Cygnus environment, which is under FONTS. >> So it looks a bit inaccurate. Uh, ... I noticed that I wrongly instruct the spell checker... The above was meant to be: The file can be created on my Cygwin environment, which is under NTFS. So it looks a bit inaccurate. >>>> - neq=$(printf "\201\202") && >>>> - git config svn.pathnameencoding cp932 && >>>> + neq=$(printf "\303\244") && >>>> + git config svn.pathnameencoding ISO8859-1 && >> >> The variable name "new" was for "NOT EQUAL TO" (0x8182 in cp932 = U+2260). >> http://unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP932.TXT Here "new" was meant to be "neq", of course. Sorry for the confusion. -- k_satoda ^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2016-03-19 6:59 UTC | newest] Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-02-08 15:19 [PATCH 0/2] enable "svn.pathnameencoding" on dcommit Kazutoshi Satoda 2016-02-08 15:20 ` [PATCH 1/2] git-svn: " Kazutoshi Satoda 2016-02-15 0:30 ` Eric Wong 2016-02-08 15:21 ` [PATCH 2/2] git-svn: apply "svn.pathnameencoding" before URL encoding Kazutoshi Satoda 2016-02-15 0:33 ` Eric Wong 2016-02-08 22:58 ` [PATCH 0/2] enable "svn.pathnameencoding" on dcommit Eric Wong 2016-02-15 0:52 ` [PULL] svn pathnameencoding for git svn dcommit Eric Wong 2016-02-15 21:32 ` Junio C Hamano 2016-02-16 3:29 ` Kazutoshi Satoda 2016-02-16 6:33 ` Eric Wong 2016-02-16 16:19 ` Kazutoshi Satoda 2016-02-20 23:37 ` Eric Wong 2016-02-21 13:12 ` Kazutoshi Satoda 2016-02-27 18:28 ` [PATCH 1/1] t9115: Skip pathnameencoding=cp932 under HFS tboegi 2016-02-28 4:59 ` Eric Wong 2016-02-28 17:52 ` Torsten Bögershausen 2016-03-15 1:59 ` Eric Wong 2016-03-15 5:23 ` Torsten Bögershausen 2016-03-15 7:09 ` Eric Wong 2016-03-16 17:37 ` Kazutoshi Satoda 2016-03-17 5:16 ` Torsten Bögershausen 2016-03-17 5:35 ` Torsten Bögershausen 2016-03-18 2:15 ` Kazutoshi Satoda 2016-03-19 6:59 ` Torsten Bögershausen 2016-03-18 2:14 ` Kazutoshi Satoda
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.