GIT+CYGWIN annoying test failure

GIT+CYGWIN annoying test failure

[SungHyun Nam]

Hello,

First of all, this problem always existed to me. I was lazy to report.
Please check the failure logs and my GIT build script below.
My build script runs 'make test' twice because of this annoying failure.
It does not mean that the first test always failed, but very often.
If it occurs, the first test failed at 't1000-read-tree-m-3way.sh'
always. And in this case, 2nd test always succeeded.

Thanks,

GIT test failure logs
====================================================================

2007-01-15
==============================================================================
GIT_VERSION = 1.5.0.rc1.gf4b6c-dirty

...
* FAIL 57: 5 - must match in !O && A && B && A==B case.
        rm -f .git/index LL &&
             cp .orig-A/LL LL &&
             git-update-index --add LL &&
             echo extra >>LL &&
             git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
997bbc4a0a51e0574168a4f637739380edebe4d7
76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
             check_result
...
* FAIL 62: 9 - must match and be up-to-date in O && A && !B && O!=A case
        rm -f .git/index MD &&
             cp .orig-A/MD MD &&
             git-update-index --add MD &&
             git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
997bbc4a0a51e0574168a4f637739380edebe4d7
76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
             check_result
...
* FAIL 65: 10 - must match and be up-to-date in O && A && !B && O==A case
        rm -f .git/index ND &&
             cp .orig-A/ND ND &&
             git-update-index --add ND &&
             git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
997bbc4a0a51e0574168a4f637739380edebe4d7
76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
             check_result
...
* failed 3 among 83 test(s)
make[1]: *** [t1000-read-tree-m-3way.sh] Error 1
make[1]: Leaving directory `/c/user/namsh/unix/git/t'
make: *** [test] Error 2


2007-01-18
==============================================================================
GIT_VERSION = 1.5.0.rc1.g556b-dirty

...
* FAIL 50: 3 - must match A in !O && A && !B case.
        rm -f .git/index AN &&
             cp .orig-A/AN AN &&
             git-update-index --add AN &&
             git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
997bbc4a0a51e0574168a4f637739380edebe4d7
76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
             check_result
...
* FAIL 53: 4 - must match and be up-to-date in !O && A && B && A!=B case.
        rm -f .git/index AA &&
             cp .orig-A/AA AA &&
             git-update-index --add AA &&
             git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
997bbc4a0a51e0574168a4f637739380edebe4d7
76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
             check_result
...
* failed 2 among 83 test(s)
make[1]: *** [t1000-read-tree-m-3way.sh] Error 1
make[1]: Leaving directory `/c/user/namsh/unix/git/t'
make: *** [test] Error 2

2007-01-24
==============================================================================
GIT_VERSION = 1.5.0.rc2.gc9a89-dirty

...
* FAIL 50: 3 - must match A in !O && A && !B case.
        rm -f .git/index AN &&
             cp .orig-A/AN AN &&
             git-update-index --add AN &&
             git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
997bbc4a0a51e0574168a4f637739380edebe4d7
76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
             check_result
...
* failed 1 among 83 test(s)
make[1]: *** [t1000-read-tree-m-3way.sh] Error 1
make[1]: Leaving directory `/c/user/namsh/unix/git/t'
make: *** [test] Error 2



GIT build script
====================================================================
	...
	if test ! -r ./configure || test ./configure.ac -nt ./configure
	then
	    printf "Run autoconf..."
	    autoconf
	    echo " DONE!"
	fi
	if test ! -r ./config.mak.autogen \
	    || test ./configure -nt ./config.mak.autogen
	then
	    ./configure --prefix=$prefix
	fi

	# generate personal configuration makefile
	[ -r config.mak ] && rm -f config.mak
	touch config.mak
	[ -r config.mak ] || die "Cannot generate config.mak"
	$E "CFLAGS=$CFLAGS" >> config.mak
	$E "DESTDIR=$DESTDIR" >> config.mak
	# git's configure script could not find the libiconv for CYGWIN.
	$E "NEEDS_LIBICONV=1\nNO_ICONV=" >> config.mak
	$E "NO_PERL_MAKEMAKER=1" >> config.mak # have no PERL MakeMaker
	$E "NO_TRUSTABLE_FILEMODE=" >> config.mak # default filemode = true
	# FAT32 has issues with 'mmap', but I only have NTFS.
	$E "NO_MMAP=" >> config.mak

	if make all doc
	then
	    # I don't use CVS/Subversion. So skip those tests.
	    export GIT_SKIP_TESTS='t91?? t92??'
	    if ! make test
	    then
		${E} "\n*** ${R}Test failed${N} ***\n"

		# I don't know why... Sometimes first 'make test' failed.
		printf "Try again... "
		if ! make test >/dev/null 2>&1
		then
		    die "2nd test failed also."
		fi
		printf "OK!"
	    fi

	    rm -f "$gitLog"
	    if ! make install install-doc > "$gitLog" 2>&1
	    then
		die "Install failed. Check $gitLog"
	    fi
	    rm -f "$gitLog"
	    ...
	fi

Re: GIT+CYGWIN annoying test failure

[Johannes Schindelin]

Hi,

On Wed, 24 Jan 2007, SungHyun Nam wrote:

> First of all, this problem always existed to me. I was lazy to report. 
> Please check the failure logs and my GIT build script below. [...] the 
> first test failed at 't1000-read-tree-m-3way.sh' always. And in this 
> case, 2nd test always succeeded.
> 
> * FAIL 57: 5 - must match in !O && A && B && A==B case.
>         rm -f .git/index LL &&
>              cp .orig-A/LL LL &&
>              git-update-index --add LL &&
>              echo extra >>LL &&
>              git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
> 997bbc4a0a51e0574168a4f637739380edebe4d7
> 76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
>              check_result

It is not directly apparent what is causing this error. Could you please 
change your script to use "-i -v" on the tests? I.e. instead of:
	
> 	    if ! make test

this:

	    if ! (cd t; GIT_TEST_OPTS="-i -v" make)

It should give us a better idea what happened, since we also see the 
error messages of the git commands.

If it is still not apparent what went wrong, there is unfortunately no way 
around doing that particular test case manually, as to find out what went 
wrong. Since you seem to be the only one who can reproduce this bug, it 
boils down to you having to do the leg work.

Please keep us posted!

Ciao,
Dscho

Re: GIT+CYGWIN annoying test failure

[SungHyun Nam]

Hello,

Johannes Schindelin wrote:
> On Wed, 24 Jan 2007, SungHyun Nam wrote:
> 
>> First of all, this problem always existed to me. I was lazy to report. 
>> Please check the failure logs and my GIT build script below. [...] the 
>> first test failed at 't1000-read-tree-m-3way.sh' always. And in this 
>> case, 2nd test always succeeded.
>>
>> * FAIL 57: 5 - must match in !O && A && B && A==B case.
>>         rm -f .git/index LL &&
>>              cp .orig-A/LL LL &&
>>              git-update-index --add LL &&
>>              echo extra >>LL &&
>>              git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
>> 997bbc4a0a51e0574168a4f637739380edebe4d7
>> 76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
>>              check_result
> 
> It is not directly apparent what is causing this error. Could you please 
> change your script to use "-i -v" on the tests? I.e. instead of:
> 	
>> 	    if ! make test
> 
> this:
> 
> 	    if ! (cd t; GIT_TEST_OPTS="-i -v" make)
> 
> It should give us a better idea what happened, since we also see the 
> error messages of the git commands.
> 
> If it is still not apparent what went wrong, there is unfortunately no way 
> around doing that particular test case manually, as to find out what went 
> wrong. Since you seem to be the only one who can reproduce this bug, it 
> boils down to you having to do the leg work.
> 

I'm not sure I can believe my system, Windows or CYGWIN.
It seems GIT has no problem.

Thanks,

2007-01-25
=========================================================================
* expecting success: rm -fr [NDMALTS][NDMALTSF] Z &&
     rm .git/index &&
     git-read-tree 997bbc4a0a51e0574168a4f637739380edebe4d7 &&
     git-checkout-index -f -u -a &&
     git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
997bbc4a0a51e0574168a4f637739380edebe4d7
76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
     check_result
t1000-read-tree-m-3way.sh: line 133: current: Permission denied
* FAIL 46: 3-way merge with git-read-tree -m, match H
        rm -fr [NDMALTS][NDMALTSF] Z &&
             rm .git/index &&
             git-read-tree 997bbc4a0a51e0574168a4f637739380edebe4d7 &&
             git-checkout-index -f -u -a &&
             git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
997bbc4a0a51e0574168a4f637739380edebe4d7
76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
             check_result
make: *** [t1000-read-tree-m-3way.sh] Error 1


2007-01-29
=========================================================================
* expecting success: rm -f .git/index LL &&
     cp .orig-A/LL LL &&
     git-update-index --add LL &&
     echo extra >>LL &&
     git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
997bbc4a0a51e0574168a4f637739380edebe4d7
76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
     check_result
t1000-read-tree-m-3way.sh: line 133: current: Permission denied
* FAIL 57: 5 - must match in !O && A && B && A==B case.
        rm -f .git/index LL &&
             cp .orig-A/LL LL &&
             git-update-index --add LL &&
             echo extra >>LL &&
             git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
997bbc4a0a51e0574168a4f637739380edebe4d7
76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
             check_result
make: *** [t1000-read-tree-m-3way.sh] Error 1

Re: GIT+CYGWIN annoying test failure

[Johannes Schindelin]

Hi,

On Mon, 29 Jan 2007, SungHyun Nam wrote:

> I'm not sure I can believe my system, Windows or CYGWIN.
> It seems GIT has no problem.

Are you perchance on a FAT32 system?

Ciao,
Dscho

Re: GIT+CYGWIN annoying test failure

[SungHyun Nam]

Johannes Schindelin wrote:
> 
> On Mon, 29 Jan 2007, SungHyun Nam wrote:
> 
>> I'm not sure I can believe my system, Windows or CYGWIN.
>> It seems GIT has no problem.
> 
> Are you perchance on a FAT32 system?

No, I have 3 NTFS partitions (C=program, D=work/data, E=temp).
There's no FAT32.

Though I have no FAT32, I even tried the NO_MMAP thing.
And it didn't help.

Thanks,
namsh

Re: GIT+CYGWIN annoying test failure

[SungHyun Nam]

Hello,

It seems the patch below work-around the problem.
I tested this patch several times after knowing the problem.

Thanks,
namsh

diff --git a/t/t1000-read-tree-m-3way.sh b/t/t1000-read-tree-m-3way.sh
index d0af8c3..422922e 100755
--- a/t/t1000-read-tree-m-3way.sh
+++ b/t/t1000-read-tree-m-3way.sh
@@ -130,6 +130,7 @@ _x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
 _x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"

 check_result () {
+    [ -r current ] && rm -f current
     git-ls-files --stage | sed -e 's/ '"$_x40"' / X /' >current &&
     diff -u expected current
 }


SungHyun Nam wrote:
> Hello,
> 
> Johannes Schindelin wrote:
>> On Wed, 24 Jan 2007, SungHyun Nam wrote:
>>
>>> First of all, this problem always existed to me. I was lazy to report. 
>>> Please check the failure logs and my GIT build script below. [...] the 
>>> first test failed at 't1000-read-tree-m-3way.sh' always. And in this 
>>> case, 2nd test always succeeded.
>>>
>>> * FAIL 57: 5 - must match in !O && A && B && A==B case.
>>>         rm -f .git/index LL &&
>>>              cp .orig-A/LL LL &&
>>>              git-update-index --add LL &&
>>>              echo extra >>LL &&
>>>              git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
>>> 997bbc4a0a51e0574168a4f637739380edebe4d7
>>> 76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
>>>              check_result
>> It is not directly apparent what is causing this error. Could you please 
>> change your script to use "-i -v" on the tests? I.e. instead of:
>> 	
>>> 	    if ! make test
>> this:
>>
>> 	    if ! (cd t; GIT_TEST_OPTS="-i -v" make)
>>
>> It should give us a better idea what happened, since we also see the 
>> error messages of the git commands.
>>
>> If it is still not apparent what went wrong, there is unfortunately no way 
>> around doing that particular test case manually, as to find out what went 
>> wrong. Since you seem to be the only one who can reproduce this bug, it 
>> boils down to you having to do the leg work.
> 
> I'm not sure I can believe my system, Windows or CYGWIN.
> It seems GIT has no problem.
> 
> Thanks,
> 
> 2007-01-25
> =========================================================================
> * expecting success: rm -fr [NDMALTS][NDMALTSF] Z &&
>      rm .git/index &&
>      git-read-tree 997bbc4a0a51e0574168a4f637739380edebe4d7 &&
>      git-checkout-index -f -u -a &&
>      git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
> 997bbc4a0a51e0574168a4f637739380edebe4d7
> 76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
>      check_result
> t1000-read-tree-m-3way.sh: line 133: current: Permission denied
> * FAIL 46: 3-way merge with git-read-tree -m, match H
>         rm -fr [NDMALTS][NDMALTSF] Z &&
>              rm .git/index &&
>              git-read-tree 997bbc4a0a51e0574168a4f637739380edebe4d7 &&
>              git-checkout-index -f -u -a &&
>              git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
> 997bbc4a0a51e0574168a4f637739380edebe4d7
> 76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
>              check_result
> make: *** [t1000-read-tree-m-3way.sh] Error 1
> 
> 
> 2007-01-29
> =========================================================================
> * expecting success: rm -f .git/index LL &&
>      cp .orig-A/LL LL &&
>      git-update-index --add LL &&
>      echo extra >>LL &&
>      git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
> 997bbc4a0a51e0574168a4f637739380edebe4d7
> 76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
>      check_result
> t1000-read-tree-m-3way.sh: line 133: current: Permission denied
> * FAIL 57: 5 - must match in !O && A && B && A==B case.
>         rm -f .git/index LL &&
>              cp .orig-A/LL LL &&
>              git-update-index --add LL &&
>              echo extra >>LL &&
>              git-read-tree -m 3fe085e0589de4327971d50e416fc292dd00fbfe
> 997bbc4a0a51e0574168a4f637739380edebe4d7
> 76d47d681d7f1d4fa975334a9a0ba8a6eeea2226 &&
>              check_result
> make: *** [t1000-read-tree-m-3way.sh] Error 1

Re: GIT+CYGWIN annoying test failure

[Johannes Schindelin]

Hi,

On Thu, 15 Feb 2007, SungHyun Nam wrote:

>  check_result () {
> +    [ -r current ] && rm -f current
>      git-ls-files --stage | sed -e 's/ '"$_x40"' / X /' >current &&
>      diff -u expected current
>  }

Are you having a restrictive umask, or what are the permissions on that 
file?

Ciao,
Dscho

Re: GIT+CYGWIN annoying test failure

[SungHyun Nam]

Hello,

Johannes Schindelin wrote:
> 
> On Thu, 15 Feb 2007, SungHyun Nam wrote:
> 
>>  check_result () {
>> +    [ -r current ] && rm -f current
>>      git-ls-files --stage | sed -e 's/ '"$_x40"' / X /' >current &&
>>      diff -u expected current
>>  }
> 
> Are you having a restrictive umask, or what are the permissions on that 
> file?

There's no problem with the umask.

$ umask
0022

I guess there is some race condition in CYGWIN or windows. And my system
triggers that condition.

Check the logic above. I cannot think there is any chance that 'rm -f
current && echo x >current' is success, but 'echo x >current' is fail?

Thanks,
namsh

Re: GIT+CYGWIN annoying test failure

[Junio C Hamano]

SungHyun Nam <goweol@gmail.com> writes:

>> On Thu, 15 Feb 2007, SungHyun Nam wrote:
>> 
>>>  check_result () {
>>> +    [ -r current ] && rm -f current
>>>      git-ls-files --stage | sed -e 's/ '"$_x40"' / X /' >current &&
>>>      diff -u expected current
>>>  }

I would have written:

	check_result () {
        	rm -f current &&
                git-ls-files --stage | ... &&
                diff -u expected current
	}

If the above still makes the tests pass for your environment,
that would be good.  I am not very happy about the lack of "&&"
after "rm -f current" in your patch.

It seems as if 'current' is sometimes writable and sometimes not,
and when the shell couldn't open it to redirect sed output the
whole test fails for you.  That does not sound like a sane,
reliable filesystem at all, and even if you worked around the
problem in that single shell function, we tend to use shell
redirection to create or overwrite files everywhere (both in
tests and in the real Porcelain scripts) so that means git is a
disaster waiting to happen on your system, which is not good.

What I am trying to get at is to find out what is causing the
problem first, before working it around.  Then, if we are lucky,
we may be able to add a notice in the INSTALL file that says "It
was reported that Filfre virus scanner makes shell redirect in
Cygwin occasionally fail, and both git itself and its testsuite
may not work as expected --- please do not use it", or something
like that.

"Filfre virus scanner" is totally made up, but I think you get
the point.