t7405.17 breakage vanishes with GETTEXT_POISON=1

t7405.17 breakage vanishes with GETTEXT_POISON=1

[Duy Nguyen]

Something fishy is going on but I don't think I'll spend time hunting
it down so I post here in case somebody else is interested. It might
also indicate a problem with poison gettext, not the test case too.
-- 
Duy

Re: t7405.17 breakage vanishes with GETTEXT_POISON=1

[SZEDER Gábor]

On Sun, Oct 28, 2018 at 06:41:06AM +0100, Duy Nguyen wrote:
> Something fishy is going on but I don't think I'll spend time hunting
> it down so I post here in case somebody else is interested. It might
> also indicate a problem with poison gettext, not the test case too.

I haven't actually run the test under GETTEXT_POISON, but I stongly
suspect it's the test, or more accurately the helper function
'test_i18ngrep'.

The test in question runs

  test_i18ngrep ! "refusing to lose untracked file at" err

which fails in normal test runs, because 'grep' does find the
undesired string; that's the known breakage.  Under GETTEXT_POISION,
however, 'test_i18ngrep' always succeeds because of this condition:

  if test -n "$GETTEXT_POISON"
  then
          # pretend success
          return 0
  fi

and then in turn the whole test succeeds.

Re: t7405.17 breakage vanishes with GETTEXT_POISON=1

[Junio C Hamano]

SZEDER Gábor <szeder.dev@gmail.com> writes:

> The test in question runs
>
>   test_i18ngrep ! "refusing to lose untracked file at" err
>
> which fails in normal test runs, because 'grep' does find the
> undesired string; that's the known breakage.  Under GETTEXT_POISION,
> however, 'test_i18ngrep' always succeeds because of this condition:
>
>   if test -n "$GETTEXT_POISON"
>   then
>           # pretend success
>           return 0
>   fi
>
> and then in turn the whole test succeeds.

Ah, good spotting.

If a test using "grep" declares that something must not exist in
plumbing message, it writes "! grep something", and because
"test_i18ngrep something" always pretends that something is found
under poisoned build (by the way, would this change when we remove
the separate poisoned build?), "test_i18ngrep ! something" must be
used for Porcelain messages to say the same thing.

Of course, this has a funny interactions with test_expect_failure.
I actually do not think the complexity to work this around is worth
it.  

Changing behaviour of "test_i18ngrep ! something" to always fail
under poisoned build would not work, of course, and changing it to
always fail under poisoned build inside test_expect_failure would
not be a good idea, either, because the know breakage may be at
steps in the same test that is different from the grep, e.g., we may
have a "git dothis" command that should keep the HEAD without
emitting an error message, and we may test it like so:

	git rev-parse HEAD >old &&
	git dothis >out 2>err &&
	test_i18ngrep ! error: err && # no error should be seen
	git rev-parse HEAD >new &&
	test_cmp old new

but currently the command may have a known bug that it moves HEAD;
the command however does not emit an error message.

SO...

Re: t7405.17 breakage vanishes with GETTEXT_POISON=1

[Duy Nguyen]

On Sun, Oct 28, 2018 at 1:15 PM SZEDER Gábor <szeder.dev@gmail.com> wrote:
>
> On Sun, Oct 28, 2018 at 06:41:06AM +0100, Duy Nguyen wrote:
> > Something fishy is going on but I don't think I'll spend time hunting
> > it down so I post here in case somebody else is interested. It might
> > also indicate a problem with poison gettext, not the test case too.
>
> I haven't actually run the test under GETTEXT_POISON, but I stongly
> suspect it's the test, or more accurately the helper function
> 'test_i18ngrep'.
>
> The test in question runs
>
>   test_i18ngrep ! "refusing to lose untracked file at" err
>
> which fails in normal test runs, because 'grep' does find the
> undesired string; that's the known breakage.  Under GETTEXT_POISION,
> however, 'test_i18ngrep' always succeeds because of this condition:
>
>   if test -n "$GETTEXT_POISON"
>   then
>           # pretend success
>           return 0
>   fi
>
> and then in turn the whole test succeeds.

And this is something your poison-with-scrambling code does help ;-) ;-)
-- 
Duy