git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: Jeff King <peff@peff.net>,
	Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>,
	Johannes Schindelin <johannes.schindelin@gmx.de>,
	Junio C Hamano <gitster@pobox.com>
Subject: [PATCH v2 0/1]  Fix t5516 flakiness in Visual Studio builds
Date: Tue, 29 Oct 2019 13:37:51 +0000	[thread overview]
Message-ID: <pull.428.v2.git.1572356272.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.428.git.1572274859.gitgitgadget@gmail.com>

Among the flaky tests, it seems that the Azure Pipeline suffers relatively
frequently from t5516 failing with the Visual Studio builds. Essentially, we
grep for an error message, but that error message is produced twice, once by
a fetch and once by the upload-pack spawned from it, and those error
messages are usually interleaved because of MSVC runtime fprintf() 
idiosyncracies. 

The commit message of this patch is based, in part, on 
https://github.com/gitgitgadget/git/pull/407. The patch itself is a much
more minimal alternative (using xwrite() instead of fprintf()) to the code
of https://github.com/gitgitgadget/git/pull/407, avoiding the complexity of
the part of the code that allows for unlimited messages.

While it would seem theoretically more elegant to allow for unlimited
messages, in practice too-long messages cause more problems than they solve,
and therefore we already clip them, and this patch does not change that
behavior.

This fixes https://github.com/gitgitgadget/git/issues/240.

Changes since v1:

 * Changed the oneline to be more accurate (thanks Junio).
 * Improved the commit message (e.g. talking about the xwrite() function
   this patch uses, rather than the write_in_full() function used by an
   earlier iteration, thanks Gábor).
 * Revamped the actual code to account for insanely long prefixes (thanks
   for the advice, Junio).

Johannes Schindelin (1):
  vreportf(): avoid relying on stdio buffering

 usage.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)


base-commit: 566a1439f6f56c2171b8853ddbca0ad3f5098770
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-428%2Fdscho%2Ffix-t5516-flakiness-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-428/dscho/fix-t5516-flakiness-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/428

Range-diff vs v1:

 1:  455026ce3e ! 1:  e426627e14 vreportf(): avoid buffered write in favor of unbuffered one
     @@ -1,6 +1,6 @@
      Author: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     -    vreportf(): avoid buffered write in favor of unbuffered one
     +    vreportf(): avoid relying on stdio buffering
      
          The MSVC runtime behavior differs from glibc's with respect to
          `fprintf(stderr, ...)` in that the former writes out the message
     @@ -16,7 +16,9 @@
          Let's avoid this predicament altogether by rendering the entire message,
          including the prefix and the trailing newline, into the buffer we
          already have (and which is still fixed size) and then write it out via
     -    `write_in_full()`.
     +    `xwrite()`.
     +
     +    We still clip the message to at most 4095 characters.
      
          The history of `vreportf()` with regard to this issue includes the
          following commits:
     @@ -36,39 +38,45 @@
                                  so it's safe to use xwrite() again
          5e5be9e2 (2016-06-28) - recv_sideband() uses xwrite() again
      
     -    Note that we need to be careful to handle the return value of
     -    `vsnprintf()` that indicates the _desired_ byte count.
     +    Note that we print nothing if the `vsnprintf()` call failed to render
     +    the error message; There is little we can do in that case, and it should
     +    not happen anyway.
      
          Also please note that we `fflush(stderr)` here to help when running in a
          Git Bash on Windows: in this case, `stderr` is not actually truly
          unbuffered, and needs the extra help.
      
     -    Co-authored-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
     +    Helped-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
     +    Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
     +    Helped-by: Junio C Hamano <gitster@pobox.com>
          Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
      
       diff --git a/usage.c b/usage.c
       --- a/usage.c
       +++ b/usage.c
      @@
     + void vreportf(const char *prefix, const char *err, va_list params)
       {
       	char msg[4096];
     - 	char *p;
     --
     --	vsnprintf(msg, sizeof(msg), err, params);
     +-	char *p;
      +	size_t off = strlcpy(msg, prefix, sizeof(msg));
     -+	int ret = vsnprintf(msg + off, sizeof(msg) - off, err, params);
     - 	for (p = msg; *p; p++) {
     ++	char *p, *pend = msg + sizeof(msg);
     + 
     +-	vsnprintf(msg, sizeof(msg), err, params);
     +-	for (p = msg; *p; p++) {
     ++	p = msg + off < pend ? msg + off : pend - 1;
     ++	if (vsnprintf(p, pend - p, err, params) < 0)
     ++		return; /* vsnprintf() failed, there is nothing we can do */
     ++
     ++	for (; p != pend - 1 && *p; p++) {
       		if (iscntrl(*p) && *p != '\t' && *p != '\n')
       			*p = '?';
       	}
      -	fprintf(stderr, "%s%s\n", prefix, msg);
     -+	if (ret > 0) {
     -+		if (off + ret > sizeof(msg) - 1)
     -+			ret = sizeof(msg) - 1 - off;
     -+		msg[off + ret] = '\n'; /* we no longer need a NUL */
     -+		fflush(stderr);
     -+		xwrite(2, msg, off + ret + 1);
     -+	}
     ++
     ++	*(p++) = '\n'; /* we no longer need a NUL */
     ++	fflush(stderr);
     ++	xwrite(2, msg, p - msg);
       }
       
       static NORETURN void usage_builtin(const char *err, va_list params)

-- 
gitgitgadget

  parent reply	other threads:[~2019-10-29 13:37 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-28 15:00 [PATCH 0/1] Fix t5516 flakiness in Visual Studio builds Johannes Schindelin via GitGitGadget
2019-10-28 15:00 ` [PATCH 1/1] vreportf(): avoid buffered write in favor of unbuffered one Johannes Schindelin via GitGitGadget
2019-10-29  3:18   ` Junio C Hamano
2019-10-29 12:30     ` Johannes Schindelin
2019-10-29 13:49       ` Jeff King
2019-10-29 14:13         ` Johannes Schindelin
2019-10-29 14:32           ` Jeff King
2019-10-29 20:09             ` Johannes Schindelin
2019-10-30  1:43               ` Junio C Hamano
2019-10-29 16:44         ` Junio C Hamano
2019-10-29 10:38   ` SZEDER Gábor
2019-10-29 12:38     ` Johannes Schindelin
2019-10-29 13:52       ` Jeff King
2019-10-29 14:18         ` Johannes Schindelin
2019-10-29 13:37 ` Johannes Schindelin via GitGitGadget [this message]
2019-10-29 13:37   ` [PATCH v2 1/1] vreportf(): avoid relying on stdio buffering Johannes Schindelin via GitGitGadget
2019-10-29 14:21     ` Alexandr Miloslavskiy
2019-10-29 19:57       ` Johannes Schindelin
2019-10-29 20:09         ` Jeff King
2019-10-29 20:24           ` Alexandr Miloslavskiy
2019-10-29 20:11         ` Alexandr Miloslavskiy
2019-10-29 20:01   ` [PATCH v3 0/1] Fix t5516 flakiness in Visual Studio builds Johannes Schindelin via GitGitGadget
2019-10-29 20:01     ` [PATCH v3 1/1] vreportf(): avoid relying on stdio buffering Johannes Schindelin via GitGitGadget
2019-10-29 20:32       ` Jeff King
2019-10-30  8:54         ` Johannes Schindelin
2019-10-31  6:24           ` Jeff King
2019-10-31 10:26             ` Johannes Schindelin
2019-10-31 15:48               ` Jeff King
2019-11-01 18:41                 ` Johannes Schindelin
2019-10-30  2:01       ` Junio C Hamano
2019-10-30  9:13         ` Johannes Schindelin
2019-10-30 10:44     ` [PATCH v4 0/1] Fix t5516 flakiness in Visual Studio builds Johannes Schindelin via GitGitGadget
2019-10-30 10:44       ` [PATCH v4 1/1] vreportf(): avoid relying on stdio buffering Johannes Schindelin via GitGitGadget
2019-11-02  4:05         ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=pull.428.v2.git.1572356272.gitgitgadget@gmail.com \
    --to=gitgitgadget@gmail.com \
    --cc=alexandr.miloslavskiy@syntevo.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=johannes.schindelin@gmx.de \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).