git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v4 00/15] add git-bugreport tool
@ 2019-12-13  0:42 Emily Shaffer
  2019-12-13  0:42 ` [PATCH v4 01/15] bugreport: add tool to generate debugging info Emily Shaffer
                   ` (15 more replies)
  0 siblings, 16 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:42 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Martin Ågren

This topic branch depends on the patch mailed in
lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
display scopes for configs gathered during "bugreport: add config values from
safelist".

Quite a few changes since v3. I'll try to summarize the big picture
stuff, plus highlight new changes and difference in patches inline in
the commit list. I'll also stick an interdiff since v3 at the bottom of
this cover letter.

Some overall changes:
- The entire command now exists in one file, bugreport.c, which is
  compiled as a standalone.
- static-ness of local functions has been cleaned up.
- strbuf lifetime management has been cleaned up.
- Mention of "whitelist" and "blacklist" has been changed to "safelist"
  and "blocklist" while dealing with Git-approved bugreportable configs.
  Thanks Dscho for pointing this out; "safelist" makes sense to me as
  these are configs we have deemed safe for public inspection.
- Inquiry into both Git's $SHELL_PATH and the user's $SHELL is new to
  v4.
- The 'man git bugreport' page which was lost from the script version of
  git-bugreport has been recovered, as have the "tests" (which only test
  whether it runs and doesn't crash).


Emily Shaffer (15):
  bugreport: add tool to generate debugging info
Moved from library + builtin to a standalone git-bugreport binary.

  help: move list_config_help to builtin/help
Resolves a circular dependency where using help.o required all builtins
to exist. This also moves generation of config-list.h into its own
script (it was previously roommates with command-list.h).

The following 4 commits were previously a single commit which used
run-command.h to call different processes; this has been removed except
for the cURL version.

  bugreport: gather git version and build info
Additionally capture the "--build-info" output from 'git version' where it
was not previously captured.

  help: add shell-path to --build-options
Adding $SHELL_PATH to 'git --version --build-options' gets it in the
bugreport for free. (This is the shell Git uses, determined at build
time.)

  bugreport: add uname info
Capture uname from the library instead of invoking a different process.

  bugreport: add glibc version
Capture glibc version info from the library instead of invoking the
'ldd' process.

  bugreport: add curl version
Capture the cURL version from git-http-fetch, and add a -V flag to that
binary for this purpose.


  bugreport: include user interactive shell
A new feature; check what $SHELL the user is using (at runtime)

  bugreport: generate config safelist based on docs
This commit squashes in suggestions both from Johannes Schindelin and
Martin Ågren, hopefully resulting in Asciidoc(tor) not showing the
safelist macros, and a more portable safelist generation script. It also
puts the safelist, sorted, directly into a header instead of into a text
file, saving us a file read at runtime.

  bugreport: add config values from safelist
At Dscho's suggestion, now we put these into a map (based on khash and
oidset.h's example) for constant-time lookup during the config parse.

  bugreport: collect list of populated hooks
Only a minor change - now this works if 'git bugreport' is invoked
outside of a Git repo.

  bugreport: count loose objects
  bugreport: add packed object summary
  bugreport: list contents of $OBJDIR/info
For the above 3, the entire 'string_list' has been refactored out; now
the obj_info buffer is written to directly during the directory
traversal. Also paid more attention to when to free strbuf used for paths.

  bugreport: summarize contents of alternates file
Rather than printing the contents of the alternates file, now the number
of specified alternate stores are listed, and they are tested to check
whether they exist. This way we stop looking at user filepaths
(sensitive) and start learning whether they have a broken alternate (new
and useful info).

 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   8 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  68 ++--
 Documentation/git-bugreport.txt         |  43 +++
 Documentation/git-http-fetch.txt        |   5 +
 Makefile                                |  31 +-
 bugreport.c                             | 434 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 generate-bugreport-config-safelist.sh   |  22 ++
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  24 ++
 help.c                                  | 131 ++-----
 help.h                                  |   2 +-
 http-fetch.c                            |  13 +-
 t/t0091-bugreport.sh                    |  41 +++
 16 files changed, 774 insertions(+), 163 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index 055a84c4a8..30935621d9 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,7 +189,9 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..5d5359fcf9 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..750bdff9af 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 69f3e4f219..92f5082013 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -1,63 +1,63 @@
-sendemail.identity:: // bugreport-exclude
+sendemail.identity bugreport:exclude[x] ::
 	A configuration identity. When given, causes values in the
 	'sendemail.<identity>' subsection to take precedence over
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption:: // bugreport-include
+sendemail.smtpEncryption bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
-sendemail.smtpssl (deprecated):: // bugreport-exclude
+sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
 
-sendemail.smtpsslcertpath:: // bugreport-exclude
+sendemail.smtpsslcertpath bugreport:exclude[x] ::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*:: // bugreport-exclude
+sendemail.<identity>.* bugreport:exclude[x] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
 	`sendemail.identity`.
 
-sendemail.aliasesFile:: // bugreport-exclude
-sendemail.aliasFileType:: // bugreport-exclude
-sendemail.annotate:: // bugreport-include
-sendemail.bcc:: // bugreport-include
-sendemail.cc:: // bugreport-include
-sendemail.ccCmd:: // bugreport-include
-sendemail.chainReplyTo:: // bugreport-include
-sendemail.confirm:: // bugreport-include
-sendemail.envelopeSender:: // bugreport-include
-sendemail.from:: // bugreport-include
-sendemail.multiEdit:: // bugreport-include
-sendemail.signedoffbycc:: // bugreport-include
-sendemail.smtpPass:: // bugreport-exclude
-sendemail.suppresscc:: // bugreport-include
-sendemail.suppressFrom:: // bugreport-include
-sendemail.to:: // bugreport-include
-sendemail.tocmd:: // bugreport-include
-sendemail.smtpDomain:: // bugreport-include
-sendemail.smtpServer:: // bugreport-include
-sendemail.smtpServerPort:: // bugreport-include
-sendemail.smtpServerOption:: // bugreport-include
-sendemail.smtpUser:: // bugreport-exclude
-sendemail.thread:: // bugreport-include
-sendemail.transferEncoding:: // bugreport-include
-sendemail.validate:: // bugreport-include
-sendemail.xmailer:: // bugreport-include
+sendemail.aliasesFile bugreport:exclude[x] ::
+sendemail.aliasFileType bugreport:exclude[x] ::
+sendemail.annotate bugreport:include[x] ::
+sendemail.bcc bugreport:include[x] ::
+sendemail.cc bugreport:include[x] ::
+sendemail.ccCmd bugreport:include[x] ::
+sendemail.chainReplyTo bugreport:include[x] ::
+sendemail.confirm bugreport:include[x] ::
+sendemail.envelopeSender bugreport:include[x] ::
+sendemail.from bugreport:include[x] ::
+sendemail.multiEdit bugreport:include[x] ::
+sendemail.signedoffbycc bugreport:include[x] ::
+sendemail.smtpPass bugreport:exclude[x] ::
+sendemail.suppresscc bugreport:include[x] ::
+sendemail.suppressFrom bugreport:include[x] ::
+sendemail.to bugreport:include[x] ::
+sendemail.tocmd bugreport:include[x] ::
+sendemail.smtpDomain bugreport:include[x] ::
+sendemail.smtpServer bugreport:include[x] ::
+sendemail.smtpServerPort bugreport:include[x] ::
+sendemail.smtpServerOption bugreport:include[x] ::
+sendemail.smtpUser bugreport:exclude[x] ::
+sendemail.thread bugreport:include[x] ::
+sendemail.transferEncoding bugreport:include[x] ::
+sendemail.validate bugreport:include[x] ::
+sendemail.xmailer bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.
 
-sendemail.signedoffcc (deprecated):: // bugreport-exclude
+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize:: // bugreport-include
+sendemail.smtpBatchSize bugreport:include[x] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay:: // bugreport-include
+sendemail.smtpReloginDelay bugreport:include[x] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..75f0c80acf
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,43 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+The following information is captured automatically:
+
+ - Git version (`git version --build-options`)
+ - Machine information (`uname -a`)
+ - Versions of various dependencies
+ - Git config contents (`git config --show-origin --list`)
+ - The names of all configured git-hooks in `.git/hooks/`
+
+OPTIONS
+-------
+-o [<path>]::
+--output [<path>]::
+	Place the resulting bug report file in <path> instead of the root of the
+	Git repository.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt
index 666b042679..2894c5e82b 100644
--- a/Documentation/git-http-fetch.txt
+++ b/Documentation/git-http-fetch.txt
@@ -10,6 +10,7 @@ SYNOPSIS
 --------
 [verse]
 'git http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin] <commit> <url>
+'git http-fetch' [-V]
 
 DESCRIPTION
 -----------
@@ -30,6 +31,10 @@ commit-id::
 -v::
 	Report what is downloaded.
 
+-V::
+	Report information about the version of git-http-fetch, including the
+	versions of its dependencies.
+
 -w <filename>::
         Writes the commit-id into the filename under $GIT_DIR/refs/<filename> on
         the local end after the transfer is complete.
diff --git a/Makefile b/Makefile
index 6e361eed64..76dc51e2b1 100644
--- a/Makefile
+++ b/Makefile
@@ -634,10 +634,6 @@ SCRIPT_PYTHON += git-p4.py
 SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH))
 SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL))
 SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON))
-SCRIPT_DEPENDENCIES = git-bugreport-config-whitelist
-
-$(SCRIPT_DEPENDENCIES): Documentation/config/*.txt
-	sh bugreport-generate-config-whitelist.sh
 
 # Individual rules to allow e.g.
 # "make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script"
@@ -662,13 +658,10 @@ clean-perl-script:
 	$(RM) $(SCRIPT_PERL_GEN)
 clean-python-script:
 	$(RM) $(SCRIPT_PYTHON_GEN)
-clean-script-dependencies:
-	$(RM) $(SCRIPT_DEPENDENCIES)
 
 SCRIPTS = $(SCRIPT_SH_GEN) \
 	  $(SCRIPT_PERL_GEN) \
 	  $(SCRIPT_PYTHON_GEN) \
-	  $(SCRIPT_DEPENDENCIES) \
 	  git-instaweb
 
 ETAGS_TARGET = TAGS
@@ -688,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -821,7 +815,9 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -846,7 +842,6 @@ LIB_OBJS += bisect.o
 LIB_OBJS += blame.o
 LIB_OBJS += blob.o
 LIB_OBJS += branch.o
-LIB_OBJS += bugreport.o
 LIB_OBJS += bulk-checkin.o
 LIB_OBJS += bundle.o
 LIB_OBJS += cache-tree.o
@@ -1049,7 +1044,6 @@ BUILTIN_OBJS += builtin/archive.o
 BUILTIN_OBJS += builtin/bisect--helper.o
 BUILTIN_OBJS += builtin/blame.o
 BUILTIN_OBJS += builtin/branch.o
-BUILTIN_OBJS += builtin/bugreport.o
 BUILTIN_OBJS += builtin/bundle.o
 BUILTIN_OBJS += builtin/cat-file.o
 BUILTIN_OBJS += builtin/check-attr.o
@@ -2135,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2155,6 +2149,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2162,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2457,6 +2463,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
@@ -2788,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -2811,7 +2821,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -3114,7 +3124,9 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo
+	$(RM) config-list.h command-list.h bugreport-config-safelist.h
+	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/bugreport-generate-config-whitelist.sh b/bugreport-generate-config-whitelist.sh
deleted file mode 100755
index ca6b232024..0000000000
--- a/bugreport-generate-config-whitelist.sh
+++ /dev/null
@@ -1,4 +0,0 @@
-#!/bin/sh
-
-grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \
-  >git-bugreport-config-whitelist
diff --git a/bugreport.c b/bugreport.c
index a7bdc72b7f..d2a3fb1f67 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,114 +1,128 @@
 #include "cache.h"
-
-#include "bugreport.h"
-#include "config.h"
-#include "exec-cmd.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
 #include "help.h"
+#include <gnu/libc-version.h>
 #include "run-command.h"
-#include "strbuf.h"
-#include "string-list.h"
-#include "version.h"
-
-#include "dirent.h"
-
-/**
- * A sorted list of config options which we will add to the bugreport. Managed
- * by 'gather_whitelist(...)'.
- */
-struct string_list whitelist = STRING_LIST_INIT_DUP;
-struct strbuf configs_and_values = STRBUF_INIT;
-
-// git version --build-options
-// uname -a
-// curl-config --version
-// ldd --version
-// echo $SHELL
-void get_system_info(struct strbuf *sys_info)
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
+#include "run-command.h"
+
+static void get_http_version_info(struct strbuf *http_info)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
-	struct strbuf std_out = STRBUF_INIT;
 
-	strbuf_reset(sys_info);
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "http-fetch");
+	argv_array_push(&cp.args, "-V");
+	if (capture_command(&cp, http_info, 0))
+	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
+}
 
-	// get git version from native cmd
-	strbuf_addstr(sys_info, "git version: ");
-	strbuf_addstr(sys_info, git_version_string);
-	strbuf_complete_line(sys_info);
+KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
 
-	// system call for other version info
-	argv_array_push(&cp.args, "uname");
-	argv_array_push(&cp.args, "-a");
-	capture_command(&cp, &std_out, 0);
+struct cfgset {
+	kh_cfg_set_t set;
+};
 
-	strbuf_addstr(sys_info, "uname -a: ");
-	strbuf_addbuf(sys_info, &std_out);
-	strbuf_complete_line(sys_info);
-
-	argv_array_clear(&cp.args);
-	strbuf_reset(&std_out);
+struct cfgset safelist;
 
+static void cfgset_init(struct cfgset *set, size_t initial_size)
+{
+	memset(&set->set, 0, sizeof(set->set));
+	if (initial_size)
+		kh_resize_cfg_set(&set->set, initial_size);
+}
 
-	argv_array_push(&cp.args, "curl-config");
-	argv_array_push(&cp.args, "--version");
-	capture_command(&cp, &std_out, 0);
+static int cfgset_insert(struct cfgset *set, const char *cfg_key)
+{
+	int added;
+	kh_put_cfg_set(&set->set, cfg_key, &added);
+	printf("ESS: added %s\n", cfg_key);
+	return !added;
+}
 
-	strbuf_addstr(sys_info, "curl-config --version: ");
-	strbuf_addbuf(sys_info, &std_out);
-	strbuf_complete_line(sys_info);
+static int cfgset_contains(struct cfgset *set, const char *cfg_key)
+{
+	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
+	return pos != kh_end(&set->set);
+}
 
-	argv_array_clear(&cp.args);
-	strbuf_reset(&std_out);
+static void cfgset_clear(struct cfgset *set)
+{
+	kh_release_cfg_set(&set->set);
+	cfgset_init(set, 0);
+}
 
+static void get_system_info(struct strbuf *sys_info)
+{
+	struct strbuf version_info = STRBUF_INIT;
+	struct utsname uname_info;
 
-	argv_array_push(&cp.args, "ldd");
-	argv_array_push(&cp.args, "--version");
-	capture_command(&cp, &std_out, 0);
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	list_version_info(&version_info, 1);
+	strbuf_addbuf(sys_info, &version_info);
+	strbuf_complete_line(sys_info);
 
-	strbuf_addstr(sys_info, "ldd --version: ");
-	strbuf_addbuf(sys_info, &std_out);
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.nodename,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "glibc version: ");
+	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
 
-	argv_array_clear(&cp.args);
-	strbuf_reset(&std_out);
-}
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    getenv("SHELL"));
 
-void gather_whitelist(struct strbuf *path)
-{
-	struct strbuf tmp = STRBUF_INIT;
-	strbuf_read_file(&tmp, path->buf, 0);
-	string_list_init(&whitelist, 1);
-	string_list_split(&whitelist, tmp.buf, '\n', -1);
-	string_list_sort(&whitelist);
+	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
+	get_http_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
-int git_config_bugreport(const char *var, const char *value, void *cb)
+static void gather_safelist()
 {
-	if (string_list_has_string(&whitelist, var)) {
-		strbuf_addf(&configs_and_values,
-			    "%s : %s\n",
-			    var, value);
-	}
+	int index;
+	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
+	cfgset_init(&safelist, safelist_len);
+	for (index = 0; index < safelist_len; index++)
+		cfgset_insert(&safelist, bugreport_config_safelist[index]);
 
-	return 0;
 }
 
-void get_whitelisted_config(struct strbuf *config_info)
+static int git_config_bugreport(const char *var, const char *value, void *cb)
 {
-	struct strbuf path = STRBUF_INIT;
+	struct strbuf *config_info = (struct strbuf *)cb;
 
-	strbuf_addstr(&path, git_exec_path());
-	strbuf_addstr(&path, "/git-bugreport-config-whitelist");
+	if (cfgset_contains(&safelist, var))
+		strbuf_addf(config_info,
+			    "%s (%s) : %s\n",
+			    var, config_scope_to_string(current_config_scope()),
+			    value);
 
-	gather_whitelist(&path);
-	strbuf_init(&configs_and_values, whitelist.nr);
-
-	git_config(git_config_bugreport, NULL);
+	return 0;
+}
 
-	strbuf_reset(config_info);
-	strbuf_addbuf(config_info, &configs_and_values);
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	gather_safelist();
+	git_config(git_config_bugreport, config_info);
+	cfgset_clear(&safelist);
 }
 
-void get_populated_hooks(struct strbuf *hook_info)
+static void get_populated_hooks(struct strbuf *hook_info)
 {
 	/*
 	 * Doesn't look like there is a list of all possible hooks; so below is
@@ -139,8 +153,15 @@ void get_populated_hooks(struct strbuf *hook_info)
 			    "post-index-changex";
 	struct string_list hooks_list = STRING_LIST_INIT_DUP;
 	struct string_list_item *iter = NULL;
+	int nongit_ok;
+
+	setup_git_directory_gently(&nongit_ok);
 
-	strbuf_reset(hook_info);
+	if (nongit_ok) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
 
 	string_list_split(&hooks_list, hooks, ',', -1);
 
@@ -152,163 +173,262 @@ void get_populated_hooks(struct strbuf *hook_info)
 	}
 }
 
-/**
- * Fill 'contents' with the contents of the dir at 'dirpath'.
- * If 'filter' is nonzero, the contents are filtered on d_type as 'type' - see
- * 'man readdir'. opendir() doesn't take string length as an arg, so don't
- * bother passing it in.
- */
-void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,
-			  int filter, unsigned char type)
+static int is_hex(const char *string, size_t count)
 {
-	struct dirent *dir = NULL;
-	DIR *dh = NULL;
-
-	dh = opendir(dirpath->buf);
-	while (dh && (dir = readdir(dh))) {
-		if (!filter || type == dir->d_type) {
-			string_list_append(contents, dir->d_name);
-		}
+	for (; count; string++, count--) {
+		if (!isxdigit(*string))
+			return 0;
 	}
+	return 1;
 }
 
-/**
- * Fills 'contents' with a list of all directories within the provided
- * directory, recursing into each directory.
- */
-void list_contents_of_dir_recursively(struct string_list *contents,
-				      struct strbuf *dirpath)
-{
-	struct string_list current_contents = STRING_LIST_INIT_DUP;
-	struct string_list current_subdirs = STRING_LIST_INIT_DUP;
-	struct string_list_item *it;
-	struct strbuf buf = STRBUF_INIT;
-
-	list_contents_of_dir(&current_contents, dirpath, 0, 0);
-	for_each_string_list_item(it, &current_contents) {
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, dirpath);
-		strbuf_complete(&buf, '/');
-		strbuf_addstr(&buf, it->string);
-
-		string_list_append(contents, buf.buf);
-	}
+static void get_loose_object_summary(struct strbuf *obj_info) {
+	struct dirent *d = NULL;
+	DIR *dir, *subdir = NULL;
+	size_t dir_len;
+	struct strbuf dirpath = STRBUF_INIT;
 
-	list_contents_of_dir(&current_subdirs, dirpath, 1, DT_DIR);
-	for_each_string_list_item(it, &current_subdirs) {
-		if (strcmp(it->string, ".") == 0
-		    || strcmp(it->string, "..") == 0)
-			continue;
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, dirpath);
-		strbuf_complete(&buf, '/');
-		strbuf_addstr(&buf, it->string);
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
 
-		list_contents_of_dir_recursively(contents, &buf);
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
 	}
-}
 
-void get_object_counts(struct strbuf *obj_info)
-{
-	struct child_process cp = CHILD_PROCESS_INIT;
-	struct strbuf std_out = STRBUF_INIT;
-
-	argv_array_push(&cp.args, "count-objects");
-	argv_array_push(&cp.args, "-vH");
-	cp.git_cmd = 1;
-	capture_command(&cp, &std_out, 0);
+	dir_len = dirpath.len;
 
-	strbuf_reset(obj_info);
-	strbuf_addstr(obj_info, "git-count-objects -vH:\n");
-	strbuf_addbuf(obj_info, &std_out);
-}
+	while ((d = readdir(dir))) {
+		int object_count = 0;
+		char subdir_name[3];
 
-void get_loose_object_summary(struct strbuf *obj_info)
-{
-	struct strbuf dirpath = STRBUF_INIT;
-	struct string_list subdirs = STRING_LIST_INIT_DUP;
-	struct string_list_item *subdir;
-
-	strbuf_reset(obj_info);
+		if (d->d_type != DT_DIR)
+			continue;
 
-	strbuf_addstr(&dirpath, get_object_directory());
-	strbuf_complete(&dirpath, '/');
+		if ((strlen(d->d_name) != 2) || (!is_hex(d->d_name, 2)))
+			continue;
 
-	list_contents_of_dir(&subdirs, &dirpath, 1, DT_DIR);
+		/* copy directory name + \0 */
+		memcpy(subdir_name, d->d_name, 3);
 
-	for_each_string_list_item(subdir, &subdirs)
-	{
-		struct strbuf subdir_buf = STRBUF_INIT;
-		struct string_list objects = STRING_LIST_INIT_DUP;
+		strbuf_setlen(&dirpath, dir_len);
+		strbuf_addstr(&dirpath, d->d_name);
 
-		/*
-		 * Only interested in loose objects - so dirs named with the
-		 * first byte of the object ID
-		 */
-		if (strlen(subdir->string) != 2 || !strcmp(subdir->string, ".."))
+		subdir = opendir(dirpath.buf);
+		if (!subdir)
 			continue;
+		while ((d = readdir(subdir)))
+			if (d->d_type == DT_REG)
+				object_count++;
+
+		closedir(subdir);
 
-		strbuf_addbuf(&subdir_buf, &dirpath);
-		strbuf_addstr(&subdir_buf, subdir->string);
-		list_contents_of_dir(&objects, &subdir_buf, 0, 0);
-		strbuf_addf(obj_info, "%s: %d objects\n", subdir->string,
-			    objects.nr);
+		strbuf_addf(obj_info, "%s: %d\n", subdir_name, object_count);
 	}
+
+
+	closedir(dir);
+	strbuf_release(&dirpath);
 }
 
-void get_packed_object_summary(struct strbuf *obj_info)
+static void get_packed_object_summary(struct strbuf *obj_info)
 {
 	struct strbuf dirpath = STRBUF_INIT;
-	struct string_list contents = STRING_LIST_INIT_DUP;
-	struct string_list_item *entry;
-
-	strbuf_reset(obj_info);
+	struct dirent *d;
+	DIR *dir = NULL;
 
 	strbuf_addstr(&dirpath, get_object_directory());
 	strbuf_complete(&dirpath, '/');
 	strbuf_addstr(&dirpath, "pack/");
-	list_contents_of_dir(&contents, &dirpath, 0, 0);
 
-	// list full contents of $GIT_OBJECT_DIRECTORY/pack/
-	for_each_string_list_item(entry, &contents) {
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	while ((d = readdir(dir))) {
 		strbuf_addbuf(obj_info, &dirpath);
-		strbuf_addstr(obj_info, entry->string);
+		strbuf_addstr(obj_info, d->d_name);
 		strbuf_complete_line(obj_info);
 	}
+
+	closedir(dir);
+	strbuf_release(&dirpath);
 }
 
-void get_object_info_summary(struct strbuf *obj_info)
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
 {
-	// strbuf += GITDIR/info/:
-	// recursively list contents of $GIT_OBJECT_DIRECTORY/info
-	struct strbuf dirpath = STRBUF_INIT;
-	struct string_list contents = STRING_LIST_INIT_DUP;
-	struct string_list_item *entry;
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
 
-	strbuf_reset(obj_info);
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
 
 	strbuf_addstr(&dirpath, get_object_directory());
 	strbuf_complete(&dirpath, '/');
 	strbuf_addstr(&dirpath, "info/");
 
-	list_contents_of_dir_recursively(&contents, &dirpath);
+	list_contents_of_dir_recursively(obj_info, &dirpath);
 
-	for_each_string_list_item(entry, &contents) {
-		strbuf_addstr(obj_info, entry->string);
-		strbuf_complete_line(obj_info);
-	}
+	strbuf_release(&dirpath);
 }
 
-void get_alternates_file(struct strbuf *alternates_info)
+static void get_alternates_summary(struct strbuf *alternates_info)
 {
 	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
 
 	strbuf_addstr(&alternates_path, get_object_directory());
 	strbuf_complete(&alternates_path, '/');
 	strbuf_addstr(&alternates_path, "info/alternates");
 
-	strbuf_reset(alternates_info);
-	strbuf_addbuf(alternates_info, &alternates_path);
-	strbuf_complete_line(alternates_info);
-	strbuf_read_file(alternates_info, alternates_path.buf, 0);
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output <file>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to send.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+
+	get_bug_template(&buffer);
+
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
+	get_header(&buffer, "Configured Hooks");
+	get_populated_hooks(&buffer);
+
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer);
+
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer);
+
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	launch_editor(report_path.buf, NULL, NULL);
+	return 0;
 }
diff --git a/bugreport.h b/bugreport.h
deleted file mode 100644
index 74d1f79960..0000000000
--- a/bugreport.h
+++ /dev/null
@@ -1,44 +0,0 @@
-#include "strbuf.h"
-
-/**
- * Adds the Git version, `uname -a`, and `curl-config --version` to sys_info.
- * The previous contents of sys_info will be discarded.
- */
-void get_system_info(struct strbuf *sys_info);
-
-/**
- * Adds the values of the config items listed in
- * 'git-bugreport-config-whitelist' to config_info. The previous contents of
- * config_info will be discarded.
- */
-void get_whitelisted_config(struct strbuf *sys_info);
-
-/**
- * Adds the paths to all configured hooks (but not their contents). The previous
- * contents of hook_info will be discarded.
- */
-void get_populated_hooks(struct strbuf *hook_info);
-
-/**
- * Adds the output of `git count-object -vH`. The previous contents of hook_info
- * will be discarded.
- */
-void get_loose_object_summary(struct strbuf *obj_info);
-
-/**
- * Adds a list of the contents of '.git/objects/pack'. The previous contents of
- * hook_info will be discarded.
- */
-void get_packed_object_summary(struct strbuf *obj_info);
-
-/**
- * Adds a list of all contents (recursively) of '.git/objects/info'. The
- * previous contents of hook_info will be discarded.
- */
-void get_object_info_summary(struct strbuf *obj_info);
-
-/**
- * Adds the contents of '.git/info/alternates'. The previous contents of
- * alternates_info will be discarded.
- */
-void get_alternates_file(struct strbuf *alt_info);
diff --git a/builtin.h b/builtin.h
index c6373d3289..5cf5df69f7 100644
--- a/builtin.h
+++ b/builtin.h
@@ -135,7 +135,6 @@ int cmd_archive(int argc, const char **argv, const char *prefix);
 int cmd_bisect__helper(int argc, const char **argv, const char *prefix);
 int cmd_blame(int argc, const char **argv, const char *prefix);
 int cmd_branch(int argc, const char **argv, const char *prefix);
-int cmd_bugreport(int argc, const char **argv, const char *prefix);
 int cmd_bundle(int argc, const char **argv, const char *prefix);
 int cmd_cat_file(int argc, const char **argv, const char *prefix);
 int cmd_checkout(int argc, const char **argv, const char *prefix);
diff --git a/builtin/bugreport.c b/builtin/bugreport.c
deleted file mode 100644
index 0784bdc42a..0000000000
--- a/builtin/bugreport.c
+++ /dev/null
@@ -1,90 +0,0 @@
-#include "builtin.h"
-#include "bugreport.h"
-#include "stdio.h"
-#include "strbuf.h"
-#include "time.h"
-
-int get_bug_template(struct strbuf *template)
-{
-	const char template_text[] =
-"Thank you for filling out a Git bug report!\n"
-"Please answer the following questions to help us understand your issue.\n"
-"\n"
-"What did you do before the bug happened? (Steps to reproduce your issue)\n"
-"\n"
-"What did you expect to happen? (Expected behavior)\n"
-"\n"
-"What happened instead? (Actual behavior)\n"
-"\n"
-"What's different between what you expected and what actually happened?\n"
-"\n"
-"Anything else you want to add:\n"
-"\n"
-"Please review the rest of the bug report below.\n"
-"You can delete any lines you don't wish to send.\n";
-
-	strbuf_reset(template);
-	strbuf_add(template, template_text, strlen(template_text));
-	return 0;
-}
-
-void add_header(FILE *report, const char *title)
-{
-	struct strbuf buffer = STRBUF_INIT;
-	strbuf_addf(&buffer, "\n\n[%s]\n", title);
-	strbuf_write(&buffer, report);
-}
-
-int cmd_bugreport(int argc, const char **argv, const char *prefix)
-{
-	struct strbuf buffer = STRBUF_INIT;
-	struct strbuf report_path = STRBUF_INIT;
-	FILE *report;
-	time_t now = time(NULL);
-
-	strbuf_addstr(&report_path, "git-bugreport-");
-	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
-	strbuf_addstr(&report_path, ".txt");
-
-	report = fopen_for_writing(report_path.buf);
-
-	get_bug_template(&buffer);
-	strbuf_write(&buffer, report);
-
-	// add other contents
-	add_header(report, "System Info");
-	get_system_info(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Whitelisted Config");
-	get_whitelisted_config(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Populated Hooks");
-	get_populated_hooks(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Object Counts");
-	get_loose_object_summary(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Packed Object Summary");
-	get_packed_object_summary(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Object Info Data");
-	get_object_info_summary(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Alternates File");
-	get_alternates_file(&buffer);
-	strbuf_write(&buffer, report);
-
-	// Close file
-	// open file in editor
-	launch_editor(report_path, NULL, NULL);
-	fclose(report);
-
-	launch_editor(report_path.buf, NULL, NULL);
-	return 0;
-}
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/config.c b/config.c
index e7052b3977..baab4a916e 100644
--- a/config.c
+++ b/config.c
@@ -3312,6 +3312,23 @@ enum config_scope current_config_scope(void)
 		return current_parsing_scope;
 }
 
+const char *config_scope_to_string(enum config_scope scope)
+{
+	switch (scope) {
+	case CONFIG_SCOPE_SYSTEM:
+		return "system";
+	case CONFIG_SCOPE_GLOBAL:
+		return "global";
+	case CONFIG_SCOPE_REPO:
+		return "repo";
+	case CONFIG_SCOPE_CMDLINE:
+		return "cmdline";
+	case CONFIG_SCOPE_UNKNOWN:
+	default:
+		return "unknown";
+	}
+}
+
 int lookup_config(const char **mapping, int nr_mapping, const char *var)
 {
 	int i;
diff --git a/config.h b/config.h
index 91fd4c5e96..c8bf296dcc 100644
--- a/config.h
+++ b/config.h
@@ -303,6 +303,7 @@ enum config_scope {
 };
 
 enum config_scope current_config_scope(void);
+const char *config_scope_to_string(enum config_scope);
 const char *current_config_origin_type(void);
 const char *current_config_name(void);
 
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..06b8e0c3c4
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
+sort |
+while read line
+do
+	echo "	\"$line\","
+done
+
+cat <<EOF
+};
+EOF
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..eca6a00c30
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/git.c b/git.c
index 2d6a64f019..ce6ab0ece2 100644
--- a/git.c
+++ b/git.c
@@ -473,7 +473,6 @@ static struct cmd_struct commands[] = {
 	{ "bisect--helper", cmd_bisect__helper, RUN_SETUP },
 	{ "blame", cmd_blame, RUN_SETUP },
 	{ "branch", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG },
-	{ "bugreport", cmd_bugreport, RUN_SETUP },
 	{ "bundle", cmd_bundle, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "cat-file", cmd_cat_file, RUN_SETUP },
 	{ "check-attr", cmd_check_attr, RUN_SETUP },
diff --git a/help.c b/help.c
index cf67624a94..6d1bd17971 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
@@ -707,8 +622,34 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void list_version_info(struct strbuf *buf, int build_options)
+{
+	strbuf_reset(buf);
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addf(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -722,25 +663,9 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	list_version_info(&buf, build_options);
+	printf("%s", buf.buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 7a455beeb7..54f6b5f793 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
@@ -38,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void list_version_info(struct strbuf *buf, int build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
diff --git a/http-fetch.c b/http-fetch.c
index a32ac118d9..31844812a1 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -3,9 +3,18 @@
 #include "exec-cmd.h"
 #include "http.h"
 #include "walker.h"
+#include "version.h"
 
 static const char http_fetch_usage[] = "git http-fetch "
-"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
+"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
+
+void NORETURN version_info()
+{
+	printf("git-http-fetch version: %s\n", git_version_string);
+	printf("built from commit: %s\n", git_built_from_commit_string);
+	printf("curl version: %s\n", curl_version());
+	exit(0);
+}
 
 int cmd_main(int argc, const char **argv)
 {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
 		} else if (argv[arg][1] == 'a') {
 		} else if (argv[arg][1] == 'v') {
 			get_verbosely = 1;
+		} else if (argv[arg][1] == 'V') {
+			version_info();
 		} else if (argv[arg][1] == 'w') {
 			write_ref = &argv[arg + 1];
 			arg++;
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..6eb2ee4f66
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then
+			read -r nextline
+			if [-z $nextline]; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	check_all_headers_populated <git-bugreport-* &&
+	rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test -f foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test ! -f git-bugreport-*
+'
+
+test_done

-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 01/15] bugreport: add tool to generate debugging info
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
@ 2019-12-13  0:42 ` Emily Shaffer
  2019-12-13  0:42 ` [PATCH v4 02/15] help: move list_config_help to builtin/help Emily Shaffer
                   ` (14 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:42 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction
steps, expected behavior, and actual behavior. Later, Git can learn how
to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic
information we would otherwise need to ask the user for, we can reduce
the number of question-and-answer round trips between the reporter and
the Git contributor.

Users may also wish to send a report like this to their local "Git
expert" if they have put their repository into a state they are confused
by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 43 ++++++++++++++++++++
 Makefile                        |  5 +++
 bugreport.c                     | 69 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 41 ++++++++++++++++++++
 5 files changed, 159 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index 89b3b79c1a..055a84c4a8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..75f0c80acf
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,43 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+The following information is captured automatically:
+
+ - Git version (`git version --build-options`)
+ - Machine information (`uname -a`)
+ - Versions of various dependencies
+ - Git config contents (`git config --show-origin --list`)
+ - The names of all configured git-hooks in `.git/hooks/`
+
+OPTIONS
+-------
+-o [<path>]::
+--output [<path>]::
+	Place the resulting bug report file in <path> instead of the root of the
+	Git repository.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index b7d7374dac..9dff91436e 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2448,6 +2449,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..5495b31674
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,69 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output <file>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to send.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	launch_editor(report_path.buf, NULL, NULL);
+	return 0;
+}
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..6eb2ee4f66
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then
+			read -r nextline
+			if [-z $nextline]; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	check_all_headers_populated <git-bugreport-* &&
+	rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test -f foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test ! -f git-bugreport-*
+'
+
+test_done
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 02/15] help: move list_config_help to builtin/help
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
  2019-12-13  0:42 ` [PATCH v4 01/15] bugreport: add tool to generate debugging info Emily Shaffer
@ 2019-12-13  0:42 ` Emily Shaffer
  2019-12-13 20:51   ` Junio C Hamano
  2019-12-13  0:43 ` [PATCH v4 03/15] bugreport: gather git version and build info Emily Shaffer
                   ` (13 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:42 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
builtin/clean.o, and builtin/config.o. This meant that help.o was
unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help()
into builtin/help.c (where it makes sense to assume other builtin libraries
are present).

When command-list.h is included but a member is not used, we start to
hear a compiler warning. Since the config list is generated in a fairly
different way than the command list, and since commands and config
options are semantically different, move the config list into its own
header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 16 ++++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 123 insertions(+), 109 deletions(-)
 create mode 100755 generate-configlist.sh

diff --git a/.gitignore b/.gitignore
index 055a84c4a8..5dde2cc4c8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 9dff91436e..c49f55a521 100644
--- a/Makefile
+++ b/Makefile
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..eca6a00c30
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 03/15] bugreport: gather git version and build info
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
  2019-12-13  0:42 ` [PATCH v4 01/15] bugreport: add tool to generate debugging info Emily Shaffer
  2019-12-13  0:42 ` [PATCH v4 02/15] help: move list_config_help to builtin/help Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:06   ` Junio C Hamano
  2019-12-17 18:45   ` Johannes Schindelin
  2019-12-13  0:43 ` [PATCH v4 04/15] help: add shell-path to --build-options Emily Shaffer
                   ` (12 subsequent siblings)
  15 siblings, 2 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Knowing which version of Git a user has and how it was built allows us
to more precisely pin down the circumstances when a certain issue
occurs, so teach bugreport how to tell us the same output as 'git
version --build-options'.

It's not ideal to directly call 'git version --build-options' because
that output goes to stdout. Instead, wrap the version string in a helper
within help.[ch] library, and call that helper from within the bugreport
library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 23 ++++++++++++++++++++++-
 help.c      | 45 +++++++++++++++++++++++++++------------------
 help.h      |  1 +
 3 files changed, 50 insertions(+), 19 deletions(-)

diff --git a/bugreport.c b/bugreport.c
index 5495b31674..59d8b5a3af 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,20 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	struct strbuf version_info = STRBUF_INIT;
+
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	list_version_info(&version_info, 1);
+	strbuf_addbuf(sys_info, &version_info);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
@@ -32,6 +44,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -60,6 +77,10 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	// add other contents
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
diff --git a/help.c b/help.c
index a21487db77..a43693fca5 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void list_version_info(struct strbuf *buf, int build_options)
+{
+	strbuf_reset(buf);
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addf(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,25 +662,9 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	list_version_info(&buf, build_options);
+	printf("%s", buf.buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..54f6b5f793 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void list_version_info(struct strbuf *buf, int build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 04/15] help: add shell-path to --build-options
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (2 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 03/15] bugreport: gather git version and build info Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13  0:43 ` [PATCH v4 05/15] bugreport: add uname info Emily Shaffer
                   ` (11 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It may be useful to know which shell Git was built to try to point to,
in the event that shell-based Git commands are failing. $SHELL_PATH is
set during the build and used to launch the manpage viewer, as well as
by git-compat-util.h, and it's used during tests. 'git version
--build-options' is encouraged for use in bug reports, so it makes sense
to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/help.c b/help.c
index a43693fca5..6d1bd17971 100644
--- a/help.c
+++ b/help.c
@@ -642,6 +642,7 @@ void list_version_info(struct strbuf *buf, int build_options)
 			strbuf_addf(buf, "no commit associated with this build\n");
 		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
 		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 }
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 05/15] bugreport: add uname info
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (3 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 04/15] help: add shell-path to --build-options Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:12   ` Junio C Hamano
  2020-01-10  2:05   ` Aaron Schrab
  2019-12-13  0:43 ` [PATCH v4 06/15] bugreport: add glibc version Emily Shaffer
                   ` (10 subsequent siblings)
  15 siblings, 2 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

The contents of uname() can give us some insight into what sort of
system the user is running on, and help us replicate their setup if need
be. The domainname field is not guaranteed to be available, so don't
collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 59d8b5a3af..9c69e3fa34 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -8,12 +8,25 @@
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
+	struct utsname uname_info;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	list_version_info(&version_info, 1);
 	strbuf_addbuf(sys_info, &version_info);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.nodename,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 06/15] bugreport: add glibc version
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (4 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 05/15] bugreport: add uname info Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:18   ` Junio C Hamano
  2019-12-13  0:43 ` [PATCH v4 07/15] bugreport: add curl version Emily Shaffer
                   ` (9 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

To help pinpoint the source of a regression, it is useful to know the
version of libc which the user's Git client was built with. Let's
include it alongside the other versioning information, which is used to
identify how the client was built.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 9c69e3fa34..af715dc157 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include <gnu/libc-version.h>
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -27,6 +28,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "glibc version: ");
+	strbuf_addstr(sys_info, gnu_get_libc_version());
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 07/15] bugreport: add curl version
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (5 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 06/15] bugreport: add glibc version Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:27   ` Junio C Hamano
  2019-12-17 18:47   ` Johannes Schindelin
  2019-12-13  0:43 ` [PATCH v4 08/15] bugreport: include user interactive shell Emily Shaffer
                   ` (8 subsequent siblings)
  15 siblings, 2 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible for git-http* to be built separately from git; in that
case we want to know what version of cURL is used by git-http*, not
necessarily which version was present at git-bugreport's build time.
So instead, ask git-http-fetch for the version information it knows
about.

git-http-fetch was chosen as git-http-backend was described as a
server-side implementation, and as an accidental fetch in case of
problems was considered less harmful than an accidental push.

Since it could have been built at a different time, also report the
version and built-from commit of git-http-fetch alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-http-fetch.txt |  5 +++++
 bugreport.c                      | 16 ++++++++++++++++
 http-fetch.c                     | 13 ++++++++++++-
 3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt
index 666b042679..2894c5e82b 100644
--- a/Documentation/git-http-fetch.txt
+++ b/Documentation/git-http-fetch.txt
@@ -10,6 +10,7 @@ SYNOPSIS
 --------
 [verse]
 'git http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin] <commit> <url>
+'git http-fetch' [-V]
 
 DESCRIPTION
 -----------
@@ -30,6 +31,10 @@ commit-id::
 -v::
 	Report what is downloaded.
 
+-V::
+	Report information about the version of git-http-fetch, including the
+	versions of its dependencies.
+
 -w <filename>::
         Writes the commit-id into the filename under $GIT_DIR/refs/<filename> on
         the local end after the transfer is complete.
diff --git a/bugreport.c b/bugreport.c
index af715dc157..f5598513d9 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include <gnu/libc-version.h>
+#include "run-command.h"
+
+static void get_http_version_info(struct strbuf *http_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "http-fetch");
+	argv_array_push(&cp.args, "-V");
+	if (capture_command(&cp, http_info, 0))
+	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "glibc version: ");
 	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
+	get_http_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/http-fetch.c b/http-fetch.c
index a32ac118d9..31844812a1 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -3,9 +3,18 @@
 #include "exec-cmd.h"
 #include "http.h"
 #include "walker.h"
+#include "version.h"
 
 static const char http_fetch_usage[] = "git http-fetch "
-"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
+"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
+
+void NORETURN version_info()
+{
+	printf("git-http-fetch version: %s\n", git_version_string);
+	printf("built from commit: %s\n", git_built_from_commit_string);
+	printf("curl version: %s\n", curl_version());
+	exit(0);
+}
 
 int cmd_main(int argc, const char **argv)
 {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
 		} else if (argv[arg][1] == 'a') {
 		} else if (argv[arg][1] == 'v') {
 			get_verbosely = 1;
+		} else if (argv[arg][1] == 'V') {
+			version_info();
 		} else if (argv[arg][1] == 'w') {
 			write_ref = &argv[arg + 1];
 			arg++;
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 08/15] bugreport: include user interactive shell
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (6 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 07/15] bugreport: add curl version Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:38   ` Junio C Hamano
  2019-12-13  0:43 ` [PATCH v4 09/15] bugreport: generate config safelist based on docs Emily Shaffer
                   ` (7 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible a user may complain about the way that Git interacts with
their interactive shell, e.g. autocompletion or shell prompt. In that
case, it's useful for us to know which shell they're using
interactively.

$SHELL isn't set by every shell, but getenv() returns NULL in the
event the variable isn't found, so we'll see a line like "$SHELL:
(null)" to tell us that variable wasn't set.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index f5598513d9..759cc0b0f8 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -45,6 +45,9 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
 
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    getenv("SHELL"));
+
 	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
 	get_http_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (7 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 08/15] bugreport: include user interactive shell Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 22:57   ` Junio C Hamano
                     ` (2 more replies)
  2019-12-13  0:43 ` [PATCH v4 10/15] bugreport: add config values from safelist Emily Shaffer
                   ` (6 subsequent siblings)
  15 siblings, 3 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Martin Ågren, Johannes Schindelin

Add a new step to the build to generate a safelist of git-config
variables which are appropriate to include in the output of
git-bugreport. New variables can be added to the safelist by annotating
their documentation in Documentation/config with the
"bugreport" macro, which is recognized by AsciiDoc and
AsciiDoctor.

Some configs are private in nature, and can contain remote URLs,
passwords, or other sensitive information. In the event that a user
doesn't notice their information while reviewing a bugreport, that user
may leak their credentials to other individuals, mailing lists, or bug
tracking tools inadvertently. Heuristic blocklisting of configuration
keys is imperfect and prone to false negatives; given the nature of the
information which can be leaked, a safelist is more reliable.

In order to prevent staleness of the safelist, add a mechanism to
generate the safelist from annotations in the config documentation,
where contributors are already used to documenting their new config
keys.

Implement a new no-op "bugreport" macro for use as
"bugreport:include[x]" to annotate the config keys that should be
included in the automatically generated safelist. Use "exclude" for the
others.

With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc
seems to want something between the brackets. A bit unfortunate, but
not a huge problem -- we'll just provide an "x".

"doc-diff" reports that this macro doesn't render at all. That is,
these are both empty after this commit:

  cd Documentation
  ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
  ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

Diffing the rendered HTML shows that there is some small amount of
whitespace and comments added. That shouldn't be a problem.

We could perhaps let the implementation verify that the "action" is one
of "include" and "exclude". For the Asciidoctor implementation that
should be straightforward, but for AsciiDoc I don't immediately know how
to do it. Anyway, if someone stumbles on the keyboard and writes
"bugreport:icndule", they'll "only" miss out on the config key being
included in the safelist. If this were a blocklist, the consequences of
a misspelled target could be a lot more severe.

Additionally, add annotations to the sendemail config documentation in
order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  8 +++
 Documentation/asciidoctor-extensions.rb |  7 +++
 Documentation/config/sendemail.txt      | 68 ++++++++++++-------------
 Makefile                                | 14 +++--
 generate-bugreport-config-safelist.sh   | 22 ++++++++
 6 files changed, 83 insertions(+), 37 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index 5dde2cc4c8..30935621d9 100644
--- a/.gitignore
+++ b/.gitignore
@@ -191,6 +191,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..5d5359fcf9 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..750bdff9af 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..92f5082013 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -1,63 +1,63 @@
-sendemail.identity::
+sendemail.identity bugreport:exclude[x] ::
 	A configuration identity. When given, causes values in the
 	'sendemail.<identity>' subsection to take precedence over
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
-sendemail.smtpssl (deprecated)::
+sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
 
-sendemail.smtpsslcertpath::
+sendemail.smtpsslcertpath bugreport:exclude[x] ::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* bugreport:exclude[x] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
 	`sendemail.identity`.
 
-sendemail.aliasesFile::
-sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.aliasesFile bugreport:exclude[x] ::
+sendemail.aliasFileType bugreport:exclude[x] ::
+sendemail.annotate bugreport:include[x] ::
+sendemail.bcc bugreport:include[x] ::
+sendemail.cc bugreport:include[x] ::
+sendemail.ccCmd bugreport:include[x] ::
+sendemail.chainReplyTo bugreport:include[x] ::
+sendemail.confirm bugreport:include[x] ::
+sendemail.envelopeSender bugreport:include[x] ::
+sendemail.from bugreport:include[x] ::
+sendemail.multiEdit bugreport:include[x] ::
+sendemail.signedoffbycc bugreport:include[x] ::
+sendemail.smtpPass bugreport:exclude[x] ::
+sendemail.suppresscc bugreport:include[x] ::
+sendemail.suppressFrom bugreport:include[x] ::
+sendemail.to bugreport:include[x] ::
+sendemail.tocmd bugreport:include[x] ::
+sendemail.smtpDomain bugreport:include[x] ::
+sendemail.smtpServer bugreport:include[x] ::
+sendemail.smtpServerPort bugreport:include[x] ::
+sendemail.smtpServerOption bugreport:include[x] ::
+sendemail.smtpUser bugreport:exclude[x] ::
+sendemail.thread bugreport:include[x] ::
+sendemail.transferEncoding bugreport:include[x] ::
+sendemail.validate bugreport:include[x] ::
+sendemail.xmailer bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.
 
-sendemail.signedoffcc (deprecated)::
+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize bugreport:include[x] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay bugreport:include[x] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index c49f55a521..76dc51e2b1 100644
--- a/Makefile
+++ b/Makefile
@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
 install-python-script: $(SCRIPT_PYTHON_GEN)
 	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 
-.PHONY: clean-perl-script clean-sh-script clean-python-script
+.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
 clean-sh-script:
 	$(RM) $(SCRIPT_SH_GEN)
 clean-perl-script:
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := config-list.h command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo
+	$(RM) config-list.h command-list.h bugreport-config-safelist.h
 	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..06b8e0c3c4
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
+sort |
+while read line
+do
+	echo "	\"$line\","
+done
+
+cat <<EOF
+};
+EOF
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 10/15] bugreport: add config values from safelist
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (8 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 09/15] bugreport: generate config safelist based on docs Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:45   ` Junio C Hamano
  2019-12-29 20:17   ` Johannes Schindelin
  2019-12-13  0:43 ` [PATCH v4 11/15] bugreport: collect list of populated hooks Emily Shaffer
                   ` (5 subsequent siblings)
  15 siblings, 2 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach bugreport to gather the values of config options which are present
in 'bugreport-config-safelist.h'.

Many config options are sensitive, and many Git add-ons use config
options which git-core does not know about; it is better only to gather
config options which we know to be safe, rather than excluding options
which we know to be unsafe.

Taking the build-time generated array and putting it into a set saves us
time - since git_config_bugreport() is called for every option the user
has configured, performing option lookup in constant time is a useful
optimization.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 71 insertions(+), 1 deletion(-)

diff --git a/bugreport.c b/bugreport.c
index 759cc0b0f8..1fca28f0b9 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include <gnu/libc-version.h>
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_http_version_info(struct strbuf *http_info)
 {
@@ -18,6 +21,41 @@ static void get_http_version_info(struct strbuf *http_info)
 	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
 }
 
+KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
+
+struct cfgset {
+	kh_cfg_set_t set;
+};
+
+struct cfgset safelist;
+
+static void cfgset_init(struct cfgset *set, size_t initial_size)
+{
+	memset(&set->set, 0, sizeof(set->set));
+	if (initial_size)
+		kh_resize_cfg_set(&set->set, initial_size);
+}
+
+static int cfgset_insert(struct cfgset *set, const char *cfg_key)
+{
+	int added;
+	kh_put_cfg_set(&set->set, cfg_key, &added);
+	printf("ESS: added %s\n", cfg_key);
+	return !added;
+}
+
+static int cfgset_contains(struct cfgset *set, const char *cfg_key)
+{
+	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
+	return pos != kh_end(&set->set);
+}
+
+static void cfgset_clear(struct cfgset *set)
+{
+	kh_release_cfg_set(&set->set);
+	cfgset_init(set, 0);
+}
+
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
@@ -53,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void gather_safelist()
+{
+	int index;
+	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
+	cfgset_init(&safelist, safelist_len);
+	for (index = 0; index < safelist_len; index++)
+		cfgset_insert(&safelist, bugreport_config_safelist[index]);
+
+}
+
+static int git_config_bugreport(const char *var, const char *value, void *cb)
+{
+	struct strbuf *config_info = (struct strbuf *)cb;
+
+	if (cfgset_contains(&safelist, var))
+		strbuf_addf(config_info,
+			    "%s (%s) : %s\n",
+			    var, config_scope_to_string(current_config_scope()),
+			    value);
+
+	return 0;
+}
+
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	gather_safelist();
+	git_config(git_config_bugreport, config_info);
+	cfgset_clear(&safelist);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -114,10 +182,12 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
-	// add other contents
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 11/15] bugreport: collect list of populated hooks
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (9 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 10/15] bugreport: add config values from safelist Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:47   ` Junio C Hamano
  2019-12-13  0:43 ` [PATCH v4 12/15] bugreport: count loose objects Emily Shaffer
                   ` (4 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific
hook which is being run, perhaps without the user realizing. While the
contents of hooks can be sensitive - containing user data or process
information specific to the user's organization - simply knowing that a
hook is being run at a certain stage can help us to understand whether
something is going wrong.

Without a definitive list of hook names within the code, we compile our
own list from the documentation. This is likely prone to bitrot. To
reduce the amount of code humans need to read, we turn the list into a
string_list and iterate over it (as we are calling the same find_hook
operation on each string). However, since bugreport should primarily be
called by the user, the performance loss from massaging the string
seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 1fca28f0b9..f89cb8d754 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_http_version_info(struct strbuf *http_info)
 {
@@ -121,6 +122,57 @@ static void get_safelisted_config(struct strbuf *config_info)
 	cfgset_clear(&safelist);
 }
 
+static void get_populated_hooks(struct strbuf *hook_info)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+	int nongit_ok;
+
+	setup_git_directory_gently(&nongit_ok);
+
+	if (nongit_ok) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -188,6 +240,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Configured Hooks");
+	get_populated_hooks(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 12/15] bugreport: count loose objects
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (10 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 11/15] bugreport: collect list of populated hooks Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:51   ` Junio C Hamano
  2019-12-13  0:43 ` [PATCH v4 13/15] bugreport: add packed object summary Emily Shaffer
                   ` (3 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Johannes Schindelin

The number of unpacked objects in a user's repository may help us
understand the root of the problem they're seeing, especially if a
command is running unusually slowly.

Rather than directly invoking 'git-count-objects', which may sometimes
fail unexpectedly on Git for Windows, manually count the contents of
.git/objects. Additionally, since we may wish to inspect other
directories' contents for bugreport in the future, put the directory
listing into a helper function.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index f89cb8d754..3abb83d77f 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -173,6 +173,67 @@ static void get_populated_hooks(struct strbuf *hook_info)
 	}
 }
 
+static int is_hex(const char *string, size_t count)
+{
+	for (; count; string++, count--) {
+		if (!isxdigit(*string))
+			return 0;
+	}
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info) {
+	struct dirent *d = NULL;
+	DIR *dir, *subdir = NULL;
+	size_t dir_len;
+	struct strbuf dirpath = STRBUF_INIT;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	dir_len = dirpath.len;
+
+	while ((d = readdir(dir))) {
+		int object_count = 0;
+		char subdir_name[3];
+
+		if (d->d_type != DT_DIR)
+			continue;
+
+		if ((strlen(d->d_name) != 2) || (!is_hex(d->d_name, 2)))
+			continue;
+
+		/* copy directory name + \0 */
+		memcpy(subdir_name, d->d_name, 3);
+
+		strbuf_setlen(&dirpath, dir_len);
+		strbuf_addstr(&dirpath, d->d_name);
+
+		subdir = opendir(dirpath.buf);
+		if (!subdir)
+			continue;
+		while ((d = readdir(subdir)))
+			if (d->d_type == DT_REG)
+				object_count++;
+
+		closedir(subdir);
+
+		strbuf_addf(obj_info, "%s: %d\n", subdir_name, object_count);
+	}
+
+
+	closedir(dir);
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -243,6 +304,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Configured Hooks");
 	get_populated_hooks(&buffer);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 13/15] bugreport: add packed object summary
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (11 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 12/15] bugreport: count loose objects Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:56   ` Junio C Hamano
  2019-12-13  0:43 ` [PATCH v4 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
                   ` (2 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Alongside the list of loose objects, it's useful to see the list of
object packs as well. It can help us to examine what Git did and did not
pack.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 3abb83d77f..992d8f9de7 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
 	strbuf_release(&dirpath);
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct dirent *d;
+	DIR *dir = NULL;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "pack/");
+
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	while ((d = readdir(dir))) {
+		strbuf_addbuf(obj_info, &dirpath);
+		strbuf_addstr(obj_info, d->d_name);
+		strbuf_complete_line(obj_info);
+	}
+
+	closedir(dir);
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 14/15] bugreport: list contents of $OBJDIR/info
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (12 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 13/15] bugreport: add packed object summary Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13  0:43 ` [PATCH v4 15/15] bugreport: summarize contents of alternates file Emily Shaffer
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Miscellaneous information used about the object store can end up in
.git/objects/info; this can help us understand what may be going on with
the object store when the user is reporting a bug. Otherwise, it could
be difficult to track down what is going wrong with an object which
isn't kept locally to .git/objects/ or .git/objects/pack. Having some
understanding of where the user's objects may be kept can save us some
hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 992d8f9de7..5d300f1c5f 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -262,6 +262,51 @@ static void get_packed_object_summary(struct strbuf *obj_info)
 	strbuf_release(&dirpath);
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -338,6 +383,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v4 15/15] bugreport: summarize contents of alternates file
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (13 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In some cases, it could be that the user is having a problem with an
object which isn't present in their normal object directory. We can get
a hint that that might be the case by examining the list of alternates
where their object may be stored instead. Since paths to alternates may
be sensitive, we'll instead count how many alternates have been
specified and note how many of them exist or are broken.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 5d300f1c5f..d2a3fb1f67 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -307,6 +307,42 @@ static void get_object_info_summary(struct strbuf *obj_info)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -386,6 +422,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 02/15] help: move list_config_help to builtin/help
  2019-12-13  0:42 ` [PATCH v4 02/15] help: move list_config_help to builtin/help Emily Shaffer
@ 2019-12-13 20:51   ` Junio C Hamano
  2019-12-16 21:36     ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2019-12-13 20:51 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
> builtin/clean.o, and builtin/config.o. This meant that help.o was
> unusable outside of the context of the main Git executable.
>
> To make help.o usable by other commands again, move list_config_help()
> into builtin/help.c (where it makes sense to assume other builtin libraries
> are present).
>
> When command-list.h is included but a member is not used, we start to
> hear a compiler warning. Since the config list is generated in a fairly
> different way than the command list, and since commands and config
> options are semantically different, move the config list into its own
> header and move the generator into its own script and build rule.
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---

OK.  Looks like a clean-up that is very worthy even without the
remainder of the topic.

Thanks.

>  .gitignore             |  1 +
>  Makefile               | 16 ++++++--
>  builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
>  generate-cmdlist.sh    | 19 ----------
>  generate-configlist.sh | 24 ++++++++++++
>  help.c                 | 85 -----------------------------------------
>  help.h                 |  1 -
>  7 files changed, 123 insertions(+), 109 deletions(-)
>  create mode 100755 generate-configlist.sh
>
> diff --git a/.gitignore b/.gitignore
> index 055a84c4a8..5dde2cc4c8 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -189,6 +189,7 @@
>  /gitweb/gitweb.cgi
>  /gitweb/static/gitweb.js
>  /gitweb/static/gitweb.min.*
> +/config-list.h
>  /command-list.h
>  *.tar.gz
>  *.dsc
> diff --git a/Makefile b/Makefile
> index 9dff91436e..c49f55a521 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -815,6 +815,7 @@ LIB_FILE = libgit.a
>  XDIFF_LIB = xdiff/lib.a
>  VCSSVN_LIB = vcs-svn/lib.a
>  
> +GENERATED_H += config-list.h
>  GENERATED_H += command-list.h
>  
>  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
> @@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
>  
>  help.sp help.s help.o: command-list.h
>  
> -builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
> +builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
>  builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
>  	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
>  	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
> @@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X
>  	ln -s $< $@ 2>/dev/null || \
>  	cp $< $@
>  
> +config-list.h: generate-configlist.sh
> +
> +config-list.h:
> +	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
> +		>$@+ && mv $@+ $@
> +
>  command-list.h: generate-cmdlist.sh command-list.txt
>  
>  command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
> @@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
>  .PHONY: sparse $(SP_OBJ)
>  sparse: $(SP_OBJ)
>  
> -GEN_HDRS := command-list.h unicode-width.h
> +GEN_HDRS := config-list.h command-list.h unicode-width.h
>  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
>  ifndef GCRYPT_SHA256
>  	EXCEPT_HDRS += sha256/gcrypt.h
> @@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
>  style:
>  	git clang-format --style file --diff --extensions c,h
>  
> -check: command-list.h
> +check: config-list.h command-list.h
>  	@if sparse; \
>  	then \
>  		echo >&2 "Use 'make sparse' instead"; \
> @@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean
>  	$(RM) $(HCC)
>  	$(RM) -r bin-wrappers $(dep_dirs)
>  	$(RM) -r po/build/
> -	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
> +	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> +	$(RM) $(ETAGS_TARGET) tags cscope*
>  	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
>  	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
>  	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
> diff --git a/builtin/help.c b/builtin/help.c
> index e5590d7787..1c5f2b9255 100644
> --- a/builtin/help.c
> +++ b/builtin/help.c
> @@ -8,6 +8,7 @@
>  #include "parse-options.h"
>  #include "run-command.h"
>  #include "column.h"
> +#include "config-list.h"
>  #include "help.h"
>  #include "alias.h"
>  
> @@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
>  	NULL
>  };
>  
> +struct slot_expansion {
> +	const char *prefix;
> +	const char *placeholder;
> +	void (*fn)(struct string_list *list, const char *prefix);
> +	int found;
> +};
> +
> +static void list_config_help(int for_human)
> +{
> +	struct slot_expansion slot_expansions[] = {
> +		{ "advice", "*", list_config_advices },
> +		{ "color.branch", "<slot>", list_config_color_branch_slots },
> +		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
> +		{ "color.diff", "<slot>", list_config_color_diff_slots },
> +		{ "color.grep", "<slot>", list_config_color_grep_slots },
> +		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
> +		{ "color.remote", "<slot>", list_config_color_sideband_slots },
> +		{ "color.status", "<slot>", list_config_color_status_slots },
> +		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
> +		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
> +		{ NULL, NULL, NULL }
> +	};
> +	const char **p;
> +	struct slot_expansion *e;
> +	struct string_list keys = STRING_LIST_INIT_DUP;
> +	int i;
> +
> +	for (p = config_name_list; *p; p++) {
> +		const char *var = *p;
> +		struct strbuf sb = STRBUF_INIT;
> +
> +		for (e = slot_expansions; e->prefix; e++) {
> +
> +			strbuf_reset(&sb);
> +			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
> +			if (!strcasecmp(var, sb.buf)) {
> +				e->fn(&keys, e->prefix);
> +				e->found++;
> +				break;
> +			}
> +		}
> +		strbuf_release(&sb);
> +		if (!e->prefix)
> +			string_list_append(&keys, var);
> +	}
> +
> +	for (e = slot_expansions; e->prefix; e++)
> +		if (!e->found)
> +			BUG("slot_expansion %s.%s is not used",
> +			    e->prefix, e->placeholder);
> +
> +	string_list_sort(&keys);
> +	for (i = 0; i < keys.nr; i++) {
> +		const char *var = keys.items[i].string;
> +		const char *wildcard, *tag, *cut;
> +
> +		if (for_human) {
> +			puts(var);
> +			continue;
> +		}
> +
> +		wildcard = strchr(var, '*');
> +		tag = strchr(var, '<');
> +
> +		if (!wildcard && !tag) {
> +			puts(var);
> +			continue;
> +		}
> +
> +		if (wildcard && !tag)
> +			cut = wildcard;
> +		else if (!wildcard && tag)
> +			cut = tag;
> +		else
> +			cut = wildcard < tag ? wildcard : tag;
> +
> +		/*
> +		 * We may produce duplicates, but that's up to
> +		 * git-completion.bash to handle
> +		 */
> +		printf("%.*s\n", (int)(cut - var), var);
> +	}
> +	string_list_clear(&keys, 0);
> +}
> +
>  static enum help_format parse_help_format(const char *format)
>  {
>  	if (!strcmp(format, "man"))
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> index 71158f7d8b..45fecf8bdf 100755
> --- a/generate-cmdlist.sh
> +++ b/generate-cmdlist.sh
> @@ -76,23 +76,6 @@ print_command_list () {
>  	echo "};"
>  }
>  
> -print_config_list () {
> -	cat <<EOF
> -static const char *config_name_list[] = {
> -EOF
> -	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> -	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> -	sort |
> -	while read line
> -	do
> -		echo "	\"$line\","
> -	done
> -	cat <<EOF
> -	NULL,
> -};
> -EOF
> -}
> -
>  exclude_programs=
>  while test "--exclude-program" = "$1"
>  do
> @@ -113,5 +96,3 @@ echo
>  define_category_names "$1"
>  echo
>  print_command_list "$1"
> -echo
> -print_config_list
> diff --git a/generate-configlist.sh b/generate-configlist.sh
> new file mode 100755
> index 0000000000..eca6a00c30
> --- /dev/null
> +++ b/generate-configlist.sh
> @@ -0,0 +1,24 @@
> +#!/bin/sh
> +
> +echo "/* Automatically generated by generate-configlist.sh */"
> +echo
> +
> +print_config_list () {
> +	cat <<EOF
> +static const char *config_name_list[] = {
> +EOF
> +	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> +	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> +	sort |
> +	while read line
> +	do
> +		echo "	\"$line\","
> +	done
> +	cat <<EOF
> +	NULL,
> +};
> +EOF
> +}
> +
> +echo
> +print_config_list
> diff --git a/help.c b/help.c
> index cf67624a94..a21487db77 100644
> --- a/help.c
> +++ b/help.c
> @@ -407,91 +407,6 @@ void list_common_guides_help(void)
>  	putchar('\n');
>  }
>  
> -struct slot_expansion {
> -	const char *prefix;
> -	const char *placeholder;
> -	void (*fn)(struct string_list *list, const char *prefix);
> -	int found;
> -};
> -
> -void list_config_help(int for_human)
> -{
> -	struct slot_expansion slot_expansions[] = {
> -		{ "advice", "*", list_config_advices },
> -		{ "color.branch", "<slot>", list_config_color_branch_slots },
> -		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
> -		{ "color.diff", "<slot>", list_config_color_diff_slots },
> -		{ "color.grep", "<slot>", list_config_color_grep_slots },
> -		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
> -		{ "color.remote", "<slot>", list_config_color_sideband_slots },
> -		{ "color.status", "<slot>", list_config_color_status_slots },
> -		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
> -		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
> -		{ NULL, NULL, NULL }
> -	};
> -	const char **p;
> -	struct slot_expansion *e;
> -	struct string_list keys = STRING_LIST_INIT_DUP;
> -	int i;
> -
> -	for (p = config_name_list; *p; p++) {
> -		const char *var = *p;
> -		struct strbuf sb = STRBUF_INIT;
> -
> -		for (e = slot_expansions; e->prefix; e++) {
> -
> -			strbuf_reset(&sb);
> -			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
> -			if (!strcasecmp(var, sb.buf)) {
> -				e->fn(&keys, e->prefix);
> -				e->found++;
> -				break;
> -			}
> -		}
> -		strbuf_release(&sb);
> -		if (!e->prefix)
> -			string_list_append(&keys, var);
> -	}
> -
> -	for (e = slot_expansions; e->prefix; e++)
> -		if (!e->found)
> -			BUG("slot_expansion %s.%s is not used",
> -			    e->prefix, e->placeholder);
> -
> -	string_list_sort(&keys);
> -	for (i = 0; i < keys.nr; i++) {
> -		const char *var = keys.items[i].string;
> -		const char *wildcard, *tag, *cut;
> -
> -		if (for_human) {
> -			puts(var);
> -			continue;
> -		}
> -
> -		wildcard = strchr(var, '*');
> -		tag = strchr(var, '<');
> -
> -		if (!wildcard && !tag) {
> -			puts(var);
> -			continue;
> -		}
> -
> -		if (wildcard && !tag)
> -			cut = wildcard;
> -		else if (!wildcard && tag)
> -			cut = tag;
> -		else
> -			cut = wildcard < tag ? wildcard : tag;
> -
> -		/*
> -		 * We may produce duplicates, but that's up to
> -		 * git-completion.bash to handle
> -		 */
> -		printf("%.*s\n", (int)(cut - var), var);
> -	}
> -	string_list_clear(&keys, 0);
> -}
> -
>  static int get_alias(const char *var, const char *value, void *data)
>  {
>  	struct string_list *list = data;
> diff --git a/help.h b/help.h
> index 7a455beeb7..9071894e8c 100644
> --- a/help.h
> +++ b/help.h
> @@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
>  void list_common_cmds_help(void);
>  void list_all_cmds_help(void);
>  void list_common_guides_help(void);
> -void list_config_help(int for_human);
>  
>  void list_all_main_cmds(struct string_list *list);
>  void list_all_other_cmds(struct string_list *list);

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 03/15] bugreport: gather git version and build info
  2019-12-13  0:43 ` [PATCH v4 03/15] bugreport: gather git version and build info Emily Shaffer
@ 2019-12-13 21:06   ` Junio C Hamano
  2019-12-20  1:46     ` Emily Shaffer
  2019-12-17 18:45   ` Johannes Schindelin
  1 sibling, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:06 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> Knowing which version of Git a user has and how it was built allows us
> to more precisely pin down the circumstances when a certain issue
> occurs, so teach bugreport how to tell us the same output as 'git
> version --build-options'.
>
> It's not ideal to directly call 'git version --build-options' because
> that output goes to stdout. Instead, wrap the version string in a helper
> within help.[ch] library, and call that helper from within the bugreport
> library.

Move to strbuf() from stdio makes sense.  

> +	// add other contents

Style.

> diff --git a/help.h b/help.h
> index 9071894e8c..54f6b5f793 100644
> --- a/help.h
> +++ b/help.h
> @@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
>  void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
>  int is_in_cmdlist(struct cmdnames *cmds, const char *name);


Many new helpers are called get_frotz(), but only one among these is
called list_version_info().  I do not think the naming of the
get_*() ones, that are private to the bugreport tool, matters that
much, but unlike "list_commands()" whose purpose is to list the
commands ;-), the new function does not list versions---it just
gives information about a single version which is the one that is
being run, so perhaps it is a misnomer.

>  void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
> +void list_version_info(struct strbuf *buf, int build_options);

It is not clear to the readers build_options is a boolean that
tells the function to include (or not to include) build options.
Perhaps rename it to "int show_build_options" or something?


^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 05/15] bugreport: add uname info
  2019-12-13  0:43 ` [PATCH v4 05/15] bugreport: add uname info Emily Shaffer
@ 2019-12-13 21:12   ` Junio C Hamano
  2020-01-10  2:05   ` Aaron Schrab
  1 sibling, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:12 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> The contents of uname() can give us some insight into what sort of
> system the user is running on, and help us replicate their setup if need
> be. The domainname field is not guaranteed to be available, so don't
> collect it.

It was surprising to me that we do use "struct utsname" somewhere in
the system ;-) ... so at least we know this is just as portable as
the remainder of Git, which is good.

> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/bugreport.c b/bugreport.c
> index 59d8b5a3af..9c69e3fa34 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -8,12 +8,25 @@
>  static void get_system_info(struct strbuf *sys_info)
>  {
>  	struct strbuf version_info = STRBUF_INIT;
> +	struct utsname uname_info;
>  
>  	/* get git version from native cmd */
>  	strbuf_addstr(sys_info, "git version:\n");
>  	list_version_info(&version_info, 1);
>  	strbuf_addbuf(sys_info, &version_info);
>  	strbuf_complete_line(sys_info);
> +
> +	/* system call for other version info */
> +	strbuf_addstr(sys_info, "uname -a: ");
> +	if (uname(&uname_info))
> +		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
> +	else
> +		strbuf_addf(sys_info, "%s %s %s %s %s\n",
> +			    uname_info.sysname,
> +			    uname_info.nodename,
> +			    uname_info.release,
> +			    uname_info.version,
> +			    uname_info.machine);
>  }
>  
>  static const char * const bugreport_usage[] = {

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 06/15] bugreport: add glibc version
  2019-12-13  0:43 ` [PATCH v4 06/15] bugreport: add glibc version Emily Shaffer
@ 2019-12-13 21:18   ` Junio C Hamano
  2019-12-16 22:39     ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> To help pinpoint the source of a regression, it is useful to know the
> version of libc which the user's Git client was built with. Let's
> include it alongside the other versioning information, which is used to
> identify how the client was built.
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 5 +++++
>  1 file changed, 5 insertions(+)

This is the first iffy one in the series.  It is unreasonable to
assume that we can dictate that Git can be built only on glibc
systems, no?

Making this conditional, perhaps make "bugreport.c" depend on
"extern void get_compiler_info(struct strbuf *sys_info)" and require
the function to be defined in compat/; the glibc variant that uses
gnu_get_libc_version() would become just one of them.

> diff --git a/bugreport.c b/bugreport.c
> index 9c69e3fa34..af715dc157 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -4,6 +4,7 @@
>  #include "strbuf.h"
>  #include "time.h"
>  #include "help.h"
> +#include <gnu/libc-version.h>
>  
>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -27,6 +28,10 @@ static void get_system_info(struct strbuf *sys_info)
>  			    uname_info.release,
>  			    uname_info.version,
>  			    uname_info.machine);
> +
> +	strbuf_addstr(sys_info, "glibc version: ");
> +	strbuf_addstr(sys_info, gnu_get_libc_version());
> +	strbuf_complete_line(sys_info);
>  }
>  
>  static const char * const bugreport_usage[] = {

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 07/15] bugreport: add curl version
  2019-12-13  0:43 ` [PATCH v4 07/15] bugreport: add curl version Emily Shaffer
@ 2019-12-13 21:27   ` Junio C Hamano
  2019-12-16 22:49     ` Emily Shaffer
  2019-12-17 18:47   ` Johannes Schindelin
  1 sibling, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:27 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> It's possible for git-http* to be built separately from git; in that
> case we want to know what version of cURL is used by git-http*, not
> necessarily which version was present at git-bugreport's build time.
> So instead, ask git-http-fetch for the version information it knows
> about.
>
> git-http-fetch was chosen as git-http-backend was described as a
> server-side implementation, and as an accidental fetch in case of
> problems was considered less harmful than an accidental push.
>
> Since it could have been built at a different time, also report the
> version and built-from commit of git-http-fetch alongside the cURL info.

One possible issue I have is that I was hoping that eventually we
can discard "git http-fetch" altogether sometime in the future.
Does anybody still use the dumb HTTP transport seriously?  

And the first move in that direction would be to allow the system be
built without http-fetch, even if git-remote-curl (and its aliases)
would still be built to access smart-http transports.

So, I am not sure.  This is just the matter of adding an out-of-line
hidden option used only for environment inspection, so if it can be
done to git-remote-curl, that would probably be much more future
proof.

> diff --git a/bugreport.c b/bugreport.c
> index af715dc157..f5598513d9 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -5,6 +5,18 @@
>  #include "time.h"
>  #include "help.h"
>  #include <gnu/libc-version.h>
> +#include "run-command.h"
> +
> +static void get_http_version_info(struct strbuf *http_info)
> +{
> +	struct child_process cp = CHILD_PROCESS_INIT;
> +
> +	argv_array_push(&cp.args, "git");
> +	argv_array_push(&cp.args, "http-fetch");
> +	argv_array_push(&cp.args, "-V");
> +	if (capture_command(&cp, http_info, 0))
> +	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");

OK.  We probably can also take the compile-time "NO_CURL" into account,
so that we can tell a misconfigured installation that wanted to have
CURL but failed to install a usable http-fetch and an installation
that deliberately omitted anything cURL?

>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)
>  	strbuf_addstr(sys_info, "glibc version: ");
>  	strbuf_addstr(sys_info, gnu_get_libc_version());
>  	strbuf_complete_line(sys_info);
> +
> +	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
> +	get_http_version_info(sys_info);
> +	strbuf_complete_line(sys_info);
>  }
>  
>  static const char * const bugreport_usage[] = {
> diff --git a/http-fetch.c b/http-fetch.c
> index a32ac118d9..31844812a1 100644
> --- a/http-fetch.c
> +++ b/http-fetch.c
> @@ -3,9 +3,18 @@
>  #include "exec-cmd.h"
>  #include "http.h"
>  #include "walker.h"
> +#include "version.h"
>  
>  static const char http_fetch_usage[] = "git http-fetch "
> -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
> +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
> +
> +void NORETURN version_info()

void NORETURN version_info(void)


> +{
> +	printf("git-http-fetch version: %s\n", git_version_string);
> +	printf("built from commit: %s\n", git_built_from_commit_string);
> +	printf("curl version: %s\n", curl_version());
> +	exit(0);
> +}
>  
>  int cmd_main(int argc, const char **argv)
>  {
> @@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
>  		} else if (argv[arg][1] == 'a') {
>  		} else if (argv[arg][1] == 'v') {
>  			get_verbosely = 1;
> +		} else if (argv[arg][1] == 'V') {
> +			version_info();
>  		} else if (argv[arg][1] == 'w') {
>  			write_ref = &argv[arg + 1];
>  			arg++;

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 08/15] bugreport: include user interactive shell
  2019-12-13  0:43 ` [PATCH v4 08/15] bugreport: include user interactive shell Emily Shaffer
@ 2019-12-13 21:38   ` Junio C Hamano
  0 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:38 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> It's possible a user may complain about the way that Git interacts with
> their interactive shell, e.g. autocompletion or shell prompt. In that
> case, it's useful for us to know which shell they're using
> interactively.
>
> $SHELL isn't set by every shell, but getenv() returns NULL in the
> event the variable isn't found, so we'll see a line like "$SHELL:
> (null)" to tell us that variable wasn't set.

Do not depend on vsnprintf() from glibc that accepts NULL and shows
"(null)".  You can segfault on systems without glibc that way.

> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/bugreport.c b/bugreport.c
> index f5598513d9..759cc0b0f8 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -45,6 +45,9 @@ static void get_system_info(struct strbuf *sys_info)
>  	strbuf_addstr(sys_info, gnu_get_libc_version());
>  	strbuf_complete_line(sys_info);
>  
> +	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
> +		    getenv("SHELL"));
> +
>  	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
>  	get_http_version_info(sys_info);
>  	strbuf_complete_line(sys_info);

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 10/15] bugreport: add config values from safelist
  2019-12-13  0:43 ` [PATCH v4 10/15] bugreport: add config values from safelist Emily Shaffer
@ 2019-12-13 21:45   ` Junio C Hamano
  2019-12-16 23:40     ` Emily Shaffer
  2019-12-29 20:17   ` Johannes Schindelin
  1 sibling, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:45 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> Teach bugreport to gather the values of config options which are present
> in 'bugreport-config-safelist.h'.
>
> Many config options are sensitive, and many Git add-ons use config
> options which git-core does not know about; it is better only to gather
> config options which we know to be safe, rather than excluding options
> which we know to be unsafe.
>
> Taking the build-time generated array and putting it into a set saves us
> time - since git_config_bugreport() is called for every option the user
> has configured, performing option lookup in constant time is a useful
> optimization.

Interesting.  I actually was expecting the look-up to go the other
way around.  We know the safe keys, so iterate over them and grab
their values, if defined.  No need for hashes or anything, but just
a simple linear list of safe stuff.

But that is too simple-minded.  If we wanted to safelist foo.*.bar,
where '*' can be anything, walking on the list of safe variables
would not work.  We must have a hash table that knows "foo.*.bar" is
allowed, and while walking all the configuration keys, when we see
foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if
it is whitelisted, or something like that.

But then I am not sure if this implementation does something like
this for three-level names.  If not, I do not see much point in use
of the hash there either.

Puzzled.

>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 71 insertions(+), 1 deletion(-)
>
> diff --git a/bugreport.c b/bugreport.c
> index 759cc0b0f8..1fca28f0b9 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -6,6 +6,9 @@
>  #include "help.h"
>  #include <gnu/libc-version.h>
>  #include "run-command.h"
> +#include "config.h"
> +#include "bugreport-config-safelist.h"
> +#include "khash.h"
>  
>  static void get_http_version_info(struct strbuf *http_info)
>  {
> @@ -18,6 +21,41 @@ static void get_http_version_info(struct strbuf *http_info)
>  	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
>  }
>  
> +KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
> +
> +struct cfgset {
> +	kh_cfg_set_t set;
> +};
> +
> +struct cfgset safelist;
> +
> +static void cfgset_init(struct cfgset *set, size_t initial_size)
> +{
> +	memset(&set->set, 0, sizeof(set->set));
> +	if (initial_size)
> +		kh_resize_cfg_set(&set->set, initial_size);
> +}
> +
> +static int cfgset_insert(struct cfgset *set, const char *cfg_key)
> +{
> +	int added;
> +	kh_put_cfg_set(&set->set, cfg_key, &added);
> +	printf("ESS: added %s\n", cfg_key);
> +	return !added;
> +}
> +
> +static int cfgset_contains(struct cfgset *set, const char *cfg_key)
> +{
> +	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
> +	return pos != kh_end(&set->set);
> +}
> +
> +static void cfgset_clear(struct cfgset *set)
> +{
> +	kh_release_cfg_set(&set->set);
> +	cfgset_init(set, 0);
> +}
> +
>  static void get_system_info(struct strbuf *sys_info)
>  {
>  	struct strbuf version_info = STRBUF_INIT;
> @@ -53,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)
>  	strbuf_complete_line(sys_info);
>  }
>  
> +static void gather_safelist()
> +{
> +	int index;
> +	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
> +	cfgset_init(&safelist, safelist_len);
> +	for (index = 0; index < safelist_len; index++)
> +		cfgset_insert(&safelist, bugreport_config_safelist[index]);
> +
> +}
> +
> +static int git_config_bugreport(const char *var, const char *value, void *cb)
> +{
> +	struct strbuf *config_info = (struct strbuf *)cb;
> +
> +	if (cfgset_contains(&safelist, var))
> +		strbuf_addf(config_info,
> +			    "%s (%s) : %s\n",
> +			    var, config_scope_to_string(current_config_scope()),
> +			    value);
> +
> +	return 0;
> +}
> +
> +static void get_safelisted_config(struct strbuf *config_info)
> +{
> +	gather_safelist();
> +	git_config(git_config_bugreport, config_info);
> +	cfgset_clear(&safelist);
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output <file>]"),
>  	NULL
> @@ -114,10 +182,12 @@ int cmd_main(int argc, const char **argv)
>  
>  	get_bug_template(&buffer);
>  
> -	// add other contents
>  	get_header(&buffer, "System Info");
>  	get_system_info(&buffer);
>  
> +	get_header(&buffer, "Safelisted Config Info");
> +	get_safelisted_config(&buffer);
> +
>  	report = fopen_for_writing(report_path.buf);
>  	strbuf_write(&buffer, report);
>  	fclose(report);

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 11/15] bugreport: collect list of populated hooks
  2019-12-13  0:43 ` [PATCH v4 11/15] bugreport: collect list of populated hooks Emily Shaffer
@ 2019-12-13 21:47   ` Junio C Hamano
  2019-12-16 23:51     ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:47 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +	/*
> +	 * Doesn't look like there is a list of all possible hooks; so below is
> +	 * a transcription of `git help hook`.
> +	 */

We probably would want to employ technique similar to what you used
for the list of safe configuration we saw in an earlier patch in the
series.  What's not even documented is not worth reporting ;-)

> +	const char *hooks = "applypatch-msg,"
> +			    "pre-applypatch,"
> +			    "post-applypatch,"
> +			    "pre-commit,"
> +			    "pre-merge-commit,"
> +			    "prepare-commit-msg,"
> +			    "commit-msg,"
> +			    "post-commit,"
> +			    "pre-rebase,"
> +			    "post-checkout,"
> +			    "post-merge,"
> +			    "pre-push,"
> +			    "pre-receive,"
> +			    "update,"
> +			    "post-receive,"
> +			    "post-update,"
> +			    "push-to-checkout,"
> +			    "pre-auto-gc,"
> +			    "post-rewrite,"
> +			    "sendemail-validate,"
> +			    "fsmonitor-watchman,"
> +			    "p4-pre-submit,"
> +			    "post-index-changex";
> +	struct string_list hooks_list = STRING_LIST_INIT_DUP;
> +	struct string_list_item *iter = NULL;
> +	int nongit_ok;
> +
> +	setup_git_directory_gently(&nongit_ok);
> +
> +	if (nongit_ok) {
> +		strbuf_addstr(hook_info,
> +			"not run from a git repository - no hooks to show\n");
> +		return;
> +	}
> +
> +	string_list_split(&hooks_list, hooks, ',', -1);
> +
> +	for_each_string_list_item(iter, &hooks_list) {
> +		if (find_hook(iter->string)) {
> +			strbuf_addstr(hook_info, iter->string);
> +			strbuf_complete_line(hook_info);
> +		}
> +	}
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output <file>]"),
>  	NULL
> @@ -188,6 +240,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Safelisted Config Info");
>  	get_safelisted_config(&buffer);
>  
> +	get_header(&buffer, "Configured Hooks");
> +	get_populated_hooks(&buffer);
> +
>  	report = fopen_for_writing(report_path.buf);
>  	strbuf_write(&buffer, report);
>  	fclose(report);

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 12/15] bugreport: count loose objects
  2019-12-13  0:43 ` [PATCH v4 12/15] bugreport: count loose objects Emily Shaffer
@ 2019-12-13 21:51   ` Junio C Hamano
  2019-12-16 23:54     ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:51 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Johannes Schindelin

Emily Shaffer <emilyshaffer@google.com> writes:

> The number of unpacked objects in a user's repository may help us
> understand the root of the problem they're seeing, especially if a
> command is running unusually slowly.
>
> Rather than directly invoking 'git-count-objects', which may sometimes
> fail unexpectedly on Git for Windows, manually count the contents of
> .git/objects.

Is for_each_loose_object() or for_each_loose_file_in_objdir() not
sufficient?  We really do *not* want a redundant implementation of
something we already use elsewhere in the system, especially for the
purpose of this program, because you would end up reporting what you
computed in a way that may be quite different from what the rest of
the system that is actually used by the end users computes.


^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 13/15] bugreport: add packed object summary
  2019-12-13  0:43 ` [PATCH v4 13/15] bugreport: add packed object summary Emily Shaffer
@ 2019-12-13 21:56   ` Junio C Hamano
  2019-12-16 23:56     ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:56 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> Alongside the list of loose objects, it's useful to see the list of
> object packs as well. It can help us to examine what Git did and did not
> pack.

Not exactly the same comment, but is in the same spirit, as the
previous step applies to this one.

Would it be too much work to libify the bulk of cmd_count_objects()
that computes numbers, making the cmd_count_objects() into a thin
wrapper that calls the libified "counter/collector" function and
prints the resulting numbers received from it?

That way, the get_packed_object_summary() can be another consumer       
of the same "counter/collector" function, no?

> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 31 +++++++++++++++++++++++++++++++
>  1 file changed, 31 insertions(+)
>
> diff --git a/bugreport.c b/bugreport.c
> index 3abb83d77f..992d8f9de7 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
>  	strbuf_release(&dirpath);
>  }
>  
> +static void get_packed_object_summary(struct strbuf *obj_info)
> +{
> +	struct strbuf dirpath = STRBUF_INIT;
> +	struct dirent *d;
> +	DIR *dir = NULL;
> +
> +	strbuf_addstr(&dirpath, get_object_directory());
> +	strbuf_complete(&dirpath, '/');
> +	strbuf_addstr(&dirpath, "pack/");
> +
> +	dir = opendir(dirpath.buf);
> +	if (!dir) {
> +		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
> +			    dirpath.buf);
> +		strbuf_release(&dirpath);
> +		return;
> +	}
> +
> +	while ((d = readdir(dir))) {
> +		strbuf_addbuf(obj_info, &dirpath);
> +		strbuf_addstr(obj_info, d->d_name);
> +		strbuf_complete_line(obj_info);
> +	}
> +
> +	closedir(dir);
> +	strbuf_release(&dirpath);
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output <file>]"),
>  	NULL
> @@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Loose Object Counts");
>  	get_loose_object_summary(&buffer);
>  
> +	get_header(&buffer, "Packed Object Summary");
> +	get_packed_object_summary(&buffer);
> +
>  	report = fopen_for_writing(report_path.buf);
>  	strbuf_write(&buffer, report);
>  	fclose(report);

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-13  0:43 ` [PATCH v4 09/15] bugreport: generate config safelist based on docs Emily Shaffer
@ 2019-12-13 22:57   ` Junio C Hamano
  2019-12-16 23:01     ` Emily Shaffer
  2019-12-15 20:17   ` Johannes Schindelin
  2019-12-17 18:38   ` Johannes Schindelin
  2 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2019-12-13 22:57 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Martin Ågren, Johannes Schindelin

Emily Shaffer <emilyshaffer@google.com> writes:

> diff --git a/Makefile b/Makefile
> index c49f55a521..76dc51e2b1 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
>  install-python-script: $(SCRIPT_PYTHON_GEN)
>  	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
>  
> -.PHONY: clean-perl-script clean-sh-script clean-python-script
> +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
>  clean-sh-script:
>  	$(RM) $(SCRIPT_SH_GEN)
>  clean-perl-script:
> @@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
>  
>  GENERATED_H += config-list.h
>  GENERATED_H += command-list.h
> +GENERATED_H += bugreport-config-safelist.h

OK.

>  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
>  	$(FIND) . \
> @@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
>  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
>  		command-list.txt >$@+ && mv $@+ $@
>  
> +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
> +
> +bugreport-config-safelist.h: Documentation/config/*.txt
> +	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
> +		>$@+ && mv $@+ $@

OK.

>  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
>  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
>  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\

But bugreport.o needs this *.h file generated before a compiler
attempts to produce it out of bugreport.c; that dependency is
missing from this update to the Makefile.

> @@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
>  .PHONY: sparse $(SP_OBJ)
>  sparse: $(SP_OBJ)
>  
> -GEN_HDRS := config-list.h command-list.h unicode-width.h
> +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
>  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
>  ifndef GCRYPT_SHA256
>  	EXCEPT_HDRS += sha256/gcrypt.h
> @@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
>  	$(RM) $(HCC)
>  	$(RM) -r bin-wrappers $(dep_dirs)
>  	$(RM) -r po/build/
> -	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> +	$(RM) *.pyc *.pyo */*.pyc */*.pyo
> +	$(RM) config-list.h command-list.h bugreport-config-safelist.h

It is kind of sad that GEN_HDRS defines the list of build artifact
that we should be able to clean, and then we manually list them to
be removed here independently.  Can we fix it up too?

We probably clean up the build/update procedure around unicode-width.h
before we can do so, probably.  It may be generatable using contrib/
script, but as far as our normal build is concerned, it is a tracked
source and not part of the build byproducts, so we probably would
want to remove it from GEN_HDRS.  When that happens, we can $(RM)
all of the $(GEN_HDRS) in the "clean" target.


^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-13  0:43 ` [PATCH v4 09/15] bugreport: generate config safelist based on docs Emily Shaffer
  2019-12-13 22:57   ` Junio C Hamano
@ 2019-12-15 20:17   ` Johannes Schindelin
  2019-12-16 22:52     ` Emily Shaffer
  2019-12-17 18:38   ` Johannes Schindelin
  2 siblings, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2019-12-15 20:17 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Martin Ågren

Hi Emily,

On Thu, 12 Dec 2019, Emily Shaffer wrote:

> diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
> new file mode 100755
> index 0000000000..06b8e0c3c4
> --- /dev/null
> +++ b/generate-bugreport-config-safelist.sh
> @@ -0,0 +1,22 @@
> +#!/bin/sh
> +
> +cat <<EOF
> +/* Automatically generated by bugreport-generate-config-safelist.sh */
> +
> +
> +static const char *bugreport_config_safelist[] = {
> +EOF
> +
> +# cat all regular files in Documentation/config
> +find Documentation/config -type f -exec cat {} \; |
> +# print the command name which matches the bugreport-include macro
> +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |

If you use `/  "\1",/` as replacement, you can totally avoid that ugly
`while` loop (that is unfortunately quite slow in MSYS2/Cygwin). You can
still pipe the result into `sort` just the same.

Ciao,
Dscho

> +sort |
> +while read line
> +do
> +	echo "	\"$line\","
> +done
> +
> +cat <<EOF
> +};
> +EOF
> --
> 2.24.1.735.g03f4e72817-goog
>
>

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 02/15] help: move list_config_help to builtin/help
  2019-12-13 20:51   ` Junio C Hamano
@ 2019-12-16 21:36     ` Emily Shaffer
  2019-12-16 22:19       ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2019-12-16 21:36 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 12:51:58PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
> > builtin/clean.o, and builtin/config.o. This meant that help.o was
> > unusable outside of the context of the main Git executable.
> >
> > To make help.o usable by other commands again, move list_config_help()
> > into builtin/help.c (where it makes sense to assume other builtin libraries
> > are present).
> >
> > When command-list.h is included but a member is not used, we start to
> > hear a compiler warning. Since the config list is generated in a fairly
> > different way than the command list, and since commands and config
> > options are semantically different, move the config list into its own
> > header and move the generator into its own script and build rule.
> >
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> 
> OK.  Looks like a clean-up that is very worthy even without the
> remainder of the topic.
> 
> Thanks.

Ok. Are you going to take it on its own? If so, I'll be relying on two
branches in pu/ - what's the best way to let you know I need them both
if/when I send another version of this chain?

> 
> >  .gitignore             |  1 +
> >  Makefile               | 16 ++++++--
> >  builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
> >  generate-cmdlist.sh    | 19 ----------
> >  generate-configlist.sh | 24 ++++++++++++
> >  help.c                 | 85 -----------------------------------------
> >  help.h                 |  1 -
> >  7 files changed, 123 insertions(+), 109 deletions(-)
> >  create mode 100755 generate-configlist.sh
> >
> > diff --git a/.gitignore b/.gitignore
> > index 055a84c4a8..5dde2cc4c8 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -189,6 +189,7 @@
> >  /gitweb/gitweb.cgi
> >  /gitweb/static/gitweb.js
> >  /gitweb/static/gitweb.min.*
> > +/config-list.h
> >  /command-list.h
> >  *.tar.gz
> >  *.dsc
> > diff --git a/Makefile b/Makefile
> > index 9dff91436e..c49f55a521 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -815,6 +815,7 @@ LIB_FILE = libgit.a
> >  XDIFF_LIB = xdiff/lib.a
> >  VCSSVN_LIB = vcs-svn/lib.a
> >  
> > +GENERATED_H += config-list.h
> >  GENERATED_H += command-list.h
> >  
> >  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
> > @@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
> >  
> >  help.sp help.s help.o: command-list.h
> >  
> > -builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
> > +builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
> >  builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
> >  	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
> >  	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
> > @@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X
> >  	ln -s $< $@ 2>/dev/null || \
> >  	cp $< $@
> >  
> > +config-list.h: generate-configlist.sh
> > +
> > +config-list.h:
> > +	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
> > +		>$@+ && mv $@+ $@
> > +
> >  command-list.h: generate-cmdlist.sh command-list.txt
> >  
> >  command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
> > @@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
> >  .PHONY: sparse $(SP_OBJ)
> >  sparse: $(SP_OBJ)
> >  
> > -GEN_HDRS := command-list.h unicode-width.h
> > +GEN_HDRS := config-list.h command-list.h unicode-width.h
> >  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
> >  ifndef GCRYPT_SHA256
> >  	EXCEPT_HDRS += sha256/gcrypt.h
> > @@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
> >  style:
> >  	git clang-format --style file --diff --extensions c,h
> >  
> > -check: command-list.h
> > +check: config-list.h command-list.h
> >  	@if sparse; \
> >  	then \
> >  		echo >&2 "Use 'make sparse' instead"; \
> > @@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean
> >  	$(RM) $(HCC)
> >  	$(RM) -r bin-wrappers $(dep_dirs)
> >  	$(RM) -r po/build/
> > -	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
> > +	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> > +	$(RM) $(ETAGS_TARGET) tags cscope*
> >  	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
> >  	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
> >  	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
> > diff --git a/builtin/help.c b/builtin/help.c
> > index e5590d7787..1c5f2b9255 100644
> > --- a/builtin/help.c
> > +++ b/builtin/help.c
> > @@ -8,6 +8,7 @@
> >  #include "parse-options.h"
> >  #include "run-command.h"
> >  #include "column.h"
> > +#include "config-list.h"
> >  #include "help.h"
> >  #include "alias.h"
> >  
> > @@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
> >  	NULL
> >  };
> >  
> > +struct slot_expansion {
> > +	const char *prefix;
> > +	const char *placeholder;
> > +	void (*fn)(struct string_list *list, const char *prefix);
> > +	int found;
> > +};
> > +
> > +static void list_config_help(int for_human)
> > +{
> > +	struct slot_expansion slot_expansions[] = {
> > +		{ "advice", "*", list_config_advices },
> > +		{ "color.branch", "<slot>", list_config_color_branch_slots },
> > +		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
> > +		{ "color.diff", "<slot>", list_config_color_diff_slots },
> > +		{ "color.grep", "<slot>", list_config_color_grep_slots },
> > +		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
> > +		{ "color.remote", "<slot>", list_config_color_sideband_slots },
> > +		{ "color.status", "<slot>", list_config_color_status_slots },
> > +		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
> > +		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
> > +		{ NULL, NULL, NULL }
> > +	};
> > +	const char **p;
> > +	struct slot_expansion *e;
> > +	struct string_list keys = STRING_LIST_INIT_DUP;
> > +	int i;
> > +
> > +	for (p = config_name_list; *p; p++) {
> > +		const char *var = *p;
> > +		struct strbuf sb = STRBUF_INIT;
> > +
> > +		for (e = slot_expansions; e->prefix; e++) {
> > +
> > +			strbuf_reset(&sb);
> > +			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
> > +			if (!strcasecmp(var, sb.buf)) {
> > +				e->fn(&keys, e->prefix);
> > +				e->found++;
> > +				break;
> > +			}
> > +		}
> > +		strbuf_release(&sb);
> > +		if (!e->prefix)
> > +			string_list_append(&keys, var);
> > +	}
> > +
> > +	for (e = slot_expansions; e->prefix; e++)
> > +		if (!e->found)
> > +			BUG("slot_expansion %s.%s is not used",
> > +			    e->prefix, e->placeholder);
> > +
> > +	string_list_sort(&keys);
> > +	for (i = 0; i < keys.nr; i++) {
> > +		const char *var = keys.items[i].string;
> > +		const char *wildcard, *tag, *cut;
> > +
> > +		if (for_human) {
> > +			puts(var);
> > +			continue;
> > +		}
> > +
> > +		wildcard = strchr(var, '*');
> > +		tag = strchr(var, '<');
> > +
> > +		if (!wildcard && !tag) {
> > +			puts(var);
> > +			continue;
> > +		}
> > +
> > +		if (wildcard && !tag)
> > +			cut = wildcard;
> > +		else if (!wildcard && tag)
> > +			cut = tag;
> > +		else
> > +			cut = wildcard < tag ? wildcard : tag;
> > +
> > +		/*
> > +		 * We may produce duplicates, but that's up to
> > +		 * git-completion.bash to handle
> > +		 */
> > +		printf("%.*s\n", (int)(cut - var), var);
> > +	}
> > +	string_list_clear(&keys, 0);
> > +}
> > +
> >  static enum help_format parse_help_format(const char *format)
> >  {
> >  	if (!strcmp(format, "man"))
> > diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> > index 71158f7d8b..45fecf8bdf 100755
> > --- a/generate-cmdlist.sh
> > +++ b/generate-cmdlist.sh
> > @@ -76,23 +76,6 @@ print_command_list () {
> >  	echo "};"
> >  }
> >  
> > -print_config_list () {
> > -	cat <<EOF
> > -static const char *config_name_list[] = {
> > -EOF
> > -	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> > -	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> > -	sort |
> > -	while read line
> > -	do
> > -		echo "	\"$line\","
> > -	done
> > -	cat <<EOF
> > -	NULL,
> > -};
> > -EOF
> > -}
> > -
> >  exclude_programs=
> >  while test "--exclude-program" = "$1"
> >  do
> > @@ -113,5 +96,3 @@ echo
> >  define_category_names "$1"
> >  echo
> >  print_command_list "$1"
> > -echo
> > -print_config_list
> > diff --git a/generate-configlist.sh b/generate-configlist.sh
> > new file mode 100755
> > index 0000000000..eca6a00c30
> > --- /dev/null
> > +++ b/generate-configlist.sh
> > @@ -0,0 +1,24 @@
> > +#!/bin/sh
> > +
> > +echo "/* Automatically generated by generate-configlist.sh */"
> > +echo
> > +
> > +print_config_list () {
> > +	cat <<EOF
> > +static const char *config_name_list[] = {
> > +EOF
> > +	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> > +	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> > +	sort |
> > +	while read line
> > +	do
> > +		echo "	\"$line\","
> > +	done
> > +	cat <<EOF
> > +	NULL,
> > +};
> > +EOF
> > +}
> > +
> > +echo
> > +print_config_list
> > diff --git a/help.c b/help.c
> > index cf67624a94..a21487db77 100644
> > --- a/help.c
> > +++ b/help.c
> > @@ -407,91 +407,6 @@ void list_common_guides_help(void)
> >  	putchar('\n');
> >  }
> >  
> > -struct slot_expansion {
> > -	const char *prefix;
> > -	const char *placeholder;
> > -	void (*fn)(struct string_list *list, const char *prefix);
> > -	int found;
> > -};
> > -
> > -void list_config_help(int for_human)
> > -{
> > -	struct slot_expansion slot_expansions[] = {
> > -		{ "advice", "*", list_config_advices },
> > -		{ "color.branch", "<slot>", list_config_color_branch_slots },
> > -		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
> > -		{ "color.diff", "<slot>", list_config_color_diff_slots },
> > -		{ "color.grep", "<slot>", list_config_color_grep_slots },
> > -		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
> > -		{ "color.remote", "<slot>", list_config_color_sideband_slots },
> > -		{ "color.status", "<slot>", list_config_color_status_slots },
> > -		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
> > -		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
> > -		{ NULL, NULL, NULL }
> > -	};
> > -	const char **p;
> > -	struct slot_expansion *e;
> > -	struct string_list keys = STRING_LIST_INIT_DUP;
> > -	int i;
> > -
> > -	for (p = config_name_list; *p; p++) {
> > -		const char *var = *p;
> > -		struct strbuf sb = STRBUF_INIT;
> > -
> > -		for (e = slot_expansions; e->prefix; e++) {
> > -
> > -			strbuf_reset(&sb);
> > -			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
> > -			if (!strcasecmp(var, sb.buf)) {
> > -				e->fn(&keys, e->prefix);
> > -				e->found++;
> > -				break;
> > -			}
> > -		}
> > -		strbuf_release(&sb);
> > -		if (!e->prefix)
> > -			string_list_append(&keys, var);
> > -	}
> > -
> > -	for (e = slot_expansions; e->prefix; e++)
> > -		if (!e->found)
> > -			BUG("slot_expansion %s.%s is not used",
> > -			    e->prefix, e->placeholder);
> > -
> > -	string_list_sort(&keys);
> > -	for (i = 0; i < keys.nr; i++) {
> > -		const char *var = keys.items[i].string;
> > -		const char *wildcard, *tag, *cut;
> > -
> > -		if (for_human) {
> > -			puts(var);
> > -			continue;
> > -		}
> > -
> > -		wildcard = strchr(var, '*');
> > -		tag = strchr(var, '<');
> > -
> > -		if (!wildcard && !tag) {
> > -			puts(var);
> > -			continue;
> > -		}
> > -
> > -		if (wildcard && !tag)
> > -			cut = wildcard;
> > -		else if (!wildcard && tag)
> > -			cut = tag;
> > -		else
> > -			cut = wildcard < tag ? wildcard : tag;
> > -
> > -		/*
> > -		 * We may produce duplicates, but that's up to
> > -		 * git-completion.bash to handle
> > -		 */
> > -		printf("%.*s\n", (int)(cut - var), var);
> > -	}
> > -	string_list_clear(&keys, 0);
> > -}
> > -
> >  static int get_alias(const char *var, const char *value, void *data)
> >  {
> >  	struct string_list *list = data;
> > diff --git a/help.h b/help.h
> > index 7a455beeb7..9071894e8c 100644
> > --- a/help.h
> > +++ b/help.h
> > @@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
> >  void list_common_cmds_help(void);
> >  void list_all_cmds_help(void);
> >  void list_common_guides_help(void);
> > -void list_config_help(int for_human);
> >  
> >  void list_all_main_cmds(struct string_list *list);
> >  void list_all_other_cmds(struct string_list *list);

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 02/15] help: move list_config_help to builtin/help
  2019-12-16 21:36     ` Emily Shaffer
@ 2019-12-16 22:19       ` Junio C Hamano
  2019-12-16 22:34         ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2019-12-16 22:19 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

>> OK.  Looks like a clean-up that is very worthy even without the
>> remainder of the topic.
>> 
>> Thanks.
>
> Ok. Are you going to take it on its own?

Not really.  Unless you abandon the rest of the topic, in which case
it may be worth resurrecting this one on its own, that is.  But that
is not what you plan to do anyway, so I expect to see it as part of
an updated series ;-)

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 02/15] help: move list_config_help to builtin/help
  2019-12-16 22:19       ` Junio C Hamano
@ 2019-12-16 22:34         ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-16 22:34 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Mon, Dec 16, 2019 at 02:19:48PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> >> OK.  Looks like a clean-up that is very worthy even without the
> >> remainder of the topic.
> >> 
> >> Thanks.
> >
> > Ok. Are you going to take it on its own?
> 
> Not really.  Unless you abandon the rest of the topic, in which case
> it may be worth resurrecting this one on its own, that is.  But that
> is not what you plan to do anyway, so I expect to see it as part of
> an updated series ;-)

Ok! That's fine and makes my life easier. Just making sure. :)

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 06/15] bugreport: add glibc version
  2019-12-13 21:18   ` Junio C Hamano
@ 2019-12-16 22:39     ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-16 22:39 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 01:18:44PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > To help pinpoint the source of a regression, it is useful to know the
> > version of libc which the user's Git client was built with. Let's
> > include it alongside the other versioning information, which is used to
> > identify how the client was built.
> >
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  bugreport.c | 5 +++++
> >  1 file changed, 5 insertions(+)
> 
> This is the first iffy one in the series.  It is unreasonable to
> assume that we can dictate that Git can be built only on glibc
> systems, no?
> 
> Making this conditional, perhaps make "bugreport.c" depend on
> "extern void get_compiler_info(struct strbuf *sys_info)" and require
> the function to be defined in compat/; the glibc variant that uses
> gnu_get_libc_version() would become just one of them.

Interesting. This sounds like a good approach - thanks, I'll look into
it.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 07/15] bugreport: add curl version
  2019-12-13 21:27   ` Junio C Hamano
@ 2019-12-16 22:49     ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-16 22:49 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 01:27:31PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > It's possible for git-http* to be built separately from git; in that
> > case we want to know what version of cURL is used by git-http*, not
> > necessarily which version was present at git-bugreport's build time.
> > So instead, ask git-http-fetch for the version information it knows
> > about.
> >
> > git-http-fetch was chosen as git-http-backend was described as a
> > server-side implementation, and as an accidental fetch in case of
> > problems was considered less harmful than an accidental push.
> >
> > Since it could have been built at a different time, also report the
> > version and built-from commit of git-http-fetch alongside the cURL info.
> 
> One possible issue I have is that I was hoping that eventually we
> can discard "git http-fetch" altogether sometime in the future.
> Does anybody still use the dumb HTTP transport seriously?  

Oh, interesting. I was about to say, "I still use it to fetch, when I
don't really care" - but that isn't even true, as I fetch via https (and
have so much muscle memory to type https that I don't even notice).`

> 
> And the first move in that direction would be to allow the system be
> built without http-fetch, even if git-remote-curl (and its aliases)
> would still be built to access smart-http transports.
> 
> So, I am not sure.  This is just the matter of adding an out-of-line
> hidden option used only for environment inspection, so if it can be
> done to git-remote-curl, that would probably be much more future
> proof.

Ok. I'll move it.

> 
> > diff --git a/bugreport.c b/bugreport.c
> > index af715dc157..f5598513d9 100644
> > --- a/bugreport.c
> > +++ b/bugreport.c
> > @@ -5,6 +5,18 @@
> >  #include "time.h"
> >  #include "help.h"
> >  #include <gnu/libc-version.h>
> > +#include "run-command.h"
> > +
> > +static void get_http_version_info(struct strbuf *http_info)
> > +{
> > +	struct child_process cp = CHILD_PROCESS_INIT;
> > +
> > +	argv_array_push(&cp.args, "git");
> > +	argv_array_push(&cp.args, "http-fetch");
> > +	argv_array_push(&cp.args, "-V");
> > +	if (capture_command(&cp, http_info, 0))
> > +	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
> 
> OK.  We probably can also take the compile-time "NO_CURL" into account,
> so that we can tell a misconfigured installation that wanted to have
> CURL but failed to install a usable http-fetch and an installation
> that deliberately omitted anything cURL?

Oh, interesting idea! I'll add that.

> 
> >  static void get_system_info(struct strbuf *sys_info)
> >  {
> > @@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)
> >  	strbuf_addstr(sys_info, "glibc version: ");
> >  	strbuf_addstr(sys_info, gnu_get_libc_version());
> >  	strbuf_complete_line(sys_info);
> > +
> > +	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
> > +	get_http_version_info(sys_info);
> > +	strbuf_complete_line(sys_info);
> >  }
> >  
> >  static const char * const bugreport_usage[] = {
> > diff --git a/http-fetch.c b/http-fetch.c
> > index a32ac118d9..31844812a1 100644
> > --- a/http-fetch.c
> > +++ b/http-fetch.c
> > @@ -3,9 +3,18 @@
> >  #include "exec-cmd.h"
> >  #include "http.h"
> >  #include "walker.h"
> > +#include "version.h"
> >  
> >  static const char http_fetch_usage[] = "git http-fetch "
> > -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
> > +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
> > +
> > +void NORETURN version_info()
> 
> void NORETURN version_info(void)
> 
> 
> > +{
> > +	printf("git-http-fetch version: %s\n", git_version_string);
> > +	printf("built from commit: %s\n", git_built_from_commit_string);
> > +	printf("curl version: %s\n", curl_version());
> > +	exit(0);
> > +}
> >  
> >  int cmd_main(int argc, const char **argv)
> >  {
> > @@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
> >  		} else if (argv[arg][1] == 'a') {
> >  		} else if (argv[arg][1] == 'v') {
> >  			get_verbosely = 1;
> > +		} else if (argv[arg][1] == 'V') {
> > +			version_info();
> >  		} else if (argv[arg][1] == 'w') {
> >  			write_ref = &argv[arg + 1];
> >  			arg++;

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-15 20:17   ` Johannes Schindelin
@ 2019-12-16 22:52     ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-16 22:52 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Martin Ågren

On Sun, Dec 15, 2019 at 09:17:57PM +0100, Johannes Schindelin wrote:
> Hi Emily,
> 
> On Thu, 12 Dec 2019, Emily Shaffer wrote:
> 
> > diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
> > new file mode 100755
> > index 0000000000..06b8e0c3c4
> > --- /dev/null
> > +++ b/generate-bugreport-config-safelist.sh
> > @@ -0,0 +1,22 @@
> > +#!/bin/sh
> > +
> > +cat <<EOF
> > +/* Automatically generated by bugreport-generate-config-safelist.sh */
> > +
> > +
> > +static const char *bugreport_config_safelist[] = {
> > +EOF
> > +
> > +# cat all regular files in Documentation/config
> > +find Documentation/config -type f -exec cat {} \; |
> > +# print the command name which matches the bugreport-include macro
> > +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
> 
> If you use `/  "\1",/` as replacement, you can totally avoid that ugly
> `while` loop (that is unfortunately quite slow in MSYS2/Cygwin). You can
> still pipe the result into `sort` just the same.

Oooof. How embarrassing - I will do that.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-13 22:57   ` Junio C Hamano
@ 2019-12-16 23:01     ` Emily Shaffer
  2019-12-17  0:41       ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2019-12-16 23:01 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Martin Ågren, Johannes Schindelin

On Fri, Dec 13, 2019 at 02:57:17PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > diff --git a/Makefile b/Makefile
> > index c49f55a521..76dc51e2b1 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
> >  install-python-script: $(SCRIPT_PYTHON_GEN)
> >  	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
> >  
> > -.PHONY: clean-perl-script clean-sh-script clean-python-script
> > +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
> >  clean-sh-script:
> >  	$(RM) $(SCRIPT_SH_GEN)
> >  clean-perl-script:
> > @@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
> >  
> >  GENERATED_H += config-list.h
> >  GENERATED_H += command-list.h
> > +GENERATED_H += bugreport-config-safelist.h
> 
> OK.
> 
> >  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
> >  	$(FIND) . \
> > @@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
> >  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
> >  		command-list.txt >$@+ && mv $@+ $@
> >  
> > +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
> > +
> > +bugreport-config-safelist.h: Documentation/config/*.txt
> > +	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
> > +		>$@+ && mv $@+ $@
> 
> OK.
> 
> >  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
> >  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
> >  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
> 
> But bugreport.o needs this *.h file generated before a compiler
> attempts to produce it out of bugreport.c; that dependency is
> missing from this update to the Makefile.

Hm. Somewhere I misformatted my commits, then. My goal was to add the
new header here, but not add it as a dependency to anybody; then, in the
next commit, add it as a dependency to git-bugreport and start to use it
(commit 1: make the thing; commit 2: use the thing). But now when I look
at the next commit, I don't see a Makefile change. So I missed something
while I was shuffling around fixups.  Thanks for noticing.

> 
> > @@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
> >  .PHONY: sparse $(SP_OBJ)
> >  sparse: $(SP_OBJ)
> >  
> > -GEN_HDRS := config-list.h command-list.h unicode-width.h
> > +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
> >  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
> >  ifndef GCRYPT_SHA256
> >  	EXCEPT_HDRS += sha256/gcrypt.h
> > @@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
> >  	$(RM) $(HCC)
> >  	$(RM) -r bin-wrappers $(dep_dirs)
> >  	$(RM) -r po/build/
> > -	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> > +	$(RM) *.pyc *.pyo */*.pyc */*.pyo
> > +	$(RM) config-list.h command-list.h bugreport-config-safelist.h
> 
> It is kind of sad that GEN_HDRS defines the list of build artifact
> that we should be able to clean, and then we manually list them to
> be removed here independently.  Can we fix it up too?

Yeah, I can do that. I thought so too when I was updating this.

I *think* what happened is that when I split config-list away (in the
earleir commit in this chain) I didn't notice that command-list.h was
already in GEN_HDRS, and instead just grepped Makefile for
"command-list.h" and added config-list.h next to it. So I'll try to fix
it much earlier, in that commit, and then simply add to the appropriate
variables in this commit.

> 
> We probably clean up the build/update procedure around unicode-width.h
> before we can do so, probably.  It may be generatable using contrib/
> script, but as far as our normal build is concerned, it is a tracked
> source and not part of the build byproducts, so we probably would
> want to remove it from GEN_HDRS.  When that happens, we can $(RM)
> all of the $(GEN_HDRS) in the "clean" target.

Noted. Thanks.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 10/15] bugreport: add config values from safelist
  2019-12-13 21:45   ` Junio C Hamano
@ 2019-12-16 23:40     ` Emily Shaffer
  2019-12-17 17:43       ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2019-12-16 23:40 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 01:45:47PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Teach bugreport to gather the values of config options which are present
> > in 'bugreport-config-safelist.h'.
> >
> > Many config options are sensitive, and many Git add-ons use config
> > options which git-core does not know about; it is better only to gather
> > config options which we know to be safe, rather than excluding options
> > which we know to be unsafe.
> >
> > Taking the build-time generated array and putting it into a set saves us
> > time - since git_config_bugreport() is called for every option the user
> > has configured, performing option lookup in constant time is a useful
> > optimization.
> 
> Interesting.  I actually was expecting the look-up to go the other
> way around.  We know the safe keys, so iterate over them and grab
> their values, if defined.  No need for hashes or anything, but just
> a simple linear list of safe stuff.

Hm. Without looking at the code, I said to myself,
"repo_config_get_value() will open all the available config files to
find the resolved value, so I don't want to do n*4 file open/closes, I
only want to do 4 total."

Now I look at the code and see that the configs are already being read
into a hashset before now. So you're right that it doesn't make sense
for me to do it this way....

> 
> But that is too simple-minded.  If we wanted to safelist foo.*.bar,
> where '*' can be anything, walking on the list of safe variables
> would not work.  We must have a hash table that knows "foo.*.bar" is
> allowed, and while walking all the configuration keys, when we see
> foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if
> it is whitelisted, or something like that.

...unless we want to use wildcards like you suggest.

But I'm not sure it's a good idea. I envision someone writing another
Git add-on, which offers someone to specify "user.password" and
automagically do some Bitbucket interaction, or something. (An extreme
and misguided example, but I hope the point remains.) Then if I say,
"all the user.* configs I can see in 'git help config' are safe, so I
will just safelist user.*," I'm not accounting for this other tool's
known configs which are a superset of what Git knows and expects.

The point of this safelist-generation exercise is to avoid accidentally
printing some sensitive user config, and the point of using a safelist
instead of a blocklist is to avoid printing a third-party config we
didn't know about (or messing up our pattern matching). So I suggest we
avoid pattern matching entirely.

> 
> But then I am not sure if this implementation does something like
> this for three-level names.  If not, I do not see much point in use
> of the hash there either.
> 
> Puzzled.

I'd prefer to solve it by iterating over the compile-time-generated
safelist using get_config_value_multi(), which I clock at O(n) for n =
safelist length: performing n calls to an O(1) hashset lookup. That
saves us the O(n*log(n)) hashset population, and the implementation of
Yet Another Set in the source.

Thanks.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 11/15] bugreport: collect list of populated hooks
  2019-12-13 21:47   ` Junio C Hamano
@ 2019-12-16 23:51     ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-16 23:51 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 01:47:27PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +	/*
> > +	 * Doesn't look like there is a list of all possible hooks; so below is
> > +	 * a transcription of `git help hook`.
> > +	 */
> 
> We probably would want to employ technique similar to what you used
> for the list of safe configuration we saw in an earlier patch in the
> series.  What's not even documented is not worth reporting ;-)

I somewhat slightly would rather not, since I hope to include a list of
the available hooks in my work with git-hook:
https://lore.kernel.org/20191211205114.GD107889@google.com

It's true that the way this patch is written now is prone to bitrot.
It's also true that I could win the lottery tomorrow and never finish my
hopes of git-hook implementation. And it's finally true that I don't
want to write a fourth build-time-generated header which scrapes docs,
especially if I also plan to delete it two months from now.

I wonder whether a better approach might be to drop hook support
entirely here, and add it later on when this particular bit of tech debt
(no canonical list of hook names) is solved, one way or another.

Another approach may be to just list all contents of $(core.hookdir)/
which doesn't end in ".sample", which is what the bash first attempt of
bugreport did. This does mean that those who have something like:

  .git/hooks/
    pre-commit
    pre-commit.d/

  pre-commit:
    sh pre-commit.d/a
    sh pre-commit.d/b

will have their multihook config exposed, although the contents of those
hooks won't be.

I'm interested to know everyone else's opinion, because mine is biased
by my hope that I can solve all the world's problems with git-hook ;)

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 12/15] bugreport: count loose objects
  2019-12-13 21:51   ` Junio C Hamano
@ 2019-12-16 23:54     ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-16 23:54 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Johannes Schindelin

On Fri, Dec 13, 2019 at 01:51:27PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > The number of unpacked objects in a user's repository may help us
> > understand the root of the problem they're seeing, especially if a
> > command is running unusually slowly.
> >
> > Rather than directly invoking 'git-count-objects', which may sometimes
> > fail unexpectedly on Git for Windows, manually count the contents of
> > .git/objects.
> 
> Is for_each_loose_object() or for_each_loose_file_in_objdir() not
> sufficient?  We really do *not* want a redundant implementation of
> something we already use elsewhere in the system, especially for the
> purpose of this program, because you would end up reporting what you
> computed in a way that may be quite different from what the rest of
> the system that is actually used by the end users computes.
> 
Oh, thanks for the pointer. That looks sufficient - I will investigate
the differences between the two and stop filesystem browsing by hand in
this case.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 13/15] bugreport: add packed object summary
  2019-12-13 21:56   ` Junio C Hamano
@ 2019-12-16 23:56     ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-16 23:56 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 01:56:07PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Alongside the list of loose objects, it's useful to see the list of
> > object packs as well. It can help us to examine what Git did and did not
> > pack.
> 
> Not exactly the same comment, but is in the same spirit, as the
> previous step applies to this one.
> 
> Would it be too much work to libify the bulk of cmd_count_objects()
> that computes numbers, making the cmd_count_objects() into a thin
> wrapper that calls the libified "counter/collector" function and
> prints the resulting numbers received from it?
> 
> That way, the get_packed_object_summary() can be another consumer       
> of the same "counter/collector" function, no?

That sounds like a good approach. Will do.

> 
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  bugreport.c | 31 +++++++++++++++++++++++++++++++
> >  1 file changed, 31 insertions(+)
> >
> > diff --git a/bugreport.c b/bugreport.c
> > index 3abb83d77f..992d8f9de7 100644
> > --- a/bugreport.c
> > +++ b/bugreport.c
> > @@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
> >  	strbuf_release(&dirpath);
> >  }
> >  
> > +static void get_packed_object_summary(struct strbuf *obj_info)
> > +{
> > +	struct strbuf dirpath = STRBUF_INIT;
> > +	struct dirent *d;
> > +	DIR *dir = NULL;
> > +
> > +	strbuf_addstr(&dirpath, get_object_directory());
> > +	strbuf_complete(&dirpath, '/');
> > +	strbuf_addstr(&dirpath, "pack/");
> > +
> > +	dir = opendir(dirpath.buf);
> > +	if (!dir) {
> > +		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
> > +			    dirpath.buf);
> > +		strbuf_release(&dirpath);
> > +		return;
> > +	}
> > +
> > +	while ((d = readdir(dir))) {
> > +		strbuf_addbuf(obj_info, &dirpath);
> > +		strbuf_addstr(obj_info, d->d_name);
> > +		strbuf_complete_line(obj_info);
> > +	}
> > +
> > +	closedir(dir);
> > +	strbuf_release(&dirpath);
> > +}
> > +
> >  static const char * const bugreport_usage[] = {
> >  	N_("git bugreport [-o|--output <file>]"),
> >  	NULL
> > @@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)
> >  	get_header(&buffer, "Loose Object Counts");
> >  	get_loose_object_summary(&buffer);
> >  
> > +	get_header(&buffer, "Packed Object Summary");
> > +	get_packed_object_summary(&buffer);
> > +
> >  	report = fopen_for_writing(report_path.buf);
> >  	strbuf_write(&buffer, report);
> >  	fclose(report);

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-16 23:01     ` Emily Shaffer
@ 2019-12-17  0:41       ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-17  0:41 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Martin Ågren, Johannes Schindelin

On Mon, Dec 16, 2019 at 03:01:24PM -0800, Emily Shaffer wrote:
> On Fri, Dec 13, 2019 at 02:57:17PM -0800, Junio C Hamano wrote:
> > Emily Shaffer <emilyshaffer@google.com> writes:
> > 
> > > diff --git a/Makefile b/Makefile
> > > index c49f55a521..76dc51e2b1 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
> > >  install-python-script: $(SCRIPT_PYTHON_GEN)
> > >  	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
> > >  
> > > -.PHONY: clean-perl-script clean-sh-script clean-python-script
> > > +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
> > >  clean-sh-script:
> > >  	$(RM) $(SCRIPT_SH_GEN)
> > >  clean-perl-script:
> > > @@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
> > >  
> > >  GENERATED_H += config-list.h
> > >  GENERATED_H += command-list.h
> > > +GENERATED_H += bugreport-config-safelist.h
> > 
> > OK.
> > 
> > >  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
> > >  	$(FIND) . \
> > > @@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
> > >  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
> > >  		command-list.txt >$@+ && mv $@+ $@
> > >  
> > > +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
> > > +
> > > +bugreport-config-safelist.h: Documentation/config/*.txt
> > > +	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
> > > +		>$@+ && mv $@+ $@
> > 
> > OK.
> > 
> > >  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
> > >  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
> > >  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
> > 
> > But bugreport.o needs this *.h file generated before a compiler
> > attempts to produce it out of bugreport.c; that dependency is
> > missing from this update to the Makefile.
> 
> Hm. Somewhere I misformatted my commits, then. My goal was to add the
> new header here, but not add it as a dependency to anybody; then, in the
> next commit, add it as a dependency to git-bugreport and start to use it
> (commit 1: make the thing; commit 2: use the thing). But now when I look
> at the next commit, I don't see a Makefile change. So I missed something
> while I was shuffling around fixups.  Thanks for noticing.
> 
> > 
> > > @@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
> > >  .PHONY: sparse $(SP_OBJ)
> > >  sparse: $(SP_OBJ)
> > >  
> > > -GEN_HDRS := config-list.h command-list.h unicode-width.h
> > > +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
> > >  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
> > >  ifndef GCRYPT_SHA256
> > >  	EXCEPT_HDRS += sha256/gcrypt.h
> > > @@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
> > >  	$(RM) $(HCC)
> > >  	$(RM) -r bin-wrappers $(dep_dirs)
> > >  	$(RM) -r po/build/
> > > -	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> > > +	$(RM) *.pyc *.pyo */*.pyc */*.pyo
> > > +	$(RM) config-list.h command-list.h bugreport-config-safelist.h
> > 
> > It is kind of sad that GEN_HDRS defines the list of build artifact
> > that we should be able to clean, and then we manually list them to
> > be removed here independently.  Can we fix it up too?
> 
> Yeah, I can do that. I thought so too when I was updating this.
> 
> I *think* what happened is that when I split config-list away (in the
> earleir commit in this chain) I didn't notice that command-list.h was
> already in GEN_HDRS, and instead just grepped Makefile for
> "command-list.h" and added config-list.h next to it. So I'll try to fix
> it much earlier, in that commit, and then simply add to the appropriate
> variables in this commit.
> 
> > 
> > We probably clean up the build/update procedure around unicode-width.h
> > before we can do so, probably.  It may be generatable using contrib/
> > script, but as far as our normal build is concerned, it is a tracked
> > source and not part of the build byproducts, so we probably would
> > want to remove it from GEN_HDRS.  When that happens, we can $(RM)
> > all of the $(GEN_HDRS) in the "clean" target.
> 
> Noted. Thanks.

Ah, after I sent this mail I saw your patch. For those playing along at
home, https://lore.kernel.org/xmqq1rt7hkp6.fsf@gitster-ct.c.googlers.com

Will review.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 10/15] bugreport: add config values from safelist
  2019-12-16 23:40     ` Emily Shaffer
@ 2019-12-17 17:43       ` Junio C Hamano
  2020-01-24  3:29         ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2019-12-17 17:43 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

>> But that is too simple-minded.  If we wanted to safelist foo.*.bar,
>> where '*' can be anything, walking on the list of safe variables
>> would not work.  We must have a hash table that knows "foo.*.bar" is
>> allowed, and while walking all the configuration keys, when we see
>> foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if
>> it is whitelisted, or something like that.
>
> ...unless we want to use wildcards like you suggest.
>
> But I'm not sure it's a good idea. I envision someone writing another
> Git add-on, which offers someone to specify "user.password" ...

Wildcarding the leaf level of two (or for that matter three) level
names like "user.*" in your example is of course a nonsense way to
use the safelist.  But think about three-level names where the
second level is designed to be used for user-supplied token to give
things of similar nature a name the user can use to distinguish
things.  If remote.origin.url is worth reporting, wouldn't
remote.upstream.url also be?  Shouldn't remote.*.url be the way to
say "the URL field for each and every remote is a safe thing to
report" in the safelist?

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-13  0:43 ` [PATCH v4 09/15] bugreport: generate config safelist based on docs Emily Shaffer
  2019-12-13 22:57   ` Junio C Hamano
  2019-12-15 20:17   ` Johannes Schindelin
@ 2019-12-17 18:38   ` Johannes Schindelin
  2 siblings, 0 replies; 273+ messages in thread
From: Johannes Schindelin @ 2019-12-17 18:38 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Martin Ågren

Hi Emily,

On Thu, 12 Dec 2019, Emily Shaffer wrote:

> diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
> index d906a00803..750bdff9af 100644
> --- a/Documentation/asciidoctor-extensions.rb
> +++ b/Documentation/asciidoctor-extensions.rb
> @@ -37,6 +37,10 @@ module Git
>            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
>          end
>          output
> +
> +    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> +      def process(parent, action, attrs)
> +        ""

The Azure Pipeline fails on this hunk, saying:

/usr/bin/asciidoctor: /home/vsts/work/1/s/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError)
/home/vsts/work/1/s/Documentation/asciidoctor-extensions.rb:55: syntax error, unexpected end-of-input, expecting keyword_end

See
https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=logs&jobId=253e5128-1058-5bd4-fdf1-9b556d3207f8&j=253e5128-1058-5bd4-fdf1-9b556d3207f8&t=95c65aec-3627-54dc-fc17-47625f123bb3
for details.

Ciao,
Dscho

>        end
>      end
>    end
> @@ -45,4 +49,7 @@ end
>  Asciidoctor::Extensions.register do
>    inline_macro Git::Documentation::LinkGitProcessor, :linkgit
>    postprocessor Git::Documentation::DocumentPostProcessor
> +  # The bugreport macro does nothing as far as rendering is
> +  # concerned -- we just grep for it in the sources.
> +  inline_macro Git::Documentation::BugReportProcessor, :bugreport
>  end
> diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
> index 0006faf800..92f5082013 100644
> --- a/Documentation/config/sendemail.txt
> +++ b/Documentation/config/sendemail.txt
> @@ -1,63 +1,63 @@
> -sendemail.identity::
> +sendemail.identity bugreport:exclude[x] ::
>  	A configuration identity. When given, causes values in the
>  	'sendemail.<identity>' subsection to take precedence over
>  	values in the 'sendemail' section. The default identity is
>  	the value of `sendemail.identity`.
>
> -sendemail.smtpEncryption::
> +sendemail.smtpEncryption bugreport:include[x] ::
>  	See linkgit:git-send-email[1] for description.  Note that this
>  	setting is not subject to the 'identity' mechanism.
>
> -sendemail.smtpssl (deprecated)::
> +sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
>  	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
>
> -sendemail.smtpsslcertpath::
> +sendemail.smtpsslcertpath bugreport:exclude[x] ::
>  	Path to ca-certificates (either a directory or a single file).
>  	Set it to an empty string to disable certificate verification.
>
> -sendemail.<identity>.*::
> +sendemail.<identity>.* bugreport:exclude[x] ::
>  	Identity-specific versions of the 'sendemail.*' parameters
>  	found below, taking precedence over those when this
>  	identity is selected, through either the command-line or
>  	`sendemail.identity`.
>
> -sendemail.aliasesFile::
> -sendemail.aliasFileType::
> -sendemail.annotate::
> -sendemail.bcc::
> -sendemail.cc::
> -sendemail.ccCmd::
> -sendemail.chainReplyTo::
> -sendemail.confirm::
> -sendemail.envelopeSender::
> -sendemail.from::
> -sendemail.multiEdit::
> -sendemail.signedoffbycc::
> -sendemail.smtpPass::
> -sendemail.suppresscc::
> -sendemail.suppressFrom::
> -sendemail.to::
> -sendemail.tocmd::
> -sendemail.smtpDomain::
> -sendemail.smtpServer::
> -sendemail.smtpServerPort::
> -sendemail.smtpServerOption::
> -sendemail.smtpUser::
> -sendemail.thread::
> -sendemail.transferEncoding::
> -sendemail.validate::
> -sendemail.xmailer::
> +sendemail.aliasesFile bugreport:exclude[x] ::
> +sendemail.aliasFileType bugreport:exclude[x] ::
> +sendemail.annotate bugreport:include[x] ::
> +sendemail.bcc bugreport:include[x] ::
> +sendemail.cc bugreport:include[x] ::
> +sendemail.ccCmd bugreport:include[x] ::
> +sendemail.chainReplyTo bugreport:include[x] ::
> +sendemail.confirm bugreport:include[x] ::
> +sendemail.envelopeSender bugreport:include[x] ::
> +sendemail.from bugreport:include[x] ::
> +sendemail.multiEdit bugreport:include[x] ::
> +sendemail.signedoffbycc bugreport:include[x] ::
> +sendemail.smtpPass bugreport:exclude[x] ::
> +sendemail.suppresscc bugreport:include[x] ::
> +sendemail.suppressFrom bugreport:include[x] ::
> +sendemail.to bugreport:include[x] ::
> +sendemail.tocmd bugreport:include[x] ::
> +sendemail.smtpDomain bugreport:include[x] ::
> +sendemail.smtpServer bugreport:include[x] ::
> +sendemail.smtpServerPort bugreport:include[x] ::
> +sendemail.smtpServerOption bugreport:include[x] ::
> +sendemail.smtpUser bugreport:exclude[x] ::
> +sendemail.thread bugreport:include[x] ::
> +sendemail.transferEncoding bugreport:include[x] ::
> +sendemail.validate bugreport:include[x] ::
> +sendemail.xmailer bugreport:include[x] ::
>  	See linkgit:git-send-email[1] for description.
>
> -sendemail.signedoffcc (deprecated)::
> +sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
>  	Deprecated alias for `sendemail.signedoffbycc`.
>
> -sendemail.smtpBatchSize::
> +sendemail.smtpBatchSize bugreport:include[x] ::
>  	Number of messages to be sent per connection, after that a relogin
>  	will happen.  If the value is 0 or undefined, send all messages in
>  	one connection.
>  	See also the `--batch-size` option of linkgit:git-send-email[1].
>
> -sendemail.smtpReloginDelay::
> +sendemail.smtpReloginDelay bugreport:include[x] ::
>  	Seconds wait before reconnecting to smtp server.
>  	See also the `--relogin-delay` option of linkgit:git-send-email[1].
> diff --git a/Makefile b/Makefile
> index c49f55a521..76dc51e2b1 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
>  install-python-script: $(SCRIPT_PYTHON_GEN)
>  	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
>
> -.PHONY: clean-perl-script clean-sh-script clean-python-script
> +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
>  clean-sh-script:
>  	$(RM) $(SCRIPT_SH_GEN)
>  clean-perl-script:
> @@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
>
>  GENERATED_H += config-list.h
>  GENERATED_H += command-list.h
> +GENERATED_H += bugreport-config-safelist.h
>
>  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
>  	$(FIND) . \
> @@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
>  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
>  		command-list.txt >$@+ && mv $@+ $@
>
> +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
> +
> +bugreport-config-safelist.h: Documentation/config/*.txt
> +	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
> +		>$@+ && mv $@+ $@
> +
>  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
>  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
>  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
> @@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
>  .PHONY: sparse $(SP_OBJ)
>  sparse: $(SP_OBJ)
>
> -GEN_HDRS := config-list.h command-list.h unicode-width.h
> +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
>  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
>  ifndef GCRYPT_SHA256
>  	EXCEPT_HDRS += sha256/gcrypt.h
> @@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
>  	$(RM) $(HCC)
>  	$(RM) -r bin-wrappers $(dep_dirs)
>  	$(RM) -r po/build/
> -	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> +	$(RM) *.pyc *.pyo */*.pyc */*.pyo
> +	$(RM) config-list.h command-list.h bugreport-config-safelist.h
>  	$(RM) $(ETAGS_TARGET) tags cscope*
>  	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
>  	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
> diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
> new file mode 100755
> index 0000000000..06b8e0c3c4
> --- /dev/null
> +++ b/generate-bugreport-config-safelist.sh
> @@ -0,0 +1,22 @@
> +#!/bin/sh
> +
> +cat <<EOF
> +/* Automatically generated by bugreport-generate-config-safelist.sh */
> +
> +
> +static const char *bugreport_config_safelist[] = {
> +EOF
> +
> +# cat all regular files in Documentation/config
> +find Documentation/config -type f -exec cat {} \; |
> +# print the command name which matches the bugreport-include macro
> +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
> +sort |
> +while read line
> +do
> +	echo "	\"$line\","
> +done
> +
> +cat <<EOF
> +};
> +EOF
> --
> 2.24.1.735.g03f4e72817-goog
>
>
>

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 03/15] bugreport: gather git version and build info
  2019-12-13  0:43 ` [PATCH v4 03/15] bugreport: gather git version and build info Emily Shaffer
  2019-12-13 21:06   ` Junio C Hamano
@ 2019-12-17 18:45   ` Johannes Schindelin
  2019-12-17 20:34     ` Junio C Hamano
  1 sibling, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2019-12-17 18:45 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 12 Dec 2019, Emily Shaffer wrote:

> diff --git a/help.c b/help.c
> index a21487db77..a43693fca5 100644
> --- a/help.c
> +++ b/help.c
> @@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
>  	exit(1);
>  }
>
> +void list_version_info(struct strbuf *buf, int build_options)
> +{
> +	strbuf_reset(buf);
> +	/*
> +	 * The format of this string should be kept stable for compatibility
> +	 * with external projects that rely on the output of "git version".
> +	 *
> +	 * Always show the version, even if other options are given.
> +	 */
> +	strbuf_addf(buf, "git version %s\n", git_version_string);
> +
> +	if (build_options) {
> +		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
> +		if (git_built_from_commit_string[0])
> +			strbuf_addf(buf, "built from commit: %s\n",
> +			       git_built_from_commit_string);
> +		else
> +			strbuf_addf(buf, "no commit associated with this build\n");

The "StaticAnalysis" job of the Azure Pipeline is not happy with this,
claiming that this should be an `strbuf_addstr()` call instead. For
details, see:

https://dev.azure.com/gitgitgadget/8fc4a374-71dc-4558-a5ea-dd1c081ea621/_apis/build/builds/23830/logs/68

Ciao,
Dscho

> +		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
> +		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
> +		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
> +	}
> +}
> +
>  int cmd_version(int argc, const char **argv, const char *prefix)
>  {
> +	struct strbuf buf = STRBUF_INIT;
>  	int build_options = 0;
>  	const char * const usage[] = {
>  		N_("git version [<options>]"),
> @@ -637,25 +662,9 @@ int cmd_version(int argc, const char **argv, const char *prefix)
>
>  	argc = parse_options(argc, argv, prefix, options, usage, 0);
>
> -	/*
> -	 * The format of this string should be kept stable for compatibility
> -	 * with external projects that rely on the output of "git version".
> -	 *
> -	 * Always show the version, even if other options are given.
> -	 */
> -	printf("git version %s\n", git_version_string);
> +	list_version_info(&buf, build_options);
> +	printf("%s", buf.buf);
>
> -	if (build_options) {
> -		printf("cpu: %s\n", GIT_HOST_CPU);
> -		if (git_built_from_commit_string[0])
> -			printf("built from commit: %s\n",
> -			       git_built_from_commit_string);
> -		else
> -			printf("no commit associated with this build\n");
> -		printf("sizeof-long: %d\n", (int)sizeof(long));
> -		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
> -		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
> -	}
>  	return 0;
>  }
>
> diff --git a/help.h b/help.h
> index 9071894e8c..54f6b5f793 100644
> --- a/help.h
> +++ b/help.h
> @@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
>  void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
>  int is_in_cmdlist(struct cmdnames *cmds, const char *name);
>  void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
> +void list_version_info(struct strbuf *buf, int build_options);
>
>  /*
>   * call this to die(), when it is suspected that the user mistyped a
> --
> 2.24.1.735.g03f4e72817-goog
>
>
>

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 07/15] bugreport: add curl version
  2019-12-13  0:43 ` [PATCH v4 07/15] bugreport: add curl version Emily Shaffer
  2019-12-13 21:27   ` Junio C Hamano
@ 2019-12-17 18:47   ` Johannes Schindelin
  1 sibling, 0 replies; 273+ messages in thread
From: Johannes Schindelin @ 2019-12-17 18:47 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 12 Dec 2019, Emily Shaffer wrote:

> diff --git a/http-fetch.c b/http-fetch.c
> index a32ac118d9..31844812a1 100644
> --- a/http-fetch.c
> +++ b/http-fetch.c
> @@ -3,9 +3,18 @@
>  #include "exec-cmd.h"
>  #include "http.h"
>  #include "walker.h"
> +#include "version.h"
>
>  static const char http_fetch_usage[] = "git http-fetch "
> -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
> +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
> +
> +void NORETURN version_info()

Pretty much all the builds in
https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=logs&jobId=253e5128-1058-5bd4-fdf1-9b556d3207f8&j=fd490c07-0b22-5182-fac9-6d67fe1e939b&t=ce91d5d6-0c55-50f5-8ab9-6695c03ab102
fail because this function definition needs `(void)` instead of `()`.

Ciao,
Dscho

> +{
> +	printf("git-http-fetch version: %s\n", git_version_string);
> +	printf("built from commit: %s\n", git_built_from_commit_string);
> +	printf("curl version: %s\n", curl_version());
> +	exit(0);
> +}
>
>  int cmd_main(int argc, const char **argv)
>  {
> @@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
>  		} else if (argv[arg][1] == 'a') {
>  		} else if (argv[arg][1] == 'v') {
>  			get_verbosely = 1;
> +		} else if (argv[arg][1] == 'V') {
> +			version_info();
>  		} else if (argv[arg][1] == 'w') {
>  			write_ref = &argv[arg + 1];
>  			arg++;
> --
> 2.24.1.735.g03f4e72817-goog
>
>
>

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 03/15] bugreport: gather git version and build info
  2019-12-17 18:45   ` Johannes Schindelin
@ 2019-12-17 20:34     ` Junio C Hamano
  2019-12-20  1:25       ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2019-12-17 20:34 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Emily Shaffer, git

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> +	if (build_options) {
>> +		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
>> +		if (git_built_from_commit_string[0])
>> +			strbuf_addf(buf, "built from commit: %s\n",
>> +			       git_built_from_commit_string);
>> +		else
>> +			strbuf_addf(buf, "no commit associated with this build\n");
>
> The "StaticAnalysis" job of the Azure Pipeline is not happy with this,
> claiming that this should be an `strbuf_addstr()` call instead.

You mean the "else" clause, right?  That feels similar to say

	printf("Hello world\n");

should better be written as

	fputs("Hello world\n", stdout);

which I do not agree with at all.  IOW, I view the distinction more
like "once it is written one way or the other way, it is not worth
spending bits and braincycles to see if it is worth changing it"
kind of minor stylistic preference.



^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 03/15] bugreport: gather git version and build info
  2019-12-17 20:34     ` Junio C Hamano
@ 2019-12-20  1:25       ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-20  1:25 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Johannes Schindelin, git

On Tue, Dec 17, 2019 at 12:34:53PM -0800, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> +	if (build_options) {
> >> +		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
> >> +		if (git_built_from_commit_string[0])
> >> +			strbuf_addf(buf, "built from commit: %s\n",
> >> +			       git_built_from_commit_string);
> >> +		else
> >> +			strbuf_addf(buf, "no commit associated with this build\n");
> >
> > The "StaticAnalysis" job of the Azure Pipeline is not happy with this,
> > claiming that this should be an `strbuf_addstr()` call instead.
> 
> You mean the "else" clause, right?  That feels similar to say
> 
> 	printf("Hello world\n");
> 
> should better be written as
> 
> 	fputs("Hello world\n", stdout);
> 
> which I do not agree with at all.  IOW, I view the distinction more
> like "once it is written one way or the other way, it is not worth
> spending bits and braincycles to see if it is worth changing it"
> kind of minor stylistic preference.

I think I side with Junio here, although it's true that when
strbuf_addstr() exists it doesn't make that much sense to use
strbuf_addf(). Since there's some other comments, though, I'll change
this too to make your CI shut up. :)

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 03/15] bugreport: gather git version and build info
  2019-12-13 21:06   ` Junio C Hamano
@ 2019-12-20  1:46     ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2019-12-20  1:46 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 01:06:29PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Knowing which version of Git a user has and how it was built allows us
> > to more precisely pin down the circumstances when a certain issue
> > occurs, so teach bugreport how to tell us the same output as 'git
> > version --build-options'.
> >
> > It's not ideal to directly call 'git version --build-options' because
> > that output goes to stdout. Instead, wrap the version string in a helper
> > within help.[ch] library, and call that helper from within the bugreport
> > library.
> 
> Move to strbuf() from stdio makes sense.  
> 
> > +	// add other contents
> 
> Style.

Sure, dropped this entirely. I think with the helpers the code is
self-documenting there.

> 
> > diff --git a/help.h b/help.h
> > index 9071894e8c..54f6b5f793 100644
> > --- a/help.h
> > +++ b/help.h
> > @@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
> >  void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
> >  int is_in_cmdlist(struct cmdnames *cmds, const char *name);
> 
> 
> Many new helpers are called get_frotz(), but only one among these is
> called list_version_info().  I do not think the naming of the
> get_*() ones, that are private to the bugreport tool, matters that
> much, but unlike "list_commands()" whose purpose is to list the
> commands ;-), the new function does not list versions---it just
> gives information about a single version which is the one that is
> being run, so perhaps it is a misnomer.

Hm, sure. I renamed it to get_version_info(); I had named it list_*
because all the other helpers in help.h are named list_*, and it does
print more than one piece of info. But I do see your point (all the info
is about the same version) so I've renamed it.

> 
> >  void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
> > +void list_version_info(struct strbuf *buf, int build_options);
> 
> It is not clear to the readers build_options is a boolean that
> tells the function to include (or not to include) build options.
> Perhaps rename it to "int show_build_options" or something?

Agree, done.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 10/15] bugreport: add config values from safelist
  2019-12-13  0:43 ` [PATCH v4 10/15] bugreport: add config values from safelist Emily Shaffer
  2019-12-13 21:45   ` Junio C Hamano
@ 2019-12-29 20:17   ` Johannes Schindelin
  1 sibling, 0 replies; 273+ messages in thread
From: Johannes Schindelin @ 2019-12-29 20:17 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 12 Dec 2019, Emily Shaffer wrote:

> diff --git a/bugreport.c b/bugreport.c
> index 759cc0b0f8..1fca28f0b9 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -6,6 +6,9 @@
>  #include "help.h"
>  #include <gnu/libc-version.h>
>  #include "run-command.h"
> +#include "config.h"
> +#include "bugreport-config-safelist.h"
> +#include "khash.h"

Seems that this patch makes things fail in the CI build
(https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=results):

-- snipsnap --
bugreport.c:10:10: fatal error: 'bugreport-config-safelist.h' file not found
#include "bugreport-config-safelist.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
Makefile:2395: recipe for target 'bugreport.o' failed
make: *** [bugreport.o] Error 1
make: *** Waiting for unfinished jobs....


^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 05/15] bugreport: add uname info
  2019-12-13  0:43 ` [PATCH v4 05/15] bugreport: add uname info Emily Shaffer
  2019-12-13 21:12   ` Junio C Hamano
@ 2020-01-10  2:05   ` Aaron Schrab
  1 sibling, 0 replies; 273+ messages in thread
From: Aaron Schrab @ 2020-01-10  2:05 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

At 16:43 -0800 12 Dec 2019, Emily Shaffer <emilyshaffer@google.com> wrote:
>The contents of uname() can give us some insight into what sort of
>system the user is running on, and help us replicate their setup if need
>be. The domainname field is not guaranteed to be available, so don't
>collect it.

The manpage on Linux says that it's a GNU extension; on Mac OS it isn't 
mentioned. So it would be more accurate to say that it's known not to be 
available on many systems rather than just not being guaranteed.

Besides that I think in some cases it may be considered sensitive 
information, so another reason to not include it. Perhaps even worthy of 
being mentioned as the primary reason.

>+			    uname_info.nodename,

I think in some cases this could also be considered as sensitive 
information, and is unlikely to help in diagnosing problems.  So I'd 
move to exclude this as well.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v4 10/15] bugreport: add config values from safelist
  2019-12-17 17:43       ` Junio C Hamano
@ 2020-01-24  3:29         ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-01-24  3:29 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Dec 17, 2019 at 09:43:23AM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> >> But that is too simple-minded.  If we wanted to safelist foo.*.bar,
> >> where '*' can be anything, walking on the list of safe variables
> >> would not work.  We must have a hash table that knows "foo.*.bar" is
> >> allowed, and while walking all the configuration keys, when we see
> >> foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if
> >> it is whitelisted, or something like that.
> >
> > ...unless we want to use wildcards like you suggest.
> >
> > But I'm not sure it's a good idea. I envision someone writing another
> > Git add-on, which offers someone to specify "user.password" ...
> 
> Wildcarding the leaf level of two (or for that matter three) level
> names like "user.*" in your example is of course a nonsense way to
> use the safelist.  But think about three-level names where the
> second level is designed to be used for user-supplied token to give
> things of similar nature a name the user can use to distinguish
> things.  If remote.origin.url is worth reporting, wouldn't
> remote.upstream.url also be?  Shouldn't remote.*.url be the way to
> say "the URL field for each and every remote is a safe thing to
> report" in the safelist?

Bah, somehow this mail slipped through the cracks for me.

When you put it that way, I see how it can be useful. Although I admit
to fatigue on this patchset and now I'm hesitant about feature creep. :)

I'm going to send the reroll for v5 as I have it, not having seen this
mail, and think more about how I'd approach wildcarding in the middle
like this. I want to take a look at the other configs which use a middle
level as a user token and try to reason about whether we ought to be
collecting them. I'll hope to report back sooner (or later).

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* [PATCH v5 00/15] add git-bugreport tool
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (14 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 15/15] bugreport: summarize contents of alternates file Emily Shaffer
@ 2020-01-24  3:34 ` emilyshaffer
  2020-01-24  3:34   ` [PATCH v5 01/15] bugreport: add tool to generate debugging info emilyshaffer
                     ` (18 more replies)
  15 siblings, 19 replies; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Martin Ågren, Aaron Schrab

From: Emily Shaffer <emilyshaffer@google.com>

This topic branch depends on the patch mailed in
lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
display scopes for configs gathered during "bugreport: add config values from
safelist".

I'll summarize v4-v5. Since v4 has languished for some time, I don't
think an interdiff is too helpful, so I won't include one. Bonus, the
code is much simplified from some suggestions from Junio on how to
inspect objects, so I hope it's easy to review anyways.

Throughout, did some style changes away from C++ bad habits. Otherwise,
as listed:

Emily Shaffer (15):
  bugreport: add tool to generate debugging info
    (no change)
  help: move list_config_help to builtin/help
    Changed to agree with f3719846134

  bugreport: gather git version and build info
    Style only

  help: add shell-path to --build-options
    (no change)

  bugreport: add uname info
    Removed nodename for privacy reasons

  bugreport: add compiler info
    Moved glibc tattling into compat/.

    I appreciate a close look at this one - I think I understood the
    right way to go about a compat/ util but it's the first one I've
    done.

  bugreport: add curl version
    Moved curl tattling from git-http-fetch into git-remote-curl

  bugreport: include user interactive shell
    Stop depending on a compiler quirk to save us from segfault

  bugreport: generate config safelist based on docs
    Changed to agree with f3719846134
    Cleaned up script a little

  bugreport: add config values from safelist
    Made git-bugreport dependent on generated safelist header

  bugreport: collect list of populated hooks
    No change.
    Per https://lore.kernel.org/git/20191216235131.GL135450@google.com,
    should we even keep this patch?

  bugreport: count loose objects
    Use helpers from object-store.h instead of manually walking the
    filesystem.

  bugreport: add packed object summary
    Use helpers from object-store.h instead of manually walking the
    filesystem.

  bugreport: list contents of $OBJDIR/info
    (no change)

  bugreport: summarize contents of alternates file
    Rephrase commit message to explain why I can't use the helpers in
    object-store.h.


Thanks.
 - Emily


 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   8 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  68 ++--
 Documentation/git-bugreport.txt         |  43 +++
 Makefile                                |  25 +-
 bugreport.c                             | 427 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 compat/compiler.h                       |  24 ++
 generate-bugreport-config-safelist.sh   |  22 ++
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  24 ++
 help.c                                  | 133 ++------
 help.h                                  |   2 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  41 +++
 16 files changed, 780 insertions(+), 160 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh

-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 273+ messages in thread

* [PATCH v5 01/15] bugreport: add tool to generate debugging info
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:18     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 02/15] help: move list_config_help to builtin/help emilyshaffer
                     ` (17 subsequent siblings)
  18 siblings, 1 reply; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Teach Git how to prompt the user for a good bug report: reproduction
steps, expected behavior, and actual behavior. Later, Git can learn how
to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic
information we would otherwise need to ask the user for, we can reduce
the number of question-and-answer round trips between the reporter and
the Git contributor.

Users may also wish to send a report like this to their local "Git
expert" if they have put their repository into a state they are confused
by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 43 ++++++++++++++++++++
 Makefile                        |  5 +++
 bugreport.c                     | 69 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 41 ++++++++++++++++++++
 5 files changed, 159 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index aebe7c0908..ca301bc890 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..75f0c80acf
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,43 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+The following information is captured automatically:
+
+ - Git version (`git version --build-options`)
+ - Machine information (`uname -a`)
+ - Versions of various dependencies
+ - Git config contents (`git config --show-origin --list`)
+ - The names of all configured git-hooks in `.git/hooks/`
+
+OPTIONS
+-------
+-o [<path>]::
+--output [<path>]::
+	Place the resulting bug report file in <path> instead of the root of the
+	Git repository.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index 09f98b777c..f271619371 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2450,6 +2451,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..5495b31674
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,69 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output <file>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to send.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	launch_editor(report_path.buf, NULL, NULL);
+	return 0;
+}
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..6eb2ee4f66
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then
+			read -r nextline
+			if [-z $nextline]; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	check_all_headers_populated <git-bugreport-* &&
+	rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test -f foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test ! -f git-bugreport-*
+'
+
+test_done
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 02/15] help: move list_config_help to builtin/help
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
  2020-01-24  3:34   ` [PATCH v5 01/15] bugreport: add tool to generate debugging info emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:19     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 03/15] bugreport: gather git version and build info emilyshaffer
                     ` (16 subsequent siblings)
  18 siblings, 1 reply; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
builtin/clean.o, and builtin/config.o. This meant that help.o was
unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help()
into builtin/help.c (where it makes sense to assume other builtin libraries
are present).

When command-list.h is included but a member is not used, we start to
hear a compiler warning. Since the config list is generated in a fairly
different way than the command list, and since commands and config
options are semantically different, move the config list into its own
header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 121 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh

diff --git a/.gitignore b/.gitignore
index ca301bc890..d89bf9e11e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -190,6 +190,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index f271619371..a01a050aa3 100644
--- a/Makefile
+++ b/Makefile
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2129,7 +2130,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2149,6 +2150,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2786,7 +2793,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2808,7 +2815,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..eca6a00c30
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 03/15] bugreport: gather git version and build info
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
  2020-01-24  3:34   ` [PATCH v5 01/15] bugreport: add tool to generate debugging info emilyshaffer
  2020-01-24  3:34   ` [PATCH v5 02/15] help: move list_config_help to builtin/help emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:19     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 04/15] help: add shell-path to --build-options emilyshaffer
                     ` (15 subsequent siblings)
  18 siblings, 1 reply; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Knowing which version of Git a user has and how it was built allows us
to more precisely pin down the circumstances when a certain issue
occurs, so teach bugreport how to tell us the same output as 'git
version --build-options'.

It's not ideal to directly call 'git version --build-options' because
that output goes to stdout. Instead, wrap the version string in a helper
within help.[ch] library, and call that helper from within the bugreport
library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 22 +++++++++++++++++++++-
 help.c      | 47 +++++++++++++++++++++++++++++------------------
 help.h      |  1 +
 3 files changed, 51 insertions(+), 19 deletions(-)

diff --git a/bugreport.c b/bugreport.c
index 5495b31674..b1a5a279ac 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,20 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	struct strbuf version_info = STRBUF_INIT;
+
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(&version_info, 1);
+	strbuf_addbuf(sys_info, &version_info);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
@@ -32,6 +44,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -60,6 +77,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
diff --git a/help.c b/help.c
index a21487db77..156a6cdea3 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	strbuf_reset(buf);
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,25 +662,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	get_version_info(&buf, build_options);
+	printf("%s", buf.buf);
+
+	strbuf_release(&buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..500521b908 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 04/15] help: add shell-path to --build-options
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (2 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 03/15] bugreport: gather git version and build info emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:21     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 05/15] bugreport: add uname info emilyshaffer
                     ` (14 subsequent siblings)
  18 siblings, 1 reply; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

It may be useful to know which shell Git was built to try to point to,
in the event that shell-based Git commands are failing. $SHELL_PATH is
set during the build and used to launch the manpage viewer, as well as
by git-compat-util.h, and it's used during tests. 'git version
--build-options' is encouraged for use in bug reports, so it makes sense
to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/help.c b/help.c
index 156a6cdea3..d3f941824f 100644
--- a/help.c
+++ b/help.c
@@ -642,6 +642,7 @@ void get_version_info(struct strbuf *buf, int show_build_options)
 			strbuf_addstr(buf, "no commit associated with this build\n");
 		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
 		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 }
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 05/15] bugreport: add uname info
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (3 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 04/15] help: add shell-path to --build-options emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-24  3:34   ` [PATCH v5 06/15] bugreport: add compiler info emilyshaffer
                     ` (13 subsequent siblings)
  18 siblings, 0 replies; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

The contents of uname() can give us some insight into what sort of
system the user is running on, and help us replicate their setup if need
be. The domainname field is not guaranteed to be available, so don't
collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index b1a5a279ac..720c91e1bd 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -8,12 +8,24 @@
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
+	struct utsname uname_info;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(&version_info, 1);
 	strbuf_addbuf(sys_info, &version_info);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 06/15] bugreport: add compiler info
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (4 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 05/15] bugreport: add uname info emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:21     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 07/15] bugreport: add curl version emilyshaffer
                     ` (12 subsequent siblings)
  18 siblings, 1 reply; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

To help pinpoint the source of a regression, it is useful to know some
info about the compiler which the user's Git client was built with. By
adding a generic get_compiler_info() in 'compat/' we can choose which
relevant information to share per compiler; to get started, let's
demonstrate the version of glibc if the user built with 'gcc'.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c       |  5 +++++
 compat/compiler.h | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
 create mode 100644 compat/compiler.h

diff --git a/bugreport.c b/bugreport.c
index 720c91e1bd..818ccb385c 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -26,6 +27,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/compat/compiler.h b/compat/compiler.h
new file mode 100644
index 0000000000..bda5098e1b
--- /dev/null
+++ b/compat/compiler.h
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 07/15] bugreport: add curl version
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (5 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 06/15] bugreport: add compiler info emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:27     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 08/15] bugreport: include user interactive shell emilyshaffer
                     ` (11 subsequent siblings)
  18 siblings, 1 reply; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

It's possible for git-remote-curl to be built separately from git; in that
case we want to know what version of cURL is used by git-remote-curl, not
necessarily which version was present at git-bugreport's build time.
So instead, ask git-remote-curl for the version information it knows
about.

For longevity purposes, invoke the alias "git-remote-https" instead of
"git-remote-http".

Since it could have been built at a different time, also report the
version and built-from commit of git-remote-curl alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c   | 16 ++++++++++++++++
 remote-curl.c |  8 ++++++++
 2 files changed, 24 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 818ccb385c..73f6d39517 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_curl_version_info(struct strbuf *curl_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, curl_info, 0))
+	    strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_curl_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/remote-curl.c b/remote-curl.c
index 350d92a074..c590fbfae3 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 08/15] bugreport: include user interactive shell
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (6 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 07/15] bugreport: add curl version emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:28     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 09/15] bugreport: generate config safelist based on docs emilyshaffer
                     ` (10 subsequent siblings)
  18 siblings, 1 reply; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

It's possible a user may complain about the way that Git interacts with
their interactive shell, e.g. autocompletion or shell prompt. In that
case, it's useful for us to know which shell they're using
interactively.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 73f6d39517..07b84b9c94 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -22,6 +22,7 @@ static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -44,6 +45,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "(NULL)");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_curl_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (7 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 08/15] bugreport: include user interactive shell emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:34     ` Martin Ågren
  2020-01-31 21:20     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 10/15] bugreport: add config values from safelist emilyshaffer
                     ` (9 subsequent siblings)
  18 siblings, 2 replies; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Martin Ågren, Johannes Schindelin

From: Emily Shaffer <emilyshaffer@google.com>

Add a new step to the build to generate a safelist of git-config
variables which are appropriate to include in the output of
git-bugreport. New variables can be added to the safelist by annotating
their documentation in Documentation/config with the
"bugreport" macro, which is recognized by AsciiDoc and
AsciiDoctor.

Some configs are private in nature, and can contain remote URLs,
passwords, or other sensitive information. In the event that a user
doesn't notice their information while reviewing a bugreport, that user
may leak their credentials to other individuals, mailing lists, or bug
tracking tools inadvertently. Heuristic blocklisting of configuration
keys is imperfect and prone to false negatives; given the nature of the
information which can be leaked, a safelist is more reliable.

In order to prevent staleness of the safelist, add a mechanism to
generate the safelist from annotations in the config documentation,
where contributors are already used to documenting their new config
keys.

Implement a new no-op "bugreport" macro for use as
"bugreport:include[x]" to annotate the config keys that should be
included in the automatically generated safelist. Use "exclude" for the
others.

With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc
seems to want something between the brackets. A bit unfortunate, but
not a huge problem -- we'll just provide an "x".

"doc-diff" reports that this macro doesn't render at all. That is,
these are both empty after this commit:

  cd Documentation
  ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
  ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

Diffing the rendered HTML shows that there is some small amount of
whitespace and comments added. That shouldn't be a problem.

We could perhaps let the implementation verify that the "action" is one
of "include" and "exclude". For the Asciidoctor implementation that
should be straightforward, but for AsciiDoc I don't immediately know how
to do it. Anyway, if someone stumbles on the keyboard and writes
"bugreport:icndule", they'll "only" miss out on the config key being
included in the safelist. If this were a blocklist, the consequences of
a misspelled target could be a lot more severe.

Additionally, add annotations to the sendemail config documentation in
order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  8 +++
 Documentation/asciidoctor-extensions.rb |  7 +++
 Documentation/config/sendemail.txt      | 68 ++++++++++++-------------
 Makefile                                |  7 +++
 generate-bugreport-config-safelist.sh   | 17 +++++++
 6 files changed, 74 insertions(+), 34 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index d89bf9e11e..bd2f49b996 100644
--- a/.gitignore
+++ b/.gitignore
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..5d5359fcf9 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..750bdff9af 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..92f5082013 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -1,63 +1,63 @@
-sendemail.identity::
+sendemail.identity bugreport:exclude[x] ::
 	A configuration identity. When given, causes values in the
 	'sendemail.<identity>' subsection to take precedence over
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
-sendemail.smtpssl (deprecated)::
+sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
 
-sendemail.smtpsslcertpath::
+sendemail.smtpsslcertpath bugreport:exclude[x] ::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* bugreport:exclude[x] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
 	`sendemail.identity`.
 
-sendemail.aliasesFile::
-sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.aliasesFile bugreport:exclude[x] ::
+sendemail.aliasFileType bugreport:exclude[x] ::
+sendemail.annotate bugreport:include[x] ::
+sendemail.bcc bugreport:include[x] ::
+sendemail.cc bugreport:include[x] ::
+sendemail.ccCmd bugreport:include[x] ::
+sendemail.chainReplyTo bugreport:include[x] ::
+sendemail.confirm bugreport:include[x] ::
+sendemail.envelopeSender bugreport:include[x] ::
+sendemail.from bugreport:include[x] ::
+sendemail.multiEdit bugreport:include[x] ::
+sendemail.signedoffbycc bugreport:include[x] ::
+sendemail.smtpPass bugreport:exclude[x] ::
+sendemail.suppresscc bugreport:include[x] ::
+sendemail.suppressFrom bugreport:include[x] ::
+sendemail.to bugreport:include[x] ::
+sendemail.tocmd bugreport:include[x] ::
+sendemail.smtpDomain bugreport:include[x] ::
+sendemail.smtpServer bugreport:include[x] ::
+sendemail.smtpServerPort bugreport:include[x] ::
+sendemail.smtpServerOption bugreport:include[x] ::
+sendemail.smtpUser bugreport:exclude[x] ::
+sendemail.thread bugreport:include[x] ::
+sendemail.transferEncoding bugreport:include[x] ::
+sendemail.validate bugreport:include[x] ::
+sendemail.xmailer bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.
 
-sendemail.signedoffcc (deprecated)::
+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize bugreport:include[x] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay bugreport:include[x] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index a01a050aa3..2bc9f112ea 100644
--- a/Makefile
+++ b/Makefile
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..44612d5538
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/  "\1",/p' | sort
+
+cat <<EOF
+};
+EOF
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 10/15] bugreport: add config values from safelist
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (8 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 09/15] bugreport: generate config safelist based on docs emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:36     ` Martin Ågren
  2020-01-31 21:25     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 11/15] bugreport: collect list of populated hooks emilyshaffer
                     ` (8 subsequent siblings)
  18 siblings, 2 replies; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Teach bugreport to gather the values of config options which are present
in 'bugreport-config-safelist.h'.

Many config options are sensitive, and many Git add-ons use config
options which git-core does not know about; it is better only to gather
config options which we know to be safe, rather than excluding options
which we know to be unsafe.

Taking the build-time generated array and putting it into a set saves us
time - since git_config_bugreport() is called for every option the user
has configured, performing option lookup in constant time is a useful
optimization.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Makefile    |  2 +-
 bugreport.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 2bc9f112ea..eb17ece120 100644
--- a/Makefile
+++ b/Makefile
@@ -2465,7 +2465,7 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
-git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(LIBS)
 
diff --git a/bugreport.c b/bugreport.c
index 07b84b9c94..7a9fd36b60 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_curl_version_info(struct strbuf *curl_info)
 {
@@ -18,6 +21,40 @@ static void get_curl_version_info(struct strbuf *curl_info)
 	    strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
 }
 
+KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
+
+struct cfgset {
+	kh_cfg_set_t set;
+};
+
+struct cfgset safelist;
+
+static void cfgset_init(struct cfgset *set, size_t initial_size)
+{
+	memset(&set->set, 0, sizeof(set->set));
+	if (initial_size)
+		kh_resize_cfg_set(&set->set, initial_size);
+}
+
+static int cfgset_insert(struct cfgset *set, const char *cfg_key)
+{
+	int added;
+	kh_put_cfg_set(&set->set, cfg_key, &added);
+	return !added;
+}
+
+static int cfgset_contains(struct cfgset *set, const char *cfg_key)
+{
+	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
+	return pos != kh_end(&set->set);
+}
+
+static void cfgset_clear(struct cfgset *set)
+{
+	kh_release_cfg_set(&set->set);
+	cfgset_init(set, 0);
+}
+
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
@@ -54,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void gather_safelist(void)
+{
+	int index;
+	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
+	cfgset_init(&safelist, safelist_len);
+	for (index = 0; index < safelist_len; index++)
+		cfgset_insert(&safelist, bugreport_config_safelist[index]);
+
+}
+
+static int git_config_bugreport(const char *var, const char *value, void *cb)
+{
+	struct strbuf *config_info = (struct strbuf *)cb;
+
+	if (cfgset_contains(&safelist, var))
+		strbuf_addf(config_info,
+			    "%s (%s) : %s\n",
+			    var, config_scope_to_string(current_config_scope()),
+			    value);
+
+	return 0;
+}
+
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	gather_safelist();
+	git_config(git_config_bugreport, config_info);
+	cfgset_clear(&safelist);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -94,12 +161,17 @@ int cmd_main(int argc, const char **argv)
 	FILE *report;
 	time_t now = time(NULL);
 	char *option_output = NULL;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output", &option_output, N_("path"),
 			   N_("specify a destination for the bugreport file")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -118,6 +190,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 11/15] bugreport: collect list of populated hooks
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (9 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 10/15] bugreport: add config values from safelist emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-02-04 18:44     ` Junio C Hamano
  2020-01-24  3:34   ` [PATCH v5 12/15] bugreport: count loose objects emilyshaffer
                     ` (7 subsequent siblings)
  18 siblings, 1 reply; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Occasionally a failure a user is seeing may be related to a specific
hook which is being run, perhaps without the user realizing. While the
contents of hooks can be sensitive - containing user data or process
information specific to the user's organization - simply knowing that a
hook is being run at a certain stage can help us to understand whether
something is going wrong.

Without a definitive list of hook names within the code, we compile our
own list from the documentation. This is likely prone to bitrot. To
reduce the amount of code humans need to read, we turn the list into a
string_list and iterate over it (as we are calling the same find_hook
operation on each string). However, since bugreport should primarily be
called by the user, the performance loss from massaging the string
seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 7a9fd36b60..4c77009f1b 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_curl_version_info(struct strbuf *curl_info)
 {
@@ -121,6 +122,55 @@ static void get_safelisted_config(struct strbuf *config_info)
 	cfgset_clear(&safelist);
 }
 
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -193,6 +243,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Configured Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 12/15] bugreport: count loose objects
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (10 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 11/15] bugreport: collect list of populated hooks emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-02-04 18:48     ` Junio C Hamano
  2020-01-24  3:34   ` [PATCH v5 13/15] bugreport: add packed object summary emilyshaffer
                     ` (6 subsequent siblings)
  18 siblings, 1 reply; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Johannes Schindelin

From: Emily Shaffer <emilyshaffer@google.com>

The number of unpacked objects in a user's repository may help us
understand the root of the problem they're seeing, especially if a
command is running unusually slowly.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
The refactor removed much of the code Dscho suggested; and yet it
remains true that he helped me while developing this commit (although
his suggestions didn't survive). Shall I leave the Helped-by line or
remove it?

 - Emily


 bugreport.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 4c77009f1b..bf10857183 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_curl_version_info(struct strbuf *curl_info)
 {
@@ -171,6 +172,48 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info) {
+
+	int local_loose_object_count = 0, total_loose_object_count = 0;
+	int local_count_questionable = 0, total_count_questionable = 0;
+
+	local_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&local_loose_object_count,
+					FOR_EACH_OBJECT_LOCAL_ONLY);
+
+	total_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&total_loose_object_count,
+					0);
+
+	strbuf_addf(obj_info, "%d local loose objects%s\n",
+		    local_loose_object_count,
+		    local_count_questionable ? " (problem during count)" : "");
+
+	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
+		    total_loose_object_count - local_loose_object_count,
+		    (local_count_questionable || total_count_questionable)
+			? " (problem during count)"
+			: "");
+
+	strbuf_addf(obj_info, "%d total loose objects%s\n",
+		    total_loose_object_count,
+		    total_count_questionable ? " (problem during count)" : "");
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -246,6 +289,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Configured Hooks");
 	get_populated_hooks(&buffer, nongit_ok);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 13/15] bugreport: add packed object summary
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (11 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 12/15] bugreport: count loose objects emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-02-04 19:00     ` Junio C Hamano
  2020-02-04 19:03     ` Junio C Hamano
  2020-01-24  3:34   ` [PATCH v5 14/15] bugreport: list contents of $OBJDIR/info emilyshaffer
                     ` (5 subsequent siblings)
  18 siblings, 2 replies; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Alongside the loose object counts, it can be useful to show the number
of packs and packed objects. This way we can check whether the repo has
an appropriate ratio of packed to loose objects to help determine
whether it's behaving correctly.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index bf10857183..45cc1764e0 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -214,6 +214,41 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+struct packed_object_cb_data {
+	struct packed_git *last_pack;
+	int pack_count;
+	int object_count;
+};
+
+static int packed_object_cb(const struct object_id *oid,
+			    struct packed_git *pack,
+			    uint32_t pos,
+			    void *data) {
+	struct packed_object_cb_data *cb_data = data;
+
+	if (!cb_data)
+		return 1;
+
+	if (pack && pack != cb_data->last_pack) {
+		cb_data->last_pack = pack;
+		cb_data->pack_count++;
+	}
+
+	cb_data->object_count++;
+
+	return 0;
+}
+
+static void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct packed_object_cb_data cb_data = {NULL, 0, 0};
+
+	for_each_packed_object(packed_object_cb, &cb_data, 0);
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n",
+		    cb_data.pack_count, cb_data.object_count);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -292,6 +327,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 14/15] bugreport: list contents of $OBJDIR/info
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (12 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 13/15] bugreport: add packed object summary emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-24  3:34   ` [PATCH v5 15/15] bugreport: summarize contents of alternates file emilyshaffer
                     ` (4 subsequent siblings)
  18 siblings, 0 replies; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Miscellaneous information used about the object store can end up in
.git/objects/info; this can help us understand what may be going on with
the object store when the user is reporting a bug. Otherwise, it could
be difficult to track down what is going wrong with an object which
isn't kept locally to .git/objects/ or .git/objects/pack. Having some
understanding of where the user's objects may be kept can save us some
hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 45cc1764e0..12463ffdc7 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -249,6 +249,51 @@ static void get_packed_object_summary(struct strbuf *obj_info)
 		    cb_data.pack_count, cb_data.object_count);
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -330,6 +375,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v5 15/15] bugreport: summarize contents of alternates file
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (13 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 14/15] bugreport: list contents of $OBJDIR/info emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-24  3:38   ` [PATCH v5 00/15] add git-bugreport tool Emily Shaffer
                     ` (3 subsequent siblings)
  18 siblings, 0 replies; 273+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

In some cases, it could be that the user is having a problem with an
object which isn't present in their normal object directory. We can get
a hint that that might be the case by examining the list of alternates
where their object may be stored instead. Since paths to alternates may
be sensitive, we'll instead count how many alternates have been
specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this
function does not provide information on broken alternates, which are
skipped over in "link_alt_odb_entry()". Since the goal is to identify
missing alternates, we can gather the contents of
.git/objects/info/alternates manually.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 12463ffdc7..2c540182dd 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -294,6 +294,42 @@ static void get_object_info_summary(struct strbuf *obj_info)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -378,6 +414,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 00/15] add git-bugreport tool
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (14 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 15/15] bugreport: summarize contents of alternates file emilyshaffer
@ 2020-01-24  3:38   ` Emily Shaffer
  2020-01-28 23:04   ` Jonathan Tan
                     ` (2 subsequent siblings)
  18 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-01-24  3:38 UTC (permalink / raw)
  To: git
  Cc: Derrick Stolee, Johannes Schindelin, Junio C Hamano,
	Martin Ågren, Aaron Schrab

A sample bugreport generated from this patchset follows:

Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

What did you expect to happen? (Expected behavior)

What happened instead? (Actual behavior)

What's different between what you expected and what actually happened?

Anything else you want to add:

Please review the rest of the bug report below.
You can delete any lines you don't wish to send.


[System Info]
git version:
git version 2.25.0.18.g682ab0d3eb
cpu: x86_64
built from commit: 682ab0d3eb8b84f8af4db1a161d24ca53d2f39fc
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname -a: Linux 5.2.17-1rodete3-amd64 #1 SMP Debian 5.2.17-1rodete3 (2019-10-21 > 2018) x86_64
compiler info: glibc: 2.28
$SHELL (typically, interactive shell): /bin/bash
git-remote-https --build-info:
git-http-fetch version: 2.25.0.18.g682ab0d3eb
built from commit: 682ab0d3eb8b84f8af4db1a161d24ca53d2f39fc
curl version: libcurl/7.66.0 GnuTLS/3.6.9 zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.20.2 (+libidn2/2.2.0) libssh2/1.8.0 nghttp2/1.39.2 librtmp/2.3


[Safelisted Config Info]
sendemail.from (global) : emilyshaffer@google.com
sendemail.from (repo) : emilyshaffer@google.com


[Configured Hooks]
pre-commit
prepare-commit-msg


[Loose Object Counts]
2549 local loose objects
2641 alternate loose objects
5190 total loose objects


[Packed Object Summary]
60 total packs (591308 objects)


[Object Info Summary]
.git/objects/info/foo
.git/objects/info/bar
.git/objects/info/bar/baz
.git/objects/info/alternates


[Alternates]
2 alternates found (1 working, 1 broken)

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 00/15] add git-bugreport tool
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (15 preceding siblings ...)
  2020-01-24  3:38   ` [PATCH v5 00/15] add git-bugreport tool Emily Shaffer
@ 2020-01-28 23:04   ` Jonathan Tan
  2020-01-28 23:26     ` Emily Shaffer
  2020-01-30 22:15   ` Martin Ågren
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
  18 siblings, 1 reply; 273+ messages in thread
From: Jonathan Tan @ 2020-01-28 23:04 UTC (permalink / raw)
  To: emilyshaffer
  Cc: git, stolee, Johannes.Schindelin, gitster, martin.agren, aaron,
	Jonathan Tan

> From: Emily Shaffer <emilyshaffer@google.com>
> 
> This topic branch depends on the patch mailed in
> lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
> display scopes for configs gathered during "bugreport: add config values from
> safelist".
> 
> I'll summarize v4-v5. Since v4 has languished for some time, I don't
> think an interdiff is too helpful, so I won't include one. Bonus, the
> code is much simplified from some suggestions from Junio on how to
> inspect objects, so I hope it's easy to review anyways.

To everyone in the developer community interested in this set: what is
the status of this?

If this needs further review, then maybe it would be best if only
patches 1-4 were put up for submission first, with a note in the
bugreport documentation that more information may be added in future Git
versions. For me, patches 1-4 look good and I don't have enough
experience with uname (especially across libcs and OSes) to determine
what should or should not be included - if this is typical of reviewers
in the Git project, it might be better to submit patches 1-4 first, and
then send each additional diagnostic separately, so that people who know
what's going on in one area but not another can just comment on the area
they know about.

Having said that, I see that a few people have already looked at the
entire patchset and made comments, so if they are OK with it, we don't
need to split it up.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 00/15] add git-bugreport tool
  2020-01-28 23:04   ` Jonathan Tan
@ 2020-01-28 23:26     ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-01-28 23:26 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: git, stolee, Johannes.Schindelin, gitster, martin.agren, aaron

On Tue, Jan 28, 2020 at 03:04:21PM -0800, Jonathan Tan wrote:
> > From: Emily Shaffer <emilyshaffer@google.com>
> > 
> > This topic branch depends on the patch mailed in
> > lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
> > display scopes for configs gathered during "bugreport: add config values from
> > safelist".
> > 
> > I'll summarize v4-v5. Since v4 has languished for some time, I don't
> > think an interdiff is too helpful, so I won't include one. Bonus, the
> > code is much simplified from some suggestions from Junio on how to
> > inspect objects, so I hope it's easy to review anyways.
> 
> To everyone in the developer community interested in this set: what is
> the status of this?
> 
> If this needs further review, then maybe it would be best if only
> patches 1-4 were put up for submission first, with a note in the
> bugreport documentation that more information may be added in future Git
> versions. For me, patches 1-4 look good and I don't have enough
> experience with uname (especially across libcs and OSes) to determine
> what should or should not be included - if this is typical of reviewers
> in the Git project, it might be better to submit patches 1-4 first, and
> then send each additional diagnostic separately, so that people who know
> what's going on in one area but not another can just comment on the area
> they know about.
> 
> Having said that, I see that a few people have already looked at the
> entire patchset and made comments, so if they are OK with it, we don't
> need to split it up.

I'd be fine with either; from now, my gut says the only ones I'm not
comfortable merging today are maybe 10, as Junio had some concerns about
whether to allow glob expansion, and 11, as it's prone to rot and I'm
doing other work in that area which would cause 11 to be
dropped/refactored anyways once it goes in. Otherwise I think they're
all pretty OK to go in and let the other work continue later in a
different thread.

Of course, as an author my opinion that they're good to go doesn't mean
much ;)

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 00/15] add git-bugreport tool
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (16 preceding siblings ...)
  2020-01-28 23:04   ` Jonathan Tan
@ 2020-01-30 22:15   ` Martin Ågren
  2020-02-04  0:07     ` Emily Shaffer
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
  18 siblings, 1 reply; 273+ messages in thread
From: Martin Ågren @ 2020-01-30 22:15 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: Git Mailing List, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Aaron Schrab

On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> This topic branch depends on the patch mailed in
> lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
> display scopes for configs gathered during "bugreport: add config values from
> safelist".

Should this use `config_scope_name()` which looks like it's about
to graduate [1]? Disclaimer: I haven't followed that patch set too closely.

[1] https://lore.kernel.org/git/xmqqzhe66dav.fsf@gitster-ct.c.googlers.com/



Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 01/15] bugreport: add tool to generate debugging info
  2020-01-24  3:34   ` [PATCH v5 01/15] bugreport: add tool to generate debugging info emilyshaffer
@ 2020-01-30 22:18     ` Martin Ågren
  2020-02-04 22:00       ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Martin Ågren @ 2020-01-30 22:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 05:56, <emilyshaffer@google.com> wrote:
>
> From: Emily Shaffer <emilyshaffer@google.com>
>
> Teach Git how to prompt the user for a good bug report: reproduction
> steps, expected behavior, and actual behavior. Later, Git can learn how
> to collect some diagnostic information from the repository.

("Later" meaning "later in this series", or "any day now"? ;-) )

> +SYNOPSIS
> +--------
> +[verse]
> +'git bugreport' [-o | --output <path>]

Hmm. Should that be "[(-o | --output) <path>]"?

> +DESCRIPTION
> +-----------
> +Captures information about the user's machine, Git client, and repository state,
> +as well as a form requesting information about the behavior the user observed,
> +into a single text file which the user can then share, for example to the Git
> +mailing list, in order to report an observed bug.

Nice description. Got it.

> +The following information is requested from the user:
> +
> + - Reproduction steps
> + - Expected behavior
> + - Actual behavior
> +
> +The following information is captured automatically:
> +
> + - Git version (`git version --build-options`)
> + - Machine information (`uname -a`)
> + - Versions of various dependencies
> + - Git config contents (`git config --show-origin --list`)
> + - The names of all configured git-hooks in `.git/hooks/`

I would have expected these points to appear later, both to make it
clear what this does commit does (and not), and to highlight what
user-visible (documentation-worthy) changes later commits bring along.

> +OPTIONS
> +-------
> +-o [<path>]::
> +--output [<path>]::

Drop the "[" and "]"? If you give -o, you'd better give a path as well?

> +       Place the resulting bug report file in <path> instead of the root of the

`<path>`

> +"Please review the rest of the bug report below.\n"
> +"You can delete any lines you don't wish to send.\n");

"send" sounds like we're *just* about to send this report somewhere, but
it's "only" going to be written to the disk. Maybe "share", instead?

> +       if (option_output) {
> +               strbuf_addstr(&report_path, option_output);
> +               strbuf_complete(&report_path, '/');
> +       }

I thought I'd use `-o` to indicate the filename, but it turns out it's
the *directory* where the file (of some semi-random, generated name)
will end up. Re-reading the docs further up, I can see how this is
consistent. I sort of wonder if this should be `--output*-directory*`
for symmetry with `git format-patch`.

> +       strbuf_addstr(&report_path, "git-bugreport-");
> +       strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
> +       strbuf_addstr(&report_path, ".txt");
> +
> +

(Double blank line?)

> +       get_bug_template(&buffer);
> +
> +       report = fopen_for_writing(report_path.buf);

Report might be NULL here.

If there's already such a file, we overwrite. Should we generate the
filename using not just today's date (two bug reports in a day wouldn't
be unheard of?) but also something like hh:mm:ss?

> +       strbuf_write(&buffer, report);
> +       fclose(report);

Maybe clear the strbuf around here...

> +       launch_editor(report_path.buf, NULL, NULL);
> +       return 0;

... and/or UNLEAK it here, together with report_path.

Maybe "return -launch_editor(...)"?

> +#!/bin/bash

Use /bin/sh instead?

> +# Headers "[System Info]" will be followed by a non-empty line if we put some
> +# information there; we can make sure all our headers were followed by some
> +# information to check if the command was successful.
> +HEADER_PATTERN="^\[.*\]$"
> +check_all_headers_populated() {
> +       while read -r line; do
> +               if [$(grep $HEADER_PATTERN $line)]; then

I think this is a bash-ism.

> +                       read -r nextline
> +                       if [-z $nextline]; then

Likewise.

> +                               return 1;
> +                       fi
> +               fi
> +       done
> +}
> +
> +test_expect_success 'creates a report with content in the right places' '
> +       git bugreport &&
> +       check_all_headers_populated <git-bugreport-* &&
> +       rm git-bugreport-*
> +'
> +
> +test_expect_success '--output puts the report in the provided dir' '
> +       mkdir foo/ &&

If foo isn't there, do we not create it? Apparently not -- in my
testing, we segfault. (We don't check for NULL after opening the file.)

> +       git bugreport -o foo/ &&
> +       test -f foo/git-bugreport-* &&

test_path_is_file

> +       rm -fr foo/
> +'
> +
> +test_expect_success 'incorrect arguments abort with usage' '
> +       test_must_fail git bugreport --false 2>output &&
> +       grep usage output &&
> +       test ! -f git-bugreport-*

test_path_is_missing


Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 02/15] help: move list_config_help to builtin/help
  2020-01-24  3:34   ` [PATCH v5 02/15] help: move list_config_help to builtin/help emilyshaffer
@ 2020-01-30 22:19     ` Martin Ågren
  2020-02-04  0:53       ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Martin Ågren @ 2020-01-30 22:19 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> To make help.o usable by other commands again, move list_config_help()
> into builtin/help.c (where it makes sense to assume other builtin libraries
> are present).

Nit: I think this could be patch 01/15. It shuffles things around --
*then*, starting with patch 02/15, we can add exciting stuff.


Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 03/15] bugreport: gather git version and build info
  2020-01-24  3:34   ` [PATCH v5 03/15] bugreport: gather git version and build info emilyshaffer
@ 2020-01-30 22:19     ` Martin Ågren
  2020-02-04 22:21       ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Martin Ågren @ 2020-01-30 22:19 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> +static void get_system_info(struct strbuf *sys_info)
> +{
> +       struct strbuf version_info = STRBUF_INIT;
> +
> +       /* get git version from native cmd */
> +       strbuf_addstr(sys_info, "git version:\n");
> +       get_version_info(&version_info, 1);
> +       strbuf_addbuf(sys_info, &version_info);
> +       strbuf_complete_line(sys_info);

This leaks version_info.

> +}


Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 04/15] help: add shell-path to --build-options
  2020-01-24  3:34   ` [PATCH v5 04/15] help: add shell-path to --build-options emilyshaffer
@ 2020-01-30 22:21     ` Martin Ågren
  0 siblings, 0 replies; 273+ messages in thread
From: Martin Ågren @ 2020-01-30 22:21 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> It may be useful to know which shell Git was built to try to point to,
> in the event that shell-based Git commands are failing. $SHELL_PATH is
> set during the build and used to launch the manpage viewer, as well as
> by git-compat-util.h, and it's used during tests. 'git version
> --build-options' is encouraged for use in bug reports, so it makes sense
> to include this information there.

Hmm, similar nit as for patch 02/15: This could go in as, e.g., 02/15 and
your 01/15 would be 03/15: This patch would be useful even without `git
bugreport`.


Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 06/15] bugreport: add compiler info
  2020-01-24  3:34   ` [PATCH v5 06/15] bugreport: add compiler info emilyshaffer
@ 2020-01-30 22:21     ` Martin Ågren
  2020-02-04 22:51       ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Martin Ågren @ 2020-01-30 22:21 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> +#else
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +       strbuf_addstr(info, "get_compiler_info() not implemented");

Maybe "no compiler info available" (or s/ available//, or
s/available/reported/), or something else more human-readable?


> +}
> +
> +#endif
> +
> +#endif /* COMPILER_H */

Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 07/15] bugreport: add curl version
  2020-01-24  3:34   ` [PATCH v5 07/15] bugreport: add curl version emilyshaffer
@ 2020-01-30 22:27     ` Martin Ågren
  2020-02-04 22:54       ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Martin Ågren @ 2020-01-30 22:27 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> +static void get_curl_version_info(struct strbuf *curl_info)
> +{
> +       struct child_process cp = CHILD_PROCESS_INIT;
> +
> +       argv_array_push(&cp.args, "git");
> +       argv_array_push(&cp.args, "remote-https");
> +       argv_array_push(&cp.args, "--build-info");
> +       if (capture_command(&cp, curl_info, 0))
> +           strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
> +}
>
>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
>         strbuf_addstr(sys_info, "compiler info: ");
>         get_compiler_info(sys_info);
>         strbuf_complete_line(sys_info);
> +
> +       strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
> +       get_curl_version_info(sys_info);

The header here looks a lot like an implementation detail of
`get_curl_version_info()`. Or put differently, these risk getting out of
sync. Maybe frame the header a bit more human readable: "curl version".
But is this "curl version", or more like "git-remote-https version"?
There's some discrepancy here.

> +       strbuf_complete_line(sys_info);
>  }


Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 08/15] bugreport: include user interactive shell
  2020-01-24  3:34   ` [PATCH v5 08/15] bugreport: include user interactive shell emilyshaffer
@ 2020-01-30 22:28     ` Martin Ågren
  2020-02-04 23:16       ` Emily Shaffer
  2020-02-05 20:06       ` Junio C Hamano
  0 siblings, 2 replies; 273+ messages in thread
From: Martin Ågren @ 2020-01-30 22:28 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> +       char *shell = NULL;

(Unnecessary initialization.)

> +       shell = getenv("SHELL");
> +       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
> +                   shell ? shell : "(NULL)");

Thanks for avoiding a classic pitfall. :-)

"<unused>" instead of "(NULL)"? "NULL" is mostly an implementation
detail.


Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-01-24  3:34   ` [PATCH v5 09/15] bugreport: generate config safelist based on docs emilyshaffer
@ 2020-01-30 22:34     ` Martin Ågren
  2020-02-05  0:44       ` Emily Shaffer
  2020-01-31 21:20     ` Martin Ågren
  1 sibling, 1 reply; 273+ messages in thread
From: Martin Ågren @ 2020-01-30 22:34 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List, Johannes Schindelin

On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> Add a new step to the build to generate a safelist of git-config
> variables which are appropriate to include in the output of
> git-bugreport. New variables can be added to the safelist by annotating
> their documentation in Documentation/config with the
> "bugreport" macro, which is recognized by AsciiDoc and
> AsciiDoctor.
>
> Some configs are private in nature, and can contain remote URLs,
> passwords, or other sensitive information. In the event that a user
> doesn't notice their information while reviewing a bugreport, that user
> may leak their credentials to other individuals, mailing lists, or bug
> tracking tools inadvertently. Heuristic blocklisting of configuration
> keys is imperfect and prone to false negatives; given the nature of the
> information which can be leaked, a safelist is more reliable.

I sort of wonder whether safelist/blocklist is to prefer over
whitelist/blacklist, or if it's the other way round. The former are new
to me, whereas the latter are the terms I would have used. But that's
just me, of course. I was a little surprised, that's all.

> Implement a new no-op "bugreport" macro for use as
> "bugreport:include[x]" to annotate the config keys that should be
> included in the automatically generated safelist. Use "exclude" for the
> others.
>
> With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc
> seems to want something between the brackets. A bit unfortunate, but
> not a huge problem -- we'll just provide an "x".

I recognize this reasoning :-) and I'm not terribly opposed to it, but
after some nights' sleeping on this, I have to wonder if
"annotate:bugreport[include]" wouldn't be better than "bugreport[x]"
with that ugly "x". Maybe this isn't the biggest problem, but if we
expect this macro to eventually sit right next to ~90% of all our config
variables...

> "doc-diff" reports that this macro doesn't render at all. That is,
> these are both empty after this commit:
>
>   cd Documentation
>   ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
>   ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

That was true in [1], but alas, no more. In that patch, it's sort of
obvious from the diff how it adds a "class" which "end"s.

[1] https://lore.kernel.org/git/20190817203846.31609-1-martin.agren@gmail.com/

> --- a/Documentation/asciidoctor-extensions.rb
> +++ b/Documentation/asciidoctor-extensions.rb
> @@ -37,6 +37,10 @@ module Git
>            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
>          end
>          output
> +
> +    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> +      def process(parent, action, attrs)
> +        ""
>        end
>      end
>    end

But this one doesn't add an "end", and Asciidoctor trips up badly.

> +  # The bugreport macro does nothing as far as rendering is
> +  # concerned -- we just grep for it in the sources.
> +  inline_macro Git::Documentation::BugReportProcessor, :bugreport

(I never much liked this copy-paste comment then, and I still don't, to
be honest.)



Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 10/15] bugreport: add config values from safelist
  2020-01-24  3:34   ` [PATCH v5 10/15] bugreport: add config values from safelist emilyshaffer
@ 2020-01-30 22:36     ` Martin Ågren
  2020-02-05  1:34       ` Emily Shaffer
  2020-01-31 21:25     ` Martin Ågren
  1 sibling, 1 reply; 273+ messages in thread
From: Martin Ågren @ 2020-01-30 22:36 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> --- a/Makefile
> +++ b/Makefile
> @@ -2465,7 +2465,7 @@ endif
>  git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
>         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
>
> -git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
> +git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS)
>         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
>                 $(LIBS)

Haven't looked at this patch at all, except I've noticed that something
is up with the dependencies. I think bugreport.o needs to depend on
bugreport-config-safelist.h. As it is, the latter may or may not be
available as bugreport.o is built. (Reproduces fairly well for me with
something like `make clean && make -j16`.)

I'll try to continue tomorrow. :-)


Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-01-24  3:34   ` [PATCH v5 09/15] bugreport: generate config safelist based on docs emilyshaffer
  2020-01-30 22:34     ` Martin Ågren
@ 2020-01-31 21:20     ` Martin Ågren
  2020-02-05  0:30       ` Emily Shaffer
  1 sibling, 1 reply; 273+ messages in thread
From: Martin Ågren @ 2020-01-31 21:20 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List, Johannes Schindelin

On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> Add a new step to the build to generate a safelist of git-config
> variables which are appropriate to include in the output of
> git-bugreport. New variables can be added to the safelist by annotating
> their documentation in Documentation/config with the
> "bugreport" macro, which is recognized by AsciiDoc and
> AsciiDoctor.

"which is recognized by" sounds like it's built-in. Maybe "with a new
no-op 'bugreport' macro" or something like that.

> -sendemail.signedoffcc (deprecated)::
> +sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
>         Deprecated alias for `sendemail.signedoffbycc`.
>
> -sendemail.smtpBatchSize::
> +sendemail.smtpBatchSize bugreport:include[x] ::
>         Number of messages to be sent per connection, after that a relogin
>         will happen.  If the value is 0 or undefined, send all messages in
>         one connection.
>         See also the `--batch-size` option of linkgit:git-send-email[1].

Do we really need to list includes *and* excludes? I could see the point
of adding an exclude here and there to signal that "this might look
innocent enough, but trust me, we really need to exclude this" in order
to avoid future commits to more or less accidentally over-include.
Should we add some internal documentation and/or a remark in the commit
message about this? As an example, is "sendemail.signedofcc" sensitive
enough that we need to explicitly exclude it? If someone wants to come
along and include it, they don't just need to argue for an inclusion,
but also for lifting the exclusion. Hmm?

Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 10/15] bugreport: add config values from safelist
  2020-01-24  3:34   ` [PATCH v5 10/15] bugreport: add config values from safelist emilyshaffer
  2020-01-30 22:36     ` Martin Ågren
@ 2020-01-31 21:25     ` Martin Ågren
  2020-02-05  2:31       ` Emily Shaffer
  1 sibling, 1 reply; 273+ messages in thread
From: Martin Ågren @ 2020-01-31 21:25 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> Taking the build-time generated array and putting it into a set saves us
> time - since git_config_bugreport() is called for every option the user
> has configured, performing option lookup in constant time is a useful
> optimization.

I'm sympathetic to your sending out what you have to obtain comments,
knowing that it's not perfect. It would have saved me some time and
effort if I'd known that this was the case though. An "[RFC]" tag,
perhaps. Or at least tweaking the above part of this commit message to
say that this might be over-engineered, with a reference to [1].

[1] https://lore.kernel.org/git/20200124032905.GA37541@google.com/

> +       int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);

I was going to suggest ARRAY_SIZE, but then I realized there are some
outstanding questions around whether you need this stuff in the first
place. I'd be inclined to guess that the first version of this would be
"for each safelisted item, obtain it and include", ignoring any "a.*.b"
business. In which case you wouldn't really need this hashset stuff.


Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 00/15] add git-bugreport tool
  2020-01-30 22:15   ` Martin Ågren
@ 2020-02-04  0:07     ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-04  0:07 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Git Mailing List, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Aaron Schrab

On Thu, Jan 30, 2020 at 11:15:25PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> > This topic branch depends on the patch mailed in
> > lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
> > display scopes for configs gathered during "bugreport: add config values from
> > safelist".
> 
> Should this use `config_scope_name()` which looks like it's about
> to graduate [1]? Disclaimer: I haven't followed that patch set too closely.

Yeah, you're right that I can use it. config.h:config_scope_name() is a
new feature on that patch since last time I looked. Thanks for the tip!

 - Emily

> [1] https://lore.kernel.org/git/xmqqzhe66dav.fsf@gitster-ct.c.googlers.com/

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 02/15] help: move list_config_help to builtin/help
  2020-01-30 22:19     ` Martin Ågren
@ 2020-02-04  0:53       ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-04  0:53 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:19:26PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > To make help.o usable by other commands again, move list_config_help()
> > into builtin/help.c (where it makes sense to assume other builtin libraries
> > are present).
> 
> Nit: I think this could be patch 01/15. It shuffles things around --
> *then*, starting with patch 02/15, we can add exciting stuff.

Sounds reasonable, here and 04/15. Will do.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 11/15] bugreport: collect list of populated hooks
  2020-01-24  3:34   ` [PATCH v5 11/15] bugreport: collect list of populated hooks emilyshaffer
@ 2020-02-04 18:44     ` Junio C Hamano
  2020-02-05  2:48       ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-04 18:44 UTC (permalink / raw)
  To: emilyshaffer; +Cc: git

emilyshaffer@google.com writes:

> +	/*
> +	 * Doesn't look like there is a list of all possible hooks; so below is
> +	 * a transcription of `git help hook`.

That's "git help hooks", if I tried my reproduction correctly.  A
straight-forward (in the sense of "what we want in the outcome is
quite clear" and not in the sense of "anybody can design and
implement it with a single 30-line patch") follow-up we can make
after this series lands is to rethink how Documentation/githooks.txt
is maintained and the list we have here is synchronized with it.

The design could me just the matter of running "grep" of some sort,
with appropriate markups that are no-op to AsciiDoctor/AsciiDoc
added to the documentation source, to produce this list.

> +	 */
> +	const char *hooks = "applypatch-msg,"
> +			    "pre-applypatch,"
> +			    "post-applypatch,"
> +			    "pre-commit,"
> +			    "pre-merge-commit,"
> +			    "prepare-commit-msg,"
> +			    "commit-msg,"
> +			    "post-commit,"
> +			    "pre-rebase,"
> +			    "post-checkout,"
> +			    "post-merge,"
> +			    "pre-push,"
> +			    "pre-receive,"
> +			    "update,"
> +			    "post-receive,"
> +			    "post-update,"
> +			    "push-to-checkout,"
> +			    "pre-auto-gc,"
> +			    "post-rewrite,"
> +			    "sendemail-validate,"
> +			    "fsmonitor-watchman,"
> +			    "p4-pre-submit,"
> +			    "post-index-changex";
> +	struct string_list hooks_list = STRING_LIST_INIT_DUP;
> +	struct string_list_item *iter = NULL;
> +
> +
> +	if (nongit) {
> +		strbuf_addstr(hook_info,
> +			"not run from a git repository - no hooks to show\n");
> +		return;
> +	}
> +
> +	string_list_split(&hooks_list, hooks, ',', -1);
> +
> +	for_each_string_list_item(iter, &hooks_list) {
> +		if (find_hook(iter->string)) {
> +			strbuf_addstr(hook_info, iter->string);
> +			strbuf_complete_line(hook_info);
> +		}
> +	}
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output <file>]"),
>  	NULL
> @@ -193,6 +243,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Safelisted Config Info");
>  	get_safelisted_config(&buffer);
>  
> +	get_header(&buffer, "Configured Hooks");

Phrase nit.  There may be many people who just enabled hooks without
configuring, so "Enabled Hooks" may be more appropriate.  We do not
have to inspect what is in the hook to determine if it is enabled,
but we do need to if we want to tell if a hook is "configured".

> +	get_populated_hooks(&buffer, nongit_ok);
> +
>  	report = fopen_for_writing(report_path.buf);
>  	strbuf_write(&buffer, report);
>  	fclose(report);

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 12/15] bugreport: count loose objects
  2020-01-24  3:34   ` [PATCH v5 12/15] bugreport: count loose objects emilyshaffer
@ 2020-02-04 18:48     ` Junio C Hamano
  2020-02-05  2:50       ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-04 18:48 UTC (permalink / raw)
  To: emilyshaffer; +Cc: git, Johannes Schindelin

emilyshaffer@google.com writes:

> From: Emily Shaffer <emilyshaffer@google.com>
>
> The number of unpacked objects in a user's repository may help us
> understand the root of the problem they're seeing, especially if a
> command is running unusually slowly.
>
> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
> The refactor removed much of the code Dscho suggested; and yet it
> remains true that he helped me while developing this commit (although
> his suggestions didn't survive). Shall I leave the Helped-by line or
> remove it?

You two collectively thought about viable alternatives and decided
to reject what was not wanted in the final result, and not having
that rejected code was good for the project, right?  If so, I would
say it still is the help that deserves recognition.  After all,
making the result better by removing things is harder than by adding
things ;-)

>  bugreport.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 46 insertions(+)
> ...

The patch text looked sensible.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 13/15] bugreport: add packed object summary
  2020-01-24  3:34   ` [PATCH v5 13/15] bugreport: add packed object summary emilyshaffer
@ 2020-02-04 19:00     ` Junio C Hamano
  2020-02-05  3:15       ` Emily Shaffer
  2020-02-04 19:03     ` Junio C Hamano
  1 sibling, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-04 19:00 UTC (permalink / raw)
  To: emilyshaffer; +Cc: git

emilyshaffer@google.com writes:

> From: Emily Shaffer <emilyshaffer@google.com>
>
> Alongside the loose object counts, it can be useful to show the number
> of packs and packed objects. This way we can check whether the repo has
> an appropriate ratio of packed to loose objects to help determine
> whether it's behaving correctly.

This lets the enumeration machinery to enumerate at the individual
object level, relies on the enumeration machinery to show your
callback all objects from the same pack before an object from a
different pack, to count both objects and packs.

But given that packfiles record how many objects there are in the
pack, and that the packfile.c layer must know what are the pack
files we have available, I wonder if we have an API already to allow
us to enumerate each packfile, and while counting the number of
times our callback was called, add the objects contained in each of
them.  If there isn't, it does not sound like a bad API function to
add (i.e. given a repository r, iterate over r->objects->packed_git
and let the API caller see each of them; the API caller can grab
num_objects and add them up).


^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 13/15] bugreport: add packed object summary
  2020-01-24  3:34   ` [PATCH v5 13/15] bugreport: add packed object summary emilyshaffer
  2020-02-04 19:00     ` Junio C Hamano
@ 2020-02-04 19:03     ` Junio C Hamano
  2020-02-05  3:09       ` Emily Shaffer
  1 sibling, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-04 19:03 UTC (permalink / raw)
  To: emilyshaffer; +Cc: git, Derrick Stolee

emilyshaffer@google.com writes:

> From: Emily Shaffer <emilyshaffer@google.com>
>
> Alongside the loose object counts, it can be useful to show the number
> of packs and packed objects. This way we can check whether the repo has
> an appropriate ratio of packed to loose objects to help determine
> whether it's behaving correctly.

This step makes me wonder if we want to see the midx as well.
Didn't we have a bug that manifests only when midx is (or is not) in
use?

Similarly, I think use (or non-use) of the commit graph may also be
a useful information for diagnosing.




^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 01/15] bugreport: add tool to generate debugging info
  2020-01-30 22:18     ` Martin Ågren
@ 2020-02-04 22:00       ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-04 22:00 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:18:55PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 05:56, <emilyshaffer@google.com> wrote:
> >
> > From: Emily Shaffer <emilyshaffer@google.com>
> >
> > Teach Git how to prompt the user for a good bug report: reproduction
> > steps, expected behavior, and actual behavior. Later, Git can learn how
> > to collect some diagnostic information from the repository.
> 
> ("Later" meaning "later in this series", or "any day now"? ;-) )
> 
> > +SYNOPSIS
> > +--------
> > +[verse]
> > +'git bugreport' [-o | --output <path>]
> 
> Hmm. Should that be "[(-o | --output) <path>]"?

Done.

> > +DESCRIPTION
> > +-----------
> > +Captures information about the user's machine, Git client, and repository state,
> > +as well as a form requesting information about the behavior the user observed,
> > +into a single text file which the user can then share, for example to the Git
> > +mailing list, in order to report an observed bug.
> 
> Nice description. Got it.
> 
> > +The following information is requested from the user:
> > +
> > + - Reproduction steps
> > + - Expected behavior
> > + - Actual behavior
> > +
> > +The following information is captured automatically:
> > +
> > + - Git version (`git version --build-options`)
> > + - Machine information (`uname -a`)
> > + - Versions of various dependencies
> > + - Git config contents (`git config --show-origin --list`)
> > + - The names of all configured git-hooks in `.git/hooks/`
> 
> I would have expected these points to appear later, both to make it
> clear what this does commit does (and not), and to highlight what
> user-visible (documentation-worthy) changes later commits bring along.

Sure, agreed.

> 
> > +OPTIONS
> > +-------
> > +-o [<path>]::
> > +--output [<path>]::
> 
> Drop the "[" and "]"? If you give -o, you'd better give a path as well?

Done.

> > +       Place the resulting bug report file in <path> instead of the root of the
> 
> `<path>`

Done.

> > +"Please review the rest of the bug report below.\n"
> > +"You can delete any lines you don't wish to send.\n");
> 
> "send" sounds like we're *just* about to send this report somewhere, but
> it's "only" going to be written to the disk. Maybe "share", instead?

Nice turn of phrase, done.

> > +       if (option_output) {
> > +               strbuf_addstr(&report_path, option_output);
> > +               strbuf_complete(&report_path, '/');
> > +       }
> 
> I thought I'd use `-o` to indicate the filename, but it turns out it's
> the *directory* where the file (of some semi-random, generated name)
> will end up. Re-reading the docs further up, I can see how this is
> consistent. I sort of wonder if this should be `--output*-directory*`
> for symmetry with `git format-patch`.

Sure.

> > +       strbuf_addstr(&report_path, "git-bugreport-");
> > +       strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
> > +       strbuf_addstr(&report_path, ".txt");
> > +
> > +
> 
> (Double blank line?)

Done.
 
> > +       get_bug_template(&buffer);
> > +
> > +       report = fopen_for_writing(report_path.buf);
> 
> Report might be NULL here.

Nice.

> If there's already such a file, we overwrite. Should we generate the
> filename using not just today's date (two bug reports in a day wouldn't
> be unheard of?) but also something like hh:mm:ss?

Sure. For the sake of brevity I'll probably neglect seconds; I hope
someone is spending more than 1 minute filling in the provided form.

I'm a little worried about including : in a filename, so I went for
'git-bugreport-YYYY-MM-DD-HHMM' (24-hour).

As I started to write a test to ensure duplicate filenames were handled
well, Jonathan Tan pointed out that it would be easy to add an arg like
--suffix to allow specifying a custom strftime string. That would allow
users to easily create a file named
`git-bugreport-fetch-failing.txt` or `git-bugreport-March-19.txt` or
whatever they want; it also makes testing easy. So I'll add this for the
next rollup.

> 
> > +       strbuf_write(&buffer, report);
> > +       fclose(report);
> 
> Maybe clear the strbuf around here...
> 
> > +       launch_editor(report_path.buf, NULL, NULL);
> > +       return 0;
> 
> ... and/or UNLEAK it here, together with report_path.
> 
> Maybe "return -launch_editor(...)"?

Hm, sure. I see that builtin/tag.c does mark strbufs this way, so I
don't see a problem using UNLEAK and tail-calling launch_editor().


As a final bonus, I also added a line to report to stderr the name of
the file that was created. I noticed it's sort of unclear what the
command actually did otherwise.

> > +#!/bin/bash
> 
> Use /bin/sh instead?

Yeah, doing so immediately pointed out the bashisms you mentioned, plus
some more. :facepalm:

> 
> > +# Headers "[System Info]" will be followed by a non-empty line if we put some
> > +# information there; we can make sure all our headers were followed by some
> > +# information to check if the command was successful.
> > +HEADER_PATTERN="^\[.*\]$"
> > +check_all_headers_populated() {
> > +       while read -r line; do
> > +               if [$(grep $HEADER_PATTERN $line)]; then
> 
> I think this is a bash-ism.
> 
> > +                       read -r nextline
> > +                       if [-z $nextline]; then
> 
> Likewise.
> 
> > +                               return 1;
> > +                       fi
> > +               fi
> > +       done
> > +}
> > +
> > +test_expect_success 'creates a report with content in the right places' '
> > +       git bugreport &&
> > +       check_all_headers_populated <git-bugreport-* &&
> > +       rm git-bugreport-*
> > +'
> > +
> > +test_expect_success '--output puts the report in the provided dir' '
> > +       mkdir foo/ &&
> 
> If foo isn't there, do we not create it? Apparently not -- in my
> testing, we segfault. (We don't check for NULL after opening the file.)

Yeah, at the moment I just added a die() if we can't open the provided
path. I think other utilties can create the path (e.g. git-format-patch)
but where it makes sense to do so, I'd prefer to keep bugreport very
simple.

I'll die instead of overwriting, too; you're right that spending quite a
while on a bug report and then accidentally writing over it with a blank
one would be a very bad user experience.

> 
> > +       git bugreport -o foo/ &&
> > +       test -f foo/git-bugreport-* &&
> 
> test_path_is_file

Sure.

> > +       rm -fr foo/
> > +'
> > +
> > +test_expect_success 'incorrect arguments abort with usage' '
> > +       test_must_fail git bugreport --false 2>output &&
> > +       grep usage output &&
> > +       test ! -f git-bugreport-*
> 
> test_path_is_missing

OK.

Thanks very much, Martin, for the thorough review. This is incredibly
helpful.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 03/15] bugreport: gather git version and build info
  2020-01-30 22:19     ` Martin Ågren
@ 2020-02-04 22:21       ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-04 22:21 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:19:55PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > +static void get_system_info(struct strbuf *sys_info)
> > +{
> > +       struct strbuf version_info = STRBUF_INIT;
> > +
> > +       /* get git version from native cmd */
> > +       strbuf_addstr(sys_info, "git version:\n");
> > +       get_version_info(&version_info, 1);
> > +       strbuf_addbuf(sys_info, &version_info);
> > +       strbuf_complete_line(sys_info);
> 
> This leaks version_info.

So it does.

I think the easiest thing is to drop the 'strbuf_reset()' at the top of
help.c:get_version_info(), and just pass in sys_info from bugreport.c.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 06/15] bugreport: add compiler info
  2020-01-30 22:21     ` Martin Ågren
@ 2020-02-04 22:51       ` Emily Shaffer
  2020-02-05 19:47         ` Martin Ågren
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-04 22:51 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:21:39PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > +#else
> > +
> > +static inline void get_compiler_info(struct strbuf *info)
> > +{
> > +       strbuf_addstr(info, "get_compiler_info() not implemented");
> 
> Maybe "no compiler info available" (or s/ available//, or
> s/available/reported/), or something else more human-readable?

Hm. I envisioned the target audience for this as "Git developers" (and
that's why I didn't mark the unimplemented string for translation); Git
developers know exactly what "get_compiler_info() not implemented"
means. But I suppose it's just as easy to grep for one string as the
other.

I am hesitant to say "no info available" or "no info reported" when I'm
certain there is some environment variable set at build time which
contains the info we'd want to show here; it really is just a question
of some Git developer having put in the time to implement this function.

So, I think I will leave it. Thanks for your suggestions.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 07/15] bugreport: add curl version
  2020-01-30 22:27     ` Martin Ågren
@ 2020-02-04 22:54       ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-04 22:54 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:27:45PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> > +static void get_curl_version_info(struct strbuf *curl_info)
> > +{
> > +       struct child_process cp = CHILD_PROCESS_INIT;
> > +
> > +       argv_array_push(&cp.args, "git");
> > +       argv_array_push(&cp.args, "remote-https");
> > +       argv_array_push(&cp.args, "--build-info");
> > +       if (capture_command(&cp, curl_info, 0))
> > +           strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
> > +}
> >
> >  static void get_system_info(struct strbuf *sys_info)
> >  {
> > @@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
> >         strbuf_addstr(sys_info, "compiler info: ");
> >         get_compiler_info(sys_info);
> >         strbuf_complete_line(sys_info);
> > +
> > +       strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
> > +       get_curl_version_info(sys_info);
> 
> The header here looks a lot like an implementation detail of
> `get_curl_version_info()`. Or put differently, these risk getting out of
> sync. Maybe frame the header a bit more human readable: "curl version".
> But is this "curl version", or more like "git-remote-https version"?
> There's some discrepancy here.

Hm, I think you're saying "If we switch to future-https-lib instead of
cURL for git-remote-https, then this command will be incorrectly named."
Sure, I agree. It's true that with this change git-remote-https also
tells us some info about itself.

Thanks.
 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 08/15] bugreport: include user interactive shell
  2020-01-30 22:28     ` Martin Ågren
@ 2020-02-04 23:16       ` Emily Shaffer
  2020-02-05 20:06       ` Junio C Hamano
  1 sibling, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-04 23:16 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:28:40PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> > +       char *shell = NULL;
> 
> (Unnecessary initialization.)
> 
> > +       shell = getenv("SHELL");
> > +       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
> > +                   shell ? shell : "(NULL)");
> 
> Thanks for avoiding a classic pitfall. :-)

Thank Junio. I fell right into it because it Just Works in gcc. ;)

> "<unused>" instead of "(NULL)"? "NULL" is mostly an implementation
> detail.

Sure, why not.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-01-31 21:20     ` Martin Ågren
@ 2020-02-05  0:30       ` Emily Shaffer
  2020-02-05  0:52         ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-05  0:30 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List, Johannes Schindelin

On Fri, Jan 31, 2020 at 10:20:05PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> > Add a new step to the build to generate a safelist of git-config
> > variables which are appropriate to include in the output of
> > git-bugreport. New variables can be added to the safelist by annotating
> > their documentation in Documentation/config with the
> > "bugreport" macro, which is recognized by AsciiDoc and
> > AsciiDoctor.
> 
> "which is recognized by" sounds like it's built-in. Maybe "with a new
> no-op 'bugreport' macro" or something like that.

Sure, I'll wordsmith this a little.

> 
> > -sendemail.signedoffcc (deprecated)::
> > +sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
> >         Deprecated alias for `sendemail.signedoffbycc`.
> >
> > -sendemail.smtpBatchSize::
> > +sendemail.smtpBatchSize bugreport:include[x] ::
> >         Number of messages to be sent per connection, after that a relogin
> >         will happen.  If the value is 0 or undefined, send all messages in
> >         one connection.
> >         See also the `--batch-size` option of linkgit:git-send-email[1].
> 
> Do we really need to list includes *and* excludes?

Need? Nah. The exclude markers are not even acknowledged by the current
implementation.

> I could see the point of adding an exclude here and there to signal
> that "this might look innocent enough, but trust me, we really need to
> exclude this" in order to avoid future commits to more or less
> accidentally over-include.

This was more of my line of thinking, along with the thought that some
folks may use bug reporting internally, where they have A) more control
over which extensions/tools are in use, and B) less concern about
sharing things like remote URLs, branch names, etc. and may want to go
for a blocklist as opposed to a safelist, to make it easier for their
support folks to diagnose.

> Should we add some internal documentation and/or a remark in the commit
> message about this? As an example, is "sendemail.signedofcc" sensitive
> enough that we need to explicitly exclude it? If someone wants to come
> along and include it, they don't just need to argue for an inclusion,
> but also for lifting the exclusion. Hmm?

Yeah, I think this is a good point. I'll try to figure out how to reword
the commit message, and take another look at the sample include/exclude
change I made to see where I can omit entirely.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-01-30 22:34     ` Martin Ågren
@ 2020-02-05  0:44       ` Emily Shaffer
  2020-02-05 19:53         ` Martin Ågren
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-05  0:44 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List, Johannes Schindelin

On Thu, Jan 30, 2020 at 11:34:24PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> > Add a new step to the build to generate a safelist of git-config
> > variables which are appropriate to include in the output of
> > git-bugreport. New variables can be added to the safelist by annotating
> > their documentation in Documentation/config with the
> > "bugreport" macro, which is recognized by AsciiDoc and
> > AsciiDoctor.
> >
> > Some configs are private in nature, and can contain remote URLs,
> > passwords, or other sensitive information. In the event that a user
> > doesn't notice their information while reviewing a bugreport, that user
> > may leak their credentials to other individuals, mailing lists, or bug
> > tracking tools inadvertently. Heuristic blocklisting of configuration
> > keys is imperfect and prone to false negatives; given the nature of the
> > information which can be leaked, a safelist is more reliable.
> 
> I sort of wonder whether safelist/blocklist is to prefer over
> whitelist/blacklist, or if it's the other way round. The former are new
> to me, whereas the latter are the terms I would have used. But that's
> just me, of course. I was a little surprised, that's all.

Eh. I think the following things are true:

 - Whitelist/blacklist has a "smell" of discrimination, whether that's
   the true etymology or not.
 - Those with experience in the field can easily understand what
   whitelist or blacklist means.
 - Safelist/blocklist do not "smell" the same way.
 - It is easy to tell what "safelist" means: "a list of stuff which is
   safe." No experience needed.

So, while it's new, I think it's not harmful. I see only a no-op or
positive impact from using this term instead of whitelist/blacklist.
Computer science seems to have quite a few terms which fall into this
long-standing but potentially negative area, so I don't mind looking for
alternatives where it's harmless to do so.

> 
> > Implement a new no-op "bugreport" macro for use as
> > "bugreport:include[x]" to annotate the config keys that should be
> > included in the automatically generated safelist. Use "exclude" for the
> > others.
> >
> > With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc
> > seems to want something between the brackets. A bit unfortunate, but
> > not a huge problem -- we'll just provide an "x".
> 
> I recognize this reasoning :-) and I'm not terribly opposed to it, but
> after some nights' sleeping on this, I have to wonder if
> "annotate:bugreport[include]" wouldn't be better than "bugreport[x]"
> with that ugly "x". Maybe this isn't the biggest problem, but if we
> expect this macro to eventually sit right next to ~90% of all our config
> variables...

Hm. I wanted to say, "Ok, but I don't know how to do that, so can you
help?" But I think that's all the more reason that I should do it ;)

Ok. I will try and change it to annotate:bugreport[include] like you
suggested, and hopefully learn more about asciidoc macros :)

> 
> > "doc-diff" reports that this macro doesn't render at all. That is,
> > these are both empty after this commit:
> >
> >   cd Documentation
> >   ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
> >   ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD
> 
> That was true in [1], but alas, no more. In that patch, it's sort of
> obvious from the diff how it adds a "class" which "end"s.
> 
> [1] https://lore.kernel.org/git/20190817203846.31609-1-martin.agren@gmail.com/
> 
> > --- a/Documentation/asciidoctor-extensions.rb
> > +++ b/Documentation/asciidoctor-extensions.rb
> > @@ -37,6 +37,10 @@ module Git
> >            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
> >          end
> >          output
> > +
> > +    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> > +      def process(parent, action, attrs)
> > +        ""
> >        end
> >      end
> >    end
> 
> But this one doesn't add an "end", and Asciidoctor trips up badly.

Ok, I'll have a look. I'm sure I copied something badly.

> 
> > +  # The bugreport macro does nothing as far as rendering is
> > +  # concerned -- we just grep for it in the sources.
> > +  inline_macro Git::Documentation::BugReportProcessor, :bugreport
> 
> (I never much liked this copy-paste comment then, and I still don't, to
> be honest.)

I'll see if I can find a reasonable alternative (or remove it).

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-02-05  0:30       ` Emily Shaffer
@ 2020-02-05  0:52         ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-05  0:52 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List, Johannes Schindelin

On Tue, Feb 04, 2020 at 04:30:43PM -0800, Emily Shaffer wrote:
> Yeah, I think this is a good point. I'll try to figure out how to reword
> the commit message, and take another look at the sample include/exclude
> change I made to see where I can omit entirely.

Eh. I'm going to try and reword the whole commit message. I combined
your scissors patch, plus something from Dscho's scissors patch, plus my
original commit message, and now it's huge and hard to follow.


 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 10/15] bugreport: add config values from safelist
  2020-01-30 22:36     ` Martin Ågren
@ 2020-02-05  1:34       ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-05  1:34 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:36:43PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -2465,7 +2465,7 @@ endif
> >  git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
> >         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
> >
> > -git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
> > +git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS)
> >         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
> >                 $(LIBS)
> 
> Haven't looked at this patch at all, except I've noticed that something
> is up with the dependencies. I think bugreport.o needs to depend on
> bugreport-config-safelist.h. As it is, the latter may or may not be
> available as bugreport.o is built. (Reproduces fairly well for me with
> something like `make clean && make -j16`.)

Hum, I thought that the ordering of the dependency list here was
ensured. But I see where help.o relies directly on command-list.h, so
I'll add a dependency for bugreport.o (and drop it from the direct
dependency for git-bugreport).

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 10/15] bugreport: add config values from safelist
  2020-01-31 21:25     ` Martin Ågren
@ 2020-02-05  2:31       ` Emily Shaffer
  2020-02-05 20:12         ` Martin Ågren
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-05  2:31 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Fri, Jan 31, 2020 at 10:25:06PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > Taking the build-time generated array and putting it into a set saves us
> > time - since git_config_bugreport() is called for every option the user
> > has configured, performing option lookup in constant time is a useful
> > optimization.
> 
> I'm sympathetic to your sending out what you have to obtain comments,
> knowing that it's not perfect. It would have saved me some time and
> effort if I'd known that this was the case though. An "[RFC]" tag,
> perhaps. Or at least tweaking the above part of this commit message to
> say that this might be over-engineered, with a reference to [1].
> 
> [1] https://lore.kernel.org/git/20200124032905.GA37541@google.com/

Yeah, you are right, and thanks for the feedback. I think I had wrapped
up the series before I realized there were those significant outstanding
comments, but I definitely should have said so in this patch.

> 
> > +       int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
> 
> I was going to suggest ARRAY_SIZE, but then I realized there are some
> outstanding questions around whether you need this stuff in the first
> place. I'd be inclined to guess that the first version of this would be
> "for each safelisted item, obtain it and include", ignoring any "a.*.b"
> business. In which case you wouldn't really need this hashset stuff.

Regardless, I think it's worth doing nicely for now.

It seems to me that supporting wildcarding in the config safelist is a
superset of supporting static strings in that safelist - that is, if I
write it simply to support static strings, a later change to support
wildcards would be welcome and non-breaking.

So I'm going to clean this up without making it more featureful, for
now.

Sorry about the confusion!
 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 11/15] bugreport: collect list of populated hooks
  2020-02-04 18:44     ` Junio C Hamano
@ 2020-02-05  2:48       ` Emily Shaffer
  2020-02-05  3:00         ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-05  2:48 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Feb 04, 2020 at 10:44:39AM -0800, Junio C Hamano wrote:
> emilyshaffer@google.com writes:
> 
> > +	/*
> > +	 * Doesn't look like there is a list of all possible hooks; so below is
> > +	 * a transcription of `git help hook`.
> 
> That's "git help hooks", if I tried my reproduction correctly.

Yep. I'll fix it.

> A straight-forward (in the sense of "what we want in the outcome is
> quite clear" and not in the sense of "anybody can design and implement
> it with a single 30-line patch") follow-up we can make after this
> series lands is to rethink how Documentation/githooks.txt is
> maintained and the list we have here is synchronized with it.
> 
> The design could me just the matter of running "grep" of some sort,
> with appropriate markups that are no-op to AsciiDoctor/AsciiDoc
> added to the documentation source, to produce this list.

Well, with Martin's suggestion[1] to use annotate:bugreport[include]
instead of bugreport:include[x], this kind of change becomes a matter of
course to just use "annotate:hook[x]" and write a script nearly
identical to generate-bugreport-config-safelist.sh, and add it to the
Makefile.

Like I said up-thread, I don't like the idea just because it's awfully
temporary (pending work that's right below this on my list for the
coming weeks).

It sounds like you're saying you don't mind any work of this nature
being done as a follow-up - and that's fine with me, too. I'd just
rather do the follow-up with enums and a library than with grep and
sh. :)

> >  static const char * const bugreport_usage[] = {
> >  	N_("git bugreport [-o|--output <file>]"),
> >  	NULL
> > @@ -193,6 +243,9 @@ int cmd_main(int argc, const char **argv)
> >  	get_header(&buffer, "Safelisted Config Info");
> >  	get_safelisted_config(&buffer);
> >  
> > +	get_header(&buffer, "Configured Hooks");
> 
> Phrase nit.  There may be many people who just enabled hooks without
> configuring, so "Enabled Hooks" may be more appropriate.  We do not
> have to inspect what is in the hook to determine if it is enabled,
> but we do need to if we want to tell if a hook is "configured".

Ah, that's a good point. OK, I'll change the wording as you suggested.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 12/15] bugreport: count loose objects
  2020-02-04 18:48     ` Junio C Hamano
@ 2020-02-05  2:50       ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-05  2:50 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Johannes Schindelin

On Tue, Feb 04, 2020 at 10:48:30AM -0800, Junio C Hamano wrote:
> emilyshaffer@google.com writes:
> 
> > From: Emily Shaffer <emilyshaffer@google.com>
> >
> > The number of unpacked objects in a user's repository may help us
> > understand the root of the problem they're seeing, especially if a
> > command is running unusually slowly.
> >
> > Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> > The refactor removed much of the code Dscho suggested; and yet it
> > remains true that he helped me while developing this commit (although
> > his suggestions didn't survive). Shall I leave the Helped-by line or
> > remove it?
> 
> You two collectively thought about viable alternatives and decided
> to reject what was not wanted in the final result, and not having
> that rejected code was good for the project, right?  If so, I would
> say it still is the help that deserves recognition.  After all,
> making the result better by removing things is harder than by adding
> things ;-)

Sounds great to me. I'd rather over-assign credit than under-assign (but
I don't think this is a case of over-assigning here).

> >  bugreport.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 46 insertions(+)
> > ...
> 
> The patch text looked sensible.

Thanks.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 11/15] bugreport: collect list of populated hooks
  2020-02-05  2:48       ` Emily Shaffer
@ 2020-02-05  3:00         ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-05  3:00 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Feb 04, 2020 at 06:48:59PM -0800, Emily Shaffer wrote:
> Well, with Martin's suggestion[1] to use annotate:bugreport[include]
> instead of bugreport:include[x], this kind of change becomes a matter of
> course to just use "annotate:hook[x]" and write a script nearly
> identical to generate-bugreport-config-safelist.sh, and add it to the
> Makefile.

Bah.

[1]: https://lore.kernel.org/git/CAN0heSq_i4EitqYH-qrZyXBU+=PUNcSXOOJDHLSnJ1ufV0WtaQ@mail.gmail.com

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 13/15] bugreport: add packed object summary
  2020-02-04 19:03     ` Junio C Hamano
@ 2020-02-05  3:09       ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-05  3:09 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Derrick Stolee

On Tue, Feb 04, 2020 at 11:03:57AM -0800, Junio C Hamano wrote:
> emilyshaffer@google.com writes:
> 
> > From: Emily Shaffer <emilyshaffer@google.com>
> >
> > Alongside the loose object counts, it can be useful to show the number
> > of packs and packed objects. This way we can check whether the repo has
> > an appropriate ratio of packed to loose objects to help determine
> > whether it's behaving correctly.
> 
> This step makes me wonder if we want to see the midx as well.
> Didn't we have a bug that manifests only when midx is (or is not) in
> use?
> 
> Similarly, I think use (or non-use) of the commit graph may also be
> a useful information for diagnosing.
> 

Sure. I'll look into both of these. Would you rather see them as add-ons
later, or do you want me to hold off on a v6 of this set until they're
ready?

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 13/15] bugreport: add packed object summary
  2020-02-04 19:00     ` Junio C Hamano
@ 2020-02-05  3:15       ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-05  3:15 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Feb 04, 2020 at 11:00:41AM -0800, Junio C Hamano wrote:
> emilyshaffer@google.com writes:
> 
> > From: Emily Shaffer <emilyshaffer@google.com>
> >
> > Alongside the loose object counts, it can be useful to show the number
> > of packs and packed objects. This way we can check whether the repo has
> > an appropriate ratio of packed to loose objects to help determine
> > whether it's behaving correctly.
> 
> This lets the enumeration machinery to enumerate at the individual
> object level, relies on the enumeration machinery to show your
> callback all objects from the same pack before an object from a
> different pack, to count both objects and packs.
> 
> But given that packfiles record how many objects there are in the
> pack, and that the packfile.c layer must know what are the pack
> files we have available, I wonder if we have an API already to allow
> us to enumerate each packfile, and while counting the number of
> times our callback was called, add the objects contained in each of
> them.  If there isn't, it does not sound like a bad API function to
> add (i.e. given a repository r, iterate over r->objects->packed_git
> and let the API caller see each of them; the API caller can grab
> num_objects and add them up).

Yeah, I don't see one. I'll go ahead and add it - that sounds
straightforward enough.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 06/15] bugreport: add compiler info
  2020-02-04 22:51       ` Emily Shaffer
@ 2020-02-05 19:47         ` Martin Ågren
  0 siblings, 0 replies; 273+ messages in thread
From: Martin Ågren @ 2020-02-05 19:47 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Tue, 4 Feb 2020 at 23:51, Emily Shaffer <emilyshaffer@google.com> wrote:
>
> On Thu, Jan 30, 2020 at 11:21:39PM +0100, Martin Ågren wrote:
> > On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > > +#else
> > > +
> > > +static inline void get_compiler_info(struct strbuf *info)
> > > +{
> > > +       strbuf_addstr(info, "get_compiler_info() not implemented");
> >
> > Maybe "no compiler info available" (or s/ available//, or
> > s/available/reported/), or something else more human-readable?
>
> Hm. I envisioned the target audience for this as "Git developers" (and
> that's why I didn't mark the unimplemented string for translation); Git
> developers know exactly what "get_compiler_info() not implemented"
> means. But I suppose it's just as easy to grep for one string as the
> other.
>
> I am hesitant to say "no info available" or "no info reported" when I'm
> certain there is some environment variable set at build time which
> contains the info we'd want to show here; it really is just a question
> of some Git developer having put in the time to implement this function.
>
> So, I think I will leave it. Thanks for your suggestions.

Ok, that makes sense.

Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-02-05  0:44       ` Emily Shaffer
@ 2020-02-05 19:53         ` Martin Ågren
  0 siblings, 0 replies; 273+ messages in thread
From: Martin Ågren @ 2020-02-05 19:53 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List, Johannes Schindelin

On Wed, 5 Feb 2020 at 01:45, Emily Shaffer <emilyshaffer@google.com> wrote:
>
> > I sort of wonder whether safelist/blocklist is to prefer over
> > whitelist/blacklist, or if it's the other way round. The former are new
> > to me, whereas the latter are the terms I would have used. But that's
> > just me, of course. I was a little surprised, that's all.
>
> Eh. I think the following things are true:
>
>  - Whitelist/blacklist has a "smell" of discrimination, whether that's
>    the true etymology or not.
>  - Those with experience in the field can easily understand what
>    whitelist or blacklist means.
>  - Safelist/blocklist do not "smell" the same way.
>  - It is easy to tell what "safelist" means: "a list of stuff which is
>    safe." No experience needed.
>
> So, while it's new, I think it's not harmful. I see only a no-op or
> positive impact from using this term instead of whitelist/blacklist.
> Computer science seems to have quite a few terms which fall into this
> long-standing but potentially negative area, so I don't mind looking for
> alternatives where it's harmless to do so.

Ok, that all makes sense.

Thanks
Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 08/15] bugreport: include user interactive shell
  2020-01-30 22:28     ` Martin Ågren
  2020-02-04 23:16       ` Emily Shaffer
@ 2020-02-05 20:06       ` Junio C Hamano
  2020-02-05 20:14         ` Martin Ågren
  1 sibling, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-05 20:06 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Emily Shaffer, Git Mailing List

Martin Ågren <martin.agren@gmail.com> writes:

> On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
>> +       char *shell = NULL;
>
> (Unnecessary initialization.)
>
>> +       shell = getenv("SHELL");
>> +       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
>> +                   shell ? shell : "(NULL)");
>
> Thanks for avoiding a classic pitfall. :-)
>
> "<unused>" instead of "(NULL)"? "NULL" is mostly an implementation
> detail.

Isn't that <unset>?

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 10/15] bugreport: add config values from safelist
  2020-02-05  2:31       ` Emily Shaffer
@ 2020-02-05 20:12         ` Martin Ågren
  0 siblings, 0 replies; 273+ messages in thread
From: Martin Ågren @ 2020-02-05 20:12 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Wed, 5 Feb 2020 at 03:31, Emily Shaffer <emilyshaffer@google.com> wrote:
>
> On Fri, Jan 31, 2020 at 10:25:06PM +0100, Martin Ågren wrote:
> > On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > > Taking the build-time generated array and putting it into a set saves us
> > > time - since git_config_bugreport() is called for every option the user
> > > has configured, performing option lookup in constant time is a useful
> > > optimization.
> >
> > I'm sympathetic to your sending out what you have to obtain comments,
> > knowing that it's not perfect. It would have saved me some time and
> > effort if I'd known that this was the case though. An "[RFC]" tag,
> > perhaps. Or at least tweaking the above part of this commit message to
> > say that this might be over-engineered, with a reference to [1].
> >
> > [1] https://lore.kernel.org/git/20200124032905.GA37541@google.com/
>
> Yeah, you are right, and thanks for the feedback. I think I had wrapped
> up the series before I realized there were those significant outstanding
> comments, but I definitely should have said so in this patch.
>
> >
> > > +       int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
> >
> > I was going to suggest ARRAY_SIZE, but then I realized there are some
> > outstanding questions around whether you need this stuff in the first
> > place. I'd be inclined to guess that the first version of this would be
> > "for each safelisted item, obtain it and include", ignoring any "a.*.b"
> > business. In which case you wouldn't really need this hashset stuff.
>
> Regardless, I think it's worth doing nicely for now.
>
> It seems to me that supporting wildcarding in the config safelist is a
> superset of supporting static strings in that safelist - that is, if I
> write it simply to support static strings, a later change to support
> wildcards would be welcome and non-breaking.

Yeah, agreed. Also, whatever magic the code for gathering the config
items knows, the annotations in Documentation/, i.e., the generated list
in bugreport-config-safelist.h will be "synced". That is, until we know
how to handle "a.*.b" in the safelist, we probably won't be marking
"a.*.b" as safe. And even if we have some crazy mixed build, it's a
*safe*list, so we should be fine. (Famous last words?)

On that topic though, and just so we don't mess up from the beginning,
in the documentation, we typically write something like
"branch.<name>.merge", i.e., not "branch.*.merge". So I guess the
"a.*.b" feature would be more like "look for '.<', something, something,
'>.'." Would that be sufficient? Or would we actually want more
fine-grained control, i.e., something like regexes?

If so, we'd need to provide those regexes somehow, e.g., as part of the
asciidoc macro notation. :-/ Do we need to prepare somehow for such a
future? I don't think we do.

> So I'm going to clean this up without making it more featureful, for
> now.
>
> Sorry about the confusion!
>  - Emily

No worries. :)

Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v5 08/15] bugreport: include user interactive shell
  2020-02-05 20:06       ` Junio C Hamano
@ 2020-02-05 20:14         ` Martin Ågren
  0 siblings, 0 replies; 273+ messages in thread
From: Martin Ågren @ 2020-02-05 20:14 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Emily Shaffer, Git Mailing List

On Wed, 5 Feb 2020 at 21:06, Junio C Hamano <gitster@pobox.com> wrote:
>
> Martin Ågren <martin.agren@gmail.com> writes:
>
> > On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> >> +       char *shell = NULL;
> >
> > (Unnecessary initialization.)
> >
> >> +       shell = getenv("SHELL");
> >> +       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
> >> +                   shell ? shell : "(NULL)");
> >
> > Thanks for avoiding a classic pitfall. :-)
> >
> > "<unused>" instead of "(NULL)"? "NULL" is mostly an implementation
> > detail.
>
> Isn't that <unset>?

Heh, yes, "SHELL: unused" sounds wrong. :-)

Martin

^ permalink raw reply	[flat|nested] 273+ messages in thread

* [PATCH v6 00/15] add git-bugreport tool
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (17 preceding siblings ...)
  2020-01-30 22:15   ` Martin Ågren
@ 2020-02-06  0:40   ` Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 01/15] help: move list_config_help to builtin/help Emily Shaffer
                       ` (15 more replies)
  18 siblings, 16 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Martin Ågren, Aaron Schrab

This patchset is based on mr/show-config-scope.

Since v5, I addressed comments from Martin and Junio.

 - Reorganized the series so the ecosystem cleanup comes before the
   interesting new features.
 - Spread out manpage updates to indicate each new feature as it's added.
 - Fixed a number of patches which assumed a repo, so bugreport can now
   run anywhere.
 - Wordsmithing in the report template.
 - die() instead of overwriting an existing report.
 - Added --suffix to allow user to change the name of the report to
   git-bugreport-<strftime format>.txt.
 - General NULL-ptr, leak, and bashism cleanup.
 - Changed safelist macro from bugreport:include[x] to
   annotate:bugreport[include].
 - Use git_config_get_value_multi() instead of a config callback to
   gather the safelist configs.
 - Added a convenience for_each_pack() macro to object-store.h and began
   using it in packfile.c and bugreport.c for pack summary

Emily Shaffer (15):
  help: move list_config_help to builtin/help
  help: add shell-path to --build-options
  bugreport: add tool to generate debugging info
  bugreport: gather git version and build info
  bugreport: add uname info
  bugreport: add compiler info
  bugreport: add git-remote-https version
  bugreport: include user interactive shell
  bugreport: generate config safelist based on docs
  bugreport: add config values from safelist
  bugreport: collect list of populated hooks
  bugreport: count loose objects
  bugreport: add packed object summary
  bugreport: list contents of $OBJDIR/info
  bugreport: summarize contents of alternates file

 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   9 +
 Documentation/asciidoctor-extensions.rb |   5 +
 Documentation/config/sendemail.txt      |  56 ++--
 Documentation/git-bugreport.txt         |  55 ++++
 Makefile                                |  27 +-
 bugreport.c                             | 409 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 compat/compiler.h                       |  24 ++
 generate-bugreport-config-safelist.sh   |  17 +
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  24 ++
 help.c                                  | 132 ++------
 help.h                                  |   2 +-
 object-store.h                          |   6 +
 packfile.c                              |   3 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  49 +++
 18 files changed, 778 insertions(+), 156 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh

-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 273+ messages in thread

* [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
@ 2020-02-06  0:40     ` Emily Shaffer
  2020-02-06  1:35       ` Danh Doan
  2020-02-06  0:40     ` [PATCH v6 02/15] help: add shell-path to --build-options Emily Shaffer
                       ` (14 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
builtin/clean.o, and builtin/config.o. This meant that help.o was
unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help()
into builtin/help.c (where it makes sense to assume other builtin libraries
are present).

When command-list.h is included but a member is not used, we start to
hear a compiler warning. Since the config list is generated in a fairly
different way than the command list, and since commands and config
options are semantically different, move the config list into its own
header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 121 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh

diff --git a/.gitignore b/.gitignore
index aebe7c0908..ea97de83f3 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 09f98b777c..5a022367d4 100644
--- a/Makefile
+++ b/Makefile
@@ -814,6 +814,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2128,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2148,6 +2149,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2781,7 +2788,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2803,7 +2810,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..eca6a00c30
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 02/15] help: add shell-path to --build-options
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 01/15] help: move list_config_help to builtin/help Emily Shaffer
@ 2020-02-06  0:40     ` Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 03/15] bugreport: add tool to generate debugging info Emily Shaffer
                       ` (13 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It may be useful to know which shell Git was built to try to point to,
in the event that shell-based Git commands are failing. $SHELL_PATH is
set during the build and used to launch the manpage viewer, as well as
by git-compat-util.h, and it's used during tests. 'git version
--build-options' is encouraged for use in bug reports, so it makes sense
to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/help.c b/help.c
index a21487db77..190722fb0a 100644
--- a/help.c
+++ b/help.c
@@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 			printf("no commit associated with this build\n");
 		printf("sizeof-long: %d\n", (int)sizeof(long));
 		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
+		printf("shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 	return 0;
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 01/15] help: move list_config_help to builtin/help Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 02/15] help: add shell-path to --build-options Emily Shaffer
@ 2020-02-06  0:40     ` Emily Shaffer
  2020-02-07 14:18       ` SZEDER Gábor
                         ` (2 more replies)
  2020-02-06  0:40     ` [PATCH v6 04/15] bugreport: gather git version and build info Emily Shaffer
                       ` (12 subsequent siblings)
  15 siblings, 3 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction
steps, expected behavior, and actual behavior. Later, Git can learn how
to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic
information we would otherwise need to ask the user for, we can reduce
the number of question-and-answer round trips between the reporter and
the Git contributor.

Users may also wish to send a report like this to their local "Git
expert" if they have put their repository into a state they are confused
by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 41 ++++++++++++++++
 Makefile                        |  5 ++
 bugreport.c                     | 85 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 49 +++++++++++++++++++
 5 files changed, 181 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index ea97de83f3..d89bf9e11e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..52d49ed7aa
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,41 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [(-o | --output-directory) <path>] [(-s | --suffix) <format>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+OPTIONS
+-------
+-o <path>::
+--output-directory <path>::
+	Place the resulting bug report file in `<path>` instead of the root of
+	the Git repository.
+
+-s <format>::
+--suffix <format>::
+	Specify an alternate suffix for the bugreport name, to create a file
+	named 'git-bugreport-<formatted suffix>'. This should take the form of a
+	link:strftime[3] format string; the current local time will be used.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index 5a022367d4..a01a050aa3 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2457,6 +2458,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..db46fb88be
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,85 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to share.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+	char *option_suffix = "%F-%H%M";
+	struct stat statbuf;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output-directory", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
+			   N_("specify a strftime format suffix for the filename")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	if (!stat(report_path.buf, &statbuf))
+		die("'%s' already exists", report_path.buf);
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+
+	if (report == NULL) {
+		strbuf_release(&report_path);
+		die("couldn't open '%s' for writing", report_path.buf);
+	}
+
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
+
+	UNLEAK(buffer);
+	UNLEAK(report_path);
+	return -launch_editor(report_path.buf, NULL, NULL);
+}
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..451badff0c
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+			echo "$line"
+			read -r nextline
+			if test -z "$nextline"; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	REPORT="$(ls git-bugreport-*)" &&
+	check_all_headers_populated <$REPORT &&
+	rm $REPORT
+'
+
+test_expect_success 'dies if file with same name as report already exists' '
+	touch git-bugreport-duplicate.txt &&
+	test_must_fail git bugreport --suffix duplicate &&
+	rm git-bugreport-duplicate.txt
+'
+
+test_expect_success '--output-directory puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test_path_is_file foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test_path_is_missing git-bugreport-*
+'
+
+test_done
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 04/15] bugreport: gather git version and build info
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (2 preceding siblings ...)
  2020-02-06  0:40     ` [PATCH v6 03/15] bugreport: add tool to generate debugging info Emily Shaffer
@ 2020-02-06  0:40     ` Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 05/15] bugreport: add uname info Emily Shaffer
                       ` (11 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Knowing which version of Git a user has and how it was built allows us
to more precisely pin down the circumstances when a certain issue
occurs, so teach bugreport how to tell us the same output as 'git
version --build-options'.

It's not ideal to directly call 'git version --build-options' because
that output goes to stdout. Instead, wrap the version string in a helper
within help.[ch] library, and call that helper from within the bugreport
library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  4 +++
 bugreport.c                     | 19 ++++++++++++-
 help.c                          | 48 ++++++++++++++++++++-------------
 help.h                          |  1 +
 4 files changed, 52 insertions(+), 20 deletions(-)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 52d49ed7aa..6ce2ae2eff 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -23,6 +23,10 @@ The following information is requested from the user:
  - Expected behavior
  - Actual behavior
 
+The following information is captured automatically:
+
+ - 'git version --build-options'
+
 OPTIONS
 -------
 -o <path>::
diff --git a/bugreport.c b/bugreport.c
index db46fb88be..dd21be9d68 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,17 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
@@ -32,6 +41,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -67,6 +81,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/help.c b/help.c
index 190722fb0a..44cee69c11 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,26 +662,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	get_version_info(&buf, build_options);
+	printf("%s", buf.buf);
+
+	strbuf_release(&buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		printf("shell-path: %s\n", SHELL_PATH);
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..500521b908 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 05/15] bugreport: add uname info
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (3 preceding siblings ...)
  2020-02-06  0:40     ` [PATCH v6 04/15] bugreport: gather git version and build info Emily Shaffer
@ 2020-02-06  0:40     ` Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 06/15] bugreport: add compiler info Emily Shaffer
                       ` (10 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

The contents of uname() can give us some insight into what sort of
system the user is running on, and help us replicate their setup if need
be. The domainname field is not guaranteed to be available, so don't
collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 13 +++++++++++++
 2 files changed, 14 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 6ce2ae2eff..4dd72c60f5 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -26,6 +26,7 @@ The following information is requested from the user:
 The following information is captured automatically:
 
  - 'git version --build-options'
+ - uname sysname, release, version, and machine strings
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index dd21be9d68..4c4b877ea1 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -7,10 +7,23 @@
 
 static void get_system_info(struct strbuf *sys_info)
 {
+	struct utsname uname_info;
+
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(sys_info, 1);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 06/15] bugreport: add compiler info
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (4 preceding siblings ...)
  2020-02-06  0:40     ` [PATCH v6 05/15] bugreport: add uname info Emily Shaffer
@ 2020-02-06  0:40     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 07/15] bugreport: add git-remote-https version Emily Shaffer
                       ` (9 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

To help pinpoint the source of a regression, it is useful to know some
info about the compiler which the user's Git client was built with. By
adding a generic get_compiler_info() in 'compat/' we can choose which
relevant information to share per compiler; to get started, let's
demonstrate the version of glibc if the user built with 'gcc'.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     |  5 +++++
 compat/compiler.h               | 24 ++++++++++++++++++++++++
 3 files changed, 30 insertions(+)
 create mode 100644 compat/compiler.h

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4dd72c60f5..8bbc4c960c 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -27,6 +27,7 @@ The following information is captured automatically:
 
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
+ - Compiler-specific info string
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 4c4b877ea1..aaeb94c10c 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/compat/compiler.h b/compat/compiler.h
new file mode 100644
index 0000000000..bda5098e1b
--- /dev/null
+++ b/compat/compiler.h
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 07/15] bugreport: add git-remote-https version
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (5 preceding siblings ...)
  2020-02-06  0:40     ` [PATCH v6 06/15] bugreport: add compiler info Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 08/15] bugreport: include user interactive shell Emily Shaffer
                       ` (8 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible for git-remote-curl to be built separately from git; in that
case we want to know what version of cURL is used by git-remote-curl, not
necessarily which version was present at git-bugreport's build time.
So instead, ask git-remote-curl for the version information it knows
about.

Today, "git-remote-http" and "git-remote-https" are aliased to
"git-remote-curl"; but in case we rely on a different library than cURL
in the future, let's not explicitly reference cURL from bugreport.

For longevity purposes, invoke the alias "git-remote-https" instead of
"git-remote-http".

Since it could have been built at a different time, also report the
version and built-from commit of git-remote-curl alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 16 ++++++++++++++++
 remote-curl.c                   |  8 ++++++++
 3 files changed, 25 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 8bbc4c960c..33df4dec7f 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -28,6 +28,7 @@ The following information is captured automatically:
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
+ - 'git remote-https --build-info'
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index aaeb94c10c..da5859c131 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_git_remote_https_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/remote-curl.c b/remote-curl.c
index 350d92a074..c590fbfae3 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 08/15] bugreport: include user interactive shell
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (6 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 07/15] bugreport: add git-remote-https version Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 09/15] bugreport: generate config safelist based on docs Emily Shaffer
                       ` (7 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible a user may complain about the way that Git interacts with
their interactive shell, e.g. autocompletion or shell prompt. In that
case, it's useful for us to know which shell they're using
interactively.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt | 1 +
 bugreport.c                     | 5 +++++
 2 files changed, 6 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 33df4dec7f..23265b0d74 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -29,6 +29,7 @@ The following information is captured automatically:
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
  - 'git remote-https --build-info'
+ - $SHELL
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index da5859c131..9b51250155 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -21,6 +21,7 @@ static void get_git_remote_https_version_info(struct strbuf *version_info)
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -42,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "<unset>");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_git_remote_https_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 09/15] bugreport: generate config safelist based on docs
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (7 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 08/15] bugreport: include user interactive shell Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-07 15:30       ` SZEDER Gábor
  2020-02-06  0:41     ` [PATCH v6 10/15] bugreport: add config values from safelist Emily Shaffer
                       ` (6 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Martin Ågren, Johannes Schindelin

Add a new step to the build to generate a safelist of git-config
variables which are appropriate to include in the output of
git-bugreport. New variables can be added to the safelist by annotating
their documentation in Documentation/config with the "annotate" macro,
which is a no-op in AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs,
passwords, or other sensitive information. In the event that a user
doesn't notice their information while reviewing a bugreport, that user
may leak their credentials to other individuals, mailing lists, or bug
tracking tools inadvertently. Heuristic blocklisting of configuration
keys is imperfect and prone to false negatives; given the nature of the
information which can be leaked, a safelist is more reliable.

However, it's possible that in some situations, an organization may be
less concerned with privacy of things like remote URLs and branch names,
and more concerned with ease of diagnosis for their support staff. In
those cases, it may make more sense for that organization to modify the
code to use a blocklist. To that end, we should try to mark configs which
are definitely safe, and configs which are definitely unsafe, and leave
blank configs which are somewhere in between. To mark a config as safe,
add "annotate:bugreport[include]" to the corresponding line in the
config documentation; to mark it as unsafe, add
"annotate:bugreport[exclude]" instead.

Generating bugreport-config-safelist.h at build time by grepping the
documentation for this new macro helps us prevent staleness. The macro
itself is a no-op and should not alter the appearance of the
documentation in either AsciiDoc or AsciiDoctor, confirmable by running:

  cd Documentation
  ./doc-diff --asciidoctor HEAD^ HEAD
  ./doc-diff --asciidoc HEAD^ HEAD

Diffing the rendered HTML shows that only inline comments were added,
which shouldn't be a problem.

Additionally, add annotations to the sendemail config documentation in
order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  9 ++++
 Documentation/asciidoctor-extensions.rb |  5 +++
 Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
 Makefile                                |  7 ++++
 generate-bugreport-config-safelist.sh   | 17 ++++++++
 6 files changed, 67 insertions(+), 28 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index d89bf9e11e..bd2f49b996 100644
--- a/.gitignore
+++ b/.gitignore
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..663e06481f 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,14 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The annotate macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources to populate
+# things like the bugreport safelist.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +33,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[annotate-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[annotate-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..382bd8f6f4 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,5 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  inline_macro Git::Documentation::AnnotateProcessor, :annotate
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..fe27473e44 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -4,7 +4,7 @@ sendemail.identity::
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
@@ -15,7 +15,7 @@ sendemail.smtpsslcertpath::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* annotate:bugreport[exclude] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
@@ -23,41 +23,41 @@ sendemail.<identity>.*::
 
 sendemail.aliasesFile::
 sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.annotate annotate:bugreport[include] ::
+sendemail.bcc annotate:bugreport[include] ::
+sendemail.cc annotate:bugreport[include] ::
+sendemail.ccCmd annotate:bugreport[include] ::
+sendemail.chainReplyTo annotate:bugreport[include] ::
+sendemail.confirm annotate:bugreport[include] ::
+sendemail.envelopeSender annotate:bugreport[include] ::
+sendemail.from annotate:bugreport[include] ::
+sendemail.multiEdit annotate:bugreport[include] ::
+sendemail.signedoffbycc annotate:bugreport[include] ::
+sendemail.smtpPass annotate:bugreport[exclude] ::
+sendemail.suppresscc annotate:bugreport[include] ::
+sendemail.suppressFrom annotate:bugreport[include] ::
+sendemail.to annotate:bugreport[include] ::
+sendemail.tocmd annotate:bugreport[include] ::
+sendemail.smtpDomain annotate:bugreport[include] ::
+sendemail.smtpServer annotate:bugreport[include] ::
+sendemail.smtpServerPort annotate:bugreport[include] ::
+sendemail.smtpServerOption annotate:bugreport[include] ::
+sendemail.smtpUser annotate:bugreport[exclude] ::
+sendemail.thread annotate:bugreport[include] ::
+sendemail.transferEncoding annotate:bugreport[include] ::
+sendemail.validate annotate:bugreport[include] ::
+sendemail.xmailer annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.
 
 sendemail.signedoffcc (deprecated)::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize annotate:bugreport[include] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay annotate:bugreport[include] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index a01a050aa3..2bc9f112ea 100644
--- a/Makefile
+++ b/Makefile
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..17d92a91c5
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the annotate-bugreport macro
+sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
+
+cat <<EOF
+};
+EOF
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 10/15] bugreport: add config values from safelist
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (8 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 09/15] bugreport: generate config safelist based on docs Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-07 14:47       ` SZEDER Gábor
  2020-02-06  0:41     ` [PATCH v6 11/15] bugreport: collect list of populated hooks Emily Shaffer
                       ` (5 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach bugreport to gather the values of config options which are present
in 'bugreport-config-safelist.h', and show their origin scope.

Many config options are sensitive, and many Git add-ons use config
options which git-core does not know about; it is better only to gather
config options which we know to be safe, rather than excluding options
which we know to be unsafe.

Rather than including the path to someone's config, which can reveal
filesystem layout and project names, just name the scope (e.g. system,
global, local) of the config source.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 Makefile                        |  2 ++
 bugreport.c                     | 34 +++++++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 23265b0d74..4e9171d1bd 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -30,6 +30,7 @@ The following information is captured automatically:
  - Compiler-specific info string
  - 'git remote-https --build-info'
  - $SHELL
+ - Selected config values
 
 OPTIONS
 -------
diff --git a/Makefile b/Makefile
index 2bc9f112ea..7b139230ae 100644
--- a/Makefile
+++ b/Makefile
@@ -2131,6 +2131,8 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
+bugreport.sp bugreport.s bugreport.o: bugreport-config-safelist.h
+
 builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
diff --git a/bugreport.c b/bugreport.c
index 9b51250155..16216bff8e 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -52,6 +55,29 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	size_t idx;
+	struct string_list_item *it = NULL;
+	struct key_value_info *kv_info = NULL;
+
+	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
+		const struct string_list *list =
+			git_config_get_value_multi(bugreport_config_safelist[idx]);
+
+		if (!list)
+			continue;
+
+		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
+		for_each_string_list_item(it, list) {
+			kv_info = it->util;
+			strbuf_addf(config_info, "  %s (%s)\n", it->string,
+				    kv_info ? config_scope_name(kv_info->scope)
+					    : "source unknown");
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -94,6 +120,7 @@ int cmd_main(int argc, const char **argv)
 	char *option_output = NULL;
 	char *option_suffix = "%F-%H%M";
 	struct stat statbuf;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output-directory", &option_output, N_("path"),
@@ -102,6 +129,10 @@ int cmd_main(int argc, const char **argv)
 			   N_("specify a strftime format suffix for the filename")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -123,6 +154,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 11/15] bugreport: collect list of populated hooks
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (9 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 10/15] bugreport: add config values from safelist Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 12/15] bugreport: count loose objects Emily Shaffer
                       ` (4 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific
hook which is being run, perhaps without the user realizing. While the
contents of hooks can be sensitive - containing user data or process
information specific to the user's organization - simply knowing that a
hook is being run at a certain stage can help us to understand whether
something is going wrong.

Without a definitive list of hook names within the code, we compile our
own list from the documentation. This is likely prone to bitrot. To
reduce the amount of code humans need to read, we turn the list into a
string_list and iterate over it (as we are calling the same find_hook
operation on each string). However, since bugreport should primarily be
called by the user, the performance loss from massaging the string
seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 53 +++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4e9171d1bd..779c9c7fd9 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -31,6 +31,7 @@ The following information is captured automatically:
  - 'git remote-https --build-info'
  - $SHELL
  - Selected config values
+ - A list of enabled hooks
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 16216bff8e..3b0a2b12cc 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -78,6 +79,55 @@ static void get_safelisted_config(struct strbuf *config_info)
 	}
 }
 
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hooks`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -157,6 +207,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Enabled Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 12/15] bugreport: count loose objects
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (10 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 11/15] bugreport: collect list of populated hooks Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 13/15] bugreport: add packed object summary Emily Shaffer
                       ` (3 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Johannes Schindelin

The number of unpacked objects in a user's repository may help us
understand the root of the problem they're seeing, especially if a
command is running unusually slowly.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 779c9c7fd9..7e1ed44cb4 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 3b0a2b12cc..9552e2922e 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
+
+	int local_loose_object_count = 0, total_loose_object_count = 0;
+	int local_count_questionable = 0, total_count_questionable = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	local_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&local_loose_object_count,
+					FOR_EACH_OBJECT_LOCAL_ONLY);
+
+	total_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&total_loose_object_count,
+					0);
+
+	strbuf_addf(obj_info, "%d local loose objects%s\n",
+		    local_loose_object_count,
+		    local_count_questionable ? " (problem during count)" : "");
+
+	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
+		    total_loose_object_count - local_loose_object_count,
+		    (local_count_questionable || total_count_questionable)
+			? " (problem during count)"
+			: "");
+
+	strbuf_addf(obj_info, "%d total loose objects%s\n",
+		    total_loose_object_count,
+		    total_count_questionable ? " (problem during count)" : "");
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -210,6 +259,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Enabled Hooks");
 	get_populated_hooks(&buffer, nongit_ok);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 13/15] bugreport: add packed object summary
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (11 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 12/15] bugreport: count loose objects Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
                       ` (2 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Alongside the loose object counts, it can be useful to show the number
of packs and packed objects. This way we can check whether the repo has
an appropriate ratio of packed to loose objects to help determine
whether it's behaving correctly.

Add a utility to easily traverse all packfiles in a given repository.
Use it in packfile.c and remove a redundant call to
prepare_packed_git(), which is already called in get_all_packs().

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
Perhaps the macro should be in another patch? I didn't spend a lot of
time looking for anybody who could conceivably be iterating packs; if I
were to do so, I'd break this change out separately.

I considered following the pattern in the rest of the file, where the
foreach takes a callback and data pointer, but found it would be
cumbersome to use with for_each_packed_object() - I'd have to add a new
callback just to handle the pack, and bundle that function's callback
data along with the callback pointer. It seemed like overkill.

 - Emily

 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 30 ++++++++++++++++++++++++++++++
 object-store.h                  |  6 ++++++
 packfile.c                      |  3 +--
 4 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 7e1ed44cb4..309b34fc42 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -33,6 +33,7 @@ The following information is captured automatically:
  - Selected config values
  - A list of enabled hooks
  - The number of loose objects in the repository
+ - The number of packs and packed objects in the repository
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 9552e2922e..d554488f3e 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -177,6 +177,33 @@ static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
+{
+	struct packed_git *pack = NULL;
+	int pack_count = 0;
+	int object_count = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	for_each_pack(the_repository, pack) {
+		pack_count++;
+		/*
+		 * To accurately count how many objects are packed, look inside
+		 * the packfile's index.
+		 */
+		open_pack_index(pack);
+		object_count += pack->num_objects;
+	}
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
+		    object_count);
+
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -262,6 +289,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/object-store.h b/object-store.h
index 55ee639350..518c73469d 100644
--- a/object-store.h
+++ b/object-store.h
@@ -6,6 +6,7 @@
 #include "list.h"
 #include "sha1-array.h"
 #include "strbuf.h"
+#include "packfile.h"
 
 struct object_directory {
 	struct object_directory *next;
@@ -403,4 +404,9 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(each_packed_object_fn, void *,
 			   enum for_each_object_flags flags);
 
+#define for_each_pack(repo, pack) 		\
+		for (pack = get_all_packs(repo);\
+		     pack;			\
+		     pack = pack->next)
+
 #endif /* OBJECT_STORE_H */
diff --git a/packfile.c b/packfile.c
index 7e7c04e4d8..6873827fb5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2061,8 +2061,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git(the_repository);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for_each_pack(the_repository, p) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 14/15] bugreport: list contents of $OBJDIR/info
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (12 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 13/15] bugreport: add packed object summary Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 15/15] bugreport: summarize contents of alternates file Emily Shaffer
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Miscellaneous information used about the object store can end up in
.git/objects/info; this can help us understand what may be going on with
the object store when the user is reporting a bug. Otherwise, it could
be difficult to track down what is going wrong with an object which
isn't kept locally to .git/objects/ or .git/objects/pack. Having some
understanding of where the user's objects may be kept can save us some
hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 54 +++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 309b34fc42..a21d081616 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -34,6 +34,7 @@ The following information is captured automatically:
  - A list of enabled hooks
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
+ - A list of the contents of .git/objects/info (or equivalent)
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index d554488f3e..3a3d33f680 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -204,6 +204,57 @@ static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
 
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info, int nongit)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - object info unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -292,6 +343,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v6 15/15] bugreport: summarize contents of alternates file
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (13 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In some cases, it could be that the user is having a problem with an
object which isn't present in their normal object directory. We can get
a hint that that might be the case by examining the list of alternates
where their object may be stored instead. Since paths to alternates may
be sensitive, we'll instead count how many alternates have been
specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this
function does not provide information on broken alternates, which are
skipped over in "link_alt_odb_entry()". Since the goal is to identify
missing alternates, we can gather the contents of
.git/objects/info/alternates manually.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 45 +++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index a21d081616..e870900c80 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -35,6 +35,7 @@ The following information is captured automatically:
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
  - A list of the contents of .git/objects/info (or equivalent)
+ - The number of valid and invalid alternates
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 3a3d33f680..9d4d5c8e6f 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -255,6 +255,48 @@ static void get_object_info_summary(struct strbuf *obj_info, int nongit)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	if (nongit) {
+		strbuf_addstr(alternates_info,
+			"not run from a git repository - alternates unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -346,6 +388,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-06  0:40     ` [PATCH v6 01/15] help: move list_config_help to builtin/help Emily Shaffer
@ 2020-02-06  1:35       ` Danh Doan
  2020-02-13 22:58         ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Danh Doan @ 2020-02-06  1:35 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

On 2020-02-05 16:40:54-0800, Emily Shaffer <emilyshaffer@google.com> wrote:
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> index 71158f7d8b..45fecf8bdf 100755
> --- a/generate-cmdlist.sh
> +++ b/generate-cmdlist.sh
> @@ -76,23 +76,6 @@ print_command_list () {
>  	echo "};"
>  }
>  
> -print_config_list () {
> -	cat <<EOF
> -static const char *config_name_list[] = {
> -EOF
> -	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> -	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> -	sort |
> -	while read line
> -	do
> -		echo "	\"$line\","
> -	done
> -	cat <<EOF
> -	NULL,
> -};
> -EOF
> -}
> -
>  exclude_programs=
>  while test "--exclude-program" = "$1"
>  do
> @@ -113,5 +96,3 @@ echo
>  define_category_names "$1"
>  echo
>  print_command_list "$1"
> -echo
> -print_config_list
> diff --git a/generate-configlist.sh b/generate-configlist.sh
> new file mode 100755
> index 0000000000..eca6a00c30
> --- /dev/null
> +++ b/generate-configlist.sh
> @@ -0,0 +1,24 @@
> +#!/bin/sh
> +
> +echo "/* Automatically generated by generate-configlist.sh */"
> +echo
> +
> +print_config_list () {
> +	cat <<EOF
> +static const char *config_name_list[] = {
> +EOF
> +	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> +	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> +	sort |
> +	while read line
> +	do
> +		echo "	\"$line\","
> +	done

This while-read-echo was moved from generate-cmdlist.sh,
which has some logic to work with read-variable.

We're moving it out, I think apply this diff on top of it will make the code easier to read.

diff --git a/generate-configlist.sh b/generate-configlist.sh
index eca6a00c30..163dbf30bb 100755
--- a/generate-configlist.sh
+++ b/generate-configlist.sh
@@ -10,10 +10,7 @@ EOF
 	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
 	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
 	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
+	sed 's/^/	"/; s/$/",/'
 	cat <<EOF
 	NULL,
 };



-- 
Danh

^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-06  0:40     ` [PATCH v6 03/15] bugreport: add tool to generate debugging info Emily Shaffer
@ 2020-02-07 14:18       ` SZEDER Gábor
  2020-02-07 18:51         ` Junio C Hamano
  2020-02-07 14:54       ` SZEDER Gábor
  2020-02-12 18:06       ` Junio C Hamano
  2 siblings, 1 reply; 273+ messages in thread
From: SZEDER Gábor @ 2020-02-07 14:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

On Wed, Feb 05, 2020 at 04:40:56PM -0800, Emily Shaffer wrote:
> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
> new file mode 100755
> index 0000000000..451badff0c
> --- /dev/null
> +++ b/t/t0091-bugreport.sh
> @@ -0,0 +1,49 @@
> +#!/bin/sh
> +
> +test_description='git bugreport'
> +
> +. ./test-lib.sh
> +
> +# Headers "[System Info]" will be followed by a non-empty line if we put some
> +# information there; we can make sure all our headers were followed by some
> +# information to check if the command was successful.
> +HEADER_PATTERN="^\[.*\]$"
> +check_all_headers_populated() {
> +	while read -r line; do
> +		if test "$(grep "$HEADER_PATTERN" "$line")"; then
> +			echo "$line"
> +			read -r nextline
> +			if test -z "$nextline"; then
> +				return 1;
> +			fi
> +		fi
> +	done
> +}
> +
> +test_expect_success 'creates a report with content in the right places' '
> +	git bugreport &&
> +	REPORT="$(ls git-bugreport-*)" &&

What if the globbing were to match more than one file?

> +	check_all_headers_populated <$REPORT &&
> +	rm $REPORT

Please register a cleanup commands like this with 'test_when_finished'.

> +'
> +
> +test_expect_success 'dies if file with same name as report already exists' '
> +	touch git-bugreport-duplicate.txt &&
> +	test_must_fail git bugreport --suffix duplicate &&
> +	rm git-bugreport-duplicate.txt
> +'
> +
> +test_expect_success '--output-directory puts the report in the provided dir' '
> +	mkdir foo/ &&

Is it really necessary to create the directory in advance?  Or to put
it in another way: shouldn't 'git bugreport' create any missing
leading directories of the path given for its '-o' option?  FWIW, 'git
format-patch -o dir ...' does create all necessary directories.

> +	git bugreport -o foo/ &&
> +	test_path_is_file foo/git-bugreport-* &&

What if the globbing were to match more than one file? :)

> +	rm -fr foo/
> +'
> +
> +test_expect_success 'incorrect arguments abort with usage' '
> +	test_must_fail git bugreport --false 2>output &&
> +	grep usage output &&

This breaks the GETTEXT_POISON CI job, because "usage" is translated,
so the test should not look for it with plain 'grep'.  Please use
'test_i18ngrep' instead.

> +	test_path_is_missing git-bugreport-*
> +'
> +
> +test_done
> -- 
> 2.25.0.341.g760bfbb309-goog
> 

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 10/15] bugreport: add config values from safelist
  2020-02-06  0:41     ` [PATCH v6 10/15] bugreport: add config values from safelist Emily Shaffer
@ 2020-02-07 14:47       ` SZEDER Gábor
  2020-02-07 15:08         ` SZEDER Gábor
  0 siblings, 1 reply; 273+ messages in thread
From: SZEDER Gábor @ 2020-02-07 14:47 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

On Wed, Feb 05, 2020 at 04:41:03PM -0800, Emily Shaffer wrote:
> +static void get_safelisted_config(struct strbuf *config_info)
> +{
> +	size_t idx;
> +	struct string_list_item *it = NULL;
> +	struct key_value_info *kv_info = NULL;
> +
> +	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {

GCC 9 complains about this loop condition:

      CC bugreport.o
  
  bugreport.c: In function 'get_safelisted_config':
  
  bugreport.c:66:20: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]
  
     66 |  for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
  
        |                    ^

I don't understand this error, that autogenerated
'bugreport_config_safelist' array clearly has a a non-zero size.
What am I missing?


> +		const struct string_list *list =
> +			git_config_get_value_multi(bugreport_config_safelist[idx]);
> +
> +		if (!list)
> +			continue;
> +
> +		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
> +		for_each_string_list_item(it, list) {
> +			kv_info = it->util;
> +			strbuf_addf(config_info, "  %s (%s)\n", it->string,
> +				    kv_info ? config_scope_name(kv_info->scope)
> +					    : "source unknown");
> +		}
> +	}
> +}
> +

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-06  0:40     ` [PATCH v6 03/15] bugreport: add tool to generate debugging info Emily Shaffer
  2020-02-07 14:18       ` SZEDER Gábor
@ 2020-02-07 14:54       ` SZEDER Gábor
  2020-02-12 18:06       ` Junio C Hamano
  2 siblings, 0 replies; 273+ messages in thread
From: SZEDER Gábor @ 2020-02-07 14:54 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

On Wed, Feb 05, 2020 at 04:40:56PM -0800, Emily Shaffer wrote:
> Teach Git how to prompt the user for a good bug report: reproduction
> steps, expected behavior, and actual behavior. Later, Git can learn how
> to collect some diagnostic information from the repository.
> 
> If users can send us a well-written bug report which contains diagnostic
> information we would otherwise need to ask the user for, we can reduce
> the number of question-and-answer round trips between the reporter and
> the Git contributor.
> 
> Users may also wish to send a report like this to their local "Git
> expert" if they have put their repository into a state they are confused
> by.
> 
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  .gitignore                      |  1 +
>  Documentation/git-bugreport.txt | 41 ++++++++++++++++
>  Makefile                        |  5 ++
>  bugreport.c                     | 85 +++++++++++++++++++++++++++++++++
>  t/t0091-bugreport.sh            | 49 +++++++++++++++++++
>  5 files changed, 181 insertions(+)
>  create mode 100644 Documentation/git-bugreport.txt
>  create mode 100644 bugreport.c
>  create mode 100755 t/t0091-bugreport.sh

Please add a corresponding entry to 'command-list.txt' as well.
According to the comment at the beginning of that file, all, even
external commands should be included there.


^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 10/15] bugreport: add config values from safelist
  2020-02-07 14:47       ` SZEDER Gábor
@ 2020-02-07 15:08         ` SZEDER Gábor
  2020-02-07 16:24           ` Eric Sunshine
  0 siblings, 1 reply; 273+ messages in thread
From: SZEDER Gábor @ 2020-02-07 15:08 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

On Fri, Feb 07, 2020 at 03:47:25PM +0100, SZEDER Gábor wrote:
> On Wed, Feb 05, 2020 at 04:41:03PM -0800, Emily Shaffer wrote:
> > +static void get_safelisted_config(struct strbuf *config_info)
> > +{
> > +	size_t idx;
> > +	struct string_list_item *it = NULL;
> > +	struct key_value_info *kv_info = NULL;
> > +
> > +	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
> 
> GCC 9 complains about this loop condition:
> 
>       CC bugreport.o
>   
>   bugreport.c: In function 'get_safelisted_config':
>   
>   bugreport.c:66:20: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]
>   
>      66 |  for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
>   
>         |                    ^
> 
> I don't understand this error, that autogenerated
> 'bugreport_config_safelist' array clearly has a a non-zero size.
> What am I missing?

macOS 'sed', that's what I was missing :)

So that array is in fact empty on macOS, because the entries of that
array are generated with:

  # cat all regular files in Documentation/config
  find Documentation/config -type f -exec cat {} \; |
  # print the command name which matches the annotate-bugreport macro
  sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort

and the 'sed' included in macOS apparently interprets that '\+'
differently than GNU 'sed', and as a result won't match anything.

FWIW, that '\+' doesn't seem to be necessary, though, and after
removing it the resulting generated array looked good to me (and to
the compiler) both on Linux and macOS.


^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 09/15] bugreport: generate config safelist based on docs
  2020-02-06  0:41     ` [PATCH v6 09/15] bugreport: generate config safelist based on docs Emily Shaffer
@ 2020-02-07 15:30       ` SZEDER Gábor
  2020-02-13 23:14         ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: SZEDER Gábor @ 2020-02-07 15:30 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Martin Ågren, Johannes Schindelin

On Wed, Feb 05, 2020 at 04:41:02PM -0800, Emily Shaffer wrote:
> Add a new step to the build to generate a safelist of git-config
> variables which are appropriate to include in the output of
> git-bugreport. New variables can be added to the safelist by annotating
> their documentation in Documentation/config with the "annotate" macro,
> which is a no-op in AsciiDoc and AsciiDoctor.
> 
> Some configs are private in nature, and can contain remote URLs,
> passwords, or other sensitive information. In the event that a user
> doesn't notice their information while reviewing a bugreport, that user
> may leak their credentials to other individuals, mailing lists, or bug
> tracking tools inadvertently. Heuristic blocklisting of configuration
> keys is imperfect and prone to false negatives; given the nature of the
> information which can be leaked, a safelist is more reliable.
> 
> However, it's possible that in some situations, an organization may be
> less concerned with privacy of things like remote URLs and branch names,
> and more concerned with ease of diagnosis for their support staff. In
> those cases, it may make more sense for that organization to modify the
> code to use a blocklist. To that end, we should try to mark configs which
> are definitely safe, and configs which are definitely unsafe, and leave
> blank configs which are somewhere in between. To mark a config as safe,
> add "annotate:bugreport[include]" to the corresponding line in the
> config documentation; to mark it as unsafe, add
> "annotate:bugreport[exclude]" instead.
> 
> Generating bugreport-config-safelist.h at build time by grepping the
> documentation for this new macro helps us prevent staleness. The macro
> itself is a no-op and should not alter the appearance of the
> documentation in either AsciiDoc or AsciiDoctor, confirmable by running:
> 
>   cd Documentation
>   ./doc-diff --asciidoctor HEAD^ HEAD
>   ./doc-diff --asciidoc HEAD^ HEAD
> 
> Diffing the rendered HTML shows that only inline comments were added,
> which shouldn't be a problem.
> 
> Additionally, add annotations to the sendemail config documentation in
> order to demonstrate a proof of concept.
> 
> Helped-by: Martin Ågren <martin.agren@gmail.com>
> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  .gitignore                              |  1 +
>  Documentation/asciidoc.conf             |  9 ++++
>  Documentation/asciidoctor-extensions.rb |  5 +++
>  Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
>  Makefile                                |  7 ++++
>  generate-bugreport-config-safelist.sh   | 17 ++++++++
>  6 files changed, 67 insertions(+), 28 deletions(-)
>  create mode 100755 generate-bugreport-config-safelist.sh
> 
> diff --git a/.gitignore b/.gitignore
> index d89bf9e11e..bd2f49b996 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -192,6 +192,7 @@
>  /gitweb/static/gitweb.min.*
>  /config-list.h
>  /command-list.h
> +/bugreport-config-safelist.h
>  *.tar.gz
>  *.dsc
>  *.deb
> diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
> index 8fc4b67081..663e06481f 100644
> --- a/Documentation/asciidoc.conf
> +++ b/Documentation/asciidoc.conf
> @@ -6,9 +6,14 @@
>  #
>  # Show Git link as: <command>(<section>); if section is defined, else just show
>  # the command.
> +#
> +# The annotate macro does nothing as far as rendering is
> +# concerned -- we just grep for it in the sources to populate
> +# things like the bugreport safelist.
>  
>  [macros]
>  (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
> +(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
>  
>  [attributes]
>  asterisk=&#42;
> @@ -28,6 +33,8 @@ ifdef::backend-docbook[]
>  {0#<citerefentry>}
>  {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
>  {0#</citerefentry>}
> +[annotate-inlinemacro]
> +{0#}
>  endif::backend-docbook[]
>  
>  ifdef::backend-docbook[]
> @@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
>  git-relative-html-prefix=
>  [linkgit-inlinemacro]
>  <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
> +[annotate-inlinemacro]
> +<!-- -->
>  endif::backend-xhtml11[]
> diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
> index d906a00803..382bd8f6f4 100644
> --- a/Documentation/asciidoctor-extensions.rb
> +++ b/Documentation/asciidoctor-extensions.rb
> @@ -37,6 +37,10 @@ module Git
>            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
>          end
>          output
> +
> +    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> +      def process(parent, target, attrs)
> +        ""

This change breaks building the documentation with Asciidoctor v1.5.8:

  /home/travis/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError)
  
  ...xtensions::InlineMacroProcessor
  
  ...                               ^
  
  /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:53: syntax error, unexpected end-of-input, expecting keyword_end


  https://travis-ci.org/git/git/jobs/647093871#L1127

>        end
>      end
>    end
> @@ -45,4 +49,5 @@ end
>  Asciidoctor::Extensions.register do
>    inline_macro Git::Documentation::LinkGitProcessor, :linkgit
>    postprocessor Git::Documentation::DocumentPostProcessor
> +  inline_macro Git::Documentation::AnnotateProcessor, :annotate
>  end

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 10/15] bugreport: add config values from safelist
  2020-02-07 15:08         ` SZEDER Gábor
@ 2020-02-07 16:24           ` Eric Sunshine
  2020-02-07 16:51             ` Andreas Schwab
  0 siblings, 1 reply; 273+ messages in thread
From: Eric Sunshine @ 2020-02-07 16:24 UTC (permalink / raw)
  To: SZEDER Gábor; +Cc: Emily Shaffer, Git List

On Fri, Feb 7, 2020 at 10:09 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
> macOS 'sed', that's what I was missing :)
>
>   sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
>
> and the 'sed' included in macOS apparently interprets that '\+'
> differently than GNU 'sed', and as a result won't match anything.

More generally, this would be a problem with any 'sed' of BSD lineage.

> FWIW, that '\+' doesn't seem to be necessary, though, and after
> removing it the resulting generated array looked good to me [...]

A reasonable replacement for "<SP>\+" would be "<SP><SP>*" (where <SP>
represents 'space').

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 10/15] bugreport: add config values from safelist
  2020-02-07 16:24           ` Eric Sunshine
@ 2020-02-07 16:51             ` Andreas Schwab
  2020-02-13 22:02               ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Andreas Schwab @ 2020-02-07 16:51 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: SZEDER Gábor, Emily Shaffer, Git List

On Feb 07 2020, Eric Sunshine wrote:

> On Fri, Feb 7, 2020 at 10:09 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
>> macOS 'sed', that's what I was missing :)
>>
>>   sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
>>
>> and the 'sed' included in macOS apparently interprets that '\+'
>> differently than GNU 'sed', and as a result won't match anything.
>
> More generally, this would be a problem with any 'sed' of BSD lineage.
>
>> FWIW, that '\+' doesn't seem to be necessary, though, and after
>> removing it the resulting generated array looked good to me [...]
>
> A reasonable replacement for "<SP>\+" would be "<SP><SP>*" (where <SP>
> represents 'space').

Another problem with that regexp is that it contains two adjacent
repetitions matching the same character.  When there are two or more
spaces before "annotate:" all but the last of them can be matched by
either '\(.*\)' or ' \+'.  To fix that '\(.*\)' needs to be modified to
not match a trailing space.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-07 14:18       ` SZEDER Gábor
@ 2020-02-07 18:51         ` Junio C Hamano
  2020-02-11 22:40           ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-07 18:51 UTC (permalink / raw)
  To: SZEDER Gábor; +Cc: Emily Shaffer, git

SZEDER Gábor <szeder.dev@gmail.com> writes:

> On Wed, Feb 05, 2020 at 04:40:56PM -0800, Emily Shaffer wrote:
>> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
>> new file mode 100755
>> index 0000000000..451badff0c
>> --- /dev/null
>> +++ b/t/t0091-bugreport.sh
>> @@ -0,0 +1,49 @@
>> +#!/bin/sh
>> +
>> +test_description='git bugreport'
>> +
>> +. ./test-lib.sh
>> +
>> +# Headers "[System Info]" will be followed by a non-empty line if we put some
>> +# information there; we can make sure all our headers were followed by some
>> +# information to check if the command was successful.
>> +HEADER_PATTERN="^\[.*\]$"
>> +check_all_headers_populated() {
>> +	while read -r line; do
>> +		if test "$(grep "$HEADER_PATTERN" "$line")"; then
>> +			echo "$line"
>> +			read -r nextline
>> +			if test -z "$nextline"; then
>> +				return 1;
>> +			fi
>> +		fi
>> +	done
>> +}
>> +
>> +test_expect_success 'creates a report with content in the right places' '
>> +	git bugreport &&
>> +	REPORT="$(ls git-bugreport-*)" &&
>
> What if the globbing were to match more than one file?

An often-useful pattern is to make the command report the output
filename, i.e.

	REPORT=$(git butreport) &&

if the design insists that "git bugreport" should allocate a
filename in order to make it easy to guarantee uniqueness.

Of course, we can make the invoker supply filename, e.g.

	REPORT=$(generate-output-filename) &&
	git bugreport -o "$REPORT" ;# or git bugreport >"$REPORT"


^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-07 18:51         ` Junio C Hamano
@ 2020-02-11 22:40           ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-11 22:40 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: SZEDER Gábor, git

On Fri, Feb 07, 2020 at 10:51:29AM -0800, Junio C Hamano wrote:
> SZEDER Gábor <szeder.dev@gmail.com> writes:
> 
> > On Wed, Feb 05, 2020 at 04:40:56PM -0800, Emily Shaffer wrote:
> >> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
> >> new file mode 100755
> >> index 0000000000..451badff0c
> >> --- /dev/null
> >> +++ b/t/t0091-bugreport.sh
> >> @@ -0,0 +1,49 @@
> >> +#!/bin/sh
> >> +
> >> +test_description='git bugreport'
> >> +
> >> +. ./test-lib.sh
> >> +
> >> +# Headers "[System Info]" will be followed by a non-empty line if we put some
> >> +# information there; we can make sure all our headers were followed by some
> >> +# information to check if the command was successful.
> >> +HEADER_PATTERN="^\[.*\]$"
> >> +check_all_headers_populated() {
> >> +	while read -r line; do
> >> +		if test "$(grep "$HEADER_PATTERN" "$line")"; then
> >> +			echo "$line"
> >> +			read -r nextline
> >> +			if test -z "$nextline"; then
> >> +				return 1;
> >> +			fi
> >> +		fi
> >> +	done
> >> +}
> >> +
> >> +test_expect_success 'creates a report with content in the right places' '
> >> +	git bugreport &&
> >> +	REPORT="$(ls git-bugreport-*)" &&
> >
> > What if the globbing were to match more than one file?
> 
> An often-useful pattern is to make the command report the output
> filename, i.e.
> 
> 	REPORT=$(git butreport) &&
> 
> if the design insists that "git bugreport" should allocate a
> filename in order to make it easy to guarantee uniqueness.
> 
> Of course, we can make the invoker supply filename, e.g.
> 
> 	REPORT=$(generate-output-filename) &&
> 	git bugreport -o "$REPORT" ;# or git bugreport >"$REPORT"

Yeah, this is an option and I've changed the offending test - which
captures a variable based on globbing - to use --suffix and force a
filename.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-06  0:40     ` [PATCH v6 03/15] bugreport: add tool to generate debugging info Emily Shaffer
  2020-02-07 14:18       ` SZEDER Gábor
  2020-02-07 14:54       ` SZEDER Gábor
@ 2020-02-12 18:06       ` Junio C Hamano
  2020-02-12 22:36         ` Emily Shaffer
  2 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-12 18:06 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

As this topic seems to break under GIT_TEST_GETTEXT_POISON=yes, I'll
apply the following band-aid on top before merging it to 'pu'.  

Most of them are style fixes, but quoting $REPORT, not just makes
the redirection honor the coding guidelines, will ensure that an
error is caught if git-bugreport-* glob matches no paths or more
than on paths.




 t/t0091-bugreport.sh | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
index 451badff0c..f2186941ce 100755
--- a/t/t0091-bugreport.sh
+++ b/t/t0091-bugreport.sh
@@ -8,9 +8,12 @@ test_description='git bugreport'
 # information there; we can make sure all our headers were followed by some
 # information to check if the command was successful.
 HEADER_PATTERN="^\[.*\]$"
-check_all_headers_populated() {
-	while read -r line; do
-		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
 			echo "$line"
 			read -r nextline
 			if test -z "$nextline"; then
@@ -23,12 +26,12 @@ check_all_headers_populated() {
 test_expect_success 'creates a report with content in the right places' '
 	git bugreport &&
 	REPORT="$(ls git-bugreport-*)" &&
-	check_all_headers_populated <$REPORT &&
-	rm $REPORT
+	check_all_headers_populated <"$REPORT" &&
+	rm "$REPORT"
 '
 
 test_expect_success 'dies if file with same name as report already exists' '
-	touch git-bugreport-duplicate.txt &&
+	>>git-bugreport-duplicate.txt &&
 	test_must_fail git bugreport --suffix duplicate &&
 	rm git-bugreport-duplicate.txt
 '
@@ -42,7 +45,7 @@ test_expect_success '--output-directory puts the report in the provided dir' '
 
 test_expect_success 'incorrect arguments abort with usage' '
 	test_must_fail git bugreport --false 2>output &&
-	grep usage output &&
+	test_i18ngrep usage output &&
 	test_path_is_missing git-bugreport-*
 '
 
-- 
2.25.0-455-gcf3c3a5ab4


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-12 18:06       ` Junio C Hamano
@ 2020-02-12 22:36         ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-12 22:36 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Wed, Feb 12, 2020 at 10:06:44AM -0800, Junio C Hamano wrote:
> As this topic seems to break under GIT_TEST_GETTEXT_POISON=yes, I'll
> apply the following band-aid on top before merging it to 'pu'.  
> 
> Most of them are style fixes, but quoting $REPORT, not just makes
> the redirection honor the coding guidelines, will ensure that an
> error is caught if git-bugreport-* glob matches no paths or more
> than on paths.

Thanks. Most of these I have locally, but not all. I'll apply this as a
fixup on patch 3/n (the one that introduces the tests) and add a
Helped-by line.

I'm hoping to get a reroll out today; it was the first thing on my list
this week, and then a high-priority internal issue shows up (so it
goes). Sorry for the inconvenience with the CI failures.

 - Emily

>  t/t0091-bugreport.sh | 17 ++++++++++-------
>  1 file changed, 10 insertions(+), 7 deletions(-)
> 
> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
> index 451badff0c..f2186941ce 100755
> --- a/t/t0091-bugreport.sh
> +++ b/t/t0091-bugreport.sh
> @@ -8,9 +8,12 @@ test_description='git bugreport'
>  # information there; we can make sure all our headers were followed by some
>  # information to check if the command was successful.
>  HEADER_PATTERN="^\[.*\]$"
> -check_all_headers_populated() {
> -	while read -r line; do
> -		if test "$(grep "$HEADER_PATTERN" "$line")"; then
> +
> +check_all_headers_populated () {
> +	while read -r line
> +	do
> +		if test "$(grep "$HEADER_PATTERN" "$line")"
> +		then
>  			echo "$line"
>  			read -r nextline
>  			if test -z "$nextline"; then
> @@ -23,12 +26,12 @@ check_all_headers_populated() {
>  test_expect_success 'creates a report with content in the right places' '
>  	git bugreport &&
>  	REPORT="$(ls git-bugreport-*)" &&
> -	check_all_headers_populated <$REPORT &&
> -	rm $REPORT
> +	check_all_headers_populated <"$REPORT" &&
> +	rm "$REPORT"
>  '
>  
>  test_expect_success 'dies if file with same name as report already exists' '
> -	touch git-bugreport-duplicate.txt &&
> +	>>git-bugreport-duplicate.txt &&
>  	test_must_fail git bugreport --suffix duplicate &&
>  	rm git-bugreport-duplicate.txt
>  '
> @@ -42,7 +45,7 @@ test_expect_success '--output-directory puts the report in the provided dir' '
>  
>  test_expect_success 'incorrect arguments abort with usage' '
>  	test_must_fail git bugreport --false 2>output &&
> -	grep usage output &&
> +	test_i18ngrep usage output &&
>  	test_path_is_missing git-bugreport-*
>  '
>  
> -- 
> 2.25.0-455-gcf3c3a5ab4
> 

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 10/15] bugreport: add config values from safelist
  2020-02-07 16:51             ` Andreas Schwab
@ 2020-02-13 22:02               ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-13 22:02 UTC (permalink / raw)
  To: Andreas Schwab; +Cc: Eric Sunshine, SZEDER Gábor, Git List

On Fri, Feb 07, 2020 at 05:51:05PM +0100, Andreas Schwab wrote:
> On Feb 07 2020, Eric Sunshine wrote:
> 
> > On Fri, Feb 7, 2020 at 10:09 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
> >> macOS 'sed', that's what I was missing :)
> >>
> >>   sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
> >>
> >> and the 'sed' included in macOS apparently interprets that '\+'
> >> differently than GNU 'sed', and as a result won't match anything.
> >
> > More generally, this would be a problem with any 'sed' of BSD lineage.
> >
> >> FWIW, that '\+' doesn't seem to be necessary, though, and after
> >> removing it the resulting generated array looked good to me [...]
> >
> > A reasonable replacement for "<SP>\+" would be "<SP><SP>*" (where <SP>
> > represents 'space').
> 
> Another problem with that regexp is that it contains two adjacent
> repetitions matching the same character.  When there are two or more
> spaces before "annotate:" all but the last of them can be matched by
> either '\(.*\)' or ' \+'.  To fix that '\(.*\)' needs to be modified to
> not match a trailing space.

Hum. I had assumed since the capture group was not greedy, it would not
capture any trailing spaces that could be captured by ' \+'. I don't see
a problem in making it non-explicit, though.

I'll hack on this some more. I do find myself pretty annoyed at the
matcher difference between 'sed' and 'also sed', though, and don't see
in the manpage a way to guarantee which matcher 'sed' should use (a la
'grep -[EFGP]'). :)

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-06  1:35       ` Danh Doan
@ 2020-02-13 22:58         ` Emily Shaffer
  2020-02-13 23:07           ` Eric Sunshine
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-13 22:58 UTC (permalink / raw)
  To: Danh Doan; +Cc: git

On Thu, Feb 06, 2020 at 08:35:33AM +0700, Danh Doan wrote:
> On 2020-02-05 16:40:54-0800, Emily Shaffer <emilyshaffer@google.com> wrote:
> > diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> > index 71158f7d8b..45fecf8bdf 100755
> > --- a/generate-cmdlist.sh
> > +++ b/generate-cmdlist.sh
> > @@ -76,23 +76,6 @@ print_command_list () {
> >  	echo "};"
> >  }
> >  
> > -print_config_list () {
> > -	cat <<EOF
> > -static const char *config_name_list[] = {
> > -EOF
> > -	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> > -	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> > -	sort |
> > -	while read line
> > -	do
> > -		echo "	\"$line\","
> > -	done
> > -	cat <<EOF
> > -	NULL,
> > -};
> > -EOF
> > -}
> > -
> >  exclude_programs=
> >  while test "--exclude-program" = "$1"
> >  do
> > @@ -113,5 +96,3 @@ echo
> >  define_category_names "$1"
> >  echo
> >  print_command_list "$1"
> > -echo
> > -print_config_list
> > diff --git a/generate-configlist.sh b/generate-configlist.sh
> > new file mode 100755
> > index 0000000000..eca6a00c30
> > --- /dev/null
> > +++ b/generate-configlist.sh
> > @@ -0,0 +1,24 @@
> > +#!/bin/sh
> > +
> > +echo "/* Automatically generated by generate-configlist.sh */"
> > +echo
> > +
> > +print_config_list () {
> > +	cat <<EOF
> > +static const char *config_name_list[] = {
> > +EOF
> > +	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> > +	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> > +	sort |
> > +	while read line
> > +	do
> > +		echo "	\"$line\","
> > +	done
> 
> This while-read-echo was moved from generate-cmdlist.sh,
> which has some logic to work with read-variable.
> 
> We're moving it out, I think apply this diff on top of it will make the code easier to read.
> 
> diff --git a/generate-configlist.sh b/generate-configlist.sh
> index eca6a00c30..163dbf30bb 100755
> --- a/generate-configlist.sh
> +++ b/generate-configlist.sh
> @@ -10,10 +10,7 @@ EOF
>  	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
>  	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
>  	sort |
> -	while read line
> -	do
> -		echo "	\"$line\","
> -	done
> +	sed 's/^/	"/; s/$/",/'
>  	cat <<EOF
>  	NULL,
>  };

Thanks for the suggestion. You're right that I didn't look into the
contents of this script much, because I was just moving it; I modified
your sed expression slightly so it performs just one operation:

  sed 's/^.*$/    "\0",/'

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-13 22:58         ` Emily Shaffer
@ 2020-02-13 23:07           ` Eric Sunshine
  2020-02-13 23:24             ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Eric Sunshine @ 2020-02-13 23:07 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Danh Doan, Git List

On Thu, Feb 13, 2020 at 5:58 PM Emily Shaffer <emilyshaffer@google.com> wrote:
> On Thu, Feb 06, 2020 at 08:35:33AM +0700, Danh Doan wrote:
> > -     while read line
> > -     do
> > -             echo "  \"$line\","
> > -     done
> > +     sed 's/^/       "/; s/$/",/'
>
> Thanks for the suggestion. You're right that I didn't look into the
> contents of this script much, because I was just moving it; I modified
> your sed expression slightly so it performs just one operation:
>
>   sed 's/^.*$/    "\0",/'

This isn't portable and won't work with 'sed' from BSD lineage,
including Mac OS. More portable would be:

    sed 's/^\(.*\)$/    "\1",/'

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 09/15] bugreport: generate config safelist based on docs
  2020-02-07 15:30       ` SZEDER Gábor
@ 2020-02-13 23:14         ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-13 23:14 UTC (permalink / raw)
  To: SZEDER Gábor; +Cc: git, Martin Ågren, Johannes Schindelin

On Fri, Feb 07, 2020 at 04:30:42PM +0100, SZEDER Gábor wrote:
> On Wed, Feb 05, 2020 at 04:41:02PM -0800, Emily Shaffer wrote:
> > diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
> > index d906a00803..382bd8f6f4 100644
> > --- a/Documentation/asciidoctor-extensions.rb
> > +++ b/Documentation/asciidoctor-extensions.rb
> > @@ -37,6 +37,10 @@ module Git
> >            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
> >          end
> >          output
> > +
> > +    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> > +      def process(parent, target, attrs)
> > +        ""
> 
> This change breaks building the documentation with Asciidoctor v1.5.8:
> 
>   /home/travis/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError)
>   
>   ...xtensions::InlineMacroProcessor
>   
>   ...                               ^
>   
>   /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:53: syntax error, unexpected end-of-input, expecting keyword_end
> 
> 
>   https://travis-ci.org/git/git/jobs/647093871#L1127

Done. It passes locally for me now if I run:

  USE_ASCIIDOCTOR=1 make doc

Thanks for the heads up.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-13 23:07           ` Eric Sunshine
@ 2020-02-13 23:24             ` Junio C Hamano
  2020-02-13 23:29               ` Eric Sunshine
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-13 23:24 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Emily Shaffer, Danh Doan, Git List

Eric Sunshine <sunshine@sunshineco.com> writes:

> On Thu, Feb 13, 2020 at 5:58 PM Emily Shaffer <emilyshaffer@google.com> wrote:
>> On Thu, Feb 06, 2020 at 08:35:33AM +0700, Danh Doan wrote:
>> > -     while read line
>> > -     do
>> > -             echo "  \"$line\","
>> > -     done
>> > +     sed 's/^/       "/; s/$/",/'
>>
>> Thanks for the suggestion. You're right that I didn't look into the
>> contents of this script much, because I was just moving it; I modified
>> your sed expression slightly so it performs just one operation:
>>
>>   sed 's/^.*$/    "\0",/'
>
> This isn't portable and won't work with 'sed' from BSD lineage,
> including Mac OS. More portable would be:
>
>     sed 's/^\(.*\)$/    "\1",/'

Or just replace Emily's \0 with &; that should be in POSIX.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-13 23:24             ` Junio C Hamano
@ 2020-02-13 23:29               ` Eric Sunshine
  2020-02-14  1:20                 ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Eric Sunshine @ 2020-02-13 23:29 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Emily Shaffer, Danh Doan, Git List

On Thu, Feb 13, 2020 at 6:24 PM Junio C Hamano <gitster@pobox.com> wrote:
> Eric Sunshine <sunshine@sunshineco.com> writes:
> > On Thu, Feb 13, 2020 at 5:58 PM Emily Shaffer <emilyshaffer@google.com> wrote:
> >> Thanks for the suggestion. You're right that I didn't look into the
> >> contents of this script much, because I was just moving it; I modified
> >> your sed expression slightly so it performs just one operation:
> >>
> >>   sed 's/^.*$/    "\0",/'
> >
> > This isn't portable and won't work with 'sed' from BSD lineage,
> > including Mac OS. More portable would be:
> >
> >     sed 's/^\(.*\)$/    "\1",/'
>
> Or just replace Emily's \0 with &; that should be in POSIX.

Yes, that's nicer.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-13 23:29               ` Eric Sunshine
@ 2020-02-14  1:20                 ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:20 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Junio C Hamano, Danh Doan, Git List

On Thu, Feb 13, 2020 at 06:29:39PM -0500, Eric Sunshine wrote:
> On Thu, Feb 13, 2020 at 6:24 PM Junio C Hamano <gitster@pobox.com> wrote:
> > Eric Sunshine <sunshine@sunshineco.com> writes:
> > > On Thu, Feb 13, 2020 at 5:58 PM Emily Shaffer <emilyshaffer@google.com> wrote:
> > >> Thanks for the suggestion. You're right that I didn't look into the
> > >> contents of this script much, because I was just moving it; I modified
> > >> your sed expression slightly so it performs just one operation:
> > >>
> > >>   sed 's/^.*$/    "\0",/'
> > >
> > > This isn't portable and won't work with 'sed' from BSD lineage,
> > > including Mac OS. More portable would be:
> > >
> > >     sed 's/^\(.*\)$/    "\1",/'
> >
> > Or just replace Emily's \0 with &; that should be in POSIX.
> 
> Yes, that's nicer.

Sure, thanks.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* [PATCH v7 00/15] add git-bugreport tool
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (14 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 15/15] bugreport: summarize contents of alternates file Emily Shaffer
@ 2020-02-14  1:53     ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 01/15] help: move list_config_help to builtin/help Emily Shaffer
                         ` (16 more replies)
  15 siblings, 17 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Martin Ågren, Aaron Schrab, Danh Doan,
	Eric Sunshine, SZEDER Gábor, Andreas Schwab

This patchset is based on mr/show-config-scope, which looks close to
ready for 'next'.

Since v6:

 - Applied Danh's suggestion[1] (with modifications recommended by Eric and
   Junio downthread)
 - Fixed a missed end-of-macro in the Asciidoctor config which was
   pointed out most recently by SZEDER[2]
 - Fixed generate-bugreport-safelist-config.sh to be more friendly to
   BSD-derived 'sed' and to capture trailing ' ' when extra whitespace is
   present(patch 9/15). This now uses POSIX character classes and {}
   notation instead of + and has been tested on OSX; I'd love to hear if
   it's still not working correctly.
 - Applied Junio's patch[3] to make GIT_TEST_GETTEXT_POISON happy and to
   make the tests more idiomatic.
 - Stop globbing into variables, start making the output directory if it
   doesn't exist before[4]
 - Add an entry in 'command-list.txt'[5]. I sure wish someone had
   explained how to do this somewhere in a tutorial about writing new
   commands ;)
 - Added a test to ensure that git-bugreport runs gracefully in a
   non-repo directory.
 - Added nongit_ok flags to relevant pieces of info to ensure the above
   test passes.

[1] https://lore.kernel.org/git/20200206013533.GA3993@danh.dev
[2] https://lore.kernel.org/git/20200207153042.GI2868@szeder.dev
[3] https://lore.kernel.org/git/xmqq36bfvfwr.fsf@gitster-ct.c.googlers.com
[4] https://lore.kernel.org/git/20200207141802.GE2868@szeder.dev
[5] https://lore.kernel.org/git/20200207145409.GG2868@szeder.dev


Emily Shaffer (15):
  help: move list_config_help to builtin/help
  help: add shell-path to --build-options
  bugreport: add tool to generate debugging info
  bugreport: gather git version and build info
  bugreport: add uname info
  bugreport: add compiler info
  bugreport: add git-remote-https version
  bugreport: include user interactive shell
  bugreport: generate config safelist based on docs
  bugreport: add config values from safelist
  bugreport: collect list of populated hooks
  bugreport: count loose objects
  bugreport: add packed object summary
  bugreport: list contents of $OBJDIR/info
  bugreport: summarize contents of alternates file

 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   9 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  56 ++--
 Documentation/git-bugreport.txt         |  55 ++++
 Makefile                                |  27 +-
 bugreport.c                             | 418 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 command-list.txt                        |   1 +
 compat/compiler.h                       |  24 ++
 generate-bugreport-config-safelist.sh   |  18 +
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  21 ++
 help.c                                  | 132 ++------
 help.h                                  |   2 +-
 object-store.h                          |   6 +
 packfile.c                              |   3 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  55 ++++
 19 files changed, 794 insertions(+), 156 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh

v6 to v7 range-diff:

diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index 382bd8f6f4..03c80af0e5 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,8 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+      end
+    end
 
     class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
       def process(parent, target, attrs)
diff --git a/bugreport.c b/bugreport.c
index 9d4d5c8e6f..3bc8cb3579 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -368,6 +368,15 @@ int cmd_main(int argc, const char **argv)
 	if (!stat(report_path.buf, &statbuf))
 		die("'%s' already exists", report_path.buf);
 
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
 	get_bug_template(&buffer);
 
 	get_header(&buffer, "System Info");
diff --git a/command-list.txt b/command-list.txt
index 2087894655..185e5e3f05 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
index 17d92a91c5..2794a8ae77 100755
--- a/generate-bugreport-config-safelist.sh
+++ b/generate-bugreport-config-safelist.sh
@@ -10,7 +10,8 @@ EOF
 # cat all regular files in Documentation/config
 find Documentation/config -type f -exec cat {} \; |
 # print the command name which matches the annotate-bugreport macro
-sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
+sed -n 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p' \
+	| sort
 
 cat <<EOF
 };
diff --git a/generate-configlist.sh b/generate-configlist.sh
index eca6a00c30..8692fe5cf4 100755
--- a/generate-configlist.sh
+++ b/generate-configlist.sh
@@ -10,10 +10,7 @@ EOF
 	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
 	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
 	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
+	sed 's/^.*$/	"&",/'
 	cat <<EOF
 	NULL,
 };
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
index 451badff0c..6585a2d144 100755
--- a/t/t0091-bugreport.sh
+++ b/t/t0091-bugreport.sh
@@ -8,9 +8,12 @@ test_description='git bugreport'
 # information there; we can make sure all our headers were followed by some
 # information to check if the command was successful.
 HEADER_PATTERN="^\[.*\]$"
-check_all_headers_populated() {
-	while read -r line; do
-		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
 			echo "$line"
 			read -r nextline
 			if test -z "$nextline"; then
@@ -21,29 +24,32 @@ check_all_headers_populated() {
 }
 
 test_expect_success 'creates a report with content in the right places' '
-	git bugreport &&
-	REPORT="$(ls git-bugreport-*)" &&
-	check_all_headers_populated <$REPORT &&
-	rm $REPORT
+	git bugreport -s check-headers &&
+	check_all_headers_populated <git-bugreport-check-headers.txt &&
+	test_when_finished rm git-bugreport-check-headers.txt
 '
 
 test_expect_success 'dies if file with same name as report already exists' '
-	touch git-bugreport-duplicate.txt &&
+	>>git-bugreport-duplicate.txt &&
 	test_must_fail git bugreport --suffix duplicate &&
-	rm git-bugreport-duplicate.txt
+	test_when_finished rm git-bugreport-duplicate.txt
 '
 
 test_expect_success '--output-directory puts the report in the provided dir' '
-	mkdir foo/ &&
 	git bugreport -o foo/ &&
 	test_path_is_file foo/git-bugreport-* &&
-	rm -fr foo/
+	test_when_finished rm -fr foo/
 '
 
 test_expect_success 'incorrect arguments abort with usage' '
 	test_must_fail git bugreport --false 2>output &&
-	grep usage output &&
+	test_i18ngrep usage output &&
 	test_path_is_missing git-bugreport-*
 '
 
+test_expect_success 'runs outside of a git dir' '
+	nongit git bugreport &&
+	test_when_finished rm non-repo/git-bugreport-*
+'
+
 test_done

-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 01/15] help: move list_config_help to builtin/help
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 02/15] help: add shell-path to --build-options Emily Shaffer
                         ` (15 subsequent siblings)
  16 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
builtin/clean.o, and builtin/config.o. This meant that help.o was
unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help()
into builtin/help.c (where it makes sense to assume other builtin libraries
are present).

When command-list.h is included but a member is not used, we start to
hear a compiler warning. Since the config list is generated in a fairly
different way than the command list, and since commands and config
options are semantically different, move the config list into its own
header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 21 +++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 118 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh

diff --git a/.gitignore b/.gitignore
index aebe7c0908..ea97de83f3 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 09f98b777c..5a022367d4 100644
--- a/Makefile
+++ b/Makefile
@@ -814,6 +814,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2128,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2148,6 +2149,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2781,7 +2788,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2803,7 +2810,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..8692fe5cf4
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	sed 's/^.*$/	"&",/'
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 02/15] help: add shell-path to --build-options
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 01/15] help: move list_config_help to builtin/help Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 03/15] bugreport: add tool to generate debugging info Emily Shaffer
                         ` (14 subsequent siblings)
  16 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It may be useful to know which shell Git was built to try to point to,
in the event that shell-based Git commands are failing. $SHELL_PATH is
set during the build and used to launch the manpage viewer, as well as
by git-compat-util.h, and it's used during tests. 'git version
--build-options' is encouraged for use in bug reports, so it makes sense
to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/help.c b/help.c
index a21487db77..190722fb0a 100644
--- a/help.c
+++ b/help.c
@@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 			printf("no commit associated with this build\n");
 		printf("sizeof-long: %d\n", (int)sizeof(long));
 		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
+		printf("shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 	return 0;
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 01/15] help: move list_config_help to builtin/help Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 02/15] help: add shell-path to --build-options Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14 17:25         ` Junio C Hamano
  2020-02-19 14:18         ` Johannes Schindelin
  2020-02-14  1:53       ` [PATCH v7 04/15] bugreport: gather git version and build info Emily Shaffer
                         ` (13 subsequent siblings)
  16 siblings, 2 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction
steps, expected behavior, and actual behavior. Later, Git can learn how
to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic
information we would otherwise need to ask the user for, we can reduce
the number of question-and-answer round trips between the reporter and
the Git contributor.

Users may also wish to send a report like this to their local "Git
expert" if they have put their repository into a state they are confused
by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 41 ++++++++++++++
 Makefile                        |  5 ++
 bugreport.c                     | 94 +++++++++++++++++++++++++++++++++
 command-list.txt                |  1 +
 t/t0091-bugreport.sh            | 55 +++++++++++++++++++
 6 files changed, 197 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index ea97de83f3..d89bf9e11e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..52d49ed7aa
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,41 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [(-o | --output-directory) <path>] [(-s | --suffix) <format>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+OPTIONS
+-------
+-o <path>::
+--output-directory <path>::
+	Place the resulting bug report file in `<path>` instead of the root of
+	the Git repository.
+
+-s <format>::
+--suffix <format>::
+	Specify an alternate suffix for the bugreport name, to create a file
+	named 'git-bugreport-<formatted suffix>'. This should take the form of a
+	link:strftime[3] format string; the current local time will be used.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index 5a022367d4..a01a050aa3 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2457,6 +2458,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..a9398e6a2a
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,94 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to share.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+	char *option_suffix = "%F-%H%M";
+	struct stat statbuf;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output-directory", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
+			   N_("specify a strftime format suffix for the filename")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	if (!stat(report_path.buf, &statbuf))
+		die("'%s' already exists", report_path.buf);
+
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+
+	if (report == NULL) {
+		strbuf_release(&report_path);
+		die("couldn't open '%s' for writing", report_path.buf);
+	}
+
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
+
+	UNLEAK(buffer);
+	UNLEAK(report_path);
+	return -launch_editor(report_path.buf, NULL, NULL);
+}
diff --git a/command-list.txt b/command-list.txt
index 2087894655..185e5e3f05 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..6585a2d144
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,55 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
+			echo "$line"
+			read -r nextline
+			if test -z "$nextline"; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport -s check-headers &&
+	check_all_headers_populated <git-bugreport-check-headers.txt &&
+	test_when_finished rm git-bugreport-check-headers.txt
+'
+
+test_expect_success 'dies if file with same name as report already exists' '
+	>>git-bugreport-duplicate.txt &&
+	test_must_fail git bugreport --suffix duplicate &&
+	test_when_finished rm git-bugreport-duplicate.txt
+'
+
+test_expect_success '--output-directory puts the report in the provided dir' '
+	git bugreport -o foo/ &&
+	test_path_is_file foo/git-bugreport-* &&
+	test_when_finished rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	test_i18ngrep usage output &&
+	test_path_is_missing git-bugreport-*
+'
+
+test_expect_success 'runs outside of a git dir' '
+	nongit git bugreport &&
+	test_when_finished rm non-repo/git-bugreport-*
+'
+
+test_done
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 04/15] bugreport: gather git version and build info
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (2 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 03/15] bugreport: add tool to generate debugging info Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 05/15] bugreport: add uname info Emily Shaffer
                         ` (12 subsequent siblings)
  16 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Knowing which version of Git a user has and how it was built allows us
to more precisely pin down the circumstances when a certain issue
occurs, so teach bugreport how to tell us the same output as 'git
version --build-options'.

It's not ideal to directly call 'git version --build-options' because
that output goes to stdout. Instead, wrap the version string in a helper
within help.[ch] library, and call that helper from within the bugreport
library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  4 +++
 bugreport.c                     | 19 ++++++++++++-
 help.c                          | 48 ++++++++++++++++++++-------------
 help.h                          |  1 +
 4 files changed, 52 insertions(+), 20 deletions(-)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 52d49ed7aa..6ce2ae2eff 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -23,6 +23,10 @@ The following information is requested from the user:
  - Expected behavior
  - Actual behavior
 
+The following information is captured automatically:
+
+ - 'git version --build-options'
+
 OPTIONS
 -------
 -o <path>::
diff --git a/bugreport.c b/bugreport.c
index a9398e6a2a..b7504fe322 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,17 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
@@ -32,6 +41,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -76,6 +90,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/help.c b/help.c
index 190722fb0a..44cee69c11 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,26 +662,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	get_version_info(&buf, build_options);
+	printf("%s", buf.buf);
+
+	strbuf_release(&buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		printf("shell-path: %s\n", SHELL_PATH);
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..500521b908 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 05/15] bugreport: add uname info
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (3 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 04/15] bugreport: gather git version and build info Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 06/15] bugreport: add compiler info Emily Shaffer
                         ` (11 subsequent siblings)
  16 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

The contents of uname() can give us some insight into what sort of
system the user is running on, and help us replicate their setup if need
be. The domainname field is not guaranteed to be available, so don't
collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 13 +++++++++++++
 2 files changed, 14 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 6ce2ae2eff..4dd72c60f5 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -26,6 +26,7 @@ The following information is requested from the user:
 The following information is captured automatically:
 
  - 'git version --build-options'
+ - uname sysname, release, version, and machine strings
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index b7504fe322..b76a1dfb2a 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -7,10 +7,23 @@
 
 static void get_system_info(struct strbuf *sys_info)
 {
+	struct utsname uname_info;
+
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(sys_info, 1);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 06/15] bugreport: add compiler info
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (4 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 05/15] bugreport: add uname info Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-19 14:23         ` Johannes Schindelin
  2020-02-14  1:53       ` [PATCH v7 07/15] bugreport: add git-remote-https version Emily Shaffer
                         ` (10 subsequent siblings)
  16 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

To help pinpoint the source of a regression, it is useful to know some
info about the compiler which the user's Git client was built with. By
adding a generic get_compiler_info() in 'compat/' we can choose which
relevant information to share per compiler; to get started, let's
demonstrate the version of glibc if the user built with 'gcc'.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     |  5 +++++
 compat/compiler.h               | 24 ++++++++++++++++++++++++
 3 files changed, 30 insertions(+)
 create mode 100644 compat/compiler.h

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4dd72c60f5..8bbc4c960c 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -27,6 +27,7 @@ The following information is captured automatically:
 
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
+ - Compiler-specific info string
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index b76a1dfb2a..4f9101caeb 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/compat/compiler.h b/compat/compiler.h
new file mode 100644
index 0000000000..bda5098e1b
--- /dev/null
+++ b/compat/compiler.h
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 07/15] bugreport: add git-remote-https version
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (5 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 06/15] bugreport: add compiler info Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-19 14:28         ` Johannes Schindelin
  2020-02-14  1:53       ` [PATCH v7 08/15] bugreport: include user interactive shell Emily Shaffer
                         ` (9 subsequent siblings)
  16 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible for git-remote-curl to be built separately from git; in that
case we want to know what version of cURL is used by git-remote-curl, not
necessarily which version was present at git-bugreport's build time.
So instead, ask git-remote-curl for the version information it knows
about.

Today, "git-remote-http" and "git-remote-https" are aliased to
"git-remote-curl"; but in case we rely on a different library than cURL
in the future, let's not explicitly reference cURL from bugreport.

For longevity purposes, invoke the alias "git-remote-https" instead of
"git-remote-http".

Since it could have been built at a different time, also report the
version and built-from commit of git-remote-curl alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 16 ++++++++++++++++
 remote-curl.c                   |  8 ++++++++
 3 files changed, 25 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 8bbc4c960c..33df4dec7f 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -28,6 +28,7 @@ The following information is captured automatically:
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
+ - 'git remote-https --build-info'
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 4f9101caeb..bfdff33368 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_git_remote_https_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/remote-curl.c b/remote-curl.c
index 350d92a074..c590fbfae3 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 08/15] bugreport: include user interactive shell
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (6 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 07/15] bugreport: add git-remote-https version Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 09/15] bugreport: generate config safelist based on docs Emily Shaffer
                         ` (8 subsequent siblings)
  16 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible a user may complain about the way that Git interacts with
their interactive shell, e.g. autocompletion or shell prompt. In that
case, it's useful for us to know which shell they're using
interactively.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt | 1 +
 bugreport.c                     | 5 +++++
 2 files changed, 6 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 33df4dec7f..23265b0d74 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -29,6 +29,7 @@ The following information is captured automatically:
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
  - 'git remote-https --build-info'
+ - $SHELL
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index bfdff33368..c1475601bf 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -21,6 +21,7 @@ static void get_git_remote_https_version_info(struct strbuf *version_info)
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -42,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "<unset>");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_git_remote_https_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 09/15] bugreport: generate config safelist based on docs
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (7 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 08/15] bugreport: include user interactive shell Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 10/15] bugreport: add config values from safelist Emily Shaffer
                         ` (7 subsequent siblings)
  16 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Martin Ågren, Johannes Schindelin

Add a new step to the build to generate a safelist of git-config
variables which are appropriate to include in the output of
git-bugreport. New variables can be added to the safelist by annotating
their documentation in Documentation/config with the "annotate" macro,
which is a no-op in AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs,
passwords, or other sensitive information. In the event that a user
doesn't notice their information while reviewing a bugreport, that user
may leak their credentials to other individuals, mailing lists, or bug
tracking tools inadvertently. Heuristic blocklisting of configuration
keys is imperfect and prone to false negatives; given the nature of the
information which can be leaked, a safelist is more reliable.

However, it's possible that in some situations, an organization may be
less concerned with privacy of things like remote URLs and branch names,
and more concerned with ease of diagnosis for their support staff. In
those cases, it may make more sense for that organization to modify the
code to use a blocklist. To that end, we should try to mark configs which
are definitely safe, and configs which are definitely unsafe, and leave
blank configs which are somewhere in between. To mark a config as safe,
add "annotate:bugreport[include]" to the corresponding line in the
config documentation; to mark it as unsafe, add
"annotate:bugreport[exclude]" instead.

Generating bugreport-config-safelist.h at build time by grepping the
documentation for this new macro helps us prevent staleness. The macro
itself is a no-op and should not alter the appearance of the
documentation in either AsciiDoc or AsciiDoctor, confirmable by running:

  cd Documentation
  ./doc-diff --asciidoctor HEAD^ HEAD
  ./doc-diff --asciidoc HEAD^ HEAD

Diffing the rendered HTML shows that only inline comments were added,
which shouldn't be a problem.

Additionally, add annotations to the sendemail config documentation in
order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  9 ++++
 Documentation/asciidoctor-extensions.rb |  7 ++++
 Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
 Makefile                                |  7 ++++
 generate-bugreport-config-safelist.sh   | 18 ++++++++
 6 files changed, 70 insertions(+), 28 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index d89bf9e11e..bd2f49b996 100644
--- a/.gitignore
+++ b/.gitignore
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..663e06481f 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,14 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The annotate macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources to populate
+# things like the bugreport safelist.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +33,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[annotate-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[annotate-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..03c80af0e5 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -39,10 +39,17 @@ module Git
         output
       end
     end
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""
+      end
+    end
   end
 end
 
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  inline_macro Git::Documentation::AnnotateProcessor, :annotate
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..fe27473e44 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -4,7 +4,7 @@ sendemail.identity::
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
@@ -15,7 +15,7 @@ sendemail.smtpsslcertpath::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* annotate:bugreport[exclude] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
@@ -23,41 +23,41 @@ sendemail.<identity>.*::
 
 sendemail.aliasesFile::
 sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.annotate annotate:bugreport[include] ::
+sendemail.bcc annotate:bugreport[include] ::
+sendemail.cc annotate:bugreport[include] ::
+sendemail.ccCmd annotate:bugreport[include] ::
+sendemail.chainReplyTo annotate:bugreport[include] ::
+sendemail.confirm annotate:bugreport[include] ::
+sendemail.envelopeSender annotate:bugreport[include] ::
+sendemail.from annotate:bugreport[include] ::
+sendemail.multiEdit annotate:bugreport[include] ::
+sendemail.signedoffbycc annotate:bugreport[include] ::
+sendemail.smtpPass annotate:bugreport[exclude] ::
+sendemail.suppresscc annotate:bugreport[include] ::
+sendemail.suppressFrom annotate:bugreport[include] ::
+sendemail.to annotate:bugreport[include] ::
+sendemail.tocmd annotate:bugreport[include] ::
+sendemail.smtpDomain annotate:bugreport[include] ::
+sendemail.smtpServer annotate:bugreport[include] ::
+sendemail.smtpServerPort annotate:bugreport[include] ::
+sendemail.smtpServerOption annotate:bugreport[include] ::
+sendemail.smtpUser annotate:bugreport[exclude] ::
+sendemail.thread annotate:bugreport[include] ::
+sendemail.transferEncoding annotate:bugreport[include] ::
+sendemail.validate annotate:bugreport[include] ::
+sendemail.xmailer annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.
 
 sendemail.signedoffcc (deprecated)::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize annotate:bugreport[include] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay annotate:bugreport[include] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index a01a050aa3..2bc9f112ea 100644
--- a/Makefile
+++ b/Makefile
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..2794a8ae77
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the annotate-bugreport macro
+sed -n 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p' \
+	| sort
+
+cat <<EOF
+};
+EOF
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 10/15] bugreport: add config values from safelist
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (8 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 09/15] bugreport: generate config safelist based on docs Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 11/15] bugreport: collect list of populated hooks Emily Shaffer
                         ` (6 subsequent siblings)
  16 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach bugreport to gather the values of config options which are present
in 'bugreport-config-safelist.h', and show their origin scope.

Many config options are sensitive, and many Git add-ons use config
options which git-core does not know about; it is better only to gather
config options which we know to be safe, rather than excluding options
which we know to be unsafe.

Rather than including the path to someone's config, which can reveal
filesystem layout and project names, just name the scope (e.g. system,
global, local) of the config source.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 Makefile                        |  2 ++
 bugreport.c                     | 34 +++++++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 23265b0d74..4e9171d1bd 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -30,6 +30,7 @@ The following information is captured automatically:
  - Compiler-specific info string
  - 'git remote-https --build-info'
  - $SHELL
+ - Selected config values
 
 OPTIONS
 -------
diff --git a/Makefile b/Makefile
index 2bc9f112ea..7b139230ae 100644
--- a/Makefile
+++ b/Makefile
@@ -2131,6 +2131,8 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
+bugreport.sp bugreport.s bugreport.o: bugreport-config-safelist.h
+
 builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
diff --git a/bugreport.c b/bugreport.c
index c1475601bf..1a9430a857 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -52,6 +55,29 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	size_t idx;
+	struct string_list_item *it = NULL;
+	struct key_value_info *kv_info = NULL;
+
+	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
+		const struct string_list *list =
+			git_config_get_value_multi(bugreport_config_safelist[idx]);
+
+		if (!list)
+			continue;
+
+		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
+		for_each_string_list_item(it, list) {
+			kv_info = it->util;
+			strbuf_addf(config_info, "  %s (%s)\n", it->string,
+				    kv_info ? config_scope_name(kv_info->scope)
+					    : "source unknown");
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -94,6 +120,7 @@ int cmd_main(int argc, const char **argv)
 	char *option_output = NULL;
 	char *option_suffix = "%F-%H%M";
 	struct stat statbuf;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output-directory", &option_output, N_("path"),
@@ -102,6 +129,10 @@ int cmd_main(int argc, const char **argv)
 			   N_("specify a strftime format suffix for the filename")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -132,6 +163,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 11/15] bugreport: collect list of populated hooks
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (9 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 10/15] bugreport: add config values from safelist Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 12/15] bugreport: count loose objects Emily Shaffer
                         ` (5 subsequent siblings)
  16 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific
hook which is being run, perhaps without the user realizing. While the
contents of hooks can be sensitive - containing user data or process
information specific to the user's organization - simply knowing that a
hook is being run at a certain stage can help us to understand whether
something is going wrong.

Without a definitive list of hook names within the code, we compile our
own list from the documentation. This is likely prone to bitrot. To
reduce the amount of code humans need to read, we turn the list into a
string_list and iterate over it (as we are calling the same find_hook
operation on each string). However, since bugreport should primarily be
called by the user, the performance loss from massaging the string
seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 53 +++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4e9171d1bd..779c9c7fd9 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -31,6 +31,7 @@ The following information is captured automatically:
  - 'git remote-https --build-info'
  - $SHELL
  - Selected config values
+ - A list of enabled hooks
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 1a9430a857..7e6147a935 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -78,6 +79,55 @@ static void get_safelisted_config(struct strbuf *config_info)
 	}
 }
 
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hooks`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -166,6 +216,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Enabled Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 12/15] bugreport: count loose objects
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (10 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 11/15] bugreport: collect list of populated hooks Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 13/15] bugreport: add packed object summary Emily Shaffer
                         ` (4 subsequent siblings)
  16 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Johannes Schindelin

The number of unpacked objects in a user's repository may help us
understand the root of the problem they're seeing, especially if a
command is running unusually slowly.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 779c9c7fd9..7e1ed44cb4 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 7e6147a935..f5201ce416 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
+
+	int local_loose_object_count = 0, total_loose_object_count = 0;
+	int local_count_questionable = 0, total_count_questionable = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	local_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&local_loose_object_count,
+					FOR_EACH_OBJECT_LOCAL_ONLY);
+
+	total_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&total_loose_object_count,
+					0);
+
+	strbuf_addf(obj_info, "%d local loose objects%s\n",
+		    local_loose_object_count,
+		    local_count_questionable ? " (problem during count)" : "");
+
+	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
+		    total_loose_object_count - local_loose_object_count,
+		    (local_count_questionable || total_count_questionable)
+			? " (problem during count)"
+			: "");
+
+	strbuf_addf(obj_info, "%d total loose objects%s\n",
+		    total_loose_object_count,
+		    total_count_questionable ? " (problem during count)" : "");
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -219,6 +268,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Enabled Hooks");
 	get_populated_hooks(&buffer, nongit_ok);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 13/15] bugreport: add packed object summary
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (11 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 12/15] bugreport: count loose objects Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
                         ` (3 subsequent siblings)
  16 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Alongside the loose object counts, it can be useful to show the number
of packs and packed objects. This way we can check whether the repo has
an appropriate ratio of packed to loose objects to help determine
whether it's behaving correctly.

Add a utility to easily traverse all packfiles in a given repository.
Use it in packfile.c and remove a redundant call to
prepare_packed_git(), which is already called in get_all_packs().

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 30 ++++++++++++++++++++++++++++++
 object-store.h                  |  6 ++++++
 packfile.c                      |  3 +--
 4 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 7e1ed44cb4..309b34fc42 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -33,6 +33,7 @@ The following information is captured automatically:
  - Selected config values
  - A list of enabled hooks
  - The number of loose objects in the repository
+ - The number of packs and packed objects in the repository
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index f5201ce416..580de13ddc 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -177,6 +177,33 @@ static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
+{
+	struct packed_git *pack = NULL;
+	int pack_count = 0;
+	int object_count = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	for_each_pack(the_repository, pack) {
+		pack_count++;
+		/*
+		 * To accurately count how many objects are packed, look inside
+		 * the packfile's index.
+		 */
+		open_pack_index(pack);
+		object_count += pack->num_objects;
+	}
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
+		    object_count);
+
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -271,6 +298,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/object-store.h b/object-store.h
index 55ee639350..518c73469d 100644
--- a/object-store.h
+++ b/object-store.h
@@ -6,6 +6,7 @@
 #include "list.h"
 #include "sha1-array.h"
 #include "strbuf.h"
+#include "packfile.h"
 
 struct object_directory {
 	struct object_directory *next;
@@ -403,4 +404,9 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(each_packed_object_fn, void *,
 			   enum for_each_object_flags flags);
 
+#define for_each_pack(repo, pack) 		\
+		for (pack = get_all_packs(repo);\
+		     pack;			\
+		     pack = pack->next)
+
 #endif /* OBJECT_STORE_H */
diff --git a/packfile.c b/packfile.c
index 7e7c04e4d8..6873827fb5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2061,8 +2061,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git(the_repository);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for_each_pack(the_repository, p) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (12 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 13/15] bugreport: add packed object summary Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14 17:04         ` Junio C Hamano
  2020-02-14  1:53       ` [PATCH v7 15/15] bugreport: summarize contents of alternates file Emily Shaffer
                         ` (2 subsequent siblings)
  16 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Miscellaneous information used about the object store can end up in
.git/objects/info; this can help us understand what may be going on with
the object store when the user is reporting a bug. Otherwise, it could
be difficult to track down what is going wrong with an object which
isn't kept locally to .git/objects/ or .git/objects/pack. Having some
understanding of where the user's objects may be kept can save us some
hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 54 +++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 309b34fc42..a21d081616 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -34,6 +34,7 @@ The following information is captured automatically:
  - A list of enabled hooks
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
+ - A list of the contents of .git/objects/info (or equivalent)
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 580de13ddc..1c67b55598 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -204,6 +204,57 @@ static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
 
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info, int nongit)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - object info unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -301,6 +352,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v7 15/15] bugreport: summarize contents of alternates file
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (13 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14 17:32       ` [PATCH v7 00/15] add git-bugreport tool Junio C Hamano
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
  16 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In some cases, it could be that the user is having a problem with an
object which isn't present in their normal object directory. We can get
a hint that that might be the case by examining the list of alternates
where their object may be stored instead. Since paths to alternates may
be sensitive, we'll instead count how many alternates have been
specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this
function does not provide information on broken alternates, which are
skipped over in "link_alt_odb_entry()". Since the goal is to identify
missing alternates, we can gather the contents of
.git/objects/info/alternates manually.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 45 +++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index a21d081616..e870900c80 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -35,6 +35,7 @@ The following information is captured automatically:
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
  - A list of the contents of .git/objects/info (or equivalent)
+ - The number of valid and invalid alternates
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 1c67b55598..3bc8cb3579 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -255,6 +255,48 @@ static void get_object_info_summary(struct strbuf *obj_info, int nongit)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	if (nongit) {
+		strbuf_addstr(alternates_info,
+			"not run from a git repository - alternates unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -355,6 +397,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info
  2020-02-14  1:53       ` [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
@ 2020-02-14 17:04         ` Junio C Hamano
  2020-02-18 23:59           ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-14 17:04 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> Miscellaneous information used about the object store can end up in
> .git/objects/info; this can help us understand what may be going on with
> the object store when the user is reporting a bug. Otherwise, it could
> be difficult to track down what is going wrong with an object which
> isn't kept locally to .git/objects/ or .git/objects/pack. Having some
> understanding of where the user's objects may be kept can save us some
> hops during the bug reporting process.

This step seems to have a new whitespace breakage that did not exist
in the previous round.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-14  1:53       ` [PATCH v7 03/15] bugreport: add tool to generate debugging info Emily Shaffer
@ 2020-02-14 17:25         ` Junio C Hamano
  2020-02-15  1:57           ` Emily Shaffer
  2020-02-19 14:18         ` Johannes Schindelin
  1 sibling, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-14 17:25 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +	switch (safe_create_leading_directories(report_path.buf)) {

This helper is about creating paths in the working tree and Git
repository, hence it has a call to adjust_shared_perm() which in
turn calls get_shared_repo(), i.e. requiring a repository.

I thought I read somewhere that this tool is meant to be usable
outside a repository?  If that is not the case, then the use of this
helper is OK.  If not, we may want to make sure that it will stay to
be safe to use the helper (I think it happens to be OK right now,
but if the reason why the user is trying to run the tool is because
the user broke Git by writing garbage into .git/config, we may
die("your configuration file is broken") before this helper returns).

Thanks.



^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 00/15] add git-bugreport tool
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (14 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 15/15] bugreport: summarize contents of alternates file Emily Shaffer
@ 2020-02-14 17:32       ` Junio C Hamano
  2020-02-14 22:00         ` Emily Shaffer
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
  16 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-14 17:32 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Derrick Stolee, Johannes Schindelin, Martin Ågren,
	Aaron Schrab, Danh Doan, Eric Sunshine, SZEDER Gábor,
	Andreas Schwab

Emily Shaffer <emilyshaffer@google.com> writes:

>    present(patch 9/15). This now uses POSIX character classes and {}
>    notation instead of + and has been tested on OSX; I'd love to hear if

I'd rather not to see unnecessary uses of POSIX character classes.

The interdiff of this step between the previous and this round looks
to me more like "I used it, just because POSIX says I *can* use it",
not "I did so because I needed to do, and it should be OK on POSIX
platforms."

Instead of overly long

's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p'

just limiting ourselves to SP and saying

's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p'

would keep the result much easier to read, I would think.


^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 00/15] add git-bugreport tool
  2020-02-14 17:32       ` [PATCH v7 00/15] add git-bugreport tool Junio C Hamano
@ 2020-02-14 22:00         ` Emily Shaffer
  2020-02-14 22:30           ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-14 22:00 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Derrick Stolee, Johannes Schindelin, Martin Ågren,
	Aaron Schrab, Danh Doan, Eric Sunshine, SZEDER Gábor,
	Andreas Schwab

On Fri, Feb 14, 2020 at 09:32:08AM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> >    present(patch 9/15). This now uses POSIX character classes and {}
> >    notation instead of + and has been tested on OSX; I'd love to hear if
> 
> I'd rather not to see unnecessary uses of POSIX character classes.
> 
> The interdiff of this step between the previous and this round looks
> to me more like "I used it, just because POSIX says I *can* use it",
> not "I did so because I needed to do, and it should be OK on POSIX
> platforms."
> 
> Instead of overly long
> 
> 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p'
> 
> just limiting ourselves to SP and saying
> 
> 's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p'
> 
> would keep the result much easier to read, I would think.

That's fine by me. I find the [[:syntax:]] extremely ugly, but I was
worried about whether that was more portable somehow. Your proposal
seems fine to me.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 00/15] add git-bugreport tool
  2020-02-14 22:00         ` Emily Shaffer
@ 2020-02-14 22:30           ` Junio C Hamano
  0 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-02-14 22:30 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Derrick Stolee, Johannes Schindelin, Martin Ågren,
	Aaron Schrab, Danh Doan, Eric Sunshine, SZEDER Gábor,
	Andreas Schwab

Emily Shaffer <emilyshaffer@google.com> writes:

> That's fine by me. I find the [[:syntax:]] extremely ugly,...

FWIW, I find it so too.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-14 17:25         ` Junio C Hamano
@ 2020-02-15  1:57           ` Emily Shaffer
  2020-02-15 18:24             ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-15  1:57 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Feb 14, 2020 at 09:25:12AM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +	switch (safe_create_leading_directories(report_path.buf)) {
> 
> This helper is about creating paths in the working tree and Git
> repository,

It's also used by cmd_format_patch() with --output-directory specified,
which is how I found it. My usual workflow is to run format-patch with
-o ~/mailed-patches/topic/ specified so I don't clutter my repo, so I'm
surprised to hear the intent of safe_create_leading_directores() is for
paths in the working tree or repo.

> hence it has a call to adjust_shared_perm() which in
> turn calls get_shared_repo(), i.e. requiring a repository.

Hmmm. I was able to run it pretty happily from a nongit dir:

  emilyshaffer@podkayne:~$ tg bugreport -o other/very/deep/path/
  Created new report at 'other/very/deep/path/git-bugreport-2020-02-14-1721.txt'.
  emilyshaffer@podkayne:~$ cat other/very/deep/path/git-bugreport-2020-02-14-1721.txt
  Thank you for filling out a Git bug report!

> I thought I read somewhere that this tool is meant to be usable
> outside a repository?  If that is not the case, then the use of this
> helper is OK.  If not, we may want to make sure that it will stay to
> be safe to use the helper (I think it happens to be OK right now,
> but if the reason why the user is trying to run the tool is because
> the user broke Git by writing garbage into .git/config, we may
> die("your configuration file is broken") before this helper returns).

Can you explain a little more about what you mean? A broken local
.git/config seems to be preventing git.c from dispatching the
'bugreport' subcommand (lots of other Git commands are broken) at all -
breakpoints in cmd_main() right after the variable declarations are not
being hit.

With junk in the .git/config, I can't run bugreport from within that
repo. With junk in ~/.gitconfig, I can't run bugreport anywhere.

(To make the configs invalid, I leaned on my keyboard and added a line
'garbaghe~*~$%)%)(@' to the bottom of the config file. Most commands
terminate early with "fatal: bad config line 37 in file .git/config".)

Do you mean there's some specific config that could be misconfigured and
prevent that utility from working?

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-15  1:57           ` Emily Shaffer
@ 2020-02-15 18:24             ` Junio C Hamano
  2020-02-18 23:46               ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-15 18:24 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> On Fri, Feb 14, 2020 at 09:25:12AM -0800, Junio C Hamano wrote:
>> Emily Shaffer <emilyshaffer@google.com> writes:
>> 
>> > +	switch (safe_create_leading_directories(report_path.buf)) {
>> 
>> This helper is about creating paths in the working tree and Git
>> repository,
>
> It's also used by cmd_format_patch() with --output-directory specified,
> which is how I found it.

And that is an example of a good use of this helper.  What will be
written out there should be visible by the same people as those who
have access to the repository; get_shared_repo() and adjust_perm()
based on what the repository you are taking patches from is
perfectly sensible.  And as it is format-patch, we know we have
get_shared_repo() working, and we know which repository we are
working on.

Output directory for bugreport is on the same boat when we know the
users are producing a report on their use of Git within a
repository.  It is not clear if the tool is started without any
repository---it happens to do a random safe thing (i.e. I think
get_shared_repo() gives PERM_UMASK, which tells adjust_perm() not to
do anything especial) right now, but there is no guarantee that we
will keep it working like that.  Somebody may come and demaind
get_shared_perm() to die() when outside a repository, for example,
and that would break the nice property that lets bugreport working
outside a repository.

I just wanted to make sure that somebody will be keeping an eye to
remind those who propose such a change in the future.  A comment
near where get_shared_repo() happens may be something that can be
done with a minimum effort when code that depends on that property
is introduced at the same time.

>> I thought I read somewhere that this tool is meant to be usable
>> outside a repository?  If that is not the case, then the use of this
>> helper is OK.  If not, we may want to make sure that it will stay to
>> be safe to use the helper (I think it happens to be OK right now,

I am actually OK if we limit the use of this tool to "use with a
repository that is not corrupt", as coping with these kinds of
breakages that in the main Git executable we deem "needs manual
intervention" inside a single process is too painful (it would have
been easier to cope with these too if we stuck with a script that
invokes many discrete commands and acts on their errors, but that is
optimizing for rare case and not recommended).  But we should tell
users about the limitation and encourage them to ask for help in non
automatable means.

Thanks.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-15 18:24             ` Junio C Hamano
@ 2020-02-18 23:46               ` Emily Shaffer
  2020-02-18 23:56                 ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-18 23:46 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Sat, Feb 15, 2020 at 10:24:40AM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > On Fri, Feb 14, 2020 at 09:25:12AM -0800, Junio C Hamano wrote:
> >> Emily Shaffer <emilyshaffer@google.com> writes:
> >> 
> >> > +	switch (safe_create_leading_directories(report_path.buf)) {
> >> 
> >> This helper is about creating paths in the working tree and Git
> >> repository,
> >
> > It's also used by cmd_format_patch() with --output-directory specified,
> > which is how I found it.
> 
> And that is an example of a good use of this helper.  What will be
> written out there should be visible by the same people as those who
> have access to the repository; get_shared_repo() and adjust_perm()
> based on what the repository you are taking patches from is
> perfectly sensible.  And as it is format-patch, we know we have
> get_shared_repo() working, and we know which repository we are
> working on.
> 
> Output directory for bugreport is on the same boat when we know the
> users are producing a report on their use of Git within a
> repository.  It is not clear if the tool is started without any
> repository---it happens to do a random safe thing (i.e. I think
> get_shared_repo() gives PERM_UMASK, which tells adjust_perm() not to
> do anything especial) right now, but there is no guarantee that we
> will keep it working like that.  Somebody may come and demaind
> get_shared_perm() to die() when outside a repository, for example,
> and that would break the nice property that lets bugreport working
> outside a repository.

Hm, this would break the convention of the name of
safe_create_leading_directories() too though right? As I understood it,
"safe_foo" in this codebase means "this function will not die()"?

> 
> I just wanted to make sure that somebody will be keeping an eye to
> remind those who propose such a change in the future.  A comment
> near where get_shared_repo() happens may be something that can be
> done with a minimum effort when code that depends on that property
> is introduced at the same time.

Ok. I want to make sure I understand you right. I think you're saying,
"This is OK, except if someone changes get_shared_repo() it could break,
so we need a way to warn someone that it could break." It sounds like
you suggested leaving a comment in the
safe_create_leading_directories() helper. My own preference is to write
a test so that it's explicit that we depend on that behavior, instead -
it's easy to gloss over a comment or read a different part of the
codebase, but it's hard to ignore a breaking test. It'd be trivial to
add one, so I will in v8 - unless I misunderstood you.

> 
> >> I thought I read somewhere that this tool is meant to be usable
> >> outside a repository?  If that is not the case, then the use of this
> >> helper is OK.  If not, we may want to make sure that it will stay to
> >> be safe to use the helper (I think it happens to be OK right now,
> 
> I am actually OK if we limit the use of this tool to "use with a
> repository that is not corrupt", as coping with these kinds of
> breakages that in the main Git executable we deem "needs manual
> intervention" inside a single process is too painful (it would have
> been easier to cope with these too if we stuck with a script that
> invokes many discrete commands and acts on their errors, but that is
> optimizing for rare case and not recommended).  But we should tell
> users about the limitation and encourage them to ask for help in non
> automatable means.

I think you're saying, "Mention this drawback in the manpage for
git-bugreport." Sounds like a good idea to me, so I'll add it for v8.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-18 23:46               ` Emily Shaffer
@ 2020-02-18 23:56                 ` Emily Shaffer
  2020-02-19 23:15                   ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-18 23:56 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Feb 18, 2020 at 03:46:28PM -0800, Emily Shaffer wrote:
> On Sat, Feb 15, 2020 at 10:24:40AM -0800, Junio C Hamano wrote:
> > I am actually OK if we limit the use of this tool to "use with a
> > repository that is not corrupt", as coping with these kinds of
> > breakages that in the main Git executable we deem "needs manual
> > intervention" inside a single process is too painful (it would have
> > been easier to cope with these too if we stuck with a script that
> > invokes many discrete commands and acts on their errors, but that is
> > optimizing for rare case and not recommended).  But we should tell
> > users about the limitation and encourage them to ask for help in non
> > automatable means.
> 
> I think you're saying, "Mention this drawback in the manpage for
> git-bugreport." Sounds like a good idea to me, so I'll add it for v8.

I'm pretty unsure about how you wanted this to sound; rather than
sending a v8 only to revise it a lot, I'll send the paragraph for
wordsmithing beforehand. Is this what you meant?

  This tool is invoked via the typical Git setup process, which means that in some
  cases, it might not be able to launch - for example, if a relevant config file
  is unreadable. In this kind of scenario, it may be helpful to manually gather
  the kind of information listed above when manually asking for help.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info
  2020-02-14 17:04         ` Junio C Hamano
@ 2020-02-18 23:59           ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-18 23:59 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Feb 14, 2020 at 09:04:05AM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Miscellaneous information used about the object store can end up in
> > .git/objects/info; this can help us understand what may be going on with
> > the object store when the user is reporting a bug. Otherwise, it could
> > be difficult to track down what is going wrong with an object which
> > isn't kept locally to .git/objects/ or .git/objects/pack. Having some
> > understanding of where the user's objects may be kept can save us some
> > hops during the bug reporting process.
> 
> This step seems to have a new whitespace breakage that did not exist
> in the previous round.

Hmph. I saw it warned at some point during the rebase-and-format-patch
workflow for v7 but thought I had quashed it. Very sorry.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-14  1:53       ` [PATCH v7 03/15] bugreport: add tool to generate debugging info Emily Shaffer
  2020-02-14 17:25         ` Junio C Hamano
@ 2020-02-19 14:18         ` Johannes Schindelin
  2020-02-19 16:55           ` Junio C Hamano
  1 sibling, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-02-19 14:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 13 Feb 2020, Emily Shaffer wrote:

> diff --git a/bugreport.c b/bugreport.c
> new file mode 100644
> index 0000000000..a9398e6a2a
> --- /dev/null
> +++ b/bugreport.c
> @@ -0,0 +1,94 @@
> +#include "builtin.h"
> +#include "parse-options.h"
> +#include "stdio.h"
> +#include "strbuf.h"
> +#include "time.h"
> +
> +static const char * const bugreport_usage[] = {
> +	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
> +	NULL
> +};
> +
> +static int get_bug_template(struct strbuf *template)
> +{
> +	const char template_text[] = N_(
> +"Thank you for filling out a Git bug report!\n"
> +"Please answer the following questions to help us understand your issue.\n"
> +"\n"
> +"What did you do before the bug happened? (Steps to reproduce your issue)\n"
> +"\n"
> +"What did you expect to happen? (Expected behavior)\n"
> +"\n"
> +"What happened instead? (Actual behavior)\n"
> +"\n"
> +"What's different between what you expected and what actually happened?\n"
> +"\n"
> +"Anything else you want to add:\n"
> +"\n"
> +"Please review the rest of the bug report below.\n"
> +"You can delete any lines you don't wish to share.\n");
> +
> +	strbuf_addstr(template, template_text);
> +	return 0;
> +}
> +
> +int cmd_main(int argc, const char **argv)
> +{
> +	struct strbuf buffer = STRBUF_INIT;
> +	struct strbuf report_path = STRBUF_INIT;
> +	FILE *report;
> +	time_t now = time(NULL);
> +	char *option_output = NULL;
> +	char *option_suffix = "%F-%H%M";
> +	struct stat statbuf;
> +
> +	const struct option bugreport_options[] = {
> +		OPT_STRING('o', "output-directory", &option_output, N_("path"),
> +			   N_("specify a destination for the bugreport file")),
> +		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
> +			   N_("specify a strftime format suffix for the filename")),
> +		OPT_END()
> +	};
> +	argc = parse_options(argc, argv, "", bugreport_options,
> +			     bugreport_usage, 0);
> +
> +	if (option_output) {
> +		strbuf_addstr(&report_path, option_output);
> +		strbuf_complete(&report_path, '/');
> +	}
> +
> +
> +	strbuf_addstr(&report_path, "git-bugreport-");
> +	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
> +	strbuf_addstr(&report_path, ".txt");
> +
> +	if (!stat(report_path.buf, &statbuf))
> +		die("'%s' already exists", report_path.buf);
> +
> +	switch (safe_create_leading_directories(report_path.buf)) {
> +	case SCLD_OK:
> +	case SCLD_EXISTS:
> +		break;
> +	default:
> +		die(_("could not create leading directories for '%s'"),
> +		    report_path.buf);
> +	}
> +
> +	get_bug_template(&buffer);
> +
> +	report = fopen_for_writing(report_path.buf);
> +
> +	if (report == NULL) {
> +		strbuf_release(&report_path);
> +		die("couldn't open '%s' for writing", report_path.buf);
> +	}
> +
> +	strbuf_write(&buffer, report);
> +	fclose(report);
> +
> +	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
> +
> +	UNLEAK(buffer);
> +	UNLEAK(report_path);
> +	return -launch_editor(report_path.buf, NULL, NULL);

This would be the first time (at least that _I_ know of) that we use `-`
in this way. We seem to use `!!` a lot more often. And now I wonder
whether there is a reason for that `-` that I missed?

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 06/15] bugreport: add compiler info
  2020-02-14  1:53       ` [PATCH v7 06/15] bugreport: add compiler info Emily Shaffer
@ 2020-02-19 14:23         ` Johannes Schindelin
  2020-02-19 22:45           ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-02-19 14:23 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 13 Feb 2020, Emily Shaffer wrote:

> To help pinpoint the source of a regression, it is useful to know some
> info about the compiler which the user's Git client was built with. By
> adding a generic get_compiler_info() in 'compat/' we can choose which
> relevant information to share per compiler; to get started, let's
> demonstrate the version of glibc if the user built with 'gcc'.

I agree with the need for the compiler information, but in the patch I
only see information about glibc being printed out. Shouldn't we use
`__GNUC__` and `__GNUC_MINOR__` here?

Don't get me wrong, the glibc version is good and all, but the compiler
information might be even more crucial. Git for Windows had to hold back
compiling with GCC v8.x for a while, for example, because the stacksmasher
was broken. Similar issues are not unheard of, and could help pinpoint
compiler-related problems a lot quicker.

Thanks,
Dscho

>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  Documentation/git-bugreport.txt |  1 +
>  bugreport.c                     |  5 +++++
>  compat/compiler.h               | 24 ++++++++++++++++++++++++
>  3 files changed, 30 insertions(+)
>  create mode 100644 compat/compiler.h
>
> diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
> index 4dd72c60f5..8bbc4c960c 100644
> --- a/Documentation/git-bugreport.txt
> +++ b/Documentation/git-bugreport.txt
> @@ -27,6 +27,7 @@ The following information is captured automatically:
>
>   - 'git version --build-options'
>   - uname sysname, release, version, and machine strings
> + - Compiler-specific info string
>
>  OPTIONS
>  -------
> diff --git a/bugreport.c b/bugreport.c
> index b76a1dfb2a..4f9101caeb 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -4,6 +4,7 @@
>  #include "strbuf.h"
>  #include "time.h"
>  #include "help.h"
> +#include "compat/compiler.h"
>
>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
>  			    uname_info.release,
>  			    uname_info.version,
>  			    uname_info.machine);
> +
> +	strbuf_addstr(sys_info, "compiler info: ");
> +	get_compiler_info(sys_info);
> +	strbuf_complete_line(sys_info);
>  }
>
>  static const char * const bugreport_usage[] = {
> diff --git a/compat/compiler.h b/compat/compiler.h
> new file mode 100644
> index 0000000000..bda5098e1b
> --- /dev/null
> +++ b/compat/compiler.h
> @@ -0,0 +1,24 @@
> +#ifndef COMPILER_H
> +#define COMPILER_H
> +
> +#include "git-compat-util.h"
> +#include "strbuf.h"
> +
> +#ifdef __GLIBC__
> +#include <gnu/libc-version.h>
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
> +}
> +
> +#else
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +	strbuf_addstr(info, "get_compiler_info() not implemented");
> +}
> +
> +#endif
> +
> +#endif /* COMPILER_H */
> --
> 2.25.0.265.gbab2e86ba0-goog
>
>

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 07/15] bugreport: add git-remote-https version
  2020-02-14  1:53       ` [PATCH v7 07/15] bugreport: add git-remote-https version Emily Shaffer
@ 2020-02-19 14:28         ` Johannes Schindelin
  2020-02-19 22:28           ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-02-19 14:28 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 13 Feb 2020, Emily Shaffer wrote:

> diff --git a/bugreport.c b/bugreport.c
> index 4f9101caeb..bfdff33368 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -5,6 +5,18 @@
>  #include "time.h"
>  #include "help.h"
>  #include "compat/compiler.h"
> +#include "run-command.h"
> +
> +static void get_git_remote_https_version_info(struct strbuf *version_info)
> +{
> +	struct child_process cp = CHILD_PROCESS_INIT;
> +
> +	argv_array_push(&cp.args, "git");
> +	argv_array_push(&cp.args, "remote-https");
> +	argv_array_push(&cp.args, "--build-info");
> +	if (capture_command(&cp, version_info, 0))

Let's use RUN_GIT_CMD instead of adding `"git"` explicitly; It documents
that we're interested in a Git command, and if we ever build a
single-binary version of Git (as some Git for Windows users already asked
for), it will make things easier.

> +	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
> +}
>
>  static void get_system_info(struct strbuf *sys_info)
>  {
> diff --git a/remote-curl.c b/remote-curl.c
> index 350d92a074..c590fbfae3 100644
> --- a/remote-curl.c
> +++ b/remote-curl.c
> @@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
>  	string_list_init(&options.deepen_not, 1);
>  	string_list_init(&options.push_options, 1);
>
> +	if (!strcmp("--build-info", argv[1])) {

The context does not say this, but at this point, we already verified that
`argc` is larger than 1. Good.

Also, in keeping with the existing code, we would need to use
`--build-options` here (this is what `git version` calls the equivalent
mode).

_However_.

I like your `--build-info` a lot more than `--build-options` (because the
latter is very misleading: the commit and the date of the build are not
"options" at all).

Thanks,
Dscho

> +		printf("git-http-fetch version: %s\n", git_version_string);
> +		printf("built from commit: %s\n", git_built_from_commit_string);
> +		printf("curl version: %s\n", curl_version());
> +		return 0;
> +	}
> +
>  	/*
>  	 * Just report "remote-curl" here (folding all the various aliases
>  	 * ("git-remote-http", "git-remote-https", and etc.) here since they
> --
> 2.25.0.265.gbab2e86ba0-goog
>
>

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-19 14:18         ` Johannes Schindelin
@ 2020-02-19 16:55           ` Junio C Hamano
  2020-02-19 21:52             ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-19 16:55 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Emily Shaffer, git

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> +int cmd_main(int argc, const char **argv)
>> +{
>> +...
>> +	if (report == NULL) {
>> +		strbuf_release(&report_path);
>> +		die("couldn't open '%s' for writing", report_path.buf);
>> +	}
>> +
>> +	strbuf_write(&buffer, report);
>> +	fclose(report);
>> +
>> +	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
>> +
>> +	UNLEAK(buffer);
>> +	UNLEAK(report_path);
>> +	return -launch_editor(report_path.buf, NULL, NULL);
>
> This would be the first time (at least that _I_ know of) that we use `-`
> in this way. We seem to use `!!` a lot more often. And now I wonder
> whether there is a reason for that `-` that I missed?

In general, our preferred way to report an error from API functions
is to return a negative number and a success is reported by
returning zero.

The argument to exit(3), which the return value from the main()
function essentially is, on the other hand, is expected to be a
small non-negative integer.  As long as we are in tight control of
the range of the returned value from launch_editor() (i.e. it must
return a small non-positive integer whose negation is suitable to be
fed to exit(3)), the above is fine.

The idiom "return !!fn();" is to canonicalize the value to 0 or 1
for the caller who asked "so, did fn() give us 0 or non-zero?",
which can also be used as the value given to exit(3), and could be
more appropriate under some conditions:

 - we MUST know launch_editor() returns zero if and only if it is
   successful.

 - we do not have to be confident to be in tight control of the
   range of the returned value from launch_editor() (e.g. it could
   return a positive upon an error).

 - we MUST NOT care to differenciate different error codes returned
   from launch_editor().  IOW, we must be fine to give the invoker
   of the program only 0 (success) or 1 (unspecified failure).

Use of "return !!fn();" that is not to give to exit(3) is probably
more common in our codebase.  See apply.c::try_create_file() and the
comment before the function about its possible return values for
example.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-19 16:55           ` Junio C Hamano
@ 2020-02-19 21:52             ` Emily Shaffer
  2020-02-19 22:09               ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-19 21:52 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Johannes Schindelin, git

On Wed, Feb 19, 2020 at 08:55:04AM -0800, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> +int cmd_main(int argc, const char **argv)
> >> +{
> >> +...
> >> +	if (report == NULL) {
> >> +		strbuf_release(&report_path);
> >> +		die("couldn't open '%s' for writing", report_path.buf);
> >> +	}
> >> +
> >> +	strbuf_write(&buffer, report);
> >> +	fclose(report);
> >> +
> >> +	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
> >> +
> >> +	UNLEAK(buffer);
> >> +	UNLEAK(report_path);
> >> +	return -launch_editor(report_path.buf, NULL, NULL);
> >
> > This would be the first time (at least that _I_ know of) that we use `-`
> > in this way. We seem to use `!!` a lot more often. And now I wonder
> > whether there is a reason for that `-` that I missed?
> 
> In general, our preferred way to report an error from API functions
> is to return a negative number and a success is reported by
> returning zero.
> 
> The argument to exit(3), which the return value from the main()
> function essentially is, on the other hand, is expected to be a
> small non-negative integer.  As long as we are in tight control of
> the range of the returned value from launch_editor() (i.e. it must
> return a small non-positive integer whose negation is suitable to be
> fed to exit(3)), the above is fine.
> 
> The idiom "return !!fn();" is to canonicalize the value to 0 or 1
> for the caller who asked "so, did fn() give us 0 or non-zero?",
> which can also be used as the value given to exit(3), and could be
> more appropriate under some conditions:

In editor.c, launch_editor() returns launch_specified_editor() without
altering the return code.

> 
>  - we MUST know launch_editor() returns zero if and only if it is
>    successful.

launch_specified_editor() has a handful of exit points, of three kinds:
 1. return error(something)
 2. raise(sigsomething)
 3. return 0
    a. when the editor process closed happily, but the user supplied
       NULL instead of a buffer. That is, the user didn't want the
       contents of the editor given back to them in a strbuf.
    b. when the editor process closed happily and the user's supplied
       buffer was filled with the file's contents with no issue.

So I think we can check "yes" here.

> 
>  - we do not have to be confident to be in tight control of the
>    range of the returned value from launch_editor() (e.g. it could
>    return a positive upon an error).

According to 'man 3 raise' (POSIX), raise() exits the current thread. If it
can't exit itself(?) it seems it returns "nonzero for failure". We've
also got a mingw_raise() in compat/mingw.h which could be used instead;
this one seems to return -1 or the result of raise(), presumably the one
from the MSC runtime[1], which also returns "a nonzero value" if not
successful.

So it's true that we aren't confident that launch_editor() returns a
positive or negative value.

>  - we MUST NOT care to differenciate different error codes returned
>    from launch_editor().  IOW, we must be fine to give the invoker
>    of the program only 0 (success) or 1 (unspecified failure).

This part is a little sticking point for me; I think I'd rather give the
user some hint of what's broken than no hint at all, especially in this
scenario, where it's conceivable someone could say "I tried to run
'git-bugreport' and it crashed, here is the return code and I have
manually collected some relevant info too".

The uncertainty coming from raise() (POSIX and MSCR both) leads me to
believe if I do want to return the error on exit, I should at least
check the sign before I flip it. Anybody else have opinions?

 - Emily

[1] https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/raise?view=vs-2019

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-19 21:52             ` Emily Shaffer
@ 2020-02-19 22:09               ` Junio C Hamano
  2020-02-19 23:06                 ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-19 22:09 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Johannes Schindelin, git

Emily Shaffer <emilyshaffer@google.com> writes:

> launch_specified_editor() has a handful of exit points, of three kinds:
>  1. return error(something)
>  2. raise(sigsomething)
>  3. return 0
>     a. when the editor process closed happily, but the user supplied
>        NULL instead of a buffer. That is, the user didn't want the
>        contents of the editor given back to them in a strbuf.
>     b. when the editor process closed happily and the user's supplied
>        buffer was filled with the file's contents with no issue.
>
> So I think we can check "yes" here.

Heh.  If we raised a signal to kill ourselves, then we won't be
returning a value from launch_editor() anyway.  That case won't
affect the "between returning negation or !!, which is more
appropriate?" discussion, I think.

>>  - we MUST NOT care to differenciate different error codes returned
>>    from launch_editor().  IOW, we must be fine to give the invoker
>>    of the program only 0 (success) or 1 (unspecified failure).

I actually think this holds for the codepath.  A failure from
start_command() returns error(), and finish_command() that waits for
the spawned editor process to complete yields the exit status from
the editor, but unless we re-raise the signal that killed the editor
process to ourselves, we just turn any non-zero exit into "return
error()", so it is safe to say launch_editor() can return either 0
or -1 and nothing else.  Would we later want to tell callers of
launch_editor() how/why the editor session failed by returning
something else?  I do not offhand think of any---we do not even
differenciate between failure to start (e.g. misspelt command name
for the editor) and other failures WITH the return value and
consider it sufficient to tell the user with different error
message right now.

So in practice returning -launch_editor() and !!launch_editor()
would not make any difference, I would think.


^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 07/15] bugreport: add git-remote-https version
  2020-02-19 14:28         ` Johannes Schindelin
@ 2020-02-19 22:28           ` Emily Shaffer
  2020-02-19 22:33             ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-19 22:28 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git

On Wed, Feb 19, 2020 at 03:28:35PM +0100, Johannes Schindelin wrote:
> Hi Emily,
> 
> On Thu, 13 Feb 2020, Emily Shaffer wrote:
> 
> > diff --git a/bugreport.c b/bugreport.c
> > index 4f9101caeb..bfdff33368 100644
> > --- a/bugreport.c
> > +++ b/bugreport.c
> > @@ -5,6 +5,18 @@
> >  #include "time.h"
> >  #include "help.h"
> >  #include "compat/compiler.h"
> > +#include "run-command.h"
> > +
> > +static void get_git_remote_https_version_info(struct strbuf *version_info)
> > +{
> > +	struct child_process cp = CHILD_PROCESS_INIT;
> > +
> > +	argv_array_push(&cp.args, "git");
> > +	argv_array_push(&cp.args, "remote-https");
> > +	argv_array_push(&cp.args, "--build-info");
> > +	if (capture_command(&cp, version_info, 0))
> 
> Let's use RUN_GIT_CMD instead of adding `"git"` explicitly; It documents
> that we're interested in a Git command, and if we ever build a
> single-binary version of Git (as some Git for Windows users already asked
> for), it will make things easier.

Hm. RUN_GIT_CMD is an argument used for the run_command_v_opt* family of
calls, but it seems that setting child_process.git_cmd has the same
effect. Done.

> 
> > +	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
> > +}
> >
> >  static void get_system_info(struct strbuf *sys_info)
> >  {
> > diff --git a/remote-curl.c b/remote-curl.c
> > index 350d92a074..c590fbfae3 100644
> > --- a/remote-curl.c
> > +++ b/remote-curl.c
> > @@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
> >  	string_list_init(&options.deepen_not, 1);
> >  	string_list_init(&options.push_options, 1);
> >
> > +	if (!strcmp("--build-info", argv[1])) {
> 
> The context does not say this, but at this point, we already verified that
> `argc` is larger than 1. Good.
> 
> Also, in keeping with the existing code, we would need to use
> `--build-options` here (this is what `git version` calls the equivalent
> mode).
> 
> _However_.
> 
> I like your `--build-info` a lot more than `--build-options` (because the
> latter is very misleading: the commit and the date of the build are not
> "options" at all).

Sure. Thanks for saying so.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 07/15] bugreport: add git-remote-https version
  2020-02-19 22:28           ` Emily Shaffer
@ 2020-02-19 22:33             ` Junio C Hamano
  2020-02-20 22:33               ` Johannes Schindelin
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-19 22:33 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Johannes Schindelin, git

Emily Shaffer <emilyshaffer@google.com> writes:

>> Also, in keeping with the existing code, we would need to use
>> `--build-options` here (this is what `git version` calls the equivalent
>> mode).
>> 
>> _However_.
>> 
>> I like your `--build-info` a lot more than `--build-options` (because the
>> latter is very misleading: the commit and the date of the build are not
>> "options" at all).
>
> Sure. Thanks for saying so.

I don't think anybody would mind introducing --build-info to "git
version" as a synonym and deprecate --build-options from it ;-)

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 06/15] bugreport: add compiler info
  2020-02-19 14:23         ` Johannes Schindelin
@ 2020-02-19 22:45           ` Emily Shaffer
  2020-02-20 22:33             ` Johannes Schindelin
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-19 22:45 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git

On Wed, Feb 19, 2020 at 03:23:34PM +0100, Johannes Schindelin wrote:
> Hi Emily,
> 
> On Thu, 13 Feb 2020, Emily Shaffer wrote:
> 
> > To help pinpoint the source of a regression, it is useful to know some
> > info about the compiler which the user's Git client was built with. By
> > adding a generic get_compiler_info() in 'compat/' we can choose which
> > relevant information to share per compiler; to get started, let's
> > demonstrate the version of glibc if the user built with 'gcc'.
> 
> I agree with the need for the compiler information, but in the patch I
> only see information about glibc being printed out. Shouldn't we use
> `__GNUC__` and `__GNUC_MINOR__` here?
> 
> Don't get me wrong, the glibc version is good and all, but the compiler
> information might be even more crucial. Git for Windows had to hold back
> compiling with GCC v8.x for a while, for example, because the stacksmasher
> was broken. Similar issues are not unheard of, and could help pinpoint
> compiler-related problems a lot quicker.

Hm, sure. Good point - thanks.

This does make me start to wonder, though - does it really make sense to
have ifdef-gated redefinitions of the whole get_compiler_info() method
like I do now? I wonder if it makes more sense to have only one
definition, so we can write down everything we know regardless of which
pieces are put together. My thinking is something like this - what if I
am using glibc, but not a GNU compiler? (The GNU docs on __GCC__
indicate this is a situation that might occur - "a non-GCC compiler that
claims to accept the GNU C dialects") Is there some striking reason not
to implement this compat command thusly instead:

  #ifdef __GLIBC__
  #include <gnu/libc-version.h>
  #endif

  static inline void get_compiler_info(struct strbuf *info)
  {
  	#ifdef __GLIBC__
	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
	#endif

	#ifdef __GNUC__
	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
	#endif

	#ifdef _MSC_VER
	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());
	#endif
  }

The thinking being - this way if I decide to use, say, LLVM + glibc,
then I don't need to reimplement this command with all the glibc
diagnostics again. Or, if someone else already wrote diagnostics for
LLVM with some other libc, then it even Just Works for me and my new
combination.

That said, I'm reasoning about these combinations of compilers and libcs
and whatever else from an inexperienced viewpoint, so maybe this isn't
necessary?

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-19 22:09               ` Junio C Hamano
@ 2020-02-19 23:06                 ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-19 23:06 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Johannes Schindelin, git

On Wed, Feb 19, 2020 at 02:09:48PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > launch_specified_editor() has a handful of exit points, of three kinds:
> >  1. return error(something)
> >  2. raise(sigsomething)
> >  3. return 0
> >     a. when the editor process closed happily, but the user supplied
> >        NULL instead of a buffer. That is, the user didn't want the
> >        contents of the editor given back to them in a strbuf.
> >     b. when the editor process closed happily and the user's supplied
> >        buffer was filled with the file's contents with no issue.
> >
> > So I think we can check "yes" here.
> 
> Heh.  If we raised a signal to kill ourselves, then we won't be
> returning a value from launch_editor() anyway.  That case won't
> affect the "between returning negation or !!, which is more
> appropriate?" discussion, I think.
> 
> >>  - we MUST NOT care to differenciate different error codes returned
> >>    from launch_editor().  IOW, we must be fine to give the invoker
> >>    of the program only 0 (success) or 1 (unspecified failure).
> 
> I actually think this holds for the codepath.  A failure from
> start_command() returns error(), and finish_command() that waits for
> the spawned editor process to complete yields the exit status from
> the editor, but unless we re-raise the signal that killed the editor
> process to ourselves, we just turn any non-zero exit into "return
> error()", so it is safe to say launch_editor() can return either 0
> or -1 and nothing else.  Would we later want to tell callers of
> launch_editor() how/why the editor session failed by returning
> something else?  I do not offhand think of any---we do not even
> differenciate between failure to start (e.g. misspelt command name
> for the editor) and other failures WITH the return value and
> consider it sufficient to tell the user with different error
> message right now.
> 
> So in practice returning -launch_editor() and !!launch_editor()
> would not make any difference, I would think.

Then, let's do the least surprising thing. I'll switch it to !! for the
next reroll.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-18 23:56                 ` Emily Shaffer
@ 2020-02-19 23:15                   ` Emily Shaffer
  2020-02-19 23:24                     ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-19 23:15 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Feb 18, 2020 at 03:56:22PM -0800, Emily Shaffer wrote:
> On Tue, Feb 18, 2020 at 03:46:28PM -0800, Emily Shaffer wrote:
> > On Sat, Feb 15, 2020 at 10:24:40AM -0800, Junio C Hamano wrote:
> > > I am actually OK if we limit the use of this tool to "use with a
> > > repository that is not corrupt", as coping with these kinds of
> > > breakages that in the main Git executable we deem "needs manual
> > > intervention" inside a single process is too painful (it would have
> > > been easier to cope with these too if we stuck with a script that
> > > invokes many discrete commands and acts on their errors, but that is
> > > optimizing for rare case and not recommended).  But we should tell
> > > users about the limitation and encourage them to ask for help in non
> > > automatable means.
> > 
> > I think you're saying, "Mention this drawback in the manpage for
> > git-bugreport." Sounds like a good idea to me, so I'll add it for v8.
> 
> I'm pretty unsure about how you wanted this to sound; rather than
> sending a v8 only to revise it a lot, I'll send the paragraph for
> wordsmithing beforehand. Is this what you meant?
> 
>   This tool is invoked via the typical Git setup process, which means that in some
>   cases, it might not be able to launch - for example, if a relevant config file
>   is unreadable. In this kind of scenario, it may be helpful to manually gather
>   the kind of information listed above when manually asking for help.

Since I saw lots of replies from Junio elsewhere in this thread, I'll
take the silence here as assent. v8 on its way momentarily.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-19 23:15                   ` Emily Shaffer
@ 2020-02-19 23:24                     ` Junio C Hamano
  0 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-02-19 23:24 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> On Tue, Feb 18, 2020 at 03:56:22PM -0800, Emily Shaffer wrote:
>> On Tue, Feb 18, 2020 at 03:46:28PM -0800, Emily Shaffer wrote:
>> > On Sat, Feb 15, 2020 at 10:24:40AM -0800, Junio C Hamano wrote:
>> > > I am actually OK if we limit the use of this tool to "use with a
>> > > repository that is not corrupt", as coping with these kinds of
>> > > breakages that in the main Git executable we deem "needs manual
>> > > intervention" inside a single process is too painful (it would have
>> > > been easier to cope with these too if we stuck with a script that
>> > > invokes many discrete commands and acts on their errors, but that is
>> > > optimizing for rare case and not recommended).  But we should tell
>> > > users about the limitation and encourage them to ask for help in non
>> > > automatable means.
>> > 
>> > I think you're saying, "Mention this drawback in the manpage for
>> > git-bugreport." Sounds like a good idea to me, so I'll add it for v8.
>> 
>> I'm pretty unsure about how you wanted this to sound; rather than
>> sending a v8 only to revise it a lot, I'll send the paragraph for
>> wordsmithing beforehand. Is this what you meant?
>> 
>>   This tool is invoked via the typical Git setup process, which means that in some
>>   cases, it might not be able to launch - for example, if a relevant config file
>>   is unreadable. In this kind of scenario, it may be helpful to manually gather
>>   the kind of information listed above when manually asking for help.
>
> Since I saw lots of replies from Junio elsewhere in this thread, I'll
> take the silence here as assent. v8 on its way momentarily.

Heh, it's just I have too little time to allocate on this single
topic X-<.  Don't take silence as anything else.


^ permalink raw reply	[flat|nested] 273+ messages in thread

* [PATCH v8 00/15] add git-bugreport tool
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (15 preceding siblings ...)
  2020-02-14 17:32       ` [PATCH v7 00/15] add git-bugreport tool Junio C Hamano
@ 2020-02-20  1:58       ` Emily Shaffer
  2020-02-20  1:58         ` [PATCH v8 01/15] help: move list_config_help to builtin/help Emily Shaffer
                           ` (15 more replies)
  16 siblings, 16 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Martin Ågren, Aaron Schrab, Danh Doan,
	Eric Sunshine, SZEDER Gábor, Andreas Schwab

No longer relies on any topic branches, since mr/show-config-scope is
merged. Range-diff at the bottom of the cover letter.

Since v7:
 - Dropped some POSIX regex magic in
   generate-bugreport-config-safelist.sh.
 - Added some advice in doc for scenarios when git-bugreport may not run
   nicely
 - Quit using unusual "return -foo(bar)" and start using "return
   !!foo(bar)"
 - Add a test to ensure path creation works outside of a git dir
 - Add some GNUC and GNUC_MINOR reporting
 - Use git_cmd instead of hardcoding "git" as the first arg when
   invoking another process
 - Fix some pesky whitespace

Thanks.
 - Emily

Emily Shaffer (15):
  help: move list_config_help to builtin/help
  help: add shell-path to --build-options
  bugreport: add tool to generate debugging info
  bugreport: gather git version and build info
  bugreport: add uname info
  bugreport: add compiler info
  bugreport: add git-remote-https version
  bugreport: include user interactive shell
  bugreport: generate config safelist based on docs
  bugreport: add config values from safelist
  bugreport: collect list of populated hooks
  bugreport: count loose objects
  bugreport: add packed object summary
  bugreport: list contents of $OBJDIR/info
  bugreport: summarize contents of alternates file

 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   9 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  56 ++--
 Documentation/git-bugreport.txt         |  60 ++++
 Makefile                                |  27 +-
 bugreport.c                             | 418 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 command-list.txt                        |   1 +
 compat/compiler.h                       |  27 ++
 generate-bugreport-config-safelist.sh   |  18 +
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  21 ++
 help.c                                  | 132 ++------
 help.h                                  |   2 +-
 object-store.h                          |   6 +
 packfile.c                              |   3 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  61 ++++
 19 files changed, 808 insertions(+), 156 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh


 1:  429c7e093b =  1:  69f54afc6b help: move list_config_help to builtin/help
 2:  0cfb24f878 =  2:  9ea1c3ab34 help: add shell-path to --build-options
 3:  02bb151176 !  3:  8a290b5445 bugreport: add tool to generate debugging info
    @@ Documentation/git-bugreport.txt (new)
     + - Expected behavior
     + - Actual behavior
     +
    ++This tool is invoked via the typical Git setup process, which means that in some
    ++cases, it might not be able to launch - for example, if a relevant config file
    ++is unreadable. In this kind of scenario, it may be helpful to manually gather
    ++the kind of information listed above when manually asking for help.
    ++
     +OPTIONS
     +-------
     +-o <path>::
    @@ bugreport.c (new)
     +
     +	UNLEAK(buffer);
     +	UNLEAK(report_path);
    -+	return -launch_editor(report_path.buf, NULL, NULL);
    ++	return !!launch_editor(report_path.buf, NULL, NULL);
     +}
     
      ## command-list.txt ##
    @@ t/t0091-bugreport.sh (new)
     +	test_when_finished rm non-repo/git-bugreport-*
     +'
     +
    ++test_expect_success 'can create leading directories outside of a git dir' '
    ++	nongit git bugreport -o foo/bar/baz &&
    ++	test_when_finished rm -fr foo/bar/baz
    ++'
    ++
    ++
     +test_done
 4:  463612e409 !  4:  4eb4a7aef3 bugreport: gather git version and build info
    @@ Documentation/git-bugreport.txt: The following information is requested from the
     +
     + - 'git version --build-options'
     +
    - OPTIONS
    - -------
    - -o <path>::
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
    + is unreadable. In this kind of scenario, it may be helpful to manually gather
     
      ## bugreport.c ##
     @@
 5:  85e11ea125 !  5:  4e18b4678e bugreport: add uname info
    @@ Documentation/git-bugreport.txt: The following information is requested from the
       - 'git version --build-options'
     + - uname sysname, release, version, and machine strings
      
    - OPTIONS
    - -------
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
     
      ## bugreport.c ##
     @@
 6:  d9eafbaa48 !  6:  c18d17443d bugreport: add compiler info
    @@ Documentation/git-bugreport.txt: The following information is captured automatic
       - uname sysname, release, version, and machine strings
     + - Compiler-specific info string
      
    - OPTIONS
    - -------
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
     
      ## bugreport.c ##
     @@
    @@ compat/compiler.h (new)
     +
     +static inline void get_compiler_info(struct strbuf *info)
     +{
    -+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
    ++	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
    ++#ifdef __GNUC__
    ++	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
    ++#endif
     +}
     +
     +#else
 7:  f5b6dd4174 !  7:  6d807cf857 bugreport: add git-remote-https version
    @@ Documentation/git-bugreport.txt: The following information is captured automatic
       - Compiler-specific info string
     + - 'git remote-https --build-info'
      
    - OPTIONS
    - -------
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
     
      ## bugreport.c ##
     @@
    @@ bugreport.c
     +{
     +	struct child_process cp = CHILD_PROCESS_INIT;
     +
    -+	argv_array_push(&cp.args, "git");
    ++	cp.git_cmd = 1;
     +	argv_array_push(&cp.args, "remote-https");
     +	argv_array_push(&cp.args, "--build-info");
     +	if (capture_command(&cp, version_info, 0))
 8:  3119c9c1e3 !  8:  f85e1b69d0 bugreport: include user interactive shell
    @@ Documentation/git-bugreport.txt: The following information is captured automatic
       - 'git remote-https --build-info'
     + - $SHELL
      
    - OPTIONS
    - -------
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
     
      ## bugreport.c ##
     @@ bugreport.c: static void get_git_remote_https_version_info(struct strbuf *version_info)
 9:  c35bab2735 !  9:  9980728360 bugreport: generate config safelist based on docs
    @@ generate-bugreport-config-safelist.sh (new)
     +# cat all regular files in Documentation/config
     +find Documentation/config -type f -exec cat {} \; |
     +# print the command name which matches the annotate-bugreport macro
    -+sed -n 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p' \
    ++sed -n 's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p' \
     +	| sort
     +
     +cat <<EOF
10:  8a54ebdbbb ! 10:  e9cd8ffec9 bugreport: add config values from safelist
    @@ Documentation/git-bugreport.txt: The following information is captured automatic
       - $SHELL
     + - Selected config values
      
    - OPTIONS
    - -------
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
     
      ## Makefile ##
     @@ Makefile: git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
11:  8bb5bcd8b8 ! 11:  881b62369d bugreport: collect list of populated hooks
    @@ Documentation/git-bugreport.txt: The following information is captured automatic
       - Selected config values
     + - A list of enabled hooks
      
    - OPTIONS
    - -------
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
     
      ## bugreport.c ##
     @@
12:  744dd6aeee ! 12:  715e48e9a7 bugreport: count loose objects
    @@ Documentation/git-bugreport.txt: The following information is captured automatic
       - A list of enabled hooks
     + - The number of loose objects in the repository
      
    - OPTIONS
    - -------
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
     
      ## bugreport.c ##
     @@
13:  2f23e272bf ! 13:  c9e2f1f1d8 bugreport: add packed object summary
    @@ Documentation/git-bugreport.txt: The following information is captured automatic
       - The number of loose objects in the repository
     + - The number of packs and packed objects in the repository
      
    - OPTIONS
    - -------
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
     
      ## bugreport.c ##
     @@ bugreport.c: static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
    @@ bugreport.c: int cmd_main(int argc, const char **argv)
     
      ## object-store.h ##
     @@
    - #include "list.h"
      #include "sha1-array.h"
      #include "strbuf.h"
    + #include "thread-utils.h"
     +#include "packfile.h"
      
      struct object_directory {
14:  7b5ae1aca1 ! 14:  89762a0de1 bugreport: list contents of $OBJDIR/info
    @@ Documentation/git-bugreport.txt: The following information is captured automatic
       - The number of packs and packed objects in the repository
     + - A list of the contents of .git/objects/info (or equivalent)
      
    - OPTIONS
    - -------
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
     
      ## bugreport.c ##
     @@ bugreport.c: static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
    @@ bugreport.c: static void get_packed_object_summary(struct strbuf *obj_info, int
      }
      
     +static void list_contents_of_dir_recursively(struct strbuf *contents,
    -+				      	     struct strbuf *dirpath)
    ++					     struct strbuf *dirpath)
     +{
     +	struct dirent *d;
     +	DIR *dir;
15:  315b00ed54 ! 15:  aa81e197aa bugreport: summarize contents of alternates file
    @@ Documentation/git-bugreport.txt: The following information is captured automatic
       - A list of the contents of .git/objects/info (or equivalent)
     + - The number of valid and invalid alternates
      
    - OPTIONS
    - -------
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
     
      ## bugreport.c ##
     @@ bugreport.c: static void get_object_info_summary(struct strbuf *obj_info, int nongit)

-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 273+ messages in thread

* [PATCH v8 01/15] help: move list_config_help to builtin/help
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20  1:58         ` [PATCH v8 02/15] help: add shell-path to --build-options Emily Shaffer
                           ` (14 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
builtin/clean.o, and builtin/config.o. This meant that help.o was
unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help()
into builtin/help.c (where it makes sense to assume other builtin libraries
are present).

When command-list.h is included but a member is not used, we start to
hear a compiler warning. Since the config list is generated in a fairly
different way than the command list, and since commands and config
options are semantically different, move the config list into its own
header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 21 +++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 118 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh

diff --git a/.gitignore b/.gitignore
index aebe7c0908..ea97de83f3 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 6134104ae6..c552312d3f 100644
--- a/Makefile
+++ b/Makefile
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2132,7 +2133,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2152,6 +2153,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2785,7 +2792,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..8692fe5cf4
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	sed 's/^.*$/	"&",/'
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 02/15] help: add shell-path to --build-options
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
  2020-02-20  1:58         ` [PATCH v8 01/15] help: move list_config_help to builtin/help Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20 19:03           ` Junio C Hamano
  2020-02-20  1:58         ` [PATCH v8 03/15] bugreport: add tool to generate debugging info Emily Shaffer
                           ` (13 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It may be useful to know which shell Git was built to try to point to,
in the event that shell-based Git commands are failing. $SHELL_PATH is
set during the build and used to launch the manpage viewer, as well as
by git-compat-util.h, and it's used during tests. 'git version
--build-options' is encouraged for use in bug reports, so it makes sense
to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/help.c b/help.c
index a21487db77..190722fb0a 100644
--- a/help.c
+++ b/help.c
@@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 			printf("no commit associated with this build\n");
 		printf("sizeof-long: %d\n", (int)sizeof(long));
 		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
+		printf("shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 	return 0;
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 03/15] bugreport: add tool to generate debugging info
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
  2020-02-20  1:58         ` [PATCH v8 01/15] help: move list_config_help to builtin/help Emily Shaffer
  2020-02-20  1:58         ` [PATCH v8 02/15] help: add shell-path to --build-options Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20 19:33           ` Junio C Hamano
  2020-02-26 16:12           ` Johannes Schindelin
  2020-02-20  1:58         ` [PATCH v8 04/15] bugreport: gather git version and build info Emily Shaffer
                           ` (12 subsequent siblings)
  15 siblings, 2 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction
steps, expected behavior, and actual behavior. Later, Git can learn how
to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic
information we would otherwise need to ask the user for, we can reduce
the number of question-and-answer round trips between the reporter and
the Git contributor.

Users may also wish to send a report like this to their local "Git
expert" if they have put their repository into a state they are confused
by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 46 ++++++++++++++++
 Makefile                        |  5 ++
 bugreport.c                     | 94 +++++++++++++++++++++++++++++++++
 command-list.txt                |  1 +
 t/t0091-bugreport.sh            | 61 +++++++++++++++++++++
 6 files changed, 208 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index ea97de83f3..d89bf9e11e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..1f9fde5cde
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,46 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [(-o | --output-directory) <path>] [(-s | --suffix) <format>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+This tool is invoked via the typical Git setup process, which means that in some
+cases, it might not be able to launch - for example, if a relevant config file
+is unreadable. In this kind of scenario, it may be helpful to manually gather
+the kind of information listed above when manually asking for help.
+
+OPTIONS
+-------
+-o <path>::
+--output-directory <path>::
+	Place the resulting bug report file in `<path>` instead of the root of
+	the Git repository.
+
+-s <format>::
+--suffix <format>::
+	Specify an alternate suffix for the bugreport name, to create a file
+	named 'git-bugreport-<formatted suffix>'. This should take the form of a
+	link:strftime[3] format string; the current local time will be used.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index c552312d3f..9e6705061d 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2461,6 +2462,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..8d4a76fdac
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,94 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to share.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+	char *option_suffix = "%F-%H%M";
+	struct stat statbuf;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output-directory", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
+			   N_("specify a strftime format suffix for the filename")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	if (!stat(report_path.buf, &statbuf))
+		die("'%s' already exists", report_path.buf);
+
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+
+	if (report == NULL) {
+		strbuf_release(&report_path);
+		die("couldn't open '%s' for writing", report_path.buf);
+	}
+
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
+
+	UNLEAK(buffer);
+	UNLEAK(report_path);
+	return !!launch_editor(report_path.buf, NULL, NULL);
+}
diff --git a/command-list.txt b/command-list.txt
index 2087894655..185e5e3f05 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..65f664fdac
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,61 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
+			echo "$line"
+			read -r nextline
+			if test -z "$nextline"; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport -s check-headers &&
+	check_all_headers_populated <git-bugreport-check-headers.txt &&
+	test_when_finished rm git-bugreport-check-headers.txt
+'
+
+test_expect_success 'dies if file with same name as report already exists' '
+	>>git-bugreport-duplicate.txt &&
+	test_must_fail git bugreport --suffix duplicate &&
+	test_when_finished rm git-bugreport-duplicate.txt
+'
+
+test_expect_success '--output-directory puts the report in the provided dir' '
+	git bugreport -o foo/ &&
+	test_path_is_file foo/git-bugreport-* &&
+	test_when_finished rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	test_i18ngrep usage output &&
+	test_path_is_missing git-bugreport-*
+'
+
+test_expect_success 'runs outside of a git dir' '
+	nongit git bugreport &&
+	test_when_finished rm non-repo/git-bugreport-*
+'
+
+test_expect_success 'can create leading directories outside of a git dir' '
+	nongit git bugreport -o foo/bar/baz &&
+	test_when_finished rm -fr foo/bar/baz
+'
+
+
+test_done
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 04/15] bugreport: gather git version and build info
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (2 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 03/15] bugreport: add tool to generate debugging info Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20 20:07           ` Junio C Hamano
  2020-02-20  1:58         ` [PATCH v8 05/15] bugreport: add uname info Emily Shaffer
                           ` (11 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Knowing which version of Git a user has and how it was built allows us
to more precisely pin down the circumstances when a certain issue
occurs, so teach bugreport how to tell us the same output as 'git
version --build-options'.

It's not ideal to directly call 'git version --build-options' because
that output goes to stdout. Instead, wrap the version string in a helper
within help.[ch] library, and call that helper from within the bugreport
library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  4 +++
 bugreport.c                     | 19 ++++++++++++-
 help.c                          | 48 ++++++++++++++++++++-------------
 help.h                          |  1 +
 4 files changed, 52 insertions(+), 20 deletions(-)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 1f9fde5cde..f44ae8cbe7 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -23,6 +23,10 @@ The following information is requested from the user:
  - Expected behavior
  - Actual behavior
 
+The following information is captured automatically:
+
+ - 'git version --build-options'
+
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
 is unreadable. In this kind of scenario, it may be helpful to manually gather
diff --git a/bugreport.c b/bugreport.c
index 8d4a76fdac..27f813643d 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,17 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
@@ -32,6 +41,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -76,6 +90,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/help.c b/help.c
index 190722fb0a..44cee69c11 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,26 +662,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	get_version_info(&buf, build_options);
+	printf("%s", buf.buf);
+
+	strbuf_release(&buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		printf("shell-path: %s\n", SHELL_PATH);
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..500521b908 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 05/15] bugreport: add uname info
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (3 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 04/15] bugreport: gather git version and build info Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20 20:12           ` Junio C Hamano
  2020-02-20  1:58         ` [PATCH v8 06/15] bugreport: add compiler info Emily Shaffer
                           ` (10 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

The contents of uname() can give us some insight into what sort of
system the user is running on, and help us replicate their setup if need
be. The domainname field is not guaranteed to be available, so don't
collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 13 +++++++++++++
 2 files changed, 14 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index f44ae8cbe7..17b0d14e8d 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -26,6 +26,7 @@ The following information is requested from the user:
 The following information is captured automatically:
 
  - 'git version --build-options'
+ - uname sysname, release, version, and machine strings
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
diff --git a/bugreport.c b/bugreport.c
index 27f813643d..06a63cc283 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -7,10 +7,23 @@
 
 static void get_system_info(struct strbuf *sys_info)
 {
+	struct utsname uname_info;
+
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(sys_info, 1);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 06/15] bugreport: add compiler info
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (4 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 05/15] bugreport: add uname info Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20  2:49           ` Danh Doan
                             ` (2 more replies)
  2020-02-20  1:58         ` [PATCH v8 07/15] bugreport: add git-remote-https version Emily Shaffer
                           ` (9 subsequent siblings)
  15 siblings, 3 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

To help pinpoint the source of a regression, it is useful to know some
info about the compiler which the user's Git client was built with. By
adding a generic get_compiler_info() in 'compat/' we can choose which
relevant information to share per compiler; to get started, let's
demonstrate the version of glibc if the user built with 'gcc'.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     |  5 +++++
 compat/compiler.h               | 27 +++++++++++++++++++++++++++
 3 files changed, 33 insertions(+)
 create mode 100644 compat/compiler.h

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 17b0d14e8d..643d1b2884 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -27,6 +27,7 @@ The following information is captured automatically:
 
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
+ - Compiler-specific info string
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
diff --git a/bugreport.c b/bugreport.c
index 06a63cc283..9a470c5588 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/compat/compiler.h b/compat/compiler.h
new file mode 100644
index 0000000000..ef41177233
--- /dev/null
+++ b/compat/compiler.h
@@ -0,0 +1,27 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
+#ifdef __GNUC__
+	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
+#endif
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 07/15] bugreport: add git-remote-https version
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (5 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 06/15] bugreport: add compiler info Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20 20:35           ` Junio C Hamano
  2020-02-20  1:58         ` [PATCH v8 08/15] bugreport: include user interactive shell Emily Shaffer
                           ` (8 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible for git-remote-curl to be built separately from git; in that
case we want to know what version of cURL is used by git-remote-curl, not
necessarily which version was present at git-bugreport's build time.
So instead, ask git-remote-curl for the version information it knows
about.

Today, "git-remote-http" and "git-remote-https" are aliased to
"git-remote-curl"; but in case we rely on a different library than cURL
in the future, let's not explicitly reference cURL from bugreport.

For longevity purposes, invoke the alias "git-remote-https" instead of
"git-remote-http".

Since it could have been built at a different time, also report the
version and built-from commit of git-remote-curl alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 16 ++++++++++++++++
 remote-curl.c                   |  8 ++++++++
 3 files changed, 25 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 643d1b2884..8c7cb9a5d4 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -28,6 +28,7 @@ The following information is captured automatically:
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
+ - 'git remote-https --build-info'
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
diff --git a/bugreport.c b/bugreport.c
index 9a470c5588..fb2adfdf14 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	cp.git_cmd = 1;
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_git_remote_https_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/remote-curl.c b/remote-curl.c
index 8eb96152f5..73e52175c0 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1375,6 +1376,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 08/15] bugreport: include user interactive shell
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (6 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 07/15] bugreport: add git-remote-https version Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20  1:58         ` [PATCH v8 09/15] bugreport: generate config safelist based on docs Emily Shaffer
                           ` (7 subsequent siblings)
  15 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible a user may complain about the way that Git interacts with
their interactive shell, e.g. autocompletion or shell prompt. In that
case, it's useful for us to know which shell they're using
interactively.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt | 1 +
 bugreport.c                     | 5 +++++
 2 files changed, 6 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 8c7cb9a5d4..f7a13c0374 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -29,6 +29,7 @@ The following information is captured automatically:
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
  - 'git remote-https --build-info'
+ - $SHELL
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
diff --git a/bugreport.c b/bugreport.c
index fb2adfdf14..f143b9a8f8 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -21,6 +21,7 @@ static void get_git_remote_https_version_info(struct strbuf *version_info)
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -42,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "<unset>");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_git_remote_https_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 09/15] bugreport: generate config safelist based on docs
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (7 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 08/15] bugreport: include user interactive shell Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20 20:40           ` Junio C Hamano
  2020-02-26 16:13           ` Johannes Schindelin
  2020-02-20  1:58         ` [PATCH v8 10/15] bugreport: add config values from safelist Emily Shaffer
                           ` (6 subsequent siblings)
  15 siblings, 2 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Martin Ågren, Johannes Schindelin

Add a new step to the build to generate a safelist of git-config
variables which are appropriate to include in the output of
git-bugreport. New variables can be added to the safelist by annotating
their documentation in Documentation/config with the "annotate" macro,
which is a no-op in AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs,
passwords, or other sensitive information. In the event that a user
doesn't notice their information while reviewing a bugreport, that user
may leak their credentials to other individuals, mailing lists, or bug
tracking tools inadvertently. Heuristic blocklisting of configuration
keys is imperfect and prone to false negatives; given the nature of the
information which can be leaked, a safelist is more reliable.

However, it's possible that in some situations, an organization may be
less concerned with privacy of things like remote URLs and branch names,
and more concerned with ease of diagnosis for their support staff. In
those cases, it may make more sense for that organization to modify the
code to use a blocklist. To that end, we should try to mark configs which
are definitely safe, and configs which are definitely unsafe, and leave
blank configs which are somewhere in between. To mark a config as safe,
add "annotate:bugreport[include]" to the corresponding line in the
config documentation; to mark it as unsafe, add
"annotate:bugreport[exclude]" instead.

Generating bugreport-config-safelist.h at build time by grepping the
documentation for this new macro helps us prevent staleness. The macro
itself is a no-op and should not alter the appearance of the
documentation in either AsciiDoc or AsciiDoctor, confirmable by running:

  cd Documentation
  ./doc-diff --asciidoctor HEAD^ HEAD
  ./doc-diff --asciidoc HEAD^ HEAD

Diffing the rendered HTML shows that only inline comments were added,
which shouldn't be a problem.

Additionally, add annotations to the sendemail config documentation in
order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  9 ++++
 Documentation/asciidoctor-extensions.rb |  7 ++++
 Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
 Makefile                                |  7 ++++
 generate-bugreport-config-safelist.sh   | 18 ++++++++
 6 files changed, 70 insertions(+), 28 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index d89bf9e11e..bd2f49b996 100644
--- a/.gitignore
+++ b/.gitignore
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..663e06481f 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,14 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The annotate macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources to populate
+# things like the bugreport safelist.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +33,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[annotate-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[annotate-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..03c80af0e5 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -39,10 +39,17 @@ module Git
         output
       end
     end
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""
+      end
+    end
   end
 end
 
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  inline_macro Git::Documentation::AnnotateProcessor, :annotate
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..fe27473e44 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -4,7 +4,7 @@ sendemail.identity::
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
@@ -15,7 +15,7 @@ sendemail.smtpsslcertpath::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* annotate:bugreport[exclude] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
@@ -23,41 +23,41 @@ sendemail.<identity>.*::
 
 sendemail.aliasesFile::
 sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.annotate annotate:bugreport[include] ::
+sendemail.bcc annotate:bugreport[include] ::
+sendemail.cc annotate:bugreport[include] ::
+sendemail.ccCmd annotate:bugreport[include] ::
+sendemail.chainReplyTo annotate:bugreport[include] ::
+sendemail.confirm annotate:bugreport[include] ::
+sendemail.envelopeSender annotate:bugreport[include] ::
+sendemail.from annotate:bugreport[include] ::
+sendemail.multiEdit annotate:bugreport[include] ::
+sendemail.signedoffbycc annotate:bugreport[include] ::
+sendemail.smtpPass annotate:bugreport[exclude] ::
+sendemail.suppresscc annotate:bugreport[include] ::
+sendemail.suppressFrom annotate:bugreport[include] ::
+sendemail.to annotate:bugreport[include] ::
+sendemail.tocmd annotate:bugreport[include] ::
+sendemail.smtpDomain annotate:bugreport[include] ::
+sendemail.smtpServer annotate:bugreport[include] ::
+sendemail.smtpServerPort annotate:bugreport[include] ::
+sendemail.smtpServerOption annotate:bugreport[include] ::
+sendemail.smtpUser annotate:bugreport[exclude] ::
+sendemail.thread annotate:bugreport[include] ::
+sendemail.transferEncoding annotate:bugreport[include] ::
+sendemail.validate annotate:bugreport[include] ::
+sendemail.xmailer annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.
 
 sendemail.signedoffcc (deprecated)::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize annotate:bugreport[include] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay annotate:bugreport[include] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index 9e6705061d..6bdd3b9337 100644
--- a/Makefile
+++ b/Makefile
@@ -818,6 +818,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2167,6 +2168,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..a9e5b6b2a0
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the annotate-bugreport macro
+sed -n 's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p' \
+	| sort
+
+cat <<EOF
+};
+EOF
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 10/15] bugreport: add config values from safelist
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (8 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 09/15] bugreport: generate config safelist based on docs Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20 20:47           ` Junio C Hamano
  2020-02-20  1:58         ` [PATCH v8 11/15] bugreport: collect list of populated hooks Emily Shaffer
                           ` (5 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach bugreport to gather the values of config options which are present
in 'bugreport-config-safelist.h', and show their origin scope.

Many config options are sensitive, and many Git add-ons use config
options which git-core does not know about; it is better only to gather
config options which we know to be safe, rather than excluding options
which we know to be unsafe.

Rather than including the path to someone's config, which can reveal
filesystem layout and project names, just name the scope (e.g. system,
global, local) of the config source.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 Makefile                        |  2 ++
 bugreport.c                     | 34 +++++++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index f7a13c0374..efe9719ea4 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -30,6 +30,7 @@ The following information is captured automatically:
  - Compiler-specific info string
  - 'git remote-https --build-info'
  - $SHELL
+ - Selected config values
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
diff --git a/Makefile b/Makefile
index 6bdd3b9337..bf4f3ede2a 100644
--- a/Makefile
+++ b/Makefile
@@ -2135,6 +2135,8 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
+bugreport.sp bugreport.s bugreport.o: bugreport-config-safelist.h
+
 builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
diff --git a/bugreport.c b/bugreport.c
index f143b9a8f8..70fb3b8fa6 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -52,6 +55,29 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	size_t idx;
+	struct string_list_item *it = NULL;
+	struct key_value_info *kv_info = NULL;
+
+	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
+		const struct string_list *list =
+			git_config_get_value_multi(bugreport_config_safelist[idx]);
+
+		if (!list)
+			continue;
+
+		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
+		for_each_string_list_item(it, list) {
+			kv_info = it->util;
+			strbuf_addf(config_info, "  %s (%s)\n", it->string,
+				    kv_info ? config_scope_name(kv_info->scope)
+					    : "source unknown");
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -94,6 +120,7 @@ int cmd_main(int argc, const char **argv)
 	char *option_output = NULL;
 	char *option_suffix = "%F-%H%M";
 	struct stat statbuf;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output-directory", &option_output, N_("path"),
@@ -102,6 +129,10 @@ int cmd_main(int argc, const char **argv)
 			   N_("specify a strftime format suffix for the filename")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -132,6 +163,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 11/15] bugreport: collect list of populated hooks
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (9 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 10/15] bugreport: add config values from safelist Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20 20:58           ` Junio C Hamano
  2020-02-20  1:58         ` [PATCH v8 12/15] bugreport: count loose objects Emily Shaffer
                           ` (4 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific
hook which is being run, perhaps without the user realizing. While the
contents of hooks can be sensitive - containing user data or process
information specific to the user's organization - simply knowing that a
hook is being run at a certain stage can help us to understand whether
something is going wrong.

Without a definitive list of hook names within the code, we compile our
own list from the documentation. This is likely prone to bitrot. To
reduce the amount of code humans need to read, we turn the list into a
string_list and iterate over it (as we are calling the same find_hook
operation on each string). However, since bugreport should primarily be
called by the user, the performance loss from massaging the string
seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 53 +++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index efe9719ea4..4d01528540 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -31,6 +31,7 @@ The following information is captured automatically:
  - 'git remote-https --build-info'
  - $SHELL
  - Selected config values
+ - A list of enabled hooks
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
diff --git a/bugreport.c b/bugreport.c
index 70fb3b8fa6..b5a0714a7f 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -78,6 +79,55 @@ static void get_safelisted_config(struct strbuf *config_info)
 	}
 }
 
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hooks`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -166,6 +216,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Enabled Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 12/15] bugreport: count loose objects
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (10 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 11/15] bugreport: collect list of populated hooks Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20 21:04           ` Junio C Hamano
  2020-02-20  1:58         ` [PATCH v8 13/15] bugreport: add packed object summary Emily Shaffer
                           ` (3 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Johannes Schindelin

The number of unpacked objects in a user's repository may help us
understand the root of the problem they're seeing, especially if a
command is running unusually slowly.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4d01528540..4fe1c60506 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
diff --git a/bugreport.c b/bugreport.c
index b5a0714a7f..fb7bc72723 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
+
+	int local_loose_object_count = 0, total_loose_object_count = 0;
+	int local_count_questionable = 0, total_count_questionable = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	local_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&local_loose_object_count,
+					FOR_EACH_OBJECT_LOCAL_ONLY);
+
+	total_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&total_loose_object_count,
+					0);
+
+	strbuf_addf(obj_info, "%d local loose objects%s\n",
+		    local_loose_object_count,
+		    local_count_questionable ? " (problem during count)" : "");
+
+	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
+		    total_loose_object_count - local_loose_object_count,
+		    (local_count_questionable || total_count_questionable)
+			? " (problem during count)"
+			: "");
+
+	strbuf_addf(obj_info, "%d total loose objects%s\n",
+		    total_loose_object_count,
+		    total_count_questionable ? " (problem during count)" : "");
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -219,6 +268,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Enabled Hooks");
 	get_populated_hooks(&buffer, nongit_ok);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 13/15] bugreport: add packed object summary
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (11 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 12/15] bugreport: count loose objects Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20 22:04           ` Junio C Hamano
  2020-02-20  1:58         ` [PATCH v8 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
                           ` (2 subsequent siblings)
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Alongside the loose object counts, it can be useful to show the number
of packs and packed objects. This way we can check whether the repo has
an appropriate ratio of packed to loose objects to help determine
whether it's behaving correctly.

Add a utility to easily traverse all packfiles in a given repository.
Use it in packfile.c and remove a redundant call to
prepare_packed_git(), which is already called in get_all_packs().

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 30 ++++++++++++++++++++++++++++++
 object-store.h                  |  6 ++++++
 packfile.c                      |  3 +--
 4 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4fe1c60506..eb41f0677f 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -33,6 +33,7 @@ The following information is captured automatically:
  - Selected config values
  - A list of enabled hooks
  - The number of loose objects in the repository
+ - The number of packs and packed objects in the repository
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
diff --git a/bugreport.c b/bugreport.c
index fb7bc72723..71191f1331 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -177,6 +177,33 @@ static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
+{
+	struct packed_git *pack = NULL;
+	int pack_count = 0;
+	int object_count = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	for_each_pack(the_repository, pack) {
+		pack_count++;
+		/*
+		 * To accurately count how many objects are packed, look inside
+		 * the packfile's index.
+		 */
+		open_pack_index(pack);
+		object_count += pack->num_objects;
+	}
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
+		    object_count);
+
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -271,6 +298,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/object-store.h b/object-store.h
index 5b047637e3..f881a1756e 100644
--- a/object-store.h
+++ b/object-store.h
@@ -7,6 +7,7 @@
 #include "sha1-array.h"
 #include "strbuf.h"
 #include "thread-utils.h"
+#include "packfile.h"
 
 struct object_directory {
 	struct object_directory *next;
@@ -447,4 +448,9 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(each_packed_object_fn, void *,
 			   enum for_each_object_flags flags);
 
+#define for_each_pack(repo, pack) 		\
+		for (pack = get_all_packs(repo);\
+		     pack;			\
+		     pack = pack->next)
+
 #endif /* OBJECT_STORE_H */
diff --git a/packfile.c b/packfile.c
index 99dd1a7d09..95afcc1187 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2095,8 +2095,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git(the_repository);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for_each_pack(the_repository, p) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 14/15] bugreport: list contents of $OBJDIR/info
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (12 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 13/15] bugreport: add packed object summary Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20 22:18           ` Junio C Hamano
  2020-02-20  1:58         ` [PATCH v8 15/15] bugreport: summarize contents of alternates file Emily Shaffer
  2020-03-02 23:03         ` [PATCH v9 0/5] add git-bugreport tool Emily Shaffer
  15 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Miscellaneous information used about the object store can end up in
.git/objects/info; this can help us understand what may be going on with
the object store when the user is reporting a bug. Otherwise, it could
be difficult to track down what is going wrong with an object which
isn't kept locally to .git/objects/ or .git/objects/pack. Having some
understanding of where the user's objects may be kept can save us some
hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 54 +++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index eb41f0677f..a7ef3360d2 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -34,6 +34,7 @@ The following information is captured automatically:
  - A list of enabled hooks
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
+ - A list of the contents of .git/objects/info (or equivalent)
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
diff --git a/bugreport.c b/bugreport.c
index 71191f1331..1d61e0f642 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -204,6 +204,57 @@ static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
 
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+					     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info, int nongit)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - object info unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -301,6 +352,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v8 15/15] bugreport: summarize contents of alternates file
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (13 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
@ 2020-02-20  1:58         ` Emily Shaffer
  2020-02-20 14:08           ` Johannes Schindelin
  2020-02-20 22:22           ` Junio C Hamano
  2020-03-02 23:03         ` [PATCH v9 0/5] add git-bugreport tool Emily Shaffer
  15 siblings, 2 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20  1:58 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In some cases, it could be that the user is having a problem with an
object which isn't present in their normal object directory. We can get
a hint that that might be the case by examining the list of alternates
where their object may be stored instead. Since paths to alternates may
be sensitive, we'll instead count how many alternates have been
specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this
function does not provide information on broken alternates, which are
skipped over in "link_alt_odb_entry()". Since the goal is to identify
missing alternates, we can gather the contents of
.git/objects/info/alternates manually.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 45 +++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index a7ef3360d2..1f60fb9456 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -35,6 +35,7 @@ The following information is captured automatically:
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
  - A list of the contents of .git/objects/info (or equivalent)
+ - The number of valid and invalid alternates
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
diff --git a/bugreport.c b/bugreport.c
index 1d61e0f642..1640a71086 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -255,6 +255,48 @@ static void get_object_info_summary(struct strbuf *obj_info, int nongit)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	if (nongit) {
+		strbuf_addstr(alternates_info,
+			"not run from a git repository - alternates unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -355,6 +397,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 06/15] bugreport: add compiler info
  2020-02-20  1:58         ` [PATCH v8 06/15] bugreport: add compiler info Emily Shaffer
@ 2020-02-20  2:49           ` Danh Doan
  2020-02-20 23:23             ` Emily Shaffer
  2020-02-20 20:23           ` Junio C Hamano
  2020-02-21  0:26           ` Junio C Hamano
  2 siblings, 1 reply; 273+ messages in thread
From: Danh Doan @ 2020-02-20  2:49 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

[-- Attachment #1: Type: text/plain, Size: 1170 bytes --]

On 2020-02-19 17:58:49-0800, Emily Shaffer <emilyshaffer@google.com> wrote:
> diff --git a/compat/compiler.h b/compat/compiler.h
> new file mode 100644
> index 0000000000..ef41177233
> --- /dev/null
> +++ b/compat/compiler.h
> @@ -0,0 +1,27 @@
> +#ifndef COMPILER_H
> +#define COMPILER_H
> +
> +#include "git-compat-util.h"
> +#include "strbuf.h"
> +
> +#ifdef __GLIBC__
> +#include <gnu/libc-version.h>
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> +#ifdef __GNUC__
> +	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> +#endif

I think we're better having

- get_compiler_info placed under #ifdef __GNUC__ gate,
- get_libc_info under #ifdef __GLIBC

Then have a function to merge those information together.
Correct me if I were wrong, as it is now,
this is only useful for Linux people with glibc.

Anyway, clang also defines __GNUC__ and __GNUC_MINOR__,
IIRC, FreeBSD people started to move to use clang instead of gcc
as their default compiler.

Gentoo forks are also known to use clang to compile their ports.


-- 
Danh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 15/15] bugreport: summarize contents of alternates file
  2020-02-20  1:58         ` [PATCH v8 15/15] bugreport: summarize contents of alternates file Emily Shaffer
@ 2020-02-20 14:08           ` Johannes Schindelin
  2020-02-20 22:22           ` Junio C Hamano
  1 sibling, 0 replies; 273+ messages in thread
From: Johannes Schindelin @ 2020-02-20 14:08 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Wed, 19 Feb 2020, Emily Shaffer wrote:

> In some cases, it could be that the user is having a problem with an
> object which isn't present in their normal object directory. We can get
> a hint that that might be the case by examining the list of alternates
> where their object may be stored instead. Since paths to alternates may
> be sensitive, we'll instead count how many alternates have been
> specified and note how many of them exist or are broken.
>
> While object-cache.h describes a function "foreach_alt_odb()", this
> function does not provide information on broken alternates, which are
> skipped over in "link_alt_odb_entry()". Since the goal is to identify
> missing alternates, we can gather the contents of
> .git/objects/info/alternates manually.

Makes sense.

> diff --git a/bugreport.c b/bugreport.c
> index 1d61e0f642..1640a71086 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -255,6 +255,48 @@ static void get_object_info_summary(struct strbuf *obj_info, int nongit)
>  	strbuf_release(&dirpath);
>  }
>
> +static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
> +{
> +	struct strbuf alternates_path = STRBUF_INIT;
> +	struct strbuf alternate = STRBUF_INIT;

I am not sure that those variables and the parameter need to repeat
"alternate", I find it rather distracting. If it were me, I would rename
the parameter to `info`, the first strbuf to `path` and the second to
`line`. This function is so specific to read the alternates file that it
is quite obvious what their roles are.

> +	FILE *file;
> +	size_t exists = 0, broken = 0;
> +
> +	if (nongit) {
> +		strbuf_addstr(alternates_info,
> +			"not run from a git repository - alternates unavailable\n");
> +		return;
> +	}
> +
> +	strbuf_addstr(&alternates_path, get_object_directory());
> +	strbuf_complete(&alternates_path, '/');
> +	strbuf_addstr(&alternates_path, "info/alternates");
> +
> +	file = fopen(alternates_path.buf, "r");
> +	if (!file) {
> +		strbuf_addstr(alternates_info, "No alternates file found.\n");
> +		strbuf_release(&alternates_path);
> +		return;
> +	}
> +
> +	while (strbuf_getline(&alternate, file) != EOF) {
> +		if (!access(alternate.buf, F_OK))

Should we make sure that the alternate is actually valid objects directory
here? Like, look whether it is a directory, not a file or a (possibly
dangling) symbolic link. This seems to be the only check
`alt_odb_usable()` performs, so that should probably be good enough here,
too.

> +			exists++;
> +		else
> +			broken++;
> +	}
> +
> +	strbuf_addf(alternates_info,
> +		    "%zd alternates found (%zd working, %zd broken)\n",

Sadly, `%zd` is not portable. Therefore, `pu` (and `es/bugreport`) do not
even _build_ on Windows. I need this to make it work:

-- snip --
diff --git a/bugreport.c b/bugreport.c
index 3770aa73fae..5033668e22f 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -287,10 +287,11 @@ static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
 	}

 	strbuf_addf(alternates_info,
-		    "%zd alternates found (%zd working, %zd broken)\n",
-		    exists + broken,
-		    exists,
-		    broken);
+		    "%"PRIuMAX" alternates found "
+		    "(%"PRIuMAX" working, %"PRIuMAX" broken)\n",
+		    (uintmax_t)(exists + broken),
+		    (uintmax_t)exists,
+		    (uintmax_t)broken);

 	fclose(file);
 	strbuf_release(&alternate);
-- snap --

Could you incorporate that into the next iteration, please?

Thanks,
Dscho

> +		    exists + broken,
> +		    exists,
> +		    broken);
> +
> +	fclose(file);
> +	strbuf_release(&alternate);
> +	strbuf_release(&alternates_path);
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
>  	NULL
> @@ -355,6 +397,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Object Info Summary");
>  	get_object_info_summary(&buffer, nongit_ok);
>
> +	get_header(&buffer, "Alternates");
> +	get_alternates_summary(&buffer, nongit_ok);
> +
>  	report = fopen_for_writing(report_path.buf);
>
>  	if (report == NULL) {
> --
> 2.25.0.265.gbab2e86ba0-goog
>
>

^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 02/15] help: add shell-path to --build-options
  2020-02-20  1:58         ` [PATCH v8 02/15] help: add shell-path to --build-options Emily Shaffer
@ 2020-02-20 19:03           ` Junio C Hamano
  2020-02-20 21:15             ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 19:03 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> It may be useful to know which shell Git was built to try to point to,
> in the event that shell-based Git commands are failing. $SHELL_PATH is
> set during the build and used to launch the manpage viewer, as well as
> by git-compat-util.h, and it's used during tests. 'git version
> --build-options' is encouraged for use in bug reports, so it makes sense
> to include this information there.

It probably makes sense to capture PERL_PATH, PYTHON_PATH,
TCLTK_PATH, and TCL_PATH, as they fall into exactly the same
category as this one.

> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  help.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/help.c b/help.c
> index a21487db77..190722fb0a 100644
> --- a/help.c
> +++ b/help.c
> @@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
>  			printf("no commit associated with this build\n");
>  		printf("sizeof-long: %d\n", (int)sizeof(long));
>  		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
> +		printf("shell-path: %s\n", SHELL_PATH);
>  		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
>  	}
>  	return 0;

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 03/15] bugreport: add tool to generate debugging info
  2020-02-20  1:58         ` [PATCH v8 03/15] bugreport: add tool to generate debugging info Emily Shaffer
@ 2020-02-20 19:33           ` Junio C Hamano
  2020-02-20 22:33             ` Emily Shaffer
  2020-02-26 16:12           ` Johannes Schindelin
  1 sibling, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 19:33 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +	argc = parse_options(argc, argv, "", bugreport_options,
> +			     bugreport_usage, 0);

Which one between an empty string and NULL is more appropriate to be
passed as "prefix" here?  I assume that this is *not* really a git
program, and no repository/working-tree discovery is involved, and
you won't be using OPT_FILENAME, so it would probably be OK.

> +
> +	if (option_output) {
> +		strbuf_addstr(&report_path, option_output);
> +		strbuf_complete(&report_path, '/');
> +	}
> +
> +
> +	strbuf_addstr(&report_path, "git-bugreport-");
> +	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
> +	strbuf_addstr(&report_path, ".txt");
> +
> +	if (!stat(report_path.buf, &statbuf))
> +		die("'%s' already exists", report_path.buf);

Missed i18n/l10n, but I do not think it is a useful thing for this
check to be here in the first place.

> +	switch (safe_create_leading_directories(report_path.buf)) {
> +	case SCLD_OK:
> +	case SCLD_EXISTS:
> +		break;
> +	default:
> +		die(_("could not create leading directories for '%s'"),
> +		    report_path.buf);
> +	}
> +
> +	get_bug_template(&buffer);
> +
> +	report = fopen_for_writing(report_path.buf);

fopen_for_writing() does not give good semantics for what you seem
to want to do here (i.e. to make sure you do not overwrite an
existing one).  It even has "if we cannot open it, retry after
unlinking" logic in it, which screams "this function is designed for
those who want to overwrite the file", and if you look at its
callers, they are _all_ about opening a file for writing with a well
known name like FETCH_HEAD, COMMIT_EDITMSG, etc.

Besides, a stat() that dies when a file exists would not
help---since that check, you've spent non-zero time, creating
leading directories and preparing boilerplate in the buffer,
and there is no guarantee somebody else used the same filename
in the meantime.

If you want to avoid overwriting an existing file (which I think is
a good idea---I just do not think the earlier "if (!stat()) die()"
is a good way to do so), the only sensible way to do so is to ask
your open/creat to fail if there already is a file---that is how
you'd avoid racing with another process that may be creating such a
file.  Grep for O_EXCL to find where the flag is used with O_CREAT
to see how it is done.

> +	if (report == NULL) {

Style. "if (!report)"

> +		strbuf_release(&report_path);
> +		die("couldn't open '%s' for writing", report_path.buf);

Do we see a use-after-free here?  Also missed i18n/l10n.

It is embarrassing on the reviewers' side that this (which is not
new in this round at all and hasn't changed since v6) hasn't been
caught for a few rounds X-<.

Thanks.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 04/15] bugreport: gather git version and build info
  2020-02-20  1:58         ` [PATCH v8 04/15] bugreport: gather git version and build info Emily Shaffer
@ 2020-02-20 20:07           ` Junio C Hamano
  2020-02-20 23:03             ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 20:07 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +static void get_system_info(struct strbuf *sys_info)
> +{
> +	/* get git version from native cmd */
> +	strbuf_addstr(sys_info, "git version:\n");
> +	get_version_info(sys_info, 1);
> +	strbuf_complete_line(sys_info);

It is a bit curious use of "don't do anything if sys_info ends with
a complete line, but complete it if it ends with an imcomplete
line".  That tells the readers that we do not know what
get_version_info() will do (now or in the future) to its output
buffer.

> +}
> ...
> diff --git a/help.c b/help.c
> index 190722fb0a..44cee69c11 100644
> --- a/help.c
> +++ b/help.c
> @@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
>  	exit(1);
>  }
>  
> +void get_version_info(struct strbuf *buf, int show_build_options)
> +{
> +	/*
> +	 * The format of this string should be kept stable for compatibility
> +	 * with external projects that rely on the output of "git version".
> +	 *
> +	 * Always show the version, even if other options are given.
> +	 */
> +	strbuf_addf(buf, "git version %s\n", git_version_string);

This ends the output with a complete line when !show_build_options ...

> +	if (show_build_options) {
> +		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
> +		if (git_built_from_commit_string[0])
> +			strbuf_addf(buf, "built from commit: %s\n",
> +			       git_built_from_commit_string);
> +		else
> +			strbuf_addstr(buf, "no commit associated with this build\n");
> +		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
> +		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
> +		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
> +		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */

... and the pattern indicates the output will end with a complete
line when !!show_build_options, too.

> +	}
> +}

So, was the strbuf_complete_line() merely defensive programming?  It
may deserve a comment if it will stay there.

>  int cmd_version(int argc, const char **argv, const char *prefix)
>  {
> +	struct strbuf buf = STRBUF_INIT;
>  	int build_options = 0;
>  	const char * const usage[] = {
>  		N_("git version [<options>]"),
> @@ -637,26 +662,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
>  
>  	argc = parse_options(argc, argv, prefix, options, usage, 0);
>  
> -	/*
> -	 * The format of this string should be kept stable for compatibility
> -	 * with external projects that rely on the output of "git version".
> -	 *
> -	 * Always show the version, even if other options are given.
> -	 */
> -	printf("git version %s\n", git_version_string);
> +	get_version_info(&buf, build_options);
> +	printf("%s", buf.buf);
> +
> +	strbuf_release(&buf);
>  
> -	if (build_options) {
> -		printf("cpu: %s\n", GIT_HOST_CPU);
> -		if (git_built_from_commit_string[0])
> -			printf("built from commit: %s\n",
> -			       git_built_from_commit_string);
> -		else
> -			printf("no commit associated with this build\n");
> -		printf("sizeof-long: %d\n", (int)sizeof(long));
> -		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
> -		printf("shell-path: %s\n", SHELL_PATH);
> -		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
> -	}
>  	return 0;

Makes sense.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 05/15] bugreport: add uname info
  2020-02-20  1:58         ` [PATCH v8 05/15] bugreport: add uname info Emily Shaffer
@ 2020-02-20 20:12           ` Junio C Hamano
  2020-02-20 23:20             ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 20:12 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +	/* system call for other version info */
> +	strbuf_addstr(sys_info, "uname -a: ");

Is that "-a" portable across systems?  That is, when given "-a",
would "uname" on all platforms show sysname, release, version and
machine in that order and nothing else?

If we are merely saying "in this section, we are showing a selected
subset from what we learned about the system with uname(2)", perhaps
just stop at saying "uname: "?

> +	if (uname(&uname_info))
> +		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);

i18n/l10n?  Don't we want to use strerror() here?

> +	else
> +		strbuf_addf(sys_info, "%s %s %s %s\n",
> +			    uname_info.sysname,
> +			    uname_info.release,
> +			    uname_info.version,
> +			    uname_info.machine);
>  }
>  
>  static const char * const bugreport_usage[] = {

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 06/15] bugreport: add compiler info
  2020-02-20  1:58         ` [PATCH v8 06/15] bugreport: add compiler info Emily Shaffer
  2020-02-20  2:49           ` Danh Doan
@ 2020-02-20 20:23           ` Junio C Hamano
  2020-02-21  0:26           ` Junio C Hamano
  2 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 20:23 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> @@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
>  			    uname_info.release,
>  			    uname_info.version,
>  			    uname_info.machine);
> +
> +	strbuf_addstr(sys_info, "compiler info: ");
> +	get_compiler_info(sys_info);
> +	strbuf_complete_line(sys_info);
>  }
>  
>  static const char * const bugreport_usage[] = {
> diff --git a/compat/compiler.h b/compat/compiler.h
> new file mode 100644
> index 0000000000..ef41177233
> --- /dev/null
> +++ b/compat/compiler.h
> @@ -0,0 +1,27 @@
> +#ifndef COMPILER_H
> +#define COMPILER_H
> +
> +#include "git-compat-util.h"
> +#include "strbuf.h"
> +
> +#ifdef __GLIBC__
> +#include <gnu/libc-version.h>
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> +#ifdef __GNUC__
> +	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> +#endif
> +}
> +
> +#else
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +	strbuf_addstr(info, "get_compiler_info() not implemented");

i18n/l10n?  Also don't deliberately leave the buffer end with an
incomplete line like so---doing so _requires_ the caller above to
have strbuf_complete_line() call, but there is no reason to do so.

strbuf_complete_line() does make sense if you insert something that
is not controlled by us (e.g. "Please type whatever other comments
you may have" to let the user give us free-form text, which may or
may not end with a complete line), but otherwise it probably is a
sign of being unnecessary sloppy.

> +}

OK, so the idea is to insert "#elif ..." and definition of
get_compiler_info() for non GLIBC systems?

I am not sure why you want to make this a header with static inline
implementations.  Is it expected for this to be included from
multiple source files?  It would be more understandable if these
were in the same file as where get_system_info() is, perhaps
immediately before that function.

Also, it would probably be easier to manage if you have two separate
helpers, one for what compiler is in use, and the other for what
libc is in use.

> +
> +#endif
> +
> +#endif /* COMPILER_H */

Thanks.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 07/15] bugreport: add git-remote-https version
  2020-02-20  1:58         ` [PATCH v8 07/15] bugreport: add git-remote-https version Emily Shaffer
@ 2020-02-20 20:35           ` Junio C Hamano
  2020-02-20 23:28             ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 20:35 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +static void get_git_remote_https_version_info(struct strbuf *version_info)
> +{
> +	struct child_process cp = CHILD_PROCESS_INIT;
> +
> +	cp.git_cmd = 1;
> +	argv_array_push(&cp.args, "remote-https");
> +	argv_array_push(&cp.args, "--build-info");
> +	if (capture_command(&cp, version_info, 0))
> +	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
> +}
>  
>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
>  	strbuf_addstr(sys_info, "compiler info: ");
>  	get_compiler_info(sys_info);
>  	strbuf_complete_line(sys_info);
> +
> +	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
> +	get_git_remote_https_version_info(sys_info);
> +	strbuf_complete_line(sys_info);
>  }
>  
>  static const char * const bugreport_usage[] = {
> diff --git a/remote-curl.c b/remote-curl.c
> index 8eb96152f5..73e52175c0 100644
> --- a/remote-curl.c
> +++ b/remote-curl.c
> @@ -17,6 +17,7 @@
>  #include "protocol.h"
>  #include "quote.h"
>  #include "transport.h"
> +#include "version.h"
>  
>  static struct remote *remote;
>  /* always ends with a trailing slash */
> @@ -1375,6 +1376,13 @@ int cmd_main(int argc, const char **argv)
>  	string_list_init(&options.deepen_not, 1);
>  	string_list_init(&options.push_options, 1);
>  
> +	if (!strcmp("--build-info", argv[1])) {
> +		printf("git-http-fetch version: %s\n", git_version_string);

We are letting bugreport grab this information from remote-curl, and
they are different binaries, so git_version_string we see here is
that of the remote-curl.  Good.

> +		printf("built from commit: %s\n", git_built_from_commit_string);
> +		printf("curl version: %s\n", curl_version());
> +		return 0;
> +	}
> +
>  	/*
>  	 * Just report "remote-curl" here (folding all the various aliases
>  	 * ("git-remote-http", "git-remote-https", and etc.) here since they

Makes sense, except that it is not clear what our overall stance on
i18n/l10n of the bugreport output.

I think there are two schools of thought.  

 - You can stick to C local, with an expectation that developers can
   read reports in C locale.  This will allow developers to avoid
   having to read reports written in a language they do not
   understand, and also makes it easier to mechanically process
   reports.

 - You can make sure what matters in the report is localized to the
   end-users' locale.  This will avoid forcing end-users to send out
   a report that they may not even able to read (which is what
   happens if we did not do i18n/l10n).

I am not sure which way you are aiming at.  The boilerplate text
we saw in a very early part of the series were marked N_() but some
of the messages in later parts of the series are not.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 09/15] bugreport: generate config safelist based on docs
  2020-02-20  1:58         ` [PATCH v8 09/15] bugreport: generate config safelist based on docs Emily Shaffer
@ 2020-02-20 20:40           ` Junio C Hamano
  2020-02-26 16:13           ` Johannes Schindelin
  1 sibling, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 20:40 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Martin Ågren, Johannes Schindelin

Emily Shaffer <emilyshaffer@google.com> writes:

> --- /dev/null
> +++ b/generate-bugreport-config-safelist.sh
> @@ -0,0 +1,18 @@
> +#!/bin/sh
> +
> +cat <<EOF

Quote the EOF if you are not doing parameter expansion in HERE-DOC.

> +/* Automatically generated by bugreport-generate-config-safelist.sh */
> +
> +
> +static const char *bugreport_config_safelist[] = {
> +EOF
> +
> +# cat all regular files in Documentation/config
> +find Documentation/config -type f -exec cat {} \; |
> +# print the command name which matches the annotate-bugreport macro
> +sed -n 's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p' \
> +	| sort
> +

End the first line with a pipe '|' instead of a backslash, and lose
the pipe from the beginning of the second line.

> +cat <<EOF
> +};
> +EOF

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 10/15] bugreport: add config values from safelist
  2020-02-20  1:58         ` [PATCH v8 10/15] bugreport: add config values from safelist Emily Shaffer
@ 2020-02-20 20:47           ` Junio C Hamano
  0 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 20:47 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> @@ -94,6 +120,7 @@ int cmd_main(int argc, const char **argv)
>  	char *option_output = NULL;
>  	char *option_suffix = "%F-%H%M";
>  	struct stat statbuf;
> +	int nongit_ok = 0;
>  
>  	const struct option bugreport_options[] = {
>  		OPT_STRING('o', "output-directory", &option_output, N_("path"),
> @@ -102,6 +129,10 @@ int cmd_main(int argc, const char **argv)
>  			   N_("specify a strftime format suffix for the filename")),
>  		OPT_END()
>  	};
> +
> +	/* Prerequisite for hooks and config checks */
> +	setup_git_directory_gently(&nongit_ok);

Now this starts to break "-o" option when the command is run from a
subdirectory of a git working tree, I suspect.

The setup() will discover that you are in a Git controlled working
tree, goes up to the top-level of the working tree, so a user may
expect

	cd t && git bugreport -o frotz

to create a new directory t/frotz, but it would instead create the
directory frotz next to t/, no?

Using OPT_FILENAME and feeding correct prefix to parse_options()
will correct this.

>  	argc = parse_options(argc, argv, "", bugreport_options,
>  			     bugreport_usage, 0);

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 11/15] bugreport: collect list of populated hooks
  2020-02-20  1:58         ` [PATCH v8 11/15] bugreport: collect list of populated hooks Emily Shaffer
@ 2020-02-20 20:58           ` Junio C Hamano
  2020-02-25 23:19             ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 20:58 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +static void get_populated_hooks(struct strbuf *hook_info, int nongit)
> +{
> +	/*
> +	 * Doesn't look like there is a list of all possible hooks; so below is
> +	 * a transcription of `git help hooks`.
> +	 */

It may want to become a NEEDSWORK comment.

> +	const char *hooks = "applypatch-msg,"
> +			    "pre-applypatch,"
> +			    "post-applypatch,"
> +			    "pre-commit,"
> +			    "pre-merge-commit,"
> +			    "prepare-commit-msg,"
> +			    "commit-msg,"
> +			    "post-commit,"
> +			    "pre-rebase,"
> +			    "post-checkout,"
> +			    "post-merge,"
> +			    "pre-push,"
> +			    "pre-receive,"
> +			    "update,"
> +			    "post-receive,"
> +			    "post-update,"
> +			    "push-to-checkout,"
> +			    "pre-auto-gc,"
> +			    "post-rewrite,"
> +			    "sendemail-validate,"
> +			    "fsmonitor-watchman,"
> +			    "p4-pre-submit,"
> +			    "post-index-changex";

Typo here?

> +	struct string_list hooks_list = STRING_LIST_INIT_DUP;
> +	struct string_list_item *iter = NULL;
> +
> +
> +	if (nongit) {
> +		strbuf_addstr(hook_info,
> +			"not run from a git repository - no hooks to show\n");
> +		return;
> +	}
> +
> +	string_list_split(&hooks_list, hooks, ',', -1);
> +
> +	for_each_string_list_item(iter, &hooks_list) {

I do not get why you want to use string_list for this, especially if
you need to use string_list_split.

To me,

	int i;
	const char *hook[] = {
		"applypatch-msg",
	        "pre-applypatch",
		...
		"post-index-change",
	};

	for (i = 0; i < ARRAY_SIZE(hook); i++) {
		if (hook[i] is enabled)
			strbuf_addf(hook_info, "%s\n", hook[i]);
	}

would be far easier to understand.  Do you have an external source
that can feed you a single long string of comma separated hook names
in mind, so that the initialization of *hooks will become simpler
that way, or something?

> +		if (find_hook(iter->string)) {
> +			strbuf_addstr(hook_info, iter->string);
> +			strbuf_complete_line(hook_info);
> +		}
> +	}
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
>  	NULL
> @@ -166,6 +216,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Safelisted Config Info");
>  	get_safelisted_config(&buffer);
>  
> +	get_header(&buffer, "Enabled Hooks");
> +	get_populated_hooks(&buffer, nongit_ok);
> +
>  	report = fopen_for_writing(report_path.buf);
>  
>  	if (report == NULL) {

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 12/15] bugreport: count loose objects
  2020-02-20  1:58         ` [PATCH v8 12/15] bugreport: count loose objects Emily Shaffer
@ 2020-02-20 21:04           ` Junio C Hamano
  2020-02-25 23:22             ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 21:04 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Johannes Schindelin

Emily Shaffer <emilyshaffer@google.com> writes:

> The number of unpacked objects in a user's repository may help us
> understand the root of the problem they're seeing, especially if a
> command is running unusually slowly.
>
> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  Documentation/git-bugreport.txt |  1 +
>  bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
>  2 files changed, 53 insertions(+)
>
> diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
> index 4d01528540..4fe1c60506 100644
> --- a/Documentation/git-bugreport.txt
> +++ b/Documentation/git-bugreport.txt
> @@ -32,6 +32,7 @@ The following information is captured automatically:
>   - $SHELL
>   - Selected config values
>   - A list of enabled hooks
> + - The number of loose objects in the repository
>  
>  This tool is invoked via the typical Git setup process, which means that in some
>  cases, it might not be able to launch - for example, if a relevant config file
> diff --git a/bugreport.c b/bugreport.c
> index b5a0714a7f..fb7bc72723 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -10,6 +10,7 @@
>  #include "bugreport-config-safelist.h"
>  #include "khash.h"
>  #include "run-command.h"
> +#include "object-store.h"
>  
>  static void get_git_remote_https_version_info(struct strbuf *version_info)
>  {
> @@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
>  	}
>  }
>  
> +static int loose_object_cb(const struct object_id *oid, const char *path,
> +			   void *data) {
> +	int *loose_object_count = data;
> +
> +	if (loose_object_count) {
> +		(*loose_object_count)++;
> +		return 0;
> +	}
> +
> +	return 1;

What is the point of returning 1 here to abort the iteration early?
Wouldn't it be a BUG() if this callback ends up gettting called with
NULL in data?

> +}
> +
> +static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
> +
> +	int local_loose_object_count = 0, total_loose_object_count = 0;
> +	int local_count_questionable = 0, total_count_questionable = 0;
> +
> +	if (nongit) {
> +		strbuf_addstr(obj_info,
> +			"not run from a git repository - no objects to show\n");
> +		return;
> +	}
> +
> +	local_count_questionable = for_each_loose_object(
> +					loose_object_cb,
> +					&local_loose_object_count,
> +					FOR_EACH_OBJECT_LOCAL_ONLY);
> +
> +	total_count_questionable = for_each_loose_object(
> +					loose_object_cb,
> +					&total_loose_object_count,
> +					0);
> +	strbuf_addf(obj_info, "%d local loose objects%s\n",
> +		    local_loose_object_count,
> +		    local_count_questionable ? " (problem during count)" : "");
> +
> +	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
> +		    total_loose_object_count - local_loose_object_count,
> +		    (local_count_questionable || total_count_questionable)
> +			? " (problem during count)"
> +			: "");
> +
> +	strbuf_addf(obj_info, "%d total loose objects%s\n",
> +		    total_loose_object_count,
> +		    total_count_questionable ? " (problem during count)" : "");
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
>  	NULL
> @@ -219,6 +268,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Enabled Hooks");
>  	get_populated_hooks(&buffer, nongit_ok);
>  
> +	get_header(&buffer, "Loose Object Counts");
> +	get_loose_object_summary(&buffer, nongit_ok);
> +
>  	report = fopen_for_writing(report_path.buf);
>  
>  	if (report == NULL) {

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 02/15] help: add shell-path to --build-options
  2020-02-20 19:03           ` Junio C Hamano
@ 2020-02-20 21:15             ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20 21:15 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Feb 20, 2020 at 11:03:59AM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > It may be useful to know which shell Git was built to try to point to,
> > in the event that shell-based Git commands are failing. $SHELL_PATH is
> > set during the build and used to launch the manpage viewer, as well as
> > by git-compat-util.h, and it's used during tests. 'git version
> > --build-options' is encouraged for use in bug reports, so it makes sense
> > to include this information there.
> 
> It probably makes sense to capture PERL_PATH, PYTHON_PATH,
> TCLTK_PATH, and TCL_PATH, as they fall into exactly the same
> category as this one.

SHELL_PATH was already made sure to exist in git-compat-util.h before
this change; these other utility paths are not. Do you want me to add
them now?

> 
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  help.c | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/help.c b/help.c
> > index a21487db77..190722fb0a 100644
> > --- a/help.c
> > +++ b/help.c
> > @@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
> >  			printf("no commit associated with this build\n");
> >  		printf("sizeof-long: %d\n", (int)sizeof(long));
> >  		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
> > +		printf("shell-path: %s\n", SHELL_PATH);
> >  		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
> >  	}
> >  	return 0;

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 13/15] bugreport: add packed object summary
  2020-02-20  1:58         ` [PATCH v8 13/15] bugreport: add packed object summary Emily Shaffer
@ 2020-02-20 22:04           ` Junio C Hamano
  2020-02-25 23:58             ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 22:04 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
> +{
> +	struct packed_git *pack = NULL;
> +	int pack_count = 0;
> +	int object_count = 0;
> +
> +	if (nongit) {
> +		strbuf_addstr(obj_info,
> +			"not run from a git repository - no objects to show\n");
> +		return;
> +	}
> +
> +	for_each_pack(the_repository, pack) {
> +		pack_count++;
> +		/*
> +		 * To accurately count how many objects are packed, look inside
> +		 * the packfile's index.
> +		 */
> +		open_pack_index(pack);
> +		object_count += pack->num_objects;
> +	}
> +
> +	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
> +		    object_count);
> +
> +}

Makes sense.

> @@ -447,4 +448,9 @@ int for_each_object_in_pack(struct packed_git *p,
>  int for_each_packed_object(each_packed_object_fn, void *,
>  			   enum for_each_object_flags flags);
>  
> +#define for_each_pack(repo, pack) 		\
> +		for (pack = get_all_packs(repo);\
> +		     pack;			\
> +		     pack = pack->next)

I generally avoid #define'ing a control loop pseudo-syntax unless it
makes the resulting code hide overly ugly implementation detail.

for_each_string_list() is there to hide the fact that items are
stored in an embedded array whose name is .items and size is .nr
that is sufficiently ugnly to expose, but iterating over packs
does not look so bad.

If you MUST have this as a pseudo-syntax macro, we need

 - to match for_each_string_list_item(), have iterating variable
   'pack' as the first parameter, and the scope of what's iterated
   'repo' as the second.

 - to make sure the syntax works correctly even if a parameter is
   *not* a simple identifier (I think the above is OK, but there may
   be cases that it does not work well).

Regarding the latter, the way 'item' is incremented at the end of
iteration in for_each_string_list_item() is subtle and correct.

#define for_each_string_list_item(item,list)            \
	for (item = (list)->items;                      \
	     item && item < (list)->items + (list)->nr; \
	     ++item)

You would break it if you changed pre-increment to post-increment
for a user like this:

	struct string_list *list;
	struct string_list_item *i, **p;
	p = &i;

	for_each_string_list_item(*p, list) {
		...
	}

because ++*p is ++(*p), while *p++ is (*p)++, and we do want the
former (i.e. increment the memory cell pointed at by pointer p).

Personally, I would prefer not to introduce this macro if I were
working on this topic.

>  #endif /* OBJECT_STORE_H */
> diff --git a/packfile.c b/packfile.c
> index 99dd1a7d09..95afcc1187 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -2095,8 +2095,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
>  	int r = 0;
>  	int pack_errors = 0;
>  
> -	prepare_packed_git(the_repository);
> -	for (p = get_all_packs(the_repository); p; p = p->next) {
> +	for_each_pack(the_repository, p) {
>  		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
>  			continue;
>  		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 14/15] bugreport: list contents of $OBJDIR/info
  2020-02-20  1:58         ` [PATCH v8 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
@ 2020-02-20 22:18           ` Junio C Hamano
  0 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 22:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +static void list_contents_of_dir_recursively(struct strbuf *contents,
> +					     struct strbuf *dirpath)
> +{
> +	struct dirent *d;
> +	DIR *dir;
> +	size_t path_len;
> +
> +	dir = opendir(dirpath->buf);
> +	if (!dir)
> +		return;

No error detected?

> +	strbuf_complete(dirpath, '/');
> +	path_len = dirpath->len;
> +
> +	while ((d = readdir(dir))) {
> +		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
> +			continue;
> +
> +		strbuf_addbuf(contents, dirpath);
> +		strbuf_addstr(contents, d->d_name);
> +		strbuf_complete_line(contents);
> +
> +		if (d->d_type == DT_DIR) {
> +			strbuf_addstr(dirpath, d->d_name);
> +			list_contents_of_dir_recursively(contents, dirpath);
> +		}
> +		strbuf_setlen(dirpath, path_len);

Shouldn't we be sorting the output?

> +	}
> +
> +	closedir(dir);
> +}

Hmph, it is somewhat sad that we have to reinvent a degraded copy of
"ls -aR" like this.

> +static void get_object_info_summary(struct strbuf *obj_info, int nongit)
> +{
> +	struct strbuf dirpath = STRBUF_INIT;
> +
> +	if (nongit) {
> +		strbuf_addstr(obj_info,
> +			"not run from a git repository - object info unavailable\n");
> +		return;
> +	}
> +
> +	strbuf_addstr(&dirpath, get_object_directory());
> +	strbuf_complete(&dirpath, '/');
> +	strbuf_addstr(&dirpath, "info/");

Would it help to use git_path() for this, perhaps like
git_path("objects/info/")?

By the way, do we store anything worth knowing in $GIT_DIR/info too?
Perhaps the per-repo attributes and excludes files whose presence
may hint at another question to ask when a user reports unexpected
behaviour from .gitignore?

> +	list_contents_of_dir_recursively(obj_info, &dirpath);
> +	strbuf_release(&dirpath);
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
>  	NULL
> @@ -301,6 +352,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Packed Object Summary");
>  	get_packed_object_summary(&buffer, nongit_ok);
>  
> +	get_header(&buffer, "Object Info Summary");
> +	get_object_info_summary(&buffer, nongit_ok);
> +
>  	report = fopen_for_writing(report_path.buf);
>  
>  	if (report == NULL) {

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 15/15] bugreport: summarize contents of alternates file
  2020-02-20  1:58         ` [PATCH v8 15/15] bugreport: summarize contents of alternates file Emily Shaffer
  2020-02-20 14:08           ` Johannes Schindelin
@ 2020-02-20 22:22           ` Junio C Hamano
  1 sibling, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 22:22 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
> +{
> +	struct strbuf alternates_path = STRBUF_INIT;
> +	struct strbuf alternate = STRBUF_INIT;
> +	FILE *file;
> +	size_t exists = 0, broken = 0;
> +
> +	if (nongit) {
> +		strbuf_addstr(alternates_info,
> +			"not run from a git repository - alternates unavailable\n");
> +		return;
> +	}
> +
> +	strbuf_addstr(&alternates_path, get_object_directory());
> +	strbuf_complete(&alternates_path, '/');
> +	strbuf_addstr(&alternates_path, "info/alternates");

git_path()?

> +	file = fopen(alternates_path.buf, "r");
> +	if (!file) {
> +		strbuf_addstr(alternates_info, "No alternates file found.\n");
> +		strbuf_release(&alternates_path);
> +		return;
> +	}
> +
> +	while (strbuf_getline(&alternate, file) != EOF) {
> +		if (!access(alternate.buf, F_OK))
> +			exists++;

F_OK reported as "exists" makes quite a lot of sense.

> +		else
> +			broken++;

So, shouldn't this rather be "missing"?

> +	}
> +
> +	strbuf_addf(alternates_info,
> +		    "%zd alternates found (%zd working, %zd broken)\n",
> +		    exists + broken,
> +		    exists,
> +		    broken);

I don't see the point of using size_t for this.  Just use int for
both, like you did for object count in the step that counds loose
and packed objects.

> +	fclose(file);
> +	strbuf_release(&alternate);
> +	strbuf_release(&alternates_path);
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
>  	NULL
> @@ -355,6 +397,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Object Info Summary");
>  	get_object_info_summary(&buffer, nongit_ok);
>  
> +	get_header(&buffer, "Alternates");
> +	get_alternates_summary(&buffer, nongit_ok);
> +
>  	report = fopen_for_writing(report_path.buf);
>  
>  	if (report == NULL) {

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 06/15] bugreport: add compiler info
  2020-02-19 22:45           ` Emily Shaffer
@ 2020-02-20 22:33             ` Johannes Schindelin
  2020-02-20 23:33               ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-02-20 22:33 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Wed, 19 Feb 2020, Emily Shaffer wrote:

>   #ifdef __GLIBC__
>   #include <gnu/libc-version.h>
>   #endif
>
>   static inline void get_compiler_info(struct strbuf *info)
>   {
>   	#ifdef __GLIBC__
> 	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> 	#endif
>
> 	#ifdef __GNUC__
> 	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> 	#endif
>
> 	#ifdef _MSC_VER
> 	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());

You could do it this way right away:

	strbuf_addf(info, "MSVC version: %d.%d\n",
		    _MSC_VER / 100, _MSC_VER % 100);

Note: this is _not_ the MSVC _runtime_ version, but really the compiler
version.

You could also use _MSC_FULL_VER, which is a bit more complete.

> 	#endif
>   }
>
> The thinking being - this way if I decide to use, say, LLVM + glibc,
> then I don't need to reimplement this command with all the glibc
> diagnostics again. Or, if someone else already wrote diagnostics for
> LLVM with some other libc, then it even Just Works for me and my new
> combination.
>
> That said, I'm reasoning about these combinations of compilers and libcs
> and whatever else from an inexperienced viewpoint, so maybe this isn't
> necessary?

I would have hoped that the argument I made earlier about the broken GCC
version would have convinced you that the answer to this question is: Yes,
this is necessary.

Ciao,
Dscho

>
>  - Emily
>

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 03/15] bugreport: add tool to generate debugging info
  2020-02-20 19:33           ` Junio C Hamano
@ 2020-02-20 22:33             ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20 22:33 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Feb 20, 2020 at 11:33:26AM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +	argc = parse_options(argc, argv, "", bugreport_options,
> > +			     bugreport_usage, 0);
> 
> Which one between an empty string and NULL is more appropriate to be
> passed as "prefix" here?  I assume that this is *not* really a git
> program, and no repository/working-tree discovery is involved, and
> you won't be using OPT_FILENAME, so it would probably be OK.
> 
> > +
> > +	if (option_output) {
> > +		strbuf_addstr(&report_path, option_output);
> > +		strbuf_complete(&report_path, '/');
> > +	}
> > +
> > +
> > +	strbuf_addstr(&report_path, "git-bugreport-");
> > +	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
> > +	strbuf_addstr(&report_path, ".txt");
> > +
> > +	if (!stat(report_path.buf, &statbuf))
> > +		die("'%s' already exists", report_path.buf);
> 
> Missed i18n/l10n, but I do not think it is a useful thing for this
> check to be here in the first place.
> 
> > +	switch (safe_create_leading_directories(report_path.buf)) {
> > +	case SCLD_OK:
> > +	case SCLD_EXISTS:
> > +		break;
> > +	default:
> > +		die(_("could not create leading directories for '%s'"),
> > +		    report_path.buf);
> > +	}
> > +
> > +	get_bug_template(&buffer);
> > +
> > +	report = fopen_for_writing(report_path.buf);
> 
> fopen_for_writing() does not give good semantics for what you seem
> to want to do here (i.e. to make sure you do not overwrite an
> existing one).  It even has "if we cannot open it, retry after
> unlinking" logic in it, which screams "this function is designed for
> those who want to overwrite the file", and if you look at its
> callers, they are _all_ about opening a file for writing with a well
> known name like FETCH_HEAD, COMMIT_EDITMSG, etc.
> 
> Besides, a stat() that dies when a file exists would not
> help---since that check, you've spent non-zero time, creating
> leading directories and preparing boilerplate in the buffer,
> and there is no guarantee somebody else used the same filename
> in the meantime.

Good point. Ouch.

> 
> If you want to avoid overwriting an existing file (which I think is
> a good idea---I just do not think the earlier "if (!stat()) die()"
> is a good way to do so), the only sensible way to do so is to ask
> your open/creat to fail if there already is a file---that is how
> you'd avoid racing with another process that may be creating such a
> file.  Grep for O_EXCL to find where the flag is used with O_CREAT
> to see how it is done.

Sure. Thanks, I reworked it to use open().

By the way, I noticed reading the GNU manual that file descriptors may
not be supported outside of GNU environments; but I also noticed that 1)
Git uses open() lots of places to use O_EXCL, and 2) fopen() doesn't
support exclusive opens (except in glibc, and nobody is using that
particular option in the Git codebase now). So I guess it's safe to use
open()...

> 
> > +	if (report == NULL) {
> 
> Style. "if (!report)"

The type of 'report' changes using open(), so this check changes too.
Now it says "if (report < 0)" - per the open() doc, it returns a
positive fd or -1.

> 
> > +		strbuf_release(&report_path);
> > +		die("couldn't open '%s' for writing", report_path.buf);
> 
> Do we see a use-after-free here?  Also missed i18n/l10n.

Hm. I suppose it's OK to UNLEAK() like we do at the successful exit
since the die() will end the process. Added i18n too.

> It is embarrassing on the reviewers' side that this (which is not
> new in this round at all and hasn't changed since v6) hasn't been
> caught for a few rounds X-<.

Well, I'm embarrassed to have written it in the first place.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 07/15] bugreport: add git-remote-https version
  2020-02-19 22:33             ` Junio C Hamano
@ 2020-02-20 22:33               ` Johannes Schindelin
  0 siblings, 0 replies; 273+ messages in thread
From: Johannes Schindelin @ 2020-02-20 22:33 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Emily Shaffer, git

Hi Junio,

On Wed, 19 Feb 2020, Junio C Hamano wrote:

> Emily Shaffer <emilyshaffer@google.com> writes:
>
> >> Also, in keeping with the existing code, we would need to use
> >> `--build-options` here (this is what `git version` calls the equivalent
> >> mode).
> >>
> >> _However_.
> >>
> >> I like your `--build-info` a lot more than `--build-options` (because the
> >> latter is very misleading: the commit and the date of the build are not
> >> "options" at all).
> >
> > Sure. Thanks for saying so.
>
> I don't think anybody would mind introducing --build-info to "git
> version" as a synonym and deprecate --build-options from it ;-)

Heh. I almost suggested this. Almost.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 04/15] bugreport: gather git version and build info
  2020-02-20 20:07           ` Junio C Hamano
@ 2020-02-20 23:03             ` Emily Shaffer
  2020-02-20 23:18               ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20 23:03 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Feb 20, 2020 at 12:07:46PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +static void get_system_info(struct strbuf *sys_info)
> > +{
> > +	/* get git version from native cmd */
> > +	strbuf_addstr(sys_info, "git version:\n");
> > +	get_version_info(sys_info, 1);
> > +	strbuf_complete_line(sys_info);
> 
> It is a bit curious use of "don't do anything if sys_info ends with
> a complete line, but complete it if it ends with an imcomplete
> line".  That tells the readers that we do not know what
> get_version_info() will do (now or in the future) to its output
> buffer.
> 
> > +}
> > ...
> > diff --git a/help.c b/help.c
> > index 190722fb0a..44cee69c11 100644
> > --- a/help.c
> > +++ b/help.c
> > @@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
> >  	exit(1);
> >  }
> >  
> > +void get_version_info(struct strbuf *buf, int show_build_options)
> > +{
> > +	/*
> > +	 * The format of this string should be kept stable for compatibility
> > +	 * with external projects that rely on the output of "git version".
> > +	 *
> > +	 * Always show the version, even if other options are given.
> > +	 */
> > +	strbuf_addf(buf, "git version %s\n", git_version_string);
> 
> This ends the output with a complete line when !show_build_options ...
> 
> > +	if (show_build_options) {
> > +		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
> > +		if (git_built_from_commit_string[0])
> > +			strbuf_addf(buf, "built from commit: %s\n",
> > +			       git_built_from_commit_string);
> > +		else
> > +			strbuf_addstr(buf, "no commit associated with this build\n");
> > +		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
> > +		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
> > +		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
> > +		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
> 
> ... and the pattern indicates the output will end with a complete
> line when !!show_build_options, too.
> 
> > +	}
> > +}
> 
> So, was the strbuf_complete_line() merely defensive programming?  It
> may deserve a comment if it will stay there.

It was meant defensively, here and elsewhere in the series. I figured
that for something like this, which is mostly bounded by human writing
in an editor and then by file IO, spurious string-checking was not such
a big deal.

Are you suggesting to comment around the strbuf_complete_line() calls,
or to comment around get_version_info() that it should end in newline?

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 04/15] bugreport: gather git version and build info
  2020-02-20 23:03             ` Emily Shaffer
@ 2020-02-20 23:18               ` Junio C Hamano
  0 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-02-20 23:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> On Thu, Feb 20, 2020 at 12:07:46PM -0800, Junio C Hamano wrote:
>> Emily Shaffer <emilyshaffer@google.com> writes:
>> 
>> > +static void get_system_info(struct strbuf *sys_info)
>> > +{
>> > +	/* get git version from native cmd */
>> > +	strbuf_addstr(sys_info, "git version:\n");
>> > +	get_version_info(sys_info, 1);
>> > +	strbuf_complete_line(sys_info);
>> 
>> It is a bit curious use of "don't do anything if sys_info ends with
>> a complete line, but complete it if it ends with an imcomplete
>> line".  That tells the readers that we do not know what
>> get_version_info() will do (now or in the future) to its output
>> buffer.
>>  ...
>> So, was the strbuf_complete_line() merely defensive programming?  It
>> may deserve a comment if it will stay there.
>
> It was meant defensively, here and elsewhere in the series. I figured
> that for something like this, which is mostly bounded by human writing
> in an editor and then by file IO, spurious string-checking was not such
> a big deal.
>
> Are you suggesting to comment around the strbuf_complete_line() calls,
> or to comment around get_version_info() that it should end in newline?

I meant a comment in get_system_info() next to the use of this
particular use of strbuf_complete_line(), if the use stays there.

But after reading the whole series through, I saw no need to use
strbuf_complete_line() in the first place, as there is no source of
input that is not under our control (if we do not count sloppy
programming, that is).

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 05/15] bugreport: add uname info
  2020-02-20 20:12           ` Junio C Hamano
@ 2020-02-20 23:20             ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20 23:20 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Feb 20, 2020 at 12:12:58PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +	/* system call for other version info */
> > +	strbuf_addstr(sys_info, "uname -a: ");
> 
> Is that "-a" portable across systems?  That is, when given "-a",
> would "uname" on all platforms show sysname, release, version and
> machine in that order and nothing else?
> 
> If we are merely saying "in this section, we are showing a selected
> subset from what we learned about the system with uname(2)", perhaps
> just stop at saying "uname: "?

Sure, that's good enough for me. I think actually 'uname -a' on a GNU
system isn't going to match what's output here anyways, since I don't
show the nodename and 'uname -a' would.

> 
> > +	if (uname(&uname_info))
> > +		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
> 
> i18n/l10n?  Don't we want to use strerror() here?

Much much earlier[1] in the series, you suggested not i18n/l10n the
contents of the bugreport, as this list would probably be ill-equipped
to process such a report. Since it's being added to sys_info, that means this line
is bound for the finished report, not for the user.

Speaking of locale, strerror() will print the error in the current
locale; if we're still assuming "the list only takes reports in English"
should I be explicit and use strerror_l()? Should I print strerror() in
current locale and include the errno, e.g. ("uname() failed with '%s'
(%d)\n", strerror(errno), errno)?

[1] https://lore.kernel.org/git/xmqqzhka2tbv.fsf@gitster-ct.c.googlers.com

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 06/15] bugreport: add compiler info
  2020-02-20  2:49           ` Danh Doan
@ 2020-02-20 23:23             ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20 23:23 UTC (permalink / raw)
  To: Danh Doan; +Cc: git

On Thu, Feb 20, 2020 at 09:49:03AM +0700, Danh Doan wrote:
> On 2020-02-19 17:58:49-0800, Emily Shaffer <emilyshaffer@google.com> wrote:
> > diff --git a/compat/compiler.h b/compat/compiler.h
> > new file mode 100644
> > index 0000000000..ef41177233
> > --- /dev/null
> > +++ b/compat/compiler.h
> > @@ -0,0 +1,27 @@
> > +#ifndef COMPILER_H
> > +#define COMPILER_H
> > +
> > +#include "git-compat-util.h"
> > +#include "strbuf.h"
> > +
> > +#ifdef __GLIBC__
> > +#include <gnu/libc-version.h>
> > +
> > +static inline void get_compiler_info(struct strbuf *info)
> > +{
> > +	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> > +#ifdef __GNUC__
> > +	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> > +#endif
> 
> I think we're better having
> 
> - get_compiler_info placed under #ifdef __GNUC__ gate,
> - get_libc_info under #ifdef __GLIBC
> 
> Then have a function to merge those information together.
> Correct me if I were wrong, as it is now,
> this is only useful for Linux people with glibc.
> 
> Anyway, clang also defines __GNUC__ and __GNUC_MINOR__,
> IIRC, FreeBSD people started to move to use clang instead of gcc
> as their default compiler.
> 
> Gentoo forks are also known to use clang to compile their ports.

Yeah, between your comments now and Dscho's comments on this patch in
the last rollup, I'll go ahead and refactor it like you suggest.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 07/15] bugreport: add git-remote-https version
  2020-02-20 20:35           ` Junio C Hamano
@ 2020-02-20 23:28             ` Emily Shaffer
  2020-02-21  3:44               ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20 23:28 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Feb 20, 2020 at 12:35:37PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +static void get_git_remote_https_version_info(struct strbuf *version_info)
> > +{
> > +	struct child_process cp = CHILD_PROCESS_INIT;
> > +
> > +	cp.git_cmd = 1;
> > +	argv_array_push(&cp.args, "remote-https");
> > +	argv_array_push(&cp.args, "--build-info");
> > +	if (capture_command(&cp, version_info, 0))
> > +	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
> > +}
> >  
> >  static void get_system_info(struct strbuf *sys_info)
> >  {
> > @@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
> >  	strbuf_addstr(sys_info, "compiler info: ");
> >  	get_compiler_info(sys_info);
> >  	strbuf_complete_line(sys_info);
> > +
> > +	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
> > +	get_git_remote_https_version_info(sys_info);
> > +	strbuf_complete_line(sys_info);
> >  }
> >  
> >  static const char * const bugreport_usage[] = {
> > diff --git a/remote-curl.c b/remote-curl.c
> > index 8eb96152f5..73e52175c0 100644
> > --- a/remote-curl.c
> > +++ b/remote-curl.c
> > @@ -17,6 +17,7 @@
> >  #include "protocol.h"
> >  #include "quote.h"
> >  #include "transport.h"
> > +#include "version.h"
> >  
> >  static struct remote *remote;
> >  /* always ends with a trailing slash */
> > @@ -1375,6 +1376,13 @@ int cmd_main(int argc, const char **argv)
> >  	string_list_init(&options.deepen_not, 1);
> >  	string_list_init(&options.push_options, 1);
> >  
> > +	if (!strcmp("--build-info", argv[1])) {
> > +		printf("git-http-fetch version: %s\n", git_version_string);
> 
> We are letting bugreport grab this information from remote-curl, and
> they are different binaries, so git_version_string we see here is
> that of the remote-curl.  Good.
> 
> > +		printf("built from commit: %s\n", git_built_from_commit_string);
> > +		printf("curl version: %s\n", curl_version());
> > +		return 0;
> > +	}
> > +
> >  	/*
> >  	 * Just report "remote-curl" here (folding all the various aliases
> >  	 * ("git-remote-http", "git-remote-https", and etc.) here since they
> 
> Makes sense, except that it is not clear what our overall stance on
> i18n/l10n of the bugreport output.
> 
> I think there are two schools of thought.  
> 
>  - You can stick to C local, with an expectation that developers can
>    read reports in C locale.  This will allow developers to avoid
>    having to read reports written in a language they do not
>    understand, and also makes it easier to mechanically process
>    reports.
> 
>  - You can make sure what matters in the report is localized to the
>    end-users' locale.  This will avoid forcing end-users to send out
>    a report that they may not even able to read (which is what
>    happens if we did not do i18n/l10n).
> 
> I am not sure which way you are aiming at.  The boilerplate text
> we saw in a very early part of the series were marked N_() but some
> of the messages in later parts of the series are not.

I mentioned it in another reply to you at a different point in the v8
conversation; I took away the localization on the boilerplate at your
suggestion. Unfortunately I agree that we wouldn't be equipped to handle
reports in other languages. Not being able to understand the
user-fillable part of the report extends, I think, to not being able to
understand the diagnostic info. If the point of the report is to skip
back-and-forth, then writing back to say "actually, run all this stuff
manually so we can see the output in English" is not going to achieve
that goal.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 06/15] bugreport: add compiler info
  2020-02-20 22:33             ` Johannes Schindelin
@ 2020-02-20 23:33               ` Emily Shaffer
  2020-02-21 15:22                 ` Johannes Schindelin
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-20 23:33 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git

On Thu, Feb 20, 2020 at 11:33:05PM +0100, Johannes Schindelin wrote:
> Hi Emily,
> 
> On Wed, 19 Feb 2020, Emily Shaffer wrote:
> 
> >   #ifdef __GLIBC__
> >   #include <gnu/libc-version.h>
> >   #endif
> >
> >   static inline void get_compiler_info(struct strbuf *info)
> >   {
> >   	#ifdef __GLIBC__
> > 	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> > 	#endif
> >
> > 	#ifdef __GNUC__
> > 	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> > 	#endif
> >
> > 	#ifdef _MSC_VER
> > 	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());
> 
> You could do it this way right away:
> 
> 	strbuf_addf(info, "MSVC version: %d.%d\n",
> 		    _MSC_VER / 100, _MSC_VER % 100);
> 
> Note: this is _not_ the MSVC _runtime_ version, but really the compiler
> version.
> 
> You could also use _MSC_FULL_VER, which is a bit more complete.

Sorry, but I'm not comfortable sending code I can't check for myself
(and already muscle-memoried into my format-patch/send-email workflow).
If you send a scissors I can roll it into the series with your SOB.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 06/15] bugreport: add compiler info
  2020-02-20  1:58         ` [PATCH v8 06/15] bugreport: add compiler info Emily Shaffer
  2020-02-20  2:49           ` Danh Doan
  2020-02-20 20:23           ` Junio C Hamano
@ 2020-02-21  0:26           ` Junio C Hamano
  2 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-02-21  0:26 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +	strbuf_addstr(sys_info, "compiler info: ");
> +	get_compiler_info(sys_info);
> +	strbuf_complete_line(sys_info);

Continuing the response to 04/15 review, I do not think this is a
good use of complete_line(), either.  On an exotic platform you
haven't seen, get_compiler_info() might stuff nothing in the output
buffer, in which case we are left with an incomplete line that just
says "compiler info: ", and it might be better not to leave that
incomplete line hanging around by calling complete_line(), but an
even better solution would be to make sure get_compiler_info()
explicitly say it encountered a system totally new to it.  And at
that point, as long as everybody in get_compiler_info() ends its
output with a complete line, there is no need for complete_line()
on the caller's side.

Of course, you can make it a convention that all case arms or ifdef
blocks in get_compiler_info() uniformly end what they have to say
with an incomplete line and make it a responsibility of the caller
to terminate the incomplete line.  But in that case, you'd not be
using complete_line(), but strbuf_addch('\n').


^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 07/15] bugreport: add git-remote-https version
  2020-02-20 23:28             ` Emily Shaffer
@ 2020-02-21  3:44               ` Junio C Hamano
  2020-02-25 22:08                 ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-21  3:44 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> ... Unfortunately I agree that we wouldn't be equipped to handle
> reports in other languages.

I actually was anticipating a far better world ;-) 

There is no reason to limit the recipient of reports only to *us*.
Use of Git and population proficient in Git would become wide enough
that we should be able to expect that users who speak language X can
be helped by experts who speak the same language X with their
issues.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 06/15] bugreport: add compiler info
  2020-02-20 23:33               ` Emily Shaffer
@ 2020-02-21 15:22                 ` Johannes Schindelin
  2020-02-22  0:04                   ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-02-21 15:22 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 20 Feb 2020, Emily Shaffer wrote:

> On Thu, Feb 20, 2020 at 11:33:05PM +0100, Johannes Schindelin wrote:
> > Hi Emily,
> >
> > On Wed, 19 Feb 2020, Emily Shaffer wrote:
> >
> > >   #ifdef __GLIBC__
> > >   #include <gnu/libc-version.h>
> > >   #endif
> > >
> > >   static inline void get_compiler_info(struct strbuf *info)
> > >   {
> > >   	#ifdef __GLIBC__
> > > 	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> > > 	#endif
> > >
> > > 	#ifdef __GNUC__
> > > 	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> > > 	#endif
> > >
> > > 	#ifdef _MSC_VER
> > > 	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());
> >
> > You could do it this way right away:
> >
> > 	strbuf_addf(info, "MSVC version: %d.%d\n",
> > 		    _MSC_VER / 100, _MSC_VER % 100);
> >
> > Note: this is _not_ the MSVC _runtime_ version, but really the compiler
> > version.
> >
> > You could also use _MSC_FULL_VER, which is a bit more complete.
>
> Sorry, but I'm not comfortable sending code I can't check for myself
> (and already muscle-memoried into my format-patch/send-email workflow).
> If you send a scissors I can roll it into the series with your SOB.

But you can check it yourself! I worked _really_ hard on that Azure
Pipeline backing the PR builds at https://github.com/git/git. _REALLY_
hard. You might just as well reap the benefits so that I did not spend all
of that time and sweat and stress in vain...

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 06/15] bugreport: add compiler info
  2020-02-21 15:22                 ` Johannes Schindelin
@ 2020-02-22  0:04                   ` Emily Shaffer
  2020-02-24  2:55                     ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-22  0:04 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git

On Fri, Feb 21, 2020 at 04:22:43PM +0100, Johannes Schindelin wrote:
> Hi Emily,
> 
> On Thu, 20 Feb 2020, Emily Shaffer wrote:
> 
> > On Thu, Feb 20, 2020 at 11:33:05PM +0100, Johannes Schindelin wrote:
> > > Hi Emily,
> > >
> > > On Wed, 19 Feb 2020, Emily Shaffer wrote:
> > >
> > > >   #ifdef __GLIBC__
> > > >   #include <gnu/libc-version.h>
> > > >   #endif
> > > >
> > > >   static inline void get_compiler_info(struct strbuf *info)
> > > >   {
> > > >   	#ifdef __GLIBC__
> > > > 	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> > > > 	#endif
> > > >
> > > > 	#ifdef __GNUC__
> > > > 	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> > > > 	#endif
> > > >
> > > > 	#ifdef _MSC_VER
> > > > 	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());
> > >
> > > You could do it this way right away:
> > >
> > > 	strbuf_addf(info, "MSVC version: %d.%d\n",
> > > 		    _MSC_VER / 100, _MSC_VER % 100);
> > >
> > > Note: this is _not_ the MSVC _runtime_ version, but really the compiler
> > > version.
> > >
> > > You could also use _MSC_FULL_VER, which is a bit more complete.
> >
> > Sorry, but I'm not comfortable sending code I can't check for myself
> > (and already muscle-memoried into my format-patch/send-email workflow).
> > If you send a scissors I can roll it into the series with your SOB.
> 
> But you can check it yourself! I worked _really_ hard on that Azure
> Pipeline backing the PR builds at https://github.com/git/git. _REALLY_
> hard. You might just as well reap the benefits so that I did not spend all
> of that time and sweat and stress in vain...

I thought a bit about this. From your Github-using point of view, "just
check my Pipeline" sounds like "just look at one more thing". From my
format-patch using point of view, "just check my Pipeline" sounds like
"ugh, I have to add this remote again... I don't have a fork already?
How do I make that? Or is my fork just 6 months behind? How do I open a
PR again? Yeesh."

So I did what any good developer who's supposed to be writing
semi-annual performance reviews would, and wrote a first pass at script
with this brand new fancy 'gh' thing (which is, by the way, clearly not
intended for scripting): https://github.com/nasamuffin/quimby. Maybe
someone will find it useful.

I'll try and add your MSC snippet for the next rollup.

 - Emily

P.S. You can thank Jonathan Nieder for the name, as I asked for ideas
and he said "Wikipedia tells me Chief Quimby gives Inspector Gadget
something to read that later self-destructs." :P

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v7 06/15] bugreport: add compiler info
  2020-02-22  0:04                   ` Emily Shaffer
@ 2020-02-24  2:55                     ` Junio C Hamano
  0 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-02-24  2:55 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Johannes Schindelin, git

Emily Shaffer <emilyshaffer@google.com> writes:

>> > Sorry, but I'm not comfortable sending code I can't check for myself
>> > (and already muscle-memoried into my format-patch/send-email workflow).
>> > If you send a scissors I can roll it into the series with your SOB.
>> 
>> But you can check it yourself! I worked _really_ hard on that Azure
>> Pipeline backing the PR builds at https://github.com/git/git. _REALLY_
>> hard. You might just as well reap the benefits so that I did not spend all
>> of that time and sweat and stress in vain...
>
> I thought a bit about this. From your Github-using point of view, "just
> check my Pipeline" sounds like "just look at one more thing". From my
> format-patch using point of view, "just check my Pipeline" sounds like
> "ugh, I have to add this remote again... I don't have a fork already?
> How do I make that? Or is my fork just 6 months behind? How do I open a
> PR again? Yeesh."

Sorry, but this is how a typical conversation between two techies who
are more intelligent than good for their social skills ;-)  Each side
is not extending its hand enough to reach the other side, expecting
that what s/he knows should be obvious to the others.

Dscho may be frustrated for you not being aware of what he worked
on, but "I worked really hard" is much less useful thing for others,
who may benefit from what he worked hard on, to hear, than "here is
one pager that tells you how to use it".  Quite honestly, reading
the above exchange from the sidelines, it is not clear to me if it
is a part of, or a completely separate from, the service offered by
GGG from the end-users' point of view.

Can some of you come up with a one-pager to cover the following (and
related but not listed) areas, that can be added as an appendix to
Documentation/SubmittingPatches?  It does not have to cover the
common stuff like style guidelines, writing good log messages, need
for test coverage, choice of fork point, etc.  "Now you have a patch
or a series of patches, but you obviously do not have direct and
personal access to all the platforms Git supports.  But there are
servies to help you test them on the more common ones."

 - How to cause TravisCI to run our test suite with your
   modification.

 - How to trigger the same on Azure Pipeline PR builds.

There may be others.

Thanks.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 07/15] bugreport: add git-remote-https version
  2020-02-21  3:44               ` Junio C Hamano
@ 2020-02-25 22:08                 ` Emily Shaffer
  2020-02-25 22:26                   ` Junio C Hamano
  2020-02-25 23:29                   ` Junio C Hamano
  0 siblings, 2 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-25 22:08 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Feb 20, 2020 at 07:44:30PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > ... Unfortunately I agree that we wouldn't be equipped to handle
> > reports in other languages.
> 
> I actually was anticipating a far better world ;-) 
> 
> There is no reason to limit the recipient of reports only to *us*.
> Use of Git and population proficient in Git would become wide enough
> that we should be able to expect that users who speak language X can
> be helped by experts who speak the same language X with their
> issues.

Hm. I guess I got the opposite impression from you way back in v1. I do
wish it had been communicated a little more clearly; it's frustrating to
perceive a reversal after seven months of review. But that's probably on
my own reading comprehension :)

Well, I certainly don't mind - but I did have a pretty long conversation
with Jonathan Nieder last week about whether it's feasible to make the
bugreport extremely stable while also being locale aware. As I
understood it, he worried about someone having a misconfigured locale
causing the bugreport tool to crash when it tries to set the locale.

But I'm having trouble figuring out how that can happen. It looks like
we use libintl.h to do almost all of our string localization, which I
have to assume is pretty robust. Or to put it another way, if the user's
environment has broken libintl.h, then it seems like they also have
bigger problems outside of Git which they would notice already.

(Side note: I went down a rabbit hole of trying to break my locale, and
did manage to generate some garbage by setting 'LANG' to a non-UTF-8
character set and setting 'LANGUAGE' to a character set which does
require UTF-8, that is, 'LANG=es_MX LANGUAGE=ko git status'.)

How about if I localize the bugreport template, headers, and formatted
comments (e.g. "3745 local loose objects"), and include a tip in 'git
help bugreport' suggesting that if it doesn't look right, maybe the user
wants to run it with 'LANG= LANGUAGE= git bugreport' to ensure it
actually gets generated?


I had another thought, actually, that this is maybe semantically similar
problem to the malformed config we discussed earlier in the review. Does
it make sense to include some kind of --safemode flag to 'git' which
asks it to not perform localization and not read configs? I would
propose adding it just to git-bugreport, and I could work around some
locale weirdness that way, but with a broken config the attempt dies
before git-bugreport binary is invoked at all.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 07/15] bugreport: add git-remote-https version
  2020-02-25 22:08                 ` Emily Shaffer
@ 2020-02-25 22:26                   ` Junio C Hamano
  2020-02-25 23:29                     ` Emily Shaffer
  2020-02-25 23:29                   ` Junio C Hamano
  1 sibling, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-25 22:26 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> How about if I localize the bugreport template, headers, and formatted
> comments (e.g. "3745 local loose objects"), and include a tip in 'git
> help bugreport' suggesting that if it doesn't look right, maybe the user
> wants to run it with 'LANG= LANGUAGE= git bugreport' to ensure it
> actually gets generated?

I think that will be what is going to happen anyway in the real
world.  We'd spend a reasonable effort for localization (and I
personally am perfectly OK if the effort for the initial round is
"almost zero"), but make sure C-locale is left as an escape hatch.

> I had another thought, actually, that this is maybe semantically similar
> problem to the malformed config we discussed earlier in the review. Does
> it make sense to include some kind of --safemode flag to 'git' which
> asks it to not perform localization and not read configs?

I suspect that "git --safemode" would always end up being buggy than
running "git" under LANG=C LC_ALL=C GIT_CONFIG=/dev/null or something
like that ;-)

Wouldn't it defeat more than 30% of the value of the tool if we
do not read and report the contents of the configuration file(s)?

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 11/15] bugreport: collect list of populated hooks
  2020-02-20 20:58           ` Junio C Hamano
@ 2020-02-25 23:19             ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-25 23:19 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Feb 20, 2020 at 12:58:32PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +static void get_populated_hooks(struct strbuf *hook_info, int nongit)
> > +{
> > +	/*
> > +	 * Doesn't look like there is a list of all possible hooks; so below is
> > +	 * a transcription of `git help hooks`.
> > +	 */
> 
> It may want to become a NEEDSWORK comment.

Oh, good point. I'll rephrase it.
> 
> > +	const char *hooks = "applypatch-msg,"
> > +			    "pre-applypatch,"
> > +			    "post-applypatch,"
> > +			    "pre-commit,"
> > +			    "pre-merge-commit,"
> > +			    "prepare-commit-msg,"
> > +			    "commit-msg,"
> > +			    "post-commit,"
> > +			    "pre-rebase,"
> > +			    "post-checkout,"
> > +			    "post-merge,"
> > +			    "pre-push,"
> > +			    "pre-receive,"
> > +			    "update,"
> > +			    "post-receive,"
> > +			    "post-update,"
> > +			    "push-to-checkout,"
> > +			    "pre-auto-gc,"
> > +			    "post-rewrite,"
> > +			    "sendemail-validate,"
> > +			    "fsmonitor-watchman,"
> > +			    "p4-pre-submit,"
> > +			    "post-index-changex";
> 
> Typo here?

Yep, I imagine from trying to press 'x' to remove the ',' from my list.

> 
> > +	struct string_list hooks_list = STRING_LIST_INIT_DUP;
> > +	struct string_list_item *iter = NULL;
> > +
> > +
> > +	if (nongit) {
> > +		strbuf_addstr(hook_info,
> > +			"not run from a git repository - no hooks to show\n");
> > +		return;
> > +	}
> > +
> > +	string_list_split(&hooks_list, hooks, ',', -1);
> > +
> > +	for_each_string_list_item(iter, &hooks_list) {
> 
> I do not get why you want to use string_list for this, especially if
> you need to use string_list_split.
> 
> To me,
> 
> 	int i;
> 	const char *hook[] = {
> 		"applypatch-msg",
> 	        "pre-applypatch",
> 		...
> 		"post-index-change",
> 	};
> 
> 	for (i = 0; i < ARRAY_SIZE(hook); i++) {
> 		if (hook[i] is enabled)
> 			strbuf_addf(hook_info, "%s\n", hook[i]);
> 	}
> 
> would be far easier to understand.  Do you have an external source
> that can feed you a single long string of comma separated hook names
> in mind, so that the initialization of *hooks will become simpler
> that way, or something?

Huh. I'm having a hard time remembering why I did it this way. I think I
wanted to use the string_list foreach? But you're right that it is
pretty hard to understand, plus expensive; I'll just use an array like
you suggest.

> 
> > +		if (find_hook(iter->string)) {
> > +			strbuf_addstr(hook_info, iter->string);
> > +			strbuf_complete_line(hook_info);
> > +		}
> > +	}
> > +}
> > +
> >  static const char * const bugreport_usage[] = {
> >  	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
> >  	NULL
> > @@ -166,6 +216,9 @@ int cmd_main(int argc, const char **argv)
> >  	get_header(&buffer, "Safelisted Config Info");
> >  	get_safelisted_config(&buffer);
> >  
> > +	get_header(&buffer, "Enabled Hooks");
> > +	get_populated_hooks(&buffer, nongit_ok);
> > +
> >  	report = fopen_for_writing(report_path.buf);
> >  
> >  	if (report == NULL) {

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 12/15] bugreport: count loose objects
  2020-02-20 21:04           ` Junio C Hamano
@ 2020-02-25 23:22             ` Emily Shaffer
  2020-02-25 23:26               ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-02-25 23:22 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Johannes Schindelin

On Thu, Feb 20, 2020 at 01:04:33PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > The number of unpacked objects in a user's repository may help us
> > understand the root of the problem they're seeing, especially if a
> > command is running unusually slowly.
> >
> > Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  Documentation/git-bugreport.txt |  1 +
> >  bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
> >  2 files changed, 53 insertions(+)
> >
> > diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
> > index 4d01528540..4fe1c60506 100644
> > --- a/Documentation/git-bugreport.txt
> > +++ b/Documentation/git-bugreport.txt
> > @@ -32,6 +32,7 @@ The following information is captured automatically:
> >   - $SHELL
> >   - Selected config values
> >   - A list of enabled hooks
> > + - The number of loose objects in the repository
> >  
> >  This tool is invoked via the typical Git setup process, which means that in some
> >  cases, it might not be able to launch - for example, if a relevant config file
> > diff --git a/bugreport.c b/bugreport.c
> > index b5a0714a7f..fb7bc72723 100644
> > --- a/bugreport.c
> > +++ b/bugreport.c
> > @@ -10,6 +10,7 @@
> >  #include "bugreport-config-safelist.h"
> >  #include "khash.h"
> >  #include "run-command.h"
> > +#include "object-store.h"
> >  
> >  static void get_git_remote_https_version_info(struct strbuf *version_info)
> >  {
> > @@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
> >  	}
> >  }
> >  
> > +static int loose_object_cb(const struct object_id *oid, const char *path,
> > +			   void *data) {
> > +	int *loose_object_count = data;
> > +
> > +	if (loose_object_count) {
> > +		(*loose_object_count)++;
> > +		return 0;
> > +	}
> > +
> > +	return 1;
> 
> What is the point of returning 1 here to abort the iteration early?
> Wouldn't it be a BUG() if this callback ends up gettting called with
> NULL in data?

Hrm. Wouldn't BUG() throw away the rest of the generated report?

Maybe it's better, in NULL case, to indicate an error occurred there and
write that information into the report, and continue gathering the rest
of the info.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 12/15] bugreport: count loose objects
  2020-02-25 23:22             ` Emily Shaffer
@ 2020-02-25 23:26               ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-25 23:26 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Johannes Schindelin

On Tue, Feb 25, 2020 at 03:22:00PM -0800, Emily Shaffer wrote:
> On Thu, Feb 20, 2020 at 01:04:33PM -0800, Junio C Hamano wrote:
> > Emily Shaffer <emilyshaffer@google.com> writes:
> > 
> > > The number of unpacked objects in a user's repository may help us
> > > understand the root of the problem they're seeing, especially if a
> > > command is running unusually slowly.
> > >
> > > Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> > > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > > ---
> > >  Documentation/git-bugreport.txt |  1 +
> > >  bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
> > >  2 files changed, 53 insertions(+)
> > >
> > > diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
> > > index 4d01528540..4fe1c60506 100644
> > > --- a/Documentation/git-bugreport.txt
> > > +++ b/Documentation/git-bugreport.txt
> > > @@ -32,6 +32,7 @@ The following information is captured automatically:
> > >   - $SHELL
> > >   - Selected config values
> > >   - A list of enabled hooks
> > > + - The number of loose objects in the repository
> > >  
> > >  This tool is invoked via the typical Git setup process, which means that in some
> > >  cases, it might not be able to launch - for example, if a relevant config file
> > > diff --git a/bugreport.c b/bugreport.c
> > > index b5a0714a7f..fb7bc72723 100644
> > > --- a/bugreport.c
> > > +++ b/bugreport.c
> > > @@ -10,6 +10,7 @@
> > >  #include "bugreport-config-safelist.h"
> > >  #include "khash.h"
> > >  #include "run-command.h"
> > > +#include "object-store.h"
> > >  
> > >  static void get_git_remote_https_version_info(struct strbuf *version_info)
> > >  {
> > > @@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
> > >  	}
> > >  }
> > >  
> > > +static int loose_object_cb(const struct object_id *oid, const char *path,
> > > +			   void *data) {
> > > +	int *loose_object_count = data;
> > > +
> > > +	if (loose_object_count) {
> > > +		(*loose_object_count)++;
> > > +		return 0;
> > > +	}
> > > +
> > > +	return 1;
> > 
> > What is the point of returning 1 here to abort the iteration early?
> > Wouldn't it be a BUG() if this callback ends up gettting called with
> > NULL in data?
> 
> Hrm. Wouldn't BUG() throw away the rest of the generated report?
> 
> Maybe it's better, in NULL case, to indicate an error occurred there and
> write that information into the report, and continue gathering the rest
> of the info.

Oh, hm. Now that I look twice, you're saying "bugreport gave its own
callback a NULL". I guess it could also be that
for_each_loose_object() decided to give a NULL context instead of the
one we gave...

Even so, I dislike the idea of BUG() called from the tool you are trying
to use to report a BUG() message. We do already report if 1 was returned
"(problem during count)". I suppose I can try and make a more specific
error if we know the callback data was definitely the problem.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 07/15] bugreport: add git-remote-https version
  2020-02-25 22:08                 ` Emily Shaffer
  2020-02-25 22:26                   ` Junio C Hamano
@ 2020-02-25 23:29                   ` Junio C Hamano
  2020-02-25 23:55                     ` Emily Shaffer
  1 sibling, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-02-25 23:29 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> Hm. I guess I got the opposite impression from you way back in v1. I do
> wish it had been communicated a little more clearly; it's frustrating to
> perceive a reversal after seven months of review. But that's probably on
> my own reading comprehension :)

Well, seven months is a long time for anybody to learn from repeated
reading and gained experience to form an opinion and get affected by
others' opinions.  In any case, this is one of the reasons why I try
to discourage people to have too many topics in flight before moving
on to a new and different topic---the risk of ending up with fliping
and flopping is just too high when you give people chance to forget.
Rather, I'd prefer to see something simple to land first and then
later refined.

On this particular issue, I actually do not have a preference.  As
long as the topic has a coherent position/stance, any one of

 (a) we do as much i18n as possible to help end-users, or

 (b) we stay away from i18n to ensure the reports are machine
     readable, or

 (c) somewhere in between with a clear criterion where you are
     drawing the line (e.g. "the introductory text is what we want
     the end-user to read, so it is i18ned, but the report about
     their environment are primarily for our use and we avoid
     localizing so that we can process mechanically"),

is fine.  The important point is that we choose what we do with a
solid guiding principle behind the decision.

In practice, every string in bugreport.c you have control over the
use (or non-use) of _() around it, but codepaths that you call from
existing parts of the system are likely to have their messages
localized if they are meant for Porcelain use.  So from that point
of view, (a) would be easier to arrange than (b), I suspect.

Thanks.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 07/15] bugreport: add git-remote-https version
  2020-02-25 22:26                   ` Junio C Hamano
@ 2020-02-25 23:29                     ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-25 23:29 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Feb 25, 2020 at 02:26:34PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > How about if I localize the bugreport template, headers, and formatted
> > comments (e.g. "3745 local loose objects"), and include a tip in 'git
> > help bugreport' suggesting that if it doesn't look right, maybe the user
> > wants to run it with 'LANG= LANGUAGE= git bugreport' to ensure it
> > actually gets generated?
> 
> I think that will be what is going to happen anyway in the real
> world.  We'd spend a reasonable effort for localization (and I
> personally am perfectly OK if the effort for the initial round is
> "almost zero"), but make sure C-locale is left as an escape hatch.

OK.

> 
> > I had another thought, actually, that this is maybe semantically similar
> > problem to the malformed config we discussed earlier in the review. Does
> > it make sense to include some kind of --safemode flag to 'git' which
> > asks it to not perform localization and not read configs?
> 
> I suspect that "git --safemode" would always end up being buggy than
> running "git" under LANG=C LC_ALL=C GIT_CONFIG=/dev/null or something
> like that ;-)
> 
> Wouldn't it defeat more than 30% of the value of the tool if we
> do not read and report the contents of the configuration file(s)?

If you mean "the tool" = "Git", I won't argue with you. But if you mean
"the tool" = "git-bugreport", I'd say that 70% of the value of the tool
is better than 0% because my config is broken and I don't know why or
how.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 07/15] bugreport: add git-remote-https version
  2020-02-25 23:29                   ` Junio C Hamano
@ 2020-02-25 23:55                     ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-25 23:55 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Feb 25, 2020 at 03:29:08PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Hm. I guess I got the opposite impression from you way back in v1. I do
> > wish it had been communicated a little more clearly; it's frustrating to
> > perceive a reversal after seven months of review. But that's probably on
> > my own reading comprehension :)
> 
> Well, seven months is a long time for anybody to learn from repeated
> reading and gained experience to form an opinion and get affected by
> others' opinions.  In any case, this is one of the reasons why I try
> to discourage people to have too many topics in flight before moving
> on to a new and different topic---the risk of ending up with fliping
> and flopping is just too high when you give people chance to forget.
> Rather, I'd prefer to see something simple to land first and then
> later refined.

I think it's exactly what happened with this topic. It's easy to say
"don't have too many things out at once" in theory - but becomes less
easy when you combine it with things like quarterly planning at dayjob,
etc. In fact I have been trying to revive and push through old topics I
had in flight because I had this same realization - my six topics were
only each updated every month or so, meaning everyone (including me)
forgot what was going on from revision to revision.

I wonder whether it makes sense to try and finalize the bare minimums of
this feature, and then move the rest of the work to their own topics. I
see a couple benefits:

 - Users can start trying out 'git bugreport' when they write us mails
   much sooner. Even the template alone would increase the quality of a
   number of the reports I see come into this list.
 - It may help insulate this topic from 'feature creep,' which in my
   opinion has plagued it from the start. While the entire topic of 15
   patches is in flux, it's hard for someone else to write their own
   patch adding, say, midex/commit-graph support, and so I feel
   obligated to write it myself - which of course increases the amount
   of time the topic lives on in review.
 - Of course smaller topics are easier to review, and a commit-graph
   expert is much more likely to click on the subject "[PATCH]
   bugreport: gather commit-graph diagnostics" than they are to click on
   the subject "[PATCH] add bugreport utility". That is, I'm guessing
   there will be more reviews on the semantics of the patch (is it the
   right diagnostic info? is it useful?) than there have been so far.

The downside is mailing list churn, and the possibility that with a
minimum 'git-bugreport' checked in I find something better to do. I'm
not too worried about the former; after all, that's what this list is
for, right? As for the latter, after 7 months of flip and flop I don't
think the risk is greater than it is now anyways ;) I think it makes
sense to send the following topics:

/* To me, this sounds like minimum viable bugreport. "What went wrong,
 * and how do I build the same binary as you so I can reproduce it?"
add git-bugreport tool:
 - bugreport: add tool to generate debugging info
 - bugreport: gather git version and build info
 - bugreport: add uname info
 - bugreport: add compiler info

bugreport: add git-remote-https version

bugreport: include shell info
 - help: add shell-path to --build-options
 - bugreport: include user interactive shell

bugreport: include config info
 - help: move list_config_help to builtin/help
 - bugreport: generate config safelist based on docs
 - bugreport: add config values from safelist

bugreport: collect list of populated hooks

bugreport: include object store info
 - bugreport: count loose objects
 - bugreport: add packed object summary
 - bugreport: list contents of $OBJDIR/info

I'd like to send that first topic as vN+1 of this one, and I can hold
the rest of the branches locally until we get it ironed out; once that
topic makes it through to 'next' then I can send the rest again. What do
you think?

> On this particular issue, I actually do not have a preference.  As
> long as the topic has a coherent position/stance, any one of
> 
>  (a) we do as much i18n as possible to help end-users, or
> 
>  (b) we stay away from i18n to ensure the reports are machine
>      readable, or
> 
>  (c) somewhere in between with a clear criterion where you are
>      drawing the line (e.g. "the introductory text is what we want
>      the end-user to read, so it is i18ned, but the report about
>      their environment are primarily for our use and we avoid
>      localizing so that we can process mechanically"),
> 
> is fine.  The important point is that we choose what we do with a
> solid guiding principle behind the decision.
> 
> In practice, every string in bugreport.c you have control over the
> use (or non-use) of _() around it, but codepaths that you call from
> existing parts of the system are likely to have their messages
> localized if they are meant for Porcelain use.  So from that point
> of view, (a) would be easier to arrange than (b), I suspect.

Hm. Sounds convincing enough to me.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 13/15] bugreport: add packed object summary
  2020-02-20 22:04           ` Junio C Hamano
@ 2020-02-25 23:58             ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-02-25 23:58 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Feb 20, 2020 at 02:04:52PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
> > +{
> > +	struct packed_git *pack = NULL;
> > +	int pack_count = 0;
> > +	int object_count = 0;
> > +
> > +	if (nongit) {
> > +		strbuf_addstr(obj_info,
> > +			"not run from a git repository - no objects to show\n");
> > +		return;
> > +	}
> > +
> > +	for_each_pack(the_repository, pack) {
> > +		pack_count++;
> > +		/*
> > +		 * To accurately count how many objects are packed, look inside
> > +		 * the packfile's index.
> > +		 */
> > +		open_pack_index(pack);
> > +		object_count += pack->num_objects;
> > +	}
> > +
> > +	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
> > +		    object_count);
> > +
> > +}
> 
> Makes sense.
> 
> > @@ -447,4 +448,9 @@ int for_each_object_in_pack(struct packed_git *p,
> >  int for_each_packed_object(each_packed_object_fn, void *,
> >  			   enum for_each_object_flags flags);
> >  
> > +#define for_each_pack(repo, pack) 		\
> > +		for (pack = get_all_packs(repo);\
> > +		     pack;			\
> > +		     pack = pack->next)
> 
> I generally avoid #define'ing a control loop pseudo-syntax unless it
> makes the resulting code hide overly ugly implementation detail.
> 
> for_each_string_list() is there to hide the fact that items are
> stored in an embedded array whose name is .items and size is .nr
> that is sufficiently ugnly to expose, but iterating over packs
> does not look so bad.
> 
> If you MUST have this as a pseudo-syntax macro, we need
> 
>  - to match for_each_string_list_item(), have iterating variable
>    'pack' as the first parameter, and the scope of what's iterated
>    'repo' as the second.
> 
>  - to make sure the syntax works correctly even if a parameter is
>    *not* a simple identifier (I think the above is OK, but there may
>    be cases that it does not work well).
> 
> Regarding the latter, the way 'item' is incremented at the end of
> iteration in for_each_string_list_item() is subtle and correct.
> 
> #define for_each_string_list_item(item,list)            \
> 	for (item = (list)->items;                      \
> 	     item && item < (list)->items + (list)->nr; \
> 	     ++item)
> 
> You would break it if you changed pre-increment to post-increment
> for a user like this:
> 
> 	struct string_list *list;
> 	struct string_list_item *i, **p;
> 	p = &i;
> 
> 	for_each_string_list_item(*p, list) {
> 		...
> 	}
> 
> because ++*p is ++(*p), while *p++ is (*p)++, and we do want the
> former (i.e. increment the memory cell pointed at by pointer p).
> 
> Personally, I would prefer not to introduce this macro if I were
> working on this topic.

Ah, I thought this is the kind of thing you meant here[1]. But I agree
with what you point out about the shortcomings of this kind of macro.
The implementation details are not so ugly; I'll drop it.

 - Emily

[1] https://lore.kernel.org/git/xmqq8sli89eu.fsf@gitster-ct.c.googlers.com

> 
> >  #endif /* OBJECT_STORE_H */
> > diff --git a/packfile.c b/packfile.c
> > index 99dd1a7d09..95afcc1187 100644
> > --- a/packfile.c
> > +++ b/packfile.c
> > @@ -2095,8 +2095,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
> >  	int r = 0;
> >  	int pack_errors = 0;
> >  
> > -	prepare_packed_git(the_repository);
> > -	for (p = get_all_packs(the_repository); p; p = p->next) {
> > +	for_each_pack(the_repository, p) {
> >  		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
> >  			continue;
> >  		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 03/15] bugreport: add tool to generate debugging info
  2020-02-20  1:58         ` [PATCH v8 03/15] bugreport: add tool to generate debugging info Emily Shaffer
  2020-02-20 19:33           ` Junio C Hamano
@ 2020-02-26 16:12           ` Johannes Schindelin
  1 sibling, 0 replies; 273+ messages in thread
From: Johannes Schindelin @ 2020-02-26 16:12 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Wed, 19 Feb 2020, Emily Shaffer wrote:

> diff --git a/bugreport.c b/bugreport.c
> new file mode 100644
> index 0000000000..8d4a76fdac
> --- /dev/null
> +++ b/bugreport.c
> @@ -0,0 +1,94 @@
> +#include "builtin.h"
> +#include "parse-options.h"
> +#include "stdio.h"
> +#include "strbuf.h"
> +#include "time.h"
> +
> +static const char * const bugreport_usage[] = {
> +	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
> +	NULL
> +};
> +
> +static int get_bug_template(struct strbuf *template)
> +{
> +	const char template_text[] = N_(
> +"Thank you for filling out a Git bug report!\n"
> +"Please answer the following questions to help us understand your issue.\n"
> +"\n"
> +"What did you do before the bug happened? (Steps to reproduce your issue)\n"
> +"\n"
> +"What did you expect to happen? (Expected behavior)\n"
> +"\n"
> +"What happened instead? (Actual behavior)\n"
> +"\n"
> +"What's different between what you expected and what actually happened?\n"
> +"\n"
> +"Anything else you want to add:\n"
> +"\n"
> +"Please review the rest of the bug report below.\n"
> +"You can delete any lines you don't wish to share.\n");
> +
> +	strbuf_addstr(template, template_text);
> +	return 0;
> +}
> +
> +int cmd_main(int argc, const char **argv)
> +{
> +	struct strbuf buffer = STRBUF_INIT;
> +	struct strbuf report_path = STRBUF_INIT;
> +	FILE *report;
> +	time_t now = time(NULL);
> +	char *option_output = NULL;
> +	char *option_suffix = "%F-%H%M";

This is not a portable `strftime()` format. Let's squash this in?

-- snipsnap --
Subject: [PATCH] fixup??? bugreport: add tool to generate debugging info

The `%F` format is an optional extension to the ISO C standard, see
https://pubs.opengroup.org/onlinepubs/9699919799/functions/strftime.html

And sure enough, in Git for Windows, this leads to a test failure
because that format is not supported in the version of the MSVC runtime
that we're stuck with.

Let's just use the long-hand instead.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 bugreport.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bugreport.c b/bugreport.c
index a3528a2e2b8..3f78db182e6 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -338,7 +338,7 @@ int cmd_main(int argc, const char **argv)
 	FILE *report;
 	time_t now = time(NULL);
 	char *option_output = NULL;
-	char *option_suffix = "%F-%H%M";
+	char *option_suffix = "%Y-%m-%d-%H%M";
 	struct stat statbuf;
 	int nongit_ok = 0;

--
2.25.1.windows.1


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 09/15] bugreport: generate config safelist based on docs
  2020-02-20  1:58         ` [PATCH v8 09/15] bugreport: generate config safelist based on docs Emily Shaffer
  2020-02-20 20:40           ` Junio C Hamano
@ 2020-02-26 16:13           ` Johannes Schindelin
  2020-02-26 16:49             ` Junio C Hamano
  1 sibling, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-02-26 16:13 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Martin Ågren

Hi Emily,

On Wed, 19 Feb 2020, Emily Shaffer wrote:

> diff --git a/Makefile b/Makefile
> index 9e6705061d..6bdd3b9337 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -818,6 +818,7 @@ VCSSVN_LIB = vcs-svn/lib.a
>
>  GENERATED_H += config-list.h
>  GENERATED_H += command-list.h
> +GENERATED_H += bugreport-config-safelist.h
>
>  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
>  	$(FIND) . \

In order to pretend that we actually care about developers on Windows,
let's squash this in?

-- snipsnap --
Subject: [PATCH] fixup??? bugreport: generate config safelist based on docs

The Visual Studio build is a special beast: as we cannot assume the
presence of any Unix tools on Windows, we have to commit all of the
files generated via shell scripts.

These two generated header files are no exception.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.mak.uname | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/config.mak.uname b/config.mak.uname
index 6d58d22cd5a..f1f36e43e47 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -788,8 +788,10 @@ vcxproj:
 	git add -f git/LinkOrCopyBuiltins.targets git-remote-http/LinkOrCopyRemoteHttp.targets

 	# Add command-list.h
-	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 command-list.h
-	git add -f command-list.h
+	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 command-list.h \
+		config-list.h bugreport-config-safelist.h
+	git add -f command-list.h \
+	 	config-list.h bugreport-config-safelist.h

 	# Add scripts
 	rm -f perl/perl.mak
--
2.25.1.windows.1


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v8 09/15] bugreport: generate config safelist based on docs
  2020-02-26 16:13           ` Johannes Schindelin
@ 2020-02-26 16:49             ` Junio C Hamano
  0 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-02-26 16:49 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Emily Shaffer, git, Martin Ågren

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Subject: [PATCH] fixup??? bugreport: generate config safelist based on docs
>
> The Visual Studio build is a special beast: as we cannot assume the
> presence of any Unix tools on Windows, we have to commit all of the
> files generated via shell scripts.
>
> These two generated header files are no exception.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  config.mak.uname | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/config.mak.uname b/config.mak.uname
> index 6d58d22cd5a..f1f36e43e47 100644
> --- a/config.mak.uname
> +++ b/config.mak.uname
> @@ -788,8 +788,10 @@ vcxproj:
>  	git add -f git/LinkOrCopyBuiltins.targets git-remote-http/LinkOrCopyRemoteHttp.targets
>
>  	# Add command-list.h
> -	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 command-list.h
> -	git add -f command-list.h
> +	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 command-list.h \
> +		config-list.h bugreport-config-safelist.h
> +	git add -f command-list.h \
> +	 	config-list.h bugreport-config-safelist.h

Two important questions are 

 - Is $(GENERATED_H) visible to you at this point in this makefile
   snippet that is included from the main Makefile?

 - Currently the list of files listed on $(GENERATED_H) match what
   you are building and adding here.  Is there a reason to believe
   two would ever diverge?

If the answers to the above are yes and no, there is an obvious
futureproofing of the suggested patch, i.e.

-	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 command-list.h
-	git add -f command-list.h
+	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 $(GENERATED_H)
+	git add -f $(GENERATED_H)


^ permalink raw reply	[flat|nested] 273+ messages in thread

* [PATCH v9 0/5] add git-bugreport tool
  2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
                           ` (14 preceding siblings ...)
  2020-02-20  1:58         ` [PATCH v8 15/15] bugreport: summarize contents of alternates file Emily Shaffer
@ 2020-03-02 23:03         ` Emily Shaffer
  2020-03-02 23:03           ` [PATCH v9 1/5] help: move list_config_help to builtin/help Emily Shaffer
                             ` (4 more replies)
  15 siblings, 5 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-03-02 23:03 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Martin Ågren, Aaron Schrab, Danh Doan,
	Eric Sunshine, SZEDER Gábor, Andreas Schwab

I did run this set through the Azure Pipelines CI [2] and hope that
means it's platform-safe; if that's not the case, please let me know
your concerns.

For the relevant patches, I made the following changes since v8:

 - More portable strftime format
 - Use open() instead of fopen_for_writing() in order to specify O_CREAT
   | O_EXCL | O_WRONLY.
 - Fix a use-after-free.
 - Add i18n/l10n macro all over the place; per [3] let's translate the
   whole contents of the report, because it's easy to force to a
   different language and we may not be the only ones reading reports
 - Use a lot less strbuf_complete_line() - when we're making library
   calls, we have control over the format, so we do know whether it ends
   in \n or not; let's not waste the time checking it.
 - Minor nit on uname header phrasing
 - Reword the structure of compat/compiler.h to separate libc and
   compiler.
 - Add MSVC reporting in compat/compiler.h (thanks Dscho).

Thanks all.
 - Emily

[1]: https://lore.kernel.org/git/20200225235558.GG212281@google.com
[2]: https://github.com/gitgitgadget/git/pull/566
[3]: https://lore.kernel.org/git/xmqqimjukzzv.fsf@gitster-ct.c.googlers.com

Emily Shaffer (5):
  help: move list_config_help to builtin/help
    Note that this patch is still required for this series because the
    help library (for version/build info) is unusable without it.
  bugreport: add tool to generate debugging info
  bugreport: gather git version and build info
  bugreport: add uname info
  bugreport: add compiler info

 .gitignore                      |   2 +
 Documentation/git-bugreport.txt |  52 ++++++++++++
 Makefile                        |  18 +++-
 bugreport.c                     | 142 ++++++++++++++++++++++++++++++++
 builtin/help.c                  |  86 +++++++++++++++++++
 command-list.txt                |   1 +
 compat/compiler.h               |  38 +++++++++
 generate-cmdlist.sh             |  19 -----
 generate-configlist.sh          |  21 +++++
 help.c                          | 131 +++++++----------------------
 help.h                          |   2 +-
 strbuf.c                        |   4 +
 strbuf.h                        |   1 +
 t/t0091-bugreport.sh            |  61 ++++++++++++++
 14 files changed, 452 insertions(+), 126 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh

-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 273+ messages in thread

* [PATCH v9 1/5] help: move list_config_help to builtin/help
  2020-03-02 23:03         ` [PATCH v9 0/5] add git-bugreport tool Emily Shaffer
@ 2020-03-02 23:03           ` Emily Shaffer
  2020-03-02 23:03           ` [PATCH v9 2/5] bugreport: add tool to generate debugging info Emily Shaffer
                             ` (3 subsequent siblings)
  4 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-03-02 23:03 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
builtin/clean.o, and builtin/config.o. This meant that help.o was
unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help()
into builtin/help.c (where it makes sense to assume other builtin libraries
are present).

When command-list.h is included but a member is not used, we start to
hear a compiler warning. Since the config list is generated in a fairly
different way than the command list, and since commands and config
options are semantically different, move the config list into its own
header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 21 +++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 118 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh

diff --git a/.gitignore b/.gitignore
index aebe7c0908..ea97de83f3 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 6134104ae6..c552312d3f 100644
--- a/Makefile
+++ b/Makefile
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2132,7 +2133,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2152,6 +2153,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2785,7 +2792,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..8692fe5cf4
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	sed 's/^.*$/	"&",/'
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-02 23:03         ` [PATCH v9 0/5] add git-bugreport tool Emily Shaffer
  2020-03-02 23:03           ` [PATCH v9 1/5] help: move list_config_help to builtin/help Emily Shaffer
@ 2020-03-02 23:03           ` Emily Shaffer
  2020-03-03 14:18             ` Johannes Schindelin
  2020-03-04 21:35             ` Johannes Schindelin
  2020-03-02 23:03           ` [PATCH v9 3/5] bugreport: gather git version and build info Emily Shaffer
                             ` (2 subsequent siblings)
  4 siblings, 2 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-03-02 23:03 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction
steps, expected behavior, and actual behavior. Later, Git can learn how
to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic
information we would otherwise need to ask the user for, we can reduce
the number of question-and-answer round trips between the reporter and
the Git contributor.

Users may also wish to send a report like this to their local "Git
expert" if they have put their repository into a state they are confused
by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |   1 +
 Documentation/git-bugreport.txt |  46 ++++++++++++++
 Makefile                        |   5 ++
 bugreport.c                     | 105 ++++++++++++++++++++++++++++++++
 command-list.txt                |   1 +
 strbuf.c                        |   4 ++
 strbuf.h                        |   1 +
 t/t0091-bugreport.sh            |  61 +++++++++++++++++++
 8 files changed, 224 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index ea97de83f3..d89bf9e11e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..1f9fde5cde
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,46 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [(-o | --output-directory) <path>] [(-s | --suffix) <format>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+This tool is invoked via the typical Git setup process, which means that in some
+cases, it might not be able to launch - for example, if a relevant config file
+is unreadable. In this kind of scenario, it may be helpful to manually gather
+the kind of information listed above when manually asking for help.
+
+OPTIONS
+-------
+-o <path>::
+--output-directory <path>::
+	Place the resulting bug report file in `<path>` instead of the root of
+	the Git repository.
+
+-s <format>::
+--suffix <format>::
+	Specify an alternate suffix for the bugreport name, to create a file
+	named 'git-bugreport-<formatted suffix>'. This should take the form of a
+	link:strftime[3] format string; the current local time will be used.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index c552312d3f..9e6705061d 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2461,6 +2462,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..f473d606f2
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,105 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to share.\n");
+
+	strbuf_addstr(template, _(template_text));
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	int report = -1;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+	char *option_suffix = "%F-%H%M";
+	int nongit_ok = 0;
+	const char *prefix = NULL;
+	const char *user_relative_path = NULL;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output-directory", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
+			   N_("specify a strftime format suffix for the filename")),
+		OPT_END()
+	};
+
+	prefix = setup_git_directory_gently(&nongit_ok);
+
+	argc = parse_options(argc, argv, prefix, bugreport_options,
+			     bugreport_usage, 0);
+
+	/* Prepare the path to put the result */
+	strbuf_addstr(&report_path,
+		      prefix_filename(prefix,
+				      option_output ? option_output : ""));
+	strbuf_complete(&report_path, '/');
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
+	/* Prepare the report contents */
+	get_bug_template(&buffer);
+
+	/* fopen doesn't offer us an O_EXCL alternative, except with glibc. */
+	report = open(report_path.buf, O_CREAT | O_EXCL | O_WRONLY, 0666);
+
+	if (report < 0) {
+		UNLEAK(report_path);
+		die(_("couldn't create a new file at '%s'"), report_path.buf);
+	}
+
+	strbuf_write_fd(&buffer, report);
+	close(report);
+
+	/*
+	 * We want to print the path relative to the user, but we still need the
+	 * path relative to us to give to the editor.
+	 */
+	if (!(prefix && skip_prefix(report_path.buf, prefix, &user_relative_path)))
+		user_relative_path = report_path.buf;
+	fprintf(stderr, _("Created new report at '%s'.\n"),
+		user_relative_path);
+
+	UNLEAK(buffer);
+	UNLEAK(report_path);
+	return !!launch_editor(report_path.buf, NULL, NULL);
+}
diff --git a/command-list.txt b/command-list.txt
index 2087894655..185e5e3f05 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
diff --git a/strbuf.c b/strbuf.c
index f19da55b07..f1d66c7848 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -539,6 +539,10 @@ ssize_t strbuf_write(struct strbuf *sb, FILE *f)
 	return sb->len ? fwrite(sb->buf, 1, sb->len, f) : 0;
 }
 
+ssize_t strbuf_write_fd(struct strbuf *sb, int fd)
+{
+	return sb->len ? write(fd, sb->buf, sb->len) : 0;
+}
 
 #define STRBUF_MAXLINK (2*PATH_MAX)
 
diff --git a/strbuf.h b/strbuf.h
index aae7ac3a82..bbf6204de7 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -462,6 +462,7 @@ int strbuf_readlink(struct strbuf *sb, const char *path, size_t hint);
  * NUL bytes.
  */
 ssize_t strbuf_write(struct strbuf *sb, FILE *stream);
+ssize_t strbuf_write_fd(struct strbuf *sb, int fd);
 
 /**
  * Read a line from a FILE *, overwriting the existing contents of
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..65f664fdac
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,61 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
+			echo "$line"
+			read -r nextline
+			if test -z "$nextline"; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport -s check-headers &&
+	check_all_headers_populated <git-bugreport-check-headers.txt &&
+	test_when_finished rm git-bugreport-check-headers.txt
+'
+
+test_expect_success 'dies if file with same name as report already exists' '
+	>>git-bugreport-duplicate.txt &&
+	test_must_fail git bugreport --suffix duplicate &&
+	test_when_finished rm git-bugreport-duplicate.txt
+'
+
+test_expect_success '--output-directory puts the report in the provided dir' '
+	git bugreport -o foo/ &&
+	test_path_is_file foo/git-bugreport-* &&
+	test_when_finished rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	test_i18ngrep usage output &&
+	test_path_is_missing git-bugreport-*
+'
+
+test_expect_success 'runs outside of a git dir' '
+	nongit git bugreport &&
+	test_when_finished rm non-repo/git-bugreport-*
+'
+
+test_expect_success 'can create leading directories outside of a git dir' '
+	nongit git bugreport -o foo/bar/baz &&
+	test_when_finished rm -fr foo/bar/baz
+'
+
+
+test_done
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v9 3/5] bugreport: gather git version and build info
  2020-03-02 23:03         ` [PATCH v9 0/5] add git-bugreport tool Emily Shaffer
  2020-03-02 23:03           ` [PATCH v9 1/5] help: move list_config_help to builtin/help Emily Shaffer
  2020-03-02 23:03           ` [PATCH v9 2/5] bugreport: add tool to generate debugging info Emily Shaffer
@ 2020-03-02 23:03           ` Emily Shaffer
  2020-03-23 21:20             ` Junio C Hamano
  2020-03-02 23:03           ` [PATCH v9 4/5] bugreport: add uname info Emily Shaffer
  2020-03-02 23:04           ` [PATCH v9 5/5] bugreport: add compiler info Emily Shaffer
  4 siblings, 1 reply; 273+ messages in thread
From: Emily Shaffer @ 2020-03-02 23:03 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Knowing which version of Git a user has and how it was built allows us
to more precisely pin down the circumstances when a certain issue
occurs, so teach bugreport how to tell us the same output as 'git
version --build-options'.

It's not ideal to directly call 'git version --build-options' because
that output goes to stdout. Instead, wrap the version string in a helper
within help.[ch] library, and call that helper from within the bugreport
library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  4 +++
 bugreport.c                     | 19 +++++++++++++-
 help.c                          | 46 ++++++++++++++++++++-------------
 help.h                          |  1 +
 4 files changed, 51 insertions(+), 19 deletions(-)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 1f9fde5cde..f44ae8cbe7 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -23,6 +23,10 @@ The following information is requested from the user:
  - Expected behavior
  - Actual behavior
 
+The following information is captured automatically:
+
+ - 'git version --build-options'
+
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
 is unreadable. In this kind of scenario, it may be helpful to manually gather
diff --git a/bugreport.c b/bugreport.c
index f473d606f2..c20437dc6a 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,17 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, _("git version:\n"));
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
@@ -32,6 +41,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -79,6 +93,9 @@ int cmd_main(int argc, const char **argv)
 	/* Prepare the report contents */
 	get_bug_template(&buffer);
 
+	get_header(&buffer, _("System Info"));
+	get_system_info(&buffer);
+
 	/* fopen doesn't offer us an O_EXCL alternative, except with glibc. */
 	report = open(report_path.buf, O_CREAT | O_EXCL | O_WRONLY, 0666);
 
diff --git a/help.c b/help.c
index a21487db77..1de9c0d589 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,32 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,25 +661,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	get_version_info(&buf, build_options);
+	printf("%s", buf.buf);
+
+	strbuf_release(&buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..500521b908 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v9 4/5] bugreport: add uname info
  2020-03-02 23:03         ` [PATCH v9 0/5] add git-bugreport tool Emily Shaffer
                             ` (2 preceding siblings ...)
  2020-03-02 23:03           ` [PATCH v9 3/5] bugreport: gather git version and build info Emily Shaffer
@ 2020-03-02 23:03           ` Emily Shaffer
  2020-03-02 23:04           ` [PATCH v9 5/5] bugreport: add compiler info Emily Shaffer
  4 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-03-02 23:03 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

The contents of uname() can give us some insight into what sort of
system the user is running on, and help us replicate their setup if need
be. The domainname field is not guaranteed to be available, so don't
collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 16 +++++++++++++++-
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index f44ae8cbe7..17b0d14e8d 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -26,6 +26,7 @@ The following information is requested from the user:
 The following information is captured automatically:
 
  - 'git version --build-options'
+ - uname sysname, release, version, and machine strings
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
diff --git a/bugreport.c b/bugreport.c
index c20437dc6a..857b4f712c 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -7,10 +7,24 @@
 
 static void get_system_info(struct strbuf *sys_info)
 {
+	struct utsname uname_info;
+
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, _("git version:\n"));
 	get_version_info(sys_info, 1);
-	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, _("uname() failed with error '%s' (%d)\n"),
+			    strerror(errno),
+			    errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* [PATCH v9 5/5] bugreport: add compiler info
  2020-03-02 23:03         ` [PATCH v9 0/5] add git-bugreport tool Emily Shaffer
                             ` (3 preceding siblings ...)
  2020-03-02 23:03           ` [PATCH v9 4/5] bugreport: add uname info Emily Shaffer
@ 2020-03-02 23:04           ` Emily Shaffer
  2020-03-03 11:46             ` Danh Doan
                               ` (2 more replies)
  4 siblings, 3 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-03-02 23:04 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

To help pinpoint the source of a regression, it is useful to know some
info about the compiler which the user's Git client was built with. By
adding a generic get_compiler_info() in 'compat/' we can choose which
relevant information to share per compiler; to get started, let's
demonstrate the version of glibc if the user built with 'gcc'.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     |  6 ++++++
 compat/compiler.h               | 38 +++++++++++++++++++++++++++++++++
 3 files changed, 45 insertions(+)
 create mode 100644 compat/compiler.h

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 17b0d14e8d..643d1b2884 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -27,6 +27,7 @@ The following information is captured automatically:
 
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
+ - Compiler-specific info string
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
diff --git a/bugreport.c b/bugreport.c
index 857b4f712c..5073c901cd 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -25,6 +26,11 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, _("compiler info: "));
+	get_compiler_info(sys_info);
+	strbuf_addstr(sys_info, _("libc info: "));
+	get_libc_info(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/compat/compiler.h b/compat/compiler.h
new file mode 100644
index 0000000000..21f098e6a4
--- /dev/null
+++ b/compat/compiler.h
@@ -0,0 +1,38 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+#endif
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	int len = info->len;
+#ifdef __GNUC__
+	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
+#endif
+
+#ifdef _MSC_VER
+	strbuf_addf(info, "MSVC version: %s\n", _MSC_FULL_VER);
+#endif
+
+	if (len == info->len)
+		strbuf_addf(info, _("no compiler information available\n"));
+}
+
+static inline void get_libc_info(struct strbuf *info)
+{
+	int len = info->len;
+
+#ifdef __GLIBC__
+	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
+#endif
+
+	if (len == info->len)
+		strbuf_addf(info, _("no libc information available\n"));
+}
+
+#endif /* COMPILER_H */
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 5/5] bugreport: add compiler info
  2020-03-02 23:04           ` [PATCH v9 5/5] bugreport: add compiler info Emily Shaffer
@ 2020-03-03 11:46             ` Danh Doan
  2020-03-03 14:07             ` Junio C Hamano
  2020-03-04 21:39             ` Johannes Schindelin
  2 siblings, 0 replies; 273+ messages in thread
From: Danh Doan @ 2020-03-03 11:46 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

On 2020-03-02 15:04:00-0800, Emily Shaffer <emilyshaffer@google.com> wrote:
> To help pinpoint the source of a regression, it is useful to know some
> info about the compiler which the user's Git client was built with. By
> adding a generic get_compiler_info() in 'compat/' we can choose which
> relevant information to share per compiler; to get started, let's
> demonstrate the version of glibc if the user built with 'gcc'.
> 
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  Documentation/git-bugreport.txt |  1 +
>  bugreport.c                     |  6 ++++++
>  compat/compiler.h               | 38 +++++++++++++++++++++++++++++++++
>  3 files changed, 45 insertions(+)
>  create mode 100644 compat/compiler.h
> 
> diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
> index 17b0d14e8d..643d1b2884 100644
> --- a/Documentation/git-bugreport.txt
> +++ b/Documentation/git-bugreport.txt
> @@ -27,6 +27,7 @@ The following information is captured automatically:
>  
>   - 'git version --build-options'
>   - uname sysname, release, version, and machine strings
> + - Compiler-specific info string
>  
>  This tool is invoked via the typical Git setup process, which means that in some
>  cases, it might not be able to launch - for example, if a relevant config file
> diff --git a/bugreport.c b/bugreport.c
> index 857b4f712c..5073c901cd 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -4,6 +4,7 @@
>  #include "strbuf.h"
>  #include "time.h"
>  #include "help.h"
> +#include "compat/compiler.h"
>  
>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -25,6 +26,11 @@ static void get_system_info(struct strbuf *sys_info)
>  			    uname_info.release,
>  			    uname_info.version,
>  			    uname_info.machine);
> +
> +	strbuf_addstr(sys_info, _("compiler info: "));
> +	get_compiler_info(sys_info);
> +	strbuf_addstr(sys_info, _("libc info: "));
> +	get_libc_info(sys_info);
>  }
>  
>  static const char * const bugreport_usage[] = {
> diff --git a/compat/compiler.h b/compat/compiler.h
> new file mode 100644
> index 0000000000..21f098e6a4
> --- /dev/null
> +++ b/compat/compiler.h
> @@ -0,0 +1,38 @@
> +#ifndef COMPILER_H
> +#define COMPILER_H
> +
> +#include "git-compat-util.h"
> +#include "strbuf.h"
> +
> +#ifdef __GLIBC__
> +#include <gnu/libc-version.h>
> +#endif
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +	int len = info->len;
> +#ifdef __GNUC__
> +	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> +#endif

This fixup can be used for clang on Linux, I guess it could be applied
for macOS too (since default gcc on macOS is symlinked to clang?)

--------------8<----------------------
diff --git a/compat/compiler.h b/compat/compiler.h
index 21f098e6a4..055dbb5e1b 100644
--- a/compat/compiler.h
+++ b/compat/compiler.h
@@ -11,7 +11,9 @@
 static inline void get_compiler_info(struct strbuf *info)
 {
 	int len = info->len;
-#ifdef __GNUC__
+#ifdef __clang__
+	strbuf_addf(info, "clang: %s\n", __clang_version__);
+#elif defined(__GNUC__)
 	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
 #endif
 
-------------------->8---------------------

> +
> +#ifdef _MSC_VER
> +	strbuf_addf(info, "MSVC version: %s\n", _MSC_FULL_VER);
> +#endif
> +
> +	if (len == info->len)
> +		strbuf_addf(info, _("no compiler information available\n"));
> +}
> +
> +static inline void get_libc_info(struct strbuf *info)
> +{
> +	int len = info->len;
> +
> +#ifdef __GLIBC__
> +	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> +#endif
> +
> +	if (len == info->len)
> +		strbuf_addf(info, _("no libc information available\n"));
> +}
> +
> +#endif /* COMPILER_H */
> -- 
> 2.25.0.265.gbab2e86ba0-goog
> 

-- 
Danh

^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 5/5] bugreport: add compiler info
  2020-03-02 23:04           ` [PATCH v9 5/5] bugreport: add compiler info Emily Shaffer
  2020-03-03 11:46             ` Danh Doan
@ 2020-03-03 14:07             ` Junio C Hamano
  2020-03-04 21:39             ` Johannes Schindelin
  2 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-03-03 14:07 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +static inline void get_compiler_info(struct strbuf *info)
> +{
> + ...
> +	if (len == info->len)
> +		strbuf_addf(info, _("no compiler information available\n"));

s/addf/addstr/ to squelch https://travis-ci.org/git/git/jobs/657572439

> +}
> +
> +static inline void get_libc_info(struct strbuf *info)
> +{
> + ...
> +	if (len == info->len)
> +		strbuf_addf(info, _("no libc information available\n"));

Ditto.

> +}
> +
> +#endif /* COMPILER_H */

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-02 23:03           ` [PATCH v9 2/5] bugreport: add tool to generate debugging info Emily Shaffer
@ 2020-03-03 14:18             ` Johannes Schindelin
  2020-03-04 21:35             ` Johannes Schindelin
  1 sibling, 0 replies; 273+ messages in thread
From: Johannes Schindelin @ 2020-03-03 14:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Mon, 2 Mar 2020, Emily Shaffer wrote:

> +	char *option_suffix = "%F-%H%M";

Unfortunately, this still is not an `strftime` format we can use on
Windows. I still needed to fix that in the previously-mentioned manner to
fix the test run on Windows.

Thanks,
Dscho

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-02 23:03           ` [PATCH v9 2/5] bugreport: add tool to generate debugging info Emily Shaffer
  2020-03-03 14:18             ` Johannes Schindelin
@ 2020-03-04 21:35             ` Johannes Schindelin
  2020-03-05 23:34               ` Jeff Hostetler
  2020-03-19 21:39               ` Emily Shaffer
  1 sibling, 2 replies; 273+ messages in thread
From: Johannes Schindelin @ 2020-03-04 21:35 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi,

On Mon, 2 Mar 2020, Emily Shaffer wrote:

>  .gitignore                      |   1 +
>  Documentation/git-bugreport.txt |  46 ++++++++++++++
>  Makefile                        |   5 ++
>  bugreport.c                     | 105 ++++++++++++++++++++++++++++++++
>  command-list.txt                |   1 +
>  strbuf.c                        |   4 ++
>  strbuf.h                        |   1 +
>  t/t0091-bugreport.sh            |  61 +++++++++++++++++++
>  8 files changed, 224 insertions(+)
>  create mode 100644 Documentation/git-bugreport.txt
>  create mode 100644 bugreport.c
>  create mode 100755 t/t0091-bugreport.sh

Hmm. I am still _quite_ convinced that this would be much better as a
built-in. Remember, non-built-ins come with a footprint, and I do not
necessarily think that you will want to spend 3MB on a `git-bugreport`
executable when you could have it for a couple dozen kilobytes inside
`git` instead.

Ciao,
Dscho

>
> diff --git a/.gitignore b/.gitignore
> index ea97de83f3..d89bf9e11e 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -25,6 +25,7 @@
>  /git-bisect--helper
>  /git-blame
>  /git-branch
> +/git-bugreport
>  /git-bundle
>  /git-cat-file
>  /git-check-attr
> diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
> new file mode 100644
> index 0000000000..1f9fde5cde
> --- /dev/null
> +++ b/Documentation/git-bugreport.txt
> @@ -0,0 +1,46 @@
> +git-bugreport(1)
> +================
> +
> +NAME
> +----
> +git-bugreport - Collect information for user to file a bug report
> +
> +SYNOPSIS
> +--------
> +[verse]
> +'git bugreport' [(-o | --output-directory) <path>] [(-s | --suffix) <format>]
> +
> +DESCRIPTION
> +-----------
> +Captures information about the user's machine, Git client, and repository state,
> +as well as a form requesting information about the behavior the user observed,
> +into a single text file which the user can then share, for example to the Git
> +mailing list, in order to report an observed bug.
> +
> +The following information is requested from the user:
> +
> + - Reproduction steps
> + - Expected behavior
> + - Actual behavior
> +
> +This tool is invoked via the typical Git setup process, which means that in some
> +cases, it might not be able to launch - for example, if a relevant config file
> +is unreadable. In this kind of scenario, it may be helpful to manually gather
> +the kind of information listed above when manually asking for help.
> +
> +OPTIONS
> +-------
> +-o <path>::
> +--output-directory <path>::
> +	Place the resulting bug report file in `<path>` instead of the root of
> +	the Git repository.
> +
> +-s <format>::
> +--suffix <format>::
> +	Specify an alternate suffix for the bugreport name, to create a file
> +	named 'git-bugreport-<formatted suffix>'. This should take the form of a
> +	link:strftime[3] format string; the current local time will be used.
> +
> +GIT
> +---
> +Part of the linkgit:git[1] suite
> diff --git a/Makefile b/Makefile
> index c552312d3f..9e6705061d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
>  # ... and all the rest that could be moved out of bindir to gitexecdir
>  PROGRAMS += $(EXTRA_PROGRAMS)
>
> +PROGRAM_OBJS += bugreport.o
>  PROGRAM_OBJS += credential-store.o
>  PROGRAM_OBJS += daemon.o
>  PROGRAM_OBJS += fast-import.o
> @@ -2461,6 +2462,10 @@ endif
>  git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
>  	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
>
> +git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
> +	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
> +		$(LIBS)
> +
>  git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
>  	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
>  		$(IMAP_SEND_LDFLAGS) $(LIBS)
> diff --git a/bugreport.c b/bugreport.c
> new file mode 100644
> index 0000000000..f473d606f2
> --- /dev/null
> +++ b/bugreport.c
> @@ -0,0 +1,105 @@
> +#include "builtin.h"
> +#include "parse-options.h"
> +#include "stdio.h"
> +#include "strbuf.h"
> +#include "time.h"
> +
> +static const char * const bugreport_usage[] = {
> +	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
> +	NULL
> +};
> +
> +static int get_bug_template(struct strbuf *template)
> +{
> +	const char template_text[] = N_(
> +"Thank you for filling out a Git bug report!\n"
> +"Please answer the following questions to help us understand your issue.\n"
> +"\n"
> +"What did you do before the bug happened? (Steps to reproduce your issue)\n"
> +"\n"
> +"What did you expect to happen? (Expected behavior)\n"
> +"\n"
> +"What happened instead? (Actual behavior)\n"
> +"\n"
> +"What's different between what you expected and what actually happened?\n"
> +"\n"
> +"Anything else you want to add:\n"
> +"\n"
> +"Please review the rest of the bug report below.\n"
> +"You can delete any lines you don't wish to share.\n");
> +
> +	strbuf_addstr(template, _(template_text));
> +	return 0;
> +}
> +
> +int cmd_main(int argc, const char **argv)
> +{
> +	struct strbuf buffer = STRBUF_INIT;
> +	struct strbuf report_path = STRBUF_INIT;
> +	int report = -1;
> +	time_t now = time(NULL);
> +	char *option_output = NULL;
> +	char *option_suffix = "%F-%H%M";
> +	int nongit_ok = 0;
> +	const char *prefix = NULL;
> +	const char *user_relative_path = NULL;
> +
> +	const struct option bugreport_options[] = {
> +		OPT_STRING('o', "output-directory", &option_output, N_("path"),
> +			   N_("specify a destination for the bugreport file")),
> +		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
> +			   N_("specify a strftime format suffix for the filename")),
> +		OPT_END()
> +	};
> +
> +	prefix = setup_git_directory_gently(&nongit_ok);
> +
> +	argc = parse_options(argc, argv, prefix, bugreport_options,
> +			     bugreport_usage, 0);
> +
> +	/* Prepare the path to put the result */
> +	strbuf_addstr(&report_path,
> +		      prefix_filename(prefix,
> +				      option_output ? option_output : ""));
> +	strbuf_complete(&report_path, '/');
> +
> +	strbuf_addstr(&report_path, "git-bugreport-");
> +	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
> +	strbuf_addstr(&report_path, ".txt");
> +
> +	switch (safe_create_leading_directories(report_path.buf)) {
> +	case SCLD_OK:
> +	case SCLD_EXISTS:
> +		break;
> +	default:
> +		die(_("could not create leading directories for '%s'"),
> +		    report_path.buf);
> +	}
> +
> +	/* Prepare the report contents */
> +	get_bug_template(&buffer);
> +
> +	/* fopen doesn't offer us an O_EXCL alternative, except with glibc. */
> +	report = open(report_path.buf, O_CREAT | O_EXCL | O_WRONLY, 0666);
> +
> +	if (report < 0) {
> +		UNLEAK(report_path);
> +		die(_("couldn't create a new file at '%s'"), report_path.buf);
> +	}
> +
> +	strbuf_write_fd(&buffer, report);
> +	close(report);
> +
> +	/*
> +	 * We want to print the path relative to the user, but we still need the
> +	 * path relative to us to give to the editor.
> +	 */
> +	if (!(prefix && skip_prefix(report_path.buf, prefix, &user_relative_path)))
> +		user_relative_path = report_path.buf;
> +	fprintf(stderr, _("Created new report at '%s'.\n"),
> +		user_relative_path);
> +
> +	UNLEAK(buffer);
> +	UNLEAK(report_path);
> +	return !!launch_editor(report_path.buf, NULL, NULL);
> +}
> diff --git a/command-list.txt b/command-list.txt
> index 2087894655..185e5e3f05 100644
> --- a/command-list.txt
> +++ b/command-list.txt
> @@ -54,6 +54,7 @@ git-archive                             mainporcelain
>  git-bisect                              mainporcelain           info
>  git-blame                               ancillaryinterrogators          complete
>  git-branch                              mainporcelain           history
> +git-bugreport                           ancillaryinterrogators
>  git-bundle                              mainporcelain
>  git-cat-file                            plumbinginterrogators
>  git-check-attr                          purehelpers
> diff --git a/strbuf.c b/strbuf.c
> index f19da55b07..f1d66c7848 100644
> --- a/strbuf.c
> +++ b/strbuf.c
> @@ -539,6 +539,10 @@ ssize_t strbuf_write(struct strbuf *sb, FILE *f)
>  	return sb->len ? fwrite(sb->buf, 1, sb->len, f) : 0;
>  }
>
> +ssize_t strbuf_write_fd(struct strbuf *sb, int fd)
> +{
> +	return sb->len ? write(fd, sb->buf, sb->len) : 0;
> +}
>
>  #define STRBUF_MAXLINK (2*PATH_MAX)
>
> diff --git a/strbuf.h b/strbuf.h
> index aae7ac3a82..bbf6204de7 100644
> --- a/strbuf.h
> +++ b/strbuf.h
> @@ -462,6 +462,7 @@ int strbuf_readlink(struct strbuf *sb, const char *path, size_t hint);
>   * NUL bytes.
>   */
>  ssize_t strbuf_write(struct strbuf *sb, FILE *stream);
> +ssize_t strbuf_write_fd(struct strbuf *sb, int fd);
>
>  /**
>   * Read a line from a FILE *, overwriting the existing contents of
> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
> new file mode 100755
> index 0000000000..65f664fdac
> --- /dev/null
> +++ b/t/t0091-bugreport.sh
> @@ -0,0 +1,61 @@
> +#!/bin/sh
> +
> +test_description='git bugreport'
> +
> +. ./test-lib.sh
> +
> +# Headers "[System Info]" will be followed by a non-empty line if we put some
> +# information there; we can make sure all our headers were followed by some
> +# information to check if the command was successful.
> +HEADER_PATTERN="^\[.*\]$"
> +
> +check_all_headers_populated () {
> +	while read -r line
> +	do
> +		if test "$(grep "$HEADER_PATTERN" "$line")"
> +		then
> +			echo "$line"
> +			read -r nextline
> +			if test -z "$nextline"; then
> +				return 1;
> +			fi
> +		fi
> +	done
> +}
> +
> +test_expect_success 'creates a report with content in the right places' '
> +	git bugreport -s check-headers &&
> +	check_all_headers_populated <git-bugreport-check-headers.txt &&
> +	test_when_finished rm git-bugreport-check-headers.txt
> +'
> +
> +test_expect_success 'dies if file with same name as report already exists' '
> +	>>git-bugreport-duplicate.txt &&
> +	test_must_fail git bugreport --suffix duplicate &&
> +	test_when_finished rm git-bugreport-duplicate.txt
> +'
> +
> +test_expect_success '--output-directory puts the report in the provided dir' '
> +	git bugreport -o foo/ &&
> +	test_path_is_file foo/git-bugreport-* &&
> +	test_when_finished rm -fr foo/
> +'
> +
> +test_expect_success 'incorrect arguments abort with usage' '
> +	test_must_fail git bugreport --false 2>output &&
> +	test_i18ngrep usage output &&
> +	test_path_is_missing git-bugreport-*
> +'
> +
> +test_expect_success 'runs outside of a git dir' '
> +	nongit git bugreport &&
> +	test_when_finished rm non-repo/git-bugreport-*
> +'
> +
> +test_expect_success 'can create leading directories outside of a git dir' '
> +	nongit git bugreport -o foo/bar/baz &&
> +	test_when_finished rm -fr foo/bar/baz
> +'
> +
> +
> +test_done
> --
> 2.25.0.265.gbab2e86ba0-goog
>
>

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 5/5] bugreport: add compiler info
  2020-03-02 23:04           ` [PATCH v9 5/5] bugreport: add compiler info Emily Shaffer
  2020-03-03 11:46             ` Danh Doan
  2020-03-03 14:07             ` Junio C Hamano
@ 2020-03-04 21:39             ` Johannes Schindelin
  2020-03-23 21:27               ` Emily Shaffer
  2 siblings, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-03-04 21:39 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Mon, 2 Mar 2020, Emily Shaffer wrote:

> diff --git a/compat/compiler.h b/compat/compiler.h
> new file mode 100644
> index 0000000000..21f098e6a4
> --- /dev/null
> +++ b/compat/compiler.h
> @@ -0,0 +1,38 @@
> +#ifndef COMPILER_H
> +#define COMPILER_H
> +
> +#include "git-compat-util.h"
> +#include "strbuf.h"
> +
> +#ifdef __GLIBC__
> +#include <gnu/libc-version.h>
> +#endif
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +	int len = info->len;
> +#ifdef __GNUC__
> +	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> +#endif
> +
> +#ifdef _MSC_VER
> +	strbuf_addf(info, "MSVC version: %s\n", _MSC_FULL_VER);

I need this to fix a crash in the Visual Studio build's test run:

-- snip --
Subject: [PATCH] fixup??? bugreport: add compiler info

As documented at
https://docs.microsoft.com/en-us/cpp/preprocessor/predefined-macros?view=vs-2019,
`_MSC_FULL_VER` is an integer, not a string. Therefore this fix is
needed to prevent a segmentation fault in the test t0091.1.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 compat/compiler.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/compat/compiler.h b/compat/compiler.h
index ce6a7f6de9a..ac90fa051dd 100644
--- a/compat/compiler.h
+++ b/compat/compiler.h
@@ -16,7 +16,8 @@ static inline void get_compiler_info(struct strbuf *info)
 #endif

 #ifdef _MSC_VER
-	strbuf_addf(info, "MSVC version: %s\n", _MSC_FULL_VER);
+	strbuf_addf(info, "MSVC version: %02d.%02d.%05d\n",
+		    _MSC_VER / 100, _MSC_VER % 100, _MSC_FULL_VER % 100000);
 #endif

 	if (len == info->len)
-- snap --

Could you squash that in, please?

BTW the two issues I reported in the earlier round are still problems.
Could you squash my fixes, please? For your convenience, you can pluck all
four of them right off of
https://github.com/git-for-windows/git/commits/shears/pu

Just replace the `???` by `!` and you can even use `git rebase
--autosquash`.

Ciao,
Dscho


> +#endif
> +
> +	if (len == info->len)
> +		strbuf_addf(info, _("no compiler information available\n"));
> +}
> +
> +static inline void get_libc_info(struct strbuf *info)
> +{
> +	int len = info->len;
> +
> +#ifdef __GLIBC__
> +	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> +#endif
> +
> +	if (len == info->len)
> +		strbuf_addf(info, _("no libc information available\n"));
> +}
> +
> +#endif /* COMPILER_H */
> --
> 2.25.0.265.gbab2e86ba0-goog
>
>

^ permalink raw reply related	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-04 21:35             ` Johannes Schindelin
@ 2020-03-05 23:34               ` Jeff Hostetler
  2020-03-06 13:57                 ` Johannes Schindelin
  2020-03-06 18:08                 ` Junio C Hamano
  2020-03-19 21:39               ` Emily Shaffer
  1 sibling, 2 replies; 273+ messages in thread
From: Jeff Hostetler @ 2020-03-05 23:34 UTC (permalink / raw)
  To: Johannes Schindelin, Emily Shaffer; +Cc: git



On 3/4/2020 4:35 PM, Johannes Schindelin wrote:
> Hi,
> 
> On Mon, 2 Mar 2020, Emily Shaffer wrote:
> 
>>   .gitignore                      |   1 +
>>   Documentation/git-bugreport.txt |  46 ++++++++++++++
>>   Makefile                        |   5 ++
>>   bugreport.c                     | 105 ++++++++++++++++++++++++++++++++
>>   command-list.txt                |   1 +
>>   strbuf.c                        |   4 ++
>>   strbuf.h                        |   1 +
>>   t/t0091-bugreport.sh            |  61 +++++++++++++++++++
>>   8 files changed, 224 insertions(+)
>>   create mode 100644 Documentation/git-bugreport.txt
>>   create mode 100644 bugreport.c
>>   create mode 100755 t/t0091-bugreport.sh
> 
> Hmm. I am still _quite_ convinced that this would be much better as a
> built-in. Remember, non-built-ins come with a footprint, and I do not
> necessarily think that you will want to spend 3MB on a `git-bugreport`
> executable when you could have it for a couple dozen kilobytes inside
> `git` instead.
> 
> Ciao,
> Dscho

Having this command be a stand-alone exe rather than a builtin allows
it to have a different linkage.  For example, you could include the
libcurl and other libraries that are only linked into the transports.
And then report version numbers for them if you wanted.

Jeff

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-05 23:34               ` Jeff Hostetler
@ 2020-03-06 13:57                 ` Johannes Schindelin
  2020-03-06 18:25                   ` Junio C Hamano
  2020-03-06 18:08                 ` Junio C Hamano
  1 sibling, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-03-06 13:57 UTC (permalink / raw)
  To: Jeff Hostetler; +Cc: Emily Shaffer, git

Hi Jeff,

On Thu, 5 Mar 2020, Jeff Hostetler wrote:

> On 3/4/2020 4:35 PM, Johannes Schindelin wrote:
> >
> > On Mon, 2 Mar 2020, Emily Shaffer wrote:
> >
> > >   .gitignore                      |   1 +
> > >   Documentation/git-bugreport.txt |  46 ++++++++++++++
> > >   Makefile                        |   5 ++
> > >   bugreport.c                     | 105 ++++++++++++++++++++++++++++++++
> > >   command-list.txt                |   1 +
> > >   strbuf.c                        |   4 ++
> > >   strbuf.h                        |   1 +
> > >   t/t0091-bugreport.sh            |  61 +++++++++++++++++++
> > >   8 files changed, 224 insertions(+)
> > >   create mode 100644 Documentation/git-bugreport.txt
> > >   create mode 100644 bugreport.c
> > >   create mode 100755 t/t0091-bugreport.sh
> >
> > Hmm. I am still _quite_ convinced that this would be much better as a
> > built-in. Remember, non-built-ins come with a footprint, and I do not
> > necessarily think that you will want to spend 3MB on a `git-bugreport`
> > executable when you could have it for a couple dozen kilobytes inside
> > `git` instead.
> >
> > Ciao,
> > Dscho
>
> Having this command be a stand-alone exe rather than a builtin allows
> it to have a different linkage.  For example, you could include the
> libcurl and other libraries that are only linked into the transports.
> And then report version numbers for them if you wanted.

While that is true, I would argue that this is quite likely using the
_wrong_ information if the user has a different `git-remote-https` in
their `PATH`. In other words, that information could _mislead_ the
recipient of the bug report.

Now, other people might argue that the different linkage lets
`git-bugreport` maybe work where `git` would fail to load a required
dynamic library. But again, I would counter that this is a false
assumption: since `git-bugreport` relies on `libgit.a`, it essentially
_has_ to have the same linkage as `git`, or a superset thereof.

So: No, I really think this is going the wrong direction.

If we want `bugreport` to be a first-class citizen, we should treat it as
such. That entails making it a built-in. That entails teaching
`git-remote-https` (and potentially other non-builtins) to sprout that
option to enquire other information that should be included in the
generated part of the bug report.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-05 23:34               ` Jeff Hostetler
  2020-03-06 13:57                 ` Johannes Schindelin
@ 2020-03-06 18:08                 ` Junio C Hamano
  2020-03-06 18:58                   ` Jeff Hostetler
  2020-03-08 22:24                   ` Johannes Schindelin
  1 sibling, 2 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-03-06 18:08 UTC (permalink / raw)
  To: Jeff Hostetler; +Cc: Johannes Schindelin, Emily Shaffer, git

Jeff Hostetler <git@jeffhostetler.com> writes:

> Having this command be a stand-alone exe rather than a builtin allows
> it to have a different linkage.  For example, you could include the
> libcurl and other libraries that are only linked into the transports.
> And then report version numbers for them if you wanted.

I actually do not think that is a good rationale.  Unless your
version of "git bugreport" links into the same binary as the
"transports", it still is possible that the version of cURL (for
example) "git bugreport" can learn internally from may not have
anything to do with the version of the library used by the
transports.  

Of course, making "bugreport" a built-in, i.e. the same binary as
the non-transport part of Git, is not a solution for that issue,
either.  As Dscho suggested and recent rounds of "git bugreport"
implements, teaching the transport binaries an option to report
relevant pieces of information regarding the libraries they use, and
making "git bugreport" ask them, is a very good solution for that.

What makes it possible by making "git bugreport" stand-alone is for
it to link with libraries that the remainder of Git, including the
transports that link with libcurl, has no business linking with (a
library to obtain system details for diagnostic purposes, for
example).  Early versions of "git bugreport" may not link with
anything special, but making sure it starts and stays standalone
leaves it easier to do so _without_ having to worry about splitting
the code that started as a built-in out to make it independent later.



^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-06 13:57                 ` Johannes Schindelin
@ 2020-03-06 18:25                   ` Junio C Hamano
  0 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-03-06 18:25 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Jeff Hostetler, Emily Shaffer, git

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> If we want `bugreport` to be a first-class citizen, we should
> treat it as such.  That entails making it a built-in.

The question is, for "bugreport", if it is better to be standalone
or to be built-in, and I do not think 'first-class' has anything to
do with the question.  The transport based on curl-library are
separate binaries but they are of course first-class citizens.

Pushing and fetching will happen a lot less often than local
operations like "add", "commit", "show", and not paying start-up
latencies to load code and link libraries that are not necessary for
local operations in built-in, while paying the cost of having to
duplicate the memory and disk footprint necessary for libgit.a stuff
by making the transport binaries separate from the built-ins, was
conscious implementation decision we made earlier to prioritize the
reduction of overhead of subcommands for local operation.  Yes, the
decision de-prioritizes the transport, but that does not mean they
are not first-class citizens.  It just means that the start-up
latency for them (e.g. linking with libcurl) is not huge burden for
the overall time cost for the network transfer, and it is necessary
cost for them to perform what they need to do anyway, while it is
something subcommands for local operations should not have to pay.

> That entails teaching
> `git-remote-https` (and potentially other non-builtins) to sprout that
> option to enquire other information that should be included in the
> generated part of the bug report.

I think later rounds of "git bugreports" already did so.

As I mentioned in a separate message to JeffH, this is a good thing
to do, whether "git bugreport" is a built-in or a standalone.




^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-06 18:08                 ` Junio C Hamano
@ 2020-03-06 18:58                   ` Jeff Hostetler
  2020-03-08 22:24                   ` Johannes Schindelin
  1 sibling, 0 replies; 273+ messages in thread
From: Jeff Hostetler @ 2020-03-06 18:58 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Johannes Schindelin, Emily Shaffer, git



On 3/6/2020 1:08 PM, Junio C Hamano wrote:
> Jeff Hostetler <git@jeffhostetler.com> writes:
> 
>> Having this command be a stand-alone exe rather than a builtin allows
>> it to have a different linkage.  For example, you could include the
>> libcurl and other libraries that are only linked into the transports.
>> And then report version numbers for them if you wanted.
> 
> I actually do not think that is a good rationale.  Unless your
> version of "git bugreport" links into the same binary as the
> "transports", it still is possible that the version of cURL (for
> example) "git bugreport" can learn internally from may not have
> anything to do with the version of the library used by the
> transports.
> 
> Of course, making "bugreport" a built-in, i.e. the same binary as
> the non-transport part of Git, is not a solution for that issue,
> either.  As Dscho suggested and recent rounds of "git bugreport"
> implements, teaching the transport binaries an option to report
> relevant pieces of information regarding the libraries they use, and
> making "git bugreport" ask them, is a very good solution for that.
> 
> What makes it possible by making "git bugreport" stand-alone is for
> it to link with libraries that the remainder of Git, including the
> transports that link with libcurl, has no business linking with (a
> library to obtain system details for diagnostic purposes, for
> example).  Early versions of "git bugreport" may not link with
> anything special, but making sure it starts and stays standalone
> leaves it easier to do so _without_ having to worry about splitting
> the code that started as a built-in out to make it independent later.
> 
> 

Yeah, that makes sense.
Thanks
Jeff

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-06 18:08                 ` Junio C Hamano
  2020-03-06 18:58                   ` Jeff Hostetler
@ 2020-03-08 22:24                   ` Johannes Schindelin
  2020-03-09 14:59                     ` Junio C Hamano
  1 sibling, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-03-08 22:24 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Jeff Hostetler, Emily Shaffer, git

Hi Junio,

On Fri, 6 Mar 2020, Junio C Hamano wrote:

> What makes it possible by making "git bugreport" stand-alone is for
> it to link with libraries that the remainder of Git, including the
> transports that link with libcurl, has no business linking with (a
> library to obtain system details for diagnostic purposes, for
> example).

That would, however, make `git-bugreport` more fragile than `git`, i.e.
the former might fail to launch under more circumstances than the latter.

Not a good idea for a tool meant to help reporting bugs, to be more
fragile to even _start_ than `git`.

A tool that is intended to help users come up with bug reports, despite
the fact that users are totally unfamiliar with the implementation details
of Git and hence unable to judge competently what information to include
or to omit, needs to be _robust_.

You do not want to end up, for example, calling a stale `git-bugreport`
from an earlier `make install` (or from a different installation
altogether). That would give the recipient not only bogus information, it
would be outright misleading.

And that's what I meant by "a stand-alone `git-bugreport` would not be a
1st-class citizen".

I would see it as a serious design issue if this command is anything but a
built-in.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-08 22:24                   ` Johannes Schindelin
@ 2020-03-09 14:59                     ` Junio C Hamano
  2020-03-09 19:17                       ` Johannes Schindelin
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-03-09 14:59 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Jeff Hostetler, Emily Shaffer, git

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Junio,
>
> On Fri, 6 Mar 2020, Junio C Hamano wrote:
>
>> What makes it possible by making "git bugreport" stand-alone is for
>> it to link with libraries that the remainder of Git, including the
>> transports that link with libcurl, has no business linking with (a
>> library to obtain system details for diagnostic purposes, for
>> example).
>
> That would, however, make `git-bugreport` more fragile than `git`, i.e.
> the former might fail to launch under more circumstances than the latter.

That's a bug.  You can go fix it when it happens.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-09 14:59                     ` Junio C Hamano
@ 2020-03-09 19:17                       ` Johannes Schindelin
  2020-03-09 19:47                         ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-03-09 19:17 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Jeff Hostetler, Emily Shaffer, git

Hi Junio,

On Mon, 9 Mar 2020, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > On Fri, 6 Mar 2020, Junio C Hamano wrote:
> >
> >> What makes it possible by making "git bugreport" stand-alone is for
> >> it to link with libraries that the remainder of Git, including the
> >> transports that link with libcurl, has no business linking with (a
> >> library to obtain system details for diagnostic purposes, for
> >> example).
> >
> > That would, however, make `git-bugreport` more fragile than `git`, i.e.
> > the former might fail to launch under more circumstances than the latter.
>
> That's a bug.  You can go fix it when it happens.

Heh... yeah, that would be a bug, and the user would not be able to report
it via `git bugreport`...

Which is the whole point of my complaint.

Isn't it obvious that we should not have an independent `git-bugreport` by
now? With a stand-alone `git-bugreport`, we might

- fail to load the .so files under more circumstances than `git` would
  (since we link to `libgit.a`, we cannot have a subset of dependencies,
  only a superset, or the same),

- launch a stale `git-bugreport` from a completely different Git,

- make users angry for wasting 3MB of their diskspace for `git-bugreport`
  when only a dozen kilobyte would suffice.

On the other hand, if we make `git-bugreport` a built-in, I cannot see any
downsides.

For me, therefore, having it as a built-in is a clear win. What am I
missing?

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-09 19:17                       ` Johannes Schindelin
@ 2020-03-09 19:47                         ` Junio C Hamano
  2020-03-10 11:42                           ` Johannes Schindelin
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-03-09 19:47 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Jeff Hostetler, Emily Shaffer, git

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On the other hand, if we make `git-bugreport` a built-in, I cannot see any
> downsides.
>
> For me, therefore, having it as a built-in is a clear win. What am I
> missing?

The right sense of relative importance between efficiently running
the rest of Git by not bloting the main binary and making sure not
to ship Git that does not run unless "git bugreport" runs (which
makes sure that "bugreport" runs) is what you are missing, I
suspect.

Another thing is that you are giving "git bugreport" too much weight
and too little credit to inexperienced users, by assuming that we
will never hear from them when "bugreport" is incapable to run for
them.  They will report, with or without "git bugreport", and the
more important thing you seem to be missing is that after the
initial contact it would become an interactive process---there is
no reason to prioritize to ensure that the initial contact has
everything that is needed to diagnose any issue without further 
interaction.  "With my build, even 'bugreport' dumps core." is
perfectly good place to start.

Besides, wouldn't the ones on platforms, on which "git bugreport"
may have trouble running, i.e. the ones on minority and exotic
platforms, tend to be self sufficient and capable (both diagnosing
the issue for themselves, and reaching out to us as appropriate)
ones in practice (e.g. I have NonStop folks in mind)?

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-09 19:47                         ` Junio C Hamano
@ 2020-03-10 11:42                           ` Johannes Schindelin
  2020-03-10 18:37                             ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-03-10 11:42 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Jeff Hostetler, Emily Shaffer, git

Hi Junio,

On Mon, 9 Mar 2020, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > On the other hand, if we make `git-bugreport` a built-in, I cannot see any
> > downsides.
> >
> > For me, therefore, having it as a built-in is a clear win. What am I
> > missing?
>
> The right sense of relative importance between efficiently running
> the rest of Git by not bloting the main binary and making sure not
> to ship Git that does not run unless "git bugreport" runs (which
> makes sure that "bugreport" runs) is what you are missing, I
> suspect.

By that reasoning, `git bugreport` should not be included in core Git.

> Another thing is that you are giving "git bugreport" too much weight
> and too little credit to inexperienced users, by assuming that we
> will never hear from them when "bugreport" is incapable to run for
> them.  They will report, with or without "git bugreport", and the
> more important thing you seem to be missing is that after the
> initial contact it would become an interactive process---there is
> no reason to prioritize to ensure that the initial contact has
> everything that is needed to diagnose any issue without further
> interaction.  "With my build, even 'bugreport' dumps core." is
> perfectly good place to start.
>
> Besides, wouldn't the ones on platforms, on which "git bugreport"
> may have trouble running, i.e. the ones on minority and exotic
> platforms, tend to be self sufficient and capable (both diagnosing
> the issue for themselves, and reaching out to us as appropriate)
> ones in practice (e.g. I have NonStop folks in mind)?

Yes, I can agree that inexperienced users will not give up and keep
up the conversation until they see their problems fixed.

I can also agree that inexperienced users will report bugs even if it is
not made super easy for them to do so.

I can agree to that _iff_ I ignore my entire experience maintaining Git
for Windows, that is.

Sarcasm aside, I think that you underestimate the importance of a good bug
reporting tool like `git bugreport`, and I suspect that Emily does not.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-10 11:42                           ` Johannes Schindelin
@ 2020-03-10 18:37                             ` Junio C Hamano
  2020-03-10 22:08                               ` Johannes Schindelin
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-03-10 18:37 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Jeff Hostetler, Emily Shaffer, git

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> The right sense of relative importance between efficiently running
>> the rest of Git by not bloting the main binary and making sure not
>> to ship Git that does not run unless "git bugreport" runs (which
>> makes sure that "bugreport" runs) is what you are missing, I
>> suspect.
>
> By that reasoning, `git bugreport` should not be included in core Git.

The world does not have to be black-or-white.  "git bugreport" that
covers mainstream platforms in widely-used configurations may be
only an 80% solution, but that is better than nothing at all.

>> Another thing is that you are giving "git bugreport" too much weight
>> and too little credit to inexperienced users, by assuming that we
>> will never hear from them when "bugreport" is incapable to run for
>> them.  They will report, with or without "git bugreport", and the
>> more important thing you seem to be missing is that after the
>> initial contact it would become an interactive process---there is
>> no reason to prioritize to ensure that the initial contact has
>> everything that is needed to diagnose any issue without further
>> interaction.  "With my build, even 'bugreport' dumps core." is
>> perfectly good place to start.
>>
>> Besides, wouldn't the ones on platforms, on which "git bugreport"
>> may have trouble running, i.e. the ones on minority and exotic
>> platforms, tend to be self sufficient and capable (both diagnosing
>> the issue for themselves, and reaching out to us as appropriate)
>> ones in practice (e.g. I have NonStop folks in mind)?
>
> Yes, I can agree that inexperienced users will not give up and keep
> up the conversation until they see their problems fixed.

Inexperienced users won't be on minority platforms for which we do
not have enough resource to get "git bugreport" to run adequetly in
the first place.  Even if those on minority platforms whose userbase
are all technically incapable ones, if they do not help us help
them, what can "git bugreport" can do, and more importantly, how
would it make a difference between "bugreport" being a built-in vs a
standalone?

At this point, I'd have to say that your quibbling does not deserve
a serious response and it would be better use of my time to
disengage from this thread.


^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-10 18:37                             ` Junio C Hamano
@ 2020-03-10 22:08                               ` Johannes Schindelin
  0 siblings, 0 replies; 273+ messages in thread
From: Johannes Schindelin @ 2020-03-10 22:08 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Jeff Hostetler, Emily Shaffer, git

Hi Junio,

On Tue, 10 Mar 2020, Junio C Hamano wrote:

> At this point, I'd have to say that your quibbling does not deserve
> a serious response and it would be better use of my time to
> disengage from this thread.

I thought we were discussing whether it makes sense to leave
`git-bugreport` as a stand-alone executable, or rather make it a built-in.

If my attempts to convince you that it would be better as a built-in
strike you as "quibbling", then I should probably pull out from trying to
review and help with this here patch series. I really thought I was
helping. My apologies for annoying you instead.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-04 21:35             ` Johannes Schindelin
  2020-03-05 23:34               ` Jeff Hostetler
@ 2020-03-19 21:39               ` Emily Shaffer
  2020-03-20  0:28                 ` Junio C Hamano
                                   ` (2 more replies)
  1 sibling, 3 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-03-19 21:39 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git

On Wed, Mar 04, 2020 at 10:35:04PM +0100, Johannes Schindelin wrote:
> Hi,
> 
> On Mon, 2 Mar 2020, Emily Shaffer wrote:
> 
> >  .gitignore                      |   1 +
> >  Documentation/git-bugreport.txt |  46 ++++++++++++++
> >  Makefile                        |   5 ++
> >  bugreport.c                     | 105 ++++++++++++++++++++++++++++++++
> >  command-list.txt                |   1 +
> >  strbuf.c                        |   4 ++
> >  strbuf.h                        |   1 +
> >  t/t0091-bugreport.sh            |  61 +++++++++++++++++++
> >  8 files changed, 224 insertions(+)
> >  create mode 100644 Documentation/git-bugreport.txt
> >  create mode 100644 bugreport.c
> >  create mode 100755 t/t0091-bugreport.sh
> 
> Hmm. I am still _quite_ convinced that this would be much better as a
> built-in. Remember, non-built-ins come with a footprint, and I do not
> necessarily think that you will want to spend 3MB on a `git-bugreport`
> executable when you could have it for a couple dozen kilobytes inside
> `git` instead.

This is the kind of stuff I really wanted to get straightened out by
sending the smaller changeset, so I'm glad to be having this
conversation (again, and hopefully for the last time). I read the
replies to this mail, which I'm truncating as I think many of them are
distracting rather than discussion-worthy, and have tried to summarize:

Builtin:
+ Don't have to call out-of-process to identify 'git version --build-options'
+ Better assurance that we aren't shipping a broken bugreport alongside a new
  version
- Binary bloat, possible startup time hit
? Libraries will behave identically to where the user is seeing issues
  (This point is a possible pro but also a possible con; see similar point in
  standalone list)

Standalone:
+ Could investigate libraries who aren't behaving the way they should.
  (This seems like it'd be better addressed by regression tests, and if we're so
  sure that git-bugreport is working with the info at hand correctly, why don't
  we just write the git binary that way too?)
+ Avoid binary bloat in the main executable.
- Have to ship a big standalone binary instead.
- To get accurate version/build info, need to query the Git executable in a new
  process

Of course if I missed capturing something, please add/correct. Some of
these concerns are quantifiable - binary size and overhead, for example
- so I'm planning on doing some experiments in the coming days. I plan
put together the handful of patches in the latest version of the topic
in both standalone and builtin mode, and then gather the following info:

 - Binary size of 'git'
 - Binary size of 'git-bugreport' when applicable
 - Time for "make" following clean
 - Time for 'git status' on an identical real-world repo (I'll use ours,
   without touching the repo state e.g. changing branch or fetching in
   between)
 - Time to editor for 'git bugreport' with the same setup as previous
 - Peak memory footprint during 'git status'

And of course, if there's more to compare that I can quantify, please
let me know that too. I expect I'll be ready to run the experiments by
Monday (taking some time off tomorrow) so there's time for me to hear
about other concerns.

I'll hold off on sharing my own preference until after we've got some
benchmarking to look at, so I can understand the whole picture.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-19 21:39               ` Emily Shaffer
@ 2020-03-20  0:28                 ` Junio C Hamano
  2020-03-20 15:35                   ` Johannes Schindelin
  2020-03-20 15:42                 ` Johannes Schindelin
  2020-03-20 17:43                 ` Junio C Hamano
  2 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-03-20  0:28 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Johannes Schindelin, git

Emily Shaffer <emilyshaffer@google.com> writes:

> Builtin:
> + Don't have to call out-of-process to identify 'git version --build-options'
> + Better assurance that we aren't shipping a broken bugreport alongside a new
>   version
> - Binary bloat, possible startup time hit
> ? Libraries will behave identically to where the user is seeing issues
>   (This point is a possible pro but also a possible con; see similar point in
>   standalone list)

 - Makes it hard to pull in libraries that "bugreport" alone will
   want to use in the future without negatively impacting the rest
   of Git.

>  - Time to editor for 'git bugreport' with the same setup as previous

This is something we absolutely should totally ignore.  Even if the
bulitin version takes 1 sec to spawn an editor while a standalone
one took only 0.1 sec, if other criteria you listed that are more
important favours the builtin solution, we should pick it.

In any case, I think we've wasted enough time on this.  Let's see a
minimum working version that won't break/affect the rest of Git and
ship it standalone.

Thanks.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-20  0:28                 ` Junio C Hamano
@ 2020-03-20 15:35                   ` Johannes Schindelin
  2020-03-23 18:52                     ` Emily Shaffer
  0 siblings, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-03-20 15:35 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Emily Shaffer, git

On Thu, 19 Mar 2020, Junio C Hamano wrote:

> Emily Shaffer <emilyshaffer@google.com> writes:
>
> > Builtin:
> > + Don't have to call out-of-process to identify 'git version --build-options'
> > + Better assurance that we aren't shipping a broken bugreport alongside a new
> >   version
> > - Binary bloat, possible startup time hit
> > ? Libraries will behave identically to where the user is seeing issues
> >   (This point is a possible pro but also a possible con; see similar point in
> >   standalone list)
>
>  - Makes it hard to pull in libraries that "bugreport" alone will
>    want to use in the future without negatively impacting the rest
>    of Git.

And of course we should never do that, lest `git bugreport` won't even
load because of it while `git` would.

So even in the purely speculative scenario that we _wanted_ to add more
libraries (which we don't, why should we?), we wouldn't actually do it.

> >  - Time to editor for 'git bugreport' with the same setup as previous
>
> This is something we absolutely should totally ignore.  Even if the
> bulitin version takes 1 sec to spawn an editor while a standalone
> one took only 0.1 sec, if other criteria you listed that are more
> important favours the builtin solution, we should pick it.
>
> In any case, I think we've wasted enough time on this.  Let's see a
> minimum working version that won't break/affect the rest of Git and
> ship it standalone.

I still disagree with that, but it seems that no amount of arguments will
convince Junio, and he's dead set on the standalone version.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-19 21:39               ` Emily Shaffer
  2020-03-20  0:28                 ` Junio C Hamano
@ 2020-03-20 15:42                 ` Johannes Schindelin
  2020-03-23 18:50                   ` Emily Shaffer
  2020-03-20 17:43                 ` Junio C Hamano
  2 siblings, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-03-20 15:42 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 19 Mar 2020, Emily Shaffer wrote:

> On Wed, Mar 04, 2020 at 10:35:04PM +0100, Johannes Schindelin wrote:
>
> > On Mon, 2 Mar 2020, Emily Shaffer wrote:
> >
> > >  .gitignore                      |   1 +
> > >  Documentation/git-bugreport.txt |  46 ++++++++++++++
> > >  Makefile                        |   5 ++
> > >  bugreport.c                     | 105 ++++++++++++++++++++++++++++++++
> > >  command-list.txt                |   1 +
> > >  strbuf.c                        |   4 ++
> > >  strbuf.h                        |   1 +
> > >  t/t0091-bugreport.sh            |  61 +++++++++++++++++++
> > >  8 files changed, 224 insertions(+)
> > >  create mode 100644 Documentation/git-bugreport.txt
> > >  create mode 100644 bugreport.c
> > >  create mode 100755 t/t0091-bugreport.sh
> >
> > Hmm. I am still _quite_ convinced that this would be much better as a
> > built-in. Remember, non-built-ins come with a footprint, and I do not
> > necessarily think that you will want to spend 3MB on a `git-bugreport`
> > executable when you could have it for a couple dozen kilobytes inside
> > `git` instead.
>
> This is the kind of stuff I really wanted to get straightened out by
> sending the smaller changeset, so I'm glad to be having this
> conversation (again, and hopefully for the last time). I read the
> replies to this mail, which I'm truncating as I think many of them are
> distracting rather than discussion-worthy, and have tried to summarize:
>
> Builtin:
> + Don't have to call out-of-process to identify 'git version --build-options'
> + Better assurance that we aren't shipping a broken bugreport alongside a new
>   version
> - Binary bloat, possible startup time hit
> ? Libraries will behave identically to where the user is seeing issues
>   (This point is a possible pro but also a possible con; see similar point in
>   standalone list)
>
> Standalone:
> + Could investigate libraries who aren't behaving the way they should.
>   (This seems like it'd be better addressed by regression tests, and if we're so
>   sure that git-bugreport is working with the info at hand correctly, why don't
>   we just write the git binary that way too?)

No, you're linking with libgit.a, you will have _at least_ the same set of
dependencies as `git`. I made that point before.

> + Avoid binary bloat in the main executable.

What? What bloat? Have you measured this?

> - Have to ship a big standalone binary instead.
> - To get accurate version/build info, need to query the Git executable in a new
>   process

Worse. You might pick up inconsistent info from unrelated Git versions,
and would be none the wiser. I made that point before.

For example, if you call `git bugreport` in a Git version that does not
even include `git-bugreport`, you could pick up a `git-bugreport` from a
_different_ Git version. This is a very real scenario on Windows, where we
have many applications that ship with their own minimal version of Git for
Windows.

> Of course if I missed capturing something, please add/correct. Some of
> these concerns are quantifiable - binary size and overhead, for example
> - so I'm planning on doing some experiments in the coming days. I plan
> put together the handful of patches in the latest version of the topic
> in both standalone and builtin mode, and then gather the following info:
>
>  - Binary size of 'git'
>  - Binary size of 'git-bugreport' when applicable
>  - Time for "make" following clean
>  - Time for 'git status' on an identical real-world repo (I'll use ours,
>    without touching the repo state e.g. changing branch or fetching in
>    between)
>  - Time to editor for 'git bugreport' with the same setup as previous
>  - Peak memory footprint during 'git status'

Something that you cannot quantify, of course, is the robustness. Which in
my mind is more important than all of the above, and of course you would
only be able to guarantee that with a built-in version.

> And of course, if there's more to compare that I can quantify, please
> let me know that too. I expect I'll be ready to run the experiments by
> Monday (taking some time off tomorrow) so there's time for me to hear
> about other concerns.
>
> I'll hold off on sharing my own preference until after we've got some
> benchmarking to look at, so I can understand the whole picture.

Why? Why not strong-arm your preference? Junio and I are not shy doing the
same, and those are _your_ patches. Junio is clearly not interested in the
command at all, but you are clearly interested in it [*1*]. You should
have more than just the final say over this.

Ciao,
Dscho

Footnote *1*: I am *obviously* interested in this because it might
potentially make my life as Git for Windows' maintainer a lot easier. If
it is robust enough that I don't get bug reports about `git-bugreport`
itself.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-19 21:39               ` Emily Shaffer
  2020-03-20  0:28                 ` Junio C Hamano
  2020-03-20 15:42                 ` Johannes Schindelin
@ 2020-03-20 17:43                 ` Junio C Hamano
  2020-03-20 22:38                   ` Johannes Schindelin
  2 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-03-20 17:43 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Johannes Schindelin, git

Emily Shaffer <emilyshaffer@google.com> writes:

> This is the kind of stuff I really wanted to get straightened out by
> sending the smaller changeset, so I'm glad to be having this
> conversation (again, and hopefully for the last time).

I actually have a suspicion that "git bugreport" that is spawned via
"git" wrapper is a bad idea (in other words, /usr/bin/git-bug that
is totally standalone may be better).

The thing is, anything launched by "git" as its subcommand (be it
standalone or builtin) sees an environment already modified by
"git", so inspecting say "$PATH" from "git bugreport" (be it
standalone or builtin) does not show what the end-user, who may be
having trouble, actually has.  The mangling of $PATH alone happens
to be simple and (we may think) we can easily reverse without losing
bits, but there would probably other differences that we think is so
subtle and insignificant, right now in this discussion without
having actual end-user who is having trouble in front of us, that
having "git" layer in between may hide from us.

Having "git-bug" a totally separate tool, that does not even go
through git.c::cmd_main(), would also allow (and force) us to treat
"git" just like any system component on the end-users' system whose
versions and configurations may affect the use of "git" by the end
user.  The tool, instead of relying on git_version_string[] that is
determined at the compile time, would ask "git version" just like
the end-users would when asked by us "what version of git do you
run?"  It also means that "git-bug" can be updated at different
cadence and the mismatch of versions between it and "git" does not
matter at all.

Having said all that, after suggesting to make the tool even more
distant from the remainder of the binary, quite honestly, I do not
even want to see us wasting any more time on builtin/standalone at
this point, and instead would like to see reports from the end-users
produced by a lets-start-small version of the tool and measure how
having the tool actually helps.

Thanks.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-20 17:43                 ` Junio C Hamano
@ 2020-03-20 22:38                   ` Johannes Schindelin
  2020-03-20 22:47                     ` Junio C Hamano
  0 siblings, 1 reply; 273+ messages in thread
From: Johannes Schindelin @ 2020-03-20 22:38 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Emily Shaffer, git

Hi Junio,

On Fri, 20 Mar 2020, Junio C Hamano wrote:

> Emily Shaffer <emilyshaffer@google.com> writes:
>
> > This is the kind of stuff I really wanted to get straightened out by
> > sending the smaller changeset, so I'm glad to be having this
> > conversation (again, and hopefully for the last time).
>
> I actually have a suspicion that "git bugreport" that is spawned via
> "git" wrapper is a bad idea (in other words, /usr/bin/git-bug that
> is totally standalone may be better).

The obvious downside of `/usr/bin/git-bug`, of course, is that it has no
way to provide accurate data regarding, say, the cURL version in use.

That's a pretty severe downside if you are truly interested in helping bug
reporters and bug receivers.

Ciao,
Dscho

>
> The thing is, anything launched by "git" as its subcommand (be it
> standalone or builtin) sees an environment already modified by
> "git", so inspecting say "$PATH" from "git bugreport" (be it
> standalone or builtin) does not show what the end-user, who may be
> having trouble, actually has.  The mangling of $PATH alone happens
> to be simple and (we may think) we can easily reverse without losing
> bits, but there would probably other differences that we think is so
> subtle and insignificant, right now in this discussion without
> having actual end-user who is having trouble in front of us, that
> having "git" layer in between may hide from us.
>
> Having "git-bug" a totally separate tool, that does not even go
> through git.c::cmd_main(), would also allow (and force) us to treat
> "git" just like any system component on the end-users' system whose
> versions and configurations may affect the use of "git" by the end
> user.  The tool, instead of relying on git_version_string[] that is
> determined at the compile time, would ask "git version" just like
> the end-users would when asked by us "what version of git do you
> run?"  It also means that "git-bug" can be updated at different
> cadence and the mismatch of versions between it and "git" does not
> matter at all.
>
> Having said all that, after suggesting to make the tool even more
> distant from the remainder of the binary, quite honestly, I do not
> even want to see us wasting any more time on builtin/standalone at
> this point, and instead would like to see reports from the end-users
> produced by a lets-start-small version of the tool and measure how
> having the tool actually helps.
>
> Thanks.
>

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-20 22:38                   ` Johannes Schindelin
@ 2020-03-20 22:47                     ` Junio C Hamano
  2020-03-21 10:53                       ` Johannes Schindelin
  0 siblings, 1 reply; 273+ messages in thread
From: Junio C Hamano @ 2020-03-20 22:47 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Emily Shaffer, git

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> I actually have a suspicion that "git bugreport" that is spawned via
>> "git" wrapper is a bad idea (in other words, /usr/bin/git-bug that
>> is totally standalone may be better).
>
> The obvious downside of `/usr/bin/git-bug`, of course, is that it has no
> way to provide accurate data regarding, say, the cURL version in use.

Sorry, but I do not see what's new in your argument this time.

I thought we've already established that the best solution for the
"accurate data regarding, say, the cURL version in use" is to use
your earlier idea, i.e. give an interface to git-remote-curl so that
"git bugreport" can ask it such details, because "git bugreport"
that is known by git.c::cmd_main(), whether it is builtin or
standalone, is *NOT* linked to the transport anyway.

And the same interface can be used by an independent "git-bug", or
even by the end user who is sitting at a terminal when asked by git
developers "what version of curl library does your build link
with?".

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-20 22:47                     ` Junio C Hamano
@ 2020-03-21 10:53                       ` Johannes Schindelin
  0 siblings, 0 replies; 273+ messages in thread
From: Johannes Schindelin @ 2020-03-21 10:53 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Emily Shaffer, git

Hi Junio,

On Fri, 20 Mar 2020, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> >> I actually have a suspicion that "git bugreport" that is spawned via
> >> "git" wrapper is a bad idea (in other words, /usr/bin/git-bug that
> >> is totally standalone may be better).
> >
> > The obvious downside of `/usr/bin/git-bug`, of course, is that it has no
> > way to provide accurate data regarding, say, the cURL version in use.
>
> Sorry, but I do not see what's new in your argument this time.
>
> I thought we've already established that the best solution for the
> "accurate data regarding, say, the cURL version in use" is to use
> your earlier idea, i.e. give an interface to git-remote-curl so that
> "git bugreport" can ask it such details, because "git bugreport"
> that is known by git.c::cmd_main(), whether it is builtin or
> standalone, is *NOT* linked to the transport anyway.
>
> And the same interface can be used by an independent "git-bug", or
> even by the end user who is sitting at a terminal when asked by git
> developers "what version of curl library does your build link
> with?".

You are _asking_ for version mismatch here. If `git-bug` is a totally
standalone program, then it cannot rely at all on `git` to give it the
information it wants. Forward compatibility becomes another concern.

In other words, all the avoidably incurred complexity will come back to
make the entire `git bugreport` not worth all the effort Emily put into
it.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-20 15:42                 ` Johannes Schindelin
@ 2020-03-23 18:50                   ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-03-23 18:50 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git

On Fri, Mar 20, 2020 at 04:42:36PM +0100, Johannes Schindelin wrote:
> Hi Emily,
> 
> On Thu, 19 Mar 2020, Emily Shaffer wrote:
> > I'll hold off on sharing my own preference until after we've got some
> > benchmarking to look at, so I can understand the whole picture.
> 
> Why? Why not strong-arm your preference? Junio and I are not shy doing the
> same, and those are _your_ patches. Junio is clearly not interested in the
> command at all, but you are clearly interested in it [*1*]. You should
> have more than just the final say over this.

We are different people, and we approach problems and disagreements
differently. I am almost never the type to strong-arm my preference; I'd
rather listen to other arguments and look at data, which is why I
proposed the experiments I mentioned here. As for you and Junio not
being shy, as you put it, I found most of the discussion offputting, and
the tone used absolutely contributed to me finding other things to work
on instead of coming back to this minefield sooner.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 2/5] bugreport: add tool to generate debugging info
  2020-03-20 15:35                   ` Johannes Schindelin
@ 2020-03-23 18:52                     ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-03-23 18:52 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Junio C Hamano, git

On Fri, Mar 20, 2020 at 04:35:45PM +0100, Johannes Schindelin wrote:
> On Thu, 19 Mar 2020, Junio C Hamano wrote:
> 
> > In any case, I think we've wasted enough time on this.  Let's see a
> > minimum working version that won't break/affect the rest of Git and
> > ship it standalone.
> 
> I still disagree with that, but it seems that no amount of arguments will
> convince Junio, and he's dead set on the standalone version.

In this case, I'll leave the build settings alone and focus on other
comments on this patch. Will try to work on it today.

 - Emily

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 3/5] bugreport: gather git version and build info
  2020-03-02 23:03           ` [PATCH v9 3/5] bugreport: gather git version and build info Emily Shaffer
@ 2020-03-23 21:20             ` Junio C Hamano
  0 siblings, 0 replies; 273+ messages in thread
From: Junio C Hamano @ 2020-03-23 21:20 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +The following information is captured automatically:
> +
> + - 'git version --build-options'

Good.  

Do make sure to give us the same information as if the end user
typed that command from the command line.  There may be more than
one directory on $PATH that has "git" in it; we want to see the one
that is giving trouble to the end user, not necessarily the one that
came from the same source as running "git bugreport" was compiled
from, so it is a good idea not to rely on the contents of static
git_version_string[] array that came in libgit.a that the bugreport
tool happened to be linked with.
   
It may not be in the scope of this step, but we'd want to see
versions of executables for the python interpreter (2vs3), the
tcl/tk interpreter (i.e. wish), the shell (i.e. bash, dash, etc.).

In addition to them, we would want to see versions of some of our
own binaries (i.e. "git", and transports that are separate
executable files).  It would lead to a confusing diagnositic session
if the end-user's PATH has /usr/bin and then $HOME/bin, and the
end-user has a newer version of "git" with "bugreport" only in
$HOME/bin.  The user may think s/he is using the latest
self-compiled "git", and "bugreport" might even agree if it trusts
anything determined when "bugreport" was compiled, but in reality
the version of "git" used largely was coming from "/usr/bin" except
for any new subcommands, or something silly like that, and asking
the environment just like the end user does is the only way to make
sure we are getting the true piture.

Thanks.

^ permalink raw reply	[flat|nested] 273+ messages in thread

* Re: [PATCH v9 5/5] bugreport: add compiler info
  2020-03-04 21:39             ` Johannes Schindelin
@ 2020-03-23 21:27               ` Emily Shaffer
  0 siblings, 0 replies; 273+ messages in thread
From: Emily Shaffer @ 2020-03-23 21:27 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git

On Wed, Mar 04, 2020 at 10:39:53PM +0100, Johannes Schindelin wrote:
> Hi Emily,
> 
> On Mon, 2 Mar 2020, Emily Shaffer wrote:
> 
> > diff --git a/compat/compiler.h b/compat/compiler.h
> > new file mode 100644
> > index 0000000000..21f098e6a4
> > --- /dev/null
> > +++ b/compat/compiler.h
> > @@ -0,0 +1,38 @@
> > +#ifndef COMPILER_H
> > +#define COMPILER_H
> > +
> > +#include "git-compat-util.h"
> > +#include "strbuf.h"
> > +
> > +#ifdef __GLIBC__
> > +#include <gnu/libc-version.h>
> > +#endif
> > +
> > +static inline void get_compiler_info(struct strbuf *info)
> > +{
> > +	int len = info->len;
> > +#ifdef __GNUC__
> > +	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
> > +#endif
> > +
> > +#ifdef _MSC_VER
> > +	strbuf_addf(info, "MSVC version: %s\n", _MSC_FULL_VER);
> 
> I need this to fix a crash in the Visual Studio build's test run:

Hummm, is that not part of the GGG suite? I saw it pass (and then failed
to include the updated patches with v9 with the changes you mentioned
below).

> 
> -- snip --
> Subject: [PATCH] fixup??? bugreport: add compiler info
> 
> As documented at
> https://docs.microsoft.com/en-us/cpp/preprocessor/predefined-macros?view=vs-2019,
> `_MSC_FULL_VER` is an integer, not a string. Therefore this fix is
> needed to prevent a segmentation fault in the test t0091.1.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  compat/compiler.h | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/compat/compiler.h b/compat/compiler.h
> index ce6a7f6de9a..ac90fa051dd 100644
> --- a/compat/compiler.h
> +++ b/compat/compiler.h
> @@ -16,7 +16,8 @@ static inline void get_compiler_info(struct strbuf *info)
>  #endif
> 
>  #ifdef _MSC_VER
> -	strbuf_addf(info, "MSVC version: %s\n", _MSC_FULL_VER);
> +	strbuf_addf(info, "MSVC version: %02d.%02d.%05d\n",
> +		    _MSC_VER / 100, _MSC_VER % 100, _MSC_FULL_VER % 100000);
>  #endif
> 
>  	if (len == info->len)
> -- snap --
> 
> Could you squash that in, please?
> 
> BTW the two issues I reported in the earlier round are still problems.
> Could you squash my fixes, please? For your convenience, you can pluck all
> four of them right off of
> https://github.com/git-for-windows/git/commits/shears/pu

Yes, I think these two issues earlier I have locally and thumbfingered
running format-patch.

> 
> Just replace the `???` by `!` and you can even use `git rebase
> --autosquash`.
> 
> Ciao,
> Dscho
> 
> 
> > +#endif
> > +
> > +	if (len == info->len)
> > +		strbuf_addf(info, _("no compiler information available\n"));
> > +}
> > +
> > +static inline void get_libc_info(struct strbuf *info)
> > +{
> > +	int len = info->len;
> > +
> > +#ifdef __GLIBC__
> > +	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
> > +#endif
> > +
> > +	if (len == info->len)
> > +		strbuf_addf(info, _("no libc information available\n"));
> > +}
> > +
> > +#endif /* COMPILER_H */
> > --
> > 2.25.0.265.gbab2e86ba0-goog
> >
> >

^ permalink raw reply	[flat|nested] 273+ messages in thread

end of thread, other threads:[~2020-03-23 21:28 UTC | newest]

Thread overview: 273+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
2019-12-13  0:42 ` [PATCH v4 01/15] bugreport: add tool to generate debugging info Emily Shaffer
2019-12-13  0:42 ` [PATCH v4 02/15] help: move list_config_help to builtin/help Emily Shaffer
2019-12-13 20:51   ` Junio C Hamano
2019-12-16 21:36     ` Emily Shaffer
2019-12-16 22:19       ` Junio C Hamano
2019-12-16 22:34         ` Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 03/15] bugreport: gather git version and build info Emily Shaffer
2019-12-13 21:06   ` Junio C Hamano
2019-12-20  1:46     ` Emily Shaffer
2019-12-17 18:45   ` Johannes Schindelin
2019-12-17 20:34     ` Junio C Hamano
2019-12-20  1:25       ` Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 04/15] help: add shell-path to --build-options Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 05/15] bugreport: add uname info Emily Shaffer
2019-12-13 21:12   ` Junio C Hamano
2020-01-10  2:05   ` Aaron Schrab
2019-12-13  0:43 ` [PATCH v4 06/15] bugreport: add glibc version Emily Shaffer
2019-12-13 21:18   ` Junio C Hamano
2019-12-16 22:39     ` Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 07/15] bugreport: add curl version Emily Shaffer
2019-12-13 21:27   ` Junio C Hamano
2019-12-16 22:49     ` Emily Shaffer
2019-12-17 18:47   ` Johannes Schindelin
2019-12-13  0:43 ` [PATCH v4 08/15] bugreport: include user interactive shell Emily Shaffer
2019-12-13 21:38   ` Junio C Hamano
2019-12-13  0:43 ` [PATCH v4 09/15] bugreport: generate config safelist based on docs Emily Shaffer
2019-12-13 22:57   ` Junio C Hamano
2019-12-16 23:01     ` Emily Shaffer
2019-12-17  0:41       ` Emily Shaffer
2019-12-15 20:17   ` Johannes Schindelin
2019-12-16 22:52     ` Emily Shaffer
2019-12-17 18:38   ` Johannes Schindelin
2019-12-13  0:43 ` [PATCH v4 10/15] bugreport: add config values from safelist Emily Shaffer
2019-12-13 21:45   ` Junio C Hamano
2019-12-16 23:40     ` Emily Shaffer
2019-12-17 17:43       ` Junio C Hamano
2020-01-24  3:29         ` Emily Shaffer
2019-12-29 20:17   ` Johannes Schindelin
2019-12-13  0:43 ` [PATCH v4 11/15] bugreport: collect list of populated hooks Emily Shaffer
2019-12-13 21:47   ` Junio C Hamano
2019-12-16 23:51     ` Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 12/15] bugreport: count loose objects Emily Shaffer
2019-12-13 21:51   ` Junio C Hamano
2019-12-16 23:54     ` Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 13/15] bugreport: add packed object summary Emily Shaffer
2019-12-13 21:56   ` Junio C Hamano
2019-12-16 23:56     ` Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 15/15] bugreport: summarize contents of alternates file Emily Shaffer
2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
2020-01-24  3:34   ` [PATCH v5 01/15] bugreport: add tool to generate debugging info emilyshaffer
2020-01-30 22:18     ` Martin Ågren
2020-02-04 22:00       ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 02/15] help: move list_config_help to builtin/help emilyshaffer
2020-01-30 22:19     ` Martin Ågren
2020-02-04  0:53       ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 03/15] bugreport: gather git version and build info emilyshaffer
2020-01-30 22:19     ` Martin Ågren
2020-02-04 22:21       ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 04/15] help: add shell-path to --build-options emilyshaffer
2020-01-30 22:21     ` Martin Ågren
2020-01-24  3:34   ` [PATCH v5 05/15] bugreport: add uname info emilyshaffer
2020-01-24  3:34   ` [PATCH v5 06/15] bugreport: add compiler info emilyshaffer
2020-01-30 22:21     ` Martin Ågren
2020-02-04 22:51       ` Emily Shaffer
2020-02-05 19:47         ` Martin Ågren
2020-01-24  3:34   ` [PATCH v5 07/15] bugreport: add curl version emilyshaffer
2020-01-30 22:27     ` Martin Ågren
2020-02-04 22:54       ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 08/15] bugreport: include user interactive shell emilyshaffer
2020-01-30 22:28     ` Martin Ågren
2020-02-04 23:16       ` Emily Shaffer
2020-02-05 20:06       ` Junio C Hamano
2020-02-05 20:14         ` Martin Ågren
2020-01-24  3:34   ` [PATCH v5 09/15] bugreport: generate config safelist based on docs emilyshaffer
2020-01-30 22:34     ` Martin Ågren
2020-02-05  0:44       ` Emily Shaffer
2020-02-05 19:53         ` Martin Ågren
2020-01-31 21:20     ` Martin Ågren
2020-02-05  0:30       ` Emily Shaffer
2020-02-05  0:52         ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 10/15] bugreport: add config values from safelist emilyshaffer
2020-01-30 22:36     ` Martin Ågren
2020-02-05  1:34       ` Emily Shaffer
2020-01-31 21:25     ` Martin Ågren
2020-02-05  2:31       ` Emily Shaffer
2020-02-05 20:12         ` Martin Ågren
2020-01-24  3:34   ` [PATCH v5 11/15] bugreport: collect list of populated hooks emilyshaffer
2020-02-04 18:44     ` Junio C Hamano
2020-02-05  2:48       ` Emily Shaffer
2020-02-05  3:00         ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 12/15] bugreport: count loose objects emilyshaffer
2020-02-04 18:48     ` Junio C Hamano
2020-02-05  2:50       ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 13/15] bugreport: add packed object summary emilyshaffer
2020-02-04 19:00     ` Junio C Hamano
2020-02-05  3:15       ` Emily Shaffer
2020-02-04 19:03     ` Junio C Hamano
2020-02-05  3:09       ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 14/15] bugreport: list contents of $OBJDIR/info emilyshaffer
2020-01-24  3:34   ` [PATCH v5 15/15] bugreport: summarize contents of alternates file emilyshaffer
2020-01-24  3:38   ` [PATCH v5 00/15] add git-bugreport tool Emily Shaffer
2020-01-28 23:04   ` Jonathan Tan
2020-01-28 23:26     ` Emily Shaffer
2020-01-30 22:15   ` Martin Ågren
2020-02-04  0:07     ` Emily Shaffer
2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
2020-02-06  0:40     ` [PATCH v6 01/15] help: move list_config_help to builtin/help Emily Shaffer
2020-02-06  1:35       ` Danh Doan
2020-02-13 22:58         ` Emily Shaffer
2020-02-13 23:07           ` Eric Sunshine
2020-02-13 23:24             ` Junio C Hamano
2020-02-13 23:29               ` Eric Sunshine
2020-02-14  1:20                 ` Emily Shaffer
2020-02-06  0:40     ` [PATCH v6 02/15] help: add shell-path to --build-options Emily Shaffer
2020-02-06  0:40     ` [PATCH v6 03/15] bugreport: add tool to generate debugging info Emily Shaffer
2020-02-07 14:18       ` SZEDER Gábor
2020-02-07 18:51         ` Junio C Hamano
2020-02-11 22:40           ` Emily Shaffer
2020-02-07 14:54       ` SZEDER Gábor
2020-02-12 18:06       ` Junio C Hamano
2020-02-12 22:36         ` Emily Shaffer
2020-02-06  0:40     ` [PATCH v6 04/15] bugreport: gather git version and build info Emily Shaffer
2020-02-06  0:40     ` [PATCH v6 05/15] bugreport: add uname info Emily Shaffer
2020-02-06  0:40     ` [PATCH v6 06/15] bugreport: add compiler info Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 07/15] bugreport: add git-remote-https version Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 08/15] bugreport: include user interactive shell Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 09/15] bugreport: generate config safelist based on docs Emily Shaffer
2020-02-07 15:30       ` SZEDER Gábor
2020-02-13 23:14         ` Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 10/15] bugreport: add config values from safelist Emily Shaffer
2020-02-07 14:47       ` SZEDER Gábor
2020-02-07 15:08         ` SZEDER Gábor
2020-02-07 16:24           ` Eric Sunshine
2020-02-07 16:51             ` Andreas Schwab
2020-02-13 22:02               ` Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 11/15] bugreport: collect list of populated hooks Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 12/15] bugreport: count loose objects Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 13/15] bugreport: add packed object summary Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 15/15] bugreport: summarize contents of alternates file Emily Shaffer
2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 01/15] help: move list_config_help to builtin/help Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 02/15] help: add shell-path to --build-options Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 03/15] bugreport: add tool to generate debugging info Emily Shaffer
2020-02-14 17:25         ` Junio C Hamano
2020-02-15  1:57           ` Emily Shaffer
2020-02-15 18:24             ` Junio C Hamano
2020-02-18 23:46               ` Emily Shaffer
2020-02-18 23:56                 ` Emily Shaffer
2020-02-19 23:15                   ` Emily Shaffer
2020-02-19 23:24                     ` Junio C Hamano
2020-02-19 14:18         ` Johannes Schindelin
2020-02-19 16:55           ` Junio C Hamano
2020-02-19 21:52             ` Emily Shaffer
2020-02-19 22:09               ` Junio C Hamano
2020-02-19 23:06                 ` Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 04/15] bugreport: gather git version and build info Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 05/15] bugreport: add uname info Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 06/15] bugreport: add compiler info Emily Shaffer
2020-02-19 14:23         ` Johannes Schindelin
2020-02-19 22:45           ` Emily Shaffer
2020-02-20 22:33             ` Johannes Schindelin
2020-02-20 23:33               ` Emily Shaffer
2020-02-21 15:22                 ` Johannes Schindelin
2020-02-22  0:04                   ` Emily Shaffer
2020-02-24  2:55                     ` Junio C Hamano
2020-02-14  1:53       ` [PATCH v7 07/15] bugreport: add git-remote-https version Emily Shaffer
2020-02-19 14:28         ` Johannes Schindelin
2020-02-19 22:28           ` Emily Shaffer
2020-02-19 22:33             ` Junio C Hamano
2020-02-20 22:33               ` Johannes Schindelin
2020-02-14  1:53       ` [PATCH v7 08/15] bugreport: include user interactive shell Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 09/15] bugreport: generate config safelist based on docs Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 10/15] bugreport: add config values from safelist Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 11/15] bugreport: collect list of populated hooks Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 12/15] bugreport: count loose objects Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 13/15] bugreport: add packed object summary Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
2020-02-14 17:04         ` Junio C Hamano
2020-02-18 23:59           ` Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 15/15] bugreport: summarize contents of alternates file Emily Shaffer
2020-02-14 17:32       ` [PATCH v7 00/15] add git-bugreport tool Junio C Hamano
2020-02-14 22:00         ` Emily Shaffer
2020-02-14 22:30           ` Junio C Hamano
2020-02-20  1:58       ` [PATCH v8 " Emily Shaffer
2020-02-20  1:58         ` [PATCH v8 01/15] help: move list_config_help to builtin/help Emily Shaffer
2020-02-20  1:58         ` [PATCH v8 02/15] help: add shell-path to --build-options Emily Shaffer
2020-02-20 19:03           ` Junio C Hamano
2020-02-20 21:15             ` Emily Shaffer
2020-02-20  1:58         ` [PATCH v8 03/15] bugreport: add tool to generate debugging info Emily Shaffer
2020-02-20 19:33           ` Junio C Hamano
2020-02-20 22:33             ` Emily Shaffer
2020-02-26 16:12           ` Johannes Schindelin
2020-02-20  1:58         ` [PATCH v8 04/15] bugreport: gather git version and build info Emily Shaffer
2020-02-20 20:07           ` Junio C Hamano
2020-02-20 23:03             ` Emily Shaffer
2020-02-20 23:18               ` Junio C Hamano
2020-02-20  1:58         ` [PATCH v8 05/15] bugreport: add uname info Emily Shaffer
2020-02-20 20:12           ` Junio C Hamano
2020-02-20 23:20             ` Emily Shaffer
2020-02-20  1:58         ` [PATCH v8 06/15] bugreport: add compiler info Emily Shaffer
2020-02-20  2:49           ` Danh Doan
2020-02-20 23:23             ` Emily Shaffer
2020-02-20 20:23           ` Junio C Hamano
2020-02-21  0:26           ` Junio C Hamano
2020-02-20  1:58         ` [PATCH v8 07/15] bugreport: add git-remote-https version Emily Shaffer
2020-02-20 20:35           ` Junio C Hamano
2020-02-20 23:28             ` Emily Shaffer
2020-02-21  3:44               ` Junio C Hamano
2020-02-25 22:08                 ` Emily Shaffer
2020-02-25 22:26                   ` Junio C Hamano
2020-02-25 23:29                     ` Emily Shaffer
2020-02-25 23:29                   ` Junio C Hamano
2020-02-25 23:55                     ` Emily Shaffer
2020-02-20  1:58         ` [PATCH v8 08/15] bugreport: include user interactive shell Emily Shaffer
2020-02-20  1:58         ` [PATCH v8 09/15] bugreport: generate config safelist based on docs Emily Shaffer
2020-02-20 20:40           ` Junio C Hamano
2020-02-26 16:13           ` Johannes Schindelin
2020-02-26 16:49             ` Junio C Hamano
2020-02-20  1:58         ` [PATCH v8 10/15] bugreport: add config values from safelist Emily Shaffer
2020-02-20 20:47           ` Junio C Hamano
2020-02-20  1:58         ` [PATCH v8 11/15] bugreport: collect list of populated hooks Emily Shaffer
2020-02-20 20:58           ` Junio C Hamano
2020-02-25 23:19             ` Emily Shaffer
2020-02-20  1:58         ` [PATCH v8 12/15] bugreport: count loose objects Emily Shaffer
2020-02-20 21:04           ` Junio C Hamano
2020-02-25 23:22             ` Emily Shaffer
2020-02-25 23:26               ` Emily Shaffer
2020-02-20  1:58         ` [PATCH v8 13/15] bugreport: add packed object summary Emily Shaffer
2020-02-20 22:04           ` Junio C Hamano
2020-02-25 23:58             ` Emily Shaffer
2020-02-20  1:58         ` [PATCH v8 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
2020-02-20 22:18           ` Junio C Hamano
2020-02-20  1:58         ` [PATCH v8 15/15] bugreport: summarize contents of alternates file Emily Shaffer
2020-02-20 14:08           ` Johannes Schindelin
2020-02-20 22:22           ` Junio C Hamano
2020-03-02 23:03         ` [PATCH v9 0/5] add git-bugreport tool Emily Shaffer
2020-03-02 23:03           ` [PATCH v9 1/5] help: move list_config_help to builtin/help Emily Shaffer
2020-03-02 23:03           ` [PATCH v9 2/5] bugreport: add tool to generate debugging info Emily Shaffer
2020-03-03 14:18             ` Johannes Schindelin
2020-03-04 21:35             ` Johannes Schindelin
2020-03-05 23:34               ` Jeff Hostetler
2020-03-06 13:57                 ` Johannes Schindelin
2020-03-06 18:25                   ` Junio C Hamano
2020-03-06 18:08                 ` Junio C Hamano
2020-03-06 18:58                   ` Jeff Hostetler
2020-03-08 22:24                   ` Johannes Schindelin
2020-03-09 14:59                     ` Junio C Hamano
2020-03-09 19:17                       ` Johannes Schindelin
2020-03-09 19:47                         ` Junio C Hamano
2020-03-10 11:42                           ` Johannes Schindelin
2020-03-10 18:37                             ` Junio C Hamano
2020-03-10 22:08                               ` Johannes Schindelin
2020-03-19 21:39               ` Emily Shaffer
2020-03-20  0:28                 ` Junio C Hamano
2020-03-20 15:35                   ` Johannes Schindelin
2020-03-23 18:52                     ` Emily Shaffer
2020-03-20 15:42                 ` Johannes Schindelin
2020-03-23 18:50                   ` Emily Shaffer
2020-03-20 17:43                 ` Junio C Hamano
2020-03-20 22:38                   ` Johannes Schindelin
2020-03-20 22:47                     ` Junio C Hamano
2020-03-21 10:53                       ` Johannes Schindelin
2020-03-02 23:03           ` [PATCH v9 3/5] bugreport: gather git version and build info Emily Shaffer
2020-03-23 21:20             ` Junio C Hamano
2020-03-02 23:03           ` [PATCH v9 4/5] bugreport: add uname info Emily Shaffer
2020-03-02 23:04           ` [PATCH v9 5/5] bugreport: add compiler info Emily Shaffer
2020-03-03 11:46             ` Danh Doan
2020-03-03 14:07             ` Junio C Hamano
2020-03-04 21:39             ` Johannes Schindelin
2020-03-23 21:27               ` Emily Shaffer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).