What's cooking in git.git (Aug 2021, #02; Tue, 3)

What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Junio C Hamano]

Here are the topics that have been cooking in my tree.  Commits
prefixed with '+' are in 'next' (being in 'next' is a sign that a
topic is stable enough to be used and are candidate to be in a
future release).  Commits prefixed with '-' are only in 'seen',
which means nothing more than that I have found them of interest for
some reason (like "it may have hard-to-resolve conflicts with
another topic already in flight" or "this may turn out to be
useful").  Do not read too much into a topic being in (or not in)
'seen'.  The ones marked with '.' do not appear in any of the
integration branches, but I am still holding onto them.

A preview release Git 2.33-rc0 has been tagged.

Copies of the source code to Git live in many repositories, and the
following is a list of the ones I push into or their mirrors.  Some
repositories have only a subset of branches.

With maint, master, next, seen, todo:

	git://git.kernel.org/pub/scm/git/git.git/
	git://repo.or.cz/alt-git.git/
	https://kernel.googlesource.com/pub/scm/git/git/
	https://github.com/git/git/
	https://gitlab.com/git-vcs/git/

With all the integration branches and topics broken out:

	https://github.com/gitster/git/

Even though the preformatted documentation in HTML and man format
are not sources, they are published in these repositories for
convenience (replace "htmldocs" with "manpages" for the manual
pages):

	git://git.kernel.org/pub/scm/git/git-htmldocs.git/
	https://github.com/gitster/git-htmldocs.git/

Release tarballs are available at:

	https://www.kernel.org/pub/software/scm/git/

--------------------------------------------------
[Graduated to 'master']

* ab/bundle-tests (2021-07-22) 2 commits
  (merged to 'next' on 2021-07-22 at 053b5d0ecf)
 + bundle tests: use test_cmp instead of grep
 + bundle tests: use ">file" not ": >file"

 "git bundle" gained more test coverage.


* fc/pull-no-rebase-merges-theirs-into-ours (2021-07-21) 1 commit
  (merged to 'next' on 2021-07-28 at f8e6567082)
 + doc: pull: fix rebase=false documentation

 Documentation fix for "git pull --rebase=no".


* jk/check-pack-valid-before-opening-bitmap (2021-07-23) 1 commit
  (merged to 'next' on 2021-07-28 at 98d5b4dc68)
 + pack-bitmap: check pack validity when opening bitmap

 A race between repacking and using pack bitmaps has been corrected.


* jk/config-env-doc (2021-07-20) 3 commits
  (merged to 'next' on 2021-07-22 at 45616c831e)
 + doc/git-config: simplify "override" advice for FILES section
 + doc/git-config: clarify GIT_CONFIG environment variable
 + doc/git-config: explain --file instead of referring to GIT_CONFIG

 Documentation around GIT_CONFIG has been updated.


* js/ci-check-whitespace-updates (2021-07-14) 2 commits
  (merged to 'next' on 2021-07-22 at cdc9aa0622)
 + ci(check-whitespace): restrict to the intended commits
 + ci(check-whitespace): stop requiring a read/write token

 CI update.


* jt/bulk-prefetch (2021-07-23) 2 commits
  (merged to 'next' on 2021-07-28 at 26e28cab21)
 + cache-tree: prefetch in partial clone read-tree
 + unpack-trees: refactor prefetching code

 "git read-tree" had a codepath where blobs are fetched one-by-one
 from the promisor remote, which has been corrected to fetch in bulk.


* pb/submodule-recurse-doc (2021-07-20) 1 commit
  (merged to 'next' on 2021-07-22 at 4129e89833)
 + doc: clarify description of 'submodule.recurse'

 Doc update.


* ps/perf-with-separate-output-directory (2021-07-02) 1 commit
  (merged to 'next' on 2021-07-22 at af51ca0a39)
 + perf: fix when running with TEST_OUTPUT_DIRECTORY

 Test update.


* tb/bitmap-type-filter-comment-fix (2021-07-20) 1 commit
  (merged to 'next' on 2021-07-22 at 8428556149)
 + pack-bitmap: clarify comment in filter_bitmap_exclude_type()

 In-code comment update.

--------------------------------------------------
[New Topics]

* ab/test-columns (2021-08-02) 3 commits
 - test-lib.sh: use GIT_TEST_COLUMNS over COLUMNS
 - test-lib-functions.sh: add a test_with_columns function
 - test-lib-functions.sh: rename test_must_fail_acceptable()


* cb/reftable-fixup (2021-08-02) 3 commits
 - openbsd: allow reftable building with zlib 1.2.3
 - reftable: clarify zlib version dependency
 - fixup! Provide zlib's uncompress2 from compat/zlib-compat.c
 (this branch uses hn/reftable.)


* cb/t7508-regexp-fix (2021-08-02) 1 commit
 - t7508: avoid non POSIX BRE


* en/merge-strategy-docs (2021-08-03) 10 commits
 - Update error message and code comment
 - merge-strategies.txt: add coverage of the `ort` merge strategy
 - git-rebase.txt: correct out-of-date and misleading text about renames
 - merge-strategies.txt: fix simple capitalization error
 - merge-strategies.txt: avoid giving special preference to patience algorithm
 - merge-strategies.txt: do not imply using copy detection is desired
 - merge-strategies.txt: update wording for the resolve strategy
 - Documentation: edit awkward references to `git merge-recursive`
 - directory-rename-detection.txt: small updates due to merge-ort optimizations
 - git-rebase.txt: correct antiquated claims about --rebase-merges
 (this branch is used by en/ort-becomes-the-default.)


* en/ort-becomes-the-default (2021-08-03) 2 commits
 - Change default merge backend from recursive to ort
 - Update docs for change of default merge backend
 (this branch uses en/merge-strategy-docs.)


* js/log-protocol-version (2021-08-03) 1 commit
 - connect, protocol: log negotiated protocol version


* ow/clone-bare-origin (2021-08-03) 1 commit
 - clone: Allow combining --bare and --origin

--------------------------------------------------
[Stalled]

* gh/gitweb-branch-sort (2021-06-10) 1 commit
 - gitweb: use HEAD as secondary sort key in git_get_heads_list()

 Tie-break branches that point at the same object in the list of
 branches on GitWeb to show the one pointed at by HEAD early.

 Waiting for reviews.


* lh/systemd-timers (2021-07-02) 3 commits
 - maintenance: add support for systemd timers on Linux
 - maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
 - cache.h: Introduce a generic "xdg_config_home_for(…)" function

 "git maintenance" scheduler learned to use systemd timers as a
 possible backend.

 Waiting for reviews.


* fc/completion-updates (2021-06-07) 4 commits
 - completion: bash: add correct suffix in variables
 - completion: bash: fix for multiple dash commands
 - completion: bash: fix for suboptions with value
 - completion: bash: fix prefix detection in branch.*

 Command line completion updates.

 Expecting a reroll.
 cf. <60be6f7fa4435_db80d208f2@natae.notmuch>


* es/superproject-aware-submodules (2021-06-16) 5 commits
 - SQUASH???
 - submodule: cache superproject gitdir during 'update'
 - submodule: cache superproject gitdir during absorbgitdirs
 - introduce submodule.superprojectGitDir cache
 - t7400-submodule-basic: modernize inspect() helper

 A configuration variable in a submodule points at the location of
 the superproject it is bound to (RFC).

 Waiting for reviews.


* en/zdiff3 (2021-06-15) 2 commits
 - update documentation for new zdiff3 conflictStyle
 - xdiff: implement a zealous diff3, or "zdiff3"

 "Zealous diff3" style of merge conflict presentation has been added.

 Expecting a reroll.
 cf. <CABPp-BE7-E03+x38EK-=AE5mwwdST+d50hiiud2eY2Nsf3rM5g@mail.gmail.com>


* ao/p4-avoid-decoding (2021-04-12) 2 commits
 - git-p4: do not decode data from perforce by default
 - git-p4: avoid decoding more data from perforce

 "git p4" in Python-2 days used to accept a lot more kinds of data
 from Perforce server as uninterrupted byte sequence, but after
 switching to Python-3, too many things are expected to be in UTF-8,
 which broke traditional use cases.

 Waiting for reviews.


* tv/p4-fallback-encoding (2021-04-30) 1 commit
 - git-p4: git-p4.fallbackEncoding to specify non UTF-8 charset

 "git p4" learns the fallbackEncoding configuration variable to
 safely accept changeset descriptions that aren't written in UTF-8.

 Waiting for reviews.

--------------------------------------------------
[Cooking]

* tb/mingw-rmdir-symlink-to-directory (2021-08-02) 1 commit
  (merged to 'next' on 2021-08-03 at a027d43cca)
 + mingw: align symlinks-related rmdir() behavior with Linux

 Windows rmdir() equivalent behaves differently from POSIX ones in
 that when used on a symbolic link that points at a directory, the
 target directory gets removed, which has been corrected.

 Will merge to 'master'.


* ab/getcwd-test (2021-07-30) 1 commit
  (merged to 'next' on 2021-08-02 at 22ecd02929)
 + t0001: fix broken not-quite getcwd(3) test in bed67874e2

 Portability test update.

 Will merge to 'master'.


* ar/doc-markup-fix (2021-07-30) 1 commit
  (merged to 'next' on 2021-08-02 at b99073fa75)
 + Documentation: render special characters correctly

 Doc mark-up fix.

 Will merge to 'master'.


* rs/use-fspathhash (2021-07-30) 1 commit
  (merged to 'next' on 2021-08-02 at 72c388e867)
 + use fspathhash() everywhere

 Code simplification.

 Will merge to 'master'.


* jc/bisect-sans-show-branch (2021-07-28) 2 commits
  (merged to 'next' on 2021-08-02 at 89a8d9a47b)
 + bisect: simplify return code from bisect_checkout()
 + bisect: do not run show-branch just to show the current commit

 "git bisect" spawned "git show-branch" only to pretty-print the
 title of the commit after checking out the next version to be
 tested; this has been rewritten in C.

 Will cook in 'next'.


* jc/trivial-threeway-binary-merge (2021-07-28) 1 commit
 - ll-merge: teach ll_binary_merge() a trivial three-way merge

 The built-in merge driver for binary files learned to resolve
 trivial three-way merges (e.g. apply change, which turns A into B,
 to content A) by itself, which would help "git apply --3way" used
 when there is no need to use "--3way".

 Will discard.
 Replace with a trivial-merge logic in apply.c::try_treeway() or
 apply.c::three_way_merge().


* ab/http-drop-old-curl (2021-07-30) 5 commits
  (merged to 'next' on 2021-08-02 at b382ac042f)
 + http: rename CURLOPT_FILE to CURLOPT_WRITEDATA
 + http: drop support for curl < 7.19.3 and < 7.17.0 (again)
 + http: drop support for curl < 7.19.4
 + http: drop support for curl < 7.16.0
 + http: drop support for curl < 7.11.1

 Support for ancient versions of cURL library has been dropped.

 Will cook in 'next'.


* ab/lib-subtest (2021-07-21) 10 commits
 - test-lib tests: assert 1 exit code, not non-zero
 - test-lib tests: refactor common part of check_sub_test_lib_test*()
 - test-lib tests: avoid subshell for "test_cmp" for readability
 - test-lib tests: get rid of copy/pasted mock test code
 - test-lib tests: don't provide a description for the sub-tests
 - test-lib tests: stop using a subshell in write_sub_test_lib_test()
 - test-lib tests: split up "write and run" into two functions
 - test-lib tests: move "run_sub_test" to a new lib-subtest.sh
 - Merge branch 'ps/t0000-output-directory-fix' into ab/lib-subtest
 - Merge branch 'jk/t0000-subtests-fix' into ab/lib-subtest

 Updates to the tests in t0000 to test the test framework.


* ds/add-with-sparse-index (2021-07-29) 5 commits
  (merged to 'next' on 2021-08-02 at ee3e1323bb)
 + add: remove ensure_full_index() with --renormalize
 + add: ignore outside the sparse-checkout in refresh()
 + pathspec: stop calling ensure_full_index
 + add: allow operating on a sparse-only index
 + t1092: test merge conflicts outside cone
 (this branch uses ds/commit-and-checkout-with-sparse-index.)

 "git add" can work better with the sparse index.

 Will cook in 'next'.


* ab/only-single-progress-at-once (2021-07-23) 8 commits
 - progress.c: add & assert a "global_progress" variable
 - pack-bitmap-write.c: add a missing stop_progress()
 - progress.c: add temporary variable from progress struct
 - progress.c: stop eagerly fflush(stderr) when not a terminal
 - progress.c: call progress_interval() from progress_test_force_update()
 - progress.c: move signal handler functions lower
 - progress.c tests: test some invalid usage
 - progress.c tests: make start/stop verbs on stdin

 Further tweaks on progress API.


* ab/progress-users-adjust-counters (2021-07-23) 3 commits
 - entry: show finer-grained counter in "Filtering content" progress line
 - midx: don't provide a total for QSORT() progress
 - commit-graph: fix bogus counter in "Scanning merged commits" progress line

 The code to show progress indicator in a few codepaths did not
 cover between 0-100%, which has been corrected.

 Waiting for a clarification.
 cf. <xmqqbl6slmer.fsf@gitster.g>


* ar/submodule-add-config (2021-07-28) 1 commit
 - submodule--helper: introduce add-config subcommand
 (this branch uses ar/submodule-add.)

 Large part of "git submodule add" gets rewritten in C.


* en/ort-perf-batch-15 (2021-08-03) 9 commits
 - merge-ort: remove compile-time ability to turn off usage of memory pools
 - merge-ort: reuse path strings in pool_alloc_filespec
 - merge-ort: store filepairs and filespecs in our mem_pool
 - diffcore-rename, merge-ort: add wrapper functions for filepair alloc/dealloc
 - merge-ort: switch our strmaps over to using memory pools
 - merge-ort: set up a memory pool
 - merge-ort: add pool_alloc, pool_calloc, and pool_strndup wrappers
 - diffcore-rename: use a mem_pool for exact rename detection's hashmap
 - merge-ort: rename str{map,intmap,set}_func()
 (this branch uses en/ort-perf-batch-14.)

 Final batch for "merge -sort" optimization.

 Will merge to 'next'.


* pb/merge-autostash-more (2021-07-23) 4 commits
  (merged to 'next' on 2021-07-30 at bfc8b41932)
 + merge: apply autostash if merge strategy fails
 + merge: apply autostash if fast-forward fails
 + Documentation: define 'MERGE_AUTOSTASH'
 + merge: add missing word "strategy" to a message

 The local changes stashed by "git merge --autostash" were lost when
 the merge failed in certain ways, which has been corrected.

 Will merge to 'master'.


* ah/plugleaks (2021-07-26) 12 commits
  (merged to 'next' on 2021-07-28 at fa15f6d1f4)
 + reset: clear_unpack_trees_porcelain to plug leak
 + builtin/rebase: fix options.strategy memory lifecycle
 + builtin/merge: free found_ref when done
 + builtin/mv: free or UNLEAK multiple pointers at end of cmd_mv
 + convert: release strbuf to avoid leak
 + read-cache: call diff_setup_done to avoid leak
 + ref-filter: also free head for ATOM_HEAD to avoid leak
 + diffcore-rename: move old_dir/new_dir definition to plug leak
 + builtin/for-each-repo: remove unnecessary argv copy to plug leak
 + builtin/submodule--helper: release unused strbuf to avoid leak
 + environment: move strbuf into block to plug leak
 + fmt-merge-msg: free newly allocated temporary strings when done

 Leak plugging.

 Will merge to 'master'.


* js/expand-runtime-prefix (2021-07-26) 6 commits
 - expand_user_path: allow in-flight topics to keep using the old name
 - interpolate_path(): allow specifying paths relative to the runtime prefix
 - Use a better name for the function interpolating paths
 - expand_user_path(): clarify the role of the `real_home` parameter
 - expand_user_path(): remove stale part of the comment
 - tests: exercise the RUNTIME_PREFIX feature

 Pathname expansion (like "~username/") learned a way to specify a
 location relative to Git installation (e.g. its $sharedir which is
 $(prefix)/share), with "%(prefix)".

 Will merge to 'next'.


* zh/cherry-pick-help-is-only-for-sequencer (2021-08-03) 2 commits
 - cherry-pick: use better advice message
 - cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP

 "git cherry-pick" loses its state file when a stray
 GIT_CHERRY_PICK_HELP environment is present, which has been
 corrected.

 Will merge to 'next'.


* dt/submodule-diff-fixes (2021-08-02) 3 commits
 - diff --submodule=diff: Don't print failure message twice
 - diff --submodule=diff: do not fail on ever-initialied deleted submodules
 - t4060: remove unused variable

 "git diff --submodule=diff" showed failure from run_command() when
 trying to run diff inside a submodule, when the user manually
 removes the submodule directory.

 Comments?


* fs/ssh-signing (2021-08-03) 9 commits
 - ssh signing: test that gpg fails for unkown keys
 - ssh signing: tests for logs, tags & push certs
 - ssh signing: duplicate t7510 tests for commits
 - ssh signing: verify signatures using ssh-keygen
 - ssh signing: provide a textual signing_key_id
 - ssh signing: retrieve a default key from ssh-agent
 - ssh signing: add ssh key format and signing code
 - ssh signing: add test prereqs
 - ssh signing: preliminary refactoring and clean-up

 Use ssh public crypto for object and push-cert signing.

 Comments?


* hn/refs-test-cleanup (2021-08-02) 11 commits
 - t6001: avoid direct file system access
 - t6500: use "ls -1" to snapshot ref database state
 - t7064: use update-ref -d to remove upstream branch
 - t1410: mark test as REFFILES
 - t1405: mark test for 'git pack-refs' as REFFILES
 - t1405: use 'git reflog exists' to check reflog existence
 - t2402: use ref-store test helper to create broken symlink
 - t3320: use git-symbolic-ref rather than filesystem access
 - t6120: use git-update-ref rather than filesystem access
 - t1503: mark symlink test as REFFILES
 - t6050: use git-update-ref rather than filesystem access

 A handful of tests that assumed implementation details of files
 backend for refs have been cleaned up.

 Will merge to 'next'.


* hn/reftable (2021-07-20) 26 commits
 - t7004: avoid direct filesystem access
 - t1404: annotate test cases with REFFILES
 - t1401,t2011: parameterize HEAD.lock for REFFILES
 - t1301: document what needs to be done for reftable
 - Add "test-tool dump-reftable" command.
 - git-prompt: prepare for reftable refs backend
 - refs: RFC: Reftable support for git-core
 - reftable: add dump utility
 - reftable: implement stack, a mutable database of reftable files.
 - reftable: implement refname validation
 - reftable: add merged table view
 - reftable: add a heap-based priority queue for reftable records
 - reftable: reftable file level tests
 - reftable: read reftable files
 - reftable: generic interface to tables
 - reftable: write reftable files
 - reftable: a generic binary tree implementation
 - reftable: reading/writing blocks
 - Provide zlib's uncompress2 from compat/zlib-compat.c
 - reftable: (de)serialization for the polymorphic record type.
 - reftable: add blocksource, an abstraction for random access reads
 - reftable: utility functions
 - reftable: add error related functionality
 - reftable: RFC: add LICENSE
 - init-db: set the_repository->hash_algo early on
 - hash.h: provide constants for the hash IDs
 (this branch is used by cb/reftable-fixup.)

 The "reftable" backend for the refs API.

 Seems to break CI jobs in 'seen'.


* ab/refs-files-cleanup (2021-08-02) 11 commits
 - refs/files: remove unused "errno != ENOTDIR" condition
 - refs/files: remove unused "errno == EISDIR" code
 - refs/files: remove unused "oid" in lock_ref_oid_basic()
 - reflog expire: don't lock reflogs using previously seen OID
 - refs/files: add a comment about refs_reflog_exists() call
 - refs: make repo_dwim_log() accept a NULL oid
 - refs/debug: re-indent argument list for "prepare"
 - refs/files: remove unused "skip" in lock_raw_ref() too
 - refs/files: remove unused "extras/skip" in lock_ref_oid_basic()
 - refs/files: remove unused REF_DELETING in lock_ref_oid_basic()
 - refs/packet: add missing BUG() invocations to reflog callbacks
 (this branch is used by hn/refs-errno-cleanup.)

 Will merge to 'next'.


* en/pull-conflicting-options (2021-07-22) 8 commits
 - pull: fix handling of multiple heads
 - pull: update docs & code for option compatibility with rebasing
 - pull: abort by default when fast-forwarding is not possible
 - pull: make --rebase and --no-rebase override pull.ff=only
 - pull: since --ff-only overrides, handle it first
 - pull: abort if --ff-only is given and fast-forwarding is impossible
 - t7601: add tests of interactions with multiple merge heads and config
 - t7601: test interaction of merge/rebase/fast-forward flags and options

 "git pull" had various corner cases that were not well thought out
 around its --rebase backend, e.g. "git pull --ff-only" did not stop
 but went ahead and rebased when the history on other side is not a
 descendant of our history.  The series tries to fix them up.

 Comments?


* bc/inactive-submodules (2021-07-02) 1 commit
 - submodule: mark submodules with update=none as inactive

 Usability update for inactive submodules.

 Comments?
 cf. <fc5ec100-1d42-4199-236e-7a99c9218f38@gmail.com>
 cf. <bf1893ee-6973-d8b2-659e-bb239a0a9ae2@gmail.com>


* en/ort-perf-batch-14 (2021-07-20) 7 commits
  (merged to 'next' on 2021-07-30 at 89cfdc4513)
 + merge-ort: restart merge with cached renames to reduce process entry cost
 + merge-ort: avoid recursing into directories when we don't need to
 + merge-ort: defer recursing into directories when merge base is matched
 + merge-ort: add a handle_deferred_entries() helper function
 + merge-ort: add data structures for allowable trivial directory resolves
 + merge-ort: add some more explanations in collect_merge_info_callback()
 + merge-ort: resolve paths early when we have sufficient information
 (this branch is used by en/ort-perf-batch-15.)

 Further optimization on "merge -sort" backend.

 Will merge to 'master'.


* cf/fetch-set-upstream-while-detached (2021-07-06) 1 commit
 - fetch: fix segfault on --set-upstream while on a detached HEAD

 "git fetch --set-upstream" while on detached HEAD segfaulted
 instead of noticing that such an operation did not make sense.

 Expecting a reroll.
 cf. <xmqqsg0ri5mq.fsf@gitster.g>


* ab/bundle-doc (2021-08-02) 4 commits
 - bundle doc: replace "basis" with "prerequsite(s)"
 - bundle doc: elaborate on rev<->ref restriction
 - bundle doc: elaborate on object prerequisites
 - bundle doc: rewrite the "DESCRIPTION" section

 Doc update.

 Will merge to 'next'.


* ab/pack-stdin-packs-fix (2021-07-09) 2 commits
 - pack-objects: fix segfault in --stdin-packs option
 - pack-objects tests: cover blindspots in stdin handling

 Input validation of "git pack-objects --stdin-packs" has been
 corrected.

 Will merge to 'next'.


* ds/commit-and-checkout-with-sparse-index (2021-07-20) 7 commits
  (merged to 'next' on 2021-07-30 at 52ed1b0091)
 + unpack-trees: resolve sparse-directory/file conflicts
 + t1092: document bad 'git checkout' behavior
 + checkout: stop expanding sparse indexes
 + sparse-index: recompute cache-tree
 + commit: integrate with sparse-index
 + p2000: compress repo names
 + p2000: add 'git checkout -' test and decrease depth
 (this branch is used by ds/add-with-sparse-index.)

 "git checkout" and "git commit" learn to work without unnecessarily
 expanding sparse indexes.

 Will merge to 'master'.


* jt/push-negotiation-fixes (2021-07-15) 3 commits
 - fetch: die on invalid --negotiation-tip hash
 - send-pack: fix push nego. when remote has refs
 - send-pack: fix push.negotiate with remote helper

 Bugfix for common ancestor negotiation recently introduced in "git
 push" codepath.

 Needs review.


* ab/make-tags-cleanup (2021-07-22) 5 commits
 - Makefile: normalize clobbering & xargs for tags targets
 - Makefile: the "cscope" target always creates a "cscope.out"
 - Makefile: don't use "FORCE" for tags targets
 - Makefile: add QUIET_GEN to "cscope" target
 - Makefile: move ".PHONY: cscope" near its target

 Build clean-up for "make tags" and friends.

 Expecting a reroll.
 4/5 may want a minor tweak to the log and the patch text but otherwise looks good.


* tb/multi-pack-bitmaps (2021-07-27) 25 commits
 - p5326: perf tests for MIDX bitmaps
 - p5310: extract full and partial bitmap tests
 - midx: respect 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP'
 - t7700: update to work with MIDX bitmap test knob
 - t5319: don't write MIDX bitmaps in t5319
 - t5310: disable GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP
 - t0410: disable GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP
 - t5326: test multi-pack bitmap behavior
 - t/helper/test-read-midx.c: add --checksum mode
 - t5310: move some tests to lib-bitmap.sh
 - pack-bitmap: write multi-pack bitmaps
 - pack-bitmap: read multi-pack bitmaps
 - pack-bitmap.c: avoid redundant calls to try_partial_reuse
 - pack-bitmap.c: introduce 'bitmap_is_preferred_refname()'
 - pack-bitmap.c: introduce 'nth_bitmap_object_oid()'
 - pack-bitmap.c: introduce 'bitmap_num_objects()'
 - midx: avoid opening multiple MIDXs when writing
 - midx: close linked MIDXs, avoid leaking memory
 - midx: infer preferred pack when not given one
 - midx: reject empty `--preferred-pack`'s
 - midx: clear auxiliary .rev after replacing the MIDX
 - Documentation: describe MIDX-based bitmaps
 - pack-bitmap-write.c: free existing bitmaps
 - pack-bitmap-write.c: gracefully fail to write non-closed bitmaps
 - pack-bitmap.c: harden 'test_bitmap_walk()' to check type bitmaps

 The reachability bitmap file used to be generated only for a single
 pack, but now we've learned to generate bitmaps for history that
 span across multiple packfiles.

 Comments?


* ab/config-based-hooks-base (2021-08-03) 36 commits
 - hooks: fix a TOCTOU in "did we run a hook?" heuristic
 - receive-pack: convert receive hooks to hook.h
 - post-update: use hook.h library
 - receive-pack: convert 'update' hook to hook.h
 - hooks: allow callers to capture output
 - run-command: allow capturing of collated output
 - reference-transaction: use hook.h to run hooks
 - hook tests: use a modern style for "pre-push" tests
 - hook tests: test for exact "pre-push" hook input
 - transport: convert pre-push hook to hook.h
 - hook: convert 'post-rewrite' hook in sequencer.c to hook.h
 - hook: provide stdin by string_list or callback
 - run-command: add stdin callback for parallelization
 - am: convert 'post-rewrite' hook to hook.h
 - hook: support passing stdin to hooks
 - run-command: allow stdin for run_processes_parallel
 - run-command: remove old run_hook_{le,ve}() hook API
 - receive-pack: convert push-to-checkout hook to hook.h
 - read-cache: convert post-index-change to use hook.h
 - commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
 - git-p4: use 'git hook' to run hooks
 - send-email: use 'git hook run' for 'sendemail-validate'
 - git hook run: add an --ignore-missing flag
 - merge: convert post-merge to use hook.h
 - hooks: convert 'post-checkout' hook to hook library
 - am: convert applypatch to use hook.h
 - rebase: convert pre-rebase to use hook.h
 - gc: use hook library for pre-auto-gc hook
 - hook: add 'run' subcommand
 - hook-list.h: add a generated list of hooks, like config-list.h
 - hook.c users: use "hook_exists()" insted of "find_hook()"
 - hook.c: add a hook_exists() wrapper and use it in bugreport.c
 - hook.[ch]: move find_hook() to this new library
 - Makefile: remove an out-of-date comment
 - Makefile: stop hardcoding {command,config}-list.h
 - Makefile: mark "check" target as .PHONY

 Restructuring of (a subset of) Emily's config-based-hooks series,
 to demonstrate that a series can be presented as a more logical and
 focused progression.

 Waiting for reviews.


* ab/serve-cleanup (2021-08-03) 13 commits
 - fixup! {upload,receive}-pack tests: add --advertise-refs tests
 - serve.[ch]: don't pass "struct strvec *keys" to commands
 - upload-pack.c: convert to new serve.c "startup" config cb
 - upload-pack: document and rename --advertise-refs
 - {upload,receive}-pack tests: add --advertise-refs tests
 - serve.[ch]: remove "serve_options", split up --advertise-refs code
 - serve.c: move version line to advertise_capabilities()
 - serve: add support for a "startup" git_config() callback
 - serve.c: add call_{advertise,command}() indirection
 - serve: use designated initializers
 - transport: use designated initializers
 - transport: rename "fetch" in transport_vtable to "fetch_refs"
 - serve: mark has_capability() as static

 Code clean-up around "git serve".


* pw/diff-color-moved-fix (2021-07-20) 12 commits
 - diff --color-moved: intern strings
 - diff: use designated initializers for emitted_diff_symbol
 - diff --color-moved-ws=allow-indentation-change: improve hash lookups
 - diff --color-moved: stop clearing potential moved blocks
 - diff --color-moved: shrink potential moved blocks as we go
 - diff --color-moved: unify moved block growth functions
 - diff --color-moved: call comparison function directly
 - diff --color-moved-ws=allow-indentation-change: simplify and optimize
 - diff: simplify allow-indentation-change delta calculation
 - diff --color-moved: avoid false short line matches and bad zerba coloring
 - diff --color-moved=zebra: fix alternate coloring
 - diff --color-moved: add perf tests

 Long-overdue correctness and performance update to "diff
 --color-moved" feature.

 Will merge to 'next'.


* hn/refs-errno-cleanup (2021-08-02) 7 commits
 - refs: make errno output explicit for refs_resolve_ref_unsafe
 - refs: explicitly return failure_errno from parse_loose_ref_contents
 - refs: add failure_errno to refs_read_raw_ref() signature
 - refs: make errno output explicit for read_raw_ref_fn
 - refs/files-backend: stop setting errno from lock_ref_oid_basic
 - refs: remove EINVAL errno output from specification of read_raw_ref_fn
 - refs file backend: move raceproof_create_file() here
 (this branch uses ab/refs-files-cleanup.)

 Futz with the way 'errno' is relied on in the refs API to carry the
 failure modes up the callchain.

 Will merge to 'next'.


* ab/test-tool-cache-cleanup (2021-06-08) 4 commits
 - read-cache perf: add a perf test for refresh_index()
 - test-tool: migrate read-cache-again to parse_options()
 - test-tool: migrate read-cache-perf to parse_options()
 - test-tool: split up test-tool read-cache

 Test code shuffling.

 Waiting for reviews.


* ab/pack-objects-stdin (2021-07-09) 5 commits
 - pack-objects.c: make use of REV_INFO_STDIN_LINE_PROCESS
 - pack-objects.c: do stdin parsing via revision.c's API
 - revision.[ch]: add a "handle_stdin_line" API
 - revision.h: refactor "disable_stdin" and "read_from_stdin"
 - upload-pack: run is_repository_shallow() before setup_revisions()

 Introduce handle_stdin_line callback to revision API and uses it.


* ar/submodule-add (2021-07-26) 5 commits
  (merged to 'next' on 2021-07-28 at 7d315a0f67)
 + submodule: drop unused sm_name parameter from show_fetch_remotes()
  (merged to 'next' on 2021-07-22 at b8b636c9a1)
 + submodule--helper: introduce add-clone subcommand
 + submodule--helper: refactor module_clone()
 + submodule: prefix die messages with 'fatal'
 + t7400: test failure to add submodule in tracked path
 (this branch is used by ar/submodule-add-config.)

 Rewrite of "git submodule" in C continues.

 Will merge to 'master'.


* ab/update-submitting-patches (2021-07-22) 2 commits
  (merged to 'next' on 2021-07-30 at 9ae2de7f7a)
 + SubmittingPatches: replace discussion of Travis with GitHub Actions
 + SubmittingPatches: move discussion of Signed-off-by above "send"

 Reorganize and update the SubmitingPatches document.

 Will merge to 'master'.


* zh/ref-filter-raw-data (2021-07-26) 6 commits
 - ref-filter: add %(rest) atom
 - ref-filter: use non-const ref_format in *_atom_parser()
 - ref-filter: --format=%(raw) support --perl
 - ref-filter: add %(raw) atom
 - ref-filter: add obj-type check in grab contents
 - Merge branch 'zh/cat-file-batch-fix' into zh/ref-filter-raw-data

 Prepare the "ref-filter" machinery that drives the "--format"
 option of "git for-each-ref" and its friends to be used in "git
 cat-file --batch".

 Will merge to 'next'.


* jh/builtin-fsmonitor (2021-07-12) 35 commits
 - BANDAID: sparse fixes
 - t7527: test FS event reporing on MacOS WRT case and Unicode
 - fsmonitor: handle shortname for .git
 - t7527: test status with untracked-cache and fsmonitor--daemon
 - fsmonitor: force update index after large responses
 - fsmonitor: enhance existing comments
 - fsmonitor--daemon: use a cookie file to sync with file system
 - fsmonitor--daemon: periodically truncate list of modified files
 - t7527: create test for fsmonitor--daemon
 - t/perf/p7519: add fsmonitor--daemon test cases
 - t/perf: avoid copying builtin fsmonitor files into test repo
 - t/perf/p7519: speed up test using "test-tool touch"
 - t/helper/test-touch: add helper to touch a series of files
 - fsmonitor--daemon: implement handle_client callback
 - fsmonitor-fs-listen-macos: implement FSEvent listener on MacOS
 - fsmonitor-fs-listen-macos: add macos header files for FSEvent
 - fsmonitor-fs-listen-win32: implement FSMonitor backend on Windows
 - fsmonitor--daemon: create token-based changed path cache
 - fsmonitor--daemon: define token-ids
 - fsmonitor--daemon: add pathname classification
 - fsmonitor: do not try to operate on bare repos
 - fsmonitor--daemon: implement 'start' command
 - fsmonitor--daemon: implement 'run' command
 - fsmonitor-fs-listen-macos: stub in backend for MacOS
 - fsmonitor-fs-listen-win32: stub in backend for Windows
 - t/helper/fsmonitor-client: create IPC client to talk to FSMonitor Daemon
 - fsmonitor--daemon: implement 'stop' and 'status' commands
 - fsmonitor--daemon: add a built-in fsmonitor daemon
 - fsmonitor: use IPC to query the builtin FSMonitor daemon
 - fsmonitor: config settings are repository-specific
 - help: include fsmonitor--daemon feature flag in version info
 - fsmonitor-ipc: create client routines for git-fsmonitor--daemon
 - fsmonitor--daemon: update fsmonitor documentation
 - fsmonitor--daemon: man page
 - simple-ipc: preparations for supporting binary messages.

 An attempt to write and ship with a watchman equivalent tailored
 for our use.

 Expecting a reroll post 2.33 release.


* es/trace2-log-parent-process-name (2021-07-22) 2 commits
 - tr2: log parent process name
 - tr2: make process info collection platform-generic

 trace2 logs learned to show parent process name to see in what
 context Git was invoked.

 Will merge to 'next'.


* ab/fsck-unexpected-type (2021-07-12) 21 commits
 - fsck: report invalid object type-path combinations
 - fsck: report invalid types recorded in objects
 - object-store.h: move read_loose_object() below 'struct object_info'
 - fsck: don't hard die on invalid object types
 - object-file.c: return -2 on "header too long" in unpack_loose_header()
 - object-file.c: return -1, not "status" from unpack_loose_header()
 - object-file.c: guard against future bugs in loose_object_info()
 - object-file.c: stop dying in parse_loose_header()
 - object-file.c: split up ternary in parse_loose_header()
 - object-file.c: simplify unpack_loose_short_header()
 - object-file.c: add missing braces to loose_object_info()
 - object-file.c: make parse_loose_header_extended() public
 - object-file.c: don't set "typep" when returning non-zero
 - cache.h: move object functions to object-store.h
 - cat-file tests: test for current --allow-unknown-type behavior
 - cat-file tests: add corrupt loose object test
 - rev-list tests: test for behavior with invalid object types
 - cat-file tests: test that --allow-unknown-type isn't on by default
 - cat-file tests: test for missing object with -t and -s
 - fsck tests: add test for fsck-ing an unknown type
 - fsck tests: refactor one test to use a sub-repo

 "git fsck" has been taught to report mismatch between expected and
 actual types of an object better.

 Needs review.

--------------------------------------------------
[Discarded]

* ab/doc-retire-alice-bob (2021-06-16) 6 commits
 . pack-protocol doc: use "www-data" in place of "alice"
 . doc: replace "alice" and "bob" with "jdoe" and "msmith"
 . fast-import doc: change "bob" in an example to "file.txt"
 . daemon doc + code comments: reword "alice" example
 . gitcvs-migration doc: replace "alice" and "bob" with "you" and "www-data"
 . gittutorial doc: replace "alice" and "bob" with "you" and "www-data"

 Documentation update to avoid Alice and Bob in contexts where Eve
 does not appear in.

 Will discard.
 Let's shelve this one for now, as it does not seem to solve any
 real problems (other than ceasing to use Alice and Bob in contexts
 that do not call for Eve).


* hn/refs-test-cleanup-contd (2021-07-22) 11 commits
  (merged to 'next' on 2021-07-28 at dd3af04939)
 + t6001: avoid direct file system access
 + t6500: use "ls -1" to snapshot ref database state
 + t7064: use update-ref -d to remove upstream branch
 + t1410: mark test as REFFILES
 + t1405: mark test for 'git pack-refs' as REFFILES
 + t1405: use 'git reflog exists' to check reflog existence
  (merged to 'next' on 2021-07-22 at 2ab8bc259a)
 + t2402: use ref-store test helper to create broken symlink
 + t3320: use git-symbolic-ref rather than filesystem access
 + t6120: use git-update-ref rather than filesystem access
 + t1503: mark symlink test as REFFILES
 + t6050: use git-update-ref rather than filesystem access

 Absorbed by the hn/refs-test-cleanup topic.


* os/bisect-runs-show-branch-without-pager (2021-07-27) 1 commit
 - bisect: disable pager while invoking show-branch

 When "git bisect" spawns "git show-branch" to pretty-print the
 title of the commit to be tested, it could have invoked user's
 pager if user configured to run pager while running show-branch.
 The invocation of show-branch has been changed to disable pager,
 even if one is configured.

 Will discard.
 This probably is unnecessary with the rewrite to get rid of the
 show-branch invocation altogether.


* es/config-based-hooks (2021-07-20) 9 commits
 - hook: implement hookcmd.<name>.skip
 - hook: teach 'hookcmd' config to alias hook scripts
 - hook: allow out-of-repo 'git hook' invocations
 - hook: include hooks from the config
 - hook: allow running non-native hooks
 - hook: treat hookdir hook specially
 - hook: introduce "git hook list"
 - hook: allow parallel hook execution
 - hook: run a list of hooks instead

 Discarded to give room for updated ab/config-based-hooks-base.

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Ævar Arnfjörð Bjarmason]

On Wed, Aug 04 2021, Junio C Hamano wrote:

Comments on my topics & other things I've got feedback on:

> * ab/bundle-tests (2021-07-22) 2 commits
>   (merged to 'next' on 2021-07-22 at 053b5d0ecf)
>  + bundle tests: use test_cmp instead of grep
>  + bundle tests: use ">file" not ": >file"
>
>  "git bundle" gained more test coverage.

Thanks!

> * ab/test-columns (2021-08-02) 3 commits
>  - test-lib.sh: use GIT_TEST_COLUMNS over COLUMNS
>  - test-lib-functions.sh: add a test_with_columns function
>  - test-lib-functions.sh: rename test_must_fail_acceptable()

We're going to need this or another solution to the v2.33.0-rc0
regression in c49a177beca (test-lib.sh: set COLUMNS=80 for --verbose
repeatability, 2021-06-29) before the final v2.33.0.

I think whatever we do we should act on that soon so it's in master
before rc1, or at least in rc1, so we can get testing of the fix well
before the release...

> * ab/getcwd-test (2021-07-30) 1 commit
>   (merged to 'next' on 2021-08-02 at 22ecd02929)
>  + t0001: fix broken not-quite getcwd(3) test in bed67874e2
>
>  Portability test update.
>
>  Will merge to 'master'.

Thanks!

> * ab/http-drop-old-curl (2021-07-30) 5 commits
>   (merged to 'next' on 2021-08-02 at b382ac042f)
>  + http: rename CURLOPT_FILE to CURLOPT_WRITEDATA
>  + http: drop support for curl < 7.19.3 and < 7.17.0 (again)
>  + http: drop support for curl < 7.19.4
>  + http: drop support for curl < 7.16.0
>  + http: drop support for curl < 7.11.1
>
>  Support for ancient versions of cURL library has been dropped.
>
>  Will cook in 'next'.

Makes sense, thanks. So post-release.

> * ab/lib-subtest (2021-07-21) 10 commits
>  - test-lib tests: assert 1 exit code, not non-zero
>  - test-lib tests: refactor common part of check_sub_test_lib_test*()
>  - test-lib tests: avoid subshell for "test_cmp" for readability
>  - test-lib tests: get rid of copy/pasted mock test code
>  - test-lib tests: don't provide a description for the sub-tests
>  - test-lib tests: stop using a subshell in write_sub_test_lib_test()
>  - test-lib tests: split up "write and run" into two functions
>  - test-lib tests: move "run_sub_test" to a new lib-subtest.sh
>  - Merge branch 'ps/t0000-output-directory-fix' into ab/lib-subtest
>  - Merge branch 'jk/t0000-subtests-fix' into ab/lib-subtest
>
>  Updates to the tests in t0000 to test the test framework.
>
> [...]
>
> * ab/only-single-progress-at-once (2021-07-23) 8 commits
>  - progress.c: add & assert a "global_progress" variable
>  - pack-bitmap-write.c: add a missing stop_progress()
>  - progress.c: add temporary variable from progress struct
>  - progress.c: stop eagerly fflush(stderr) when not a terminal
>  - progress.c: call progress_interval() from progress_test_force_update()
>  - progress.c: move signal handler functions lower
>  - progress.c tests: test some invalid usage
>  - progress.c tests: make start/stop verbs on stdin
>
>  Further tweaks on progress API.

Thanks, it would be great to have both of these move soon after the
release. I think the chances of unexpected breakage here are minimal
given their nature.

> * ab/progress-users-adjust-counters (2021-07-23) 3 commits
>  - entry: show finer-grained counter in "Filtering content" progress line
>  - midx: don't provide a total for QSORT() progress
>  - commit-graph: fix bogus counter in "Scanning merged commits" progress line
>
>  The code to show progress indicator in a few codepaths did not
>  cover between 0-100%, which has been corrected.
>
>  Waiting for a clarification.
>  cf. <xmqqbl6slmer.fsf@gitster.g>

I think that what SZEDER had to say in
https://lore.kernel.org/git/20210802220506.GF23408@szeder.dev/ should be
enough to clear this to proceed forward.

I.e. this topic is missing his subsequent
https://lore.kernel.org/git/20210620200303.2328957-7-szeder.dev@gmail.com/
patch.

But as he notes we only encounter this in one of our tests (and are
unlikely to in the wild).

Between this topic and my ab/only-single-progress-at-once I think we're
better off getting those basic fixes in before proceeding with any
tricky additions of BUG() and other assertions, per what I noted in
https://public-inbox.org/git/cover-00.25-00000000000-20210623T155626Z-avarab@gmail.com/;
i.e. that some of those assertions were themselves buggy.

> * ab/refs-files-cleanup (2021-08-02) 11 commits
>  - refs/files: remove unused "errno != ENOTDIR" condition
>  - refs/files: remove unused "errno == EISDIR" code
>  - refs/files: remove unused "oid" in lock_ref_oid_basic()
>  - reflog expire: don't lock reflogs using previously seen OID
>  - refs/files: add a comment about refs_reflog_exists() call
>  - refs: make repo_dwim_log() accept a NULL oid
>  - refs/debug: re-indent argument list for "prepare"
>  - refs/files: remove unused "skip" in lock_raw_ref() too
>  - refs/files: remove unused "extras/skip" in lock_ref_oid_basic()
>  - refs/files: remove unused REF_DELETING in lock_ref_oid_basic()
>  - refs/packet: add missing BUG() invocations to reflog callbacks
>  (this branch is used by hn/refs-errno-cleanup.)
>
>  Will merge to 'next'.

Thanks!

> * ab/bundle-doc (2021-08-02) 4 commits
>  - bundle doc: replace "basis" with "prerequsite(s)"
>  - bundle doc: elaborate on rev<->ref restriction
>  - bundle doc: elaborate on object prerequisites
>  - bundle doc: rewrite the "DESCRIPTION" section
>
>  Doc update.
>
>  Will merge to 'next'.

Yay, we finally got this in good enough shape :)

> * ab/pack-stdin-packs-fix (2021-07-09) 2 commits
>  - pack-objects: fix segfault in --stdin-packs option
>  - pack-objects tests: cover blindspots in stdin handling
>
>  Input validation of "git pack-objects --stdin-packs" has been
>  corrected.
>
>  Will merge to 'next'.

Thanks!

> * ab/make-tags-cleanup (2021-07-22) 5 commits
>  - Makefile: normalize clobbering & xargs for tags targets
>  - Makefile: the "cscope" target always creates a "cscope.out"
>  - Makefile: don't use "FORCE" for tags targets
>  - Makefile: add QUIET_GEN to "cscope" target
>  - Makefile: move ".PHONY: cscope" near its target
>
>  Build clean-up for "make tags" and friends.
>
>  Expecting a reroll.
>  4/5 may want a minor tweak to the log and the patch text but otherwise looks good.

This is the 3rd What's Cooking where I comment to the effect that I
don't think we need a re-roll (previously at
https://lore.kernel.org/git/87sg00qfbp.fsf@evledraar.gmail.com/ &
https://lore.kernel.org/git/87wnp4p4xo.fsf@evledraar.gmail.com/).

I.e. my comment in
https://lore.kernel.org/git/87k0lit57x.fsf@evledraar.gmail.com/ of:
    
    As long as we have no user of a -q flag ever, what's the point of
    forever carrying the "rm foo*" when we know it's foo.out, just
    because a future Makefile change might add foo.blah?
    
    Doesn't seem like something we'd trip over, and the .gitignore/rm
    rule is just misleading now...

So I think this is fine as-is, if you or Taylor still feel it should be
kept I can re-roll it, seems rather weird as long as we have no
$(CSCOPE_FLAGS), but whatever.

> * ab/config-based-hooks-base (2021-08-03) 36 commits
>  - hooks: fix a TOCTOU in "did we run a hook?" heuristic
>  - receive-pack: convert receive hooks to hook.h
>  - post-update: use hook.h library
>  - receive-pack: convert 'update' hook to hook.h
>  - hooks: allow callers to capture output
>  - run-command: allow capturing of collated output
>  - reference-transaction: use hook.h to run hooks
>  - hook tests: use a modern style for "pre-push" tests
>  - hook tests: test for exact "pre-push" hook input
>  - transport: convert pre-push hook to hook.h
>  - hook: convert 'post-rewrite' hook in sequencer.c to hook.h
>  - hook: provide stdin by string_list or callback
>  - run-command: add stdin callback for parallelization
>  - am: convert 'post-rewrite' hook to hook.h
>  - hook: support passing stdin to hooks
>  - run-command: allow stdin for run_processes_parallel
>  - run-command: remove old run_hook_{le,ve}() hook API
>  - receive-pack: convert push-to-checkout hook to hook.h
>  - read-cache: convert post-index-change to use hook.h
>  - commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
>  - git-p4: use 'git hook' to run hooks
>  - send-email: use 'git hook run' for 'sendemail-validate'
>  - git hook run: add an --ignore-missing flag
>  - merge: convert post-merge to use hook.h
>  - hooks: convert 'post-checkout' hook to hook library
>  - am: convert applypatch to use hook.h
>  - rebase: convert pre-rebase to use hook.h
>  - gc: use hook library for pre-auto-gc hook
>  - hook: add 'run' subcommand
>  - hook-list.h: add a generated list of hooks, like config-list.h
>  - hook.c users: use "hook_exists()" insted of "find_hook()"
>  - hook.c: add a hook_exists() wrapper and use it in bugreport.c
>  - hook.[ch]: move find_hook() to this new library
>  - Makefile: remove an out-of-date comment
>  - Makefile: stop hardcoding {command,config}-list.h
>  - Makefile: mark "check" target as .PHONY
>
>  Restructuring of (a subset of) Emily's config-based-hooks series,
>  to demonstrate that a series can be presented as a more logical and
>  focused progression.
>
>  Waiting for reviews.

Indeed: https://lore.kernel.org/git/87zgtxms4s.fsf@evledraar.gmail.com/

> * ab/serve-cleanup (2021-08-03) 13 commits
>  - fixup! {upload,receive}-pack tests: add --advertise-refs tests
>  - serve.[ch]: don't pass "struct strvec *keys" to commands
>  - upload-pack.c: convert to new serve.c "startup" config cb
>  - upload-pack: document and rename --advertise-refs
>  - {upload,receive}-pack tests: add --advertise-refs tests
>  - serve.[ch]: remove "serve_options", split up --advertise-refs code
>  - serve.c: move version line to advertise_capabilities()
>  - serve: add support for a "startup" git_config() callback
>  - serve.c: add call_{advertise,command}() indirection
>  - serve: use designated initializers
>  - transport: use designated initializers
>  - transport: rename "fetch" in transport_vtable to "fetch_refs"
>  - serve: mark has_capability() as static
>
>  Code clean-up around "git serve".

It seems the reception to the "config callback" part of this was rather
lukewarm. I'll re-roll to eject those changes and see if it's better
received.

> * hn/refs-errno-cleanup (2021-08-02) 7 commits
>  - refs: make errno output explicit for refs_resolve_ref_unsafe
>  - refs: explicitly return failure_errno from parse_loose_ref_contents
>  - refs: add failure_errno to refs_read_raw_ref() signature
>  - refs: make errno output explicit for read_raw_ref_fn
>  - refs/files-backend: stop setting errno from lock_ref_oid_basic
>  - refs: remove EINVAL errno output from specification of read_raw_ref_fn
>  - refs file backend: move raceproof_create_file() here
>  (this branch uses ab/refs-files-cleanup.)
>
>  Futz with the way 'errno' is relied on in the refs API to carry the
>  failure modes up the callchain.
>
>  Will merge to 'next'.

Yay!

> * ab/test-tool-cache-cleanup (2021-06-08) 4 commits
>  - read-cache perf: add a perf test for refresh_index()
>  - test-tool: migrate read-cache-again to parse_options()
>  - test-tool: migrate read-cache-perf to parse_options()
>  - test-tool: split up test-tool read-cache
>
>  Test code shuffling.
>
>  Waiting for reviews.

Been saying I'll re-roll this one, willdo....

> * ab/pack-objects-stdin (2021-07-09) 5 commits
>  - pack-objects.c: make use of REV_INFO_STDIN_LINE_PROCESS
>  - pack-objects.c: do stdin parsing via revision.c's API
>  - revision.[ch]: add a "handle_stdin_line" API
>  - revision.h: refactor "disable_stdin" and "read_from_stdin"
>  - upload-pack: run is_repository_shallow() before setup_revisions()
>
>  Introduce handle_stdin_line callback to revision API and uses it.

Still keeen to get this moving, if there's any takers for reviews...

> * ab/update-submitting-patches (2021-07-22) 2 commits
>   (merged to 'next' on 2021-07-30 at 9ae2de7f7a)
>  + SubmittingPatches: replace discussion of Travis with GitHub Actions
>  + SubmittingPatches: move discussion of Signed-off-by above "send"
>
>  Reorganize and update the SubmitingPatches document.
>
>  Will merge to 'master'.

Thanks...

> * ab/fsck-unexpected-type (2021-07-12) 21 commits
>  - fsck: report invalid object type-path combinations
>  - fsck: report invalid types recorded in objects
>  - object-store.h: move read_loose_object() below 'struct object_info'
>  - fsck: don't hard die on invalid object types
>  - object-file.c: return -2 on "header too long" in unpack_loose_header()
>  - object-file.c: return -1, not "status" from unpack_loose_header()
>  - object-file.c: guard against future bugs in loose_object_info()
>  - object-file.c: stop dying in parse_loose_header()
>  - object-file.c: split up ternary in parse_loose_header()
>  - object-file.c: simplify unpack_loose_short_header()
>  - object-file.c: add missing braces to loose_object_info()
>  - object-file.c: make parse_loose_header_extended() public
>  - object-file.c: don't set "typep" when returning non-zero
>  - cache.h: move object functions to object-store.h
>  - cat-file tests: test for current --allow-unknown-type behavior
>  - cat-file tests: add corrupt loose object test
>  - rev-list tests: test for behavior with invalid object types
>  - cat-file tests: test that --allow-unknown-type isn't on by default
>  - cat-file tests: test for missing object with -t and -s
>  - fsck tests: add test for fsck-ing an unknown type
>  - fsck tests: refactor one test to use a sub-repo
>
>  "git fsck" has been taught to report mismatch between expected and
>  actual types of an object better.
>
>  Needs review.

...ditto a call-out for reviewer interest...

> * es/config-based-hooks (2021-07-20) 9 commits
>  - hook: implement hookcmd.<name>.skip
>  - hook: teach 'hookcmd' config to alias hook scripts
>  - hook: allow out-of-repo 'git hook' invocations
>  - hook: include hooks from the config
>  - hook: allow running non-native hooks
>  - hook: treat hookdir hook specially
>  - hook: introduce "git hook list"
>  - hook: allow parallel hook execution
>  - hook: run a list of hooks instead
>
>  Discarded to give room for updated ab/config-based-hooks-base.

I understand that Emily is re-rolling this on top of my
ab/config-based-hooks-base.

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Derrick Stolee]

On 8/4/2021 3:03 AM, Junio C Hamano wrote:
...
> --------------------------------------------------
> [Stalled]
...
> * lh/systemd-timers (2021-07-02) 3 commits
>  - maintenance: add support for systemd timers on Linux
>  - maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
>  - cache.h: Introduce a generic "xdg_config_home_for(…)" function
> 
>  "git maintenance" scheduler learned to use systemd timers as a
>  possible backend.
> 
>  Waiting for reviews.

I just took another look at this series and see that there were a few
items that have yet to be addressed. CC'ing Lénaïc and reviewers to
see if those items will come in a v8. Here is a quick summary of my
understanding:

* There are some non-ASCII characters in a code comment that are a
  bit non-standard and could be replaced with ASCII representations.
  (nit, but if re-rolling already this might be worth doing.)

* There is some discussion about using string_list_split() instead
  of hand-rolling a string splitter. Discussion decided that we
  should _not_ use strbuf_split_buf(). It would be nice to later
  create a version of strvec_split() that takes an arbitrary 
  delimiter, but isn't necessary for this series.

* There was some discussion about using #ifdef to make certain
  logic be compiled in or not. This seems (to me) less important
  in the case of returning 0 or 1, but in the third patch there
  is a large set of logic that is only compiled on Linux, which
  seems like it should match the pattern of the other methods,
  if possible.

Again, this is just my drive-by summary, but hopefully it renews
work on this topic.

Thanks,
-Stolee

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>> * ab/test-columns (2021-08-02) 3 commits
>>  - test-lib.sh: use GIT_TEST_COLUMNS over COLUMNS
>>  - test-lib-functions.sh: add a test_with_columns function
>>  - test-lib-functions.sh: rename test_must_fail_acceptable()
>
> We're going to need this or another solution to the v2.33.0-rc0
> regression in c49a177beca (test-lib.sh: set COLUMNS=80 for --verbose
> repeatability, 2021-06-29) before the final v2.33.0.

Yeah, this is ugly but not so ugly as the original version ;-)
Thanks for prodding.

>> * ab/lib-subtest (2021-07-21) 10 commits
>> * ab/only-single-progress-at-once (2021-07-23) 8 commits
>
> Thanks, it would be great to have both of these move soon after the
> release. I think the chances of unexpected breakage here are minimal
> given their nature.

Have they seen enough eyeballs already to gain support?

>> * ab/progress-users-adjust-counters (2021-07-23) 3 commits
> ...
> I think that what SZEDER had to say in
> https://lore.kernel.org/git/20210802220506.GF23408@szeder.dev/ should be
> enough to clear this to proceed forward.

OK, thanks.

>> * ab/make-tags-cleanup (2021-07-22) 5 commits
> ...
>     As long as we have no user of a -q flag ever, what's the point of
>     forever carrying the "rm foo*" when we know it's foo.out, just
>     because a future Makefile change might add foo.blah?

That is not a forward-looking thinking.  A cheap and easy way to
future-proof was suggested by a reviewer.  Why ignore repeated
suggestions?

>> * ab/serve-cleanup (2021-08-03) 13 commits
>>  - fixup! {upload,receive}-pack tests: add --advertise-refs tests
>>  - serve.[ch]: don't pass "struct strvec *keys" to commands
>>  - upload-pack.c: convert to new serve.c "startup" config cb
>>  - upload-pack: document and rename --advertise-refs
>>  - {upload,receive}-pack tests: add --advertise-refs tests
>>  - serve.[ch]: remove "serve_options", split up --advertise-refs code
>>  - serve.c: move version line to advertise_capabilities()
>>  - serve: add support for a "startup" git_config() callback
>>  - serve.c: add call_{advertise,command}() indirection
>>  - serve: use designated initializers
>>  - transport: use designated initializers
>>  - transport: rename "fetch" in transport_vtable to "fetch_refs"
>>  - serve: mark has_capability() as static
>>
>>  Code clean-up around "git serve".
>
> It seems the reception to the "config callback" part of this was rather
> lukewarm. I'll re-roll to eject those changes and see if it's better
> received.

I'll eject this topic and other "will reroll" topics until the
final, with the hope to restart post release, then.  Our focus ought
to be to stabilize the upcoming release and having more topics or
topics that won't be near 'next' updated adds to distraction.

Thanks.

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>> * ab/test-columns (2021-08-02) 3 commits
>>  - test-lib.sh: use GIT_TEST_COLUMNS over COLUMNS
>>  - test-lib-functions.sh: add a test_with_columns function
>>  - test-lib-functions.sh: rename test_must_fail_acceptable()
>
> We're going to need this or another solution to the v2.33.0-rc0
> regression in c49a177beca (test-lib.sh: set COLUMNS=80 for --verbose
> repeatability, 2021-06-29) before the final v2.33.0.

Just a question.  Is that true?  Wouldn't a system that needs these
on top of c49a177beca already break the tests without c49a177beca?

IOW, is c49a177beca truly a "regression", or is it merely "a half
solution that solves for most but not all platforms"?

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[SZEDER Gábor]

On Wed, Aug 04, 2021 at 12:22:44PM +0200, Ævar Arnfjörð Bjarmason wrote:
> > * ab/progress-users-adjust-counters (2021-07-23) 3 commits
> >  - entry: show finer-grained counter in "Filtering content" progress line
> >  - midx: don't provide a total for QSORT() progress
> >  - commit-graph: fix bogus counter in "Scanning merged commits" progress line
> >
> >  The code to show progress indicator in a few codepaths did not
> >  cover between 0-100%, which has been corrected.
> >
> >  Waiting for a clarification.
> >  cf. <xmqqbl6slmer.fsf@gitster.g>
> 
> I think that what SZEDER had to say in
> https://lore.kernel.org/git/20210802220506.GF23408@szeder.dev/ should be
> enough to clear this to proceed forward.

Note that I also pointed out errors in the commit messages that should
be addressed before this patch series advances; see:

  https://public-inbox.org/git/20210802210759.GD23408@szeder.dev/
  https://public-inbox.org/git/20210802214827.GE23408@szeder.dev/

But none of these patches fixes regressions introduced in this release
cycle, so it's not urgent.


> I.e. this topic is missing his subsequent
> https://lore.kernel.org/git/20210620200303.2328957-7-szeder.dev@gmail.com/
> patch.
> 
> But as he notes we only encounter this in one of our tests (and are
> unlikely to in the wild).
> 
> Between this topic and my ab/only-single-progress-at-once I think we're
> better off getting those basic fixes in before proceeding with any
> tricky additions of BUG() and other assertions, per what I noted in
> https://public-inbox.org/git/cover-00.25-00000000000-20210623T155626Z-avarab@gmail.com/;
> i.e. that some of those assertions were themselves buggy.

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Ævar Arnfjörð Bjarmason]

On Wed, Aug 04 2021, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>>> * ab/test-columns (2021-08-02) 3 commits
>>>  - test-lib.sh: use GIT_TEST_COLUMNS over COLUMNS
>>>  - test-lib-functions.sh: add a test_with_columns function
>>>  - test-lib-functions.sh: rename test_must_fail_acceptable()
>>
>> We're going to need this or another solution to the v2.33.0-rc0
>> regression in c49a177beca (test-lib.sh: set COLUMNS=80 for --verbose
>> repeatability, 2021-06-29) before the final v2.33.0.
>
> Just a question.  Is that true?  Wouldn't a system that needs these
> on top of c49a177beca already break the tests without c49a177beca?
>
> IOW, is c49a177beca truly a "regression", or is it merely "a half
> solution that solves for most but not all platforms"?

Yes, because with c49a177beca your tests only break if you use the
--verbose option, i.e. if your stderr is connected to a terminal. I.e.:

    ./t0500-progress-display.sh --verbose

So in practice it mostly affects git developers who run with --verbose,
but probably nobody doing a build in the wild.

With c49a177beca they break on e.g.:

    /bin/bash ./t0500-progress-display.sh

If your bash is recent enough, so "make test" if you're on a system with
a recent bash whose /bin/sh is /bin/bash.

This is because post-c49a177beca we don't "unset" COLUMNS anymore, which
bash takes as license to update it.

So we really do need that series in before the release to avoid that
common annoyance, a revert of c49a177beca is also possible, i.e. it
would still leave things broken under --verbose, but that breakage is
rare and existed before v2.33.

I think given the triviality of the fixes and that the cause is
well-understood it makes sense to go forward in this case.

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Ævar Arnfjörð Bjarmason]

On Wed, Aug 04 2021, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>>> * ab/make-tags-cleanup (2021-07-22) 5 commits
>> ...
>>     As long as we have no user of a -q flag ever, what's the point of
>>     forever carrying the "rm foo*" when we know it's foo.out, just
>>     because a future Makefile change might add foo.blah?
>
> That is not a forward-looking thinking.  A cheap and easy way to
> future-proof was suggested by a reviewer.  Why ignore repeated
> suggestions?

Raising it repeatedly is an odd way to ignore it :)

But sure, all I was looking for was some reply to
https://lore.kernel.org/git/87k0lit57x.fsf@evledraar.gmail.com/; if you
want to keep it sure, I'll keep it. Will re-roll.

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> This is because post-c49a177beca we don't "unset" COLUMNS anymore, which
> bash takes as license to update it.

Ah, OK.

Thanks.

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Jeff King]

On Wed, Aug 04, 2021 at 10:22:41AM -0400, Derrick Stolee wrote:

> > * lh/systemd-timers (2021-07-02) 3 commits
> >  - maintenance: add support for systemd timers on Linux
> >  - maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
> >  - cache.h: Introduce a generic "xdg_config_home_for(…)" function
> > 
> >  "git maintenance" scheduler learned to use systemd timers as a
> >  possible backend.
> > 
> >  Waiting for reviews.
> 
> I just took another look at this series and see that there were a few
> items that have yet to be addressed. CC'ing Lénaïc and reviewers to
> see if those items will come in a v8. Here is a quick summary of my
> understanding:
> 
> * There are some non-ASCII characters in a code comment that are a
>   bit non-standard and could be replaced with ASCII representations.
>   (nit, but if re-rolling already this might be worth doing.)
> 
> * There is some discussion about using string_list_split() instead
>   of hand-rolling a string splitter. Discussion decided that we
>   should _not_ use strbuf_split_buf(). It would be nice to later
>   create a version of strvec_split() that takes an arbitrary 
>   delimiter, but isn't necessary for this series.

I think this strvec part was the only thing I had commented on
previously. I agree with your summary here (especially that we can punt
on refactoring strvec_split() for this series).

-Peff

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[SZEDER Gábor]

On Wed, Aug 04, 2021 at 09:53:02PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> On Wed, Aug 04 2021, Junio C Hamano wrote:
> 
> > Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> >
> >>> * ab/test-columns (2021-08-02) 3 commits
> >>>  - test-lib.sh: use GIT_TEST_COLUMNS over COLUMNS
> >>>  - test-lib-functions.sh: add a test_with_columns function
> >>>  - test-lib-functions.sh: rename test_must_fail_acceptable()
> >>
> >> We're going to need this or another solution to the v2.33.0-rc0
> >> regression in c49a177beca (test-lib.sh: set COLUMNS=80 for --verbose
> >> repeatability, 2021-06-29) before the final v2.33.0.
> >
> > Just a question.  Is that true?  Wouldn't a system that needs these
> > on top of c49a177beca already break the tests without c49a177beca?
> >
> > IOW, is c49a177beca truly a "regression", or is it merely "a half
> > solution that solves for most but not all platforms"?
> 
> Yes, because with c49a177beca your tests only break if you use the
> --verbose option, i.e. if your stderr is connected to a terminal. I.e.:
> 
>     ./t0500-progress-display.sh --verbose
> 
> So in practice it mostly affects git developers who run with --verbose,
> but probably nobody doing a build in the wild.
> 
> With c49a177beca they break on e.g.:
> 
>     /bin/bash ./t0500-progress-display.sh
> 
> If your bash is recent enough, so "make test" if you're on a system with
> a recent bash whose /bin/sh is /bin/bash.
> 
> This is because post-c49a177beca we don't "unset" COLUMNS anymore, which
> bash takes as license to update it.
> 
> So we really do need that series in before the release to avoid that
> common annoyance, a revert of c49a177beca is also possible, i.e. it
> would still leave things broken under --verbose, but that breakage is
> rare and existed before v2.33.
> 
> I think given the triviality of the fixes and that the cause is
> well-understood it makes sense to go forward in this case.

On one hand, there is feedback to be addressed:

  https://public-inbox.org/git/20210802171429.GC23408@szeder.dev/

OTOH, setting the checkwinsize is the truly trivial, minimal, reliable
and uncontroversial fix for this issue, and IMO that should go into
the next release.

This fix in this patch series is not trivial: it introduces yet
another GIT_TEST variable and a helper function that developers will
have to remember to use in the future.  Worse, this means that despite
aiming for future proofing I can't consider this approach future
proof, because it's easy to forget about such a rarely used test
helper function, and if anyone introduces yet another test setting
COLUMNS, then that will be prone to similar failures when run with
Bash.

I don't think that in this case we should aim for future proofing when
the cost is the additional cognitive load of yet another helper
function.  I would instead prefer to go with the really trivial fix
for now and wait whether this issue pops up again with other shell or
terminal, hoping that this issue is a "one-hit-wonder" [1] and it
won't happen ever again.


[1] "one-hit-wonder" as in:
    https://en.wikipedia.org/wiki/Bloom_filter#Cache_filtering

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[SZEDER Gábor]

On Wed, Aug 04, 2021 at 11:28:25PM +0200, SZEDER Gábor wrote:
> On Wed, Aug 04, 2021 at 09:53:02PM +0200, Ævar Arnfjörð Bjarmason wrote:
> > 
> > On Wed, Aug 04 2021, Junio C Hamano wrote:
> > 
> > > Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> > >
> > >>> * ab/test-columns (2021-08-02) 3 commits
> > >>>  - test-lib.sh: use GIT_TEST_COLUMNS over COLUMNS
> > >>>  - test-lib-functions.sh: add a test_with_columns function
> > >>>  - test-lib-functions.sh: rename test_must_fail_acceptable()
> > >>
> > >> We're going to need this or another solution to the v2.33.0-rc0
> > >> regression in c49a177beca (test-lib.sh: set COLUMNS=80 for --verbose
> > >> repeatability, 2021-06-29) before the final v2.33.0.
> > >
> > > Just a question.  Is that true?  Wouldn't a system that needs these
> > > on top of c49a177beca already break the tests without c49a177beca?
> > >
> > > IOW, is c49a177beca truly a "regression", or is it merely "a half
> > > solution that solves for most but not all platforms"?
> > 
> > Yes, because with c49a177beca your tests only break if you use the
> > --verbose option, i.e. if your stderr is connected to a terminal. I.e.:
> > 
> >     ./t0500-progress-display.sh --verbose
> > 
> > So in practice it mostly affects git developers who run with --verbose,
> > but probably nobody doing a build in the wild.
> > 
> > With c49a177beca they break on e.g.:
> > 
> >     /bin/bash ./t0500-progress-display.sh
> > 
> > If your bash is recent enough, so "make test" if you're on a system with
> > a recent bash whose /bin/sh is /bin/bash.
> > 
> > This is because post-c49a177beca we don't "unset" COLUMNS anymore, which
> > bash takes as license to update it.
> > 
> > So we really do need that series in before the release to avoid that
> > common annoyance, a revert of c49a177beca is also possible, i.e. it
> > would still leave things broken under --verbose, but that breakage is
> > rare and existed before v2.33.
> > 
> > I think given the triviality of the fixes and that the cause is
> > well-understood it makes sense to go forward in this case.
> 
> On one hand, there is feedback to be addressed:
> 
>   https://public-inbox.org/git/20210802171429.GC23408@szeder.dev/
> 
> OTOH, setting the checkwinsize is the truly trivial, minimal, reliable

Oh, this should have been s/setting/turning off/

> and uncontroversial fix for this issue, and IMO that should go into
> the next release.
> 
> This fix in this patch series is not trivial: it introduces yet
> another GIT_TEST variable and a helper function that developers will
> have to remember to use in the future.  Worse, this means that despite
> aiming for future proofing I can't consider this approach future
> proof, because it's easy to forget about such a rarely used test
> helper function, and if anyone introduces yet another test setting
> COLUMNS, then that will be prone to similar failures when run with
> Bash.
> 
> I don't think that in this case we should aim for future proofing when
> the cost is the additional cognitive load of yet another helper
> function.  I would instead prefer to go with the really trivial fix
> for now and wait whether this issue pops up again with other shell or
> terminal, hoping that this issue is a "one-hit-wonder" [1] and it
> won't happen ever again.
> 
> 
> [1] "one-hit-wonder" as in:
>     https://en.wikipedia.org/wiki/Bloom_filter#Cache_filtering
> 

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Ævar Arnfjörð Bjarmason]

On Wed, Aug 04 2021, SZEDER Gábor wrote:

> On Wed, Aug 04, 2021 at 09:53:02PM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> On Wed, Aug 04 2021, Junio C Hamano wrote:
>> 
>> > Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>> >
>> >>> * ab/test-columns (2021-08-02) 3 commits
>> >>>  - test-lib.sh: use GIT_TEST_COLUMNS over COLUMNS
>> >>>  - test-lib-functions.sh: add a test_with_columns function
>> >>>  - test-lib-functions.sh: rename test_must_fail_acceptable()
>> >>
>> >> We're going to need this or another solution to the v2.33.0-rc0
>> >> regression in c49a177beca (test-lib.sh: set COLUMNS=80 for --verbose
>> >> repeatability, 2021-06-29) before the final v2.33.0.
>> >
>> > Just a question.  Is that true?  Wouldn't a system that needs these
>> > on top of c49a177beca already break the tests without c49a177beca?
>> >
>> > IOW, is c49a177beca truly a "regression", or is it merely "a half
>> > solution that solves for most but not all platforms"?
>> 
>> Yes, because with c49a177beca your tests only break if you use the
>> --verbose option, i.e. if your stderr is connected to a terminal. I.e.:
>> 
>>     ./t0500-progress-display.sh --verbose
>> 
>> So in practice it mostly affects git developers who run with --verbose,
>> but probably nobody doing a build in the wild.
>> 
>> With c49a177beca they break on e.g.:
>> 
>>     /bin/bash ./t0500-progress-display.sh
>> 
>> If your bash is recent enough, so "make test" if you're on a system with
>> a recent bash whose /bin/sh is /bin/bash.
>> 
>> This is because post-c49a177beca we don't "unset" COLUMNS anymore, which
>> bash takes as license to update it.
>> 
>> So we really do need that series in before the release to avoid that
>> common annoyance, a revert of c49a177beca is also possible, i.e. it
>> would still leave things broken under --verbose, but that breakage is
>> rare and existed before v2.33.
>> 
>> I think given the triviality of the fixes and that the cause is
>> well-understood it makes sense to go forward in this case.
>
> On one hand, there is feedback to be addressed:
>
>   https://public-inbox.org/git/20210802171429.GC23408@szeder.dev/

I read that a couple of days ago, but managed to forget about it. Sorry,
re-rolled with it addressed:
https://lore.kernel.org/git/cover-v3-0.3-00000000000-20210804T230335Z-avarab@gmail.com/

> OTOH, setting the checkwinsize is the truly trivial, minimal, reliable
> and uncontroversial fix for this issue, and IMO that should go into
> the next release.

Addressed below.

> This fix in this patch series is not trivial: it introduces yet
> another GIT_TEST variable and a helper function that developers will
> have to remember to use in the future.  Worse, this means that despite
> aiming for future proofing I can't consider this approach future
> proof, because it's easy to forget about such a rarely used test
> helper function, and if anyone introduces yet another test setting
> COLUMNS, then that will be prone to similar failures when run with
> Bash.

How would it be forgotten? If you introduce tests like the ones changed
in 1/3 of the series and expect git to pay attention to COLUMNS you'll
find that they won't work, because if you set COLUMNS=123 we won't take
it over the GIT_TEST_COLUMNS=80 set in test-lib.sh.

So it seems like a fairly easy-to-discover thing. You'll grep for that
setting and find test_with_columns().

Most tests (such as t0500-progress-display.sh) won't need to concern
themselves with the new helper, it's only for those tests that want to
check how git itself behaves with a custom COLUMNS value, as opposed to
just wanting consistent repeatable results.

In any case, I didn't think a helper was needed in this case, it's
something Junio requested:
https://lore.kernel.org/git/xmqqzgu7b6of.fsf@gitster.g/ ...

> I don't think that in this case we should aim for future proofing when
> the cost is the additional cognitive load of yet another helper
> function.  I would instead prefer to go with the really trivial fix
> for now and wait whether this issue pops up again with other shell or
> terminal, hoping that this issue is a "one-hit-wonder" [1] and it
> won't happen ever again.

...I'd be happy to remove the helper if Junio would take that version of
the patch; :)

But in the topic of the overall approach, I think it's worth
future-proofing here mainly because it's useful to be able to reliably
run "make test" on old commits for bisecting, which is a property we
mostly manage to uphold.

By narrowly targeting a fix at one specific shell's cleverness around
COLUMNS we'll leave open a window where we'll fail on other shells if
they introduce similar cleverness.

It hardly seems like a stretch that once bash starts doing that sort of
thing other shells might think to follow suit, and all have their own
non-standard way to turn it off.

You also didn't address the other rationale for it, namely that it's
also future-proofing us for submarine breakages in non-git programs
which'll understand the new COLUMNS=10, but not GIT_TEST_COLUMNS=80.

I.e. should our tests rely on their output, and those programs
themselves change how they treat e.g. COLUMNS v.s. TIOCGWINSZ any tests
relying on their output will change their behavior.

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Taylor Blau]

On Wed, Aug 04, 2021 at 12:03:02AM -0700, Junio C Hamano wrote:
> * tb/multi-pack-bitmaps (2021-07-27) 25 commits
>
>  The reachability bitmap file used to be generated only for a single
>  pack, but now we've learned to generate bitmaps for history that
>  span across multiple packfiles.
>
>  Comments?

Peff completed a thorough review of the most substantial patches. We
found a few issues, which I addressed in v3. I'd like to have him read
through the tests, although I expect there to be far fewer changes
there, so I would anticipate that v4 would be suitable for merging.

But let's definitely wait until post-release to move this along further,
since we should be dedicating our efforts to polishing the release until
then. Feel free to eject it in the meantime if it's more convenient

Thanks,
Taylor

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> How would it be forgotten? If you introduce tests like the ones changed
> in 1/3 of the series and expect git to pay attention to COLUMNS you'll
> find that they won't work, because if you set COLUMNS=123 we won't take
> it over the GIT_TEST_COLUMNS=80 set in test-lib.sh.
> ...
> ...I'd be happy to remove the helper if Junio would take that version of
> the patch; :)

FWIW, I didn't *request* it; the resulting test scripts that set and
unset both the standard COLUMNS and another test-only environment
variable looked typo-prone and hard to read, and that is why I
suggested to hide that behind a helper.

If we do not have to add a test-only enviroment variable at all, I
do not see the reason why we need a helper.

> By narrowly targeting a fix at one specific shell's cleverness around
> COLUMNS we'll leave open a window where we'll fail on other shells if
> they introduce similar cleverness.
>
> It hardly seems like a stretch that once bash starts doing that sort of
> thing other shells might think to follow suit, and all have their own
> non-standard way to turn it off.

Hmph. Wouldn't the same argument apply to the much simpler single
liner "shopt -u" solution?  When writing new tests, there is nothing
to remember, and a new shell that needs a different trick to defeat
the auto-COLUMNS would be detected quickly by running the tests in a
terminal whose width is different from 80, no?

> You also didn't address the other rationale for it, namely that it's
> also future-proofing us for submarine breakages in non-git programs
> which'll understand the new COLUMNS=10, but not GIT_TEST_COLUMNS=80.

Isn't that another downside of the approach you are advocating?

If we make Git rely on GIT_TEST_COLUMNS, we may honor it while
everybody else ignores it.  If we only have to deal with COLUMNS
like everybody else does, Git and other tools that are used in our
tests will be affected the same way by overly-clever shells, no?

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Ævar Arnfjörð Bjarmason]

On Wed, Aug 04 2021, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> [...]
>> By narrowly targeting a fix at one specific shell's cleverness around
>> COLUMNS we'll leave open a window where we'll fail on other shells if
>> they introduce similar cleverness.
>>
>> It hardly seems like a stretch that once bash starts doing that sort of
>> thing other shells might think to follow suit, and all have their own
>> non-standard way to turn it off.
>
> Hmph. Wouldn't the same argument apply to the much simpler single
> liner "shopt -u" solution?  When writing new tests, there is nothing
> to remember, and a new shell that needs a different trick to defeat
> the auto-COLUMNS would be detected quickly by running the tests in a
> terminal whose width is different from 80, no?

It's different in the "bisecting" case I mentioned. I.e. "shopt -u" is a
shell-specific solution, the approach I'm suggesting bypasses any sort
of shell-specific solutions, both current and future ones.


>> You also didn't address the other rationale for it, namely that it's
>> also future-proofing us for submarine breakages in non-git programs
>> which'll understand the new COLUMNS=10, but not GIT_TEST_COLUMNS=80.
>
> Isn't that another downside of the approach you are advocating?
>
> If we make Git rely on GIT_TEST_COLUMNS, we may honor it while
> everybody else ignores it.  If we only have to deal with COLUMNS
> like everybody else does, Git and other tools that are used in our
> tests will be affected the same way by overly-clever shells, no?

I think it's an upside. We know how git handles the combination of
GIT_TEST_COLUMNS, COLUMNS and TIOCGWINSZ. We have no idea how any
third-party program decides to behave.

So by intentionally spoiling COLUMNS=* and only having our tests pay
attention to GIT_TEST_COLUMNS we pretty much guarantee that we won't
grow some dependency on a non-git program's handling of COLUMNS, which
e.g. could differ from platform to platform.

I don't know about any such case, and spoiling COLUMNS didn't reveal
any. I just thought it was worthwhile to once-and-for-all get us away
from any sort of shell or 3rd party code cleverness around COLUMNS, as
opposed to a very narrow bugfix for just the issue we've spotted with
bash recently.

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[Phillip Wood]

On 04/08/2021 15:22, Derrick Stolee wrote:
> On 8/4/2021 3:03 AM, Junio C Hamano wrote:
> ...
>> --------------------------------------------------
>> [Stalled]
> ...
>> * lh/systemd-timers (2021-07-02) 3 commits
>>   - maintenance: add support for systemd timers on Linux
>>   - maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
>>   - cache.h: Introduce a generic "xdg_config_home_for(…)" function
>>
>>   "git maintenance" scheduler learned to use systemd timers as a
>>   possible backend.
>>
>>   Waiting for reviews.
> 
> I just took another look at this series and see that there were a few
> items that have yet to be addressed. CC'ing Lénaïc and reviewers to
> see if those items will come in a v8. Here is a quick summary of my
> understanding:
> 
> * There are some non-ASCII characters in a code comment that are a
>    bit non-standard and could be replaced with ASCII representations.
>    (nit, but if re-rolling already this might be worth doing.)
> 
> * There is some discussion about using string_list_split() instead
>    of hand-rolling a string splitter. Discussion decided that we
>    should _not_ use strbuf_split_buf(). It would be nice to later
>    create a version of strvec_split() that takes an arbitrary
>    delimiter, but isn't necessary for this series.
> 
> * There was some discussion about using #ifdef to make certain
>    logic be compiled in or not. This seems (to me) less important
>    in the case of returning 0 or 1, but in the third patch there
>    is a large set of logic that is only compiled on Linux, which
>    seems like it should match the pattern of the other methods,
>    if possible.
> 
> Again, this is just my drive-by summary, but hopefully it renews
> work on this topic.

I think that is a fair summary, I'd be happy to see us take this series 
as-is, the code is fine and any small warts could be cleaned up in the 
future. I think Eric expressed a similar opinion in [1]. Lénaïc has been 
responsive to fixing bugs as well as taking on board other comments, I 
think we've reached the stage where it is not reasonable to keep asking 
for small changes.

Best Wishes

Phillip

[1] 
https://lore.kernel.org/git/CAPig+cT-x4_YuxmmoFw62jFqKRFJrS_UkyNOkdQz9-Gwec3QCQ@mail.gmail.com

> Thanks,
> -Stolee
> 

Re: What's cooking in git.git (Aug 2021, #02; Tue, 3)

[SZEDER Gábor]

On Thu, Aug 05, 2021 at 01:06:55AM +0200, Ævar Arnfjörð Bjarmason wrote:
> > I don't think that in this case we should aim for future proofing when
> > the cost is the additional cognitive load of yet another helper
> > function.  I would instead prefer to go with the really trivial fix
> > for now and wait whether this issue pops up again with other shell or
> > terminal, hoping that this issue is a "one-hit-wonder" [1] and it
> > won't happen ever again.
> 
> ...I'd be happy to remove the helper if Junio would take that version of
> the patch; :)

I have to agree with Junio here: setting 'GIT_TEST_COLUMNS=
COLUMNS=80' for each affected command like the first patch did was
just too ugly.

> But in the topic of the overall approach, I think it's worth
> future-proofing here mainly because it's useful to be able to reliably
> run "make test" on old commits for bisecting, which is a property we
> mostly manage to uphold.
> 
> By narrowly targeting a fix at one specific shell's cleverness around
> COLUMNS we'll leave open a window where we'll fail on other shells if
> they introduce similar cleverness.
> 
> It hardly seems like a stretch that once bash starts doing that sort of
> thing other shells might think to follow suit, and all have their own
> non-standard way to turn it off.

Bash has a lot of non-standard features that other shells decided to
avoid, even when they are really useful and unconroversial.  To me it
does seems like a stretch that other shells will follow suit with
COLUMNS, when our first reaction to Bash's changed behavior was that
it's buggy.

> You also didn't address the other rationale for it, namely that it's
> also future-proofing us for submarine breakages in non-git programs
> which'll understand the new COLUMNS=10, but not GIT_TEST_COLUMNS=80.
> 
> I.e. should our tests rely on their output, and those programs
> themselves change how they treat e.g. COLUMNS v.s. TIOCGWINSZ any tests
> relying on their output will change their behavior.

I occasionally scan through the test logs looking for any new
suspicious error messages from non-git commands and shell builtins, and
to find those error messages I had to collect the list of non-git
commands [1] executed in the test suite:

  awk|basename|cat|chmod|cmp|comm|curl|cut|date|dd|diff|dirname|egrep|env|envsubst|expr|fgrep|find|getfacl|gettext|grep|gunzip|gzip|head|hostname|iconv|id|jgit|kill|ln|mkdir|mkfifo|mktemp|mv|perl|readlink|sed|setfacl|sleep|sort|tail|tar|touch|tr|uname|uniq|unzip|wc|xargs|xmllint|zipinfo|

Most of these are fairly low-level system commands, where supporting
COLUMNS in any way wouldn't make sense at all, or, worse, would be a
serious regression.  Commands like 'curl', 'gunzip' or 'gzip' could
show a progress bar matching the width of the terminal, but our tests
should never rely on something like that, ever.  Then there is 'jgit'
which could support COLUMNS for output intended to the user, just like
git does, but, again, our tests should never rely on that.


[1] It's an incomplete list, e.g. 'cp', 'ls', and 'rm' are
    intentionally left out, because they are too often used in cases
    when they can legitimately fail.