What's cooking in git.git (Apr 2014, #09; Tue, 29)

What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed with
'-' are only in 'pu' (proposed updates) while commits prefixed with
'+' are in 'next'.

The tip of the 'master' branch has passed v2.0.0-rc1.  Last minute
fixes to newly added code keep flowing in, which is good.  I've
picked up some topics that will not be part of the upcoming release
to 'pu' not to lose them, but I didn't have time to give them a deep
reading.

You can find the changes described here in the integration branches
of the repositories listed at

    http://git-blame.blogspot.com/p/git-public-repositories.html

--------------------------------------------------
[New Topics]

* cc/replace-edit (2014-04-29) 4 commits
 - replace: add --edit option
 - replace: factor object resolution out of replace_object
 - replace: use OPT_CMDMODE to handle modes
 - replace: refactor command-mode determination


* da/imap-send-use-credential-helper (2014-04-29) 1 commit
 - imap-send: use git-credential


* dk/blame-reorg (2014-04-28) 1 commit
 - blame: large-scale performance rewrite


* je/pager-do-not-recurse (2014-04-28) 1 commit
 - pager: do allow spawning pager recursively


* jk/commit-C-pick-empty (2014-04-28) 1 commit
 - commit: do not complain of empty messages from -C


* jk/utf8-switch-between-nfd-and-nfc (2014-04-29) 1 commit
 - t3910: show failure of core.precomposeunicode with decomposed filenames


* mt/send-email-cover-to-cc (2014-04-29) 2 commits
 - test/send-email: to-cover, cc-cover tests
 - git-send-email: two new options: to-cover, cc-cover


* nd/split-index (2014-04-29) 33 commits
 - SQUASH???
 - t1700: new tests for split-index mode
 - t2104: make sure split index mode is off for the version test
 - read-cache: force split index mode with GIT_TEST_SPLIT_INDEX
 - read-tree: note about dropping split-index mode or index version
 - read-tree: force split-index mode off on --index-output
 - rev-parse: add --shared-index-path to get shared index path
 - update-index --split-index: do not split if $GIT_DIR is read only
 - update-index: new options to enable/disable split index mode
 - split-index: strip pathname of on-disk replaced entries
 - split-index: do not invalidate cache-tree at read time
 - split-index: the reading part
 - split-index: the writing part
 - read-cache: mark updated entries for split index
 - read-cache: save deleted entries in split index
 - read-cache: mark new entries for split index
 - read-cache: split-index mode
 - read-cache: save index SHA-1 after reading
 - entry.c: update cache_changed if refresh_cache is set in checkout_entry()
 - cache-tree: mark istate->cache_changed on prime_cache_tree()
 - cache-tree: mark istate->cache_changed on cache tree update
 - cache-tree: mark istate->cache_changed on cache tree invalidation
 - unpack-trees: be specific what part of the index has changed
 - resolve-undo: be specific what part of the index has changed
 - update-index: be specific what part of the index has changed
 - read-cache: be specific what part of the index has changed
 - read-cache: be strict about "changed" in remove_marked_cache_entries()
 - read-cache: store in-memory flags in the first 12 bits of ce_flags
 - read-cache: relocate and unexport commit_locked_index()
 - read-cache: new API write_locked_index instead of write_index/write_cache
 - sequencer: do not update/refresh index if the lock cannot be held
 - ewah: delete unused ewah_read_mmap_native declaration
 - ewah: fix constness of ewah_read_mmap


* tl/relax-in-poll-emulation (2014-04-29) 1 commit
 - compat/poll: sleep 1 millisecond to avoid busy wait

--------------------------------------------------
[Stalled]

* tr/merge-recursive-index-only (2014-02-05) 3 commits
 - merge-recursive: -Xindex-only to leave worktree unchanged
 - merge-recursive: internal flag to avoid touching the worktree
 - merge-recursive: remove dead conditional in update_stages()
 (this branch is used by tr/remerge-diff.)

 Will hold.


* tr/remerge-diff (2014-02-26) 5 commits
 . log --remerge-diff: show what the conflict resolution changed
 . name-hash: allow dir hashing even when !ignore_case
 . merge-recursive: allow storing conflict hunks in index
 . revision: fold all merge diff variants into an enum merge_diff_mode
 . combine-diff: do not pass revs->dense_combined_merges redundantly
 (this branch uses tr/merge-recursive-index-only.)

 "log -p" output learns a new way to let users inspect a merge
 commit by showing the differences between the automerged result
 with conflicts the person who recorded the merge would have seen
 and the final conflict resolution that was recorded in the merge.

 Needs to be rebased, now kb/fast-hashmap topic is in.


* jk/makefile (2014-02-05) 16 commits
 - FIXUP
 - move LESS/LV pager environment to Makefile
 - Makefile: teach scripts to include make variables
 - FIXUP
 - Makefile: auto-build C strings from make variables
 - Makefile: drop *_SQ variables
 - FIXUP
 - Makefile: add c-quote helper function
 - Makefile: introduce sq function for shell-quoting
 - Makefile: always create files via make-var
 - Makefile: store GIT-* sentinel files in MAKE/
 - Makefile: prefer printf to echo for GIT-*
 - Makefile: use tempfile/mv strategy for GIT-*
 - Makefile: introduce make-var helper function
 - Makefile: fix git-instaweb dependency on gitweb
 - Makefile: drop USE_GETTEXT_SCHEME from GIT-CFLAGS
 (this branch is used by mm/pager-less-sans-S.)

 Simplify the Makefile rules and macros that exist primarily for
 quoting purposes, and make it easier to robustly express the
 dependency rules.

 Expecting a reroll.


* po/everyday-doc (2014-01-27) 1 commit
 - Make 'git help everyday' work

 This may make the said command to emit something, but the source is
 not meant to be formatted into a manual pages to begin with, and
 also its contents are a bit stale.  It may be a good first step in
 the right direction, but needs more work to at least get the
 mark-up right before public consumption.

 Will hold.


* jk/branch-at-publish-rebased (2014-01-17) 5 commits
 . t1507 (rev-parse-upstream): fix typo in test title
 . implement @{publish} shorthand
 . branch_get: provide per-branch pushremote pointers
 . branch_get: return early on error
 . sha1_name: refactor upstream_mark

 Give an easier access to the tracking branches from "other" side in
 a triangular workflow by introducing B@{publish} that works in a
 similar way to how B@{upstream} does.

 Meant to be used as a basis for whatever Ram wants to build on.

 Ejected from 'pu' to make room for fc/publish-vs-upstream topic.


* rb/merge-prepare-commit-msg-hook (2014-01-10) 4 commits
 - merge: drop unused arg from abort_commit method signature
 - merge: make prepare_to_commit responsible for write_merge_state
 - t7505: ensure cleanup after hook blocks merge
 - t7505: add missing &&

 Expose more merge states (e.g. $GIT_DIR/MERGE_MODE) to hooks that
 run during "git merge".  The log message stresses too much on one
 hook, prepare-commit-msg, but it would equally apply to other hooks
 like post-merge, I think.

 Waiting for a reroll.


* jl/submodule-recursive-checkout (2013-12-26) 5 commits
 - Teach checkout to recursively checkout submodules
 - submodule: teach unpack_trees() to update submodules
 - submodule: teach unpack_trees() to repopulate submodules
 - submodule: teach unpack_trees() to remove submodule contents
 - submodule: prepare for recursive checkout of submodules

 An RFCv2 exists ($gmane/241455) with sizable review comments.
 Expecting a reroll.


* jc/graph-post-root-gap (2013-12-30) 3 commits
 - WIP: document what we want at the end
 - graph: remove unused code a bit
 - graph: stuff the current commit into graph->columns[]

 This was primarily a RFH ($gmane/239580).


* np/pack-v4 (2013-09-18) 90 commits
 . packv4-parse.c: add tree offset caching
 . t1050: replace one instance of show-index with verify-pack
 . index-pack, pack-objects: allow creating .idx v2 with .pack v4
 . unpack-objects: decode v4 trees
 . unpack-objects: allow to save processed bytes to a buffer
 - ...

 Nico and Duy advancing the eternal vaporware pack-v4.  This is here
 primarily for wider distribution of the preview edition.

 Needs to be rebased, now the pack-bitmap series is in.


* tg/perf-lib-test-perf-cleanup (2013-09-19) 2 commits
 - perf-lib: add test_perf_cleanup target
 - perf-lib: split starting the test from the execution

 Add test_perf_cleanup shell function to the perf suite, that allows
 the script writers to define a test with a clean-up action.

 Will hold.


* jc/format-patch (2013-04-22) 2 commits
 - format-patch: --inline-single
 - format-patch: rename "no_inline" field

 A new option to send a single patch to the standard output to be
 appended at the bottom of a message.  I personally have no need for
 this, but it was easy enough to cobble together.  Tests, docs and
 stripping out more MIMEy stuff are left as exercises to interested
 parties.


* jc/show-branch (2014-03-24) 5 commits
 - show-branch: use commit slab to represent bitflags of arbitrary width
 - show-branch.c: remove "all_mask"
 - show-branch.c: abstract out "flags" operation
 - show-branch.c: lift all_mask/all_revs to a global static
 - show-branch.c: update comment style

 Waiting for the final step to lift the hard-limit before sending it out.

--------------------------------------------------
[Cooking]

* bc/blame-crlf-test (2014-04-28) 1 commit
 - blame: correctly handle files regardless of autocrlf.


* rs/ref-transaction (2014-04-29) 27 commits
 - refs.c: make lock_ref_sha1 static
 - refs.c: make write_ref_sha1 static
 - walker.c: use ref transaction for ref updates
 - fast-import.c: use a ref transaction when dumping tags
 - receive-pack.c: use a reference transaction for updating the refs
 - fetch.c: use a single ref transaction for all ref updates
 - fetch.c: change s_update_ref to use a ref transaction
 - fetch.c: clear errno before calling functions that might set it
 - refs.c: ref_transaction_commit should not free the transaction
 - refs.c: free the transaction before returning when number of updates is 0
 - refs.c: change update_ref to use a transaction
 - branch.c: use ref transaction for all ref updates
 - fast-import.c: change update_branch to use ref transactions
 - sequencer.c: use ref transactions for all ref updates
 - commit.c: use ref transactions for updates
 - replace.c: use the ref transaction functions for updates
 - tag.c: use ref transactions when doing updates
 - refs.c: ref_transaction_delete to check for error and return status
 - refs.c: change ref_transaction_create to do error checking and return status
 - refs.c: change ref_transaction_update() to do error checking and return status
 - refs.c: remove the onerr argument to ref_transaction_commit
 - refs.c: make update_ref_write update a strbuf on failure
 - update-ref.c: log transaction error from the update_ref
 - refs.c: make ref_update_reject_duplicates take a strbuf argument for errors
 - refs.c: add a strbuf argument to ref_transaction_commit for error logging
 - refs.c: allow passing NULL to ref_transaction_free
 - refs.c: constify the sha arguments for ref_transaction_create|delete|update
 (this branch uses mh/ref-transaction.)

 Updates most of the callsites to write_sha1_ref(), the low-level
 mechanism to update a ref, to use the ref-transaction API.

 Expecting a reroll.


* ep/shell-command-substitution (2014-04-29) 27 commits
 - t1050-large.sh: use the $( ... ) construct for command substitution
 - t1020-subdirectory.sh: use the $( ... ) construct for command substitution
 - t1004-read-tree-m-u-wf.sh: use the $( ... ) construct for command substitution
 - t1003-read-tree-prefix.sh: use the $( ... ) construct for command substitution
 - t1002-read-tree-m-u-2way.sh: use the $( ... ) construct for command substitution
 - t1001-read-tree-m-2way.sh: use the $( ... ) construct for command substitution
 - t1000-read-tree-m-3way.sh: use the $( ... ) construct for command substitution
 - t0300-credentials.sh: use the $( ... ) construct for command substitution
 - t0030-stripspace.sh: use the $( ... ) construct for command substitution
 - t0026-eol-config.sh: use the $( ... ) construct for command substitution
 - t0025-crlf-auto.sh: use the $( ... ) construct for command substitution
 - t0020-crlf.sh: use the $( ... ) construct for command substitution
 - t0010-racy-git.sh: use the $( ... ) construct for command substitution
 - t0001-init.sh: use the $( ... ) construct for command substitution
 - p5302-pack-index.sh: use the $( ... ) construct for command substitution
 - lib-gpg.sh: use the $( ... ) construct for command substitution
 - lib-cvs.sh: use the $( ... ) construct for command substitution
 - lib-credential.sh: use the $( ... ) construct for command substitution
 - git-web--browse.sh: use the $( ... ) construct for command substitution
 - git-stash.sh: use the $( ... ) construct for command substitution
 - git-rebase.sh: use the $( ... ) construct for command substitution
 - git-rebase--merge.sh: use the $( ... ) construct for command substitution
 - git-pull.sh: use the $( ... ) construct for command substitution
 - appp.sh: use the $( ... ) construct for command substitution
 - t7900-subtree.sh: use the $( ... ) construct for command substitution
 - test-gitmw-lib.sh: use the $( ... ) construct for command substitution
 - t9365-continuing-queries.sh: use the $( ... ) construct for command substitution

 Adjust shell scripts to use $(cmd) instead of `cmd`.

 Will merge to 'next' and keep it there for the remainder of the cycle.


* ib/test-selectively-run (2014-04-23) 3 commits
 - test-lib: '--run' to run only specific tests
 - test-lib: tests skipped by GIT_SKIP_TESTS say so
 - test-lib: Document short options in t/README

 Allow specifying only certain individual test pieces to be run
 using a range notation (e.g. "t1234-test.sh --run='1-4 6 8 9-'").


* mm/mediawiki-encoding-fix (2014-04-23) 2 commits
 - git-remote-mediawiki: fix encoding issue for UTF-8 media files
 - git-remote-mediawiki: allow stop/start-ing the test server

 Will merge to 'next' and keep it there for the remainder of the cycle.


* mw/symlinks (2014-04-24) 1 commit
  (merged to 'next' on 2014-04-25 at 20b2af6)
 + setup: fix windows path buffer over-stepping

 A finishing touch fix to a new change already in 'master'.

 Will merge to 'master' by -rc2.


* sk/tag-contains-wo-recursion (2014-04-25) 1 commit
  (merged to 'next' on 2014-04-25 at f320750)
 + git tag --contains: avoid stack overflow

 Will keep in 'next' for the remainder of the cycle.


* fc/remote-helpers-hg-bzr-graduation (2014-04-29) 11 commits
 - remote-hg: trivial cleanups
 - remote-hg: make sure we omit multiple heads
 - git-remote-hg: use internal clone's hgrc
 - t: remote-hg: split into setup test
 - remote-hg: properly detect missing contexts
 - remote-{hg,bzr}: store marks only on success
 - remote-hg: update to 'public' phase when pushing
 - remote-hg: fix parsing of custom committer
  (merged to 'next' on 2014-04-22 at fed170a)
 + remote-helpers: move tests out of contrib
 + remote-helpers: move out of contrib
 + remote-helpers: squelch python import exceptions

 Move remote-hg and remote-bzr out of contrib/.  There were some
 suggestions on the follow-up fix patches still not in 'next', which
 may result in a reroll.

 Will merge to 'next' and keep it there for the remainder of the
 cycle.


* fc/remote-helper-refmap (2014-04-21) 8 commits
  (merged to 'next' on 2014-04-22 at fb5a4c2)
 + transport-helper: remove unnecessary strbuf resets
 + transport-helper: add support to delete branches
 + fast-export: add support to delete refs
 + fast-import: add support to delete refs
 + transport-helper: add support to push symbolic refs
 + transport-helper: add support for old:new refspec
 + fast-export: add new --refspec option
 + fast-export: improve argument parsing

 Allow remote-helper/fast-import based transport to rename the refs
 while transferring the history.

 Will keep in 'next' for the remainder of the cycle.


* jk/external-diff-use-argv-array (2014-04-21) 5 commits
  (merged to 'next' on 2014-04-22 at e6d92d7)
 + run_external_diff: refactor cmdline setup logic
 + run_external_diff: hoist common bits out of conditional
 + run_external_diff: drop fflush(NULL)
 + run_external_diff: clean up error handling
 + run_external_diff: use an argv_array for the environment

 Code clean-up (and a bugfix which has been merged for 2.0).

 Will keep in 'next' for the remainder of the cycle.


* jx/blame-align-relative-time (2014-04-23) 2 commits
  (merged to 'next' on 2014-04-23 at 858df39)
 + blame: dynamic blame_date_width for different locales
 + blame: fix broken time_buf paddings in relative timestamp

 "git blame" miscounted number of columns needed to show localized
 timestamps, resulting in jaggy left-side-edge of the source code
 lines in its output.

 Will keep in 'next' for the remainder of the cycle.


* fc/merge-default-to-upstream (2014-04-22) 1 commit
  (merged to 'next' on 2014-04-22 at 4f98483)
 + merge: enable defaulttoupstream by default

 "git merge" without argument, even when there is an upstream
 defined for the current branch, refused to run until
 merge.defaultToUpstream is set to true. Flip the default of that
 configuration variable to true.

 Will keep in 'next' for the remainder of the cycle.


* fc/mergetool-prompt (2014-04-24) 2 commits
 - mergetool: document the default for --[no-]prompt
  (merged to 'next' on 2014-04-22 at dcaec94)
 + mergetool: run prompt only if guessed tool

 mergetool.prompt used to default to 'true', always causing a confirmation
 "do you really want to run the tool on this path" to be shown.

 Among the two purposes the prompt serves, ignore the use case to
 confirm that the user wants to view particular path with the named
 tool, and make the prompt only to confirm the choice of the tool
 made by autodetection and defaulting.  For those who configured the
 tool explicitly, the prompt shown for the latter purpose is simply
 annoying.

 Strictly speaking, this is a backward incompatible change and the
 users need to explicitly set the variable to 'true' if they want to
 resurrect the now-ignored use case.

 Will merge to 'next' and keep it there for the remainder of the cycle.


* fc/mergetools-vimdiff3 (2014-04-22) 1 commit
  (merged to 'next' on 2014-04-22 at d843e75)
 + mergetools: add vimdiff3 mode

 Will keep in 'next' for the remainder of the cycle.


* km/git-svn-workaround-older-getopt-long (2014-04-23) 1 commit
  (merged to 'next' on 2014-04-23 at 3f35586)
 + t9117: use --prefix "" instead of --prefix=""

 Will merge to 'master' by -rc2.


* lr/git-run-setup-gently (2014-04-22) 1 commit
  (merged to 'next' on 2014-04-22 at 5c2523f)
 + git.c: treat RUN_SETUP_GENTLY and RUN_SETUP as mutually exclusive

 Will keep in 'next' for the remainder of the cycle.


* mk/doc-git-gui-display-untracked (2014-04-21) 1 commit
  (merged to 'next' on 2014-04-22 at 385d39a)
 + Documentation: git-gui: describe gui.displayuntracked

 Will merge to 'master' by -rc2.


* rh/prompt-pcmode-avoid-eval-on-refname (2014-04-22) 1 commit
  (merged to 'next' on 2014-04-22 at 3a1506f)
 + git-prompt.sh: don't put unsanitized branch names in $PS1

 Will merge to 'master' by -rc2.


* fc/publish-vs-upstream (2014-04-21) 8 commits
 - sha1_name: add support for @{publish} marks
 - sha1_name: simplify track finding
 - sha1_name: cleanup interpret_branch_name()
 - branch: display publish branch
 - push: add --set-publish option
 - branch: add --set-publish-to option
 - Add concept of 'publish' branch
 - t5516 (fetch-push): fix test restoration

 Add branch@{publish}; it seems that this is somewhat different from
 Ram and Peff started working on.  There were many discussion
 messages going back and forth but it does not appear that the
 design issues have been worked out among participants yet.


* jl/git-gui-show-added-submodule-changes (2014-04-15) 1 commit
 - git-gui: show staged submodules regardless of ignore config

 Tentatively queued what I expect to receive via Pat Thoyts.


* jl/gitk-show-added-submodule-changes (2014-04-15) 3 commits
 - gitk: show staged submodules regardless of ignore config
 - gitk: Merge branch 'new' of https://github.com/vnwildman/gitk
 - l10n: Init Vietnamese translation

 Tentatively queued what I expect to receive via Paul Mackerras.


* bg/rebase-off-of-previous-branch (2014-04-16) 1 commit
 - git-rebase: print name of rev when using shorthand

 Teach "git rebase -" to report the concrete name of the branch
 (i.e. the previous one).

 But it stops short and does not do the same for "git rebase @{-1}".
 Expecting a reroll.


* ef/send-email-absolute-path-to-the-command (2014-04-23) 2 commits
  (merged to 'next' on 2014-04-23 at a657e5e)
 + send-email: windows drive prefix (e.g. C:) appears only at the beginning
  (merged to 'next' on 2014-04-21 at 43bebb5)
 + send-email: recognize absolute path on Windows

 Will keep in 'next' for the remainder of the cycle.


* jh/submodule-tests (2014-04-17) 1 commit
 - t7410: 210 tests for various 'git submodule update' scenarios


* rs/ref-update-check-errors-early (2014-04-17) 2 commits
  (merged to 'next' on 2014-04-21 at acc62aa)
 + commit.c: check for lock error and return early
 + sequencer.c: check for lock failure and bail early in fast_forward_to

 Will keep in 'next' for the remainder of the cycle.


* sk/svn-parse-datestamp (2014-04-17) 1 commit
  (merged to 'next' on 2014-04-21 at 5ff519f)
 + SVN.pm::parse_svn_date: allow timestamps with a single-digit hour

 Will keep in 'next' for the remainder of the cycle.


* nd/index-pack-one-fd-per-thread (2014-04-16) 1 commit
  (merged to 'next' on 2014-04-16 at b38d5a9)
 + index-pack: work around thread-unsafe pread()

 Enable threaded index-pack on platforms without thread-unsafe
 pread() emulation.

 Will keep in 'next' for the remainder of the cycle.


* ym/fix-opportunistic-index-update-race (2014-04-10) 2 commits
  (merged to 'next' on 2014-04-16 at cb92f4f)
 + read-cache.c: verify index file before we opportunistically update it
 + wrapper.c: add xpread() similar to xread()

 Read-only operations such as "git status" that internally refreshes
 the index write out the refreshed index to the disk to optimize
 future accesses to the working tree, but this could race with a
 "read-write" operation that modify the index while it is running.
 Detect such a race and avoid overwriting the index.

 Duy raised a good point that we may need to do the same for the
 normal writeout codepath, not just the "opportunistic" update
 codepath.  While that is true, nobody sane would be running two
 simultaneous operations that are clearly write-oriented competing
 with each other against the same index file.  So in that sense that
 can be done as a less urgent follow-up for this topic.

 Will keep in 'next' for the remainder of the cycle.


* jl/status-added-submodule-is-never-ignored (2014-04-07) 2 commits
 - commit -m: commit staged submodules regardless of ignore config
 - status/commit: show staged submodules regardless of ignore config

 There also are a few patches Ronald Weiss and Jens are working on
 polishing around this topic, and a patch from Jens each for gitk
 and git-gui.

 Waiting for the dust to settle until picking them up all.


* mh/lockfile (2014-04-15) 25 commits
 - trim_last_path_elm(): replace last_path_elm()
 - resolve_symlink(): take a strbuf parameter
 - resolve_symlink(): use a strbuf for internal scratch space
 - change lock_file::filename into a strbuf
 - commit_lock_file(): use a strbuf to manage temporary space
 - try_merge_strategy(): use a statically-allocated lock_file object
 - try_merge_strategy(): remove redundant lock_file allocation
 - struct lock_file: declare some fields volatile
 - lockfile: avoid transitory invalid states
 - commit_lock_file(): die() if called for unlocked lockfile object
 - commit_lock_file(): inline temporary variable
 - remove_lock_file(): call rollback_lock_file()
 - lock_file(): exit early if lockfile cannot be opened
 - write_packed_entry_fn(): convert cb_data into a (const int *)
 - prepare_index(): declare return value to be (const char *)
 - delete_ref_loose(): don't muck around in the lock_file's filename
 - cache.h: define constants LOCK_SUFFIX and LOCK_SUFFIX_LEN
 - lockfile.c: document the various states of lock_file objects
 - lock_file(): always add lock_file object to lock_file_list
 - hold_lock_file_for_append(): release lock on errors
 - lockfile: unlock file if lockfile permissions cannot be adjusted
 - rollback_lock_file(): set fd to -1
 - rollback_lock_file(): do not clear filename redundantly
 - api-lockfile: expand the documentation
 - unable_to_lock_die(): rename function from unable_to_lock_index_die()

 Refactor and fix corner-case bugs in the lockfile API, all looked
 sensible.

 Expecting a reroll.


* mt/patch-id-stable (2014-04-29) 5 commits
 - t4204-patch-id.sh: default is now stable
 - patch-id: change default to stable
 - patch-id-test: test stable and unstable behaviour
 - test: add test_write_lines helper
 - patch-id: make it stable against hunk reordering

 Introduce a new way to compute patch-id for a patch that is not
 affected by the order of the paths that appear in the input.

 Will merge to 'next' and keep it there for the remainder of the cycle.


* mh/ref-transaction (2014-04-07) 27 commits
  (merged to 'next' on 2014-04-16 at a99f84d)
 + ref_transaction_commit(): work with transaction->updates in place
 + struct ref_update: add a type field
 + struct ref_update: add a lock field
 + ref_transaction_commit(): simplify code using temporary variables
 + struct ref_update: store refname as a FLEX_ARRAY
 + struct ref_update: rename field "ref_name" to "refname"
 + refs: remove API function update_refs()
 + update-ref --stdin: reimplement using reference transactions
 + refs: add a concept of a reference transaction
 + update-ref --stdin: harmonize error messages
 + update-ref --stdin: improve the error message for unexpected EOF
 + t1400: test one mistake at a time
 + update-ref --stdin -z: deprecate interpreting the empty string as zeros
 + update-ref.c: extract a new function, parse_next_sha1()
 + t1400: test that stdin -z update treats empty <newvalue> as zeros
 + update-ref --stdin: simplify error messages for missing oldvalues
 + update-ref --stdin: make error messages more consistent
 + update-ref --stdin: improve error messages for invalid values
 + update-ref.c: extract a new function, parse_refname()
 + parse_cmd_verify(): copy old_sha1 instead of evaluating <oldvalue> twice
 + update-ref --stdin: read the whole input at once
 + update_refs(): fix constness
 + refs.h: rename the action_on_err constants
 + t1400: add some more tests involving quoted arguments
 + parse_arg(): really test that argument is properly terminated
 + t1400: provide more usual input to the command
 + t1400: fix name and expected result of one test
 (this branch is used by rs/ref-transaction.)

 Update "update-ref --stdin [-z]" and then introduce a transactional
 support for (multi-)reference updates.

 Will keep in 'next' for the remainder of the cycle.


* jc/apply-ignore-whitespace (2014-03-26) 1 commit
  (merged to 'next' on 2014-04-04 at 53779a7)
 + apply --ignore-space-change: lines with and without leading whitespaces do not match

 "--ignore-space-change" option of "git apply" ignored the
 spaces at the beginning of line too aggressively, which is
 inconsistent with the option of the same name "diff" and "git diff"
 have.

 Will keep in 'next' for the remainder of the cycle.


* as/grep-fullname-config (2014-03-20) 1 commit
  (merged to 'next' on 2014-03-28 at 810a076)
 + grep: add grep.fullName config variable

 Add a configuration variable to force --full-name to be default for
 "git grep".

 This may cause regressions on scripted users that do not expect
 this new behaviour.

 Will keep in 'next' for the remainder of the cycle.


* nd/multiple-work-trees (2014-03-25) 28 commits
 - count-objects: report unused files in $GIT_DIR/repos/...
 - gc: support prune --repos
 - gc: style change -- no SP before closing bracket
 - prune: strategies for linked checkouts
 - checkout: detach if the branch is already checked out elsewhere
 - checkout: clean up half-prepared directories in --to mode
 - checkout: support checking out into a new working directory
 - use new wrapper write_file() for simple file writing
 - wrapper.c: wrapper to open a file, fprintf then close
 - setup.c: support multi-checkout repo setup
 - setup.c: detect $GIT_COMMON_DIR check_repository_format_gently()
 - setup.c: convert check_repository_format_gently to use strbuf
 - setup.c: detect $GIT_COMMON_DIR in is_git_directory()
 - setup.c: convert is_git_directory() to use strbuf
 - git-stash: avoid hardcoding $GIT_DIR/logs/....
 - *.sh: avoid hardcoding $GIT_DIR/hooks/...
 - git-sh-setup.sh: use rev-parse --git-path to get $GIT_DIR/objects
 - $GIT_COMMON_DIR: a new environment variable
 - commit: use SEQ_DIR instead of hardcoding "sequencer"
 - fast-import: use git_path() for accessing .git dir instead of get_git_dir()
 - reflog: avoid constructing .lock path with git_path
 - *.sh: respect $GIT_INDEX_FILE
 - git_path(): be aware of file relocation in $GIT_DIR
 - path.c: group git_path(), git_pathdup() and strbuf_git_path() together
 - path.c: rename vsnpath() to do_git_path()
 - git_snpath(): retire and replace with strbuf_git_path()
 - path.c: make get_pathname() call sites return const char *
 - path.c: make get_pathname() return strbuf instead of static buffer

 A replacement for contrib/workdir/git-new-workdir that does not
 rely on symbolic links and make sharing of objects and refs safer
 by making the borrowee and borrowers aware of each other.

 Will hold.


* ks/tree-diff-nway (2014-04-09) 20 commits
  (merged to 'next' on 2014-04-09 at c17228e)
 + mingw: activate alloca
  (merged to 'next' on 2014-04-08 at 6b74773)
 + combine-diff: speed it up, by using multiparent diff tree-walker directly
 + tree-diff: rework diff_tree() to generate diffs for multiparent cases as well
 + Portable alloca for Git
  (merged to 'next' on 2014-03-31 at 16a7bd4)
 + tree-diff: reuse base str(buf) memory on sub-tree recursion
 + tree-diff: no need to call "full" diff_tree_sha1 from show_path()
 + tree-diff: rework diff_tree interface to be sha1 based
 + tree-diff: diff_tree() should now be static
 + tree-diff: remove special-case diff-emitting code for empty-tree cases
  (merged to 'next' on 2014-03-25 at cfcbdac)
 + tree-diff: simplify tree_entry_pathcmp
 + tree-diff: show_path prototype is not needed anymore
 + tree-diff: rename compare_tree_entry -> tree_entry_pathcmp
 + tree-diff: move all action-taking code out of compare_tree_entry()
 + tree-diff: don't assume compare_tree_entry() returns -1,0,1
  (merged to 'next' on 2014-03-21 at d872679)
 + tree-diff: consolidate code for emitting diffs and recursion in one place
 + tree-diff: show_tree() is not needed
 + tree-diff: no need to pass match to skip_uninteresting()
 + tree-diff: no need to manually verify that there is no mode change for a path
 + combine-diff: move changed-paths scanning logic into its own function
 + combine-diff: move show_log_first logic/action out of paths scanning

 Instead of running N pair-wise diff-trees when inspecting a
 N-parent merge, find the set of paths that were touched by walking
 N+1 trees in parallel.  These set of paths can then be turned into
 N pair-wise diff-tree results to be processed through rename
 detections and such.  And N=2 case nicely degenerates to the usual
 2-way diff-tree, which is very nice.

 Will keep in 'next' for the remainder of the cycle.


* cc/interpret-trailers (2014-04-29) 11 commits
 - Documentation: add documentation for 'git interpret-trailers'
 - trailer: add tests for commands in config file
 - trailer: execute command from 'trailer.<name>.command'
 - trailer: add tests for "git interpret-trailers"
 - trailer: add interpret-trailers command
 - trailer: put all the processing together and print
 - trailer: parse trailers from file or stdin
 - trailer: process command line trailer arguments
 - trailer: read and process config information
 - trailer: process trailers from input message and arguments
 - trailer: add data structures and basic functions

 A new filter to programatically edit the tail end of the commit log
 messages.

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[John Keeping]

On Tue, Apr 29, 2014 at 03:38:07PM -0700, Junio C Hamano wrote:
> * fc/remote-helpers-hg-bzr-graduation (2014-04-29) 11 commits
>  - remote-hg: trivial cleanups
>  - remote-hg: make sure we omit multiple heads
>  - git-remote-hg: use internal clone's hgrc
>  - t: remote-hg: split into setup test
>  - remote-hg: properly detect missing contexts
>  - remote-{hg,bzr}: store marks only on success
>  - remote-hg: update to 'public' phase when pushing
>  - remote-hg: fix parsing of custom committer
>   (merged to 'next' on 2014-04-22 at fed170a)
>  + remote-helpers: move tests out of contrib
>  + remote-helpers: move out of contrib
>  + remote-helpers: squelch python import exceptions
> 
>  Move remote-hg and remote-bzr out of contrib/.  There were some
>  suggestions on the follow-up fix patches still not in 'next', which
>  may result in a reroll.
> 
>  Will merge to 'next' and keep it there for the remainder of the
>  cycle.

I'd like to register my opposition to moving git-remote-{bzr,hg} out of
contrib/.

I am not convinced that tools for interoperating with other VCSs need to
be part of core Git; as Junio has pointed out previously, while contrib/
was necessary for promoting associated tools when Git was young and had
not established mindshare, Git is now by far the most popular DVCS and
is rapidly catching up with centralized systems.  Associated tools can
therefore live on their own and do not need to be promoted as part of
Git itself (as git-imerge is doing successfully).

In the case of git-remote-hg specifically, the remote helper has to use
an interface that the Mercurial developers consider unstable [1]; the
version currently on 'pu' fails the test suite for me because I have
Mercurial 3.0:

	AttributeError: 'mqrepo' object has no attribute 'getbundle'

I do not want to end up in a situation where an update to Git is blocked
by a distribution because git-remote-hg is not updated to support newer
versions of Mercurial sufficiently quickly; this previously happened in
Gentoo due to git-svn and meant that was stuck on 1.7.8 until 1.7.13 was
released [2].

Since the remote helper interface is stable and the remote helpers do
not use any of the Git internals, I consider the risks of including them
in core Git to outweigh the benefits of wider distribution.  In fact,
the remote helpers may benefit from having their own release cadences
so that they can respond to changes in related projects more quickly
than the normal Git release cycle.


[1] http://mercurial.selenic.com/wiki/MercurialApi#Why_you_shouldn.27t_use_Mercurial.27s_internal_API
[2] https://bugs.gentoo.org/show_bug.cgi?id=418431

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

John Keeping wrote:
> I am not convinced that tools for interoperating with other VCSs need to
> be part of core Git; as Junio has pointed out previously, while contrib/
> was necessary for promoting associated tools when Git was young and had
> not established mindshare, Git is now by far the most popular DVCS and
> is rapidly catching up with centralized systems.  Associated tools can
> therefore live on their own and do not need to be promoted as part of
> Git itself (as git-imerge is doing successfully).

Then let's remove git-p4.

> In the case of git-remote-hg specifically, the remote helper has to use
> an interface that the Mercurial developers consider unstable [1];

There is no other sensible way of doing them.

> the version currently on 'pu' fails the test suite for me because I
> have Mercurial 3.0:
> 
> 	AttributeError: 'mqrepo' object has no attribute 'getbundle'

And because this patch has not been picked up[1].

> I do not want to end up in a situation where an update to Git is blocked
> by a distribution because git-remote-hg is not updated to support newer
> versions of Mercurial sufficiently quickly; this previously happened in
> Gentoo due to git-svn and meant that was stuck on 1.7.8 until 1.7.13 was
> released [2].

Travis-CI ensures that won't happen[2].

> Since the remote helper interface is stable and the remote helpers do
> not use any of the Git internals, I consider the risks of including them
> in core Git to outweigh the benefits of wider distribution.  In fact,
> the remote helpers may benefit from having their own release cadences
> so that they can respond to changes in related projects more quickly
> than the normal Git release cycle.

Maybe, but git-remote-hg has already benefitted a lot from the wider
exposure of being in 'contrib/', I'm sure it would benefit even more if
it's distributed by default.

Moreover, there's a ton of subpar tools out there[3], and I believe
giving the burden of choosing one to the user is detrimental to the Git
project. If we as a project say: this is the one we recommend, and has
the Git stamp, that helps the users tremendously.

Your point is valid though, but it's valid not just for
git-remote-hg/bzr.

So I think these are the two options:

  1) Include git-remote-hg/bzr to the core and distribute them by
     default (as is the current intention)

  2) Remove git-remote-hg/bzr entirely from the Git tree. And do the
     same for other tools: git-p4, git-svn, git-cvs*. Given the huge
     amount of people using Subversion, we might want to defer that one
     for later, but eventually do it.

I'd say for v2.0 we should go for 1), and 2) should be considered for
v3.0, perhaps.

[1] http://article.gmane.org/gmane.comp.version-control.git/248065
[2] https://travis-ci.org/felipec/git
[3] https://github.com/felipec/git/wiki/Comparison-of-git-remote-hg-alternatives

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[John Keeping]

On Mon, May 05, 2014 at 02:08:28PM -0500, Felipe Contreras wrote:
> John Keeping wrote:
> > I am not convinced that tools for interoperating with other VCSs need to
> > be part of core Git; as Junio has pointed out previously, while contrib/
> > was necessary for promoting associated tools when Git was young and had
> > not established mindshare, Git is now by far the most popular DVCS and
> > is rapidly catching up with centralized systems.  Associated tools can
> > therefore live on their own and do not need to be promoted as part of
> > Git itself (as git-imerge is doing successfully).
> 
> Then let's remove git-p4.

If git-p4 were not already in the core, I would be making precisely the
same argument regarding it (and the others you identify below).

> > In the case of git-remote-hg specifically, the remote helper has to use
> > an interface that the Mercurial developers consider unstable [1];
> 
> There is no other sensible way of doing them.
> 
> > the version currently on 'pu' fails the test suite for me because I
> > have Mercurial 3.0:
> > 
> > 	AttributeError: 'mqrepo' object has no attribute 'getbundle'
> 
> And because this patch has not been picked up[1].

And it is now probably too late for that to make Git 2.0, which means it
may be another 12 weeks before it makes it into a Git release.  Since
this is quite a minor change it may make it into a stable release, but
what would happen if the required changes were much more involved?

> > I do not want to end up in a situation where an update to Git is blocked
> > by a distribution because git-remote-hg is not updated to support newer
> > versions of Mercurial sufficiently quickly; this previously happened in
> > Gentoo due to git-svn and meant that was stuck on 1.7.8 until 1.7.13 was
> > released [2].
> 
> Travis-CI ensures that won't happen[2].

I don't see that building against Mercurial's default branch, so it will
not help with future releases.

> > Since the remote helper interface is stable and the remote helpers do
> > not use any of the Git internals, I consider the risks of including them
> > in core Git to outweigh the benefits of wider distribution.  In fact,
> > the remote helpers may benefit from having their own release cadences
> > so that they can respond to changes in related projects more quickly
> > than the normal Git release cycle.
> 
> Maybe, but git-remote-hg has already benefitted a lot from the wider
> exposure of being in 'contrib/', I'm sure it would benefit even more if
> it's distributed by default.

Is that because it was included in contrib/ or just as a result of being
publicised on this list and elsewhere?  I don't think git-imerge is
suffering from being its own project and git-subtree appears to have
received very little attention despite being in contrib/.

> Moreover, there's a ton of subpar tools out there[3], and I believe
> giving the burden of choosing one to the user is detrimental to the Git
> project. If we as a project say: this is the one we recommend, and has
> the Git stamp, that helps the users tremendously.

But by choosing one now, we are stuck promoting that one even if a
better alternative comes along in the future.  We have seen that with
git-cvsimport and it's not dissimilar to the situation with git-pull.

> Your point is valid though, but it's valid not just for
> git-remote-hg/bzr.
> 
> So I think these are the two options:
> 
>   1) Include git-remote-hg/bzr to the core and distribute them by
>      default (as is the current intention)
> 
>   2) Remove git-remote-hg/bzr entirely from the Git tree. And do the
>      same for other tools: git-p4, git-svn, git-cvs*. Given the huge
>      amount of people using Subversion, we might want to defer that one
>      for later, but eventually do it.

Don't forget git-archimport...

My personal vote would be for 2), splitting the bridges to other VCSs
into their own repositories but there would need to be some guarantee
that they would continue to be maintained.  I'm not sure it needs to
wait for a major Git release since most of the impact is on package
maintainers and not end users.

> I'd say for v2.0 we should go for 1), and 2) should be considered for
> v3.0, perhaps.

I don't think there is any advantage to adding new tools now if we only
intend to remove them in the future.

> [1] http://article.gmane.org/gmane.comp.version-control.git/248065
> [2] https://travis-ci.org/felipec/git
> [3] https://github.com/felipec/git/wiki/Comparison-of-git-remote-hg-alternatives

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

John Keeping wrote:
> On Mon, May 05, 2014 at 02:08:28PM -0500, Felipe Contreras wrote:
> > John Keeping wrote:
> > > I am not convinced that tools for interoperating with other VCSs need to
> > > be part of core Git; as Junio has pointed out previously, while contrib/
> > > was necessary for promoting associated tools when Git was young and had
> > > not established mindshare, Git is now by far the most popular DVCS and
> > > is rapidly catching up with centralized systems.  Associated tools can
> > > therefore live on their own and do not need to be promoted as part of
> > > Git itself (as git-imerge is doing successfully).
> > 
> > Then let's remove git-p4.
> 
> If git-p4 were not already in the core, I would be making precisely the
> same argument regarding it (and the others you identify below).

So basically you are arguing against any change.

> > > the version currently on 'pu' fails the test suite for me because I
> > > have Mercurial 3.0:
> > > 
> > > 	AttributeError: 'mqrepo' object has no attribute 'getbundle'
> > 
> > And because this patch has not been picked up[1].
> 
> And it is now probably too late for that to make Git 2.0,

That's not for you to decide.

> which means it may be another 12 weeks before it makes it into a Git
> release.  Since this is quite a minor change it may make it into a
> stable release, but what would happen if the required changes were
> much more involved?

All the Mercurial API compatibility issues I have seen are trivial.

> > > I do not want to end up in a situation where an update to Git is blocked
> > > by a distribution because git-remote-hg is not updated to support newer
> > > versions of Mercurial sufficiently quickly; this previously happened in
> > > Gentoo due to git-svn and meant that was stuck on 1.7.8 until 1.7.13 was
> > > released [2].
> > 
> > Travis-CI ensures that won't happen[2].
> 
> I don't see that building against Mercurial's default branch, so it will
> not help with future releases.

I can easily add that.

> > > Since the remote helper interface is stable and the remote helpers do
> > > not use any of the Git internals, I consider the risks of including them
> > > in core Git to outweigh the benefits of wider distribution.  In fact,
> > > the remote helpers may benefit from having their own release cadences
> > > so that they can respond to changes in related projects more quickly
> > > than the normal Git release cycle.
> > 
> > Maybe, but git-remote-hg has already benefitted a lot from the wider
> > exposure of being in 'contrib/', I'm sure it would benefit even more if
> > it's distributed by default.
> 
> Is that because it was included in contrib/ or just as a result of being
> publicised on this list and elsewhere?

The former I'd bet.

> I don't think git-imerge is suffering from being its own project and
> git-subtree appears to have received very little attention despite
> being in contrib/.

Apples and oranges. There aren't scores of tools out there trying to do
what git-subtree does.

> > Moreover, there's a ton of subpar tools out there[3], and I believe
> > giving the burden of choosing one to the user is detrimental to the Git
> > project. If we as a project say: this is the one we recommend, and has
> > the Git stamp, that helps the users tremendously.
> 
> But by choosing one now, we are stuck promoting that one even if a
> better alternative comes along in the future.

Are there better alternatives coming in the future?

> > Your point is valid though, but it's valid not just for
> > git-remote-hg/bzr.
> > 
> > So I think these are the two options:
> > 
> >   1) Include git-remote-hg/bzr to the core and distribute them by
> >      default (as is the current intention)
> > 
> >   2) Remove git-remote-hg/bzr entirely from the Git tree. And do the
> >      same for other tools: git-p4, git-svn, git-cvs*. Given the huge
> >      amount of people using Subversion, we might want to defer that one
> >      for later, but eventually do it.
> 
> Don't forget git-archimport...
> 
> My personal vote would be for 2), splitting the bridges to other VCSs
> into their own repositories but there would need to be some guarantee
> that they would continue to be maintained.  I'm not sure it needs to
> wait for a major Git release since most of the impact is on package
> maintainers and not end users.

Sure, we might not need to wait for v3.0, but that's not the point. The
point is that we should be consistent, and that means going for 1) in
v2.0.

> > I'd say for v2.0 we should go for 1), and 2) should be considered for
> > v3.0, perhaps.
> 
> I don't think there is any advantage to adding new tools now if we only
> intend to remove them in the future.

Do we intend to remove them in the future? That hasn't been decided.

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

Felipe Contreras wrote:
> John Keeping wrote:
> > I don't see that building against Mercurial's default branch, so it
> > will not help with future releases.
> 
> I can easily add that.

There:
https://travis-ci.org/felipec/git

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Junio C Hamano]

John Keeping <john@keeping.me.uk> writes:

> On Tue, Apr 29, 2014 at 03:38:07PM -0700, Junio C Hamano wrote:
>> * fc/remote-helpers-hg-bzr-graduation (2014-04-29) 11 commits
>>  ...
>>  Move remote-hg and remote-bzr out of contrib/.  There were some
>>  suggestions on the follow-up fix patches still not in 'next', which
>>  may result in a reroll.
>> 
>>  Will merge to 'next' and keep it there for the remainder of the
>>  cycle.
>
> I'd like to register my opposition to moving git-remote-{bzr,hg} out of
> contrib/.
> ...
> In the case of git-remote-hg specifically, the remote helper has to use
> an interface that the Mercurial developers consider unstable [1];...
> I do not want to end up in a situation where an update to Git is blocked
> by a distribution because git-remote-hg is not updated to support newer
> versions of Mercurial sufficiently quickly; this previously happened in
> Gentoo due to git-svn and meant that was stuck on 1.7.8 until 1.7.13 was
> released [2].

The same argument would apply to git-svn, git-p4, and git-cvsimport,
I would think.

Among these, I am not sure if we can find willing maintainers who
can give enough love to them.  But unlike these other importers,
remote-hg and remote-bzr do have an active maintainer (and IIRC I
think I heard that Hg one even has an active competitor or two?) so
I am reasonably confident that these can live on their own merit
outside of my tree.  In the ideal world, I would think it may be
even beneficial to the end users of these helpers to unbundle them.

You raised a good point on the issue of external dependencies may
impact Git as a whole, even for those who are not interested in the
particular remote helpers at all.  I'll have to think about it.

The silly thing is that I totally forgot that we almost got
ourselves into a very similar situation on cvsimport when a series
wanted to make it cvsps3-only.  It is very possible nobody would
have picked up the entire new release, if we merged that change.

Having said all that, there is one caveat.

> Since the remote helper interface is stable and the remote helpers do
> not use any of the Git internals, I consider the risks of including them
> in core Git to outweigh the benefits of wider distribution.

You are correct to say that a remote helper has to talk with a
foreign system and it would not help to dictate the update schedule
of helpers to match the release cycle of Git itself.  At the same
time, however, the interface the remote helpers use to talk to Git
has not been as stable as you seem to think, I am afraid.  For
example, a recent remote-hg/bzr series needed some enhancements to
fast-import to achieve the feature parity with native transports by
adding a missing feature or two on the Git side.

So in reality, a helper has to talk with two sides, needs to adjust
to changes in the both sides, and both sides are changing.

Unbundling a helper from Git would place more burden on the helper's
maintainer, because the helper has to know enough about versions and
features of both sides (the foreign system and Git) to adjust its
behaviour, to stay compatible with wider versions of both foreign
systems and Git.  Unbundling, when done properly, may give more
ideal user experience to the end users, because such a helper would
allow them to pick up the latest (or stay on an older but known to
be stable) version of the helper and expect it to work with the
foreign system and Git they happen to have.

It however would be easier to maintain if the helper maintainer
knows a change to Git itself will be released at the same time as
the new version of the helper that takes advantage of the modified
Git.  The helper maintainer only has to worry about compatibility
with the foreign side if it is bundled with Git.

So it boils down to "how much resource are there to make sure a
helper will stay compatible with wider versions of both sides?" and
"how far backwards are helper maintainers willing to bend to support
users better?".

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

Junio C Hamano wrote:
> John Keeping <john@keeping.me.uk> writes:

> > In the case of git-remote-hg specifically, the remote helper has to use
> > an interface that the Mercurial developers consider unstable [1];...
> > I do not want to end up in a situation where an update to Git is blocked
> > by a distribution because git-remote-hg is not updated to support newer
> > versions of Mercurial sufficiently quickly; this previously happened in
> > Gentoo due to git-svn and meant that was stuck on 1.7.8 until 1.7.13 was
> > released [2].
> 
> The same argument would apply to git-svn, git-p4, and git-cvsimport,
> I would think.
> 
> Among these, I am not sure if we can find willing maintainers who
> can give enough love to them.  But unlike these other importers,
> remote-hg and remote-bzr do have an active maintainer (and IIRC I
> think I heard that Hg one even has an active competitor or two?)

Unfortunately there are no more real competitors to remote-hg. A far as
I can tell msysgit has dropped their remote helper, and gitifyhg is not
being actively maintatined and it's even pointing to our git-remote-hg
as probably the best alternative to use at the moment.

> so I am reasonably confident that these can live on their own merit
> outside of my tree.  In the ideal world, I would think it may be even
> beneficial to the end users of these helpers to unbundle them.

It might be benefitial in the future, but right now I'm willing to bet
there's many people that don't know git-remote-hg/bzr even exist. If Git
v2.0 distributes them by default, and they are mentioned in the release
notes:

 * Transparent support to pull and push to and from Mercurial and Bazaar
   repositories is now enabled by default.

Many more people will know about that, and in the future when we try to
unbundle them they can shout if for some reason it would be inconvenient
for them. At the moment I don't think we can say for sure.

Even if people don't use these bridges, I think just mentioning that
feature helps the project in general.

> You raised a good point on the issue of external dependencies may
> impact Git as a whole, even for those who are not interested in the
> particular remote helpers at all.  I'll have to think about it.

Yes, it's worth thinking about it because it's a real possibility.
However, real possibilities are many times not likely to happen, and I
think this is one of those cases.

As I've said, if history is any indication these issues won't happen. As
far as I can remember the only issues that have happened are backwards
compatibility issues, not present or future. And as I said I've setup
TravisCI builds to detect those, which is why we haven't had those
issues since then.

> So it boils down to "how much resource are there to make sure a helper
> will stay compatible with wider versions of both sides?" and "how far
> backwards are helper maintainers willing to bend to support users
> better?".

This is not that big of an issue. For example, notice how the changes in
the transport-helper to enable say --force and --dry-run did not
require to align changes in remote-hg/bzr. That's because remote-hg/bzr
had already the code for these features, it was just not exercised until
the transport-helper was modified.

I think the current transport-helper infraestructure is already good
enough to detect the features and options of the remote helpers so
unbundling wouldn't be a major problem.

Having said that alignment issues do happen, and we have one of those in
Git v2.0, but I don't think they are a major concern (at least for
remote-hg/bzr).

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

Felipe Contreras wrote:
> Having said that alignment issues do happen, and we have one of those
> in Git v2.0, but I don't think they are a major concern (at least for
> remote-hg/bzr).

Actually I just noticed that the remote-helpers side is not in the
"master" branch.

I don't know what is your plan with fc/remote-helpers-hg-bzr-graduation,
but for v2.0 we really want the patch 'remote-{hg,bzr}: store marks only
on success'. Explaining precisely why would take a lot of effort, but
basically it's related to 3994e64 (transport-helper: fix sync issue on
crashes).

If you are worried about merging the whole branch, I could pick only the
important patches and reroll.

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[John Keeping]

On Mon, May 05, 2014 at 04:50:58PM -0700, Junio C Hamano wrote:
> John Keeping <john@keeping.me.uk> writes:
> 
> Having said all that, there is one caveat.
> 
> > Since the remote helper interface is stable and the remote helpers do
> > not use any of the Git internals, I consider the risks of including them
> > in core Git to outweigh the benefits of wider distribution.
> 
> You are correct to say that a remote helper has to talk with a
> foreign system and it would not help to dictate the update schedule
> of helpers to match the release cycle of Git itself.  At the same
> time, however, the interface the remote helpers use to talk to Git
> has not been as stable as you seem to think, I am afraid.  For
> example, a recent remote-hg/bzr series needed some enhancements to
> fast-import to achieve the feature parity with native transports by
> adding a missing feature or two on the Git side.

This doesn't qualify as an unstable interface for me.  In this case, the
remote helpers could not support a feature without Git supporting it
first, which is quite natural and the remote helper can then guard that
feature with a capability check.  I do not think it likely that the
remote helper interface will ever change in such a way that all remote
helpers must be updated, at least not without a long deprecation period.

The Mercurial API makes no such guarantee; it is considered a private
implementation detail and most releases seem to contain some changes
that require all consumers to be updated.

There is a different level of urgency between "you cannot use this new
feature until you update Git" and "if you update Mercurial then the
remote helper will stop working", and that's why I think the remote
helpers may benefit from a separate release schedule.

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

John Keeping wrote:
> The Mercurial API makes no such guarantee; it is considered a private
> implementation detail and most releases seem to contain some changes
> that require all consumers to be updated.
> 
> There is a different level of urgency between "you cannot use this new
> feature until you update Git" and "if you update Mercurial then the
> remote helper will stop working",

s/the remote helper will stop working/certain features of the remote
helper *might* stop working, but we are trying hard to make sure that
doesn't happen/

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Junio C Hamano]

John Keeping <john@keeping.me.uk> writes:

> And it is now probably too late for that to make Git 2.0,...

Anything with end-user visible changes in the core part that is not
a fix to a regression introduced between v1.9.0..master is too late
for the upcoming release.  We are way past -rc1.

>> So I think these are the two options:
>> 
>>   1) Include git-remote-hg/bzr to the core and distribute them by
>>      default (as is the current intention)
>> 
>>   2) Remove git-remote-hg/bzr entirely from the Git tree. And do the
>>      same for other tools: git-p4, git-svn, git-cvs*. Given the huge
>>      amount of people using Subversion, we might want to defer that one
>>      for later, but eventually do it.

Isn't there a middle ground?  The option 1.5 may be like this:

 - Eject tools in contrib/ that would benefit the users better if
   they were outside my tree.  There are a few points to consider
   when judging "benefit better if outside":

   * Their release cycle requirements are better met outside my tree
     (the "remote-hg depends not just on Git but Hg internal" issue
     we have discussed).

   * They are actively maintained.  The overall Git maintainer would
     merely be being a bottleneck than being a helpful editor with
     respect to these tools if we keep them in my tree, and we
     expect that the tool maintainer would do a much better job
     without me.

 - Keep tools that are not actively maintained but still used by the
   users widely in my tree, but when their external dependencies
   become baggage to Git as a whole, demote them to contrib/ and
   stop installing them by default.

 - I would not mind having install.contrib-frotz target in the
   top-level Makefile for each of the remaining contrib/frotz
   hierarchies for those users and distro packagers who know their
   platform meets the dependency requirements.

> I'm not sure it needs to
> wait for a major Git release since most of the impact is on package
> maintainers and not end users.

Removal of features is a big deal, I would think, though.

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

Junio C Hamano wrote:
> John Keeping <john@keeping.me.uk> writes:
> 
> > And it is now probably too late for that to make Git 2.0,...
> 
> Anything with end-user visible changes in the core part that is not
> a fix to a regression introduced between v1.9.0..master is too late
> for the upcoming release.  We are way past -rc1.

The patch in question only affects users of hg v3.0 since it's
surrounded by a 'check_version(3, 0)'. Therefore it cannot introduce
regressions, there's no reason not to apply it.

> >> So I think these are the two options:
> >> 
> >>   1) Include git-remote-hg/bzr to the core and distribute them by
> >>      default (as is the current intention)
> >> 
> >>   2) Remove git-remote-hg/bzr entirely from the Git tree. And do the
> >>      same for other tools: git-p4, git-svn, git-cvs*. Given the huge
> >>      amount of people using Subversion, we might want to defer that one
> >>      for later, but eventually do it.
> 
> Isn't there a middle ground?  The option 1.5 may be like this:
> 
>  - Eject tools in contrib/ that would benefit the users better if
>    they were outside my tree.  There are a few points to consider
>    when judging "benefit better if outside":
> 
>    * Their release cycle requirements are better met outside my tree
>      (the "remote-hg depends not just on Git but Hg internal" issue
>      we have discussed).

Shouldn't *I* be the one most qualified to know if the release cycle
requirements are better met outside the git.git tree?

>    * They are actively maintained.  The overall Git maintainer would
>      merely be being a bottleneck than being a helpful editor with
>      respect to these tools if we keep them in my tree, and we
>      expect that the tool maintainer would do a much better job
>      without me.

Perhaps. But only if the patches are reviewed throught the git mailing
list.

And what about the tools that are not actively maintainted? For example
'contrib/hg-to-git'.
 
>  - Keep tools that are not actively maintained but still used by the
>    users widely in my tree, but when their external dependencies
>    become baggage to Git as a whole, demote them to contrib/ and
>    stop installing them by default.

That implies that git-remote-hg would become a baggage to Git as a
whole.

If you are arguing that git-remote-hg should be distributed by default,
and only if the dependencies become a problem, demote to 'contrib/' that
is fine. The same for git-p4 and other tools already out of contrib.

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Junio C Hamano]

John Keeping <john@keeping.me.uk> writes:

> On Mon, May 05, 2014 at 04:50:58PM -0700, Junio C Hamano wrote:
>> ...
>> At the same
>> time, however, the interface the remote helpers use to talk to Git
>> has not been as stable as you seem to think, I am afraid.  For
>> example, a recent remote-hg/bzr series needed some enhancements to
>> fast-import to achieve the feature parity with native transports by
>> adding a missing feature or two on the Git side.
>
> This doesn't qualify as an unstable interface for me.

That is true, but that does not change the equation very much, no?
To a remote-helper maintainer, bundled is easier to maintain than
unbundled, because both sides are changing, and regardless of the
nature of the change, s/he would know how the Git side looks like if
bundled.

Having said that, I agree with the conclusion of your message:

> There is a different level of urgency between "you cannot use this new
> feature until you update Git" and "if you update Mercurial then the
> remote helper will stop working", and that's why I think the remote
> helpers may benefit from a separate release schedule.

and I am inclined to be persuaded that the users of remote-hg/bzr
may better off if they are unbundled from my tree.

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

Junio C Hamano wrote:
> Having said that, I agree with the conclusion of your message:
> 
> > There is a different level of urgency between "you cannot use this
> > new feature until you update Git" and "if you update Mercurial then
> > the remote helper will stop working", and that's why I think the
> > remote helpers may benefit from a separate release schedule.

The conclusion is correct, the premises are not.

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Junio C Hamano]

John Keeping <john@keeping.me.uk> writes:

> I'd like to register my opposition to moving git-remote-{bzr,hg} out of
> contrib/.
>
> I am not convinced that tools for interoperating with other VCSs need to
> be part of core Git; as Junio has pointed out previously, while contrib/
> was necessary ... Associated tools can
> therefore live on their own and do not need to be promoted as part of
> Git itself (as git-imerge is doing successfully).

Another thing to keep in mind is that we need to ensure that we give
a good way for these third-party tools to integrate well with the
core Git tools to form a single toolchest for the users.  I would
love to be able to do

    $ (cd git.git && make install)
    $ (cd git-imerge.git && make install)

and then say "git imerge", "git --help imerge", etc.  The same for
the remote helpers that we may be splitting out of my tree into
their own stand-alone projects.

I _think_ it probably is OK for git-imerge.git/Makefile to peek into
our Makefile, e.g.

    $ cd git-imerge.git
    $ make GIT_SOURCE_DIR=../git.git install

to learn where imerge should install its subcommand implementation
and documentation.  It might even want to borrow the test framework
by using $GIT_SOURCE_DIR/t/test-lib.sh or somesuch.  There may be
some changes the third-party tool authors would want to have in our
Makefile to help them better when building their tools this way; I
dunno.

I also think that there should be a way to make it really easy to
install these third-party tools to augment the installed version of
Git without having the source tree of Git.  We have ways for them to
ask us where things are expected to be, e.g.

    $ git --html-path
    $ git --man-path
    $ git --exec-path

but I am not sure if these are enough, or if it would help them to
add a bit more, then what these "a bit more" are.

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

Junio C Hamano wrote:
> I _think_ it probably is OK for git-imerge.git/Makefile to peek into
> our Makefile, e.g.
> 
>     $ cd git-imerge.git
>     $ make GIT_SOURCE_DIR=../git.git install
> 
> to learn where imerge should install its subcommand implementation and
> documentation.  It might even want to borrow the test framework by
> using $GIT_SOURCE_DIR/t/test-lib.sh or somesuch.

Since Git's test framework heavily tied to git.git, sharness[1] is the
only sensible option. It might not have all the latest features of Git's
test framework, but it's standalone.

[1] https://github.com/mlafeldt/sharness

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[John Keeping]

On Tue, May 06, 2014 at 05:01:59PM -0700, Junio C Hamano wrote:
> John Keeping <john@keeping.me.uk> writes:
> 
> > I'd like to register my opposition to moving git-remote-{bzr,hg} out of
> > contrib/.
> >
> > I am not convinced that tools for interoperating with other VCSs need to
> > be part of core Git; as Junio has pointed out previously, while contrib/
> > was necessary ... Associated tools can
> > therefore live on their own and do not need to be promoted as part of
> > Git itself (as git-imerge is doing successfully).
> 
> Another thing to keep in mind is that we need to ensure that we give
> a good way for these third-party tools to integrate well with the
> core Git tools to form a single toolchest for the users.  I would
> love to be able to do
> 
>     $ (cd git.git && make install)
>     $ (cd git-imerge.git && make install)
> 
> and then say "git imerge", "git --help imerge", etc.  The same for
> the remote helpers that we may be splitting out of my tree into
> their own stand-alone projects.

This can already work given suitable installation.  With
git-integration[1] I can type `git help integration` and it shows me the
man page in the same way that `git help commit` does.  When I manually
linked the HTML file to the right place `git help -w integration` worked
as well.

> I _think_ it probably is OK for git-imerge.git/Makefile to peek into
> our Makefile, e.g.
> 
>     $ cd git-imerge.git
>     $ make GIT_SOURCE_DIR=../git.git install
> 
> to learn where imerge should install its subcommand implementation
> and documentation.  It might even want to borrow the test framework
> by using $GIT_SOURCE_DIR/t/test-lib.sh or somesuch.  There may be
> some changes the third-party tool authors would want to have in our
> Makefile to help them better when building their tools this way; I
> dunno.
> 
> I also think that there should be a way to make it really easy to
> install these third-party tools to augment the installed version of
> Git without having the source tree of Git.  We have ways for them to
> ask us where things are expected to be, e.g.
> 
>     $ git --html-path
>     $ git --man-path
>     $ git --exec-path
> 
> but I am not sure if these are enough, or if it would help them to
> add a bit more, then what these "a bit more" are.

I think this is enough - now I need to go and make git-integration's
Makefile use them by default rather than just using the same defaults as
git.git.

Perhaps it would be useful to have a skeleton "external Git utility"
project under contrib/ which could demonstrate best practice for
installing utilties that augment Git.

[1] http://johnkeeping.github.io/git-integration/

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

John Keeping wrote:
> > I also think that there should be a way to make it really easy to
> > install these third-party tools to augment the installed version of
> > Git without having the source tree of Git.  We have ways for them to
> > ask us where things are expected to be, e.g.
> > 
> >     $ git --html-path
> >     $ git --man-path
> >     $ git --exec-path
> > 
> > but I am not sure if these are enough, or if it would help them to
> > add a bit more, then what these "a bit more" are.
> 
> I think this is enough - now I need to go and make git-integration's
> Makefile use them by default rather than just using the same defaults as
> git.git.

This is wrong. Subprojects should use /usr/bin/ and /usr/share/man/ and
not rely on the output of `git --exec-path` and so on.

For example if the user has installed Git in his $home, when building a
package the package manager would use ~/libexec/git-core, which is
wrong.

Moreover, if you are cross-compiling you won't be able to run the
target's `git` binary.

If anything, it should be `pkg-config --variable=exec-path git`.

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 2006 bytes --]


Junio C Hamano <gitster@pobox.com> writes:

> You raised a good point on the issue of external dependencies may
> impact Git as a whole, even for those who are not interested in the
> particular remote helpers at all.  I'll have to think about it.

(As I'm sure you know, but starting from the beginning.)  There are
basically two ways that a program can be built.  One is by a user on a
particular system.  There, checking to see if various libraries are
installed and if so enabling some additional feature (that isn't that
hard/time-consuming to build) is entirely reasonable.

In a packaging system, dependencies are much more troublesome.
Dependencies have to be declared, and the build limited to use only
those declared dependencies, in order to get repeatable builds and
binary packages that can be used on other systems.  Dependencies that
really are required are fine.  But optional dependencies are a problem,
because e.g. one doesn't want to require the presence of qt to build
something (that isn't already enormous).   So if git needs mercurial and
subversion installed, plus perhaps 5 other things for less popular
remote helpers, that starts to be a real burden.

(I realize some packaging systems have a style where the union of the
possible dependencies must be present to build, and then the resulting
binaries are allocated to split packages.  But that's not universal, and
it still requires large amounts of unnecessary dependencies to build a
package from source.)

Ideally, the core of git would have a small set of dependencies, and
optional language bindings or remote helpers could be built
independently (by running a different build, after git proper was built
and installed).  It seems more likely that the property of independent
builds of optional components will be preserved if the various git-foo
pieces are in seaprate trees.  But if they are in subdirs of the main
git tree, and build by "./configure && make && make install" in the
subdir, that's arguably equivalent.

[-- Attachment #2: Type: application/pgp-signature, Size: 180 bytes --]

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Junio C Hamano]

John Keeping <john@keeping.me.uk> writes:

> On Tue, May 06, 2014 at 05:01:59PM -0700, Junio C Hamano wrote:
> ...
>> Another thing to keep in mind is that we need to ensure that we give
>> a good way for these third-party tools to integrate well with the
>> core Git tools to form a single toolchest for the users.  I would
>> love to be able to do
>> 
>>     $ (cd git.git && make install)
>>     $ (cd git-imerge.git && make install)
>> 
>> and then say "git imerge", "git --help imerge", etc.  The same for
>> the remote helpers that we may be splitting out of my tree into
>> their own stand-alone projects.
>
> This can already work given suitable installation.  With
> git-integration[1] I can type `git help integration` and it shows me the
> man page in the same way that `git help commit` does.  When I manually
> linked the HTML file to the right place `git help -w integration` worked
> as well.

That "when I manually" part is what I meant by "we give a good way
for these third-party tools" above, and "make it really easy to
install these third-party tools" in the remaining part of the
message you are responding to.

> I think this is enough...

Thanks.

The reason why I CC'ed Michael was primarily because I thought you
were not one of those third-party tools maintainers (and secondarily
I am a fairly big fan of imerge), but it is good to hear your
opinion as another third-party provider.  Your git-integrate might
turn into something I could augment my workflow with with some
additions.  What is missing (I only read the full manual page at
http://johnkeeping.github.io/git-integration/git-integration.html)
to support my workflow seems to be:

 - specifying a merge strategy per branch being merged;
 - support evil merges or picking a fix-up commit;
 - leaving an empty commit only to leave comment in the history.

and until that happens, I'll keep using the Reintegrate script found
in my 'todo' branch.

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[John Keeping]

On Wed, May 07, 2014 at 11:56:18AM -0700, Junio C Hamano wrote:
> John Keeping <john@keeping.me.uk> writes:
> 
> > On Tue, May 06, 2014 at 05:01:59PM -0700, Junio C Hamano wrote:
> > ...
> >> Another thing to keep in mind is that we need to ensure that we give
> >> a good way for these third-party tools to integrate well with the
> >> core Git tools to form a single toolchest for the users.  I would
> >> love to be able to do
> >> 
> >>     $ (cd git.git && make install)
> >>     $ (cd git-imerge.git && make install)
> >> 
> >> and then say "git imerge", "git --help imerge", etc.  The same for
> >> the remote helpers that we may be splitting out of my tree into
> >> their own stand-alone projects.
> >
> > This can already work given suitable installation.  With
> > git-integration[1] I can type `git help integration` and it shows me the
> > man page in the same way that `git help commit` does.  When I manually
> > linked the HTML file to the right place `git help -w integration` worked
> > as well.
> 
> That "when I manually" part is what I meant by "we give a good way
> for these third-party tools" above, and "make it really easy to
> install these third-party tools" in the remaining part of the
> message you are responding to.
> 
> > I think this is enough...

Having thought about it a bit more after reading Felipe's reply, it
would be nice if there were some way for third-party tools to install
HTML documentation without relying on `git --html-path` but I cannot see
an obvious way to do that as there isn't a standard $HTML_PATH to match
$MAN_PATH and $PATH.

I've never tried `git help --info` until this thread, but I think we
could make some trivial improvements to that in order to support .info
documentation for third-party tools.

> The reason why I CC'ed Michael was primarily because I thought you
> were not one of those third-party tools maintainers (and secondarily
> I am a fairly big fan of imerge), but it is good to hear your
> opinion as another third-party provider.  Your git-integrate might
> turn into something I could augment my workflow with with some
> additions.  What is missing (I only read the full manual page at
> http://johnkeeping.github.io/git-integration/git-integration.html)
> to support my workflow seems to be:
> 
>  - specifying a merge strategy per branch being merged;

This is already supported by the "merge" instruction:

	If any options are given after the ref (and on the same line)
	then these are passed to git merge. This may be useful for
	specifying an alternative merge strategy for a branch.

>  - support evil merges or picking a fix-up commit;

I have an implementation of this on a branch, but have never merged it
because it's not something I need to do often and it is very hard to
support for git-integration's "status" output.

One of my primary use cases for git-integration involves pulling
together branches owned by others (either in the same repository or by
having fetched from their repositories); in this case it is interesting
to see if/how a branch has changed since the last time the integration
branch was built.  This also handles changes to the instruction sheet
without an immediate rebuild.

I have not found a good way of figuring out whether a fixup commit has
been applied and squashed into a merge) so I have let the branch sit
there awaiting a perfect solution (which I doubt exists).  It may be
that the status of a fixup is unimportant, so it could just be marked as
unknown; I am mostly convinced that marking it as unknown is going to be
better than an heuristic that is right most of the time.

>  - leaving an empty commit only to leave comment in the history.

This would be easy to add.

> and until that happens, I'll keep using the Reintegrate script found
> in my 'todo' branch.

When I originally wrote git-integration I purposefully did not target
your workflow because I (perhaps wrongly) assumed that the interaction
between the different integration branches would mean that Git was
better served sticking to the custom Reintegrate script.

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

John Keeping wrote:
> Having thought about it a bit more after reading Felipe's reply, it
> would be nice if there were some way for third-party tools to install
> HTML documentation without relying on `git --html-path` but I cannot
> see an obvious way to do that as there isn't a standard $HTML_PATH to
> match $MAN_PATH and $PATH.

Using `git --html-path` for that is wrong.

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

Greg Troxel wrote:
> In a packaging system, dependencies are much more troublesome.
> Dependencies have to be declared, and the build limited to use only
> those declared dependencies, in order to get repeatable builds and
> binary packages that can be used on other systems.  Dependencies that
> really are required are fine.  But optional dependencies are a
> problem, because e.g. one doesn't want to require the presence of qt
> to build something (that isn't already enormous).   So if git needs
> mercurial and subversion installed, plus perhaps 5 other things for
> less popular remote helpers, that starts to be a real burden.

It doesn't *need* them to build. The Mercurial/Bazaar dependencies are
optional, both at build-time and at run-time. Most distributions would
want to test the functionality they are distributing, and for testing
they do need these dependencies.

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

Junio C Hamano wrote:
> That "when I manually" part is what I meant by "we give a good way for
> these third-party tools" above, and "make it really easy to install
> these third-party tools" in the remaining part of the message you are
> responding to.

We need two things:

  1) Provied a pkg-config, as all sane shared components do
  2) Split the testing framework so third-parties don't have to rely on
     yet another third-parth (shareness)

> Your git-integrate might turn into something I could augment my
> workflow with with some additions.
> 
>  - specifying a merge strategy per branch being merged;

git-reintegrate[1] supports this.

>  - support evil merges or picking a fix-up commit;

git-reintegrate supports this.

>  - leaving an empty commit only to leave comment in the history.

Done[2].


> and until that happens, I'll keep using the Reintegrate script found
> in my 'todo' branch.

My git-reintegrate supports everything John's git-integrate and in
addition it supports generating the commands from an existing branch,
like your Reintegrate. IOW; it's superior.

[1] https://github.com/felipec/git-reintegrate
[2] https://github.com/felipec/git-reintegrate/commit/332412470c6e084f10ac2f8dc11e86ab4680974a

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[John Keeping]

On Wed, May 07, 2014 at 03:26:15PM -0500, Felipe Contreras wrote:
> Junio C Hamano wrote:
> > Your git-integrate might turn into something I could augment my
> > workflow with with some additions.
> > 
> >  - specifying a merge strategy per branch being merged;
> 
> git-reintegrate[1] supports this.
> 
> >  - support evil merges or picking a fix-up commit;
> 
> git-reintegrate supports this.
> 
> >  - leaving an empty commit only to leave comment in the history.
> 
> Done[2].
> 
> 
> > and until that happens, I'll keep using the Reintegrate script found
> > in my 'todo' branch.
> 
> My git-reintegrate supports everything John's git-integrate and in
> addition it supports generating the commands from an existing branch,
> like your Reintegrate. IOW; it's superior.

And yet the documentation is unchanged from the version you copied in
from git-integration.  Personally I would much rather use a project
which takes time to document all of the features rather than relying on
reading the code to figure out the options.

More features does not make a project superior.

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

John Keeping wrote:
> On Wed, May 07, 2014 at 03:26:15PM -0500, Felipe Contreras wrote:
> > Junio C Hamano wrote:
> > > Your git-integrate might turn into something I could augment my
> > > workflow with with some additions.
> > > 
> > >  - specifying a merge strategy per branch being merged;
> > 
> > git-reintegrate[1] supports this.
> > 
> > >  - support evil merges or picking a fix-up commit;
> > 
> > git-reintegrate supports this.
> > 
> > >  - leaving an empty commit only to leave comment in the history.
> > 
> > Done[2].
> > 
> > 
> > > and until that happens, I'll keep using the Reintegrate script found
> > > in my 'todo' branch.
> > 
> > My git-reintegrate supports everything John's git-integrate and in
> > addition it supports generating the commands from an existing branch,
> > like your Reintegrate. IOW; it's superior.
> 
> And yet the documentation is unchanged from the version you copied in
> from git-integration.

Not much has changed since v0.1 since that version already worked
perfectly. But I'll update it.

> Personally I would much rather use a project which takes time to
> document all of the features rather than relying on reading the code
> to figure out the options.

And I would rather use a project that concentrates on having the
features users need.

> More features does not make a project superior.

No, better features do.

Either way. Documentation updated.

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 2165 bytes --]


Felipe Contreras <felipe.contreras@gmail.com> writes:

> Greg Troxel wrote:
>> In a packaging system, dependencies are much more troublesome.
>> Dependencies have to be declared, and the build limited to use only
>> those declared dependencies, in order to get repeatable builds and
>> binary packages that can be used on other systems.  Dependencies that
>> really are required are fine.  But optional dependencies are a
>> problem, because e.g. one doesn't want to require the presence of qt
>> to build something (that isn't already enormous).   So if git needs
>> mercurial and subversion installed, plus perhaps 5 other things for
>> less popular remote helpers, that starts to be a real burden.
>
> It doesn't *need* them to build. The Mercurial/Bazaar dependencies are
> optional, both at build-time and at run-time. Most distributions would
> want to test the functionality they are distributing, and for testing
> they do need these dependencies.

My point is that a packaging of git needs to either decide to forego
these optional parts, or to include them, in the default case.  One
choice means that anyone who builds the package from source has to have
the dependencies, and the other means that users of the built package(s)
can't use the features.  I realize that in Linux it's perhaps typical to
not worry about burdening builders because actually building is very
rare, but that's only reasonable because of having only one OS and
perhaps three CPUs; with dozens each, building from source becomes more
frequent.  So I'm merely trying to suggest that it's better to have a
core package with a restrained set of dependencies, and then a way to
build the other things independently (perhaps assuming the core is
built/installed), each with their own dependencies.

It turns out in pkgsrc that git-svn is a meta-package (with no files)
that depends on git-base (no man pages, no gitk) and p5-subversion.
hg-git appears to be a separate source distribution, depending on a
python implementation of the git formats.  So perhaps the situation is
currently ok.  I was just trying to point out the issue to avoid
regressions in the packaging situation.



[-- Attachment #2: Type: application/pgp-signature, Size: 180 bytes --]

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

Greg Troxel wrote:
> 
> Felipe Contreras <felipe.contreras@gmail.com> writes:
> 
> > Greg Troxel wrote:
> >> In a packaging system, dependencies are much more troublesome.
> >> Dependencies have to be declared, and the build limited to use only
> >> those declared dependencies, in order to get repeatable builds and
> >> binary packages that can be used on other systems.  Dependencies that
> >> really are required are fine.  But optional dependencies are a
> >> problem, because e.g. one doesn't want to require the presence of qt
> >> to build something (that isn't already enormous).   So if git needs
> >> mercurial and subversion installed, plus perhaps 5 other things for
> >> less popular remote helpers, that starts to be a real burden.
> >
> > It doesn't *need* them to build. The Mercurial/Bazaar dependencies are
> > optional, both at build-time and at run-time. Most distributions would
> > want to test the functionality they are distributing, and for testing
> > they do need these dependencies.
> 
> My point is that a packaging of git needs to either decide to forego
> these optional parts, or to include them, in the default case.

That is currently the case. They would be included by default, but not
usable unless the *optional* dependencies are installed.

> So I'm merely trying to suggest that it's better to have a core
> package with a restrained set of dependencies, and then a way to build
> the other things independently (perhaps assuming the core is
> built/installed), each with their own dependencies.

I'm all in favor of 'make install' installing only the core of Git, and
different targets for all the other parts.

However, if you take a look at any distribution's packaing of Git you
would see why that wouldn't be desirable (they are full of hacks and
fixes). If the build system is eventually fixed so one package can do
'make install', another 'make install-p4', another 'make install-hg' and
so on, that would be great. But we are pretty far from that.

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Chris Packham]

Hi,

On 06/05/14 11:50, Junio C Hamano wrote:
> John Keeping <john@keeping.me.uk> writes:
> 
>> On Tue, Apr 29, 2014 at 03:38:07PM -0700, Junio C Hamano wrote:
>>> * fc/remote-helpers-hg-bzr-graduation (2014-04-29) 11 commits
>>>  ...
>>>  Move remote-hg and remote-bzr out of contrib/.  There were some
>>>  suggestions on the follow-up fix patches still not in 'next', which
>>>  may result in a reroll.
>>>
>>>  Will merge to 'next' and keep it there for the remainder of the
>>>  cycle.
>>
>> I'd like to register my opposition to moving git-remote-{bzr,hg} out of
>> contrib/.
>> ...
>> In the case of git-remote-hg specifically, the remote helper has to use
>> an interface that the Mercurial developers consider unstable [1];...
>> I do not want to end up in a situation where an update to Git is blocked
>> by a distribution because git-remote-hg is not updated to support newer
>> versions of Mercurial sufficiently quickly; this previously happened in
>> Gentoo due to git-svn and meant that was stuck on 1.7.8 until 1.7.13 was
>> released [2].
> 
> The same argument would apply to git-svn, git-p4, and git-cvsimport,
> I would think.

A bit of a crazy suggestion and a little off-topic. Assuming maintainers
can be found what about having these foreign vcs interfaces as
submodules. That way they can be in Junio's tree as well as having their
own release cycles. The same could apply to git-gui, gitk and gitweb. It
would also be a chance to eat-our-own-dogfood with submodules.

> Among these, I am not sure if we can find willing maintainers who
> can give enough love to them.  But unlike these other importers,
> remote-hg and remote-bzr do have an active maintainer (and IIRC I
> think I heard that Hg one even has an active competitor or two?) so
> I am reasonably confident that these can live on their own merit
> outside of my tree.  In the ideal world, I would think it may be
> even beneficial to the end users of these helpers to unbundle them.
> 
> You raised a good point on the issue of external dependencies may
> impact Git as a whole, even for those who are not interested in the
> particular remote helpers at all.  I'll have to think about it.
> 
> The silly thing is that I totally forgot that we almost got
> ourselves into a very similar situation on cvsimport when a series
> wanted to make it cvsps3-only.  It is very possible nobody would
> have picked up the entire new release, if we merged that change.
> 
> Having said all that, there is one caveat.
> 
>> Since the remote helper interface is stable and the remote helpers do
>> not use any of the Git internals, I consider the risks of including them
>> in core Git to outweigh the benefits of wider distribution.
> 
> You are correct to say that a remote helper has to talk with a
> foreign system and it would not help to dictate the update schedule
> of helpers to match the release cycle of Git itself.  At the same
> time, however, the interface the remote helpers use to talk to Git
> has not been as stable as you seem to think, I am afraid.  For
> example, a recent remote-hg/bzr series needed some enhancements to
> fast-import to achieve the feature parity with native transports by
> adding a missing feature or two on the Git side.
> 
> So in reality, a helper has to talk with two sides, needs to adjust
> to changes in the both sides, and both sides are changing.
> 
> Unbundling a helper from Git would place more burden on the helper's
> maintainer, because the helper has to know enough about versions and
> features of both sides (the foreign system and Git) to adjust its
> behaviour, to stay compatible with wider versions of both foreign
> systems and Git.  Unbundling, when done properly, may give more
> ideal user experience to the end users, because such a helper would
> allow them to pick up the latest (or stay on an older but known to
> be stable) version of the helper and expect it to work with the
> foreign system and Git they happen to have.
> 
> It however would be easier to maintain if the helper maintainer
> knows a change to Git itself will be released at the same time as
> the new version of the helper that takes advantage of the modified
> Git.  The helper maintainer only has to worry about compatibility
> with the foreign side if it is bundled with Git.
> 
> So it boils down to "how much resource are there to make sure a
> helper will stay compatible with wider versions of both sides?" and
> "how far backwards are helper maintainers willing to bend to support
> users better?".
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

Chris Packham wrote:
> On 06/05/14 11:50, Junio C Hamano wrote:
> > The same argument would apply to git-svn, git-p4, and git-cvsimport,
> > I would think.
> 
> A bit of a crazy suggestion and a little off-topic. Assuming maintainers
> can be found what about having these foreign vcs interfaces as
> submodules. That way they can be in Junio's tree as well as having their
> own release cycles. The same could apply to git-gui, gitk and gitweb. It
> would also be a chance to eat-our-own-dogfood with submodules.

If submodules were an integral part of Git that would be a possibility,
but they are more like a hack.

-- 
Felipe Contreras

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Junio C Hamano]

Chris Packham <judge.packham@gmail.com> writes:

> A bit of a crazy suggestion and a little off-topic. Assuming maintainers
> can be found what about having these foreign vcs interfaces as
> submodules. That way they can be in Junio's tree as well as having their
> own release cycles. The same could apply to git-gui, gitk and gitweb. It
> would also be a chance to eat-our-own-dogfood with submodules.

While I agree that submodules may be useful for git-gui and gitk
(which already have their own repository and history), I do not
think that affects the issue of release cycles for third-party tools,
especially the ones with heavier foreign system dependencies like
vcs interfaces.

The release schedule of Git itself places a lot of stress not to
regress anything for existing users of Git, and the gitlink that
points at the specific commit in a submodule will stop advancing in
the top-level superproject (i.e. my tree) during the feature-freeze
period before releases, just like any other paths (i.e. regular file
blobs).

A third-party product maintainer may have other ideas about
stability of their product.  They may want to issue an unproven new
release to adjust to a recent update made to their external
dependencies as soon as code is written, relying on their ability to
issue follow-up maintenance updates on their product in quick
succession.  If many of them are bundled with Git, then we would
have to keep following these "oops, that was wrong" updates from all
of these, which would add unscalable burden to a single choking
point.

Not bundling gives third-party tools the freedom to evolve and worry
about compatibility with their dependencies on their own, and allows
them to treat Git as just one of the dependencies at the same level
as their other dependencies.

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[David Lang]

On Thu, 8 May 2014, Felipe Contreras wrote:

> Chris Packham wrote:
>> On 06/05/14 11:50, Junio C Hamano wrote:
>>> The same argument would apply to git-svn, git-p4, and git-cvsimport,
>>> I would think.
>>
>> A bit of a crazy suggestion and a little off-topic. Assuming maintainers
>> can be found what about having these foreign vcs interfaces as
>> submodules. That way they can be in Junio's tree as well as having their
>> own release cycles. The same could apply to git-gui, gitk and gitweb. It
>> would also be a chance to eat-our-own-dogfood with submodules.
>
> If submodules were an integral part of Git that would be a possibility,
> but they are more like a hack.

Well, if git.git can't use them, then how can anyone else be expected to.

I haven't been paying close attention for a while, what would have to be done to 
make submodules "an integral part of Git"?

David Lang

Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)

[Felipe Contreras]

David Lang wrote:
> On Thu, 8 May 2014, Felipe Contreras wrote:
> > If submodules were an integral part of Git that would be a possibility,
> > but they are more like a hack.
> 
> Well, if git.git can't use them, then how can anyone else be expected to.

That is a very good question.

> I haven't been paying close attention for a while, what would have to be done to 
> make submodules "an integral part of Git"?

This comes to mind:

http://article.gmane.org/gmane.comp.version-control.git/220047

-- 
Felipe Contreras

Submodule improvements (Re: What's cooking in git.git (Apr 2014, #09; Tue, 29))

[Jonathan Nieder]

Hi,

David Lang wrote:

> I haven't been paying close attention for a while, what would have
> to be done to make submodules "an integral part of Git"?

The series at
http://thread.gmane.org/gmane.comp.version-control.git/241455 is a
start.  I'm hoping to get a reroll done soon and then I can talk about
later steps.

https://github.com/jlehmann/git-submod-enhancements/wiki has a rough
roadmap, but really there's lots of commands that could be improved to
recurse into submodules and not many interdependencies involved so
anyone can bite off a chunk.

Thanks,
Jonathan