What's cooking in git.git (Apr 2010, #05; Wed, 14)

What's cooking in git.git (Apr 2010, #05; Wed, 14)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed with '-' are
only in 'pu' while commits prefixed with '+' are in 'next'.  The ones
marked with '.' do not appear in any of the integration branches, but I am
still holding onto them.

We are at 1.7.1-rc1; please test 'master', report & fix regressions.

--------------------------------------------------
[Graduated to "master"]

* jc/doc-submit-gmail (2010-04-07) 1 commit
  (merged to 'next' on 2010-04-10 at cd4e8cf)
 + SubmittingPatches: update GMail section

--------------------------------------------------
[New Topics]

* hg/id-munging (2010-04-06) 2 commits
 - convert: Keep foreign $Id$ on checkout.
 - convert: Safer handling of $Id$ contraction.

* jn/submodule-basic-test (2010-04-10) 3 commits
 - t7400: clarify submodule update tests
 - t7400: clarify 'submodule add' tests
 - t7400: split setup into multiple tests

* np/index-pack-memsave (2010-04-12) 3 commits
 - index-pack: smarter memory usage when appending objects
 - index-pack: rationalize unpack_entry_data()
 - index-pack: smarter memory usage when resolving deltas

* jg/auto-initialize-notes-with-percent-n-in-format (2010-04-13) 1 commit
 - pretty: Initialize notes if %N is used

* jc/am-3-show-corrupted-patch (2010-04-09) 1 commit
 - am -3: recover the diagnostic messages for corrupt patches

* jc/maint-no-reflog-expire-unreach-for-head (2010-04-09) 3 commits
 - reflog: ignore expire-unreachable for "HEAD" reflog
 - more war on "sleep" in tests
 - Document gc.<pattern>.reflogexpire variables
 (this branch uses jc/maint-reflog-expire-unreachable.)

* cw/maint-exec-defpath (2010-04-13) 1 commit
 - exec_cmd.c: replace hard-coded path list with one from <paths.h>

--------------------------------------------------
[Stalled]

* rr/remote-helper-doc (2010-04-07) 3 commits
 - Documentation/remote-helpers: Add invocation section
 - Documentation/urls: Rewrite to accomodate <transport>::<address>
 - Documentation/remote-helpers: Rewrite description

I'd like acks or rewrites from transport people on this topic.

* js/rebase-origin-x (2010-02-05) 1 commit
 - [RFC w/o test and incomplete] rebase: add -x option to record original commit name

I retract my objection against the idea of -x; needs polishing before
moving forward.

* js/grep-open (2010-03-26) 2 commits
 - grep -O: allow optional argument specifying the pager (or editor)
 - grep: Add the option '--open-files-in-pager'
 (this branch is used by jp/hold-string-list-sanity.)

Probably needs to support --no-index mode as well.

* jp/hold-string-list-sanity (2010-04-06) 9 commits
 . string_list: Fix argument order for string_list_append
 . Merge branch 'sr/remote-helper-export' into HEAD
 . Merge branch 'js/grep-open' into HEAD
 . Merge branch 'sb/fmt-merge-msg' into HEAD
 . string_list: Fix argument order for string_list_lookup
 . string_list: Fix argument order for string_list_insert_at_index
 . string_list: Fix argument order for string_list_insert
 . string_list: Fix argument order for for_each_string_list
 . string_list: Fix argument order for print_string_list
 (this branch uses js/grep-open and sr/remote-helper-export.)

Building this on top of slushy codebase is not a very promising endeavor.
Good thing to do, but bad timing.

--------------------------------------------------
[Cooking]

* tr/word-diff (2010-04-14) 1 commit
 - diff: add --word-diff option that generalizes --color-words

There are gitk patches that go with this series.

* sp/maint-describe-tiebreak-with-tagger-date (2010-04-12) 5 commits
 - describe: Break annotated tag ties by tagger date
 - tag.c: Parse tagger date (if present)
 - tag.c: Refactor parse_tag_buffer to be saner to program
 - tag.h: Remove unused signature field
 - tag.c: Correct indentation

* bg/apply-blank-trailing-context (2010-04-08) 1 commit
  (merged to 'next' on 2010-04-14 at a393c0b)
 + apply: Allow blank *trailing* context lines to match beyond EOF

* bg/send-email-smtpdomain (2010-04-10) 4 commits
  (merged to 'next' on 2010-04-14 at 88fff99)
 + send-email: Cleanup smtp-domain and add config
 + Document send-email --smtp-domain
 + send-email: Don't use FQDNs without a '.'
 + send-email: Cleanup { style

* rc/ls-remote-default (2010-04-09) 1 commit
  (merged to 'next' on 2010-04-14 at d5081bd)
 + ls-remote: fall-back to default remotes when no remote specified

* rc/maint-curl-helper (2010-04-08) 3 commits
  (merged to 'next' on 2010-04-14 at 2eb9177)
 + remote-curl: ensure that URLs have a trailing slash
 + http: make end_url_with_slash() public
 + t5541-http-push: add test for URLs with trailing slash

* hg/maint-attr-fix (2010-04-06) 3 commits
  (merged to 'next' on 2010-04-14 at a27c7d7)
 + attr: Expand macros immediately when encountered.
 + attr: Allow multiple changes to an attribute on the same line.
 + attr: Fixed debug output for macro expansion.

* jc/status-show-ignored (2010-04-10) 5 commits
 - status: --ignored option shows ignored files
 - wt-status: rename and restructure status-print-untracked
 - wt-status: collect ignored files
 - wt-status: plug memory leak while collecting untracked files
 - wt-status: remove unused workdir_untracked member

This is merely an illustration patch.  I am negative about conflating the
JSON output to the mix at this point.  If we are going to add JSON option
to everywhere we would generate "machine readable" output with the -z
and/or --porcelain options, that is fine, but let's stabilize "status"
with the existing --porcelain/-z option first.

* eb/unpretty-b-format (2010-03-24) 1 commit
  (merged to 'next' on 2010-04-07 at 7f5c112)
 + Add `%B' in format strings for raw commit body in `git log' and friends

* ab/commit-empty-message (2010-04-06) 1 commit
  (merged to 'next' on 2010-04-07 at 701e863)
 + Add option to git-commit to allow empty log messages

* jc/test-sleepless (2010-04-06) 1 commit
  (merged to 'next' on 2010-04-07 at 8ccf40a)
 + war on "sleep" in tests

* jc/maint-reflog-expire-unreachable (2010-04-07) 1 commit
 - reflog --expire-unreachable: avoid merge-base computation
 (this branch is used by jc/maint-no-reflog-expire-unreach-for-head.)

* np/malloc-threading (2010-04-08) 2 commits
  (merged to 'next' on 2010-04-10 at e1730fb)
 + Thread-safe xmalloc and xrealloc needs a recursive mutex
 + Make xmalloc and xrealloc thread-safe

The fix should eventually go to 'maint' and 'master'.  This queues the one
from J6t for Windows compatibility.

* js/async-thread (2010-03-09) 7 commits
 - Enable threaded async procedures whenever pthreads is available
  (merged to 'next' on 2010-03-20 at 9939243)
 + Dying in an async procedure should only exit the thread, not the process.
 + Reimplement async procedures using pthreads
 + Windows: more pthreads functions
 + Fix signature of fcntl() compatibility dummy
 + Make report() from usage.c public as vreportf() and use it.
 + Modernize t5530-upload-pack-error.

Been waiting for resolution of locking issues in malloc-threading.

* sd/log-decorate (2010-04-08) 6 commits
  (merged to 'next' on 2010-04-08 at 68fee56)
 + log.decorate: only ignore it under "log --pretty=raw"
 + script with rev-list instead of log
  (merged to 'next' on 2010-04-06 at 992c9ad)
 + log --pretty/--oneline: ignore log.decorate
  (merged to 'next' on 2010-03-08 at 58a6fba)
 + log.decorate: usability fixes
 + Add `log.decorate' configuration variable.
 + git_config_maybe_bool()

* mh/status-optionally-refresh (2010-04-03) 3 commits
  (merged to 'next' on 2010-04-05 at 0e64aac)
 + t7508: add a test for "git status" in a read-only repository
 + git status: refresh the index if possible
 + t7508: add test for "git status" refreshing the index

* cw/ws-indent-with-tab (2010-04-03) 6 commits
  (merged to 'next' on 2010-04-05 at 5b5e579)
 + whitespace: tests for git-apply --whitespace=fix with tab-in-indent
 + whitespace: add tab-in-indent support for --whitespace=fix
 + whitespace: replumb ws_fix_copy to take a strbuf *dst instead of char *dst
 + whitespace: tests for git-diff --check with tab-in-indent error class
 + whitespace: add tab-in-indent error class
 + whitespace: we cannot "catch all errors known to git" anymore

* sr/remote-helper-export (2010-04-12) 9 commits
  (merged to 'next' on 2010-04-12 at cefbb2f)
 + t5800: testgit helper requires Python support
  (merged to 'next' on 2010-04-09 at 94f6e3d)
 + Makefile: Simplify handling of python scripts
  (merged to 'next' on 2010-04-07 at 5651307)
 + remote-helpers: add tests for testgit helper
 + remote-helpers: add testgit helper
 + remote-helpers: add support for an export command
 + remote-helpers: allow requesing the path to the .git directory
 + fast-import: always create marks_file directories
 + clone: also configure url for bare clones
 + clone: pass the remote name to remote_get
 (this branch is used by jp/hold-string-list-sanity.)

* cc/revert-strategy (2010-03-31) 5 commits
  (merged to 'next' on 2010-04-05 at 50909c1)
 + revert: add "--strategy" option to choose merge strategy
 + merge: make function try_merge_command non static
 + merge: refactor code that calls "git merge-STRATEGY"
 + revert: refactor merge recursive code into its own function
 + revert: use strbuf to refactor the code that writes the merge message

This is more about debugging and we are not in a great hurry.

* sc/http-late-auth (2010-04-01) 1 commit
  (merged to 'next' on 2010-04-02 at c991acf)
 + Prompt for a username when an HTTP request 401s

* jk/cached-textconv (2010-04-01) 5 commits
  (merged to 'next' on 2010-04-02 at a023e3c)
 + diff: avoid useless filespec population
 + diff: cache textconv output
 + textconv: refactor calls to run_textconv
 + introduce notes-cache interface
 + make commit_tree a library function

The small leak fixes in the beginning of the series has been merged to
'master' after a minor fixup.

* ld/discovery-limit-to-fs (2010-04-04) 6 commits
 - write-index: check and warn when worktree crosses a filesystem boundary
  (merged to 'next' on 2010-04-07 at 11ea09a)
 + Rename ONE_FILESYSTEM to DISCOVERY_ACROSS_FILESYSTEM
 + GIT_ONE_FILESYSTEM: flip the default to stop at filesystem boundaries
 + Add support for GIT_ONE_FILESYSTEM
 + truncate cwd string before printing error message
 + config.c: remove static keyword from git_env_bool()

The tip one is a bit iffy; the whole series changes behaviour in a corner
case, and is not a 1.7.1 material.

* pc/remove-warn (2010-03-26) 4 commits
  (merged to 'next' on 2010-04-02 at 52fc00d)
 + Remove a redundant errno test in a usage of remove_path
 + Introduce remove_or_warn function
 + Implement the rmdir_or_warn function
 + Generalise the unlink_or_warn function

I cherry-picked only obviously sensible bits.

* em/checkout-orphan (2010-03-21) 1 commit
  (merged to 'next' on 2010-03-28 at 86b6a4f)
 + git checkout: create unparented branch by --orphan

Perhaps needs a bit of documentation updates, describing the "going open
source" scenario.

* ar/config-from-command-line (2010-03-26) 2 commits
  (merged to 'next' on 2010-04-07 at e50fd3a)
 + Use strbufs instead of open-coded string manipulation
 + Allow passing of configuration parameters in the command line

--------------------------------------------------
[Ejected from 'pu']

* ne/rev-cache (2010-04-05) 7 commits
 . graft awareness
 . object name support
 . integration into revision walker
 . administrative api and tools
 . support for non-commit objects
 . basic api and porcelain
 . man page and technical docs

For some reason this was extremely hard to read, partly because it had too
many distracting style violations and too many long lines.

This duplicates quite a similar traversal logic and hijacks the control
from sideways in a very core part of the revision walker, making it a
maintenance nightmare.  After I noticed that it still writes/reads
structures with bitfields directly on-disk, which I pointed out number of
times is a no-no during the initial review, I decided to eject this round
for now.

* nd/setup (2010-04-05) 43 commits
 . builtins: do not commit pager choice early
 . builtins: setup repository before print unknown command error
 . t0001: Add test cases for "git init" with aliases
 . Guard unallowed access to repository when it's not set up
 . alias: keep repository found while collecting aliases as long as possible
 . Allow to undo setup_git_directory_gently() gracefully (and fix alias code)
 . builtins: check for startup_info->help, print and exit early
 . builtins: utilize startup_info->help where possible
 . run_builtin(): save "-h" detection result for later use
 . config: do not read .git/config if there is no repository
 . apply: do not check sha1 if there is no repository
 . Do not read .git/info/attributes if there is no repository
 . Do not read .git/info/exclude if there is no repository
 . git_config(): do not read .git/config if there is no repository
 . init/clone: turn on startup->have_repository properly
 . worktree setup: restore original state when things go wrong
 . Use git_config_early() instead of git_config() during repo setup
 . Add git_config_early()
 . worktree setup: call set_git_dir explicitly
 . rev-parse --git-dir: print relative gitdir correctly
 . enter_repo(): initialize other variables as setup_git_directory_gently() does
 . Move enter_repo() to setup.c
 . index-pack: use RUN_SETUP_GENTLY
 . index-pack: trust the prefix returned by setup_git_directory_gently()
 . worktree setup: calculate prefix even if no worktree is found
 . merge-file: use RUN_SETUP_GENTLY
 . var: use RUN_SETUP_GENTLY
 . ls-remote: use RUN_SETUP_GENTLY
 . help: take note why this command is not applicable for RUN_SETUP_GENTLY
 . diff: use RUN_SETUP_GENTLY
 . bundle: use RUN_SETUP_GENTLY
 . apply: use RUN_SETUP_GENTLY
 . verify-pack: use RUN_SETUP_GENTLY
 . check-ref-format: use RUN_SETUP_GENTLY
 . mailinfo: use RUN_SETUP_GENTLY
 . archive: use RUN_SETUP_GENTLY
 . builtin: USE_PAGER should not be used without RUN_SETUP*
 . grep: use RUN_SETUP_GENTLY
 . shortlog: use RUN_SETUP_GENTLY
 . hash-object: use RUN_SETUP_GENTLY
 . config: use RUN_SETUP_GENTLY
 . builtin: Support RUN_SETUP_GENTLY to set up repository early if found
 . builtin: introduce startup_info struct

Jonathan Nieder is disecting this series into more easily digestible
chunks; in the meantime I'll get out of the way not to step on the toes of
his effort.

Re: What's cooking in git.git (Apr 2010, #05; Wed, 14)

[Ramkumar Ramachandra]

HI,

> * rr/remote-helper-doc (2010-04-07) 3 commits
>  - Documentation/remote-helpers: Add invocation section
>  - Documentation/urls: Rewrite to accomodate <transport>::<address>
>  - Documentation/remote-helpers: Rewrite description
>
> I'd like acks or rewrites from transport people on this topic.

Can I do something to help?

-- Ram

Re: What's cooking in git.git (Apr 2010, #05; Wed, 14)

[Johannes Sixt]

Am 4/15/2010 1:37, schrieb Junio C Hamano:
> * ld/discovery-limit-to-fs (2010-04-04) 6 commits
>  - write-index: check and warn when worktree crosses a filesystem boundary
>   (merged to 'next' on 2010-04-07 at 11ea09a)
>  + Rename ONE_FILESYSTEM to DISCOVERY_ACROSS_FILESYSTEM
>  + GIT_ONE_FILESYSTEM: flip the default to stop at filesystem boundaries
>  + Add support for GIT_ONE_FILESYSTEM
>  + truncate cwd string before printing error message
>  + config.c: remove static keyword from git_env_bool()
> 
> The tip one is a bit iffy; the whole series changes behaviour in a corner
> case, and is not a 1.7.1 material.

The problem I reported after your previous announcement (that I always see
the warning on Windows) was a false alarm. Appearently, I still had some
random st_dev values recorded in the index for some old files, like
Documentation/RelNotes-1.5.4.*, and it went away after I rebuilt the index.

-- Hannes

Re: What's cooking in git.git (Apr 2010, #05; Wed, 14)

[Nick Edelen]

>* ne/rev-cache (2010-04-05) 7 commits
> . graft awareness
> . object name support
> . integration into revision walker
> . administrative api and tools
> . support for non-commit objects
> . basic api and porcelain
> . man page and technical docs
>
>For some reason this was extremely hard to read, partly because it had too
>many distracting style violations and too many long lines.

What sort of style violations?  I tried to be quite careful about
that, but evidently I wasn't successful :-/

I'm sorry about the long lines -- all but a handful should fit on a
standard xga resolution screen, which I figured was something of a
minimum for screen width now?

>This duplicates quite a similar traversal logic and hijacks the control
>from sideways in a very core part of the revision walker, making it a
>maintenance nightmare.

I realize that might be a problem, but it seems to be sorta
unavoidable.  The goal is to allow highly-efficient retrieval of
metadata in revision walking, and in order to figure out what metadata
is required (preferably with minimum redundancy), at least some
portion of the revision walker is going to be duplicated.

It was intended to something of an alternative walker in its own
right, which could supplement standard revision traversal by passing
control back and forth based on what was available in the caches.  In
this way the API was to additionally provide a possible doorway for
third-parties to have ready, fast access to the commit tree, without
trudging through individual object files.

I don't think maintenance would really be that nightmarish.  First,
rev-cache is quite distinct from git's core, and is only (tentatively)
integrated by a few lines of code.  It keeps its own cache-slice
format independent from git's objects, and is transparent to almost
every git function (so, e.g., completely changing git's daemon would
have no influence on the rev-cache's effectiveness, as long as
revision walking is not significantly altered).

Second, the key logic is does duplicate (= mainly the interestingness)
is so fundamental that any large change would require a huge rewrite
of most of gits internals anyway, let alone rev-cache.  Other
duplication (e.g. max/min dates) is no harder to alter than in
revision.c; indeed, most of the time it'd be simpler.  Additionally,
for anything more complicated than simple traversal (e.g. pruning)
revision caching is disabled.

Care is taken to make it as unobtrusive as possible: if it can provide
extra help, it will, but as far as git's concerned it dosn't exist.

>After I noticed that it still writes/reads
>structures with bitfields directly on-disk, which I pointed out number of
>times is a no-no during the initial review

Unless I missed something, I had thought I specifically addressed this
issue.  I even went so far as to prevent direct structure writes of
any kind (contrary to the read-cache.c functions, my initial guide for
such strictness), explicitly converting structures to/from char
arrays.

 - Nick