git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jeff King <peff@peff.net>
To: Junio C Hamano <gitster@pobox.com>
Cc: Christian Couder <christian.couder@gmail.com>, git@vger.kernel.org
Subject: [PATCH 0/9] trailer-parsing false positives
Date: Wed, 22 Aug 2018 20:43:01 -0400	[thread overview]
Message-ID: <20180823004300.GA1355@sigill.intra.peff.net> (raw)
In-Reply-To: <xmqq36v78ml8.fsf@gitster-ct.c.googlers.com>

On Tue, Aug 21, 2018 at 01:57:07PM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > Ah, yeah, I think you're right. We call find_patch_start(), which thinks
> > the "---" line is the end of the commit message. That makes sense when
> > parsing trailers out of "format-patch" output, but not when we know we
> > have just the commit message.
> 
> Yes, but that does not explain what we are seeing.  If the code
> mistakenly thinks that the log message ends before that table, then
> it should have inserted the S-o-b: _before_ that table, but that is
> not happening.
> 
> So there are three issues; (1) find-patch-start uses too weak a
> logic to find the beginning of a patch section (2) even if it found
> the right place, its caller does not tell "commit --amend -s" where
> the log message ends correctly and (3) some callchains that get
> there know they only have a log message but there is no way to take
> advantage of that information and skip the call to find-patch-start.

So this turned into a bit of a rabbit hole. Here's what I have so far
(which I think is not quite ready, but I wanted to start discussing).

Issues (2) and (3) are actually the same issue. The caller that does the
bogus appending for (2) is always append_signoff(). But it always has
just a commit message (modulo some complications, which I'll get to in a
minute), and so fixing it with respect to (3) magically solves (2).
I.e., the code was simply not prepared for the case of "end of string is
not end of trailers". But since that case cannot exist, we do not have
to deal with it. :)

Now here's the tricky part. I think patches 1-8 are mostly sensible. But
while doing 7/8, I noticed some weirdness around the ignore_footer
parameter. It seems git-commit strips some cruft off the end of the
buffer, including "#" comments, before looking for the trailers. But
that doesn't make any sense. We should just handle the cleaned-up commit
message. Worse, the stripping does not take into account options like
--cleanup=verbatim, where "#" comments are _not_ cruft anymore. Double
worse, it uses the current notion of core.commentChar, but we might be
operating on historical data (i.e., something created with a different
comment char). Triple-worse, this same cleanup is used in the trailer
parsing code!

So:

  git interpret-trailers --parse <<-\EOF
  subject

  body

  Signed-off-by: me

  # is this a comment or not?
  EOF

will find that s-o-b. Should that "#" thing be a comment? I'd say no. We
are not dealing with a commit message template at all. And even in
git-commit, I think we aren't (or at least ought not to be) passing the
template around to the signoff code.

So I think there may be further opportunities for cleanup here. I'm not
sure if we'd need to retain this behavior for git-interpret-trailers.
AFAICT it is not documented, and I suspect is mostly historical
accident, and not anything anybody ever wanted.

If we do keep it by default, then the "--no-divider" option I added in
patch 4 should probably get a more generic name and cover this.
Something like "--verbatim-input".

I'm going to sleep on it and see how I feel tomorrow.

  [1/9]: trailer: use size_t for string offsets
  [2/9]: trailer: use size_t for iterating trailer list
  [3/9]: trailer: pass process_trailer_opts to trailer_info_get()
  [4/9]: interpret-trailers: tighten check for "---" patch boundary
  [5/9]: interpret-trailers: allow suppressing "---" divider
  [6/9]: pretty, ref-filter: format %(trailers) with no_divider option
  [7/9]: sequencer: ignore "---" divider when parsing trailers
  [8/9]: append_signoff: use size_t for string offsets
  [9/9]: sequencer: handle ignore_footer when parsing trailers

 Documentation/git-interpret-trailers.txt | 10 +++-
 builtin/interpret-trailers.c             |  1 +
 commit.c                                 |  6 +--
 commit.h                                 |  2 +-
 pretty.c                                 |  3 ++
 ref-filter.c                             |  2 +
 sequencer.c                              | 20 ++++++--
 sequencer.h                              |  9 +++-
 t/t4205-log-pretty-formats.sh            | 23 +++++++++
 t/t6300-for-each-ref.sh                  | 23 +++++++++
 t/t7501-commit.sh                        | 16 ++++++
 t/t7513-interpret-trailers.sh            | 42 ++++++++++++++++
 trailer.c                                | 62 +++++++++++++-----------
 trailer.h                                |  4 +-
 14 files changed, 184 insertions(+), 39 deletions(-)

-Peff

  parent reply	other threads:[~2018-08-23  0:43 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-21 18:41 [PATCH] test-tool.h: include git-compat-util.h Jeff King
2018-08-21 19:03 ` Junio C Hamano
2018-08-21 19:06 ` [PATCH 1/6] t/perf: factor boilerplate out of test_perf Jeff King
2018-08-21 19:06 ` [PATCH 2/6] t/perf: factor out percent calculations Jeff King
2018-08-21 19:06 ` [PATCH 3/6] t/perf: add infrastructure for measuring sizes Jeff King
2018-08-22 13:40   ` Derrick Stolee
2018-08-22 15:31     ` Jeff King
2018-08-21 19:06 ` [PATCH 4/6] t/perf: add perf tests for fetches from a bitmapped server Jeff King
2018-08-21 19:07 ` [PATCH 5/6] pack-bitmap: save "have" bitmap from walk Jeff King
2018-08-21 19:47   ` Derrick Stolee
2018-08-21 19:54     ` Jeff King
2018-08-31 15:23   ` Ævar Arnfjörð Bjarmason
2018-08-31 22:55     ` Jeff King
2018-09-01  7:41       ` [PATCH 0/4] un-breaking pack-objects with bitmaps Jeff King
2018-09-01  7:44         ` [PATCH 1/4] bitmap_has_sha1_in_uninteresting(): drop BUG check Jeff King
2018-09-01  7:48         ` [PATCH 2/4] t5310: test delta reuse with bitmaps Jeff King
2018-09-01  8:03           ` Jeff King
2018-09-01 20:29             ` Ævar Arnfjörð Bjarmason
2018-09-01 22:46               ` Ben Peart
2018-09-02  5:51               ` Jeff King
2018-09-04 19:05             ` Stefan Beller
2018-09-04 19:45               ` Junio C Hamano
2018-09-04 20:02               ` Jeff King
2018-09-01  7:49         ` [PATCH 3/4] traverse_bitmap_commit_list(): don't free result Jeff King
2018-09-01  7:50         ` [PATCH 4/4] pack-bitmap: drop "loaded" flag Jeff King
2018-09-04 19:30         ` [PATCH 0/4] un-breaking pack-objects with bitmaps Stefan Beller
2018-09-04 20:03           ` Jeff King
2018-09-08  6:43         ` Ævar Arnfjörð Bjarmason
2018-09-10 16:53           ` Junio C Hamano
2018-09-10 18:48             ` Jeff King
2018-09-10 19:23               ` Junio C Hamano
2018-08-21 19:07 ` [PATCH 6/6] pack-objects: reuse on-disk deltas for thin "have" objects Jeff King
2018-08-21 19:43   ` Junio C Hamano
2018-08-21 19:50     ` Junio C Hamano
2018-08-21 20:07       ` Jeff King
2018-08-21 20:14         ` Jeff King
2018-08-21 20:52           ` Junio C Hamano
2018-08-21 21:30             ` Jeff King
2018-08-21 20:57         ` Junio C Hamano
2018-08-21 21:32           ` Jeff King
2018-08-23  0:43           ` Jeff King [this message]
2018-08-23  0:44             ` [PATCH 1/9] trailer: use size_t for string offsets Jeff King
2018-08-23  0:45             ` [PATCH 2/9] trailer: use size_t for iterating trailer list Jeff King
2018-08-23  0:46             ` [PATCH 3/9] trailer: pass process_trailer_opts to trailer_info_get() Jeff King
2018-08-23  0:48             ` [PATCH 4/9] interpret-trailers: tighten check for "---" patch boundary Jeff King
2018-08-23  0:49             ` [PATCH 5/9] interpret-trailers: allow suppressing "---" divider Jeff King
2018-08-23  0:50             ` [PATCH 6/9] pretty, ref-filter: format %(trailers) with no_divider option Jeff King
2018-08-23  0:50             ` [PATCH 7/9] sequencer: ignore "---" divider when parsing trailers Jeff King
2018-08-23  0:50             ` [PATCH 8/9] append_signoff: use size_t for string offsets Jeff King
2018-08-23  0:51             ` [PATCH 9/9] sequencer: handle ignore_footer when parsing trailers Jeff King
2018-08-23 18:30             ` [PATCH 0/9] trailer-parsing false positives Junio C Hamano
2018-08-24  7:26               ` Jeff King
2018-08-21 20:00     ` [PATCH 6/6] pack-objects: reuse on-disk deltas for thin "have" objects Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180823004300.GA1355@sigill.intra.peff.net \
    --to=peff@peff.net \
    --cc=christian.couder@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --subject='Re: [PATCH 0/9] trailer-parsing false positives' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).