git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jonathan Nieder <jrnieder@gmail.com>
To: Jeff King <peff@peff.net>
Cc: git@vger.kernel.org
Subject: Re: [PATCH v2 7/8] verify_path(): disallow symlinks in .gitattributes and .gitignore
Date: Mon, 26 Oct 2020 20:35:18 -0700	[thread overview]
Message-ID: <20201027033518.GH2645313@google.com> (raw)
In-Reply-To: <20201005121645.GG2907394@coredump.intra.peff.net>

Hi,

Jeff King wrote:

> However, it's still a reasonable idea to forbid symlinks for these
> files:
>
>   - As noted, they can still be used to read out-of-repo files (which is
>     fairly restricted, but in some circumstances you can probe file
>     content by speculatively creating files and seeing if they get
>     ignored)
>
>   - They don't currently behave well in all cases. We sometimes read
>     these files from the index, where we _don't_ follow symlinks (we'd
>     just treat the symlink target as the .gitignore or .gitattributes
>     content, which is actively wrong).
>
> This patch forbids symlinked versions of these files from entering the
> index. We already have helpers for obscured forms of the names from
> e7cb0b4455 (is_ntfs_dotgit: match other .git files, 2018-05-11) and
> 0fc333ba20 (is_hfs_dotgit: match other .git files, 2018-05-02), which
> were done as part of the series touching .gitmodules.

Thanks again for this.  Since this patch has been in "next", we've
gotten a little experience of it at Google.

We've been running with the fsck check for a while (more than a year),
but not the verify_dotfile check.  The verify_dotfile check didn't
trigger for anyone with .gitattributes, but it hit several people for
.gitignore.  Some examples that users have mentioned:

Before https://android-review.googlesource.com/c/platform/tools/test/connectivity/+/1462771/
Android used a .gitignore symlink for two directories that had similar
gitignore requirements.  Diagnosing the error was confusing for them,
especially because the "repo" wrapper tool produced messages like

	error.GitError: Cannot initialize work tree for platform/tools/test/connectivity

Eventually someone manually ran

	$ git add --a
	error: invalid path 'acts_tests/.gitignore'
	error: unable to add 'acts_tests/.gitignore' to index
	fatal: adding files failed

which helped them realize it was a git issue and helped me point them
to the need to replace the symlink with a plain file.

As another example, a user working with the
https://github.com/bakerstu/openmrn.git repository noticed "git
checkout" commands failing.  In this user's case, the checkout failed
part-way through, producing confusing behavior ("git status" showing
entries missing from the index).  When I tried to reproduce this, I
wasn't able to clone the repository at all because it failed fsck;
after disabling transfer.fsckObjects, I still wasn't able to check out
HEAD.

Observations:

- since some widely used repositories have .gitignore symlinks, I
  think we can't forbid it in fsck, alas

- it would be useful to be able to check whether these symlinks would
  not escape the worktree, for a more targeted check.  It might be
  nice to even respect these settings when they would not escape the
  worktree, but not necessarily

- we could use a clearer error message than "invalid path".  There's
  some room for improvement in "git checkout"'s error handling, too
  --- I think my ideal would be if the operation would fail entirely,
  with an advice message describing a checkout command that would
  succeed (But how do I checkout another commit while excluding some
  files? Should it suggest a sparse checkout?).

That's all I have time for today --- will revisit again tomorrow.

Thoughts?

Thanks,
Jonathan

  reply	other threads:[~2020-10-27  3:35 UTC|newest]

Thread overview: 55+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-05  7:17 [PATCH 0/7] forbidding symlinked .gitattributes and .gitignore Jeff King
2020-10-05  7:19 ` [PATCH 1/7] fsck_tree(): fix shadowed variable Jeff King
2020-10-05  7:44   ` Jonathan Nieder
2020-10-05  8:20     ` Jeff King
2020-10-05  8:29       ` Jonathan Nieder
2020-10-05  7:19 ` [PATCH 2/7] fsck_tree(): wrap some long lines Jeff King
2020-10-05  7:46   ` Jonathan Nieder
2020-10-05  7:19 ` [PATCH 3/7] t7415: rename to expand scope Jeff King
2020-10-05  7:50   ` Jonathan Nieder
2020-10-05  8:24     ` Jeff King
2020-10-05  8:34       ` Jonathan Nieder
2020-10-05  8:49         ` Jeff King
2020-10-05  7:20 ` [PATCH 4/7] t7450: test verify_path() handling of gitmodules Jeff King
2020-10-05  7:53   ` Jonathan Nieder
2020-10-05  8:30     ` Jeff King
2020-10-05  8:38       ` Jonathan Nieder
2020-10-05  7:21 ` [PATCH 5/7] t0060: test obscured .gitattributes and .gitignore matching Jeff King
2020-10-05  8:03   ` Jonathan Nieder
2020-10-05  8:40     ` Jeff King
2020-10-05 21:20       ` Johannes Schindelin
2020-10-06 14:01         ` Jeff King
2020-10-05  7:24 ` [PATCH 6/7] verify_path(): disallow symlinks in .gitattributes and .gitignore Jeff King
2020-10-05  8:09   ` Jonathan Nieder
2020-10-05 12:07     ` Jeff King
2020-10-05  7:25 ` [PATCH 7/7] fsck: complain when .gitattributes or .gitignore is a symlink Jeff King
2020-10-05  8:12   ` Jonathan Nieder
2020-10-05  8:53     ` Jeff King
2020-10-05  7:32 ` [PATCH 0/7] forbidding symlinked .gitattributes and .gitignore Jonathan Nieder
2020-10-05  8:58   ` Jeff King
2020-10-05 12:16 ` [PATCH v2 0/8] " Jeff King
2020-10-05 12:16   ` [PATCH v2 1/8] fsck_tree(): fix shadowed variable Jeff King
2020-10-05 12:16   ` [PATCH v2 2/8] fsck_tree(): wrap some long lines Jeff King
2020-10-05 12:16   ` [PATCH v2 3/8] t7415: rename to expand scope Jeff King
2020-10-05 12:16   ` [PATCH v2 4/8] t7450: test verify_path() handling of gitmodules Jeff King
2020-10-05 12:16   ` [PATCH v2 5/8] t7450: test .gitmodules symlink matching against obscured names Jeff King
2020-10-05 12:16   ` [PATCH v2 6/8] t0060: test obscured .gitattributes and .gitignore matching Jeff King
2020-10-05 12:16   ` [PATCH v2 7/8] verify_path(): disallow symlinks in .gitattributes and .gitignore Jeff King
2020-10-27  3:35     ` Jonathan Nieder [this message]
2020-10-27  7:58       ` Jeff King
2020-10-27 22:00         ` Junio C Hamano
2020-10-28  9:41           ` Jeff King
2020-10-27 23:43         ` Jonathan Nieder
2020-10-28 19:18           ` Junio C Hamano
2020-10-05 12:16   ` [PATCH v2 8/8] fsck: complain when .gitattributes or .gitignore is a symlink Jeff King
2020-10-06 20:41   ` [PATCH v2 0/8] forbidding symlinked .gitattributes and .gitignore Junio C Hamano
2020-10-20 23:19   ` Philip Oakley
2020-10-23  8:17     ` [PATCH] documentation symlink restrictions for .git* files Jeff King
2020-10-23  8:27       ` Jeff King
2020-10-26 22:18       ` Philip Oakley
2020-10-26 22:53         ` Jeff King
2020-10-26 23:32           ` Junio C Hamano
2020-10-27  7:26             ` Jeff King
2020-10-27 18:45               ` Junio C Hamano
2020-10-27 21:00                 ` Philip Oakley
2020-10-28 19:14                   ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201027033518.GH2645313@google.com \
    --to=jrnieder@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).