From: "Philip Oakley" <philipoakley@iee.org>
To: "Jeff King" <peff@peff.net>,
"Jeff Mitchell" <jeffrey.mitchell@gmail.com>
Cc: "Duy Nguyen" <pclouds@gmail.com>,
"Ævar Arnfjörð" <avarab@gmail.com>,
"Git List" <git@vger.kernel.org>
Subject: Re: propagating repo corruption across clone
Date: Tue, 26 Mar 2013 21:59:42 -0000 [thread overview]
Message-ID: <102DBFFC4475445D9180A9C7D2A9D97C@PhilipOakley> (raw)
In-Reply-To: 20130326165553.GA7282@sigill.intra.peff.net
From: "Jeff King" <peff@peff.net>
Sent: Tuesday, March 26, 2013 4:55 PM
> On Tue, Mar 26, 2013 at 09:43:01AM -0400, Jeff Mitchell wrote:
>
>> On Mon, Mar 25, 2013 at 4:07 PM, Jeff King <peff@peff.net> wrote:
>> > On Mon, Mar 25, 2013 at 12:32:50PM -0400, Jeff Mitchell wrote:
>> >> For commit corruptions, the --no-hardlinks, non --mirror case
>> >> refused
>> >> to create the new repository and exited with an error code of 128.
>> >> The
>> >> --no-hardlinks, --mirror case spewed errors to the console, yet
>> >> *still* created the new clone *and* returned an error code of
>> >> zero.
>> >
>> > I wasn't able to reproduce this; can you post a succint test case?
>>
>> [...link to tar.gz...]
>> Once you extract that, you should be able to run a clone using paths
>> (not file://) with --no-hardlinks --mirror and replicate the behavior
>> I saw. FYI, I'm on Git 1.8.2.
>
> Thanks for providing an example.
>
> The difference is the same "--mirror implies --bare" issue; the
> non-bare
> case dies during the checkout (even before my patches, as the
> corruption
> is not in a blob, but rather in the HEAD commit object itself). You
> can
> replace --mirror with --bare and see the same behavior.
>
> The troubling part is that we see errors in the bare case, but do not
> die. Those errors all come from upload-pack, the "sending" side of a
> clone/fetch. Even though we do not transfer the objects via the git
> protocol, we still invoke upload-pack to get the ref list (and then
> copy
> the objects themselves out-of-band).
>
> What happens is that upload-pack sees the errors while trying to see
> if
> the object is a tag that can be peeled (the server advertises both
> tags
> and the objects they point to). It does not distinguish between
> "errors
> did not let me peel this object" and "this object is not a tag, and
> therefore there is nothing to peel".
>
> We could change that, but I'm not sure whether it is a good idea. I
> think the intent is that upload-pack's ref advertisement would remain
> resilient to corruption in the repository (e.g., even if that commit
> is
> corrupt, you can still fetch the rest of the data). We should not
> worry
> about advertising broken objects, because we will encounter the same
> error when we actually do try to send the objects. Dying at the
> advertisement phase would be premature, since we do not yet know what
> the client will request.
>
> The problem, of course, is that the --local optimization _skips_ the
> part where we actually ask upload-pack for data, and instead blindly
> copies it. So this is the same issue as usual, which is that the local
> transport is not thorough enough to catch corruption. It seems like a
> failing in this case, because upload-pack does notice the problem, but
> that is only luck; if the corruption were in a non-tip object, it
> would
> not notice it at all. So trying to die on errors in the ref
> advertisement would just be a band-aid. Fundamentally the problem is
> that the --local transport is not safe from propagating corruption,
> and
> should not be used if that's a requirement.
>
> -Peff
> --
Which way does `git bundle file.bundl --all` perform after the changes
for both the 'transport' checking and being reliable during updates.
Is it an option for creating an archivable file that can be used for a
later `clone`?
I wasn't sure if the bundle capability had been considered.
Philip
next prev parent reply other threads:[~2013-03-26 22:00 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-03-24 18:31 propagating repo corruption across clone Jeff King
2013-03-24 19:01 ` Ævar Arnfjörð Bjarmason
2013-03-24 19:23 ` Jeff King
2013-03-25 13:43 ` Jeff Mitchell
2013-03-25 14:56 ` Jeff King
2013-03-25 15:31 ` Duy Nguyen
2013-03-25 15:56 ` Jeff King
2013-03-25 16:32 ` Jeff Mitchell
2013-03-25 20:07 ` Jeff King
2013-03-26 13:43 ` Jeff Mitchell
2013-03-26 16:55 ` Jeff King
2013-03-26 21:59 ` Philip Oakley [this message]
2013-03-26 22:03 ` Jeff King
2013-03-26 23:20 ` Rich Fromm
2013-03-27 1:25 ` Jonathan Nieder
2013-03-27 18:23 ` Rich Fromm
2013-03-27 19:49 ` Jeff King
2013-03-27 20:04 ` Jeff King
2013-03-27 3:47 ` Junio C Hamano
2013-03-27 6:19 ` Sitaram Chamarty
2013-03-27 15:03 ` Junio C Hamano
2013-03-27 15:47 ` Sitaram Chamarty
2013-03-27 18:51 ` Rich Fromm
2013-03-27 19:13 ` Junio C Hamano
2013-03-28 13:52 ` Jeff Mitchell
2013-03-28 13:48 ` Jeff Mitchell
2013-03-26 1:06 ` Duy Nguyen
2013-03-24 19:16 ` Ilari Liusvaara
2013-03-25 20:01 ` Junio C Hamano
2013-03-25 20:05 ` Jeff King
2013-03-25 20:14 ` [PATCH 0/9] corrupt object potpourri Jeff King
2013-03-25 20:16 ` [PATCH 1/9] stream_blob_to_fd: detect errors reading from stream Jeff King
2013-03-26 21:27 ` Junio C Hamano
2013-03-25 20:17 ` [PATCH 2/9] check_sha1_signature: check return value from read_istream Jeff King
2013-03-25 20:18 ` [PATCH 3/9] read_istream_filtered: propagate read error from upstream Jeff King
2013-03-25 20:21 ` [PATCH 4/9] avoid infinite loop in read_istream_loose Jeff King
2013-03-25 20:21 ` [PATCH 5/9] add test for streaming corrupt blobs Jeff King
2013-03-25 21:10 ` Jonathan Nieder
2013-03-25 21:26 ` Jeff King
2013-03-27 20:27 ` Jeff King
2013-03-27 20:35 ` Junio C Hamano
2013-03-25 20:22 ` [PATCH 6/9] streaming_write_entry: propagate streaming errors Jeff King
2013-03-25 21:35 ` Eric Sunshine
2013-03-25 21:37 ` Jeff King
2013-03-25 21:39 ` Jonathan Nieder
2013-03-25 21:49 ` [PATCH v2 " Jeff King
2013-03-25 23:29 ` Jonathan Nieder
2013-03-26 21:38 ` Junio C Hamano
2013-03-25 20:22 ` [PATCH 7/9] add tests for cloning corrupted repositories Jeff King
2013-03-25 20:23 ` [PATCH 8/9] clone: die on errors from unpack_trees Jeff King
2013-03-26 21:40 ` Junio C Hamano
2013-03-26 22:22 ` [PATCH 10/9] clone: leave repo in place after checkout errors Jeff King
2013-03-26 22:32 ` Jonathan Nieder
2013-03-27 1:03 ` Jeff King
2013-03-25 20:26 ` [PATCH 9/9] clone: run check_everything_connected Jeff King
2013-03-26 0:53 ` Duy Nguyen
2013-03-26 22:24 ` Jeff King
2013-03-26 21:50 ` Junio C Hamano
2013-03-28 0:40 ` Duy Nguyen
2013-03-31 7:57 ` Duy Nguyen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=102DBFFC4475445D9180A9C7D2A9D97C@PhilipOakley \
--to=philipoakley@iee.org \
--cc=avarab@gmail.com \
--cc=git@vger.kernel.org \
--cc=jeffrey.mitchell@gmail.com \
--cc=pclouds@gmail.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).