From: Linus Torvalds <torvalds@osdl.org>
To: Carl Baldwin <cnb@fc.hp.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>, Git Mailing List <git@vger.kernel.org>
Subject: Re: auto-packing on kernel.org? please?
Date: Mon, 21 Nov 2005 11:24:11 -0800 (PST) [thread overview]
Message-ID: <Pine.LNX.4.64.0511211110480.13959@g5.osdl.org> (raw)
In-Reply-To: <20051121190151.GA2568@hpsvcnb.fc.hp.com>
On Mon, 21 Nov 2005, Carl Baldwin wrote:
>
> I have a question about automatic repacking.
>
> I am thinking of turning something like Linus' repacking heuristic loose
> on my repositories. I just want to make sure it is as safe as possible.
>
> At the core of the incremental and full repack strategies are these
> statements.
>
> Incremental...
> > git repack &&
> > git prune-packed
>
> Full...
> > git repack -a -d &&
> > git prune-packed
NOTE! Since that email, "git repack" has gotten a "local" option (-l),
which is very useful if the repositories have pointers to alternates.
So do
git repack -l
instead, to get much better packs (and "-a -d" for the full case, of
course).
Other that than, the old email suggestion should still be fine.
> Are there some built in safety checks in 'git repack' and/or 'git
> prune-packed' to guard against corruption? In the long run, I would
> feel more comfortable with somelike like this:
>
> git repack
> git verify-pack <new pack>
> git prune-packed
You can certainly do that if you are nervous. It might even be a good
idea: just for fun, I just did
git clone -l git git-clone
cd git-clone
# pick an object at random
rm .git/objects/f7/c3d39fe3db6da3a307da385a7a1cb563ed15f7
git repack -a -d
and it said:
error: Could not read f7c3d39fe3db6da3a307da385a7a1cb563ed15f7
fatal: bad tree object f7c3d39fe3db6da3a307da385a7a1cb563ed15f7
but then it created the pack _anyway_, and said:
Packing 27 objects
Pack pack-13bfca704078175c1c1c59964553b14f7b952651 created.
and happily removed all the old ones.
So right now, repacking a broken archive can actually break it even more.
NOTE! Your "git verify-pack" wouldn't even catch this: the _pack_ is fine,
it's just incomplete.
Of course, this only happens if the repository was broken to begin with,
so arguably it's not that bad. But it does show that git-repack should be
more careful and return an error more aggressively.
Can anybody tell me how to do that sanely? Right now we do
..
name=$(git-rev-list --objects $rev_list $(git-rev-parse $rev_parse) |
git-pack-objects --non-empty $pack_objects .tmp-pack) ||
exit 1
..
and the thing is, the "git-pack-objects" thing is happy, it's the
"git-rev-list" that fails. So because the last command in the pipeline
returns ok, we think it all is ok..
(This is one of the reasons I much prefer working in C over working in
shell: it may be twenty times more lines, but when you have a problem, the
fix is always obvious..)
Anyway, with that fixed, a "git repack" in many ways would be a mini-fsck,
so it should be very safe in general. Modulo any other bugs like the
above.
Linus
next prev parent reply other threads:[~2005-11-21 19:30 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2005-10-13 18:44 auto-packing on kernel.org? please? Linus Torvalds
[not found] ` <434EABFD.5070604@zytor.com>
[not found] ` <434EC07C.30505@pobox.com>
2005-10-13 21:23 ` [kernel.org users] " Linus Torvalds
2005-10-16 14:33 ` Dirk Behme
2005-10-16 15:44 ` Daniel Barkalow
2005-10-16 16:12 ` Nick Hengeveld
2005-10-16 16:23 ` Brian Gerst
2005-10-16 16:56 ` Junio C Hamano
2005-10-16 21:33 ` Nick Hengeveld
2005-10-16 22:12 ` Junio C Hamano
2005-10-17 6:06 ` Nick Hengeveld
2005-10-17 8:21 ` Junio C Hamano
2005-10-17 17:41 ` Nick Hengeveld
2005-10-17 20:08 ` Junio C Hamano
2005-10-17 22:56 ` Daniel Barkalow
2005-10-17 23:19 ` Linus Torvalds
2005-10-17 23:54 ` Nick Hengeveld
2005-10-17 19:13 ` Daniel Barkalow
2005-10-16 17:10 ` Johannes Schindelin
2005-10-16 17:15 ` Brian Gerst
2005-11-21 19:01 ` Carl Baldwin
2005-11-21 19:24 ` Linus Torvalds [this message]
2005-11-21 19:58 ` Junio C Hamano
2005-11-21 20:38 ` Linus Torvalds
2005-11-21 21:35 ` Junio C Hamano
2005-11-22 5:26 ` Chuck Lever
2005-11-22 5:41 ` Linus Torvalds
2005-11-22 14:13 ` Catalin Marinas
2005-11-22 17:05 ` Linus Torvalds
[not found] ` <7v64qkfwhe.fsf@assigned-by-dhcp.cox.net>
[not found] ` <b0943d9e0511220946o3b62842ey@mail.gmail.com>
[not found] ` <7v1x18eddp.fsf@assigned-by-dhcp.cox.net>
2005-11-23 14:10 ` Catalin Marinas
2005-11-22 18:18 ` Chuck Lever
2005-11-23 14:18 ` Catalin Marinas
2005-11-22 17:25 ` Carl Baldwin
2005-11-22 17:58 ` Linus Torvalds
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Pine.LNX.4.64.0511211110480.13959@g5.osdl.org \
--to=torvalds@osdl.org \
--cc=cnb@fc.hp.com \
--cc=git@vger.kernel.org \
--cc=hpa@zytor.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).