ignoring extra bitmap file?

ignoring extra bitmap file?

[Andreas Krey]

Hi everyone,

I'm seeing the message

   remote: warning: ignoring extra bitmap file: ./objects/pack/pack-2943dc24....pack

and indeed, there is such a thing (two, actually):

   171736188 Aug 17 08:20 pack-2943dc2477026f87b280ebcefa93fe28412688df.idx
    12662268 Aug 17 08:24 pack-2943dc2477026f87b280ebcefa93fe28412688df.bitmap
 12927989355 Aug 17 08:27 pack-2943dc2477026f87b280ebcefa93fe28412688df.pack
   164857412 Aug 17 08:33 pack-8b4a42ca7aa2aca6f354292007910de1110117b2.idx
    13164932 Aug 17 08:49 pack-8b4a42ca7aa2aca6f354292007910de1110117b2.bitmap
      281872 Aug 17 09:40 pack-bddb40f984124ba8c2a4e5c55b0d1b2804fd5817.pack
       13280 Aug 17 09:40 pack-bddb40f984124ba8c2a4e5c55b0d1b2804fd5817.idx
        7904 Aug 17 15:51 pack-0f8b1478e17174c562d9a52cf577e0e050bdb7c5.idx
     2373948 Aug 17 16:09 pack-23253e17510cacaae3bb38fb5429073b3bc59480.pack
        6980 Aug 17 16:09 pack-23253e17510cacaae3bb38fb5429073b3bc59480.idx
      144158 Aug 17 17:03 pack-0f8b1478e17174c562d9a52cf577e0e050bdb7c5.pack
 12927996484 Aug 17 19:19 pack-8b4a42ca7aa2aca6f354292007910de1110117b2.pack
      153332 Aug 17 20:17 pack-65ff13a10c29a6c1604017c50dc9a320044ee605.pack
       14036 Aug 17 20:17 pack-65ff13a10c29a6c1604017c50dc9a320044ee605.idx

But it looks like something went wrong in that repack cycle (that
pack-2943dc247702 is the full repo), and it won't get removed later
in the next repack in the evening.

Question: Can I safely remove the .bitmap file, and repack will then
clean up the .pack and .idx files as will?

(This is still that repo in bitbucket (latest 4.x) server
with git 2.6.2, now with cg.auto=0.)

- Andreas

-- 
"Totally trivial. Famous last words."
From: Linus Torvalds <torvalds@*.org>
Date: Fri, 22 Jan 2010 07:29:21 -0800

Re: ignoring extra bitmap file?

[Jeff King]

On Thu, Aug 17, 2017 at 09:24:36PM +0200, Andreas Krey wrote:

> I'm seeing the message
> 
>    remote: warning: ignoring extra bitmap file: ./objects/pack/pack-2943dc24....pack
> 
> and indeed, there is such a thing (two, actually):

Only one is the extra. :) The other is doing something useful.

Basically, the bitmap code was written to a handle a single bitmap file.
It would be possible to handle multiple, but it simplified the
implementation greatly to only handle one. And in practice, since a
bitmap can only be made for a pack which contains all of the reachable
objects, you'd have only one bitmap per repo, for the one big "main"
pack.

> But it looks like something went wrong in that repack cycle (that
> pack-2943dc247702 is the full repo), and it won't get removed later
> in the next repack in the evening.

Yes, it looks like you got a full repack that failed to remove the old
pack. Or more likely you had two full repacks racing with each other,
each creating a new big pack.

So the extra bitmap here is harmless. Both of them contain more or less
the same data, and whichever one we use will be fine (and remember that
the .bitmap files are purely an optimization, so that "more or less"
will only make a minor impact on the speed of operations, not on the
output).

> Question: Can I safely remove the .bitmap file, and repack will then
> clean up the .pack and .idx files as will?

Yes, it's always safe to remove a .bitmap file (though if you remove the
last one, you may expect performance to drop for some operations).

Whether there's a .bitmap doesn't impact whether .pack and .idx files
are deleted. The next full repack would pack everything into a new big
pack, and then delete any existing files, including .pack, .idx, and
.bitmap.

-Peff

Re: ignoring extra bitmap file?

[Andreas Krey]

On Fri, 18 Aug 2017 02:53:34 +0000, Jeff King wrote:
...
> Whether there's a .bitmap doesn't impact whether .pack and .idx files
> are deleted. The next full repack would pack everything into a new big
> pack, and then delete any existing files, including .pack, .idx, and
> .bitmap.

It took a bit of patience, but the extra packs and .bitmaps
finally went away without intervention.

However, now I have a few GB sitting in loose objects that
refuse to vanish yet. Bryan might be interested here;
how often is bitbucket 4.latest doing prune-objects
(with auto-gc disabled)?

Andreas

-- 
"Totally trivial. Famous last words."
From: Linus Torvalds <torvalds@*.org>
Date: Fri, 22 Jan 2010 07:29:21 -0800