[BUG] Destructive access to an "objects/info/alternates" repository

[BUG] Destructive access to an "objects/info/alternates" repository

[Stefan Monnier]

[ Resending with a subject that clarifies that it's a bug report.  ]

> Thank you for filling out a Git bug report!
> Please answer the following questions to help us understand your issue.

> What did you do before the bug happened? (Steps to reproduce your
> issue)

I did basically:

    git clone --reference ~otheruser/nongnu
    [ Days passed and I did various things.  ]
    cd nongnu; git gc

Note that `otheruser` is [..wait for it..] another user, so I only have
read access to the referenced repository.

> What did you expect to happen? (Expected behavior)

That it would GC the local part of the repository and that it wouldn't
try to modify the referenced repository.

> What happened instead? (Actual behavior)

I got warnings indicating that Git was trying to delete files from
the referenced repository:

    Enumerating objects: 35095, done.
    Nothing new to pack.
    warning: unable to unlink
'/home/otheruser/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx':
Permission denied
    warning: unable to unlink
'/home/otheruser/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx':
Permission denied

> What's different between what you expected and what actually happened?

The warnings.

> Anything else you want to add:

The warnings make me wonder if I should worry about other possible
misbehaviors when using such a setup.

> Please review the rest of the bug report below.
> You can delete any lines you don't wish to share.

This is on a Debian stable system but with Git from Debian testing
(because the version from Debian stable crashed in a `git log` command
I was using in my scripts).

[System Info]
git version:
git version 2.29.2
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname: Linux 4.19.0-9-amd64 #1 SMP Debian 4.19.118-2 (2020-04-29) x86_64
compiler info: gnuc: 10.2
libc info: glibc: 2.28
$SHELL (typically, interactive shell): /bin/bash


[Enabled Hooks]

Re: [BUG] Destructive access to an "objects/info/alternates" repository

[Konstantin Ryabitsev]

On Mon, Dec 28, 2020 at 01:50:43PM -0500, Stefan Monnier wrote:
> [ Resending with a subject that clarifies that it's a bug report.  ]
> 
> > Thank you for filling out a Git bug report!
> > Please answer the following questions to help us understand your issue.
> 
> > What did you do before the bug happened? (Steps to reproduce your
> > issue)
> 
> I did basically:
> 
>     git clone --reference ~otheruser/nongnu

This doesn't appear to be a proper command, because it's missing actual
repository to clone, only the repository to use as reference. This is the
proper usage of --reference:

git clone --reference ~/some/repo https://some/remote/repo.git

Perhaps you meant --shared?

Without knowing exactly what you did, it's hard to tell what went wrong. You
can try running "find nongnu" to see if what you think is your own checkout is
actually a symlink or some combination thereof.

-K

Re: [BUG] Destructive access to an "objects/info/alternates" repository

[Stefan Monnier]

>>     git clone --reference ~otheruser/nongnu
>
> This doesn't appear to be a proper command, because it's missing actual
> repository to clone, only the repository to use as reference. This is the
> proper usage of --reference:
>
> git clone --reference ~/some/repo https://some/remote/repo.git

Indeed, thanks:

git clone --reference ~otheruser/nongnu https://git.savannah.gnu.org/r/emacs/nongnu.git

> Without knowing exactly what you did, it's hard to tell what went wrong. You
> can try running "find nongnu" to see if what you think is your own checkout is
> actually a symlink or some combination thereof.

Not sure what you mean by that.

In any case, I just reproduced the problem:

    % git clone --reference ~otheruser/nongnu https://git.savannah.gnu.org/r/emacs/nongnu.git
    Cloning into 'nongnu'...
    Fetching objects: 36974, done.
    % cd nongnu 
    % git gc
    Enumerating objects: 36974, done.
    Nothing new to pack.
    warning: unable to unlink '/home/otheruser/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx': Permission denied
    warning: unable to unlink '/home/otheruser/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx': Permission denied
    % ls -l /home/otheruser/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx
    -rw-r--r-- 1 otheruser otheruser 975612 Dec 11 16:08 /home/otheruser/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx
    % ls -ld /home/otheruser/nongnu/.git/objects/pack
    drwxr-xr-x 2 otheruser otheruser 4096 Dec 27 17:57 /home/otheruser/nongnu/.git/objects/pack/
    %


-- Stefan

Re: [BUG] Destructive access to an "objects/info/alternates" repository

[Stefan Monnier]

>>>     git clone --reference ~otheruser/nongnu
>>
>> This doesn't appear to be a proper command, because it's missing actual
>> repository to clone, only the repository to use as reference. This is the
>> proper usage of --reference:
>>
>> git clone --reference ~/some/repo https://some/remote/repo.git
>
> Indeed, thanks:
>
> git clone --reference ~otheruser/nongnu https://git.savannah.gnu.org/r/emacs/nongnu.git
>
>> Without knowing exactly what you did, it's hard to tell what went wrong. You
>> can try running "find nongnu" to see if what you think is your own checkout is
>> actually a symlink or some combination thereof.
>
> Not sure what you mean by that.
>
> In any case, I just reproduced the problem:

FWIW, I just tried it with

    git clone --shared ~otheruser/nongnu

and got the same result.
I also tried with another repository, still with the same result (tho
different pack.idx names, of course).


        Stefan

Re: [BUG] Destructive access to an "objects/info/alternates" repository

[Konstantin Ryabitsev]

On Mon, Dec 28, 2020 at 03:17:01PM -0500, Stefan Monnier wrote:
> FWIW, I just tried it with
> 
>     git clone --shared ~otheruser/nongnu
> 
> and got the same result.
> I also tried with another repository, still with the same result (tho
> different pack.idx names, of course).

I've tried several times to reproduce this, but I am unable to do so. You may
need to provide the tarball with ~otheruser/nongnu contents for someone to be
able to properly debug this.

-K

Re: [BUG] Destructive access to an "objects/info/alternates" repository

[Stefan Monnier]

>> FWIW, I just tried it with
>> 
>>     git clone --shared ~otheruser/nongnu
>> 
>> and got the same result.
>> I also tried with another repository, still with the same result (tho
>> different pack.idx names, of course).
>
> I've tried several times to reproduce this, but I am unable to do so. You may
> need to provide the tarball with ~otheruser/nongnu contents for someone to be
> able to properly debug this.

OK, you can try with the following tarball:

    https://www.iro.umontreal.ca/~monnier/nongnu.tar.gz

untar it (as "otheruser") into ~otheruser (which will create
~otheruser/nongnu) and then do (as your normal user):

    git clone --shared ~otheruser/nongnu
    cd nongnu
    git gc

It "worked" (i.e. gave me the unlink failure warnings) for me, so
hopefully it will "work" for others as well.

The `git clone` will likely give you some message about HEAD not being
found, which is normal.


        Stefan

Re: [BUG] Destructive access to an "objects/info/alternates" repository

[Konstantin Ryabitsev]

On Sun, Jan 03, 2021 at 11:50:35AM -0500, Stefan Monnier wrote:
> >> FWIW, I just tried it with
> >> 
> >>     git clone --shared ~otheruser/nongnu
> >> 
> >> and got the same result.
> >> I also tried with another repository, still with the same result (tho
> >> different pack.idx names, of course).
> >
> > I've tried several times to reproduce this, but I am unable to do so. You may
> > need to provide the tarball with ~otheruser/nongnu contents for someone to be
> > able to properly debug this.
> 
> OK, you can try with the following tarball:
> 
>     https://www.iro.umontreal.ca/~monnier/nongnu.tar.gz

Great, thank you for providing that. At this point, I can verify the behaviour
you are seeing:

$ cat .git/objects/info/alternates
/tmp/test/nongnu/.git/objects
$ git gc
Enumerating objects: 119, done.
Nothing new to pack.
warning: unable to unlink '/tmp/test/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx': Permission denied
warning: unable to unlink '/tmp/test/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx': Permission denied

Notably, if I run repack/prune by themselves, this does not happen:

$ git repack -Adln
Enumerating objects: 119, done.
Nothing new to pack.
$ git prune --expire=now
$

I would guess that this is happening in builtin/gc.c#686 [1], where it calls
clean_pack_garbage(). Running "git count-objects -v" shows:

$ git count-objects -v
warning: no corresponding .pack: /tmp/test/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx
warning: no corresponding .pack: /tmp/test/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx
count: 0
size: 0
in-pack: 0
packs: 0
size-pack: 0
prune-packable: 0
garbage: 2
size-garbage: 6573
alternate: /tmp/test/nongnu/.git/objects

I agree with Stefan that this is undesired behaviour, even when it only
happens when attempting to clean up garbage -- git-gc (and git-count-objects)
should distinguish between garbage in the parent repository and its own
repository and not attempt any modification of the parent repository during
gc or any other operation.

[1] https://git.kernel.org/pub/scm/git/git.git/tree/builtin/gc.c#n686

-K

Re: [BUG] Destructive access to an "objects/info/alternates" repository

[Stefan Monnier]

Konstantin Ryabitsev [2021-01-04 08:54:10] wrote:
> I would guess that this is happening in builtin/gc.c#686 [1], where it calls
> clean_pack_garbage(). Running "git count-objects -v" shows:
>
> $ git count-objects -v
> warning: no corresponding .pack:
> /tmp/test/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx
> warning: no corresponding .pack:
> /tmp/test/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx
> count: 0
> size: 0
> in-pack: 0
> packs: 0
> size-pack: 0
> prune-packable: 0
> garbage: 2
> size-garbage: 6573
> alternate: /tmp/test/nongnu/.git/objects

Ah, that'd make sense, yes.

> I agree with Stefan that this is undesired behaviour, even when it only
> happens when attempting to clean up garbage -- git-gc (and git-count-objects)
> should distinguish between garbage in the parent repository and its own
> repository and not attempt any modification of the parent repository during
> gc or any other operation.

Great, so IIUC:
- this is a harmless warning
- it's considered as a bug
- so my use case should work fine in practice and is not considered as
  "too weird to be supported"
Thank you,


        Stefan

Re: [BUG] Destructive access to an "objects/info/alternates" repository

[Konstantin Ryabitsev]

On Mon, Jan 04, 2021 at 03:47:53PM -0500, Stefan Monnier wrote:
> > I agree with Stefan that this is undesired behaviour, even when it only
> > happens when attempting to clean up garbage -- git-gc (and git-count-objects)
> > should distinguish between garbage in the parent repository and its own
> > repository and not attempt any modification of the parent repository during
> > gc or any other operation.
> 
> Great, so IIUC:
> - this is a harmless warning

Well, this is a benign situation in general, since git only tries to delete
files that it determined to be garbage. Even if it succeeded in your case, it
wouldn't really modify the actual shared repository state/contents.

> - it's considered as a bug

I'm not really someone who decides if it's a bug or not. :) I *think* it's a
bug, since git modifies the repository from which it should be merely
borrowing objects (a read-only operation). Even if the results are benign,
it's not something that should be happening, in my opinion.

> - so my use case should work fine in practice and is not considered as
>   "too weird to be supported"

It's easy to replicate, if someone feels like fixing it:

mkdir repo1
cd repo1
git init
echo test > test
git add test
git commit -am test
touch .git/objects/pack/pack-abcd.idx
cd ..
git clone -s repo1 repo2
cd repo2
git gc
ls -al ../repo1/.git/objects/pack/pack-abcd.idx
/bin/ls: cannot access '../repo1/.git/objects/pack/pack-abcd.idx': No such file or directory

Hope this helps.

-K