Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Joakim Tjernlund]

I cannot push:
# > git push origin
Login for jocke@git.transmode.se
Password: 
Counting objects: 7, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 13.73 KiB | 0 bytes/s, done.
Total 7 (delta 4), reused 0 (delta 0)
fatal: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied
fatal: The remote end hung up unexpectedly
fatal: The remote end hung up unexpectedly
cu-km022 tmv3-target-overlay # ls -l
total 0
drwxr-xr-x 1 root root  72 Aug 19 17:59 ./
drwxr-xr-x 1 root root  38 Aug 19 17:59 ../
drwxr-xr-x 1 root root 216 Aug 21 13:07 .git/
drwxr-xr-x 1 root root  22 Aug 19 17:59 metadata/
drwxr-xr-x 1 root root  58 Aug 19 18:27 profiles/
drwxr-xr-x 1 root root  74 Aug 19 17:59 sys-apps/
drwxr-xr-x 1 root root  42 Aug 19 17:59 sys-libs/

On server I have:
# >ls -la
total 24
dr-xr-sr-x   6 apache tm-3000  123 Jun 10 15:25 .
drwxr-xr-x  36 root   root    4096 Jun 25 11:11 ..
-r--r--r--   1 root   tm-3000  263 Jun 10 15:18 config
-r--r--r--   1 apache tm-3000   73 Jun 10 15:18 description
-rw-r--r--   1 root   tm-3000    0 Jun 10 15:25 git-daemon-export-ok
-r--r--r--   1 apache tm-3000   23 Jun 10 15:18 HEAD
drwxr-sr-x   2 root   tm-3000 4096 Jun 10 15:18 hooks
drwxrwsr-x   2 apache tm-3000   20 Jun 10 15:18 info
drwxrwsr-x 135 apache tm-3000 4096 Aug 20 19:07 objects
drwxrwsr-x   4 apache tm-3000   29 Jun 10 15:18 refs


I suspect this is because no one is allowed to write the repos top dir.
It would be much better if git allowed a tmp dir inside the repo and used that 
for push etc.
Meanwhile, is there some workaround I can use?

 Jocke

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Joakim Tjernlund]

On Fri, 2015-08-21 at 13:36 +0200, Joakim Tjernlund wrote:
> I cannot push:
> # > git push origin
> Login for jocke@git.transmode.se
> Password: 
> Counting objects: 7, done.
> Delta compression using up to 4 threads.
> Compressing objects: 100% (7/7), done.
> Writing objects: 100% (7/7), 13.73 KiB | 0 bytes/s, done.
> Total 7 (delta 4), reused 0 (delta 0)
> fatal: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied
> fatal: The remote end hung up unexpectedly
> fatal: The remote end hung up unexpectedly
> cu-km022 tmv3-target-overlay # ls -l
> total 0
> drwxr-xr-x 1 root root  72 Aug 19 17:59 ./
> drwxr-xr-x 1 root root  38 Aug 19 17:59 ../
> drwxr-xr-x 1 root root 216 Aug 21 13:07 .git/
> drwxr-xr-x 1 root root  22 Aug 19 17:59 metadata/
> drwxr-xr-x 1 root root  58 Aug 19 18:27 profiles/
> drwxr-xr-x 1 root root  74 Aug 19 17:59 sys-apps/
> drwxr-xr-x 1 root root  42 Aug 19 17:59 sys-libs/
> 
> On server I have:
> # >ls -la
> total 24
> dr-xr-sr-x   6 apache tm-3000  123 Jun 10 15:25 .
> drwxr-xr-x  36 root   root    4096 Jun 25 11:11 ..
> -r--r--r--   1 root   tm-3000  263 Jun 10 15:18 config
> -r--r--r--   1 apache tm-3000   73 Jun 10 15:18 description
> -rw-r--r--   1 root   tm-3000    0 Jun 10 15:25 git-daemon-export-ok
> -r--r--r--   1 apache tm-3000   23 Jun 10 15:18 HEAD
> drwxr-sr-x   2 root   tm-3000 4096 Jun 10 15:18 hooks
> drwxrwsr-x   2 apache tm-3000   20 Jun 10 15:18 info
> drwxrwsr-x 135 apache tm-3000 4096 Aug 20 19:07 objects
> drwxrwsr-x   4 apache tm-3000   29 Jun 10 15:18 refs
> 
> 
> I suspect this is because no one is allowed to write the repos top dir.
> It would be much better if git allowed a tmp dir inside the repo and used that 
> for push etc.
> Meanwhile, is there some workaround I can use?

This error seems to only affect shallow clones.
Forgot, git version 2.4.6

 Jocke

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Joakim Tjernlund]

On Fri, 2015-08-21 at 13:50 +0200, Joakim Tjernlund wrote:
> On Fri, 2015-08-21 at 13:36 +0200, Joakim Tjernlund wrote:
> > I cannot push:
> > # > git push origin
> > Login for jocke@git.transmode.se
> > Password: 
> > Counting objects: 7, done.
> > Delta compression using up to 4 threads.
> > Compressing objects: 100% (7/7), done.
> > Writing objects: 100% (7/7), 13.73 KiB | 0 bytes/s, done.
> > Total 7 (delta 4), reused 0 (delta 0)
> > fatal: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission 
> > denied
> > fatal: The remote end hung up unexpectedly
> > fatal: The remote end hung up unexpectedly
> > cu-km022 tmv3-target-overlay # ls -l
> > total 0
> > drwxr-xr-x 1 root root  72 Aug 19 17:59 ./
> > drwxr-xr-x 1 root root  38 Aug 19 17:59 ../
> > drwxr-xr-x 1 root root 216 Aug 21 13:07 .git/
> > drwxr-xr-x 1 root root  22 Aug 19 17:59 metadata/
> > drwxr-xr-x 1 root root  58 Aug 19 18:27 profiles/
> > drwxr-xr-x 1 root root  74 Aug 19 17:59 sys-apps/
> > drwxr-xr-x 1 root root  42 Aug 19 17:59 sys-libs/
> > 
> > On server I have:
> > # >ls -la
> > total 24
> > dr-xr-sr-x   6 apache tm-3000  123 Jun 10 15:25 .
> > drwxr-xr-x  36 root   root    4096 Jun 25 11:11 ..
> > -r--r--r--   1 root   tm-3000  263 Jun 10 15:18 config
> > -r--r--r--   1 apache tm-3000   73 Jun 10 15:18 description
> > -rw-r--r--   1 root   tm-3000    0 Jun 10 15:25 git-daemon-export-ok
> > -r--r--r--   1 apache tm-3000   23 Jun 10 15:18 HEAD
> > drwxr-sr-x   2 root   tm-3000 4096 Jun 10 15:18 hooks
> > drwxrwsr-x   2 apache tm-3000   20 Jun 10 15:18 info
> > drwxrwsr-x 135 apache tm-3000 4096 Aug 20 19:07 objects
> > drwxrwsr-x   4 apache tm-3000   29 Jun 10 15:18 refs
> > 
> > 
> > I suspect this is because no one is allowed to write the repos top dir.
> > It would be much better if git allowed a tmp dir inside the repo and used that 
> > for push etc.
> > Meanwhile, is there some workaround I can use?
> 
> This error seems to only affect shallow clones.
> Forgot, git version 2.4.6

Ping?

 Jocke

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Duy Nguyen]

On Fri, Aug 21, 2015 at 6:36 PM, Joakim Tjernlund
<joakim.tjernlund@transmode.se> wrote:
> I cannot push:
> # > git push origin
> Login for jocke@git.transmode.se
> Password:
> Counting objects: 7, done.
> Delta compression using up to 4 threads.
> Compressing objects: 100% (7/7), done.
> Writing objects: 100% (7/7), 13.73 KiB | 0 bytes/s, done.
> Total 7 (delta 4), reused 0 (delta 0)
> fatal: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied
> fatal: The remote end hung up unexpectedly
> fatal: The remote end hung up unexpectedly

Noted. Will try to fix (but probably not fast). At first I thought
this was an old bug, but that old bug [1] is in the fetch/clone path,
not push. Not sure if the same approach can be reused here (i.e.avoid
temp files altoghether).

[1] b790e0f (upload-pack: send shallow info over stdin to pack-objects
- 2014-03-11)
-- 
Duy

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Joakim Tjernlund]

On Mon, 2015-08-31 at 16:56 +0700, Duy Nguyen wrote:
> On Fri, Aug 21, 2015 at 6:36 PM, Joakim Tjernlund
> <joakim.tjernlund@transmode.se> wrote:
> > I cannot push:
> > # > git push origin
> > Login for jocke@git.transmode.se
> > Password:
> > Counting objects: 7, done.
> > Delta compression using up to 4 threads.
> > Compressing objects: 100% (7/7), done.
> > Writing objects: 100% (7/7), 13.73 KiB | 0 bytes/s, done.
> > Total 7 (delta 4), reused 0 (delta 0)
> > fatal: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission
> > denied
> > fatal: The remote end hung up unexpectedly
> > fatal: The remote end hung up unexpectedly
> 
> Noted. Will try to fix (but probably not fast). At first I thought
> this was an old bug, but that old bug [1] is in the fetch/clone path,
> not push. Not sure if the same approach can be reused here (i.e.avoid
> temp files altoghether).
> 
> [1] b790e0f (upload-pack: send shallow info over stdin to pack-objects
> - 2014-03-11)

Noticed I had forgotten to reply ...

An even simpler fix would be to have an tmp dir within the repo, aka:
 /var/git/tmv3-target-overlay.git/tmp/shallow_Un8ZOR
This would cover all cases when one must create a tmp file

 Jocke

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Duy Nguyen]

On Mon, Sep 14, 2015 at 10:37 PM, Joakim Tjernlund
<joakim.tjernlund@transmode.se> wrote:
> On Mon, 2015-08-31 at 16:56 +0700, Duy Nguyen wrote:
>> On Fri, Aug 21, 2015 at 6:36 PM, Joakim Tjernlund
>> <joakim.tjernlund@transmode.se> wrote:
>> > I cannot push:
>> > # > git push origin
>> > Login for jocke@git.transmode.se
>> > Password:
>> > Counting objects: 7, done.
>> > Delta compression using up to 4 threads.
>> > Compressing objects: 100% (7/7), done.
>> > Writing objects: 100% (7/7), 13.73 KiB | 0 bytes/s, done.
>> > Total 7 (delta 4), reused 0 (delta 0)
>> > fatal: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission
>> > denied
>> > fatal: The remote end hung up unexpectedly
>> > fatal: The remote end hung up unexpectedly
>>
>> Noted. Will try to fix (but probably not fast). At first I thought
>> this was an old bug, but that old bug [1] is in the fetch/clone path,
>> not push. Not sure if the same approach can be reused here (i.e.avoid
>> temp files altoghether).
>>
>> [1] b790e0f (upload-pack: send shallow info over stdin to pack-objects
>> - 2014-03-11)
>
> Noticed I had forgotten to reply ...
>
> An even simpler fix would be to have an tmp dir within the repo, aka:
>  /var/git/tmv3-target-overlay.git/tmp/shallow_Un8ZOR
> This would cover all cases when one must create a tmp file

Sorry for my silence. I intend to put these temp files in $TMPDIR by
resurrecting (part of) this patch [1]. Maybe tomorrow.

But if you build your own, you can put them in $GIT_DIR/tmp by
replacing "shallow_XXXXXX" in setup_temporary_shallow() in shallow.c
with "tmp/shallow_XXXX". You need to create the directory "tmp" in
advance though, or do
"safe_create_leading_directories_const(git_path("tmp/shallow_XXXXX"));"
before xmkstemp()

[1] http://article.gmane.org/gmane.comp.version-control.git/242787
-- 
Duy

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Joakim Tjernlund]

On Thu, 2015-09-17 at 20:18 +0700, Duy Nguyen wrote:
> On Mon, Sep 14, 2015 at 10:37 PM, Joakim Tjernlund
> <joakim.tjernlund@transmode.se> wrote:
> > On Mon, 2015-08-31 at 16:56 +0700, Duy Nguyen wrote:
> > > On Fri, Aug 21, 2015 at 6:36 PM, Joakim Tjernlund
> > > <joakim.tjernlund@transmode.se> wrote:
> > > > I cannot push:
> > > > # > git push origin
> > > > Login for jocke@git.transmode.se
> > > > Password:
> > > > Counting objects: 7, done.
> > > > Delta compression using up to 4 threads.
> > > > Compressing objects: 100% (7/7), done.
> > > > Writing objects: 100% (7/7), 13.73 KiB | 0 bytes/s, done.
> > > > Total 7 (delta 4), reused 0 (delta 0)
> > > > fatal: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission
> > > > denied
> > > > fatal: The remote end hung up unexpectedly
> > > > fatal: The remote end hung up unexpectedly
> > > 
> > > Noted. Will try to fix (but probably not fast). At first I thought
> > > this was an old bug, but that old bug [1] is in the fetch/clone path,
> > > not push. Not sure if the same approach can be reused here (i.e.avoid
> > > temp files altoghether).
> > > 
> > > [1] b790e0f (upload-pack: send shallow info over stdin to pack-objects
> > > - 2014-03-11)
> > 
> > Noticed I had forgotten to reply ...
> > 
> > An even simpler fix would be to have an tmp dir within the repo, aka:
> >  /var/git/tmv3-target-overlay.git/tmp/shallow_Un8ZOR
> > This would cover all cases when one must create a tmp file
> 
> Sorry for my silence. I intend to put these temp files in $TMPDIR by
> resurrecting (part of) this patch [1]. Maybe tomorrow.
> 
> But if you build your own, you can put them in $GIT_DIR/tmp by
> replacing "shallow_XXXXXX" in setup_temporary_shallow() in shallow.c
> with "tmp/shallow_XXXX". You need to create the directory "tmp" in
> advance though, or do
> "safe_create_leading_directories_const(git_path("tmp/shallow_XXXXX"));"
> before xmkstemp()

I think you should do both, safe_create_leading_directories_const(git_path("tmp/shallow_XXXXX")) and
TMPDIR.

Anyhow, can you send me a patch when you are done?

  Jocke

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Duy Nguyen]

On Thu, Sep 17, 2015 at 11:54 PM, Joakim Tjernlund
<joakim.tjernlund@transmode.se> wrote:
> On Thu, 2015-09-17 at 20:18 +0700, Duy Nguyen wrote:
>> On Mon, Sep 14, 2015 at 10:37 PM, Joakim Tjernlund
>> <joakim.tjernlund@transmode.se> wrote:
>> > On Mon, 2015-08-31 at 16:56 +0700, Duy Nguyen wrote:
>> > > On Fri, Aug 21, 2015 at 6:36 PM, Joakim Tjernlund
>> > > <joakim.tjernlund@transmode.se> wrote:
>> > > > I cannot push:
>> > > > # > git push origin
>> > > > Login for jocke@git.transmode.se
>> > > > Password:
>> > > > Counting objects: 7, done.
>> > > > Delta compression using up to 4 threads.
>> > > > Compressing objects: 100% (7/7), done.
>> > > > Writing objects: 100% (7/7), 13.73 KiB | 0 bytes/s, done.
>> > > > Total 7 (delta 4), reused 0 (delta 0)
>> > > > fatal: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission
>> > > > denied

I'm about to do it, but now I'm not sure if I should move
shallow_XXXXXX out of $GIT_DIR. It will not be the only command that
may write to $GIT_DIR. "git gc --auto" (which can be triggered at the
server side at push time) can write $GIT_DIR/gc.pid (and soon,
gc.log). Even if you disable gc --auto and run it periodically (with
cron or something), it will write gc.pid.

Is it really necessary to remove write access in $GIT_DIR? Do we (git
devs) have some guidelines about things in $GIT_DIR?
-- 
Duy

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Duy Nguyen]

On Sat, Sep 19, 2015 at 9:21 AM, Duy Nguyen <pclouds@gmail.com> wrote:
> Even if you disable gc --auto and run it periodically (with
> cron or something), it will write gc.pid.

Ignore this sentence. Of course you can run manual gc using a
different user and with write access.
-- 
Duy

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Johannes Schindelin]

Hi Duy,

On 2015-09-19 04:21, Duy Nguyen wrote:
> On Thu, Sep 17, 2015 at 11:54 PM, Joakim Tjernlund
> <joakim.tjernlund@transmode.se> wrote:
>> On Thu, 2015-09-17 at 20:18 +0700, Duy Nguyen wrote:
>>> On Mon, Sep 14, 2015 at 10:37 PM, Joakim Tjernlund
>>> <joakim.tjernlund@transmode.se> wrote:
>>> > On Mon, 2015-08-31 at 16:56 +0700, Duy Nguyen wrote:
>>> > > On Fri, Aug 21, 2015 at 6:36 PM, Joakim Tjernlund
>>> > > <joakim.tjernlund@transmode.se> wrote:
>>> > > > I cannot push:
>>> > > > # > git push origin
>>> > > > Login for jocke@git.transmode.se
>>> > > > Password:
>>> > > > Counting objects: 7, done.
>>> > > > Delta compression using up to 4 threads.
>>> > > > Compressing objects: 100% (7/7), done.
>>> > > > Writing objects: 100% (7/7), 13.73 KiB | 0 bytes/s, done.
>>> > > > Total 7 (delta 4), reused 0 (delta 0)
>>> > > > fatal: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission
>>> > > > denied
> 
> I'm about to do it, but now I'm not sure if I should move
> shallow_XXXXXX out of $GIT_DIR. It will not be the only command that
> may write to $GIT_DIR. "git gc --auto" (which can be triggered at the
> server side at push time) can write $GIT_DIR/gc.pid (and soon,
> gc.log). Even if you disable gc --auto and run it periodically (with
> cron or something), it will write gc.pid.
> 
> Is it really necessary to remove write access in $GIT_DIR? Do we (git
> devs) have some guidelines about things in $GIT_DIR?

IMO it makes little sense to remove write access from users who you want to push.

They need to write objects to the directory, after all, and update refs.

This problem sounds more like the doing of an overzealous sysadmin to me than a careful one who researched diligently what locations require write access for the intended operations.

Personally, I see little sense in bending over to try to support such an intentionally tampered setup.

Ciao,
Dscho

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Joakim Tjernlund]

On Sat, 2015-09-19 at 09:21 +0700, Duy Nguyen wrote:
> On Thu, Sep 17, 2015 at 11:54 PM, Joakim Tjernlund
> <joakim.tjernlund@transmode.se> wrote:
> > On Thu, 2015-09-17 at 20:18 +0700, Duy Nguyen wrote:
> > > On Mon, Sep 14, 2015 at 10:37 PM, Joakim Tjernlund
> > > <joakim.tjernlund@transmode.se> wrote:
> > > > On Mon, 2015-08-31 at 16:56 +0700, Duy Nguyen wrote:
> > > > > On Fri, Aug 21, 2015 at 6:36 PM, Joakim Tjernlund
> > > > > <joakim.tjernlund@transmode.se> wrote:
> > > > > > I cannot push:
> > > > > > # > git push origin
> > > > > > Login for jocke@git.transmode.se
> > > > > > Password:
> > > > > > Counting objects: 7, done.
> > > > > > Delta compression using up to 4 threads.
> > > > > > Compressing objects: 100% (7/7), done.
> > > > > > Writing objects: 100% (7/7), 13.73 KiB | 0 bytes/s, done.
> > > > > > Total 7 (delta 4), reused 0 (delta 0)
> > > > > > fatal: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR':
> > > > > > Permission
> > > > > > denied
> 
> I'm about to do it, but now I'm not sure if I should move
> shallow_XXXXXX out of $GIT_DIR. It will not be the only command that
> may write to $GIT_DIR. "git gc --auto" (which can be triggered at the
> server side at push time) can write $GIT_DIR/gc.pid (and soon,
> gc.log). Even if you disable gc --auto and run it periodically (with
> cron or something), it will write gc.pid.
> 
> Is it really necessary to remove write access in $GIT_DIR? Do we (git
> devs) have some guidelines about things in $GIT_DIR?

It feels a lot cleaner to not let everybody create stuff in $GIT_DIR
we have:
# > ls -l
total 24
dr-xr-sr-x   6 apache tm-3000  123 Jun 10 15:30 ./
drwxr-xr-x  36 root   root    4096 Jun 25 11:11 ../
-r--r--r--   1 root   tm-3000  263 Jun 10 15:28 config
-r--r--r--   1 apache tm-3000   73 Jun 10 15:28 description
-rw-r--r--   1 root   tm-3000    0 Jun 10 15:30 git-daemon-export-ok
-r--r--r--   1 apache tm-3000   23 Jun 10 15:28 HEAD
drwxr-sr-x   2 root   tm-3000 4096 Jun 10 15:28 hooks/
drwxrwsr-x   2 apache tm-3000   20 Jun 10 15:28 info/
drwxrwsr-x 157 apache tm-3000 4096 Jul 14 15:06 objects/
drwxrwsr-x   4 apache tm-3000   29 Jun 10 15:28 refs/

Why mess this up with tmp files? Would be cleaner to have a specific tmp dir in $GIT_DIR for that.

 Jocke

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Joakim Tjernlund]

On Sat, 2015-09-19 at 09:13 +0200, Johannes Schindelin wrote:
> Hi Duy,
> 
> On 2015-09-19 04:21, Duy Nguyen wrote:
> > On Thu, Sep 17, 2015 at 11:54 PM, Joakim Tjernlund
> > <joakim.tjernlund@transmode.se> wrote:
> > > On Thu, 2015-09-17 at 20:18 +0700, Duy Nguyen wrote:
> > > > On Mon, Sep 14, 2015 at 10:37 PM, Joakim Tjernlund
> > > > <joakim.tjernlund@transmode.se> wrote:
> > > > > On Mon, 2015-08-31 at 16:56 +0700, Duy Nguyen wrote:
> > > > > > On Fri, Aug 21, 2015 at 6:36 PM, Joakim Tjernlund
> > > > > > <joakim.tjernlund@transmode.se> wrote:
> > > > > > > I cannot push:
> > > > > > > # > git push origin
> > > > > > > Login for jocke@git.transmode.se
> > > > > > > Password:
> > > > > > > Counting objects: 7, done.
> > > > > > > Delta compression using up to 4 threads.
> > > > > > > Compressing objects: 100% (7/7), done.
> > > > > > > Writing objects: 100% (7/7), 13.73 KiB | 0 bytes/s, done.
> > > > > > > Total 7 (delta 4), reused 0 (delta 0)
> > > > > > > fatal: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR':
> > > > > > > Permission
> > > > > > > denied
> > 
> > I'm about to do it, but now I'm not sure if I should move
> > shallow_XXXXXX out of $GIT_DIR. It will not be the only command that
> > may write to $GIT_DIR. "git gc --auto" (which can be triggered at the
> > server side at push time) can write $GIT_DIR/gc.pid (and soon,
> > gc.log). Even if you disable gc --auto and run it periodically (with
> > cron or something), it will write gc.pid.
> > 
> > Is it really necessary to remove write access in $GIT_DIR? Do we (git
> > devs) have some guidelines about things in $GIT_DIR?
> 
> IMO it makes little sense to remove write access from users who you want to push.
> 
> They need to write objects to the directory, after all, and update refs.
> 
> This problem sounds more like the doing of an overzealous sysadmin to me than a careful one who researched
> diligently what locations require write access for the intended operations.

We did and it all worked just fine , uses can push as they should. It is just shallow clones
that are non working. Why are pushes against normal clones and shallow clones not handled the
same way w.r.t tmp file creation?

> 
> Personally, I see little sense in bending over to try to support such an intentionally tampered setup.

Before bringing out your shootgun, make sure you are aiming it the right target.

 Jocke

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Junio C Hamano]

Duy Nguyen <pclouds@gmail.com> writes:

> Is it really necessary to remove write access in $GIT_DIR? Do we (git
> devs) have some guidelines about things in $GIT_DIR?

Those who are allowed to "git push" into it should be able to write
there.  It is a different matter that "git" program itself may make
a policy decision to disallow some operations that the filesystem
bits alone would have allowed (e.g. you can arrange the "pusher" to
only come over the wire via "receive-pack" and "receive-pack" may
deliberately lack support for writing into $GIT_DIR/config).

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Joakim Tjernlund]

On Mon, 2015-09-21 at 09:48 -0700, Junio C Hamano wrote:
> Duy Nguyen <pclouds@gmail.com> writes:
> 
> > Is it really necessary to remove write access in $GIT_DIR? Do we (git
> > devs) have some guidelines about things in $GIT_DIR?
> 
> Those who are allowed to "git push" into it should be able to write
> there.  It is a different matter that "git" program itself may make
> a policy decision to disallow some operations that the filesystem
> bits alone would have allowed (e.g. you can arrange the "pusher" to
> only come over the wire via "receive-pack" and "receive-pack" may
> deliberately lack support for writing into $GIT_DIR/config).
> 

I view $GIT_DIR similar to "/" and "/tmp". Normally one does not let normal users write to "/"
as you want to keep this level clean. It is not obvious to everybody what files are important
under $GIT_DIR when mixed with tmp files etc.
$GIT_DIR/tmp would solve this nicely.

 Jocke

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Johannes Schindelin]

Hi Joakim,

On 2015-09-21 19:08, Joakim Tjernlund wrote:
> On Mon, 2015-09-21 at 09:48 -0700, Junio C Hamano wrote:
>> Duy Nguyen <pclouds@gmail.com> writes:
>>
>> > Is it really necessary to remove write access in $GIT_DIR? Do we (git
>> > devs) have some guidelines about things in $GIT_DIR?
>>
>> Those who are allowed to "git push" into it should be able to write
>> there.  It is a different matter that "git" program itself may make
>> a policy decision to disallow some operations that the filesystem
>> bits alone would have allowed (e.g. you can arrange the "pusher" to
>> only come over the wire via "receive-pack" and "receive-pack" may
>> deliberately lack support for writing into $GIT_DIR/config).
>>
> 
> I view $GIT_DIR similar to "/" and "/tmp". Normally one does not let
> normal users write to "/"
> as you want to keep this level clean. It is not obvious to everybody
> what files are important
> under $GIT_DIR when mixed with tmp files etc.
> $GIT_DIR/tmp would solve this nicely.

By now it is pretty clear that you won't find many people here you share your opinion about locking down the Git directory.

The reason should be easy to understand: Git's concept is based on the idea that you have full control over your repository. Other repositories you might only have read access.

But this idea you have, to somehow introduce fine-grained levels of control, this idea would imply that all of a sudden Git is no longer free to write to its files as it likes. And as far as Git is concerned, everything inside .git/ *are* its files.

So in essence, the core concept of Git -- you clone a repository you cannot write to so that you have a local repository you can do *anything you like* to -- is pretty much incompatible with this idea of a selective lock down of files in .git/ that not only would require you to know very exactly what files Git might want to write, but also to keep yourself up-to-date with Git's development as to which files it might want to write for *every* new version. Making only .git/tmp/ a writable location further fails to acknowledge the fact that the hierarchy of to-be-written files follows the function of those files, not any write permission hierarchy. Since the idea seems so alien to Git's core concept, I called it "overzealous". If that hurt your feelings, I am sorry and would like to apologize
 .

Having said all that, I believe that reiterating this idea without pointing to any benefit will continue to fail to convince people that the idea is sound and that Git's core concept should change. If you need to exert more control in a specific repository, you simply make it accessible only as a non-file-system remote (where only `git`, `git-receive-pack` and `git-upload-pack` are allowed to be executed) and define hooks that can accept or deny on a *much* finer level than file system permissions ever could, after all.

Ciao,
Johannes

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Joakim Tjernlund]

On Tue, 2015-09-22 at 22:00 +0200, Johannes Schindelin wrote:
> Hi Joakim,
> 
> On 2015-09-21 19:08, Joakim Tjernlund wrote:
> > On Mon, 2015-09-21 at 09:48 -0700, Junio C Hamano wrote:
> > > Duy Nguyen <pclouds@gmail.com> writes:
> > > 
> > > > Is it really necessary to remove write access in $GIT_DIR? Do we (git
> > > > devs) have some guidelines about things in $GIT_DIR?
> > > 
> > > Those who are allowed to "git push" into it should be able to write
> > > there.  It is a different matter that "git" program itself may make
> > > a policy decision to disallow some operations that the filesystem
> > > bits alone would have allowed (e.g. you can arrange the "pusher" to
> > > only come over the wire via "receive-pack" and "receive-pack" may
> > > deliberately lack support for writing into $GIT_DIR/config).
> > > 
> > 
> > I view $GIT_DIR similar to "/" and "/tmp". Normally one does not let
> > normal users write to "/"
> > as you want to keep this level clean. It is not obvious to everybody
> > what files are important
> > under $GIT_DIR when mixed with tmp files etc.
> > $GIT_DIR/tmp would solve this nicely.
> 
> By now it is pretty clear that you won't find many people here you share your opinion about locking down the
> Git directory.

So I note.

> 
> The reason should be easy to understand: Git's concept is based on the idea that you have full control over
> your repository. Other repositories you might only have read access.

Yes and some repos I only have partial write access to(config, hooks etc. might be readonly) 

> 
> But this idea you have, to somehow introduce fine-grained levels of control, this idea would imply that all
> of a sudden Git is no longer free to write to its files as it likes. And as far as Git is concerned,
> everything inside .git/ *are* its files.

This does not compute for me, files inside git are git's files, I only think that not all users
to a repo should have the same (write) access. In this case it mostly to protect the repo from "creative"
users and accidents.

> 
> So in essence, the core concept of Git -- you clone a repository you cannot write to so that you have a
> local repository you can do *anything you like* to -- is pretty much incompatible with this idea of a
> selective lock down of files in .git/ that not only would require you to know very exactly what files Git
> might want to write, but also to keep yourself up-to-date with Git's development as to which files it might


Don't see how I can avoid some of that if you want to protect areas of the repo from accidents etc.

> want to write for *every* new version. Making only .git/tmp/ a writable location further fails to
> acknowledge the fact that the hierarchy of to-be-written files follows the function of those files, not any


Curious, how would you set up some level of protection on a repo?

A .git/tmp/ would make housekeeping easier, you would know that every file under .git
should be there and if you find something you don't recognize you would react.

> write permission hierarchy. Since the idea seems so alien to Git's core concept, I called it "overzealous".
> If that hurt your feelings, I am sorry and would like to apologize.

No feelings hurt, I too regret my choise of words.

> Having said all that, I believe that reiterating this idea without pointing to any benefit will continue to
> fail to convince people that the idea is sound and that Git's core concept should change. If you need to
> exert more control in a specific repository, you simply make it accessible only as a non-file-system remote
> (where only `git`, `git-receive-pack` and `git-upload-pack` are allowed to be executed) and define hooks
> that can accept or deny on a *much* finer level than file system permissions ever could, after all.

Even if I did go through this hassle, I would prefer if temporary data were put somewhere else
than .git/ as I think mixing config/persistent data with temporary data in the same directory is something
that should be avoided.

Anyhow, I see that this idea is not something upstream agrees on so I will back off now.

  Jocke

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Johannes Schindelin]

Hi Joakim,

On 2015-09-22 22:58, Joakim Tjernlund wrote:
> On Tue, 2015-09-22 at 22:00 +0200, Johannes Schindelin wrote:
>>
>> The reason should be easy to understand: Git's concept is based on the idea that you have full control over
>> your repository. Other repositories you might only have read access.
> 
> Yes and some repos I only have partial write access to(config, hooks
> etc. might be readonly)

The partial write access idea is definitely not part of the original idea of Git, and your use case is actually the first I heard of.

The original idea was really that you either own your repository, or you do not. And that includes the repositories that can be accessed publicly: you own them or you don't.

Now, I know that in particular in some corporate setups, there needs to be a permission system in place that disallows certain users from doing certain things (such as editing the config).

The Git solution is to set up a server, usually with SSH, and allow users to push and fetch from the repositories, but nothing else (i.e. no shell access), then set up hooks to implement the permission system.

This is much less error prone than partially locking down a repository on some network drive because the file system structure simply does not reflect the permission structure. That is where all your troubles come from.

>> But this idea you have, to somehow introduce fine-grained levels of control, this idea would imply that all
>> of a sudden Git is no longer free to write to its files as it likes. And as far as Git is concerned,
>> everything inside .git/ *are* its files.
> 
> This does not compute for me, files inside git are git's files, I only
> think that not all users
> to a repo should have the same (write) access.

But then it is your duty to tell *Git* what it can and what it cannot do. Typically via those hooks I mentioned.

> A .git/tmp/ would make housekeeping easier, you would know that every
> file under .git
> should be there and if you find something you don't recognize you would react.

No, it would actually make it harder. I seem to recall that there was some problem with renaming a file unless it was already in the same directory as the destination. If all files were to be written to .git/tmp/ first...

>> If you need to exert more control in a specific repository, you simply make it accessible only as a non-file-system remote
>> (where only `git`, `git-receive-pack` and `git-upload-pack` are allowed to be executed) and define hooks
>> that can accept or deny on a *much* finer level than file system permissions ever could, after all.
> 
> Even if I did go through this hassle, I would prefer if temporary data
> were put somewhere else
> than .git/ as I think mixing config/persistent data with temporary
> data in the same directory is something
> that should be avoided.

Sure, I understand what you ask for. It's just that Git worked in a different direction for 10 years now ;-)

Ciao,
Johannes

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

>> A .git/tmp/ would make housekeeping easier, you would know that every
>> file under .git
>> should be there and if you find something you don't recognize you would react.
>
> No, it would actually make it harder. I seem to recall that there was
> some problem with renaming a file unless it was already in the same
> directory as the destination. If all files were to be written to
> .git/tmp/ first...

I agree with you here.  This "some filesystems do not like
cross-directory renames" was also why I thought the idea to create
everything in $GIT_DIR/tmp and moving to the final destination would
not fly well.

For the specific issue with the shallow-cutoff list temporarily
created, however, it is created, written to, read from and removed
without ever leaving the original place, so we could designate one
writable "temporary" place somewhere under $GIT_DIR, and $GIT_DIR/tmp
could certainly that designated temporary place.

By the way, the temporary shallow-cutoff-list is created by callingy
xmks_tempfile().  I wonder if it can safely changed to use
mks_tempfile() instead.  That way, people can set up a wrapper for
receive-pack that sets up a suitable TMPDIR exported to the real
thing to relocate it.

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Joakim Tjernlund]

On Wed, 2015-09-23 at 13:10 +0200, Johannes Schindelin wrote:
> Hi Joakim,
> 
> On 2015-09-22 22:58, Joakim Tjernlund wrote:
> > On Tue, 2015-09-22 at 22:00 +0200, Johannes Schindelin wrote:
> > > 
> > > The reason should be easy to understand: Git's concept is based on the idea that you have full control
> > > over
> > > your repository. Other repositories you might only have read access.
> > 
> > Yes and some repos I only have partial write access to(config, hooks
> > etc. might be readonly)
> 
> The partial write access idea is definitely not part of the original idea of Git, and your use case is
> actually the first I heard of.

Ouch, that cannot be so?? The first thing one would do for some level of accident protection 
would be to just change privs on a few selected files/dirs. 

> 
> The original idea was really that you either own your repository, or you do not. And that includes the
> repositories that can be accessed publicly: you own them or you don't.
> 
> Now, I know that in particular in some corporate setups, there needs to be a permission system in place that
> disallows certain users from doing certain things (such as editing the config).

Exactly! This is what we are doing.

> 
> The Git solution is to set up a server, usually with SSH, and allow users to push and fetch from the
> repositories, but nothing else (i.e. no shell access), then set up hooks to implement the permission system.

But this is too big of an ax just to get any protection at all. Dedicating a server just for this
is very costly, both the physical/virtual server and to maintain it. 

> 
> This is much less error prone than partially locking down a repository on some network drive because the
> file system structure simply does not reflect the permission structure. That is where all your troubles come
> from.

Sure, but here is room for improvement.

 Jocke

Re: Unable to create temporary file '/var/git/tmv3-target-overlay.git/shallow_Un8ZOR': Permission denied

[Johannes Schindelin]

Hi Joakim,

On 2015-09-23 22:41, Joakim Tjernlund wrote:
> On Wed, 2015-09-23 at 13:10 +0200, Johannes Schindelin wrote:
>>
>> On 2015-09-22 22:58, Joakim Tjernlund wrote:
>> > On Tue, 2015-09-22 at 22:00 +0200, Johannes Schindelin wrote:
>> > >
>> > > The reason should be easy to understand: Git's concept is based on the idea that you have full control
>> > > over
>> > > your repository. Other repositories you might only have read access.
>> >
>> > Yes and some repos I only have partial write access to(config, hooks
>> > etc. might be readonly)
>>
>> The partial write access idea is definitely not part of the original idea of Git, and your use case is
>> actually the first I heard of.
> 
> Ouch, that cannot be so??

Yes, it can be so. In fat, it is so.

Please note that I *did* encounter valid scenarios where some operations might not be desirable (and therefore need to be prevented).

One such scenario (maybe even the first one) was to prevent non-fast-forward pushes. But you will certainly agree that this cannot be prevented by mere file system permission: they are not fine-grained enough. So we introduced a config option -- because in contrast to file system permissions, Git *does* have the means to enforce that rule.

So it all comes back to the point I made earlier, and that I really would like you to understand: Git's concepts do not align well with file system permissions. Not well at all, in fact.

So the method of choice is indeed what you called that "big axe" which is not such a big axe after all. You just need to set up an SSH server and define very clearly in the hooks what you consider permissible. Yep, that's a bit of work, but it is less work than would be required of Git to bend it so the same could be done via file system permissions. And stay that way.

Now, it might be possible for some operations, to *make* Git align with that permission system. But that sounds more and more like the desired changes would require Git developers to put in a lot of work in favor of others being able to avoid work, just for the sake of keeping with an idea that has been demonstrated to be flawed. If you are looking for fans of that idea, count me out ;-) Of course, if you are willing to put in the work to make it possible to restrict certain Git operations simply by using `chmod`, and to pay attention that it stays that way, go right ahead and submit a patch series to that end... Junio already indicated that he would not be flatly opposed to accept such changes ;-)

Ciao,
Johannes