What's happening to the index

What's happening to the index

[Andy Falanga (afalanga)]

Hi,

I'm using git version 1.8.3.1.  I have a process for building RPMs in my 
repository.  The RPMs are versioned using a "build number". In order to 
maintain uniqueness for this build number, I have stored this number to 
a file which exists on only a unique branch.

The build process, for an actual release, is intended to be done by 
branching on a tag.  During that process, a BASH script is called with 
checks out "rpm", fetches the repo from remote, merges with origin/rpm, 
increments the number and pushes back to origin.  It then returns the 
branch to the original branch in which the script was called.  The make 
recipe looks like this:


release:
     make clean
     cd ../..  && ../tools/increlnum && cd -
     if [[ $(TAG) = ......... ]]; then \
         git checkout -b rpm_build_$(TAG) $(TAG); \
     fi
     make rpm RPM_BUILD_NUM=$(shell git show rpm:./rpm_build_num)

After the line calling increlnum is executed, I often have issues with 
make unable to spawn the next command because it can't read the current 
directory info.  Make stops with errors and I'm done.  I have the branch 
name displayed in my PS1 prompt.  I've noticed that, when make errors at 
this point, the branch isn't displayed.  It's as if the index has become 
unstable (or something similar).  If I do: cd .. && cd -; all is well.

What is the problem (I'd really like to understand) and what might I do 
to correct it?

Andy

Re: What's happening to the index

[Matthieu Moy]

"Andy Falanga (afalanga)" <afalanga@micron.com> writes:

> After the line calling increlnum is executed, I often have issues with 
> make unable to spawn the next command because it can't read the current 
> directory info.

This may happen if you delete the current directory, even if your
re-create it afterwards. For example:

/tmp/test$ rm -fr /tmp/test && mkdir /tmp/test
/tmp/test$ touch foo
touch: cannot touch ‘foo’: No such file or directory
/tmp/test$ cd /tmp/test
/tmp/test$ touch foo   
/tmp/test$ 

This is unrelated from Git, but maybe you asked Git to delete a
directory (by switching to a branch which doesn't contain a directory
for example).

> If I do: cd .. && cd -; all is well.

This is a typical symptom of the issue above.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: What's happening to the index

[Andy Falanga (afalanga)]

On 06/28/2016 09:17 AM, Matthieu Moy wrote:
> "Andy Falanga (afalanga)" <afalanga@micron.com> writes:
>
>> After the line calling increlnum is executed, I often have issues with
>> make unable to spawn the next command because it can't read the current
>> directory info.
> This may happen if you delete the current directory, even if your
> re-create it afterwards. For example:
>
> /tmp/test$ rm -fr /tmp/test && mkdir /tmp/test
> /tmp/test$ touch foo
> touch: cannot touch ‘foo’: No such file or directory
> /tmp/test$ cd /tmp/test
> /tmp/test$ touch foo
> /tmp/test$
>
> This is unrelated from Git, but maybe you asked Git to delete a
> directory (by switching to a branch which doesn't contain a directory
> for example).
>
>> If I do: cd .. && cd -; all is well.
> This is a typical symptom of the issue above.
>
Thank you for the insight: very interesting.  After asking another 
colleague how he solved this issue, I've re-written my increlnum script 
to, instead of working within my working tree, clone a temporary of this 
one branch only.  Then, it increments the number and pushes back to the 
origin.  Once completed, the temporary clone is deleted.

The strange thing now is, after the script exits, I then call "git 
fetch" in the recipe.  I can see from the output of make that the remote 
db is fetched.  However, when I call "git show 
origin/rpm:path/to/rpm_build_num" from the makefile I get the *previous* 
number.  Yet, as soon as the make process exits, I call "git show 
origin/rpm:path/to/rpm_build_num" and it shows the correct number!  What 
gives?  Is there some sort of strange file caching that happening when 
make starts that, although the local db is updated, I don't get what I'm 
after?

Andy

Re: What's happening to the index

[Matthieu Moy]

"Andy Falanga (afalanga)" <afalanga@micron.com> writes:

> The strange thing now is, after the script exits, I then call "git 
> fetch" in the recipe.  I can see from the output of make that the remote 
> db is fetched.  However, when I call "git show 
> origin/rpm:path/to/rpm_build_num" from the makefile I get the *previous* 
> number.  Yet, as soon as the make process exits, I call "git show 
> origin/rpm:path/to/rpm_build_num" and it shows the correct number!

My bet would be that you are running the commands in two different
repositories. Try running

  pwd && git show origin/rpm:path/to/rpm_build_num

In the Makefile and after make completes, and check that pwd returns the
same thing. Also, to avoid getting the issue you previously had, try

  cd $(pwd) && pwd && git show origin/rpm:path/to/rpm_build_num

too.

> Is there some sort of strange file caching that happening
> when make starts that, although the local db is updated, I don't get
> what I'm after?

Git can keep information either in RAM, hence not shared between git
invocations (so running git within or outside the Makefile wouldn't
matter), or on disk, but then inside or outside the Makefile doesn't
matter either.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: What's happening to the index

[David]

On 29 June 2016 at 08:49, Andy Falanga (afalanga) <afalanga@micron.com> wrote:

>  Is there some sort of strange file caching that happening when
> make starts that, although the local db is updated, I don't get what I'm
> after?

I don't have time to look at your git issue, but I write this quick
note just in case it might help you to be aware that 'gnu make'
apparently does implement internal directory caching.

This is known to cause unexpected results in makefiles whose recipes
change the filesystem in ways that make does not notice.

See for example:
http://lists.gnu.org/archive/html/help-make/2015-02/msg00012.html

and related bug reports, for example:
https://savannah.gnu.org/bugs/index.php?41273

You might want to create a simple runnable example and ask on the
extremely helpful gnu-make mailing list:
https://lists.gnu.org/mailman/listinfo/help-make