Using Format/export-subst Howto.

Using Format/export-subst Howto.

[Mestnik, Michael J - Eagan, MN - Contractor]

I must have missed something reading through the documentation for this.  git version 1.7.11.3

$ git check-attr -a -- autorepair.d/AR02_new_rttest.sh
autorepair.d/AR02_new_rttest.sh: ident: set
autorepair.d/AR02_new_rttest.sh: export-subst: set

  echo "0..$_expected_tests"
  diag 'Script Version: $Id: 1ca40f8395ea361cc07d2ec1a2961c3df749dc3c $'
  diag 'By: $Format:%cn$ $Format:%ce$'
  diag 'At: $Format:%cD$'

I also believe that the documentation could try and better explain under what conditions "$Id" will be processed, it doesn't seam to happen on commit and even after a checkout this is not updated.  It does seam to update during a pull and that's basically all I need.

Thanks!

Mike Mestnik, Michael J
The ESM Tools
Enterprise Systems Monitoring
United States Postal Service
 O: (651) 406-2048
Michael.J.Mestnik@usps.gov
ITEnterpriseSystemsMonitoring@usps.gov

Re: Using Format/export-subst Howto.

[Michael J Gruber]

Mestnik, Michael J - Eagan, MN - Contractor venit, vidit, dixit
14.09.2012 14:20:
> I must have missed something reading through the documentation for this.  git version 1.7.11.3
> 
> $ git check-attr -a -- autorepair.d/AR02_new_rttest.sh
> autorepair.d/AR02_new_rttest.sh: ident: set
> autorepair.d/AR02_new_rttest.sh: export-subst: set
> 
>   echo "0..$_expected_tests"
>   diag 'Script Version: $Id: 1ca40f8395ea361cc07d2ec1a2961c3df749dc3c $'
>   diag 'By: $Format:%cn$ $Format:%ce$'
>   diag 'At: $Format:%cD$'
> 

'$Format:' is processed when creating an archive. It's mentioned with
export-subst only under the heading "create archive". So, that is as
described, I think.

> I also believe that the documentation could try and better explain under what conditions "$Id" will be processed, it doesn't seam to happen on commit and even after a checkout this is not updated.  It does seam to update during a pull and that's basically all I need.

"git replaces $Id$... upon checkout.  Any byte sequence that begins with
$Id: and ends with $ in the worktree file is replaced with $Id$ upon
check-in."

Now, the there are two problems after you add $Id$ and check-in (commit):

- commit does not check out, i.e. your work-tree copy is not updated
with expanded $Id$
- Not even "git checkout thatFile" updates your work-tree copy.

The first one could be considered OK, but at least the second one seems
to be a bug. Together they create the following problem: Say, you've
corrected that problem (rm that file and checkout) and then update your
file, add and commit. It will keeping having the old (now wrong) Id
expansion.

We should do something about this.

Michael

Re: Using Format/export-subst Howto.

[Johannes Sixt]

Am 9/14/2012 15:03, schrieb Michael J Gruber:
> "git replaces $Id$... upon checkout.  Any byte sequence that begins
> with $Id: and ends with $ in the worktree file is replaced with $Id$
> upon check-in."
> 
> Now, the there are two problems after you add $Id$ and check-in
> (commit):
> 
> - commit does not check out, i.e. your work-tree copy is not updated 
> with expanded $Id$ - Not even "git checkout thatFile" updates your
> work-tree copy.
> 
> The first one could be considered OK, but at least the second one
> seems to be a bug. Together they create the following problem: Say,
> you've corrected that problem (rm that file and checkout) and then
> update your file, add and commit. It will keeping having the old (now
> wrong) Id expansion.

If EOL conversion or a clean filter was applied during 'git add file', is
the version in the worktree suddenly wrong? Of course, not.

I would place $Id$ treatment in the same ball park and declare it as a
mistake of the editor that it did not remove the now "wrong" SHA1 from $Id:$.

> We should do something about this.

Not necessary, IMHO.

-- Hannes

RE: Using Format/export-subst Howto.

[Mestnik, Michael J - Eagan, MN - Contractor]

> -----Original Message-----
> From: Johannes Sixt [mailto:j.sixt@viscovery.net] 
> Sent: Friday, September 14, 2012 10:07 AM
> To: Michael J Gruber
> Cc: Mestnik, Michael J - Eagan, MN - Contractor; git@vger.kernel.org
> Subject: Re: Using Format/export-subst Howto.
> 
> Am 9/14/2012 15:03, schrieb Michael J Gruber:
> > "git replaces $Id$... upon checkout.  Any byte sequence that begins
> > with $Id: and ends with $ in the worktree file is replaced with $Id$
> > upon check-in."
> > 
> > Now, the there are two problems after you add $Id$ and check-in
> > (commit):
> > 
> > - commit does not check out, i.e. your work-tree copy is 
> not updated 
> > with expanded $Id$ - Not even "git checkout thatFile" updates your
> > work-tree copy.
> > 
> > The first one could be considered OK, but at least the second one
> > seems to be a bug. Together they create the following problem: Say,
> > you've corrected that problem (rm that file and checkout) and then
> > update your file, add and commit. It will keeping having 
> the old (now
> > wrong) Id expansion.
> 
> If EOL conversion or a clean filter was applied during 'git 
> add file', is
> the version in the worktree suddenly wrong? Of course, not.
> 
> I would place $Id$ treatment in the same ball park and declare it as a
> mistake of the editor that it did not remove the now "wrong" 
> SHA1 from $Id:$.

I think the difference here is that git does not *currently change the OS's LEF.  In this case each commit alters the Id and git is the one altering the Id.  If git did change the expected/system LEF then it would seam reasonable that it would also be responsible for forward porting files to the new regime.

* If git could fix some misguided operating systems into following the correct LEF, that would be great!

What I mean is that I agree that git is not the tool to tackle every technical challenge, but I think it should carry though with any decisions it makes and that it should not ignore the effects(or changes) made as a result of **these decisions.

** Any and all, within reason.

Cheers!

> 
> > We should do something about this.
> 
> Not necessary, IMHO.
> 
> -- Hannes
> 



Mike Mestnik, Michael J
The ESM Tools
Enterprise Systems Monitoring
United States Postal Service
 O: (651) 406-2048
Michael.J.Mestnik@usps.gov
ITEnterpriseSystemsMonitoring@usps.gov
 

Re: Using Format/export-subst Howto.

[Johannes Sixt]

Am 9/14/2012 17:27, schrieb Mestnik, Michael J - Eagan, MN - Contractor:
> 
>> -----Original Message----- From: Johannes Sixt
>> If EOL conversion or a clean filter was applied during 'git add
>> file', is the version in the worktree suddenly wrong? Of course,
>> not.
>> 
>> I would place $Id$ treatment in the same ball park and declare it as
>> a mistake of the editor that it did not remove the now "wrong" SHA1
>> from $Id:$.
> 
> I think the difference here is that git does not *currently change the
> OS's LEF.  In this case each commit alters the Id and git is the one
> altering the Id.

Maybe you misunderstood $Id$? The value you get there is the blob's SHA1,
not the commit's. That is, it does not change on every commit, but only
when the file changes.

You are right that the value itself is something that is dictated by git's
database model, but the change logically happens when the editor modifies
the file.

(My contribution to this thread should be regarded as food for thought.
Personally, I don't mind whether or not and when $Id$ is updated.)

-- Hannes

RE: Using Format/export-subst Howto.

[Mestnik, Michael J - Eagan, MN - Contractor]

> -----Original Message-----
> From: Johannes Sixt [mailto:j.sixt@viscovery.net] 
> Sent: Friday, September 14, 2012 10:40 AM
> To: Mestnik, Michael J - Eagan, MN - Contractor
> Cc: Michael J Gruber; git@vger.kernel.org
> Subject: Re: Using Format/export-subst Howto.
> 
> Am 9/14/2012 17:27, schrieb Mestnik, Michael J - Eagan, MN - 
> Contractor:
> > 
> >> -----Original Message----- From: Johannes Sixt
> >> If EOL conversion or a clean filter was applied during 'git add
> >> file', is the version in the worktree suddenly wrong? Of course,
> >> not.
> >> 
> >> I would place $Id$ treatment in the same ball park and 
> declare it as
> >> a mistake of the editor that it did not remove the now "wrong" SHA1
> >> from $Id:$.
> > 
> > I think the difference here is that git does not *currently 
> change the
> > OS's LEF.  In this case each commit alters the Id and git is the one
> > altering the Id.
> 
> Maybe you misunderstood $Id$? The value you get there is the 
> blob's SHA1,
> not the commit's. That is, it does not change on every 
> commit, but only
> when the file changes.
> 
> You are right that the value itself is something that is 
> dictated by git's
> database model, but the change logically happens when the 
> editor modifies
> the file.
> 
> (My contribution to this thread should be regarded as food 
> for thought.
> Personally, I don't mind whether or not and when $Id$ is updated.)
>
Thank you for correcting me, I've always noticed this number doesn't seam to correlate to anything of use for me.  However it's been helpful when reading these reports to see what version generated it and that's why I wanted to further expand the information provided...  The date and time of the commit are specifically useful to me.

> -- Hannes
> 



Mike Mestnik, Michael J
The ESM Tools
Enterprise Systems Monitoring
United States Postal Service
 O: (651) 406-2048
Michael.J.Mestnik@usps.gov
ITEnterpriseSystemsMonitoring@usps.gov
 

Re: Using Format/export-subst Howto.

[Michael J Gruber]

On Fri, Sep 14, 2012, at 05:39 PM, Johannes Sixt wrote:
> Am 9/14/2012 17:27, schrieb Mestnik, Michael J - Eagan, MN - Contractor:
> > 
> >> -----Original Message----- From: Johannes Sixt
> >> If EOL conversion or a clean filter was applied during 'git add
> >> file', is the version in the worktree suddenly wrong? Of course,
> >> not.
> >> 
> >> I would place $Id$ treatment in the same ball park and declare it as
> >> a mistake of the editor that it did not remove the now "wrong" SHA1
> >> from $Id:$.
> > 
> > I think the difference here is that git does not *currently change the
> > OS's LEF.  In this case each commit alters the Id and git is the one
> > altering the Id.
> 
> Maybe you misunderstood $Id$? The value you get there is the blob's SHA1,
> not the commit's. That is, it does not change on every commit, but only
> when the file changes.
> 
> You are right that the value itself is something that is dictated by
> git's
> database model, but the change logically happens when the editor modifies
> the file.

Exactly, but the problem is that neither $Id$ nor $Id: deadbeef$ in the
work-tree copy of the file
are updated with $Id: abacbeef$ after the file's content has changed and
has been committed, i.e.
after the blob's sha1 has changed. What's worse, even a "git checkout
file" does not
correct this (because git sees that there's no change to the file
compared to the index),
you need to "rm file && git checkout file"). If the user has to update
$Id$ to match the current sha1
(by remembering to do a more forceful checkout than checkout -f) then
one half of that feature is useless. 

> (My contribution to this thread should be regarded as food for thought.
> Personally, I don't mind whether or not and when $Id$ is updated.)
> 
> -- Hannes

I think at least we should do a commit.renormalize akin to
merge.renormalize if we can't do
more for hysterical raisins. But maybe the behavior even has changed
during some
stat/lstat related optimizations. I'll check next week if nobody beats
me to it.

Cheers,
Michael

Re: Using Format/export-subst Howto.

[Junio C Hamano]

Michael J Gruber <git@drmicha.warpmail.net> writes:

> you need to "rm file && git checkout file"). If the user has to update
> $Id$ to match the current sha1
> (by remembering to do a more forceful checkout than checkout -f) then
> one half of that feature is useless. 

As if there is any value in "$Id$" _feature_.  It's a checkbox item,
nothing more ;-).

Re: Using Format/export-subst Howto.

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Michael J Gruber <git@drmicha.warpmail.net> writes:
>
>> you need to "rm file && git checkout file"). If the user has to update
>> $Id$ to match the current sha1
>> (by remembering to do a more forceful checkout than checkout -f) then
>> one half of that feature is useless. 
>
> As if there is any value in "$Id$" _feature_.  It's a checkbox item,
> nothing more ;-).

Having said that, I think you could do something along this line (I
am thinking aloud, so there may be leaps in the logic below).

 * Introduce a new on-disk flag in the index.  Call it X.  After
   entry.c:write_entry() writes it out to the working tree, this
   flag is cleared.  And this codepath is the only place that clears
   this flag.

 * When applying a clean filter (from here on, everything that
   breaks byte-for-byte identity between the copy on the working
   tree and the contents that is hashed and stored in the object
   store are considered "clean filter", including CRLF->LF and
   ident), internally apply the corresponding smudge filter to the
   cleaned result and compare it with the original input we obtained
   from the working tree.  If they differ, flip the X bit on for the
   path in the index.

 * When "checkout" and any potential callers of write_entry() decide
   whether it is worth calling write_entry() [*1*], consider any
   path with the X bit on as "dirty" and call write_entry().

You have to be very careful when designing the third point, though.
There will now be two kinds of "the working tree file is different
from the version registerd in the index" once you do the above.
Different only because of "clean->smudge" roundtrip comparison, and
different because it does have a real local modification.  The
former must be considered "no local modification" for the purpose of
merges and branch switching (otherwise you will get "cannot merge,
you have local modifications" error).


[Footnote]

*1* This currently is done primarily with ie_match_stat(), that 
essentially is "Does the result of applying 'clean' to the working
tree contents match what is registered to the index?  Do not bother
doing this check over and over again once you checked this until
the file in the working tree is modified again".

RE: Using Format/export-subst Howto.

[Mestnik, Michael J - Eagan, MN - Contractor]

> -----Original Message-----
> From: Junio C Hamano [mailto:gitster@pobox.com] 
> Sent: Friday, September 14, 2012 4:24 PM
> To: Michael J Gruber
> Cc: Johannes Sixt; Mestnik, Michael J - Eagan, MN - 
> Contractor; git@vger.kernel.org
> Subject: Re: Using Format/export-subst Howto.
> 
> Michael J Gruber <git@drmicha.warpmail.net> writes:
> 
> > you need to "rm file && git checkout file"). If the user 
> has to update
> > $Id$ to match the current sha1
> > (by remembering to do a more forceful checkout than 
> checkout -f) then
> > one half of that feature is useless. 
> 
> As if there is any value in "$Id$" _feature_.  It's a checkbox item,
> nothing more ;-).
> 

I agree here, $Id$ is much more useful in rcs/cvs.  That's why I chose to dive into export-subst, because I wanted my output to not only contain the uniq Id of the script but also the time and user of the last edit.

I'm going to read up on export today.  Just for context I'm using git to manage daily cron scripts, run on about 1,000 hosts, located on NFS.  I've moved the editing of the scripts into user's home directories as I'm a fan of not using editors on live data.  The key point is that when sudoing to the shared user who manages the files the commands run should be minimal and "pushd; ln -s ... .git; git pull; rm .git; popd;" works well.  If I can replace this with something that would populate these fields that would be awesome.

Cheers.

Mike Mestnik, Michael J
The ESM Tools
Enterprise Systems Monitoring
United States Postal Service
 O: (651) 406-2048
Michael.J.Mestnik@usps.gov
ITEnterpriseSystemsMonitoring@usps.gov
 

Re: Using Format/export-subst Howto.

[Michael J Gruber]

Junio C Hamano venit, vidit, dixit 14.09.2012 23:23:
> Michael J Gruber <git@drmicha.warpmail.net> writes:
> 
>> you need to "rm file && git checkout file"). If the user has to
>> update $Id$ to match the current sha1 (by remembering to do a more
>> forceful checkout than checkout -f) then one half of that feature
>> is useless.
> 
> As if there is any value in "$Id$" _feature_.  It's a checkbox item, 
> nothing more ;-).

It's your favorite feature^Wcheckbox item, I know ;)

I wouldn't mind dropping it or making it export-only, but the current
state of that item is quite confusing. I seem to remember this has been
brought up before.

Junio C Hamano venit, vidit, dixit 15.09.2012 00:26:
> Junio C Hamano <gitster@pobox.com> writes:
> 
>> Michael J Gruber <git@drmicha.warpmail.net> writes:
>> 
>>> you need to "rm file && git checkout file"). If the user has to
>>> update $Id$ to match the current sha1 (by remembering to do a
>>> more forceful checkout than checkout -f) then one half of that
>>> feature is useless.
>> 
>> As if there is any value in "$Id$" _feature_.  It's a checkbox
>> item, nothing more ;-).
> 
> Having said that, I think you could do something along this line (I 
> am thinking aloud, so there may be leaps in the logic below).
> 
> * Introduce a new on-disk flag in the index.  Call it X.  After 
> entry.c:write_entry() writes it out to the working tree, this flag is
> cleared.  And this codepath is the only place that clears this flag.
> 
> * When applying a clean filter (from here on, everything that breaks
> byte-for-byte identity between the copy on the working tree and the
> contents that is hashed and stored in the object store are considered
> "clean filter", including CRLF->LF and ident), internally apply the
> corresponding smudge filter to the cleaned result and compare it with
> the original input we obtained from the working tree.  If they
> differ, flip the X bit on for the path in the index.
> 
> * When "checkout" and any potential callers of write_entry() decide 
> whether it is worth calling write_entry() [*1*], consider any path
> with the X bit on as "dirty" and call write_entry().
> 
> You have to be very careful when designing the third point, though. 
> There will now be two kinds of "the working tree file is different 
> from the version registerd in the index" once you do the above. 
> Different only because of "clean->smudge" roundtrip comparison, and 
> different because it does have a real local modification.  The former
> must be considered "no local modification" for the purpose of merges
> and branch switching (otherwise you will get "cannot merge, you have
> local modifications" error).
> 
> 
> [Footnote]
> 
> *1* This currently is done primarily with ie_match_stat(), that 
> essentially is "Does the result of applying 'clean' to the working 
> tree contents match what is registered to the index?  Do not bother 
> doing this check over and over again once you checked this until the
> file in the working tree is modified again".
> 

You've convinced me not to try this in-core...

Maybe a post-commit hook is enough for Id. It's just that we also have
no in-core way of doing a forceful enough checkout on those files. But I
wouldn't mind making Id export-only (export-subst). Really, most people
who look at Id really want something like VERSION_GEN (without having to
use a build script/make/...).

Michael

RE: Using Format/export-subst Howto.

[Mestnik, Michael J - Eagan, MN - Contractor]

> -----Original Message-----
> From: Michael J Gruber [mailto:git@drmicha.warpmail.net] 
> Sent: Monday, September 17, 2012 8:17 AM
> To: Junio C Hamano
> Cc: Johannes Sixt; Mestnik, Michael J - Eagan, MN - 
> Contractor; git@vger.kernel.org
> Subject: Re: Using Format/export-subst Howto.
> 
> Junio C Hamano venit, vidit, dixit 14.09.2012 23:23:
> > Michael J Gruber <git@drmicha.warpmail.net> writes:
> > 
> >> you need to "rm file && git checkout file"). If the user has to
> >> update $Id$ to match the current sha1 (by remembering to do a more
> >> forceful checkout than checkout -f) then one half of that feature
> >> is useless.
> > 
> > As if there is any value in "$Id$" _feature_.  It's a 
> checkbox item, 
> > nothing more ;-).
> 
> It's your favorite feature^Wcheckbox item, I know ;)
> 
> I wouldn't mind dropping it or making it export-only, but the current
> state of that item is quite confusing. I seem to remember 
> this has been
> brought up before.
> 
> Junio C Hamano venit, vidit, dixit 15.09.2012 00:26:
> > Junio C Hamano <gitster@pobox.com> writes:
> > 
> >> Michael J Gruber <git@drmicha.warpmail.net> writes:
> >> 
> >>> you need to "rm file && git checkout file"). If the user has to
> >>> update $Id$ to match the current sha1 (by remembering to do a
> >>> more forceful checkout than checkout -f) then one half of that
> >>> feature is useless.
> >> 
> >> As if there is any value in "$Id$" _feature_.  It's a checkbox
> >> item, nothing more ;-).
> > 
> > Having said that, I think you could do something along this line (I 
> > am thinking aloud, so there may be leaps in the logic below).
> > 
> > * Introduce a new on-disk flag in the index.  Call it X.  After 
> > entry.c:write_entry() writes it out to the working tree, 
> this flag is
> > cleared.  And this codepath is the only place that clears this flag.
> > 
> > * When applying a clean filter (from here on, everything that breaks
> > byte-for-byte identity between the copy on the working tree and the
> > contents that is hashed and stored in the object store are 
> considered
> > "clean filter", including CRLF->LF and ident), internally apply the
> > corresponding smudge filter to the cleaned result and 
> compare it with
> > the original input we obtained from the working tree.  If they
> > differ, flip the X bit on for the path in the index.
> > 
> > * When "checkout" and any potential callers of write_entry() decide 
> > whether it is worth calling write_entry() [*1*], consider any path
> > with the X bit on as "dirty" and call write_entry().
> > 
> > You have to be very careful when designing the third point, though. 
> > There will now be two kinds of "the working tree file is different 
> > from the version registerd in the index" once you do the above. 
> > Different only because of "clean->smudge" roundtrip comparison, and 
> > different because it does have a real local modification.  
> The former
> > must be considered "no local modification" for the purpose of merges
> > and branch switching (otherwise you will get "cannot merge, you have
> > local modifications" error).
> > 
> > 
> > [Footnote]
> > 
> > *1* This currently is done primarily with ie_match_stat(), that 
> > essentially is "Does the result of applying 'clean' to the working 
> > tree contents match what is registered to the index?  Do not bother 
> > doing this check over and over again once you checked this until the
> > file in the working tree is modified again".
> > 
> 
> You've convinced me not to try this in-core...
> 
> Maybe a post-commit hook is enough for Id. It's just that we also have
> no in-core way of doing a forceful enough checkout on those 
> files. But I
> wouldn't mind making Id export-only (export-subst). Really, 
> most people
> who look at Id really want something like VERSION_GEN 
I'm just reporting that I didn't think Id was behaving as it should in my working folder, this is of no consequence for me.  After git pull, AFAIK, the Id has the correct/new value if the file has changed.

> (without having to
> use a build script/make/...).
> 
> Michael
> 

I would not be opposed to going the build script/make route, if I could do it in a hook.  I'm using git to produce the final product in my case, I don't want this live folder to be used as a temporary place for doing builds.  I guess I'm looking for behavior similar to install.

Cheers.

Mike Mestnik, Michael J
The ESM Tools
Enterprise Systems Monitoring
United States Postal Service
 O: (651) 406-2048
Michael.J.Mestnik@usps.gov
ITEnterpriseSystemsMonitoring@usps.gov
 

Re: Using Format/export-subst Howto.

[Michael J Gruber]

Mestnik, Michael J - Eagan, MN - Contractor venit, vidit, dixit
17.09.2012 14:12:
> 
>> -----Original Message-----
>> From: Junio C Hamano [mailto:gitster@pobox.com] 
>> Sent: Friday, September 14, 2012 4:24 PM
>> To: Michael J Gruber
>> Cc: Johannes Sixt; Mestnik, Michael J - Eagan, MN - 
>> Contractor; git@vger.kernel.org
>> Subject: Re: Using Format/export-subst Howto.
>>
>> Michael J Gruber <git@drmicha.warpmail.net> writes:
>>
>>> you need to "rm file && git checkout file"). If the user 
>> has to update
>>> $Id$ to match the current sha1
>>> (by remembering to do a more forceful checkout than 
>> checkout -f) then
>>> one half of that feature is useless. 
>>
>> As if there is any value in "$Id$" _feature_.  It's a checkbox item,
>> nothing more ;-).
>>
> 
> I agree here, $Id$ is much more useful in rcs/cvs.  That's why I chose to dive into export-subst, because I wanted my output to not only contain the uniq Id of the script but also the time and user of the last edit.
> 
> I'm going to read up on export today.  Just for context I'm using git to manage daily cron scripts, run on about 1,000 hosts, located on NFS.  I've moved the editing of the scripts into user's home directories as I'm a fan of not using editors on live data.  The key point is that when sudoing to the shared user who manages the files the commands run should be minimal and "pushd; ln -s ... .git; git pull; rm .git; popd;" works well.  If I can replace this with something that would populate these fields that would be awesome.

I'm not completely sure about the setup, but something like

GIT_DIR=fromthisrepo git archive usethisbranchortag |
git tar -xf- -C thereyougo

may do the job (and expand export-subst formats).

Cheers
Michael

RE: Using Format/export-subst Howto.

[Mestnik, Michael J - Eagan, MN - Contractor]

> -----Original Message-----
> From: Michael J Gruber [mailto:git@drmicha.warpmail.net] 
> Sent: Monday, September 17, 2012 8:45 AM
> To: Mestnik, Michael J - Eagan, MN - Contractor
> Cc: Junio C Hamano; Johannes Sixt; git@vger.kernel.org
> Subject: Re: Using Format/export-subst Howto.
> 
> Mestnik, Michael J - Eagan, MN - Contractor venit, vidit, dixit
> 17.09.2012 14:12:
> > 
> >> -----Original Message-----
> >> From: Junio C Hamano [mailto:gitster@pobox.com] 
> >> Sent: Friday, September 14, 2012 4:24 PM
> >> To: Michael J Gruber
> >> Cc: Johannes Sixt; Mestnik, Michael J - Eagan, MN - 
> >> Contractor; git@vger.kernel.org
> >> Subject: Re: Using Format/export-subst Howto.
> >>
> >> Michael J Gruber <git@drmicha.warpmail.net> writes:
> >>
> >>> you need to "rm file && git checkout file"). If the user 
> >> has to update
> >>> $Id$ to match the current sha1
> >>> (by remembering to do a more forceful checkout than 
> >> checkout -f) then
> >>> one half of that feature is useless. 
> >>
> >> As if there is any value in "$Id$" _feature_.  It's a 
> checkbox item,
> >> nothing more ;-).
> >>
> > 
> > I agree here, $Id$ is much more useful in rcs/cvs.  That's 
> why I chose to dive into export-subst, because I wanted my 
> output to not only contain the uniq Id of the script but also 
> the time and user of the last edit.
> > 
> > I'm going to read up on export today.  Just for context I'm 
> using git to manage daily cron scripts, run on about 1,000 
> hosts, located on NFS.  I've moved the editing of the scripts 
> into user's home directories as I'm a fan of not using 
> editors on live data.  The key point is that when sudoing to 
> the shared user who manages the files the commands run should 
> be minimal and "pushd; ln -s ... .git; git pull; rm .git; 
> popd;" works well.  If I can replace this with something that 
> would populate these fields that would be awesome.
> 
> I'm not completely sure about the setup, but something like
> 
> GIT_DIR=fromthisrepo git archive usethisbranchortag |
> git tar -xf- -C thereyougo
> 
Yes, that's about what the man page says.  I though I could pass --keep-newer-files to tar and that would prevent files from being re-created, but a "git archive --format=tar HEAD | tar -tvv" indicates that every file has the current date/time and not, for example, the time of the last commit that modified that file.  Not only would pushing every file be undesirable it would also cause every server to pull every file and that's unacceptable.  Another down side, or perhaps this is the downside, to this method is that it lacks any form of merge capability and will destroy local changes.

It looks like I'll be doing this in a hook.

BTW: I couldn't find "git tar", if it exists that might be the way for me...  Depending on what it would do.

Thank you for all the insight!

> may do the job (and expand export-subst formats).
> 
> Cheers
> Michael
> 


Mike Mestnik, Michael J
The ESM Tools
Enterprise Systems Monitoring
United States Postal Service
 O: (651) 406-2048
Michael.J.Mestnik@usps.gov
ITEnterpriseSystemsMonitoring@usps.gov
 

Re: Using Format/export-subst Howto.

[Michael J Gruber]

Mestnik, Michael J - Eagan, MN - Contractor venit, vidit, dixit
17.09.2012 16:08:
> 
>> -----Original Message-----
>> From: Michael J Gruber [mailto:git@drmicha.warpmail.net] 
>> Sent: Monday, September 17, 2012 8:45 AM
>> To: Mestnik, Michael J - Eagan, MN - Contractor
>> Cc: Junio C Hamano; Johannes Sixt; git@vger.kernel.org
>> Subject: Re: Using Format/export-subst Howto.
>>
>> Mestnik, Michael J - Eagan, MN - Contractor venit, vidit, dixit
>> 17.09.2012 14:12:
>>>
>>>> -----Original Message-----
>>>> From: Junio C Hamano [mailto:gitster@pobox.com] 
>>>> Sent: Friday, September 14, 2012 4:24 PM
>>>> To: Michael J Gruber
>>>> Cc: Johannes Sixt; Mestnik, Michael J - Eagan, MN - 
>>>> Contractor; git@vger.kernel.org
>>>> Subject: Re: Using Format/export-subst Howto.
>>>>
>>>> Michael J Gruber <git@drmicha.warpmail.net> writes:
>>>>
>>>>> you need to "rm file && git checkout file"). If the user 
>>>> has to update
>>>>> $Id$ to match the current sha1
>>>>> (by remembering to do a more forceful checkout than 
>>>> checkout -f) then
>>>>> one half of that feature is useless. 
>>>>
>>>> As if there is any value in "$Id$" _feature_.  It's a 
>> checkbox item,
>>>> nothing more ;-).
>>>>
>>>
>>> I agree here, $Id$ is much more useful in rcs/cvs.  That's 
>> why I chose to dive into export-subst, because I wanted my 
>> output to not only contain the uniq Id of the script but also 
>> the time and user of the last edit.
>>>
>>> I'm going to read up on export today.  Just for context I'm 
>> using git to manage daily cron scripts, run on about 1,000 
>> hosts, located on NFS.  I've moved the editing of the scripts 
>> into user's home directories as I'm a fan of not using 
>> editors on live data.  The key point is that when sudoing to 
>> the shared user who manages the files the commands run should 
>> be minimal and "pushd; ln -s ... .git; git pull; rm .git; 
>> popd;" works well.  If I can replace this with something that 
>> would populate these fields that would be awesome.
>>
>> I'm not completely sure about the setup, but something like
>>
>> GIT_DIR=fromthisrepo git archive usethisbranchortag |
>> git tar -xf- -C thereyougo
>>
> Yes, that's about what the man page says.  I though I could pass --keep-newer-files to tar and that would prevent files from being re-created, but a "git archive --format=tar HEAD | tar -tvv" indicates that every file has the current date/time and not, for example, the time of the last commit that modified that file.  Not only would pushing every file be undesirable it would also cause every server to pull every file and that's unacceptable.  Another down side, or perhaps this is the downside, to this method is that it lacks any form of merge capability and will destroy local changes.

I thought it's only a few files and you don't care about merging (in
sudo), only about deploying a checkout, since you're removing the git dir.

> It looks like I'll be doing this in a hook.
> 
> BTW: I couldn't find "git tar", if it exists that might be the way for me...  Depending on what it would do.

Oh yeah. Done too much git work lately. I know I keep using "git help
foo" rather than "man foo", but this one is new :)

I just meant tar.

Michael

RE: Using Format/export-subst Howto.

[Mestnik, Michael J - Eagan, MN - Contractor]

> -----Original Message-----
> From: Michael J Gruber [mailto:git@drmicha.warpmail.net] 
> Sent: Monday, September 17, 2012 9:21 AM
> To: Mestnik, Michael J - Eagan, MN - Contractor
> Cc: Junio C Hamano; Johannes Sixt; git@vger.kernel.org
> Subject: Re: Using Format/export-subst Howto.
> 
> Mestnik, Michael J - Eagan, MN - Contractor venit, vidit, dixit
> 17.09.2012 16:08:
> > 
> >> -----Original Message-----
> >> From: Michael J Gruber [mailto:git@drmicha.warpmail.net] 
> >> Sent: Monday, September 17, 2012 8:45 AM
> >> To: Mestnik, Michael J - Eagan, MN - Contractor
> >> Cc: Junio C Hamano; Johannes Sixt; git@vger.kernel.org
> >> Subject: Re: Using Format/export-subst Howto.
> >>
> >> Mestnik, Michael J - Eagan, MN - Contractor venit, vidit, dixit
> >> 17.09.2012 14:12:
> >>>
> >>>> -----Original Message-----
> >>>> From: Junio C Hamano [mailto:gitster@pobox.com] 
> >>>> Sent: Friday, September 14, 2012 4:24 PM
> >>>> To: Michael J Gruber
> >>>> Cc: Johannes Sixt; Mestnik, Michael J - Eagan, MN - 
> >>>> Contractor; git@vger.kernel.org
> >>>> Subject: Re: Using Format/export-subst Howto.
> >>>>
> >>>> Michael J Gruber <git@drmicha.warpmail.net> writes:
> >>>>
> >>>>> you need to "rm file && git checkout file"). If the user 
> >>>> has to update
> >>>>> $Id$ to match the current sha1
> >>>>> (by remembering to do a more forceful checkout than 
> >>>> checkout -f) then
> >>>>> one half of that feature is useless. 
> >>>>
> >>>> As if there is any value in "$Id$" _feature_.  It's a 
> >> checkbox item,
> >>>> nothing more ;-).
> >>>>
> >>>
> >>> I agree here, $Id$ is much more useful in rcs/cvs.  That's 
> >> why I chose to dive into export-subst, because I wanted my 
> >> output to not only contain the uniq Id of the script but also 
> >> the time and user of the last edit.
> >>>
> >>> I'm going to read up on export today.  Just for context I'm 
> >> using git to manage daily cron scripts, run on about 1,000 
> >> hosts, located on NFS.  I've moved the editing of the scripts 
> >> into user's home directories as I'm a fan of not using 
> >> editors on live data.  The key point is that when sudoing to 
> >> the shared user who manages the files the commands run should 
> >> be minimal and "pushd; ln -s ... .git; git pull; rm .git; 
> >> popd;" works well.  If I can replace this with something that 
> >> would populate these fields that would be awesome.
> >>
> >> I'm not completely sure about the setup, but something like
> >>
> >> GIT_DIR=fromthisrepo git archive usethisbranchortag |
> >> git tar -xf- -C thereyougo
> >>
> > Yes, that's about what the man page says.  I though I could 
> pass --keep-newer-files to tar and that would prevent files 
> from being re-created, but a "git archive --format=tar HEAD | 
> tar -tvv" indicates that every file has the current date/time 
> and not, for example, the time of the last commit that 
> modified that file.  Not only would pushing every file be 
> undesirable it would also cause every server to pull every 
> file and that's unacceptable.  Another down side, or perhaps 
> this is the downside, to this method is that it lacks any 
> form of merge capability and will destroy local changes.
> 
> I thought it's only a few files and you don't care about merging (in
> sudo), only about deploying a checkout, since you're removing 
> the git dir.
> 
Allow me to explain.  There was an issue with copying the .git folder to every host, we literally don't have the space for that and I wonder what happens when these files grow by a few hundred k.  I had put in the symlink, but our replicator script was chocking on it.  Thus I've resorted to not only using a symlink, but also removing it when not in use.  Correcting these issues is lower priority then making the logs have all the information they need.

Merging has been done when edits are needed ASAP and git becomes an after thought.  I've done pushes from the live files too, though it is something that happens after an emergency.

I know I've got a rather strange environment, most of it surprises me as well.

> > It looks like I'll be doing this in a hook.
> > 
> > BTW: I couldn't find "git tar", if it exists that might be 
> the way for me...  Depending on what it would do.
> 
> Oh yeah. Done too much git work lately. I know I keep using "git help
> foo" rather than "man foo", but this one is new :)
> 
> I just meant tar.
> 
> Michael
> 

Mike Mestnik, Michael J
The ESM Tools
Enterprise Systems Monitoring
United States Postal Service
 O: (651) 406-2048
Michael.J.Mestnik@usps.gov
ITEnterpriseSystemsMonitoring@usps.gov
 

Re: Using Format/export-subst Howto.

[Junio C Hamano]

Michael J Gruber <git@drmicha.warpmail.net> writes:

> Junio C Hamano venit, vidit, dixit 15.09.2012 00:26:
>> Junio C Hamano <gitster@pobox.com> writes:
>> 
>>> Michael J Gruber <git@drmicha.warpmail.net> writes:
>>> 
>>>> you need to "rm file && git checkout file"). If the user has to
>>>> update $Id$ to match the current sha1 (by remembering to do a
>>>> more forceful checkout than checkout -f) then one half of that
>>>> feature is useless.
>>> 
>>> As if there is any value in "$Id$" _feature_.  It's a checkbox
>>> item, nothing more ;-).
>> 
>> Having said that, I think you could do something along this line (I 
>> am thinking aloud, so there may be leaps in the logic below).
>> ...
>
> You've convinced me not to try this in-core...

Quite contrary, I think the approach outlined is a reasonable way
forward.  I do not care about a checkbox feature like $Id$ that is
incomatible with the fundamental concept in Git, but the approach
would help people who started their history with wrong CRLF
settings, tweak the attributes and then scratch their head trying to
repeatedly run "checkout", "reset --hard", etc. and still never
seeing the right thing to happen.

Re: Using Format/export-subst Howto.

[Michael J Gruber]

Junio C Hamano venit, vidit, dixit 17.09.2012 22:21:
> Michael J Gruber <git@drmicha.warpmail.net> writes:
> 
>> Junio C Hamano venit, vidit, dixit 15.09.2012 00:26:
>>> Junio C Hamano <gitster@pobox.com> writes:
>>>
>>>> Michael J Gruber <git@drmicha.warpmail.net> writes:
>>>>
>>>>> you need to "rm file && git checkout file"). If the user has to
>>>>> update $Id$ to match the current sha1 (by remembering to do a
>>>>> more forceful checkout than checkout -f) then one half of that
>>>>> feature is useless.
>>>>
>>>> As if there is any value in "$Id$" _feature_.  It's a checkbox
>>>> item, nothing more ;-).
>>>
>>> Having said that, I think you could do something along this line (I 
>>> am thinking aloud, so there may be leaps in the logic below).
>>> ...
>>
>> You've convinced me not to try this in-core...
> 
> Quite contrary, I think the approach outlined is a reasonable way
> forward.  I do not care about a checkbox feature like $Id$ that is
> incomatible with the fundamental concept in Git, but the approach
> would help people who started their history with wrong CRLF
> settings, tweak the attributes and then scratch their head trying to
> repeatedly run "checkout", "reset --hard", etc. and still never
> seeing the right thing to happen.
> 

Maybe I should have said "...not to try this myself..." to make it
clearer: The approach you outlined sounds not only reasonable but the
perfect solution to this and other problems. But it's over my head and
time limit (and it's not my itch either). It touches the inner core
(index format and handling) and needs to be done diligently, too much
for a quick patch on the side of my $DAYJOB. (Though, thinking about it:
some colleagues use a similar hg feature to revision stamp their LaTeX
preprints, and working Id could help win them over.)

Michael