split directories into branches

split directories into branches

[shawn wilson]

I've got a chef cookbook repo where everyone started developing
cookbooks in a single dev branch (not project specific). Minus a few
edge cases, it should be fairly simple to split this up into feature
branches based on /cookbooks/<feature>.

I tried:
$ git filter-branch --subdirectory-filter cookbooks/<feature>-- <feature>
And
$ git subtree split --prefix cookbooks/<feature> -b <feature>

Which both seem to do the same thing (haven't looked at the subtree
bash - guessing it does exactly the filter-branch). The issue is that
it removes the directory tree (so obviously merges wouldn't work). I'm
thinking some type of filter-branch --index-filter with a cherry pick
(or similar) should work...?

Re: split directories into branches

[Konstantin Khomoutov]

On Mon, 4 Jul 2016 12:45:39 -0400
shawn wilson <ag4ve.us@gmail.com> wrote:

> I've got a chef cookbook repo where everyone started developing
> cookbooks in a single dev branch (not project specific). Minus a few
> edge cases, it should be fairly simple to split this up into feature
> branches based on /cookbooks/<feature>.
> 
> I tried:
> $ git filter-branch --subdirectory-filter cookbooks/<feature>--
> <feature> And
> $ git subtree split --prefix cookbooks/<feature> -b <feature>
> 
> Which both seem to do the same thing (haven't looked at the subtree
> bash - guessing it does exactly the filter-branch). The issue is that
> it removes the directory tree (so obviously merges wouldn't work). I'm
> thinking some type of filter-branch --index-filter with a cherry pick
> (or similar) should work...?

To achieve what you're after I used `git subtree split` followed by
`git filter-branch --tree-filter ...` which moved all the files under
the directory hierarchy `git subtree split` removes.

I'd love if `git subtree split` had an option to preserve the prefix
(even better would be to have another option to rewrite the prefix)
because that would greatly simplify another use case for `git subtree`:
moving "a directory" with its full history from one repository into
another.  Presently, the user is able to split that directory out from
the source repository but when they subtree-merge it back in the new
repository, they are puzzled by the fact `git log` invoked on the new
history with the pathname prefix designating the "inserted" directory
does not traverse past the merge point which brought that directory in.

I know Git tracks content, not files (and directory) but still many
folks have "stable" directories for their files, assign certain
semantics to them etc.  I've needed such transfers myself, and this
topic has been raised more than once by folks over there on the
git-users mailing list.

Re: split directories into branches

[shawn wilson]

On Mon, Jul 4, 2016 at 1:39 PM, Konstantin Khomoutov
<kostix+git@007spb.ru> wrote:
> On Mon, 4 Jul 2016 12:45:39 -0400
> shawn wilson <ag4ve.us@gmail.com> wrote:
>

>
> To achieve what you're after I used `git subtree split` followed by
> `git filter-branch --tree-filter ...` which moved all the files under
> the directory hierarchy `git subtree split` removes.
>

Ah, that looks like it should work - thanks

> I know Git tracks content, not files (and directory) but still many
> folks have "stable" directories for their files, assign certain
> semantics to them etc.  I've needed such transfers myself, and this
> topic has been raised more than once by folks over there on the
> git-users mailing list.

Or just an option for git {rm,mv,cp} that would allow simple rewriting
of history. Ie:
git mv HEAD~3.. -- foo bar
(obviously not the sanest thing in most cases but would be a shorter
reach than filter-branch - and if newest isn't head then it'd what?
move it back during commits for who knows what - still...)

Thanks for pointing out the users list - didn't notice it and sorry
for posting a user question on a dev list.

Re: split directories into branches

[Konstantin Khomoutov]

On Mon, 4 Jul 2016 14:15:58 -0400
shawn wilson <ag4ve.us@gmail.com> wrote:

[...]
> > I know Git tracks content, not files (and directory) but still many
> > folks have "stable" directories for their files, assign certain
> > semantics to them etc.  I've needed such transfers myself, and this
> > topic has been raised more than once by folks over there on the
> > git-users mailing list.
[...]
> Thanks for pointing out the users list - didn't notice it and sorry
> for posting a user question on a dev list.

I don't possess the official stance on this topic but AFAIK user-level
questions are fine on this list.  The git-users mailing list was
created -- as I understand it -- because of three reasons: 1) it's
easier to post to; 2) you don't receive all the bug reports and patch
traffic irrelevant to mere mortals; 3) you have good chances to get
even RTFM questions answered (mostly by those who just had RTFM
recently), though I'd say stackoverflow is better at this one. ;-)

Re: split directories into branches

[shawn wilson]

On Mon, Jul 4, 2016 at 2:29 PM, Konstantin Khomoutov
<kostix+git@007spb.ru> wrote:
> On Mon, 4 Jul 2016 14:15:58 -0400
> shawn wilson <ag4ve.us@gmail.com> wrote:
>

> I don't possess the official stance on this topic but AFAIK user-level
> questions are fine on this list.

In that case :)
... still having issues w/ filter-branch:

$ git filter-branch --tree-filter "shopt -s extglob && mkdir -p
cookbooks/base_sys && mv !(cookbooks) cookbooks/base_sys"
Rewrite a90fb34230e02d9494934e5549f36bdbdd1b6ce6 (1/6) (0 seconds
passed, remaining 0 predicted)
/usr/local/libexec/git-core/git-filter-branch: eval: line 360: syntax
error near unexpected token `('
/usr/local/libexec/git-core/git-filter-branch: eval: line 360: `shopt
-s extglob && mkdir -p cookbooks/base_sys && mv !(cookbooks)
cookbooks/base_sys'
/usr/local/libexec/git-core/git-filter-branch: line 360: warning:
syntax errors in . or eval will cause future versions of the shell to
abort as Posix requires
tree filter failed: shopt -s extglob && mkdir -p cookbooks/base_sys &&
mv !(cookbooks) cookbooks/base_sys

I'm guessing it's because of how filter-branch is eval'ing stuff. But
if I do !\(cookbooks\) it interprets the '(' and ')' as text - ideas?

Re: split directories into branches

[Andreas Schwab]

shawn wilson <ag4ve.us@gmail.com> writes:

> $ git filter-branch --tree-filter "shopt -s extglob && mkdir -p
> cookbooks/base_sys && mv !(cookbooks) cookbooks/base_sys"

extglob changes the parser, you need to set that on a separate line.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: split directories into branches

[shawn wilson]

On Mon, Jul 4, 2016 at 6:10 PM, Andreas Schwab <schwab@linux-m68k.org> wrote:
> shawn wilson <ag4ve.us@gmail.com> writes:
>
>> $ git filter-branch --tree-filter "shopt -s extglob && mkdir -p
>> cookbooks/base_sys && mv !(cookbooks) cookbooks/base_sys"
>
> extglob changes the parser, you need to set that on a separate line.
>

by "separate line", I'm guessing you mean outside of the filter-branch
eval? Something like this?
shopt -s extglob; declare -a f=(!(cookbooks)); git filter-branch
--tree-filter "mkdir -p cookbooks/base_sys && mv $f
cookbooks/base_sys"

(don't have access to the repo atm and am curious if this is what you mean)

Re: split directories into branches

[shawn wilson]

correction

On Mon, Jul 4, 2016 at 7:30 PM, shawn wilson <ag4ve.us@gmail.com> wrote:

> shopt -s extglob; declare -a f=(!(cookbooks)); git filter-branch
> --tree-filter "mkdir -p cookbooks/base_sys && mv ${f[@]}
> cookbooks/base_sys"
>

Re: split directories into branches

[Konstantin Khomoutov]

On Mon, 4 Jul 2016 17:03:46 -0400
shawn wilson <ag4ve.us@gmail.com> wrote:

[...]
> > I don't possess the official stance on this topic but AFAIK
> > user-level questions are fine on this list.
> 
> In that case :)
> ... still having issues w/ filter-branch:
[...]

I used something along these lines:

  git filter-branch -f --tree-filter \
    'test -e my/new/dir || mkdir -p my/new/dir
      find . -mindepth 1 -maxdepth 1 \
      -type d -path ./my -prune -o -print \
      | xargs -n 30 mv -t ./my/new/dir' master 

More background on how it works [1] if you want.

1. https://groups.google.com/d/msg/git-users/hxFmfUZpj_k/9IQAQq40BwAJ