Git & Project Workflow

Git & Project Workflow

[Tommy Wang]

Hi,

I'm trying to set up a project using git, and I'm having a hard time
figuring out the best way to implement one of my workflows.

Lets say I have a repository "main".  Some of my developers will have
full access to this repository, while others should have access only
to certain parts of it (and, no, unfortunately the files are scattered
all over the place, and reorganizing isn't an option, so I can't use
subtrees).

I've attempted to solve this project by cloning the project into
another repository "limited".  Now, half of my developers can work off
from "main", while the other half can work from "limited".
Incidentally, its okay that the original source is visible in limited,
I just don't want the work we put into "main" to show up in "limited".

So, lets say my "main" developer makes some changes to the project.
Some of his changes are to kept in "main" only, while other changes
need to be pushed into "limited".  I can maybe force developers to
keep changes in common files in separate commits, and just cherry pick
them into "limited".  "Limited" developers can then pull those changes
into his local repository, resolve conflicts, (rebase), and push them
back into "limited".  At some point later in time, lets suppose we
need to push changes made in limited back to main.  So, our main
developers will cherry pick again the newest changes into limited (to
keep the conflicts down).  Limited developers will then
commit/resolve/rebase their work into limited, and let the main
developers know that they're ready.  The main developers will now pull
from limited -- and since we (likely) haven't changed main, the pull
should be pretty conflict free (git does keep track of cherry-commits
in merges, right?).

My question is, is cherry-picking really the best way to do this?
What I'd really like to do is to create the limited repository by
doing a filter-branch and removing secure files out of the repository
entirely.  After doing this, though, it makes it difficult/impossible
to move changes between the two repositories.  I tried a number of
approaches, including trying to graft the two repositories together
after the filter-branch to make push/pull work as simple
merges/fast-forwards.  I would have to figure out way to filter
commits to make sure that main doesn't push a secure file into
limited, and that limited doesn't try to do something silly like
creating a secure file.  Perhaps I could use filter-branch to rewrite
these commits as well.  But, being that filter-branch totally changes
the commit ids, I figure that this approach really doesn't have any
merit.

I also considered doing something like this:

# create main & limited from upstream source
git clone upstream main
git clone main limited

# remove secure files from limited
cd limited
git rm -r secure files
git commit

# fake a merge to keep histories correct
cd main
git pull -s ours limited

# pulling from limited worked well, and would only get changes made after the rm
cd main
git pull limited master

# pull from main didn't work so well --> but I probably should be
cherry-picking here anyway?
cd limited
git pull main master

I'm not sure why I'm trying so hard to avoid cherry-picking.  Probably
due to the fact that developers need to be more intimate with git, and
that it takes considerably more time to look through a list of
commits, and pick out the ones that can be moved from main -> limited.

Is there a way to generate a list of commits based on a list of files
(paths aren't good enough, since my files are scattered all over the
place) that can be found in main but not limited?  Perhaps I can write
a script that can be used to automate the cherry-picking from main ->
limited.  If we run this script right after we resolve all the
conflicts from a limited -> main merge, we should be able to avoid all
conflicts.

Anyway, I'm hoping for some thoughts/direction on how I may be able to
solve my problem.  Perhaps there is a magical solution that is plainly
obvious to someone with more git experience?

Thanks,
Tommy

Re: Git & Project Workflow

[Nicolas Sebrecht]

The 05/11/09, Tommy Wang wrote:

> Hi,

Hi,

>      it takes considerably more time to look through a list of
> commits, and pick out the ones that can be moved from main -> limited.

Instead of doing this massive (time consuming) cherry-picking, why don't
you avoid the whole "main -> limited" workflow ? This looks a bit ugly
for me.

I think that the main developers should work by default with the limited
repo and only commit for the secret files into main. This would result
in only pulling limited work into main ("limited -> main" workflow).

Let's rename main to secret and limited to normal to have the semantic
stick with my point. In this case, whole developers work with normal for
the "normal" tasks. If they need a secret change, secret developers
commit and push to secret. Then, you only need to pull normal from
secret and resolve conflicts in one place.


-- 
Nicolas Sebrecht