Submodules or similar for exercise/exam management

Submodules or similar for exercise/exam management

[Thomas Rast]

Hi all

[Jens, I was hoping you'd have some insight, but of course everyone is
welcome to join in.  I honestly never thought I'd get into the
submodule business so shortly after gittogether...]


Out of $DAYJOB issues with tracking exercises and exams, I had a
little brainstorming session with a fellow assistant yesterday.  It
seems none of this has been done before?  Is anyone else interested
(mainly in getting the requirements right so that the next poor soul
won't have to roll their own)?

The scenario is that we have a bunch of exercise questions stored in
one or several files, each living in a directory.  Then, we assemble
those into exercise sheets (super-repos) spanning a group of questions
(submodules).  We would like to track this in such a way that changes
to the questions propagate to other sheets (possibly in other
lectures) that use them.

Training everyone in full git usage is probably not an option, so any
solution would have to have some frontend that can be explained
easily.

The requirements we came up with are roughly:

1) commit across all sub-repos at the same time (atomicity nice but
   optional)

  1b) tag across all sub-repos

2) do recursive clone/update of one super-repo easily

3) never need to be aware of repo boundaries or manipulate sub-repo

4) sanely cope with branches etc. in case the user wants them

A sample workflow might be:

  foo clone git@example.com/some/super/repo
  # time passes
  cd repo
  foo update
  # work
  foo status
  foo diff
  foo commit -m 'one message fits all'
  # possibly, but this could also be commit --no-push instead
  foo push

Merges are expected to be rare but would happen every so often.

It seems repo can do (2) and (4) but violates (3) [requires use of
git-commit and others in the sub-repo].  git-subtree can do (1) and
(2) but probably violates (3) [need to rebase changes to sub-repo
branch] and (4).  Submodules can do (2) and (4) but violate (3) and
currently have no support for (1).  I think svn externals could do
(1)-(3) but violate (4) and probably (1b).

Has this been done before?  Or do we need to hack up a new wrapper
around submodules?

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Re: Submodules or similar for exercise/exam management

[Seth Robertson]

In message <201011181109.08345.trast@student.ethz.ch>, Thomas Rast writes:

    The requirements we came up with are roughly:

    1) commit across all sub-repos at the same time (atomicity nice but
       optional)
      1b) tag across all sub-repos
    2) do recursive clone/update of one super-repo easily
    3) never need to be aware of repo boundaries or manipulate sub-repo
    4) sanely cope with branches etc. in case the user wants them

This is exactly the sort of situation I wrote gitslave for.
http://gitslave.sf.net  I have a superproject of many subprojects where
I want to perform the same git operation on all of them at the same
time as if it was one giant repository (without the drawbacks of a
giant repository while losing as few as the benefits as possible).

It does not perform atomic commits across sub-repos, but otherwise
handles 1 and 1b.  2 and 4 are handled as well.  3 is mostly handled,
and by mostly, I mean if you get, for instance, a merge conflict in
the superproject or the subproject, you would need to resolve that
using the normal git commands and then typically retry the gitslave
command.  gitslave does not prevent you from using any git command in
any way you might desire, it simply allows you to use "gits" instead
of "git" when you want your command to run over all repos.

    A sample workflow might be:

      foo clone git@example.com/some/super/repo
      # time passes
      cd repo
      foo update
      # work
      foo status
      foo diff
      foo commit -m 'one message fits all'
      # possibly, but this could also be commit --no-push instead
      foo push

In the gitslave world, this would be done:

git clone git@example.com/some/super/repo
cd repo
gits checkout master
# time passes
gits pull
# work
gits status
gits diff
gits commit -a -m 'one message fits none'
# I am not familiar with the "git commit --no-push" option
gits push

    Merges are expected to be rare but would happen every so often.

gits checkout featurebranch
# work
gits commit -a -m 'features are misunderstood'
gits checkout master
gits merge featurebranch
gits tag v1.0 -a -m "feature available"
gits push
gits push --tags

    Has this been done before?  Or do we need to hack up a new wrapper
    around submodules?

Hopefully gitslave will work for you (or could be modified to work for
you).  It is not perfect but has been working well enough for a team
of developers for over two years.

					-Seth Robertson

Re: Submodules or similar for exercise/exam management

[Jens Lehmann]

Hi Thomas

Am 18.11.2010 11:09, schrieb Thomas Rast:
> The scenario is that we have a bunch of exercise questions stored in
> one or several files, each living in a directory.  Then, we assemble1
> those into exercise sheets (super-repos) spanning a group of questions
> (submodules).

That sounds like each directory is maintained by a different group of
people and then there is another bunch of people choosing the content
of the next exercise sheets, right?

> We would like to track this in such a way that changes
> to the questions propagate to other sheets (possibly in other
> lectures) that use them.

This could mean you would want an 'always-tip' mode for the
submodules, or am I misunderstanding you here?

> Training everyone in full git usage is probably not an option, so any
> solution would have to have some frontend that can be explained
> easily.

Yup, makes sense (at least until something goes wrong, see 3). ;-)

> The requirements we came up with are roughly:
> 
> 1) commit across all sub-repos at the same time (atomicity nice but
>    optional)
> 
>   1b) tag across all sub-repos

"git commit" and "git tag" could be taught the "--recurse-submodule"
option (but the commit part only makes sense when "git branch" can
do that too, so you have something to commit on. And I think you
want to enable a - yet to be implemented - file-based recursive diff
and status too, to see what changes your next commit will include).
And until all that materializes for submodules, a script would have
to do that.

> 2) do recursive clone/update of one super-repo easily

That will be handled by recursive checkout and is already achieved
by "git clone --recursive" (but at least in the first version both
don't support an "always-tip" mode).

> 3) never need to be aware of repo boundaries or manipulate sub-repo

I think that this requirement is the hardest for any solution I know
of or can imagine, as you hit these boundaries sooner or later either
when you want to commit, push and/or when you have to resolve merge
conflicts.

> 4) sanely cope with branches etc. in case the user wants them

A "--recurse-submodules" option to "git-branch" might be what you
want here, but as this isn't there yet a script will have to do that
for now.

> A sample workflow might be:
> 
>   foo clone git@example.com/some/super/repo
>   # time passes
>   cd repo
>   foo update
>   # work
>   foo status
>   foo diff
>   foo commit -m 'one message fits all'
>   # possibly, but this could also be commit --no-push instead
>   foo push
> 
> Merges are expected to be rare but would happen every so often.
> 
> It seems repo can do (2) and (4) but violates (3) [requires use of
> git-commit and others in the sub-repo].  git-subtree can do (1) and
> (2) but probably violates (3) [need to rebase changes to sub-repo
> branch] and (4).  Submodules can do (2) and (4) but violate (3) and
> currently have no support for (1).  I think svn externals could do
> (1)-(3) but violate (4) and probably (1b).
> 
> Has this been done before?  Or do we need to hack up a new wrapper
> around submodules?

If you would base that on submodule functionality, you would have
to hack up a wrapper script for the foreseeable future because the
"fully integrated" world view you seem to need is not worked on yet
(and I didn't see anyone coming forward to do that).

I took a cursory glance at "gitslave" Seth mentioned, it might do
what you want, but I can't tell for sure as I never used it myself.


Jens

Re: Submodules or similar for exercise/exam management

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

>> 3) never need to be aware of repo boundaries or manipulate sub-repo
>
> I think that this requirement is the hardest for any solution I know
> of or can imagine, as you hit these boundaries sooner or later either
> when you want to commit, push and/or when you have to resolve merge
> conflicts.

Just a quick sanity check.  When this "requirement" makes sense, does the
whole thing need to be a superproject with bunch of submodules, and why?

Re: Submodules or similar for exercise/exam management

[Jonathan Nieder]

Junio C Hamano wrote:
> Jens Lehmann <Jens.Lehmann@web.de> writes:

>>> 3) never need to be aware of repo boundaries or manipulate sub-repo
>>
>> I think that this requirement is the hardest for any solution I know
>> of or can imagine, as you hit these boundaries sooner or later either
>> when you want to commit, push and/or when you have to resolve merge
>> conflicts.
>
> Just a quick sanity check.  When this "requirement" makes sense, does the
> whole thing need to be a superproject with bunch of submodules, and why?

In this example: because the submodule represents individual
questions that are used by multiple exams.

Another instance of the same (slightly uncomfortable :)) practice:
suppose a certain chapter is part of multiple works I have
published --- maybe in an article and a larger book.  When working on
the book:

 - I do not want to make changes to that chapter and forget to commit
   them.

 - After making changes to many chapters, I do not want the fuss of
   going from chapter to chapter and commiting them one by one.

 - I certainly do not want to publish a version of the book that
   "includes" versions of the chapters as dead links, so to speak.
   That is, when I publish the current version of the book, I want
   to publish the current version of all chapters, too.

 - When starting work on the book again after long absence, I would
   like to be able to see and have the chance to adopt changes to
   chapters made as part of this book and as part of others.

If I understand correctly.

Re: Submodules or similar for exercise/exam management

[Thomas Rast]

Seth Robertson wrote:
> git clone git@example.com/some/super/repo
> cd repo
> gits checkout master
> # time passes
> gits pull
> # work
> gits status
> gits diff
> gits commit -a -m 'one message fits none'
> # I am not familiar with the "git commit --no-push" option
> gits push

Thanks a lot for pointing that out!  I just did a few simple tests and
it looks like it's the tool for the job.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Re: Submodules or similar for exercise/exam management

[Thomas Rast]

Jonathan Nieder wrote:
> Junio C Hamano wrote:
> > Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
> >>> 3) never need to be aware of repo boundaries or manipulate sub-repo
> >>
> >> I think that this requirement is the hardest for any solution I know
> >> of or can imagine, as you hit these boundaries sooner or later either
> >> when you want to commit, push and/or when you have to resolve merge
> >> conflicts.
> >
> > Just a quick sanity check.  When this "requirement" makes sense, does the
> > whole thing need to be a superproject with bunch of submodules, and why?
> 
> In this example: because the submodule represents individual
> questions that are used by multiple exams.

Indeed.

I tried to not go too deeply into the status quo to keep the original
mail short, but if you really care, here goes (there's nothing else
below this point so you can also stop reading):

In my own group (theoretical CS), we have a Big Ugly SVN repository
with a directory per class taught, with subdirectories for the
exercises of every year.  Either at the beginning or during the year,
we copy the previous exercises and solutions to a new directory and
modify/fix them to taste.  Similar for exams.

This works reasonably well because all our lectures are on mostly
disjoint topics, so we rarely share anything between them.  The
directories act more as a "tag" of the old version than anything else,
and also help lazy users find the old version without talking to SVN.
Converting it to git would be fairly trivial since the usage maps
straight to lecture-projects with tags at the end of every semester,
but largely due to inertia nobody wants to do it.

The statistics institute where my colleague works is different in two
major respects.

First, they do not currently have any kind of enforced versioning.
There apparently are a bunch of directories along similar lines as the
above, but they live on a big shared NFS and thus it's easy to lose
existing state simply by overwriting it.  This is good for the
purposes of this discussion because we can weigh the pain of living
with the existing solution against the pain of using the new solution
;-)

More importantly, and this spawned my question, they have a lot more
overlaps between lectures.  (I assume this is because they give
statistics courses at various levels of difficulty to various other
departments/subjects of study, but I don't really know.)

Thus it often happens that an exercise is copied from lecture A to B,
then later some mistake is fixed in A and the fix does not propagate.
Worse, for whatever reason they might later copy back the exercise
from B to A and thus have a regression.

Handling at least this case sanely would already be good.  The
requirement that Junio criticized above is geared not towards sanity
of the use-case, but towards not making adoption so complicated as to
make it impossible.

It is true that in cases where submodules really pay off, such
exercises (or perhaps more realistically, guides and lecture notes)
being customized slightly for each course.  This would need branches.
For now I am putting this case a bit into the background since it can
be solved by working on such documents in a separate repo, splitting
the customizations into branches and merging the fixes from the main
copy; and then just binding the appropriate version as a sub-repo.

And yes, this would in theory also go for books as you described:

> suppose a certain chapter is part of multiple works I have
> published --- maybe in an article and a larger book.  When working on
> the book:
> 
>  - I do not want to make changes to that chapter and forget to commit
>    them.
> 
>  - After making changes to many chapters, I do not want the fuss of
>    going from chapter to chapter and commiting them one by one.
> 
>  - I certainly do not want to publish a version of the book that
>    "includes" versions of the chapters as dead links, so to speak.
>    That is, when I publish the current version of the book, I want
>    to publish the current version of all chapters, too.
> 
>  - When starting work on the book again after long absence, I would
>    like to be able to see and have the chance to adopt changes to
>    chapters made as part of this book and as part of others.

Though I wonder how that pans out in practice since it is rarely
possible to just "drop in" a chapter without rewriting significant
parts of it to match the rest of the book.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch