RFC: a plugin architecture for git extensions?

RFC: a plugin architecture for git extensions?

[Jon Seymour]

Has anyone ever given consideration to git supporting a plugin
architecture for git extensions?

The idea would be to provide a consistent way to install, and address
extensions to the core git functionality in a manner that does not
require the extension to actually be integrated into the git core.

For example, I have recently proposed a new command 'git work'
https://github.com/jonseymour/git/blob/master/README.md which I think
is a really useful extension to git.

I haven't had much feedback for the concept. I am not sure if it is
because people are too busy, just don't grok it, or grok it and don't
think it is useful.

So, perhaps it won't be included in git. That's fine, I can build my
own fork of git which includes the proposed extension [ indeed, this
is how I originally developed it]. That's fine for
me, but it isn't the most practical way to distribute it to others
since I'll have to produce distribution packages for a variety of
different distribution formats or fallback to tars and zips.

What would be call is if I could package my extension in a standard
way and then anyone who was interested could simply do something like:

    git plugin install gitwork

and then start using the commands as if they had built my fork of git
themselves.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jonathan Nieder]

Hi Jon,

Jon Seymour wrote:

> Has anyone ever given consideration to git supporting a plugin
> architecture for git extensions?
>
> The idea would be to provide a consistent way to install, and address
> extensions to the core git functionality in a manner that does not
> require the extension to actually be integrated into the git core.

I haven't looked into 'git work' yet, but for my own private tweaks,
two mechanisms have sufficed:

 * adding a program named git-foo to the $PATH introduces a
   'git foo' command.  For the git command look and feel, scripts
   tend to start with

	. "$(git --exec-path)"/git-sh-setup

   (see git-sh-setup(1) for details).

 * various existing git commands can have their behavior modified
   through configuration and hooks.

Does 'git work' require changing the behavior of existing commands,
and if so, are there hooks that could be introduced to help in doing
that?

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Wednesday, April 27, 2011, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Hi Jon,
>
> I haven't looked into 'git work' yet, but for my own private tweaks,
> two mechanisms have sufficed:
>
>  * adding a program named git-foo to the $PATH introduces a
>    'git foo' command.  For the git command look and feel, scripts
>    tend to start with
>
>         . "$(git --exec-path)"/git-sh-setup
>
>    (see git-sh-setup(1) for details).
>
>  * various existing git commands can have their behavior modified
>    through configuration and hooks.
>
> Does 'git work' require changing the behavior of existing commands,
> and if so, are there hooks that could be introduced to help in doing
> that?

git work is all new, so such considerations don't apply. Actually
there is one thing - I proposed a new config variable whose
description I left out of the tar ball version because I didn"t want
to touch existing files in a git install. The other thing II needed to
do was include my man pages so they get picked up by git help.

I can see that there might be a class of extension that 'enhances'
existing git commands. No doubt that would be a tricky problem to
solve and such a solution would not necessarily be good for the git
platform in any case.

 I guess one thing a git plugin architecture could do is provide
management of the PATH for plugins, a way to manage man pages and a
way to manage plugin specific configuration help.

jon.

| missed the list unintentionally, the first time through

Re: RFC: a plugin architecture for git extensions?

[Junio C Hamano]

Jon Seymour <jon.seymour@gmail.com> writes:

> Has anyone ever given consideration to git supporting a plugin
> architecture for git extensions?

Hint.  The output from "git help --all" is produced by finding any
executable whose name match "git-*" on your $PATH.

So if you have /home/js/bin on your $PATH, you can install your "git-work"
script as /home/js/bin/git-work, and that should be sufficient.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Wed, Apr 27, 2011 at 3:07 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jon Seymour <jon.seymour@gmail.com> writes:
>
>> Has anyone ever given consideration to git supporting a plugin
>> architecture for git extensions?
>
> Hint.  The output from "git help --all" is produced by finding any
> executable whose name match "git-*" on your $PATH.
>
> So if you have /home/js/bin on your $PATH, you can install your "git-work"
> script as /home/js/bin/git-work, and that should be sufficient.
>

Yep, that's a start, but does not a a complete plugin architecture make :-)

jon.

Re: RFC: a plugin architecture for git extensions?

[Junio C Hamano]

Jon Seymour <jon.seymour@gmail.com> writes:

>> So if you have /home/js/bin on your $PATH, you can install your "git-work"
>> script as /home/js/bin/git-work, and that should be sufficient.
>
> Yep, that's a start, but does not a a complete plugin architecture make :-)

Please explain yourself.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

Hide quoted text -
On Wed, Apr 27, 2011 at 3:17 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jon Seymour <jon.seymour@gmail.com> writes:
>
>>> So if you have /home/js/bin on your $PATH, you can install your "git-work"
>>> script as /home/js/bin/git-work, and that should be sufficient.
>>
>> Yep, that's a start, but does not a a complete plugin architecture make :-)
>
> Please explain yourself.
>

So, I think at a very minimum, a plugin architecture should specify
the file system layout of packages to be managed by a plugin/package
manager.

So, where to find scripts, where to find man pages, bash completions,
configuration help etc.

A slightly more functional architecture would provide support for
unpacking package archives into a "standard" repository location and
for removing unwanted plugins.

A plugin architecture might specify a standard way to access
extensions. (git blah is easy for local use, but what if a plugin
grabs a "noun" that the core wants to use that "noun" itself in
future. Perhaps gitx blah would be a better standard way to access
extensions. But that is an aside, I am sure this question has been
considered previously).

An even more functional architecture would provide support for a
global registry of plugins. I understand that git may not want to
write its own package manager (how many times has that been done),
but it'd be nice if competing "git package managers" had a standard
target to deploy to.

My thoughts about this are inspired by how the node project manages
packages with its npm package manager and also the fact that I have
several ideas on the boil at the moment that would definitely benefit
from a standard way to manage these concerns.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Wed, Apr 27, 2011 at 3:33 PM, Jon Seymour <jon.seymour@gmail.com> wrote:
> Hide quoted text -
> On Wed, Apr 27, 2011 at 3:17 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Jon Seymour <jon.seymour@gmail.com> writes:
>>
>>>> So if you have /home/js/bin on your $PATH, you can install your "git-work"
>>>> script as /home/js/bin/git-work, and that should be sufficient.
>>>
>>> Yep, that's a start, but does not a a complete plugin architecture make :-)
>>
>> Please explain yourself.
>>
>
> So, I think at a very minimum, a plugin architecture should specify
> the file system layout of packages to be managed by a plugin/package
> manager.
>
> So, where to find scripts, where to find man pages, bash completions,
> configuration help etc.
>
> A slightly more functional architecture would provide support for
> unpacking package archives into a "standard" repository location and
> for removing unwanted plugins.
>
> A plugin architecture might specify a standard way to access
> extensions. (git blah is easy for local use, but what if a plugin
> grabs a "noun" that the core wants to use that "noun" itself in
> future. Perhaps gitx blah would be a better standard way to access
> extensions. But that is an aside, I am sure this question has been
> considered previously).
>
> An even more functional architecture would provide support for a
> global registry of plugins. I understand that git may not want to
> write its own package manager (how many times has that been done),
> but it'd be nice if competing "git package managers" had a standard
> target to deploy to.
>
> My thoughts about this are inspired by how the node project manages
> packages with its npm package manager and also the fact that I have
> several ideas on the boil at the moment that would definitely benefit
> from a standard way to manage these concerns.
>
> jon.
>

Also, a hypothetical git package manager might also provide build support
for packages to easy the creation of help, archives, validating
layouts and descriptors, etc.

jon.

Re: RFC: a plugin architecture for git extensions?

[Junio C Hamano]

Jon Seymour <jon.seymour@gmail.com> writes:

> My thoughts about this are inspired by how the node project manages
> packages with its npm package manager and also the fact that I have
> several ideas on the boil at the moment that would definitely benefit
> from a standard way to manage these concerns.

Sounds like you have a plan ;-)

It would be ideal if you can arrange things so that the only thing the
user needs to do is to point your package manager to one subdirectory of
contrib/ and everything necessary would be installed...

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Wed, Apr 27, 2011 at 3:39 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jon Seymour <jon.seymour@gmail.com> writes:
>
>> My thoughts about this are inspired by how the node project manages
>> packages with its npm package manager and also the fact that I have
>> several ideas on the boil at the moment that would definitely benefit
>> from a standard way to manage these concerns.
>
> Sounds like you have a plan ;-)
>
> It would be ideal if you can arrange things so that the only thing the
> user needs to do is to point your package manager to one subdirectory of
> contrib/ and everything necessary would be installed...
>
>
>
>

By that you mean the contrib/ directory as the repository of
extensions, with one such extension being the plugin manager itself?

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Wed, Apr 27, 2011 at 3:39 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jon Seymour <jon.seymour@gmail.com> writes:
>
>> My thoughts about this are inspired by how the node project manages
>> packages with its npm package manager and also the fact that I have
>> several ideas on the boil at the moment that would definitely benefit
>> from a standard way to manage these concerns.
>
> Sounds like you have a plan ;-)
>

Ok, here we go:

    https://github.com/jonseymour/gpm

Anyone who violently objects to the suggested name of a package
manager interface - gpm, please speak up now because it'll be easier
to change now.

(And yes, gpm is intended to be an *interface*. The idea would be to
allow the interface to be back-ended by different implementations
depending on taste, platform etc.).

I suggest using this list for discussion, but I also think the github
issue manager would be a pretty good option.

Speak up if, you see anything wrong so far!

jon.

NAME
====
gpm - a package manager for git extensions

DESCRIPTION
===========
This is place holder for a proposed plugin architecture for git extensions.

The initial deliverable of the project will be a plugin architecture proposal.
In parallel to this deliverable a reference plugin manager will be developed.

PRINCIPLES
==========
* define a repository layout specification
* define a package descriptor (using the git config syntax)
* package manager agnostic repo and package specifications
 * perhaps define a pluggable package manager interface
* support for:
 * existing contrib/ directory
 * existing git patterns for finding and using extensions
 * man pages
 * package specific configuration help
 * bash completions
 * scripts
* build support:
 * for documentation, archives
* layers
 * repo, package descriptors
 * tool interface specifications
 * tool implementations
 * global package registry and repository

INTENDED SCENARIOS
==================

Installation and Removal
------------------------
	gpm install package-name | package-url | package-archive
	gpm remove package-name
	gpm update package-name
	gpm list installed|available|active|inactive
	gpm status package-name

Activation/De-activation
------------------------
	gpm activate package-name
	gpm deactivate package-name

MAILING LIST
============
Until noted otherwise, please use the
[http://dir.gmane.org/gmane.comp.version-control.git](git@vger.kernel.org)
mailing list for discussions about design decisions. gpm: is suggested
a good prefix for such discussions. We might
also use the github issues manager to integrate and discuss
suggestions, subject to agreement from the list.

AUTHOR
======
Jon Seymour

Re: RFC: a plugin architecture for git extensions?

[Michael J Gruber]

Jon Seymour venit, vidit, dixit 27.04.2011 09:15:
> On Wed, Apr 27, 2011 at 3:39 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Jon Seymour <jon.seymour@gmail.com> writes:
>>
>>> My thoughts about this are inspired by how the node project manages
>>> packages with its npm package manager and also the fact that I have
>>> several ideas on the boil at the moment that would definitely benefit
>>> from a standard way to manage these concerns.
>>
>> Sounds like you have a plan ;-)
>>
> 
> Ok, here we go:
> 
>     https://github.com/jonseymour/gpm
> 
> Anyone who violently objects to the suggested name of a package
> manager interface - gpm, please speak up now because it'll be easier
> to change now.
> 
> (And yes, gpm is intended to be an *interface*. The idea would be to
> allow the interface to be back-ended by different implementations
> depending on taste, platform etc.).
> 
> I suggest using this list for discussion, but I also think the github
> issue manager would be a pretty good option.
> 
> Speak up if, you see anything wrong so far!

I'm sorry to spoil the party before it started but I'm not very fond of
having yet another package manager orthogonal to what distributions have
already. This is definitely not a way to get anything like that into a
distribution which has proper policies.

What we could need now to help users is a working "make -C contrib/foo"
to easily install selected features from our contrib area (with
"install" or "doc" targets), i.e. set things up so that we can have very
simple makefiles in contrib (with an include).

If you really want to go forward with a bigger solution, we could
distribute "extensions" by installing but not "activating" them, and
have some command which activates them selectively (like hg extensions).
That is something distributions could ship, although I don't see any
need for it personally. Our architecture makes it so easy to "integrate"
git-foo, git-foo.1 and config variables without touching the core git
installation at all!

Michael

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Wed, Apr 27, 2011 at 5:57 PM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
>
> I'm sorry to spoil the party before it started but I'm not very fond of
> having yet another package manager orthogonal to what distributions have
> already. This is definitely not a way to get anything like that into a
> distribution which has proper policies.

I am happy to defer work on a full-blown package manager for now. I do
agree, the world already has a surfeit of
package managers. As I say, I'd prefer a minimal interface that could
be back-ended by one or more of those if possible.

Initially I'd like to focus on:

* a descriptor format for packages providing hints about how to
activate an extension
* guidelines for the filesystem layout of extensions themselves
* a way to locally locate extensions that might be subject to activation
* the interface between the git runtime and the activated package repository
* the interface to activate/deactivate a locally located package

I also want to avoid baking in any too many decisions about things
like registries and distribution models. That can come later.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Wed, Apr 27, 2011 at 6:15 PM, Jon Seymour <jon.seymour@gmail.com> wrote:
> On Wed, Apr 27, 2011 at 5:57 PM, Michael J Gruber
> <git@drmicha.warpmail.net> wrote:
>
> I also want to avoid baking in any too many decisions about things
> like registries and distribution models. That can come later.

Ok, may be some thoughts about a registry, at least to set a direction...

I like the way brew on MAC OSX uses git to manage formulas managed in
a git repo.

The idea would be to maintain a registry of "git package descriptors".
The descriptors would be a copy of the whatever descriptor an
activated package would be described by, but probably simply a git
config text file, since we already have the tools to parse those.

Such a descriptor would have hints about how to use a real package
manager to get the actual package, but would not actually contain any
files from the package itself.

So, for example, the package source might be bundled in git-core
contrib/ directory, fetchable as a git repo, fetchable as a tar ball,
fetchable as an apt-get package, as brew package etc. The idea is that
gpm would know enough about invoking a real package manager to handle
the actual distribution details.

Once the package is installed by gpm, it is then subject to the local
activation process.

jon.

Re: RFC: a plugin architecture for git extensions?

[Motiejus Jakštys]

On Wed, Apr 27, 2011 at 06:40:07PM +1000, Jon Seymour wrote:
> On Wed, Apr 27, 2011 at 6:15 PM, Jon Seymour <jon.seymour@gmail.com> wrote:
> > On Wed, Apr 27, 2011 at 5:57 PM, Michael J Gruber
> > <git@drmicha.warpmail.net> wrote:
> >
> The idea would be to maintain a registry of "git package descriptors".
> The descriptors would be a copy of the whatever descriptor an
> activated package would be described by, but probably simply a git
> config text file, since we already have the tools to parse those.
> 
> Such a descriptor would have hints about how to use a real package
> manager to get the actual package, but would not actually contain any
> files from the package itself.

> 
> So, for example, the package source might be bundled in git-core
> contrib/ directory, fetchable as a git repo, fetchable as a tar ball,
> fetchable as an apt-get package, as brew package etc. The idea is that
> gpm would know enough about invoking a real package manager to handle
> the actual distribution details.

Let me check If I get this right:
Say I am a maintainer of enchanced git log -- git logx. It is one .c
file, which has to be simply compiled (cc -o git-logx logx.c).

If I want to maintain the package in a way you are suggesting, I have to
prepare deb, rpm, tarball, zipball, brew (what ever it is, sorry for
non-intelligence) and what not. Otherwise, I lose users? *ball is not
an option, since we are supporting different architectures and cannot
distribute compiled files (unless statically compiled for all
architectures we know, which is not good for obvious reasons).

Morevoer, different debs for different debian releases (if package
depends on libc version)?

Although this looks very nice from user point of view, it would be a
pain for the extension maintainer... Though it's only one C file.

I think shipping it in contrib/ and having extension system is a better
option. Though it leaves a hole for dependencies -- if my logx depends
on boost or imagemagick, we don't want make git depend on it...

Motiejus

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

2011/4/27 Motiejus Jakštys <desired.mta@gmail.com>:
> On Wed, Apr 27, 2011 at 06:40:07PM +1000, Jon Seymour wrote:
>> On Wed, Apr 27, 2011 at 6:15 PM, Jon Seymour <jon.seymour@gmail.com> wrote:
>> > On Wed, Apr 27, 2011 at 5:57 PM, Michael J Gruber
>> > <git@drmicha.warpmail.net> wrote:
>> >
>> The idea would be to maintain a registry of "git package descriptors".
>> The descriptors would be a copy of the whatever descriptor an
>> activated package would be described by, but probably simply a git
>> config text file, since we already have the tools to parse those.
>>
>> Such a descriptor would have hints about how to use a real package
>> manager to get the actual package, but would not actually contain any
>> files from the package itself.
>
>>
>> So, for example, the package source might be bundled in git-core
>> contrib/ directory, fetchable as a git repo, fetchable as a tar ball,
>> fetchable as an apt-get package, as brew package etc. The idea is that
>> gpm would know enough about invoking a real package manager to handle
>> the actual distribution details.
>
> Let me check If I get this right:
> Say I am a maintainer of enchanced git log -- git logx. It is one .c
> file, which has to be simply compiled (cc -o git-logx logx.c).
>
> If I want to maintain the package in a way you are suggesting, I have to
> prepare deb, rpm, tarball, zipball, brew (what ever it is, sorry for
> non-intelligence) and what not. Otherwise, I lose users? *ball is not
> an option, since we are supporting different architectures and cannot
> distribute compiled files (unless statically compiled for all
> architectures we know, which is not good for obvious reasons).
>
> Morevoer, different debs for different debian releases (if package
> depends on libc version)?
>
> Although this looks very nice from user point of view, it would be a
> pain for the extension maintainer... Though it's only one C file.
>

I don't see why. You wouldn't be forced to maintain one of those
packages, anymore than you
are now. If you can, you just publish a git url, and your job is done.
If your tool requires
compilation, you have already solved the distribution problem for the
package managers you choose to support.

All I am suggesting you do is that you list pointers to these
packages, which gpm could then use to actually do the installation for
you.

The thing I really care about is: once the package has been installed
locally, how do I activate it and makes its features available to the
git runtime, without having to futz around with my PATH, MANPATH etc?

Yes, this is user focused. I want users of my tool to be able to something like:

   git pm install gitwork

or perhaps:

   git pm install git://github.com/jonseymour/gitwork

And then start using it. I want the git completions to be there, I
want the man pages to be there. I want the scripts to be
there. In my case, there I really shouldn't have to do anything other
than point at a git url, get the package transported
to a local directory and activate it.

I want to tell my users what they need to do to install my extension
without having to have variants of the instructions for zip/tar
people, apt-get people, brew people, MAC ports people, yum people etc.
Yes, for compiled stuff, someone has to prepare the packages. But once
they are prepared, use whatever package manager the user uses to
install it and expose the gpm config required to activate it.

> I think shipping it in contrib/ and having extension system is a better
> option. Though it leaves a hole for dependencies -- if my logx depends
> on boost or imagemagick, we don't want make git depend on it...
>

Unless I am misunderstanding something about how contrib/ is managed
that still requires Junio to be in the loop, which defeats one of my
objectives here - which is to allow anyone to contribute and share
their own extensions without being bottlenecked by someone else's
release cycle.

jon.

Re: RFC: a plugin architecture for git extensions?

[Carlos Martín Nieto]

[-- Attachment #1: Type: text/plain, Size: 1043 bytes --]

On Wed, Apr 27, 2011 at 05:15:50PM +1000, Jon Seymour wrote:
> On Wed, Apr 27, 2011 at 3:39 PM, Junio C Hamano <gitster@pobox.com> wrote:
> > Jon Seymour <jon.seymour@gmail.com> writes:
> >
> >> My thoughts about this are inspired by how the node project manages
> >> packages with its npm package manager and also the fact that I have
> >> several ideas on the boil at the moment that would definitely benefit
> >> from a standard way to manage these concerns.
> >
> > Sounds like you have a plan ;-)
> >
> 
> Ok, here we go:
> 
>     https://github.com/jonseymour/gpm
> 
> Anyone who violently objects to the suggested name of a package
> manager interface - gpm, please speak up now because it'll be easier
> to change now.

I'm not objecting, but when I see gpm, I think of the mouse daemon for
Linux virtual consoles[0] and whose git clone address also ends in gpm.

I find a name like gem (git extension manager) nicer, though that's
taken by ruby.

[0] http://www.nico.schottelius.org/software/gpm/

   cmn

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

>
> I'm not objecting, but when I see gpm, I think of the mouse daemon for
> Linux virtual consoles[0] and whose git clone address also ends in gpm.
>
> I find a name like gem (git extension manager) nicer, though that's
> taken by ruby.
>
> [0] http://www.nico.schottelius.org/software/gpm/
>
>    cmn
>

Yes, I guess name clashes are not with out precedent, but perhaps we
should try harder this time or people will start to talk :-)

How about git-pm? I am thinking the script would be called git-pm
anyway, so it would make sense.

jon.

Re: RFC: a plugin architecture for git extensions?

[Motiejus Jakštys]

On Wed, Apr 27, 2011 at 07:59:33PM +1000, Jon Seymour wrote:
> 2011/4/27 Motiejus Jakštys <desired.mta@gmail.com>:
> > On Wed, Apr 27, 2011 at 06:40:07PM +1000, Jon Seymour wrote:
> > Although this looks very nice from user point of view, it would be a
> > pain for the extension maintainer... Though it's only one C file.
> >
> 
> I don't see why. You wouldn't be forced to maintain one of those
> packages, anymore than you are now. If you can, you just publish a git
> url, and your job is done.  If your tool requires compilation, you
> have already solved the distribution problem for the package managers
> you choose to support.
> 
> All I am suggesting you do is that you list pointers to these
> packages, which gpm could then use to actually do the installation for
> you.
> 
> The thing I really care about is: once the package has been installed
> locally, how do I activate it and makes its features available to the
> git runtime, without having to futz around with my PATH, MANPATH etc?
> 
> Yes, this is user focused. I want users of my tool to be able to
> something like:
> 
>    git pm install gitwork
> 
> or perhaps:
> 
>    git pm install git://github.com/jonseymour/gitwork
> 
> And then start using it. I want the git completions to be there, I
> want the man pages to be there. I want the scripts to be there. In my
> case, there I really shouldn't have to do anything other than point at
> a git url, get the package transported to a local directory and
> activate it.
> 
> I want to tell my users what they need to do to install my extension
> without having to have variants of the instructions for zip/tar
> people, apt-get people, brew people, MAC ports people, yum people etc.
> Yes, for compiled stuff, someone has to prepare the packages. But once
> they are prepared, use whatever package manager the user uses to
> install it and expose the gpm config required to activate it.

python virtualenv has very similar goals and does the same thing, the
only difference is you have to "source env/bin/activate" before using
your local modules (or run env/bin/python). I can clarify what features
from it could be used for git, if you like.

> > I think shipping it in contrib/ and having extension system is a better
> > option. Though it leaves a hole for dependencies -- if my logx depends
> > on boost or imagemagick, we don't want make git depend on it...
> >
> 
> Unless I am misunderstanding something about how contrib/ is managed
> that still requires Junio to be in the loop, which defeats one of my
> objectives here - which is to allow anyone to contribute and share
> their own extensions without being bottlenecked by someone else's
> release cycle.
Yes, you are right. It shouldn't. Like in python.

Motiejus

Re: RFC: a plugin architecture for git extensions?

[Ævar Arnfjörð Bjarmason]

On Wed, Apr 27, 2011 at 05:36, Jon Seymour <jon.seymour@gmail.com> wrote:

> What would be call is if I could package my extension in a standard
> way and then anyone who was interested could simply do something like:
>
>    git plugin install gitwork
>
> and then start using the commands as if they had built my fork of git
> themselves.

We already have a plugin system, you can drop "git-work" in your
$PATH, what you're talking about is a packaging system, and solving
that problem in a user facing application like Git is IMNSHO a
terrible idea.

Instead you could just make gitwork available somewhere and then
install it as:

    sudo aptitude install git-work

Or whatever incarnation of that your distro or OS provides.

Having our own "gpm" system would mean having to solve all these
issues of distribution, cross-platform & arch compatibility that real
packaging systems already deal with.

If you *really* wanted to go through with this I'd suggest just using
some existing package manager like apt, aliasing it to "gpm", then
configure it to download packages from your own custom repository, and
install them in ~/ somewhere.

You'd still be re-inventing the wheel, but at least minimally so.

Re: RFC: a plugin architecture for git extensions?

[Fredrik Gustafsson]

On Wed, Apr 27, 2011 at 03:33:57PM +1000, Jon Seymour wrote:
> So, I think at a very minimum, a plugin architecture should specify
> the file system layout of packages to be managed by a plugin/package
> manager.

As I recall, Junios initial plan was to have gitk-git as a submodule at
some point. I still thinks this is a good idea.

If we extends the submodule concept, of not only having a list of
submodules, but also state weather a submodule is 'active' or 'inactive'
we could easily get a _very_ customable git.git.

Imagine git.git only containing git. A 'git submodule init' would load
the default 'active' submodules (for example git-gui, gitk-gui and
gitweb), everything in contrib is 'inactive'. The point here is to be
able to ship references to nice things to have (contrib) but not force
the use (download, diskspace, etc.) of it.

If a user finds an other awesome "plugin" to use with git, it's easy to
add it to h{er,i}s repository with 'git submodule add'.

Once the code is downloaded to the git-workspace (via 'git submodule
update') the git build system ('make') would take care of building and
installation, just as it does today.

-- 
Med vänliga hälsningar
Fredrik Gustafsson

tel: 0733-608274
e-post: iveqy@iveqy.com

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

>
> We already have a plugin system, you can drop "git-work" in your
> $PATH, what you're talking about is a packaging system, and solving
> that problem in a user facing application like Git is IMNSHO a
> terrible idea.
>
> Instead you could just make gitwork available somewhere and then
> install it as:
>
>     sudo aptitude install git-work
>
> Or whatever incarnation of that your distro or OS provides.
>
> Having our own "gpm" system would mean having to solve all these
> issues of distribution, cross-platform & arch compatibility that real
> packaging systems already deal with.
>
> If you *really* wanted to go through with this I'd suggest just using
> some existing package manager like apt, aliasing it to "gpm", then
> configure it to download packages from your own custom repository, and
> install them in ~/ somewhere.
>
> You'd still be re-inventing the wheel, but at least minimally so.
>


No. As I explained in the posts
 that you chose not to read, such concerns would be dealt with by real
package managers.

One you have read those posts, let me know if you still have concerns,
humble or otherwise.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Wednesday, April 27, 2011, Fredrik Gustafsson <iveqy@iveqy.com> wrote:
> On Wed, Apr 27, 2011 at 03:33:57PM +1000, Jon Seymour wrote:
>> So, I think at a very minimum, a plugin architecture should specify
>> the file system layout of packages to be managed by a plugin/package
>> manager.
>
> As I recall, Junios initial plan was to have gitk-git as a submodule at
> some point. I still thinks this is a good idea.
>
> If we extends the submodule concept, of not only having a list of
> submodules, but also state weather a submodule is 'active' or 'inactive'
> we could easily get a _very_ customable git.git.
>
> Imagine git.git only containing git. A 'git submodule init' would load
> the default 'active' submodules (for example git-gui, gitk-gui and
> gitweb), everything in contrib is 'inactive'. The point here is to be
> able to ship references to nice things to have (contrib) but not force
> the use (download, diskspace, etc.) of it.
>
> If a user finds an other awesome "plugin" to use with git, it's easy to
> add it to h{er,i}s repository with 'git submodule add'.
>
> Once the code is downloaded to the git-workspace (via 'git submodule
> update') the git build system ('make') would take care of building and
> installation, just as it does today.
>

Frederiick,

I have also been think about submodules as one possible back-end
package distribution manager, It certainly makes sense since once can
at least assume git is available :-)

However, it would be only one way, precisely because I will need to
rely on real package managers tif he build step is non-trivial.

Anyway, thanks for your feedback.

Jon.


> --
> Med vänliga hälsningar
> Fredrik Gustafsson
>
> tel: 0733-608274
> e-post: iveqy@iveqy.com
>

Re: RFC: a plugin architecture for git extensions?

[Andreas Ericsson]

On 04/27/2011 05:36 AM, Jon Seymour wrote:
> Has anyone ever given consideration to git supporting a plugin
> architecture for git extensions?
> 
> The idea would be to provide a consistent way to install, and address
> extensions to the core git functionality in a manner that does not
> require the extension to actually be integrated into the git core.
> 

Horrible idea. There are already as many package managers as there
are packages without us throwing another one into the mix.

> For example, I have recently proposed a new command 'git work'
> https://github.com/jonseymour/git/blob/master/README.md which I think
> is a really useful extension to git.
> 
> I haven't had much feedback for the concept. I am not sure if it is
> because people are too busy, just don't grok it, or grok it and don't
> think it is useful.
> 

I had a look at the manpage. It seems to do more or less exactly what
the same command would do without the word "work" thrown in, so either
it's quite useless or you've failed to describe its usefulness in the
manpage.

"git atomic" seems nice though.

> So, perhaps it won't be included in git. That's fine, I can build my
> own fork of git which includes the proposed extension [ indeed, this
> is how I originally developed it]. That's fine for
> me, but it isn't the most practical way to distribute it to others
> since I'll have to produce distribution packages for a variety of
> different distribution formats or fallback to tars and zips.
> 

What you can do is let your Makefile (or some other install-script)
take the destination path for "make install" (or equivalent) from
the output of "git --exec-path".

That way, you can ship "git extadd" or whatever you want to call it
as a simple installer that installs executable and man-page in their
proper locations. If the commands you install require configuration
by default I'd say they're broken to begin with, but even that can
be remedied by running "git config --add key value" from the installer.

So in a way, git is already its own pkg-config binary and anyone
clever enough to write useful scripts that enhances git will almost
certainly see that and use it from their favourite language quite
without having to learn some new magic format for package management.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Wednesday, April 27, 2011, Andreas Ericsson  wrote:
> On 04/27/2011 05:36 AM, Jon Seymour wrote:
>> Has anyone ever given consideration to git supporting a plugin
>> architecture for git extensions?
>>
>> The idea would be to provide a consistent way to install, and address
>> extensions to the core git functionality in a manner that does not
>> require the extension to actually be integrated into the git core.
>>
>

> Horrible idea. There are already as many package managers as there
> are packages without us throwing another one into the mix.
>

I agree that there are too many package managers. But do you know
what? There isn't a single package manager that reliably works across
platform. apt-get? great. Except you need something else for Mac,
cywgin, or, um Fedora. Brew? Fine then you only need to worry about
Linux and cygwin. Cygwin? ...

The platform for my extension is git. Not Mac. Not Debian. Not Fedora.
Not cygwin. git.

The lowest common denominator across these environments is, um, git.

I challenge the sceptIcs to specify a one line command script that
works across all possible environment that is more succinct than:

   git pm install gitwork

It shouldn't be too hard. A tar command here, an enviroment  variable
edit there. Perhaps a curl command or a browser download.

You have 4 words. Knock yourself out.

>> For example, I have recently proposed a new command 'git work'
>> https://github.com/jonseymour/git/blob/master/README.md which I think
>> is a really useful extension to git.
>>
>> I haven't had much feedback for the concept. I am not sure if it is
>> because people are too busy, just don't grok it, or grok it and don't
>> think it is useful.
>>
>
> I had a look at the manpage. It seems to do more or less exactly what
> the same command would do without the word "work" thrown in, so either
> it's quite useless or you've failed to describe its usefulness in the
> manpage.
>

It is far from useless, so I have clearly failed with the explanation.
I will post later,perhaps with some diagrams.

> "git atomic" seems nice though.
>

Thank you!

>> So, perhaps it won't be included in git. That's fine, I can build my
>> own fork of git which includes the proposed extension [ indeed, this
>> is how I originally developed it]. That's fine for
>> me, but it isn't the most practical way to distribute it to others
>> since I'll have to produce distribution packages for a variety of
>> different distribution formats or fallback to tars and zips.
>>
>
> What you can do is let your Makefile (or some other install-script)
> take the destination path for "make install" (or equivalent) from
> the output of "git --exec-path".
>
> That way, you can ship "git extadd" or whatever you want to call it
> as a simple installer that installs executable and man-page in their
> proper locations. If the commands you install require configuration
> by default I'd say they're broken to begin with, but even that can
> be remedied by running "git config --add key value" from the installer.
>
> So in a way, git is already its own pkg-config binary and anyone
> clever enough to write useful scripts that enhances git will almost
> certainly see that and use it from their favourite language quite
> without having to learn some new magic format for package management.
>

Those 3 paragraphs were substantially longer than 4 words. Again,
there is a tar ball, let me know how I can install it across alll
environments that got runs on. Make sure the man pages work.

jon.


> --
> Andreas Ericsson                   andreas.ericsson@op5.se
> OP5 AB                             www.op5.se
> Tel: +46 8-230225                  Fax: +46 8-230231
>
> Considering the successes of the wars on alcohol, poverty, drugs and
> terror, I think we should give some serious thought to declaring war
> on peace.
>

Re: RFC: a plugin architecture for git extensions?

[Felipe Contreras]

On Wed, Apr 27, 2011 at 3:50 PM, Jon Seymour <jon.seymour@gmail.com> wrote:
> On Wednesday, April 27, 2011, Andreas Ericsson  wrote:
>> Horrible idea. There are already as many package managers as there
>> are packages without us throwing another one into the mix.
>
> I agree that there are too many package managers. But do you know
> what? There isn't a single package manager that reliably works across
> platform. apt-get? great. Except you need something else for Mac,
> cywgin, or, um Fedora. Brew? Fine then you only need to worry about
> Linux and cygwin. Cygwin? ...

gem works on all of them :)

-- 
Felipe Contreras

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Wednesday, April 27, 2011, Felipe Contreras
<felipe.contreras@gmail.com> wrote:
> On Wed, Apr 27, 2011 at 3:50 PM, Jon Seymour <jon.seymour@gmail.com> wrote:
>> On Wednesday, April 27, 2011, Andreas Ericsson  wrote:
>>> Horrible idea. There are already as many package managers as there
>>> are packages without us throwing another one into the mix.
>>
>> I agree that there are too many package managers. But do you know
>> what? There isn't a single package manager that reliably works across
>> platform. apt-get? great. Except you need something else for Mac,
>> cywgin, or, um Fedora. Brew? Fine then you only need to worry about
>> Linux and cygwin. Cygwin? ...
>
> gem works on all of them :)
>

Ah. but you have just made things harder for yourself.

Assuming you use two words to install ruby, one as a command
separator, you need to deploy gitwork with just one word.

That said, I have heard that ruby is a _very_ concise language :-)

jon.

Re: RFC: a plugin architecture for git extensions?

[Andreas Ericsson]

On 04/27/2011 02:50 PM, Jon Seymour wrote:
> On Wednesday, April 27, 2011, Andreas Ericsson  wrote:
>> On 04/27/2011 05:36 AM, Jon Seymour wrote:
>>> Has anyone ever given consideration to git supporting a plugin
>>> architecture for git extensions?
>>>
>>> The idea would be to provide a consistent way to install, and address
>>> extensions to the core git functionality in a manner that does not
>>> require the extension to actually be integrated into the git core.
>>>
>>
> 
>> Horrible idea. There are already as many package managers as there
>> are packages without us throwing another one into the mix.
>>
> 
> I agree that there are too many package managers. But do you know
> what? There isn't a single package manager that reliably works across
> platform. apt-get? great. Except you need something else for Mac,
> cywgin, or, um Fedora. Brew? Fine then you only need to worry about
> Linux and cygwin. Cygwin? ...
> 
> The platform for my extension is git. Not Mac. Not Debian. Not Fedora.
> Not cygwin. git.
> 
> The lowest common denominator across these environments is, um, git.
> 

You're utterly horribly wrong. It'll work well enough for scripted
languages but when you start mixing in compiler requirements and
whatnot the scheme falls apart. Quickly. Binary packages are popular
for (very good) reasons: They are simple, fast and there's a
reasonable chance they've been tested fairly well with the rest of
the system so nothing breaks horribly once you install it.

Perl, Ruby, Python and PHP all have their own extension installers.
That makes perfect sense since the same code runs unchanged on all
platforms (with some few exceptions).

> I challenge the sceptIcs to specify a one line command script that
> works across all possible environment that is more succinct than:
> 
>     git pm install gitwork
> 

That's not the point. Mac users supposedly already know about brew.
Fedora users already know about yum. Cygwin users... well, I have
no idea what they know about, but whatever it is, it's fairly safe
to assume they already know about it. That means they'll turn to
that familiar tool for managing packages when they want to install
something new. What you're proposing would force users on *all*
systems to have to learn a new one.

> It shouldn't be too hard. A tar command here, an enviroment  variable
> edit there. Perhaps a curl command or a browser download.
> 

And what you get in the end is a f*cking mess of spaghetti shell
code that works worse than the existing package managers.

And you're right. It's not too hard, so long as every extension
manager maintains some short list of requirements in the proper
format, which current package maintainers will have to learn if
they want some modules to be part of the "default" system install,
the way a whole bunch of Perl modules are.

> You have 4 words. Knock yourself out.
> 

make install

Made it in 2. What you described is what the user does to get
new extensions. What I described below is what developers have
to do to make their extensions easy to install *without* a
package manager even if the distro the user is on doesn't ship
that particular extension.

> 
>>> So, perhaps it won't be included in git. That's fine, I can build my
>>> own fork of git which includes the proposed extension [ indeed, this
>>> is how I originally developed it]. That's fine for
>>> me, but it isn't the most practical way to distribute it to others
>>> since I'll have to produce distribution packages for a variety of
>>> different distribution formats or fallback to tars and zips.
>>>
>>
>> What you can do is let your Makefile (or some other install-script)
>> take the destination path for "make install" (or equivalent) from
>> the output of "git --exec-path".
>>
>> That way, you can ship "git extadd" or whatever you want to call it
>> as a simple installer that installs executable and man-page in their
>> proper locations. If the commands you install require configuration
>> by default I'd say they're broken to begin with, but even that can
>> be remedied by running "git config --add key value" from the installer.
>>
>> So in a way, git is already its own pkg-config binary and anyone
>> clever enough to write useful scripts that enhances git will almost
>> certainly see that and use it from their favourite language quite
>> without having to learn some new magic format for package management.
>>
> 
> Those 3 paragraphs were substantially longer than 4 words. Again,
> there is a tar ball, let me know how I can install it across alll
> environments that got runs on. Make sure the man pages work.
> 

So the complete description would be

  git clone git://somerepo/gitworks
  cd gitworks
  make install

and the rest is in developer hands. If you make the extension
manager run *ONLY* those commands, you've made it fairly simple for
extension developers and users both, but then the gain is so
extremely small that it's hardly worth it anymore, although I agree
it would be nifty to keep a list of popular extensions on a wiki
somewhere so users can actually find them.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 12:52 AM, Andreas Ericsson <ae@op5.se> wrote:
> On 04/27/2011 02:50 PM, Jon Seymour wrote:
>> On Wednesday, April 27, 2011, Andreas Ericsson  wrote:
>>> On 04/27/2011 05:36 AM, Jon Seymour wrote:
>>>> Has anyone ever given consideration to git supporting a plugin
>>>> architecture for git extensions?
>>>>
>>>> The idea would be to provide a consistent way to install, and address
>>>> extensions to the core git functionality in a manner that does not
>>>> require the extension to actually be integrated into the git core.
>>>>
>>>
>>
>>> Horrible idea. There are already as many package managers as there
>>> are packages without us throwing another one into the mix.
>>>
>>
>> I agree that there are too many package managers. But do you know
>> what? There isn't a single package manager that reliably works across
>> platform. apt-get? great. Except you need something else for Mac,
>> cywgin, or, um Fedora. Brew? Fine then you only need to worry about
>> Linux and cygwin. Cygwin? ...
>>
>> The platform for my extension is git. Not Mac. Not Debian. Not Fedora.
>> Not cygwin. git.
>>
>> The lowest common denominator across these environments is, um, git.
>>
>
> You're utterly horribly wrong. It'll work well enough for scripted
> languages but when you start mixing in compiler requirements and
> whatnot the scheme falls apart. Quickly. Binary packages are popular
> for (very good) reasons: They are simple, fast and there's a
> reasonable chance they've been tested fairly well with the rest of
> the system so nothing breaks horribly once you install it.
> Perl, Ruby, Python and PHP all have their own extension installers.
> That makes perfect sense since the same code runs unchanged on all
> platforms (with some few exceptions).
>

Yeah, but that's when you delegate to a OS-specific package manager.

Concens. Separated. Good principle, that.

>> I challenge the sceptIcs to specify a one line command script that
>> works across all possible environment that is more succinct than:
>>
>>     git pm install gitwork
>>
>
> That's not the point. Mac users supposedly already know about brew.
> Fedora users already know about yum. Cygwin users... well, I have
> no idea what they know about, but whatever it is, it's fairly safe
> to assume they already know about it. That means they'll turn to
> that familiar tool for managing packages when they want to install
> something new. What you're proposing would force users on *all*
> systems to have to learn a new one.
>
>> It shouldn't be too hard. A tar command here, an enviroment  variable
>> edit there. Perhaps a curl command or a browser download.
>>
>
> And what you get in the end is a f*cking mess of spaghetti shell
> code that works worse than the existing package managers.
>

I guess that really depends on who you ask to write the shell script.

Most package managers have fairly straight forward interfaces.

   brew install blah
   apt-get install blah
   git clone git://github.com/blah/blah

There is no reason why some with a modicum of shell scripting nous
could not whip together
a simple meta interface for platform-specific package managers that
knows how to:

* read a specification from a git config file
* apply that specification to the task of invoking a platform specific
package manager.

Some one really smart could probably do it in an extensible way that
coped with the concept that different OS-platforms have different
package managers.

Most of this pasta can be cooked once, by the person who writes
gpm/git-pm. Sauce would be extra, of course.

> And you're right. It's not too hard, so long as every extension
> manager maintains some short list of requirements in the proper
> format, which current package maintainers will have to learn if
> they want some modules to be part of the "default" system install,
> the way a whole bunch of Perl modules are.
>
>> You have 4 words. Knock yourself out.
>>
>
> make install

>
> Made it in 2. What you described is what the user does to get
> new extensions. What I described below is what developers have
> to do to make their extensions easy to install *without* a
> package manager even if the distro the user is on doesn't ship
> that particular extension.
>

Again, there is a package called gitwork, available. It is available
as a tarball. Somewhere.
Install it.

* look up the url (google, might help)
* dowload it with your favourite download tool (browser, curl)
* unpack it
* install its dependencies, if required
* configure it
* buiild it

Your two words only specified the very last step. I needed 6 bullet
points merely to explain the details you omitted.

>
> So the complete description would be
>
>  git clone git://somerepo/gitworks
>  cd gitworks
>  make install

Still more than 4 words.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

I think my use of the word package was unfortunate, since it suggests
I am proposing an alternative to tools such as apt-get, brew, rpm etc.

This is not the intention. The intention is to manage _plugins_ to
git, treating git itself as a platform.

Plugins will be delivered via platform-specific package managers
(perhaps sequenced by git-pm), but once they arrive on the OS platform
they will be _activated_ by the plugin manager and this made available
to the git command line.

Platform specific concerns such as building and (most) dependency
management will be delegated to platform specific package managers.

The overriding objective is to allow a git user to install a git
plugin called foobar with 4 words:

     git pm install foobar

given that someone, somwhere, has done the work to create a plugin
descriptor and create an installable package of some kind for whatever
package managers are required in order to successfully install the
plugin on the target platform.

The same command should work whether your git platform is hosted on
MAC OSX, cygwin, Debian, Fedora, AIX or Windows.

Where git can be used as the underlying package manager, it will be
(for extensions which really are just source repos). If more
sophisticated build support is required, then that will be delegated
to a platform specific package manager via one of a small number of
package manager adapters.

Here is an updated README that hopefully makes this clear.

[ An easier to read version is here - https://github.com/jonseymour/gpm ]

--------------------

NAME
====
git-pm - a plugin manager for git extensions

DESCRIPTION
===========
The initial deliverable of the project will be a plugin architecture
proposal. The intent of this deliverable is to create specifications
that will allow plugin authors to create descriptors for their plugins
that will be sufficient to enable git-pm to locate an appropriate
package manager and delegate installation of the plugin to that package manager.

In parallel to this deliverable, a plugin manager interface (git-pm)
will be developed. Such an interface will manage git plugins on
behalf of git by delegating to whatever package-managers are available
and installed on the platform.

PRINCIPLES
==========
* define a repository layout specification
* define a plugin descriptor (using the git config syntax)
* package manager agnostic repo and plugin specifications
 * perhaps define a pluggable plugin manager interface
* support for:
 * existing contrib/ directory
 * existing git patterns for finding and using extensions
 * man pages
 * plugin specific configuration help
 * bash completions
 * scripts
* build support:
 * for documentation, archives
* layers
 * repo, plugin descriptors
 * tool interface specifications
 * tool implementations
 * global plugin registry and repository

INTENDED SCENARIOS
==================

Installation and Removal
------------------------
There is completely understandable resistance to YAPM - yet another
package manager. This is not the goal of git-pm. In particular, git-pm
will delegate platform-specific build
and deployment concerns to platform-specific package managers such as
apt-get, rpm, brew and cygwin. That said, there is no good reason why
git-pm shouldn't know how to delegate such concerns to a
platform-specific package manager.

The basic intent is to allow plugin authors to provide simple
instructions to potential consumers of their plugin.

<hr/>
Installing a plugin should be as simple as:

	   git pm install foobar

Such a comamnd should work whether the platform is Linux, cygwin or
Mac OSX. In fact, the only
common denominator should be git itself, and its dependencies (POSIX
shell and perl).
<hr/>
Install a plugin, given a URL that can locate its descriptor:

	   git pm install [ plugin-url | plugin-archive | plugin-name ]

<hr/>
Remove a plugin, given a name that can identify it:

	   git pm remove plugin-name

<hr/>
Update an installed plugin:

	   git pm update plugin-name

<hr/>
Inspect a registry of available available, installed, active or
inactive plugins:

	   git pm list available|installed|active|inactive

<hr/>
Describe the current installation, availability or activation status
of a plugin:

	   git pm status plugin-name

<hr/>

Activation/De-activation
------------------------
Activation is the means by which a locally available plugin can be
made available to the git command line. The idea is to

	   git-pm activate plugin-name
	   git-pm deactivate plugin-name

WHAT GIT-PM IS:
===============
* a native extension manager for the git platform
* an _activator_ for git extensions
* distribution and platform package mamager friendly
* git-aware

WHAT GIT-PM IS NOT:
===================
* a build tool
* a replacement for a package manager
* useful for anything other than git extensions

CONCEPTS
========

plugin
------
An extension to git that exports 1 or more commands to the git command line

package
-------
A platform-specific archive that is installable by a platform-specific
package manager. Packages will _package_ plugins.

plugin-descriptor
-----------------
A package-manager agnostic descriptor that describes a plugin to the
git platform, in particular git-pm.

package-descriptor
------------------
A package-manager specific descriptor that describes a package to a
package manager.

plugin manager
--------------
A command, such as git-pm, that can install, remove, activate or
deactive plugins by delegating platform specific
concerns to a platform-specific package manager.

package-manager
---------------
A platform-specific manager of packages.

plugin author
-------------
An author of git plugins.

package author
--------------
An author of package specifications for a package manager. In
particular, the author of a package specification that allows
a git plugin to be bundled into a package for the purposes of
distribution and management.

package-manager-adapter
-----------------------
A pluggable component of git-pm that exposes a platform-specific
package manager via a uniform interface.

DEPENDENCY MANAGEMENT
=====================
It is unclear yet how important dependency management will be. Where
possible, such concerns will be delegated to
platform-specific package managers. That said, there may still be
value in managing dependencies between git-pm
packages at the git-pm level.

PACKAGE NAME
============
The package name is currently gpm. Howeve, the "General Purpose Mouse"
package has already claimed this name in the apt space, at least.

So, it might be better to use 'git-pm' as the package name, which
isn't so bad since idiomatic use of the package should be 'git pm
blah'.

SUPPORTED PACKAGE MANGERS
=========================
The intent of git-pm is not to duplicate the functionality of existing
package managers. The intent is to provide a package manager for git
as a platform itself. To the extent that a build process is required
to install a git extension, then such concerns will be delegated
to a real package manager that knows how to deal with such concerns.

The intent is that a minimal plugin that depends only on the
availability of a shell, should be installable with something as
simple as:

    git pm install foobar

This should work whether your git install is running on Linux, cywgin,
Windows (MSYSGIT), MAC, AIX or whatever.

If a compilation is required, then delegation to a platform package
manager will be requried.

Linux
-----
* git
* rpm
* apt

Mac
---
* git
* brew

cygwin
---
* git

CONTRACTS
=========
The following contracts will be required between:

<dl>
<dt>the git user and git-pm</dt>
<dd>
This contract will be specified in terms of the git-pm man page. It
will specify the plugin management commands offered by
the git-pm interface to the user.
</dd>
<dt>the git runtime and git-pm</dt>
<dd>
This contract will specify the technique by which git-pm exposes git
extensions to the git command line. It will include
support for exposing:
<ul>
<li>sub-commands</li>
<li>man pages</li>
<li>shell completions</li>
</ul>
</dd>
<dt>git-pm and package-manager adapters</dt>
<dd>
This contract will specify how to add a new package-manager adapter.
The purpose will be to allow git-pm to delegate
its interface to backend package managers that know how to manage
platform specific concerns such as building,
dependency management, package distribution etc.
</dd>
</dd>
<dt>package-manager adapters and package managers</dt>
<dt>extension authors and the git-pm package manager</dt>
</dl>

INFLUENCES
==========
The following programs have influenced the design of git-pm.

npm
---
The node package manager.

Node is an interesting, V8-JavaScript based runtime for implementing
servers. Node comes with a tightly coupled package manager npm. If
node is your platform, npm
is the package manager for that platform. In particular, npm works the
same way irrespective of which platform you have node installed on.

brew
----
A Mac OSX specific package manager.

Brew is a relatively new package manager for the Mac OSX operating
system. It has several nice features, including its formula registry
that allows
authors to specify succinct package management formulae as short ruby
scripts. Brew also uses git as an integral part of its registry and
run-time.

SCEPTICS CHALLENGE
==================
Some people doubt there is value in a 'git plugin mamager'. In
response to this scepticism, I pose the following
challenge.

1. take an arbitrary git extension.
2. specify installation instructions for that extension in 4 words or less.
3. make sure those 4 words work on Linux, Cygwin, MAC OSX and AIX.
4. verify that following installation, the man pages work.

As an example, there is a package called gitwork. It is available
somewhere on the Internet. I think, as a tarball. Or perhaps a zip. I
don't think it has any dependencies, but I am really not sure. Why
don't you suck it and see?

MAILING LIST
============
Until noted otherwise, please use the
[http://dir.gmane.org/gmane.comp.version-control.git](git@vger.kernel.org)
mailing list for discussions about design decisions. gpm: is suggested
a good prefix for such discussions.

REVISIONS
=========
Ordered most recent, to less recent:

* changed 'gpm' to 'git pm' on the assumption that 'git-pm' will be in
the path and that 'General Purpose Mouse' has already nabbed 'gpm'
* changed terminology from 'package' to 'plugin' to help reduce
consfusion about the objectives of the project

AUTHOR
======
Jon Seymour

Re: RFC: a plugin architecture for git extensions?

[A Large Angry SCM]

On 04/27/2011 12:13 PM, Jon Seymour wrote:
> I think my use of the word package was unfortunate, since it suggests
> I am proposing an alternative to tools such as apt-get, brew, rpm etc.
>
> This is not the intention. The intention is to manage _plugins_ to
> git, treating git itself as a platform.
>
> Plugins will be delivered via platform-specific package managers
> (perhaps sequenced by git-pm), but once they arrive on the OS platform
> they will be _activated_ by the plugin manager and this made available
> to the git command line.
>
> Platform specific concerns such as building and (most) dependency
> management will be delegated to platform specific package managers.
>
> The overriding objective is to allow a git user to install a git
> plugin called foobar with 4 words:
>
>       git pm install foobar
>
> given that someone, somwhere, has done the work to create a plugin
> descriptor and create an installable package of some kind for whatever
> package managers are required in order to successfully install the
> plugin on the target platform.
>
> The same command should work whether your git platform is hosted on
> MAC OSX, cygwin, Debian, Fedora, AIX or Windows.
>
> Where git can be used as the underlying package manager, it will be
> (for extensions which really are just source repos). If more
> sophisticated build support is required, then that will be delegated
> to a platform specific package manager via one of a small number of
> package manager adapters.

For a git plugin ecosystem to work, a (relatively) stable API/ABI is 
necessary for the plugin authors to code to. Where is your proposal for 
that.

Re: RFC: a plugin architecture for git extensions?

[Junio C Hamano]

Andreas Ericsson <ae@op5.se> writes:

> You're utterly horribly wrong. ...
> ...
> So the complete description would be
>
>   git clone git://somerepo/gitworks
>   cd gitworks
>   make install
>
> and the rest is in developer hands.

Yeah, I like this as the conclusion of this thread ;-).

Re: RFC: a plugin architecture for git extensions?

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Andreas Ericsson <ae@op5.se> writes:
>
>> You're utterly horribly wrong. ...
>> ...
>> So the complete description would be
>>
>>   git clone git://somerepo/gitworks
>>   cd gitworks
>>   make install
>>
>> and the rest is in developer hands.
>
> Yeah, I like this as the conclusion of this thread ;-).

Having said that, to make this work well not just for the command but for
documentation and help, there needs a way for the build procedure of such
user-script project to query the manpage and the documentation paths, just
like we let them query the executable path via "git --exec-path".

Re: RFC: a plugin architecture for git extensions?

[Drew Northup]

On Wed, 2011-04-27 at 11:28 -0700, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Andreas Ericsson <ae@op5.se> writes:
> >
> >> You're utterly horribly wrong. ...
> >> ...
> >> So the complete description would be
> >>
> >>   git clone git://somerepo/gitworks
> >>   cd gitworks
> >>   make install
> >>
> >> and the rest is in developer hands.
> >
> > Yeah, I like this as the conclusion of this thread ;-).
> 
> Having said that, to make this work well not just for the command but for
> documentation and help, there needs a way for the build procedure of such
> user-script project to query the manpage and the documentation paths, just
> like we let them query the executable path via "git --exec-path".

I was just thinking of that, and for hoots and hollers I
copied /usr/share/man/man1/git-am.1.gz
to /usr/share/man/man1/git-amp.1.gz and tried "git help amp" on it.

	[dnorthup@drew-northup ~]$ git help amp
	No manual entry for gitamp

So, that doesn't work. I haven't checked yet how Git "knows" what valid
pages are available for "git help" but just putting another file in the
same directory as the others didn't do the job (at least not on my
workstation).

However, as noted earlier, copying /usr/bin/git-am to /usr/bin/git-amp
did work. Executing "git amp -h" resulted in the built-in help text for
'git am' being printed to the screen, exactly as expected.

-- 
-Drew Northup
________________________________________________
"As opposed to vegetable or mineral error?"
-John Pescatore, SANS NewsBites Vol. 12 Num. 59

Re: RFC: a plugin architecture for git extensions?

[Motiejus Jakštys]

On Thu, Apr 28, 2011 at 02:13:56AM +1000, Jon Seymour wrote:
> Platform specific concerns such as building and (most) dependency
> management will be delegated to platform specific package managers.
> 
> The overriding objective is to allow a git user to install a git
> plugin called foobar with 4 words:
> 
>      git pm install foobar
> 
> given that someone, somwhere, has done the work to create a plugin
> descriptor and create an installable package of some kind for whatever
> package managers are required in order to successfully install the
> plugin on the target platform.
> 
> The same command should work whether your git platform is hosted on
> MAC OSX, cygwin, Debian, Fedora, AIX or Windows.
> 
> Where git can be used as the underlying package manager, it will be
> (for extensions which really are just source repos). If more
> sophisticated build support is required, then that will be delegated
> to a platform specific package manager via one of a small number of
> package manager adapters.

Ok, I write my "git logx", which is a single file -- git-logx.c.
Compilation steps:
    $ gcc -lz git-logx.c. 

> then that will be delegated to a platform specific package manager 

Which one? Say I am on debian. dpkg-buildpackage? So I would get a
git-logx.deb, which I should install as root? In that case, you forgot
one word:

      sudo git pm install foobar

Or I got it wrong?

Motiejus

Re: RFC: a plugin architecture for git extensions?

[Motiejus Jakštys]

On Wed, Apr 27, 2011 at 08:16:12PM +0100, Motiejus Jakštys wrote:
> Which one? Say I am on debian. dpkg-buildpackage? So I would get a
> git-logx.deb, which I should install as root? In that case, you forgot
> one word:
> 
>       sudo git pm install foobar
> 
> Or I got it wrong?

Maybe you mean this...
       $ git pm install foobar
       $ sudo dpkg -i foobar.deb

Which does not make really happy...

Motiejus

Re: RFC: a plugin architecture for git extensions?

[Joey Hess]

[-- Attachment #1: Type: text/plain, Size: 791 bytes --]

Drew Northup wrote:
> I was just thinking of that, and for hoots and hollers I
> copied /usr/share/man/man1/git-am.1.gz
> to /usr/share/man/man1/git-amp.1.gz and tried "git help amp" on it.
> 
> 	[dnorthup@drew-northup ~]$ git help amp
> 	No manual entry for gitamp
> 
> So, that doesn't work. I haven't checked yet how Git "knows" what valid
> pages are available for "git help" but just putting another file in the
> same directory as the others didn't do the job (at least not on my
> workstation).

If git-amp is available in $PATH, 'git help amp' runs 'man git-amp'.
Otherwise, it runs 'man gitamp'.

Tools like autoconf etc already know how to install man pages into the
right places. I don't see the need for any additional support for git
here.

-- 
see shy jo

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: RFC: a plugin architecture for git extensions?

[Drew Northup]

On Wed, 2011-04-27 at 15:42 -0400, Joey Hess wrote:
> Drew Northup wrote:
> > I was just thinking of that, and for hoots and hollers I
> > copied /usr/share/man/man1/git-am.1.gz
> > to /usr/share/man/man1/git-amp.1.gz and tried "git help amp" on it.
> > 
> > 	[dnorthup@drew-northup ~]$ git help amp
> > 	No manual entry for gitamp
> > 
> > So, that doesn't work. I haven't checked yet how Git "knows" what valid
> > pages are available for "git help" but just putting another file in the
> > same directory as the others didn't do the job (at least not on my
> > workstation).
> 
> If git-amp is available in $PATH, 'git help amp' runs 'man git-amp'.
> Otherwise, it runs 'man gitamp'.

That's what I thought too, but it didn't work! (<still scratching head
here>) I'll have to try a couple of other ways to break it later on.

> Tools like autoconf etc already know how to install man pages into the
> right places. I don't see the need for any additional support for git
> here.

I'm not saying there should be--in fact quite the opposite.

-- 
-Drew Northup
________________________________________________
"As opposed to vegetable or mineral error?"
-John Pescatore, SANS NewsBites Vol. 12 Num. 59

Re: RFC: a plugin architecture for git extensions?

[Motiejus Jakštys]

On Wed, Apr 27, 2011 at 01:36:44PM +1000, Jon Seymour wrote:
> Has anyone ever given consideration to git supporting a plugin
> architecture for git extensions?
> 

How about this proposition? From the user perspective:
$ cd /somewhere/might/be/home/or/project/.git/ext/
$ git clone git://github.com/jonseymour/gitwork.git

What user finds in gitwork/ is totally up to maintainer. git cares about
two files:
    .git/ext/git-work(.exe)
    .git/ext/git-work.1.gz

$ cd gitwork;
$ (c)make|./waf|scons|what_the_hell_just_produce_git-work
$ git config ext.work.enabled true

What git does when git <command> is invoked:
-------------------------------------------
* config.ext.<command>.enabled is true. If not, do nothing new.
* if above is true, search for git-work in:
  ** project directory
  ** user home directory
* if found, execute it
* if not found, fallback to default mechanism (check in
    cmd_struct_commands, etc)

Similar with man pages.

Think about how Vim plugins are distributed. Vim developers assumed that
users are educated enough to download the plugin and extract it to the
right place. Are git users less intelligent to download & install it,
especially in a case when they need an *extension*?

Are you trying to kill a bird with a rock?

Motiejus

Re: RFC: a plugin architecture for git extensions?

[Junio C Hamano]

Joey Hess <joey@kitenet.net> writes:

> Tools like autoconf etc already know how to install man pages into the
> right places. I don't see the need for any additional support for git
> here.

But the original point by Andreas was that third-party tools like "git
work" should be able to just say "make install", and of course the distro
people can take it from there, but the thing is, distro people can take it
from a very low starting point.  Unless "make install" does not figuire
out how the user's git is configured and installed, it won't be very
useful for end users who do say "make install" for whatever reason,
instead of installing it from their distro.

For example, on my primary development box, I do not have any git
installed from distribution, but I do have git on my $PATH.  For such
users, "make install" should be able to find out that the right place to
install git-work.1 is in $HOME/some/where/man/man1 directory.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

2011/4/28 Motiejus Jakštys <desired.mta@gmail.com>:
- Hide quoted text -
> On Wed, Apr 27, 2011 at 01:36:44PM +1000, Jon Seymour wrote:
>> Has anyone ever given consideration to git supporting a plugin
>> architecture for git extensions?
>>
>
> How about this proposition? From the user perspective:
> $ cd /somewhere/might/be/home/or/project/.git/ext/
> $ git clone git://github.com/jonseymour/gitwork.git
>
> What user finds in gitwork/ is totally up to maintainer. git cares about
> two files:
>    .git/ext/git-work(.exe)
>    .git/ext/git-work.1.gz
>
> $ cd gitwork;
> $ (c)make|./waf|scons|what_the_hell_just_produce_git-work
> $ git config ext.work.enabled true
>
> What git does when git <command> is invoked:
> -------------------------------------------
> * config.ext.<command>.enabled is true. If not, do nothing new.
> * if above is true, search for git-work in:
>  ** project directory
>  ** user home directory
> * if found, execute it
> * if not found, fallback to default mechanism (check in
>    cmd_struct_commands, etc)
>
> Similar with man pages.
>
> Think about how Vim plugins are distributed. Vim developers assumed that
> users are educated enough to download the plugin and extract it to the
> right place. Are git users less intelligent to download & install it,
> especially in a case when they need an *extension*?
>
> Are you trying to kill a bird with a rock?
>
> Motiejus
>

I think that is a little more invasive of git-core than it needs to be.

I was thinking more along the lines of, having solved the distribution
and build problem _some other way_, resulting in a foobar.gpm file
ending up in, say, /usr/local/lib/git-pm/foobar.gpm.

You would then run:

   git pm activate foobar

which would then link:

  ln -sf /usr/local/bin/git-foobar ~/.git-pm/activated/libexec/git-foobar
  ln -sf /usr/local/share/man/man1 ~/.git-pm/activated/share/man/man1

One would have to have previously arranged for the sub directories of
~/git-pm/activated to be in the PATH and MANPATH paths.

Deactivating would involve removing the links.

Deciding which links to link during activation and de-activation would
be driven by a short descriptor file.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jonathan Nieder]

Junio C Hamano wrote:

> For example, on my primary development box, I do not have any git
> installed from distribution, but I do have git on my $PATH.  For such
> users, "make install" should be able to find out that the right place to
> install git-work.1 is in $HOME/some/where/man/man1 directory.

Sorry to be dense, but: isn't the right place to install git-work.1
one of

	/usr/local/share/man
	/usr/share/man
	/opt/man
	$HOME/man
	$prefix/man

depending on where the git-work binary was installed?  In the $prefix
case, the same snippet in .profile that adds $prefix/bin to the $PATH
would also say

	MANPATH=$prefix/man:$(manpath)

Or is the idea to blindly install (a symlink to) git-work to $(git
--exec-path)/ rather than a place on the $PATH?  In this case, I would
be a little worried.  How will the helper deal with uninstallation and
with namespace conflicts?  (On the $PATH, these are expected problems
and I'd expect each user has some way of dealing with them already.)

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 8:08 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Junio C Hamano wrote:

> Or is the idea to blindly install (a symlink to) git-work to $(git
> --exec-path)/ rather than a place on the $PATH?  In this case, I would
> be a little worried.  How will the helper deal with uninstallation and
> with namespace conflicts?  (On the $PATH, these are expected problems
> and I'd expect each user has some way of dealing with them already.)

I see 'git pm activate' managing symbolic links in a directory
dedicated to the purpose (.e.g. ~/.git-pm/activated).

One thing 'git pm activate' could do is check that the commands
exported by the 'gitwork' descriptor do not conflict with what is
already activated.

If the user has done something like:

    git clone https://repo/gitwork.git ~/hub/gitwork

and then:

    git activate pm ~/hub/gitwork

the symbolic links would be established to ~/hub/gitwork, wherever
that happens to be.

If the user has done:

   apt-get install gitwork

then given a package-manager adapter for apt-get, it could extract the
.gpm file from the list of installed files, and resume activation from
there. Ultimately, the end result is the same ~/.git-pm/activated is
updated, it has always been on the paths it needs to be.

If the descriptor did have a list of exported commands (e.g. git-work,
git-base, git-atomic, git-test), then a global registry could use this
list of exported commands to detect conflicts early - at package
registration time which might help avoid grief down the track.

Re: RFC: a plugin architecture for git extensions?

[Pau Garcia i Quiles]

Hi,

Can we please split this debate into the two threads that have arisen?

a) git extensions (the original point)

b) git package manager


Let me give my unrequested opinion:

a) I like it. Mercurial has it. It requires more or less what Jon says
below: let's define a hierarchy of where to place the executables,
documentation, the extenions' porcelain (which IMHO would require one
directory per extension), etc

b) Please no. As a Debian developer, I'd rather see extensions
distributed as source, then I would package them. It's what Debian
(and other distributions) are doing now with Ruby gems, Python eggs,
etc: we provide packages for them so that you do not use gem, etc





On Wed, Apr 27, 2011 at 7:33 AM, Jon Seymour <jon.seymour@gmail.com> wrote:
> Hide quoted text -
> On Wed, Apr 27, 2011 at 3:17 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Jon Seymour <jon.seymour@gmail.com> writes:
>>
>>>> So if you have /home/js/bin on your $PATH, you can install your "git-work"
>>>> script as /home/js/bin/git-work, and that should be sufficient.
>>>
>>> Yep, that's a start, but does not a a complete plugin architecture make :-)
>>
>> Please explain yourself.
>>
>
> So, I think at a very minimum, a plugin architecture should specify
> the file system layout of packages to be managed by a plugin/package
> manager.
>
> So, where to find scripts, where to find man pages, bash completions,
> configuration help etc.
>
> A slightly more functional architecture would provide support for
> unpacking package archives into a "standard" repository location and
> for removing unwanted plugins.
>
> A plugin architecture might specify a standard way to access
> extensions. (git blah is easy for local use, but what if a plugin
> grabs a "noun" that the core wants to use that "noun" itself in
> future. Perhaps gitx blah would be a better standard way to access
> extensions. But that is an aside, I am sure this question has been
> considered previously).
>
> An even more functional architecture would provide support for a
> global registry of plugins. I understand that git may not want to
> write its own package manager (how many times has that been done),
> but it'd be nice if competing "git package managers" had a standard
> target to deploy to.
>
> My thoughts about this are inspired by how the node project manages
> packages with its npm package manager and also the fact that I have
> several ideas on the boil at the moment that would definitely benefit
> from a standard way to manage these concerns.
>
> jon.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Pau Garcia i Quiles
http://www.elpauer.org
(Due to my workload, I may need 10 days to answer)

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 8:32 AM, Pau Garcia i Quiles
<pgquiles@elpauer.org> wrote:
> Hi,
>
> Can we please split this debate into the two threads that have arisen?
>
> a) git extensions (the original point)
>
> b) git package manager
>
>
> Let me give my unrequested opinion:
>
> a) I like it. Mercurial has it. It requires more or less what Jon says
> below: let's define a hierarchy of where to place the executables,
> documentation, the extenions' porcelain (which IMHO would require one
> directory per extension), etc
>


> b) Please no. As a Debian developer, I'd rather see extensions
> distributed as source, then I would package them. It's what Debian
> (and other distributions) are doing now with Ruby gems, Python eggs,
> etc: we provide packages for them so that you do not use gem, etc
>

I absolutely agree that is the right approach. To the extent that my
proposal supports:

    git pm install foobar

it would do so by delegation to package-manager adapters (in effect
acting as a meta package manager).

However, I don't want to further distract discussion by having a
debate about whether a meta package
manager is a good idea or not.  So let's concentrate on what:

   git pm activate

would look like and do.

jon.

Re: RFC: a plugin architecture for git extensions?

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Junio C Hamano wrote:
>
>> For example, on my primary development box, I do not have any git
>> installed from distribution, but I do have git on my $PATH.  For such
>> users, "make install" should be able to find out that the right place to
>> install git-work.1 is in $HOME/some/where/man/man1 directory.
>
> Sorry to be dense, but: isn't the right place to install git-work.1
> one of
>
> 	/usr/local/share/man
> 	/usr/share/man
> 	/opt/man
> 	$HOME/man
> 	$prefix/man
>
> depending on where the git-work binary was installed?

I was thinking it, and the location git-work binary gets installed, should
depend on where "git" and its subcommand binaries are installed.  The word
plug-in mentioned in the thread implied that whatever plugs in is not by
itself full fledged thing that is useful standalone, so it seemed a very
natural thing to do.

> In the $prefix
> case, the same snippet in .profile that adds $prefix/bin to the $PATH
> would also say
>
> 	MANPATH=$prefix/man:$(manpath)

You are correct only if "git" the user is building is _not_ changed to
look for other places for its own manpages.  If "git" was built to look at
somewhere else, the relationship between the output of "git --exec-path"
and that location shouldn't be assumed to be ../../share/man or anything.

The layout should be discoverable, by exposing system_path(GIT_MAN_PATH)
and friends (see builtin/help.c), just like we expose GIT_EXEC_PATH.

> Or is the idea to blindly install (a symlink to) git-work to $(git
> --exec-path)/ rather than a place on the $PATH?

You can call it _blindly_ if you like, but that is what I meant.  "git"
tells where the binary and help material for a "plugin" to be installed,
so that it can find them where it expects to.

After all, I am not interested at all in adding "git pm" or other crap.  I
am just trying to help people write their own "make install" of a plugin
project, like "git work".  And writing "make uninstall" for that project
should be doable with the same information I am trying to give in this
thread, no?

Re: RFC: a plugin architecture for git extensions?

[Jonathan Nieder]

Junio C Hamano wrote:

> I was thinking it, and the location git-work binary gets installed, should
> depend on where "git" and its subcommand binaries are installed.  The word
> plug-in mentioned in the thread implied that whatever plugs in is not by
> itself full fledged thing that is useful standalone, so it seemed a very
> natural thing to do.

With the goal of making some commands available to git without putting
them on the $PATH in mind, this all makes more sense.  Sorry I missed
that before.

>> Or is the idea to blindly install (a symlink to) git-work to $(git
>> --exec-path)/ rather than a place on the $PATH?
>
> You can call it _blindly_ if you like, but that is what I meant.  "git"
> tells where the binary and help material for a "plugin" to be installed,
> so that it can find them where it expects to.

Right, my worry was based on the usual way programs find their way
onto my $PATH.  That is:

 - if they are installed via a package from the distro, they are in
   /usr/bin.

 - if they are installed with "make install" by the local sysadmin for
   all users, they are in /usr/local/bin.

 - if I am trying them out for myself, they are in $HOME/opt/foo/bin
   and when it is time to remove it, "rm -fr $HOME/opt/foo".

 - if I have adopted them, symlinks go in $HOME/bin.

With a local gcc-4.6 in $HOME/bin, if the sysadmin upgrades gcc so
gcc-4.6 is to appear in /usr/bin or /usr/local/bin, my setup still
works without trouble.  So, barring bugs, each installation method
does not interfere with the other ones.

Call it overengineering, but I would want a way for installing new git
commands to have the same attributes (installable by normal users in
multiuser systems and name conflicts not being a terrible
administrative burden).  A simple way would be to introduce
GIT_MAN_PATH as you described and to teach "git help" to accept a
GIT_EXEC_PATH consisting of a colon-delimited (semicolon-delimited on
Windows) list of directories.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 9:27 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>> Junio C Hamano wrote:
>
> I was thinking it, and the location git-work binary gets installed, should
> depend on where "git" and its subcommand binaries are installed.  The word
> plug-in mentioned in the thread implied that whatever plugs in is not by
> itself full fledged thing that is useful standalone, so it seemed a very
> natural thing to do.
>

You are correct about this - the ~/.git-pm/activation was, um, a horrible idea.

>> In the $prefix
>> case, the same snippet in .profile that adds $prefix/bin to the $PATH
>> would also say
>>
>>       MANPATH=$prefix/man:$(manpath)
>
> You are correct only if "git" the user is building is _not_ changed to
> look for other places for its own manpages.  If "git" was built to look at
> somewhere else, the relationship between the output of "git --exec-path"
> and that location shouldn't be assumed to be ../../share/man or anything.
>
> The layout should be discoverable, by exposing system_path(GIT_MAN_PATH)
> and friends (see builtin/help.c), just like we expose GIT_EXEC_PATH.
>
>> Or is the idea to blindly install (a symlink to) git-work to $(git
>> --exec-path)/ rather than a place on the $PATH?
>
> You can call it _blindly_ if you like, but that is what I meant.  "git"
> tells where the binary and help material for a "plugin" to be installed,
> so that it can find them where it expects to.
>

> After all, I am not interested at all in adding "git pm" or other crap.  I
> am just trying to help people write their own "make install" of a plugin
> project, like "git work".  And writing "make uninstall" for that project
> should be doable with the same information I am trying to give in this
> thread, no?

Ok, how about this?

plugins/
  gitwork/
     git-work.sh
     git-work.1

libexec/
  plugins/
    git-work -> ../../plugins/gitwork/git-work.sh

share/
  man/
     man1/
         git-work.1 -> ../../../plugins/git-work.1

The function of a:

    git plugin activate git-work

would simply be to establish the symlinks.

gitwork would get into plugins/ by your existing platform manager or
make install do some minimal validation of conflicts, etc.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 10:06 AM, Jon Seymour <jon.seymour@gmail.com> wrote:

>
> gitwork would get into plugins/ by your existing platform manager or
> make install do some minimal validation of conflicts, etc.
>

Sorry, garbled

I meant:

gitwork would get into plugins/ via the path of an existing platform
manager or make install.

One function of git plugin activate would be to perform some minimal
validation to report on and/or prevent conflicts.

jon.

Re: RFC: a plugin architecture for git extensions?

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Right, my worry was based on the usual way programs find their way
> onto my $PATH.  That is:
>
>  - if they are installed via a package from the distro, they are in
>    /usr/bin.
>
>  - if they are installed with "make install" by the local sysadmin for
>    all users, they are in /usr/local/bin.
>
>  - if I am trying them out for myself, they are in $HOME/opt/foo/bin
>    and when it is time to remove it, "rm -fr $HOME/opt/foo".
>
>  - if I have adopted them, symlinks go in $HOME/bin.
>
> With a local gcc-4.6 in $HOME/bin, if the sysadmin upgrades gcc so
> gcc-4.6 is to appear in /usr/bin or /usr/local/bin, my setup still
> works without trouble.  So, barring bugs, each installation method
> does not interfere with the other ones.
>
> Call it overengineering, but I would want a way for installing new git
> commands to have the same attributes (installable by normal users in
> multiuser systems and name conflicts not being a terrible
> administrative burden).

Ok, I wasn't thinking about folks who use repackaged /usr/bin/git together
with their own choice of third-party enhancements.

Probably we would be better off if we define a new set of paths that is
independent from GIT_EXEC_PATH and friends.  The installed git and nothing
else will occupy GIT_EXEC_PATH etc., but at the runtime, git would look at
a user-writable location GIT_PLUGIN_PATH/{bin,man,...} to see if the user
has her own customization, and add them to its vocabulary.

Or something like that.  I am not all that interested, but it feels like a
good direction.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 10:10 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>> Right, my worry was based on the usual way programs find their way
>> onto my $PATH.  That is:
>>
>>  - if they are installed via a package from the distro, they are in
>>    /usr/bin.
>>
>>  - if they are installed with "make install" by the local sysadmin for
>>    all users, they are in /usr/local/bin.
>>
>>  - if I am trying them out for myself, they are in $HOME/opt/foo/bin
>>    and when it is time to remove it, "rm -fr $HOME/opt/foo".
>>
>>  - if I have adopted them, symlinks go in $HOME/bin.
>>
>> With a local gcc-4.6 in $HOME/bin, if the sysadmin upgrades gcc so
>> gcc-4.6 is to appear in /usr/bin or /usr/local/bin, my setup still
>> works without trouble.  So, barring bugs, each installation method
>> does not interfere with the other ones.
>>
>> Call it overengineering, but I would want a way for installing new git
>> commands to have the same attributes (installable by normal users in
>> multiuser systems and name conflicts not being a terrible
>> administrative burden).
>
> Ok, I wasn't thinking about folks who use repackaged /usr/bin/git together
> with their own choice of third-party enhancements.
>
> Probably we would be better off if we define a new set of paths that is
> independent from GIT_EXEC_PATH and friends.  The installed git and nothing
> else will occupy GIT_EXEC_PATH etc., but at the runtime, git would look at
> a user-writable location GIT_PLUGIN_PATH/{bin,man,...} to see if the user
> has her own customization, and add them to its vocabulary.
>
> Or something like that.  I am not all that interested, but it feels like a
> good direction.
>

I agree. Apologies for confusing things by talking too much about a
git pm install command.

I think there are 3 levels of functionality. FWIW, I am suggesting
git-core stops at #2.

0. unmanaged plugins

git doesn't provide any explicit management of plugins, but will use
them if finds them.

Without some kind of management, however, you will be forced to dump
the man pages and scripts
for the plugins in one place.

This would be very distribution manager unfriendly since there could
be conflicts galore.

I guess an unmanaged solution could use separate directories for each
plugin, but this would imply scanning all these paths each time you
invoke git. In my view, symbolic links from a dir already
GIT_EXEC_PATH to plugin directories would be a more efficient way to
do this.

1. managed plugins

git provides minimal plugin management functionality. Each plugin has
its own directory, but an activate step is required to make the plugin
available to the GIT_EXEC_PATH and GIT_MAN_PATH.

This has the advantage that conflicts between plugins would be more
readily avoided and is potentially more performant. As Pau suggests,
this option is much more package manager friendly

It probably does require a git plugin command of some kind, however,
in order to perform the activation step.

2. managed packages

A meta-package manager for plugins, that delegates plugin installation
concerns to a platform package manager.

The thing is, you may absolutely hate #2, but if approach #1 is
adopted by git-core, someone can at least attempt this by, well,
writing a plugin for it.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 10:50 AM, Jon Seymour <jon.seymour@gmail.com> wrote:
> On Thu, Apr 28, 2011 at 10:10 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Jonathan Nieder <jrnieder@gmail.com> writes:
>>
>>> Right, my worry was based on the usual way programs find their way
>>> onto my $PATH.  That is:
>>>
>>>  - if they are installed via a package from the distro, they are in
>>>    /usr/bin.
>>>
>>>  - if they are installed with "make install" by the local sysadmin for
>>>    all users, they are in /usr/local/bin.
>>>
>>>  - if I am trying them out for myself, they are in $HOME/opt/foo/bin
>>>    and when it is time to remove it, "rm -fr $HOME/opt/foo".
>>>
>>>  - if I have adopted them, symlinks go in $HOME/bin.
>>>
>>> With a local gcc-4.6 in $HOME/bin, if the sysadmin upgrades gcc so
>>> gcc-4.6 is to appear in /usr/bin or /usr/local/bin, my setup still
>>> works without trouble.  So, barring bugs, each installation method
>>> does not interfere with the other ones.
>>>
>>> Call it overengineering, but I would want a way for installing new git
>>> commands to have the same attributes (installable by normal users in
>>> multiuser systems and name conflicts not being a terrible
>>> administrative burden).
>>
>> Ok, I wasn't thinking about folks who use repackaged /usr/bin/git together
>> with their own choice of third-party enhancements.
>>
>> Probably we would be better off if we define a new set of paths that is
>> independent from GIT_EXEC_PATH and friends.  The installed git and nothing
>> else will occupy GIT_EXEC_PATH etc., but at the runtime, git would look at
>> a user-writable location GIT_PLUGIN_PATH/{bin,man,...} to see if the user
>> has her own customization, and add them to its vocabulary.
>>
>> Or something like that.  I am not all that interested, but it feels like a
>> good direction.
>>
>
> I agree. Apologies for confusing things by talking too much about a
> git pm install command.
>
> I think there are 3 levels of functionality. FWIW, I am suggesting
> git-core stops at #2.
>
> 0. unmanaged plugins
>
> git doesn't provide any explicit management of plugins, but will use
> them if finds them.
>
> Without some kind of management, however, you will be forced to dump
> the man pages and scripts
> for the plugins in one place.
>
> This would be very distribution manager unfriendly since there could
> be conflicts galore.
>
> I guess an unmanaged solution could use separate directories for each
> plugin, but this would imply scanning all these paths each time you
> invoke git. In my view, symbolic links from a dir already
> GIT_EXEC_PATH to plugin directories would be a more efficient way to
> do this.
>
> 1. managed plugins
>
> git provides minimal plugin management functionality. Each plugin has
> its own directory, but an activate step is required to make the plugin
> available to the GIT_EXEC_PATH and GIT_MAN_PATH.
>
> This has the advantage that conflicts between plugins would be more
> readily avoided and is potentially more performant. As Pau suggests,
> this option is much more package manager friendly
>
> It probably does require a git plugin command of some kind, however,
> in order to perform the activation step.
>
> 2. managed packages
>
> A meta-package manager for plugins, that delegates plugin installation
> concerns to a platform package manager.
>
> The thing is, you may absolutely hate #2, but if approach #1 is
> adopted by git-core, someone can at least attempt this by, well,
> writing a plugin for it.
>

Sorry, sorry/. git-core stops at #1!!

> jon.
>

Re: RFC: a plugin architecture for git extensions?

[david]

On Thu, 28 Apr 2011, Jon Seymour wrote:

> On Thu, Apr 28, 2011 at 10:10 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Jonathan Nieder <jrnieder@gmail.com> writes:
>>
>
> I agree. Apologies for confusing things by talking too much about a
> git pm install command.
>
> I think there are 3 levels of functionality. FWIW, I am suggesting
> git-core stops at #2.
>
> 0. unmanaged plugins
>
> git doesn't provide any explicit management of plugins, but will use
> them if finds them.
>
> Without some kind of management, however, you will be forced to dump
> the man pages and scripts
> for the plugins in one place.
>
> This would be very distribution manager unfriendly since there could
> be conflicts galore.

every package manager I know of has no problem with multiple packages 
owning files in one directory.

if the files are named the same thing you will have a conflict, but if the 
files are named the same thing, the commands are probably going to be 
named the same, and so you will have conflicts in any case.

> I guess an unmanaged solution could use separate directories for each
> plugin, but this would imply scanning all these paths each time you
> invoke git. In my view, symbolic links from a dir already
> GIT_EXEC_PATH to plugin directories would be a more efficient way to
> do this.

I think you are overanalyzing the problem

> 1. managed plugins
>
> git provides minimal plugin management functionality. Each plugin has
> its own directory, but an activate step is required to make the plugin
> available to the GIT_EXEC_PATH and GIT_MAN_PATH.
>
> This has the advantage that conflicts between plugins would be more
> readily avoided and is potentially more performant. As Pau suggests,
> this option is much more package manager friendly

I don't see how this will avoid conflicts. what files are you thinking 
that the different plugins will make that won't conflict any more than the 
commands themselves will?

> It probably does require a git plugin command of some kind, however,
> in order to perform the activation step.

only if you think you need a 'installed but not active' mode of operation, 
and I don't understand why you would want that.

David Lang

> 2. managed packages
>
> A meta-package manager for plugins, that delegates plugin installation
> concerns to a platform package manager.
>
> The thing is, you may absolutely hate #2, but if approach #1 is
> adopted by git-core, someone can at least attempt this by, well,
> writing a plugin for it.
>
> jon.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 10:55 AM,  <david@lang.hm> wrote:
> On Thu, 28 Apr 2011, Jon Seymour wrote:
>
>> On Thu, Apr 28, 2011 at 10:10 AM, Junio C Hamano <gitster@pobox.com>
>> wrote:
>>>
>>> Jonathan Nieder <jrnieder@gmail.com> writes:
>>>
>>
>> I agree. Apologies for confusing things by talking too much about a
>> git pm install command.
>>
>> I think there are 3 levels of functionality. FWIW, I am suggesting
>> git-core stops at #2.
>>
>> 0. unmanaged plugins
>>
>> git doesn't provide any explicit management of plugins, but will use
>> them if finds them.
>>
>> Without some kind of management, however, you will be forced to dump
>> the man pages and scripts
>> for the plugins in one place.
>>
>> This would be very distribution manager unfriendly since there could
>> be conflicts galore.
>
> every package manager I know of has no problem with multiple packages owning
> files in one directory.
>
> if the files are named the same thing you will have a conflict, but if the
> files are named the same thing, the commands are probably going to be named
> the same, and so you will have conflicts in any case.
>

Suppose a plugin contains a file called LICENSE or README.txt. Which
LICENSE or README.txt wins?

>> I guess an unmanaged solution could use separate directories for each
>> plugin, but this would imply scanning all these paths each time you
>> invoke git. In my view, symbolic links from a dir already
>> GIT_EXEC_PATH to plugin directories would be a more efficient way to
>> do this.
>
> I think you are overanalyzing the problem

I don't think so.  Perhaps Pau can give us his view on the
desirability of a single directory for all plugins artifacts from a
distribution maintainers perspective.

>> 1. managed plugins
>>
>> git provides minimal plugin management functionality. Each plugin has
>> its own directory, but an activate step is required to make the plugin
>> available to the GIT_EXEC_PATH and GIT_MAN_PATH.
>>
>> This has the advantage that conflicts between plugins would be more
>> readily avoided and is potentially more performant. As Pau suggests,
>> this option is much more package manager friendly
>
> I don't see how this will avoid conflicts. what files are you thinking that
> the different plugins will make that won't conflict any more than the
> commands themselves will?

No, but you can manage the conflicts. Hence _managed_ plugins.

   git plugin activate

can fail with a non-zero exit code if there will be a conflict.

>
>> It probably does require a git plugin command of some kind, however,
>> in order to perform the activation step.
>
> only if you think you need a 'installed but not active' mode of operation,
> and I don't understand why you would want that.
>

For the reasons of managed conflict management.

I don't want to drop a new plugin into a common directory only to find
it has blitzed some other plugin I previously did the same with.

Still, I guess it is horses for courses.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

Assuming for the moment that the concept of a managed plugin is accepted, then.

The relationship between distribution managers and the git-plugin
architecture would be as follows:

- distributions would know how to locate the git instance it manages
- distributions would know how to install their own packages that
contain plugins into plugins/ subdirectory of this git instance
- distributions would know how to run git plugin activate and properly
handle non-zero return codes from same

make install scripts would act like a kind of distribution in this regard.

Now consider this:

* suppose that git-core defined a git install _interface contract_ but
did not define an implementation.

Then, a distribution could install its own implementation of the
git-install plugin into git installations it manages.

Then a command like:

    git install gitwork

would trivially work across all distributions precisely because the
distribution has provided the implementation of the git install
interface contract that git-core has helpfully mandated.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 12:15 PM, Jon Seymour <jon.seymour@gmail.com> wrote:
> Then a command like:
>
>    git install gitwork
>
> would trivially work across all distributions precisely because the
> distribution has provided the implementation of the git install
> interface contract that git-core has helpfully mandated.
>

Or better yet, git-core could provide a trivial git install
implementation that selects between different distribution manager
supplied plugins selected according to some heuristic, allowing
several distribution managers to happily manage plugins in the same
git instance.

I have to ask.

Is such an architecture really "absolutely horrid"? Is it  "crap"? Really?

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 3:07 AM, A Large Angry SCM <gitzilla@gmail.com> wrote:
>
> For a git plugin ecosystem to work, a (relatively) stable API/ABI is
> necessary for the plugin authors to code to. Where is your proposal for
> that.
>

To answer your question, the intent is to provide plugins git's
command line interface.

As has already explained by Junio amongst others, git already provides
support for such extensions via its idiom of treating any executable
of the form git-{command} found in the PATH.

The intention of my proposal is simply to formalise the plugin
architecture and provide a degree of plugin management.

Of course, others might think there is a need for ABI plugins, and
they are free to propose such extensions, but that is not my intent.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

Just in case other people haven't noticed...

A sane plugin management architcture also provides a sane way to
manage hooks that might contributed my multiple parties and to provide
a sane way to deal with conflicts between same in a
distribution-agnostic manner.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

The concept of a division of responsibilities between 3 roles seems to
be misunderstood.

There are 3 roles:

   application, plugin and package.

The application provides the plugin manager.
The disribution provides the package manager.
The package manager packages the plugin as a OS-specific package.
The package manager installs the package, containing the plugin.
The package manager calls the application's plugin manager to activate
the plugin

Optionally, the application delegates plugin installation request to
one or more package managers.

Simple, isn't it?

Ruby (and to be honest, npm) got it wrong because they have a plugin
manager that wants to be a package manager.


jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

FWIW: I have renamed my github package from gpm to git-plugin.

Contributors who grok the concept as I conceive it are welcome to
submit pull requests.

jon.

https://github.com/jonseymour/git-plugin

Re: RFC: a plugin architecture for git extensions?

[Pau Garcia i Quiles]

On Thu, Apr 28, 2011 at 4:08 AM, Jon Seymour <jon.seymour@gmail.com> wrote:

>>> I guess an unmanaged solution could use separate directories for each
>>> plugin, but this would imply scanning all these paths each time you
>>> invoke git. In my view, symbolic links from a dir already
>>> GIT_EXEC_PATH to plugin directories would be a more efficient way to
>>> do this.
>>
>> I think you are overanalyzing the problem
>
> I don't think so.  Perhaps Pau can give us his view on the
> desirability of a single directory for all plugins artifacts from a
> distribution maintainers perspective.

You are thinking too much, this is simpler than what you are trying to do :-)

What packages in Linux distributions do is essentially placing files
where the FHS says, i. e:

- Binaries intended to be used by an average Joe go to /usr/bin
- Binaries for internal consumption go to /usr/lib/packagename
- Libraries go to /usr/lib
- Documentation goes to /usr/share/doc/packagename
- Manual pages to go /usr/share/man/manX/command.X.gz
- etc

Define something like that inside the prefix where git is installed
and you are done. As Junio said, git already checks for git-* presence
for help, etc.

If you want to see more about package organization, go to
http://www.debian.org/Packages/unstable/ , go to that package's page
and at the bottom there is a "List of files" link. For instance

http://packages.debian.org/experimental/git
http://packages.debian.org/experimental/amd64/git/filelist

BTW, I fail to see why an "activation" step is needed. Either it is
installed or it is not.

-- 
Pau Garcia i Quiles
http://www.elpauer.org
(Due to my workload, I may need 10 days to answer)

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 5:40 PM, Pau Garcia i Quiles
<pgquiles@elpauer.org> wrote:
> On Thu, Apr 28, 2011 at 4:08 AM, Jon Seymour <jon.seymour@gmail.com> wrote:
>
>>>> I guess an unmanaged solution could use separate directories for each
>>>> plugin, but this would imply scanning all these paths each time you
>>>> invoke git. In my view, symbolic links from a dir already
>>>> GIT_EXEC_PATH to plugin directories would be a more efficient way to
>>>> do this.
>>>
>>> I think you are overanalyzing the problem
>>
>> I don't think so.  Perhaps Pau can give us his view on the
>> desirability of a single directory for all plugins artifacts from a
>> distribution maintainers perspective.
>
> You are thinking too much, this is simpler than what you are trying to do :-)
>
> What packages in Linux distributions do is essentially placing files
> where the FHS says, i. e:
>
> - Binaries intended to be used by an average Joe go to /usr/bin
> - Binaries for internal consumption go to /usr/lib/packagename
> - Libraries go to /usr/lib
> - Documentation goes to /usr/share/doc/packagename
> - Manual pages to go /usr/share/man/manX/command.X.gz
> - etc
>
> Define something like that inside the prefix where git is installed
> and you are done. As Junio said, git already checks for git-* presence
> for help, etc.
>
> If you want to see more about package organization, go to
> http://www.debian.org/Packages/unstable/ , go to that package's page
> and at the bottom there is a "List of files" link. For instance
>
> http://packages.debian.org/experimental/git
> http://packages.debian.org/experimental/amd64/git/filelist
>
> BTW, I fail to see why an "activation" step is needed. Either it is
> installed or it is not.
>
> --
> Pau Garcia i Quiles
> http://www.elpauer.org
> (Due to my workload, I may need 10 days to answer)
>

Ok, I have tried to explain why separating the concerns of package
management and plugin management is an appropriate thing to do, and
why one directory for each plugin is also a good thing to do. BTW: I
thought you actually suggested this concept yourself in your earlier
post.

I also explained why an activation step is desirable. It is good for
performance (since you don't have to resolve across plugin directories
on each invocation) and it is good for conflict management (since you
get a precise means to identify and prevent conflicts). More over, the
application which provides the plugin manager gets to do this conflict
detection and prevention, which is exactly the right component to do
it. Why? because it understands its own plugin architecture better
than any and all plugin authors, package authors or package managers.

There is probably only so many times I can explain these principles.

As they saying goes - code talks. I'll implement things as I think
they should be implemented, incorporating the constructive suggestions
that have been made.

To be sure, even the insults have helped.

Calling the idea of:

    git install foobar

"horrid" and then "utterly horrid" and then "crap" has just helped
crystalise in my mind exactly where the correct lines of
responsibility should be drawn between, application, plugin and
package.

It appears that many people still don't get it but there is only so
much I can do about that.

Anyway, thanks for all your help (constructive and otherwise),

jon.

Re: RFC: a plugin architecture for git extensions?

[Andreas Ericsson]

On 04/28/2011 12:32 AM, Jon Seymour wrote:
> On Thu, Apr 28, 2011 at 8:08 AM, Jonathan Nieder<jrnieder@gmail.com>  wrote:
>> Junio C Hamano wrote:
> 
>> Or is the idea to blindly install (a symlink to) git-work to $(git
>> --exec-path)/ rather than a place on the $PATH?  In this case, I would
>> be a little worried.  How will the helper deal with uninstallation and
>> with namespace conflicts?  (On the $PATH, these are expected problems
>> and I'd expect each user has some way of dealing with them already.)
> 
> I see 'git pm activate' managing symbolic links in a directory
> dedicated to the purpose (.e.g. ~/.git-pm/activated).
> 

Why not just flip executable bits on commands in the git directory?
That way, built-ins can't be disabled no matter what you type, and the
list of out-of-core helpers will be kept shorter.

Either way, the "activate" thing requires you to maintain state of
downloaded extensions that aren't part of the core package. I'd say
it's safe to assume that anything downloaded is supposed to be active
and deactivating it is done by means of uninstalling it.

> One thing 'git pm activate' could do is check that the commands
> exported by the 'gitwork' descriptor do not conflict with what is
> already activated.
> 
> If the user has done something like:
> 
>      git clone https://repo/gitwork.git ~/hub/gitwork
> 
> and then:
> 
>      git activate pm ~/hub/gitwork
> 
> the symbolic links would be established to ~/hub/gitwork, wherever
> that happens to be.
> 
> If the user has done:
> 
>     apt-get install gitwork
> 
> then given a package-manager adapter for apt-get, it could extract the
> .gpm file from the list of installed files, and resume activation from
> there. Ultimately, the end result is the same ~/.git-pm/activated is
> updated, it has always been on the paths it needs to be.
> 
> If the descriptor did have a list of exported commands (e.g. git-work,
> git-base, git-atomic, git-test), then a global registry could use this
> list of exported commands to detect conflicts early - at package
> registration time which might help avoid grief down the track.

But seriously. Don't you see how complex this is? If you want to make
things easier for people, put up a website or a database somewhere and
ship a (very, very small) addon with core git that just goes to fetch
that list and present it to users upon request. The list should contain
3 fields and 3 fields only.
1. Short name of extension (must be unique)
2. Url for where to get the extension
3. Free-form text, containing info of why anyone would want to use it,
   preferrably in the style of git.git commit messages so the brief
   description is the first line and the full info comes later.

Then your extension manager becomes really straightforward and can
easily be extended over time to also uninstall applications.

This assumes that git core itself can provide paths for where the
user wants man-pages and whatnot, ofcourse, but such a patch is so
trivial anyone who's seen code in any language can easily hack it
up blindfolded. Even for versions that doesn't support it, the man-
pages of any extension can be guessed from the --exec-path or just
ignored and all of them put into /usr/share/man or whatever the path
might be on the system we're on, so long as the extension manager
remembers where they ended up. This could easily be handled by the
extension itself creating a manifest file after it's done installing
which is stashed away by the extension manager.

A decent first step would be to have "git pm list" just show
what extensions are available, what they do and where to get them.

However, that info could just as easily be distributed to users by
letting "git help" or something print a link to a wiki page and by
letting extension devs know about some officially blessed way of
installing and uninstalling extensions properly. Once all extensions
use that same way and there are more than a small handful of
extensions, the idea of having a script download and automagically
install those extensions will probably be opened up again, but by
then with more than a single user backing it up with a request that
it be implemented (without showing any code).

Shipping a bunch of extensions with git core or making distro package
maintainers ship them as deactivated by default makes absolutely no
sense what so ever. Shipping a (very small) extension manager with
git that aids users obtaining more extensions makes sense when there's
a proper flora of them rather than just a few.

Either way, it's time to put your money where your mouth is and show
us some code that does what you think is a good idea.

If I were you, I'd start with the blindfolded patch mentioned above
and then follow up with a configurable directory where git will
search for additional commands, with a sensible default in the
user's home directory (.git-plugins, perhaps?).

Let conflicts go hang for now. They will be thoroughly frowned upon
by the community anyway, since providing any sort of help for the
command "git fniggle" will be totally impossible if there are 100
different implementations of it.
 
With those two patches in place in core git (which I'm fairly sure
Junio won't reject out of hand given his opinions in this thread),
coding up the "make install" part of extensions become trivial and
starting on the real extension manager becomes at least possible for
both types of core git users, namely:
1. Those who get core git from yum/brew/apt-get
2. Those who build it from sources.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.

Re: RFC: a plugin architecture for git extensions?

[Pau Garcia i Quiles]

On Thu, Apr 28, 2011 at 10:09 AM, Jon Seymour <jon.seymour@gmail.com> wrote:

> Ok, I have tried to explain why separating the concerns of package
> management and plugin management is an appropriate thing to do, and
> why one directory for each plugin is also a good thing to do. BTW: I
> thought you actually suggested this concept yourself in your earlier
> post.

Please, re-read my mails. I *am* suggesting that plugins store data in
different directories!

- The "main command" (git-atomic, for instance) would be stored in
GIT_PREFIX/lib/git-plugins (instead of GIT_PREFIX/lib/git-core, which
is where git stores its commands). Git would have to learn to search
in GIT_PREFIX/lib/git-plugins in addition to git-core, of course.

- Porcelain for git-atomic would go to
GIT_PREFIX/lib/git-plugins/git-atomic (or something like this)

- The documentation would be stored in GIT_PREFIX/doc/git-atomic

- Resources in GIT_PREFIX/share/git-atomic

- etc

I. e. each plugin stores its stuff in a separate directory, it's just
that directory is not a *single* directory but *many* directories,
each one inside the proper path under GIT_PREFIX, just like it is for
anything you install on a Unix system

-- 
Pau Garcia i Quiles
http://www.elpauer.org
(Due to my workload, I may need 10 days to answer)

Re: RFC: a plugin architecture for git extensions?

[Andreas Ericsson]

On 04/28/2011 04:15 AM, Jon Seymour wrote:
> Assuming for the moment that the concept of a managed plugin is accepted, then.
> 
> The relationship between distribution managers and the git-plugin
> architecture would be as follows:
> 
> - distributions would know how to locate the git instance it manages
> - distributions would know how to install their own packages that
> contain plugins into plugins/ subdirectory of this git instance
> - distributions would know how to run git plugin activate and properly
> handle non-zero return codes from same
> 
> make install scripts would act like a kind of distribution in this regard.
> 
> Now consider this:
> 
> * suppose that git-core defined a git install _interface contract_ but
> did not define an implementation.
> 

Please. I'm already on my way to a seriously boring sales meeting without
having developers throw garbage terms on me. You've done a lot of that in
this thread and I for one am confused by them as to what you want to
achieve and how you want to achieve it.

> Then, a distribution could install its own implementation of the
> git-install plugin into git installations it manages.
> 
> Then a command like:
> 
>      git install gitwork
> 
> would trivially work across all distributions precisely because the
> distribution has provided the implementation of the git install
> interface contract that git-core has helpfully mandated.
> 

And so we force package maintainers to become git extension developers.
Brilliant. They'll love you for it.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.

Re: RFC: a plugin architecture for git extensions?

[Andreas Ericsson]

On 04/28/2011 04:49 AM, Jon Seymour wrote:
> On Thu, Apr 28, 2011 at 12:15 PM, Jon Seymour<jon.seymour@gmail.com>  wrote:
>> Then a command like:
>>
>>     git install gitwork
>>
>> would trivially work across all distributions precisely because the
>> distribution has provided the implementation of the git install
>> interface contract that git-core has helpfully mandated.
>>
> 
> Or better yet, git-core could provide a trivial git install
> implementation that selects between different distribution manager
> supplied plugins selected according to some heuristic, allowing
> several distribution managers to happily manage plugins in the same
> git instance.
> 
> I have to ask.
> 
> Is such an architecture really "absolutely horrid"? Is it  "crap"? Really?
> 

Yes, because it forces all distributions to name their extensions
the same and it forces all distributions to carry the same extensions.
It also makes it impossible to support 3rd party extensions that core
git doesn't know about and that aren't already packaged, unless you
want the "git install" command to have knowledge about all package
management systems and *very, very good* heuristics when guessing what
a particular extension is called on that system.

What will happen though is that the distributions will happily ignore
that and the "git install" command will fail for some extensions on
some distributions.

Besides that, it forces users to install git from their distribution
packages so we're hard-shafting the git developers who usually have
at least some installations done from source.

We just went far beyond "absolutely horrid" and into the realms of
"steaming pile of abominably manure-like ideas whose inventor should
be slapped silly for their own good".

So let's get back to the basic wish you have here. You want people
to be able to easily find, download and install "git work" so that it
works nicely with man-pages and all.

The wiki-page with known extensions and the patch to core git so that
"make install" can put man-pages in the right directory are the first,
simplest and smallest steps that takes us the farthest towards that
wish without burdening people you have no control over with more labor.
In short; It's both good engineering and polite to implement that and
then consider what new possibilities open up and see what people do
with those possibilities. You might be surprised.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thursday, April 28, 2011, Pau Garcia i Quiles <pgquiles@elpauer.org> wrote:
> On Thu, Apr 28, 2011 at 10:09 AM, Jon Seymour <jon.seymour@gmail.com> wrote:
>
>> Ok, I have tried to explain why separating the concerns of package
>> management and plugin management is an appropriate thing to do, and
>> why one directory for each plugin is also a good thing to do. BTW: I
>> thought you actually suggested this concept yourself in your earlier
>> post.
>
> Please, re-read my mails. I *am* suggesting that plugins store data in
> different directories!

Good. Sorry I explained why an activation step is good for
performance, particularly if you have a plugin architecture that uses
multiple directories, one per plugin. I misinterpreted your confusion
about why an activate step was necessary as an explicit rejection of
my arguments that an activation step was a plus for performance and
conflict detection in an environment in which each plugin is installed
in it's own directory. Apologies.

Supposed you have 100 plugins. Suppose the user invoked git foobar
where foobar is an installed plugin. How does git find the plugin to
invoke? Why it has to scan 100 directories looking for a match.

If you use an activation step (just after installation) you get O(1)
performance instead of O(n). Small gain for 2 plugins, I agree.

But then that isn't the only reason to use an activation step. As I
said, an activation step alllows precise detection and prevention of
conflicts that just isn't possible if you rely on first-in-PATH
semantics.

It is true, you can delegate conflict detection to the package
manager, but do you know what, that would be in Andreas"s words
"Brilliant" as it would force package maintainers become git extension
experts.

My proposal makes the git-core maintainerrthathe experts on git
extension management and relieves plugin authors, package authors and
package-manager authors of this responsibility.

I ask. is that not a very good thing?

>
> - The "main command" (git-atomic, for instance) would be stored in
> GIT_PREFIX/lib/git-plugins (instead of GIT_PREFIX/lib/git-core, which
> is where git stores its commands). Git would have to learn to search
> in GIT_PREFIX/lib/git-plugins in addition to git-core, of course.
>
> - Porcelain for git-atomic would go to
> GIT_PREFIX/lib/git-plugins/git-atomic (or something like this)
>
> - The documentation would be stored in GIT_PREFIX/doc/git-atomic
>
> - Resources in GIT_PREFIX/share/git-atomic
>
> - etc
>
> I. e. each plugin stores its stuff in a separate directory, it's just
> that directory is not a *single* directory but *many* directories,
> each one inside the proper path under GIT_PREFIX, just like it is for
> anything you install on a Unix system
>
> --
> Pau Garcia i Quiles
> http://www.elpauer.org
> (Due to my workload, I may need 10 days to answer)
>

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 7:11 PM, Andreas Ericsson <ae@op5.se> wrote:
> On 04/28/2011 04:15 AM, Jon Seymour wrote:
>>
>> * suppose that git-core defined a git install _interface contract_ but
>> did not define an implementation.
>>
>
> Please. I'm already on my way to a seriously boring sales meeting without
> having developers throw garbage terms on me. You've done a lot of that in
> this thread and I for one am confused by them as to what you want to
> achieve and how you want to achieve it.
>

I am confused. Is "interface contract" a "garbage term" for you?

As I understand the term, it is a good way to enable strong separation
of concerns and information hiding and these techniques are both
valuable tools in the battle against complexity, a battle that I
understand from other posts this in thread you are vitally engaged in.

My apologies, if terms such as "separation of concerns" and
"information hiding" are viscerally offensive to you. I must learn to
deploy phrases such as "horrid" and "utterly horrid" with more panache
and aplomb.

>> Then, a distribution could install its own implementation of the
>> git-install plugin into git installations it manages.
>>
>> Then a command like:
>>
>>      git install gitwork
>>
>> would trivially work across all distributions precisely because the
>> distribution has provided the implementation of the git install
>> interface contract that git-core has helpfully mandated.
>>
>
> And so we force package maintainers to become git extension developers.
> Brilliant. They'll love you for it.

This exactly the wrong way around, but I can understand your mistake
since I used this phrase:

- distributions would know how to run git plugin activate and properly
 handle non-zero return codes from same

when precision dictates that I should have used the more wordy, but
more accurate:

- distributions would know how to invoke plugin installation scripts
provided by the plugin author, and these plugin installation scripts
would, by virtue of being intimately aware of the public interfaces
(oops, there is that word again) of git, know how to invoke the git
plugin command in order to activate their own plugin.

I have explained in my most recent reply to Pau that in the absence of
a plugin management script provided by the git core, any hope of
plugin conflict management has to be delegated to other parties such
as the package manager author, the package  author or the plugin
author or, alas, to the poor user.

I fail to see why management of plugin conflicts is better handled by
N authors and M users, rather than a single author of the git-plugin
script [ except for the minor detail that the only potential author of
such a script seems to spend all his time writing e-mails instead of
cutting code ].

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 7:25 PM, Andreas Ericsson <ae@op5.se> wrote:
> On 04/28/2011 04:49 AM, Jon Seymour wrote:
>> On Thu, Apr 28, 2011 at 12:15 PM, Jon Seymour<jon.seymour@gmail.com>  wrote:
>>> Then a command like:
>>>
>>>     git install gitwork
>>>
>>> would trivially work across all distributions precisely because the
>>> distribution has provided the implementation of the git install
>>> interface contract that git-core has helpfully mandated.
>>>
>>
>> Or better yet, git-core could provide a trivial git install
>> implementation that selects between different distribution manager
>> supplied plugins selected according to some heuristic, allowing
>> several distribution managers to happily manage plugins in the same
>> git instance.
>>
>> I have to ask.
>>
>> Is such an architecture really "absolutely horrid"? Is it  "crap"? Really?
>>
>
> Yes, because it forces all distributions to name their extensions
> the same and it forces all distributions to carry the same extensions.

False.

Distributions maintain a N:1 registry of git plugin names to
distribution specific package names. This registry would most
naturally be maintained by the author who maintains the git package,
but of course, such responsibilities could be delegated to others if
required.

During installation, the package manage aware "install" plugin
consults such a registry to determine which packages need to be
installed in order to obtain the requested plugin. It then asks the
package manager to install said packages.

> It also makes it impossible to support 3rd party extensions that core
> git doesn't know about and that aren't already packaged, unless you
> want the "git install" command to have knowledge about all package
> management systems and *very, very good* heuristics when guessing what
> a particular extension is called on that system.
>

Again false. That is the role of the registry that is maintained by
some package author.

> What will happen though is that the distributions will happily ignore
> that and the "git install" command will fail for some extensions on
> some distributions.

Yes, but it can fail in a extremely precise way. In those cases, it
could fall back another package manager
such as one that is capable of checking out a git repo and running make install.

I realise that most people could not accomplish such a task without
writing vast amounts of spaghetti code, but there you go.

>
> Besides that, it forces users to install git from their distribution
> packages so we're hard-shafting the git developers who usually have
> at least some installations done from source.
>

No it doesn't. My suggestion allows deploy by drop-in configuration to
a subdirectory of plugins/ followed by the invocation of a command
such as.

  git plugin activate foobar

I realise this may "hard-shaft" the git developers who are unable to
invoke a git command, but I guess they could also put such a command
in their make install scripts. Now there's an idea.

> We just went far beyond "absolutely horrid" and into the realms of
> "steaming pile of abominably manure-like ideas whose inventor should
> be slapped silly for their own good".
>

Thanks, I'll add that to my vocabulary of terms that git developers
use when they let fly to their emotions for completely unaccountable
reasons. I'd suggest,. perhaps "grow up", but that would strike some
as rude. So, I won't.

> So let's get back to the basic wish you have here. You want people
> to be able to easily find, download and install "git work" so that it
> works nicely with man-pages and all.
>

What is so  hard about:

  app install plugin.

Forget git. Forget git work.

What is so hard about the concept of an application providing a
facility that allows it to request, merely request, the installation
of a plugin for itself by what ever happens to be the users choice of
package manager or distribution.

Is such a concept really, fundamentally flawed?

if so, replace "git" with "linux", "app" with "apt-get", "plugin" with
"git-core" and explain to me why

apt-get install git-core

is such a bad idea.

Yes, different levels of abstraction, but the principles are the same.

Like it or not, git is a platform, there is absolutely no reason why
it can't have sane plugin manager, other than complete lack of
imagination.


> The wiki-page with known extensions and the patch to core git so that
> "make install" can put man-pages in the right directory are the first,
> simplest and smallest steps that takes us the farthest towards that
> wish without burdening people you have no control over with more labor.
> In short; It's both good engineering and polite to implement that and
> then consider what new possibilities open up and see what people do
> with those possibilities. You might be surprised.

Yep, that's what unix package management was like before people
invented the idea of package managers.

How quaint.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jonathan Nieder]

Hi,

Jon Seymour wrote:

> What is so  hard about:
>
>   app install plugin.
>
> Forget git. Forget git work.
>
> What is so hard about the concept of
[...]

Is it so important that the people on this list agree with you
right away that your proposal is a great idea?

You asked for comments, so people are giving them.  But typically,
the procedure might be something like:

1. person describes a problem they are solving, and perhaps a
   proposed solution

2. people on the mailing list try to help out in figuring out the
   parameters of the problem, giving relevant background about the
   tools available, and describing experiences they have had in the
   past that might help in solving it.

3. person reports on how it panned out.  If this involved changing
   git or is nicely documented, we might get a patch out of it.  If
   so:

4. people on the mailing list review the patch, considering how
   useful it might be for their own nefarious ends, potential
   downsides, and how maintainable it is.

5. at some point, some patches appear that seem to be a good idea,
   and Junio applies them.

And so forth.  Notice how at no step everyone needs to agree.

Earlier in this thread you mentioned that git needs some basic
facilities to make a prototype of a command installer possible.
But git already has those facilities.  Even if the installed commands
should not be on the path, all you need to get a "git install"
available on early adopters' machines is to put an executable at
/usr/bin/git-install or ~/bin/git-install.

So you can show people what they have missing, and once they can't
live without it, maybe they'll clamor for its inclusion in git.git
proper and distro packages.  If that is your goal, getting something
working and helping interested people to try it out (and adding it
to https://git.wiki.kernel.org/index.php/InterfacesFrontendsAndTools
when ready) comes first.

Sorry to spend so long on abstractions.  Still, hope that helps.

Regards,
Jonathan

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Thu, Apr 28, 2011 at 9:11 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Hi,
>
> Jon Seymour wrote:
>
>> What is so  hard about:
>>
>>   app install plugin.
>>
>> Forget git. Forget git work.
>>
>> What is so hard about the concept of
> [...]
>
> Is it so important that the people on this list agree with you
> right away that your proposal is a great idea?
>
> You asked for comments, so people are giving them.  But typically,
> the procedure might be something like:
>
> 1. person describes a problem they are solving, and perhaps a
>   proposed solution
>
> 2. people on the mailing list try to help out in figuring out the
>   parameters of the problem, giving relevant background about the
>   tools available, and describing experiences they have had in the
>   past that might help in solving it.
>
> 3. person reports on how it panned out.  If this involved changing
>   git or is nicely documented, we might get a patch out of it.  If
>   so:
>
> 4. people on the mailing list review the patch, considering how
>   useful it might be for their own nefarious ends, potential
>   downsides, and how maintainable it is.
>
> 5. at some point, some patches appear that seem to be a good idea,
>   and Junio applies them.
>
> And so forth.  Notice how at no step everyone needs to agree.
>
> Earlier in this thread you mentioned that git needs some basic
> facilities to make a prototype of a command installer possible.
> But git already has those facilities.  Even if the installed commands
> should not be on the path, all you need to get a "git install"
> available on early adopters' machines is to put an executable at
> /usr/bin/git-install or ~/bin/git-install.
>
> So you can show people what they have missing, and once they can't
> live without it, maybe they'll clamor for its inclusion in git.git
> proper and distro packages.  If that is your goal, getting something
> working and helping interested people to try it out (and adding it
> to https://git.wiki.kernel.org/index.php/InterfacesFrontendsAndTools
> when ready) comes first.
>
> Sorry to spend so long on abstractions.  Still, hope that helps.
>
> Regards,
> Jonathan
>

Agreed. I'll get coding. Thanks for the advice.

jon.

Re: RFC: a plugin architecture for git extensions?

[david]

On Thu, 28 Apr 2011, Jon Seymour wrote:

> On Thu, Apr 28, 2011 at 3:07 AM, A Large Angry SCM <gitzilla@gmail.com> wrote:
>>
>> For a git plugin ecosystem to work, a (relatively) stable API/ABI is
>> necessary for the plugin authors to code to. Where is your proposal for
>> that.
>>
>
> To answer your question, the intent is to provide plugins git's
> command line interface.
>
> As has already explained by Junio amongst others, git already provides
> support for such extensions via its idiom of treating any executable
> of the form git-{command} found in the PATH.
>
> The intention of my proposal is simply to formalise the plugin
> architecture and provide a degree of plugin management.

start with documenting (and therefor formalizing) how plugins can work 
today. Then propose your change to how they work, what the benifits of 
your change are, and the code needed to implement your change.

David Lang

> Of course, others might think there is a need for ABI plugins, and
> they are free to propose such extensions, but that is not my intent.
>
> jon.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

<david@lang.hm> wrote:
> On Thu, 28 Apr 2011, Jon Seymour
> start with documenting (and therefor formalizing) how plugins can work today. Then propose your change to how they work, what the benifits of your change are, and the code needed to implement your change.
>


Good suggestion, thank you,

jon

Re: RFC: a plugin architecture for git extensions?

[David Aguilar]

On Apr 28, 2011, at 3:56 AM, Jon Seymour <jon.seymour@gmail.com> wrote:
> What is so  hard about:
> 
>  app install plugin.
> 
> Forget git. Forget git work.
> 
> What is so hard about the concept of an application providing a
> facility that allows it to request, merely request, the installation
> of a plugin for itself by what ever happens to be the users choice of
> package manager or distribution.

It's not hard.  We simply don't need it. 

Why do I need to activate my "plugin"?  That seems like a needless feature. If I don't want "git gui" I apt-get uninstall git-gui.

I've read this thread and do not understand what problem this is trying to solve. I have personally gotten along just fine writing git applications and deploying them with existing package managers.  I've not once ran into any situation where I felt I needed more support from git in order to deploy git commands.

Users already know about their package manager. Why do we need them to learn about Yet Another system?


> Is such a concept really, fundamentally flawed?

Yes

> if so, replace "git" with "linux", "app" with "apt-get", "plugin" with
> "git-core" and explain to me why
> 
> apt-get install git-core
> 
> is such a bad idea.
> 
> Yes, different levels of abstraction, but the principles are the same.
> 
> Like it or not, git is a platform, there is absolutely no reason why
> it can't have sane plugin manager, other than complete lack of
> imagination.

It doesn't need one. I'm happy with apt, yum, etc.  I probably don't understand why I need to learn more, but ignorance is bliss and users like to be blissful.

Let's assume we did have this system for a minute. Now we are worse off because many git apps do not and will never use the new plugin system.

Are we going to one day remove the awesome "search for git-foo in path" behavior and break everyone?  I think not.  In other words, I see little incentive for existing, established git apps to use such a system.

Sorry if I seem like a hard-liner, but we should always strive to keep things as simple as possible. I don't see any downsides to the current situation which is why I would resist such a proposal.  The upside would have to be pretty significant to convince established git apps to switch.
-- 
                                        David

Re: RFC: a plugin architecture for git extensions?

[Junio C Hamano]

David Aguilar <davvid@gmail.com> writes:

> On Apr 28, 2011, at 3:56 AM, Jon Seymour <jon.seymour@gmail.com> wrote:
> 
>> What is so hard about the concept of an application providing a
>> facility that allows it to request, merely request, the installation
>> of a plugin for itself by what ever happens to be the users choice of
>> package manager or distribution.
>
> It's not hard.  We simply don't need it. 
>
> Why do I need to activate my "plugin"?  That seems like a needless
> feature. If I don't want "git gui" I apt-get uninstall git-gui.

I mostly agree with you that what Jon has wrote so far is overengineering
to solve a problem that does not exist [*1*].

But here is one thought.

Imagine this "git gui" is not "git gui" but "git superadd" package that
changes the behaviour of "git add" slightly.

    Side note: Of course, for this kind of usage, the "git potty" needs to
    be extended to look for things in different places, and also it needs
    to be made easy for the implementation of "superadd" to call the
    underlying "git add", bypassing itself, when necessary.

You do not want that new interface, you are old timer and you like the old
way of doing things like me ;-).  But your wife wants to use it.  You two
share a computer.

Do you or do you not run "apt-get install git-superadd"?

One possible answer may be to run "apt-get install git-superadd", and then
the users who want "git add" to behave in a new way to opt-in to use the
"plug-in".  I think that is what Jon is getting at.


[Footnote]

*1* I admit that I didn't read all of them carefully, as I was repelled by
them as soon as I saw phrases like "for people who can grok this concept".

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

| Junio: apologies, reply missed list. Edited slightly.

On Fri, May 6, 2011 at 7:53 AM, Junio C Hamano <gitster@pobox.com> wrote:
> David Aguilar <davvid@gmail.com> writes:
>
>> On Apr 28, 2011, at 3:56 AM, Jon Seymour <jon.seymour@gmail.com> wrote:
>>
>>> What is so hard about the concept of an application providing a
>>> facility that allows it to request, merely request, the installation
>>> of a plugin for itself by what ever happens to be the users choice of
>>> package manager or distribution.
>>
>> It's not hard.  We simply don't need it.
>>
>> Why do I need to activate my "plugin"?  That seems like a needless
>> feature. If I don't want "git gui" I apt-get uninstall git-gui.
>
> I mostly agree with you that what Jon has wrote so far is overengineering
> to solve a problem that does not exist [*1*].
>

I do accept that the consensus of the list is that any kind of package
or plugin management is over-engineering.

> But here is one thought.
>
> Imagine this "git gui" is not "git gui" but "git superadd" package that
> changes the behaviour of "git add" slightly.
>
>    Side note: Of course, for this kind of usage, the "git potty" needs to
>    be extended to look for things in different places, and also it needs
>    to be made easy for the implementation of "superadd" to call the
>    underlying "git add", bypassing itself, when necessary.
>
> You do not want that new interface, you are old timer and you like the old
> way of doing things like me ;-).  But your wife wants to use it.  You two
> share a computer.
>
> Do you or do you not run "apt-get install git-superadd"?
>
> One possible answer may be to run "apt-get install git-superadd", and then
> the users who want "git add" to behave in a new way to opt-in to use the
> "plug-in".  I think that is what Jon is getting at.
>

Yes, the basic premise is that an author of a git extension, which
depends only on git, should be able to provide an extension in a form
that allows others to drop the extension onto a system, knowing only
that the system has git installed.

The extension author should not need to know about how to build an
apt-get, yum, cygwin or brew package or too much about how to install
into a git installation that wasn't installed by a distro. Why should
they? Their dependency is simply git.

It would be good if something like:

    unzip -d $(git --plugins-dir) foobar.zip

installed scripts, info files and man pages into a place where git
would find them and then have git foobar start working without any
additional effort by the package author or user.

I accept that anything more sophisticated than "drop and go" style
deployment of extensions (e.g. plugin management) is overkill, given
that the current market for git extensions is miniscule. That said, I
would like to get the basic extension mechanism right so that if the
"market" for git plugins ever became wildly successful, then at least
we could implement more sophisticated plugin management if we wanted
to.

This was the point of the other thread: "RFC: a minimal plugin
architecture" where I tried to elicit feedback about what an minimal
solution would look like, sans my more elaborate visions.

http://permalink.gmane.org/gmane.comp.version-control.git/172419

>
> [Footnote]
>
> *1* I admit that I didn't read all of them carefully, as I was repelled by
> them as soon as I saw phrases like "for people who can grok this concept".

Junio: at least quote me accurately. I actually wrote:

"Contributors who grok the concept as I conceive it are welcome to
submit pull requests."

I am a little mystified why use of the word "grok" in this way, given the
circumstances, caused you to be "repelled".

Re: RFC: a plugin architecture for git extensions?

[Junio C Hamano]

Jon Seymour <jon.seymour@gmail.com> writes:

> | Junio: apologies, reply missed list. Edited slightly.
>
> On Fri, May 6, 2011 at 7:53 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> David Aguilar <davvid@gmail.com> writes:
>>
>>> On Apr 28, 2011, at 3:56 AM, Jon Seymour <jon.seymour@gmail.com> wrote:
>>>
>>>> What is so hard about the concept of an application providing a
>>>> ...
>
> It would be good if something like:
>
>     unzip -d $(git --plugins-dir) foobar.zip
>
> installed scripts, info files and man pages into a place where git
> would find them and then have git foobar start working without any
> additional effort by the package author or user.

That should be doable without any elaborate "plugin management".  We only
need to enhance the git potty and help system to look for things in a set
of new directories, and have these unzip installation put their stuff
there.

>> [Footnote]
>>
>> *1* I admit that I didn't read all of them carefully, as I was repelled by
>> them as soon as I saw phrases like "for people who can grok this concept".
>
> Junio: at least quote me accurately. I actually wrote:
>
> "Contributors who grok the concept as I conceive it are welcome to
> submit pull requests."
>
> I am a little mystified why use of the word "grok" in this way, given the
> circumstances, caused you to be "repelled".

What repelled me was not any particular word (and that is exactly why I
did not even bother to "quote") but your general attitude in the
discussion.  Your tone throughout the discussion appeared at least to me
that you thought anybody who did not agree with you was incapable of
understanding something so obvious and clearly right, and only those who
could "grok" it deserved to join the discussion.  We occasionally have
seen such people on this list in the past, but luckily not very often.

I am not saying that your thinking should always start from "I could be
wrong".  However, I do not think "What is so hard about the concept ...?"
is the question you should be asking others before asking yourself "Is
there a better way I could have explained this idea I want others to agree
with?  The reason why they are still not on the same page as I am could
well be because all of them are morons, but it may be possible that it is
because the way I have explained my idea to them was not optimal.  Perhaps
I did not present the motivation and the background well enough to justify
that the problem I am trying to solve is worth solving, before throwing my
idea on how to solve it".

After all, different people have different needs and expectations.  The
discussion on a particular _solution_ can only start after you get people
on board and share the sense of _need_ to solve something common.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Fri, May 6, 2011 at 2:47 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jon Seymour <jon.seymour@gmail.com> writes:
>> It would be good if something like:
>>
>>     unzip -d $(git --plugins-dir) foobar.zip
>>
>> installed scripts, info files and man pages into a place where git
>> would find them and then have git foobar start working without any
>> additional effort by the package author or user.

> That should be doable without any elaborate "plugin management".  We only
> need to enhance the git potty and help system to look for things in a set
> of new directories, and have these unzip installation put their stuff
> there.

I do agree - plugin management is superfluous to current requirements.

>>>
>>> *1* I admit that I didn't read all of them carefully, as I was repelled by
>>> them as soon as I saw phrases like "for people who can grok this concept".
>>
>> Junio: at least quote me accurately. I actually wrote:
>>
>> "Contributors who grok the concept as I conceive it are welcome to
>> submit pull requests."
>>
>> I am a little mystified why use of the word "grok" in this way, given the
>> circumstances, caused you to be "repelled".
>
> What repelled me was not any particular word (and that is exactly why I
> did not even bother to "quote") but your general attitude in the
> discussion.  Your tone throughout the discussion appeared at least to me
> that you thought anybody who did not agree with you was incapable of
> understanding something so obvious and clearly right, and only those who
> could "grok" it deserved to join the discussion.  We occasionally have
> seen such people on this list in the past, but luckily not very often.
>
> I am not saying that your thinking should always start from "I could be
> wrong".  However, I do not think "What is so hard about the concept ...?"
> is the question you should be asking others before asking yourself "Is
> there a better way I could have explained this idea I want others to agree
> with?  The reason why they are still not on the same page as I am could
> well be because all of them are morons, but it may be possible that it is
> because the way I have explained my idea to them was not optimal.  Perhaps
> I did not present the motivation and the background well enough to justify
> that the problem I am trying to solve is worth solving, before throwing my
> idea on how to solve it".
>
> After all, different people have different needs and expectations.  The
> discussion on a particular _solution_ can only start after you get people
> on board and share the sense of _need_ to solve something common.
>

Ok, well thanks for taking the time to explain what you really meant.

I agree that I have come across as arrogant; I reacted badly to what I
considered to be insults being heaped in my direction by some.
However, I did myself and my ideas no favors by treating such
criticism with the public disdain that I did.

I also agree that I did a lousy job explaining the concepts let alone
convincing others of their merits.

Finally, I agree that it would be more productive for me to be more
sensitive to what is agreed to be a common need and to try to restrict
my proposed solutions to exactly those needs.

I would appreciate any feedback you (or others) have about:

    http://permalink.gmane.org/gmane.comp.version-control.git/172419

In particular, I would be interested in feedback about how to best support:

- multiple extensions - do we want support installing extensions in
their own directories, per Pau's suggestion or simply allow them to
write to a common directory?
- multiple extension directories - how to support Jonathan's
requirement to allow user specific extension directories?

I have some ideas about how to do this which I will propose in a patch
over the next few weeks, but any input I have now would be useful.

Regards,

jon.

Re: RFC: a plugin architecture for git extensions?

[Jonathan Nieder]

Jon Seymour wrote:

> I would appreciate any feedback you (or others) have about:
>
>     http://permalink.gmane.org/gmane.comp.version-control.git/172419
>
> In particular, I would be interested in feedback about how to best support:
>
> - multiple extensions - do we want support installing extensions in
> their own directories, per Pau's suggestion or simply allow them to
> write to a common directory?
> - multiple extension directories - how to support Jonathan's
> requirement to allow user specific extension directories?

Well, let's step back for a moment.  What problem are we solving?
I still don't even know the answer to that!

Once upon a time, all git commands lived on the $PATH (typically in
/usr/bin, $HOME/bin, or some similar place) and could even be
invoked directly as git-commandname.  At some point someone noticed
that by running

	git-<TAB>

it was possible to read the list of all git commands.  Unfortunately
the list was very long, and this seemed like a much worse introduction
to git than the shorter list shown by "git --help".

There were also some related minor problems --- for example, git was
putting more pressure than necessary on filesystems and other
facilities to keep track of all the files in $bindir, and providing
the dashed forms of commands provides a temptation to use them
exclusively, making features (like aliases) of the git wrapper less
discoverable.  But the main thing was the tab completion.

The fix was to tuck away the individual commands somewhere under
libexecdir, outside $PATH.

Now at some point in this discussion I thought you were solving a
related problem.  If a person were to install 100 new commands for
git, or a single package with 100 commands in it, then

	git-<TAB>

would be daunting again.  So the task becomes to find a place to tuck
away these new commands without placing them on the $PATH.

But now I am less sure.  The motivating example has less than 10
commands; that doesn't seem worth all the fuss at all.  Why not just
install the command on the $PATH?  "git help work" _would_ work on all
the systems I have easy access to.  For example, if I write:

	install:
		install -m0755 git-work $(prefix)/bin
		install -d -m0755 $(prefix)/share/man/man1
		gzip -9 <git-work.1 >git-work.1.gz
		install -m0644 git-work.1.gz $(prefix)/share/man/man1/
		install -d -m0755 $(prefix)/bin

and the user runs

	make install
	PATH=$HOME/bin:$PATH

then "man git-work" will just work.  Similarly, "git work --help"
(which the git wrapper transforms to "git help work") would just work.

I see only a few potential problems remaining:

 1. There is no automatically generated documentation page pointing
    to the documentation for all new commands of this kind.  So
    I can run "git help -a" to learn about installed commands, but
    I cannot run "man git" to do so.  Likewise for info.

 2. On platforms like Windows that do not use manpages, my "git work"
    documentation will not show up with "git work --help".  For this,
    it would certainly be useful to have a GIT_HTML_PATH environment
    variable (or some similar name) that could be used to point to a
    list of directories with additional documentation.  The default
    could be something along the lines of the default library search
    path (but simpler), like:

	/usr/local/share/git/help:/usr/share/git/help

    Users installing new commands under $HOME might want to prepend
    something like

	$HOME/share/git/help:

    or whatever directory names suit their tastes.

    Even better might be a way for "git help" to ask the command
    where it puts its documentation, so "git help work" would
    internally run "git work --html-path".

 3. On a machine with multiple installations of git, my new command
    is not tied to any particular installation but shared by all of
    them.  This is a feature, not a bug.

 4. My command is visible with git-<TAB>, as mentioned above.

My comments about unprivileged users installing new commands were to
explain why an alternative solution to (2) that uses a single
directory where "git help" always looks would be inadequate.  I still
think that but luckily there is a large space of possible designs
without that problem; two are mentioned in the description of (2)
above.

Now problems 2 and 4 can be solved at the same time by introducing
something like a GIT_PLUGINS_PATH variable that could be shared for
both uses.  I'm not so convinced that's a good idea (I prefer to see
decoupled solutions to independent problems when it's simple to do)
but it could very well turn out okay.

Re: RFC: a plugin architecture for git extensions?

[Jonathan Nieder]

Jonathan Nieder wrote:

> 	install:
> 		install -m0755 git-work $(prefix)/bin
> 		install -d -m0755 $(prefix)/share/man/man1
> 		gzip -9 <git-work.1 >git-work.1.gz
> 		install -m0644 git-work.1.gz $(prefix)/share/man/man1/
> 		install -d -m0755 $(prefix)/bin

I'm sorry; I don't even know what sequence of keystrokes I used to
make this upside-down series of commands.  The intent was rather

	install: git-work git-work.1.gz
		: executable
		install -d -m0755 $(prefix)/bin
		install -m0755 git-work $(prefix)/bin
		: manual page
		install -d -m0755 $(prefix)/share/man/man1
		install -m0644 git-work.1.gz $(prefix)/share/man/man1

Sorry for the noise.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Fri, May 6, 2011 at 4:56 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Jon Seymour wrote:
>
>> I would appreciate any feedback you (or others) have about:
>>
>>     http://permalink.gmane.org/gmane.comp.version-control.git/172419
>>
>> In particular, I would be interested in feedback about how to best support:
>>
>> - multiple extensions - do we want support installing extensions in
>> their own directories, per Pau's suggestion or simply allow them to
>> write to a common directory?
>> - multiple extension directories - how to support Jonathan's
>> requirement to allow user specific extension directories?
>
> Well, let's step back for a moment.  What problem are we solving?
> I still don't even know the answer to that!
>

Jonathan,

Thanks for your detailed reply.

I think the problem we are trying to solve is this: how to make it as
easy as possible to install, and get operational, an extension to git.

If git supported the concept of a standard place to put extensions,
then it could be as simple as:

    unzip -d $(git --plugins-dir) plugin.zip

with no need to configure or choose a prefix and no need to edit the
an .profile or .bashrc to permanently add a directory to the PATH.

If there is no post-installation configuration step, then it becomes
much easier to write an adhoc script that produces a working
environment that has the extension already operational.

The idea is that the person or distribution who(/which) installed git
onto the system has already made decisions about where artifacts
should go. Neither the user nor the extension author should have to
second guess or reverse engineer these decisions. Given that git has
been installed in X, then let's use convention to say that extensions
can install in X/lib/git-plugins/ (say) and then have git able to use
those extensions immediately with no additional configuration step
because this directory is already (effectively) in the PATH.

To the extent that a configuration step is necessary, post install, it
would be good if the configuration was limited to facilities that are
available in every git installation (e.g. git config --global).

jon.

Re: RFC: a plugin architecture for git extensions?

[Jonathan Nieder]

Jon Seymour wrote:

> If git supported the concept of a standard place to put extensions,
> then it could be as simple as:
>
>     unzip -d $(git --plugins-dir) plugin.zip
>
> with no need to configure or choose a prefix and no need to edit the
> an .profile or .bashrc to permanently add a directory to the PATH.

Why not use "/usr/local" in place of "git --plugins-dir"?  (I can
think of one answer --- namely root privileges --- but it would apply
to any system with one standard place to put extensions.)

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Sat, May 7, 2011 at 12:17 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Jon Seymour wrote:
>
>> If git supported the concept of a standard place to put extensions,
>> then it could be as simple as:
>>
>>     unzip -d $(git --plugins-dir) plugin.zip
>>
>> with no need to configure or choose a prefix and no need to edit the
>> an .profile or .bashrc to permanently add a directory to the PATH.
>
> Why not use "/usr/local" in place of "git --plugins-dir"?  (I can
> think of one answer --- namely root privileges --- but it would apply
> to any system with one standard place to put extensions.)
>

Partly because that is second guessing &/or reverse engineering the
distribution's decisions and
it won't work for a Windows install where there is no /usr/local

Not that I currently, have a need, but Junio did mention the case
where someone wants to enhance an existing git command with a wrapper
of some kind. This could be done more precisely if git itself
controlled the relative order of the path components than if one
relied on /usr/local being in a particular place in the path.


jon.

Re: RFC: a plugin architecture for git extensions?

[Jonathan Nieder]

Jon Seymour wrote:

> Partly because that is second guessing &/or reverse engineering the
> distribution's decisions and

Well, no --- that's what /usr/local is _for_:
http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLOCALLOCALHIERARCHY

> it won't work for a Windows install where there is no /usr/local

That's true.  I believe command-line users on Windows who install by
unzipping to a directory are used to having to set a PATH for
themselves.  Perhaps it would be convenient for git to learn to add a
specific standard directory to its private PATH as a Windows-specific
extension, though.

If your goal is to make installing new commands for git easier than
installing native apps --- why?  It seems backwards.  Consider that
the end result ought to be easy not only for the app developer but for
the end user, and if every program with the ability to call other
programs sets up its own better replacement for standard operating
system facilities, that will make for a complicated system to
administer indeed.  So with that in mind, it might be simpler to take
advantage of existing project that simplifies installation of native
apps, like <http://zero-install.sourceforge.net/>.

The development environment for git on Windows (mysgit) does provide a
directory hierarchy complete with /usr et al, so people using that
very well might want to install to /usr/local.  Likewise with Cygwin.

> Not that I currently, have a need, but Junio did mention the case
> where someone wants to enhance an existing git command with a wrapper
> of some kind.

For reasons you've hinted at before, Git deliberately does not allow
that (similarly, it does not allow git aliases to override existing
commands).  $GIT_EXEC_PATH comes first on the PATH that git uses
internally.  That's a feature, not a bug, imho.

Re: RFC: a plugin architecture for git extensions?

[Jeff King]

On Sat, May 07, 2011 at 12:07:14AM +1000, Jon Seymour wrote:

> I think the problem we are trying to solve is this: how to make it as
> easy as possible to install, and get operational, an extension to git.
> 
> If git supported the concept of a standard place to put extensions,
> then it could be as simple as:
> 
>     unzip -d $(git --plugins-dir) plugin.zip
> 
> with no need to configure or choose a prefix and no need to edit the
> an .profile or .bashrc to permanently add a directory to the PATH.

This seems slightly backwards to me. You are asking git "where should
plugins go?" and then putting them there. But that leaves no room for
plugins going in _multiple_ places. IOW, the usual hierarchy of:

  1. distribution-packaged extensions (in /usr/share/git/plugins)

  2. local system-wide extensions (in /usr/local/share/git/plugins)

  3. per-user extensions (in $HOME/.gitplugins)

It seems like we should not be asking git, but _telling_ git about where
our plugins are. I understand that you don't want the user to have to do
any additional steps, and I think that is a reasonable goal. But can't
that be easily accomplished by:

  1. The git wrapper learns to look in a set of plugin paths, something
     like:

       foreach path in (list of plugin paths)
         foreach plugin in "path/*"
           add plugin/bin to PATH
           add plugin/man to MANPATH

  2. At compile time, we give some stock system directories like
     /usr/share/git/plugins and /usr/local/share/git/plugins.
     Distribution packages of git override as appropriate for the target
     system.

  3. We always check $HOME/.gitplugins by default.

  4. Users can set GIT_PLUGIN_PATH in the environment if they want to do
     something fancy (they can also always just set PATH and MANPATH
     manually if they want, too).

This is how many systems already work. For example, look at how vim
handles plugins.

Distro-packaged extensions obviously know where to go (the packager
knows their distro's rules). People with personal extensions don't have
to know anything special; their packages go in $HOME/.gitplugins.

In general I would expect /usr/local/share/git/plugins to be pretty
standard, and not needing of being repeated for admins who want to
install something system-wide. But if you want to be really thorough,
then your "git --plugins-dir" should probably report the "system-wide
but not distro" directory for that (but I would call it something like
"git --system-plugins-dir" or something to make it more clear).

-Peff

Re: RFC: a plugin architecture for git extensions?

[John Szakmeister]

On Fri, May 6, 2011 at 1:23 PM, Jeff King <peff@peff.net> wrote:
> On Sat, May 07, 2011 at 12:07:14AM +1000, Jon Seymour wrote:
>
>> I think the problem we are trying to solve is this: how to make it as
>> easy as possible to install, and get operational, an extension to git.
>>
>> If git supported the concept of a standard place to put extensions,
>> then it could be as simple as:
>>
>>     unzip -d $(git --plugins-dir) plugin.zip
>>
>> with no need to configure or choose a prefix and no need to edit the
>> an .profile or .bashrc to permanently add a directory to the PATH.
>
> This seems slightly backwards to me. You are asking git "where should
> plugins go?" and then putting them there. But that leaves no room for
> plugins going in _multiple_ places. IOW, the usual hierarchy of:
>
>  1. distribution-packaged extensions (in /usr/share/git/plugins)
>
>  2. local system-wide extensions (in /usr/local/share/git/plugins)
>
>  3. per-user extensions (in $HOME/.gitplugins)

Exactly!  I'd really like to see (3) and stop cluttering up my
~/.local/bin folder with wrapper scripts.

-John

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Sat, May 7, 2011 at 12:50 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Jon Seymour wrote:
>
>> Partly because that is second guessing &/or reverse engineering the
>> distribution's decisions and
>
> Well, no --- that's what /usr/local is _for_:
> http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLOCALLOCALHIERARCHY
>

Sorry, what I really meant to say is that /usr/local/bin is not always
in a user's path *** . Whether it is or not depends in part on which
distribution manager you use on MAC OSX [ brew does user /usr/local,
but MAC Ports uses /opt/local ].

*** I may be wrong about this - /usr/local/bin is actually in my MAC's
path, but I think I had to edit /etc/profile to get it there.

That said, once git is installed, there is usually a directory like
/usr/local or /opt/local that could be used as the default prefix to
target the installation of a plugin.

Perhaps there is a case for a git --prefix to provide that hint?

>> it won't work for a Windows install where there is no /usr/local
>
> That's true.  I believe command-line users on Windows who install by
> unzipping to a directory are used to having to set a PATH for
> themselves.  Perhaps it would be convenient for git to learn to add a
> specific standard directory to its private PATH as a Windows-specific
> extension, though.
>
> If your goal is to make installing new commands for git easier than
> installing native apps --- why?  It seems backwards.  Consider that
> the end result ought to be easy not only for the app developer but for
> the end user, and if every program with the ability to call other
> programs sets up its own better replacement for standard operating
> system facilities, that will make for a complicated system to
> administer indeed.  So with that in mind, it might be simpler to take
> advantage of existing project that simplifies installation of native
> apps, like <http://zero-install.sourceforge.net/>.
>

I will have a look at zero-install, though quick inspection seems to
indicate that it is best for running standalone apps and not
necessarily great for the task at hand here.

The idea isn't to replace standard operating system facilities for
application installation. The idea is to provide a uniform interface
for accomplishing the rather limited task of extending git that can
work the same way, irrespective of platform, irrespective of file
system layouts, irrespective of assumptions about which directories
are already in the user's paths.

One answer is for each extension author to invent their own way to
produce packages that can be installed in a cross-platform way across
multiple platforms but this seems like an unnecessary duplication of
effort compared to a possible alternative.

If at the end of the day, we say make and install are the way to do
it, then fine. However, this makes the dependencies for a successful
install strictly greater than the existence of git and a POSIX shell
which we can assume if git is already installed.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Sat, May 7, 2011 at 3:23 AM, Jeff King <peff@peff.net> wrote:
> On Sat, May 07, 2011 at 12:07:14AM +1000, Jon Seymour wrote:
>
>> I think the problem we are trying to solve is this: how to make it as
>> easy as possible to install, and get operational, an extension to git.
>>
>> If git supported the concept of a standard place to put extensions,
>> then it could be as simple as:
>>
>>     unzip -d $(git --plugins-dir) plugin.zip
>>
>> with no need to configure or choose a prefix and no need to edit the
>> an .profile or .bashrc to permanently add a directory to the PATH.
>
> This seems slightly backwards to me. You are asking git "where should
> plugins go?" and then putting them there. But that leaves no room for
> plugins going in _multiple_ places. IOW, the usual hierarchy of:
>
>  1. distribution-packaged extensions (in /usr/share/git/plugins)
>
>  2. local system-wide extensions (in /usr/local/share/git/plugins)
>
>  3. per-user extensions (in $HOME/.gitplugins)
>

Actually, that was part of my RFC - how to let git find places where
plugins may be installed.

Your suggestions seem good to me.

> It seems like we should not be asking git, but _telling_ git about where
> our plugins are. I understand that you don't want the user to have to do
> any additional steps, and I think that is a reasonable goal. But can't
> that be easily accomplished by:
>
>  1. The git wrapper learns to look in a set of plugin paths, something
>     like:
>
>       foreach path in (list of plugin paths)
>         foreach plugin in "path/*"
>           add plugin/bin to PATH
>           add plugin/man to MANPATH
>
>  2. At compile time, we give some stock system directories like
>     /usr/share/git/plugins and /usr/local/share/git/plugins.
>     Distribution packages of git override as appropriate for the target
>     system.
>
>  3. We always check $HOME/.gitplugins by default.
>

All seems reasonable, assuming that there is a consensus that we need
to do something other than recommend installation into /usr/local (or
the platform equivalent) and I am not sure that is a given at this
point.

>  4. Users can set GIT_PLUGIN_PATH in the environment if they want to do
>     something fancy (they can also always just set PATH and MANPATH
>     manually if they want, too).
>

If the need for multiple plugin directories were accepted, then I
wonder if there might not be some advantages for the configuration of
this path being in git configuration rather than an environment
variable? Again, assuming only git, this would provide installation
helpers with a standard way to make git aware of such directories that
does not depend on the particulars of the platform on which git is
installed.

> This is how many systems already work. For example, look at how vim
> handles plugins.
>
> Distro-packaged extensions obviously know where to go (the packager
> knows their distro's rules). People with personal extensions don't have
> to know anything special; their packages go in $HOME/.gitplugins.
>
> In general I would expect /usr/local/share/git/plugins to be pretty
> standard, and not needing of being repeated for admins who want to
> install something system-wide. But if you want to be really thorough,
> then your "git --plugins-dir" should probably report the "system-wide
> but not distro" directory for that (but I would call it something like
> "git --system-plugins-dir" or something to make it more clear).
>

I think that your suggestion of --system-plugins-dir is a good one for
the reasons you suggest.

> -Peff
>

Re: RFC: a plugin architecture for git extensions?

[Jonathan Nieder]

Jon Seymour wrote:

> The idea isn't to replace standard operating system facilities for
> application installation. The idea is to provide a uniform interface
> for accomplishing the rather limited task of extending git that can
> work the same way, irrespective of platform, irrespective of file
> system layouts, irrespective of assumptions about which directories
> are already in the user's paths.
[...]
> If at the end of the day, we say make and install are the way to do
> it, then fine. However, this makes the dependencies for a successful
> install strictly greater than the existence of git and a POSIX shell
> which we can assume if git is already installed.

Thanks for explaining, and sorry to have given so much grief by not
understanding.

If this whole conversation were about how to add a new menu item to
git gui, I would have understood completely.  Having to figure out how
to get something in a directory listed in PATH would be undue
complication.

After going in circles a few times, I think you're saying there are
also some people using git on the command line for whom "make
prefix=<whereever> install" won't cut it.  With such a person in mind,
what you're trying to do makes sense --- and why not do it, when it
will bring some other benefits as a side-effect, like the ability to
add new commands without them showing up in git-<TAB> tab completion?
So I'll be quiet now.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Sun, May 8, 2011 at 4:49 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Jon Seymour wrote:
>
>> The idea isn't to replace standard operating system facilities for
>> application installation. The idea is to provide a uniform interface
>> for accomplishing the rather limited task of extending git that can
>> work the same way, irrespective of platform, irrespective of file
>> system layouts, irrespective of assumptions about which directories
>> are already in the user's paths.
> [...]
>> If at the end of the day, we say make and install are the way to do
>> it, then fine. However, this makes the dependencies for a successful
>> install strictly greater than the existence of git and a POSIX shell
>> which we can assume if git is already installed.
>
> Thanks for explaining, and sorry to have given so much grief by not
> understanding.
>
> If this whole conversation were about how to add a new menu item to
> git gui, I would have understood completely.  Having to figure out how
> to get something in a directory listed in PATH would be undue
> complication.
>
> After going in circles a few times, I think you're saying there are
> also some people using git on the command line for whom "make
> prefix=<whereever> install" won't cut it.  With such a person in mind,
> what you're trying to do makes sense --- and why not do it, when it
> will bring some other benefits as a side-effect, like the ability to
> add new commands without them showing up in git-<TAB> tab completion?
> So I'll be quiet now.
>

Admittedly, I think there might be concrete advantages in making at
hard as possible for such users to get anywhere near a git command
line, but hey, I work in an environment where ClearCase looks like a
reasonable option to some people :-)

Your point about support for extension supplied HTML is an interesting
one, that I don't think I have an answer for other than an
installation script that is aware of git --html-path. The reason is,
you want links to descriptions of core git commands to just work in
such html, and the only way you can get that to happen reliably is to
install in some directory that is strictly relative to the location
that git's own HTML is stored in.

So, in the end, I think I would be happy with a solution like:

    git --system-extensions-prefix

which answers the recommended prefix for the installation of
extensions. This might default git's own prefix, but may be configured
differently by distribution or builder preference. If we did allow it
to be different to the git prefix, then I think the git runtime would
need to be extended to allow $(git --system-plugins-prefix)/bin to
appear in the path, and then use the existing gt help mechanisms to
find the
man pages.

Would a patch along these lines be acceptable, do you think?

jon seymour.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Fri, May 6, 2011 at 4:56 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
>
> But now I am less sure.  The motivating example has less than 10
> commands; that doesn't seem worth all the fuss at all.  Why not just
> install the command on the $PATH?  "git help work" _would_ work on all
> the systems I have easy access to.  For example, if I write:
>

An aspect of the motivational example that I would welcome your
feedback on is this.

git-work is an example of an extension that depends on 3 other
extensions (git-base, git-atomic and git-test).

Similarly:

   git-base depends on git-test.
   git-atomic depends on git-test

It seems that if I am to represent these dependencies, I need to
create 4 * N packages, where N is the number of packaging systems I
want to support.

It is unclear how I make:

    make install prefix=/usr/local

work unless the 3 other dependencies have been installed first. If I
need to explain that these 3 other dependencies need to be installed,
then my installation instructions have just become longer.

This seems a shame, since each extension has a dependency only on
other git extensions and git itself. Yet, to provide a workable
installation from source option for my users, I need to either get
them to fetch and build all 4 packages from source, or I need to build
packages for N different distribution managers.

[ FWIW: I recognise that my current proposals for a simple git
--system-extensions-dir don't make this problem any easier to solve,
but I wanted to use the example to demonstrate why a simple make
install script isn't the silver bullet to every installation scenario
].

jon.

Re: RFC: a plugin architecture for git extensions?

[Miles Bader]

Junio C Hamano <gitster@pobox.com> writes:
> Do you or do you not run "apt-get install git-superadd"?
>
> One possible answer may be to run "apt-get install git-superadd", and then
> the users who want "git add" to behave in a new way to opt-in to use the
> "plug-in".  I think that is what Jon is getting at.

If aliases could override built-in command names, it'd be easy enough
("alias add=superadd") ... [with some feature to allow suppressing the
alias to prevent recursion, e.g. an environment variable or something.]

-Miles

-- 
Justice, n. A commodity which in a more or less adulterated condition the
State sells to the citizen as a reward for his allegiance, taxes and personal
service.

Re: RFC: a plugin architecture for git extensions?

[Jeff King]

On Sun, May 08, 2011 at 02:44:54PM +1000, Jon Seymour wrote:

> >  4. Users can set GIT_PLUGIN_PATH in the environment if they want to do
> >     something fancy (they can also always just set PATH and MANPATH
> >     manually if they want, too).
> 
> If the need for multiple plugin directories were accepted, then I
> wonder if there might not be some advantages for the configuration of
> this path being in git configuration rather than an environment
> variable?

I think in general most users would not need to set this (since after
all, the point of your proposal is to avoid such tweaks), so it may not
be worth troubling about. But it is much simpler to tell users to run:

  git config core.pluginpath /path/to/wherever

than trying to figure out whether they need to use bashrc, cshrc,
whatever Windows uses, etc.

-Peff

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Mon, May 9, 2011 at 5:35 PM, Jeff King <peff@peff.net> wrote:
> On Sun, May 08, 2011 at 02:44:54PM +1000, Jon Seymour wrote:
>
>> >  4. Users can set GIT_PLUGIN_PATH in the environment if they want to do
>> >     something fancy (they can also always just set PATH and MANPATH
>> >     manually if they want, too).
>>
>> If the need for multiple plugin directories were accepted, then I
>> wonder if there might not be some advantages for the configuration of
>> this path being in git configuration rather than an environment
>> variable?
>
> I think in general most users would not need to set this (since after
> all, the point of your proposal is to avoid such tweaks), so it may not
> be worth troubling about. But it is much simpler to tell users to run:
>
>  git config core.pluginpath /path/to/wherever
>
> than trying to figure out whether they need to use bashrc, cshrc,
> whatever Windows uses, etc.

Yep, that was part of the motivation for the suggestion - something
that works consistently, assuming only a working git installation.

Per one of my other notes, my initial inclination is to provide a
patch that implements support for

     git --system-extensions-dir

which would:
   - provide the caller with location that extensions could be
installed in (assuming the caller can acquire write privileges)
   - provide a guarantee that $(git --system-extensions-dir)/bin will
be on the path set up by the git wrapper and $(git
--system-extensions-dir)/man will be in the MANPATH searched by git
help

Extensions could then use this information, together with git
--html-path to install themselves into these places using whatever
mechanism seems appropriate (either a POSIX shell script or a
make/install script).

The default value of git --system-extensions-dir could be supplied
during the build and might naturally default to the build prefix, but
distributions and other builders could specify an alternative.

An enhancement to this would then allow core.system-extensions-dir to
override the compiled in version.

Anything wrong with this so far?

jon

Re: RFC: a plugin architecture for git extensions?

[Jeff King]

On Mon, May 09, 2011 at 05:49:27PM +1000, Jon Seymour wrote:

> Yep, that was part of the motivation for the suggestion - something
> that works consistently, assuming only a working git installation.
> 
> Per one of my other notes, my initial inclination is to provide a
> patch that implements support for
> 
>      git --system-extensions-dir
> 
> which would:
>    - provide the caller with location that extensions could be
> installed in (assuming the caller can acquire write privileges)
>    - provide a guarantee that $(git --system-extensions-dir)/bin will
> be on the path set up by the git wrapper and $(git
> --system-extensions-dir)/man will be in the MANPATH searched by git
> help
> 
> Extensions could then use this information, together with git
> --html-path to install themselves into these places using whatever
> mechanism seems appropriate (either a POSIX shell script or a
> make/install script).

But is the system extension dir always the right place to do so? If I'm
not root, then that probably won't be writable (or even if I am, I may
want to install the extension only for the root user).

If your proposal is for the user to decide on one of:

  unzip -d "$(git --system-extension-dir)" git-foo.zip

or

  unzip -d "$HOME/.gitplugins" git-foo.zip

then they can make that decision. But if you're proposing that the
extension-writer distribute a script, then it's more complicated. They
would probably need to provide a "--user" versus "--system" option.

It would also be tempting to write something like:

  install_dir() {
    if test "`id -u`" = "0"; then
      git --system-extension-dir
    else
      echo $HOME/.gitplugins
    fi
  }

but that is:

  1. Not portable.

  2. Does not allow for user-only installation by root.

But all of this is a packaging best-practices issue, not an issue of
what git needs to do to support it (you _could_ address the portability
issue by having "git --preferred-extension-path" that did the
appropriate platform-specific UID check, but that still doesn't address
the second point).

-Peff

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Mon, May 9, 2011 at 6:12 PM, Jeff King <peff@peff.net> wrote:
> On Mon, May 09, 2011 at 05:49:27PM +1000, Jon Seymour wrote:
>
>> Yep, that was part of the motivation for the suggestion - something
>> that works consistently, assuming only a working git installation.
>>
>> Per one of my other notes, my initial inclination is to provide a
>> patch that implements support for
>>
>>      git --system-extensions-dir
>>
>> which would:
>>    - provide the caller with location that extensions could be
>> installed in (assuming the caller can acquire write privileges)
>>    - provide a guarantee that $(git --system-extensions-dir)/bin will
>> be on the path set up by the git wrapper and $(git
>> --system-extensions-dir)/man will be in the MANPATH searched by git
>> help
>>
>> Extensions could then use this information, together with git
>> --html-path to install themselves into these places using whatever
>> mechanism seems appropriate (either a POSIX shell script or a
>> make/install script).
>
> But is the system extension dir always the right place to do so? If I'm
> not root, then that probably won't be writable (or even if I am, I may
> want to install the extension only for the root user).
>
> If your proposal is for the user to decide on one of:
>
>  unzip -d "$(git --system-extension-dir)" git-foo.zip
>
> or
>
>  unzip -d "$HOME/.gitplugins" git-foo.zip

>
> then they can make that decision. But if you're proposing that the
> extension-writer distribute a script, then it's more complicated. They
> would probably need to provide a "--user" versus "--system" option.
>

I am starting to think that deploy-via-zip/tar is unworkable for the
case where the extension wants to supply html, since I think an
attempt has to be made to deploy HTML in the path reported by git
--html-path for reasons of HTML  linkability from extension back to
the pages from git-core.

Now, this might not always work, and the install script can either
fail (to be re-run under sudo at users discretion) or degrade
gracefully (install what it can, and warn).

> It would also be tempting to write something like:
>
>  install_dir() {
>    if test "`id -u`" = "0"; then
>      git --system-extension-dir
>    else
>      echo $HOME/.gitplugins
>    fi
>  }
>
> but that is:
>
>  1. Not portable.
>
>  2. Does not allow for user-only installation by root.
>
> But all of this is a packaging best-practices issue, not an issue of
> what git needs to do to support it (you _could_ address the portability
> issue by having "git --preferred-extension-path" that did the
> appropriate platform-specific UID check, but that still doesn't address
> the second point).

So, suppose we call it --preferred-extension-path*, then if the user
(root or otherwise) defines

    git config core.preferrred-extension-path ${HOME}/.gitplugins

then they get to choose where the installer next run will install extensions.

Also, this would only be a default - installation scripts could have a
mechanism to specify an override on the command line. Of course, if
they supply the override and it is not consistent with
--preferred-extensions-path/core.preferred-extensions-path then they
need to take steps to install that the bin directory is in the PATH,
but that's a decision they take themselves as an alternative to
acquiring the privileges required to modify the directory reported by
git --preferred-extensions-path

The idea is for core to provide extension installers with just enough
information that their installs will work and be discoverable by the
core and enough control for distributions to determine the preferred
location for such extensions.

* I would prefer --preferred-extension-dir rather than
--preferred-extension-path, since I think this name should specify a
single directory and not a list (otherwise the choice of destination
would be ambigiuous). I realise this creates an inconsistency with
--html-path and if consistency is preferred (or I have misinterpreted
what path normally means), I am happy to use
--preferred-extension-path.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jeff King]

On Mon, May 09, 2011 at 06:45:56PM +1000, Jon Seymour wrote:

> I am starting to think that deploy-via-zip/tar is unworkable for the
> case where the extension wants to supply html, since I think an
> attempt has to be made to deploy HTML in the path reported by git
> --html-path for reasons of HTML  linkability from extension back to
> the pages from git-core.

Yeah, I don't see a way around that without post-processing the HTML
links at install time (or creating a symlink farm with all of the HTML
pages).

> So, suppose we call it --preferred-extension-path*, then if the user
> (root or otherwise) defines
> 
>     git config core.preferrred-extension-path ${HOME}/.gitplugins
> 
> then they get to choose where the installer next run will install extensions.

I thought about suggesting that, but a config option didn't seem a good
fit to me. The decision of where to put a package seems more likely to
be related to which _package_ it is, than which user you are. So a
command-line option would make more sense. And even if you have a config
option, you would sitll need a command-line one to override, so it's not
like you are reducing the amount of code or complexity.

Which again makes it not a git issue at all, but an issue for
package-writers who want to provide a script. It's their job to interact
with the user and find out where the user wants to put things (i.e.,
personal or system directories).

I don't really see any need for git's role in this to be more than:

  1. Check a set list of directories for extra paths to add to PATH and
     MANPATH.

  2. Tell the packager's script what that set list is, so they can be
     sure to put their files in the right spot.

-Peff

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Mon, May 9, 2011 at 8:44 PM, Jeff King <peff@peff.net> wrote:
> On Mon, May 09, 2011 at 06:45:56PM +1000, Jon Seymour wrote:
>
>> I am starting to think that deploy-via-zip/tar is unworkable for the
>> case where the extension wants to supply html, since I think an
>> attempt has to be made to deploy HTML in the path reported by git
>> --html-path for reasons of HTML  linkability from extension back to
>> the pages from git-core.
>
> Yeah, I don't see a way around that without post-processing the HTML
> links at install time (or creating a symlink farm with all of the HTML
> pages).
>
>> So, suppose we call it --preferred-extension-path*, then if the user
>> (root or otherwise) defines
>>
>>     git config core.preferrred-extension-path ${HOME}/.gitplugins
>>
>> then they get to choose where the installer next run will install extensions.
>
> I thought about suggesting that, but a config option didn't seem a good
> fit to me. The decision of where to put a package seems more likely to
> be related to which _package_ it is, than which user you are. So a
> command-line option would make more sense. And even if you have a config
> option, you would sitll need a command-line one to override, so it's not
> like you are reducing the amount of code or complexity.
>
> Which again makes it not a git issue at all, but an issue for
> package-writers who want to provide a script. It's their job to interact
> with the user and find out where the user wants to put things (i.e.,
> personal or system directories).
>
> I don't really see any need for git's role in this to be more than:
>
>  1. Check a set list of directories for extra paths to add to PATH and
>     MANPATH.
>
>  2. Tell the packager's script what that set list is, so they can be
>     sure to put their files in the right spot.
>

The problem with presenting a list of possible directories is that
given a list, the choice is ambiguous. Sure a decision procedure can
be arrived at to choose, but it might be hard for the user to predict
what decision the procedure will reach. In that case, the user must be
prompted for a selection.

Ideally most extension installs would not require any kind of
configuration or decision by the user - simply a statement, I want
this installed and I want it to be available immediately.

The idea, I think, is to make this decision once, not every single
extension install. Chances are
git's default guess will be pretty good (say, /usr/local), but if a
user decides that he wants to install
this, and all future extensions in ~/.gitplugins, then this can be
indicated once, with global configuration
that overrides the compiled in default.

Again, part of the idea is to give the extension installer some degree
of confidence that the selected prefix/bin is, in fact,
in the path and to give the user an override in case the compiled in
default isn't workable for some reason (for example, due to a
permissions issue).

jon.

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Mon, May 9, 2011 at 9:07 PM, Jon Seymour <jon.seymour@gmail.com> wrote:
> On Mon, May 9, 2011 at 8:44 PM, Jeff King <peff@peff.net> wrote:
>> On Mon, May 09, 2011 at 06:45:56PM +1000, Jon Seymour wrote:
>>
>>> I am starting to think that deploy-via-zip/tar is unworkable for the
>>> case where the extension wants to supply html, since I think an
>>> attempt has to be made to deploy HTML in the path reported by git
>>> --html-path for reasons of HTML  linkability from extension back to
>>> the pages from git-core.
>>
>> Yeah, I don't see a way around that without post-processing the HTML
>> links at install time (or creating a symlink farm with all of the HTML
>> pages).
>>
>>> So, suppose we call it --preferred-extension-path*, then if the user
>>> (root or otherwise) defines
>>>
>>>     git config core.preferrred-extension-path ${HOME}/.gitplugins
>>>
>>> then they get to choose where the installer next run will install extensions.
>>
>> I thought about suggesting that, but a config option didn't seem a good
>> fit to me. The decision of where to put a package seems more likely to
>> be related to which _package_ it is, than which user you are. So a
>> command-line option would make more sense. And even if you have a config
>> option, you would sitll need a command-line one to override, so it's not
>> like you are reducing the amount of code or complexity.
>>
>> Which again makes it not a git issue at all, but an issue for
>> package-writers who want to provide a script. It's their job to interact
>> with the user and find out where the user wants to put things (i.e.,
>> personal or system directories).
>>
>> I don't really see any need for git's role in this to be more than:
>>
>>  1. Check a set list of directories for extra paths to add to PATH and
>>     MANPATH.
>>
>>  2. Tell the packager's script what that set list is, so they can be
>>     sure to put their files in the right spot.
>>
>

BTW: the reason why I want the git actually installed to provide this
hint about the best place to install,
rather than use installation script heuristics is that the locally
installed git + the user supplied configuration
is in a much better situation to suggest a good choice than an install
script that is trying to be platform and distribution agnostic.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jeff King]

On Mon, May 09, 2011 at 09:07:31PM +1000, Jon Seymour wrote:

> > I don't really see any need for git's role in this to be more than:
> >
> >  1. Check a set list of directories for extra paths to add to PATH and
> >     MANPATH.
> >
> >  2. Tell the packager's script what that set list is, so they can be
> >     sure to put their files in the right spot.
> >
> 
> The problem with presenting a list of possible directories is that
> given a list, the choice is ambiguous.

Well, yeah. Though between git and the package script, I think it ends
up being simplified to the --system-extension-dir thing, because out of
the three places I mentioned you might want to install something:

  1. You definitely don't want to install in the distro location. If you
     did, then you would be a distro package, and therefore would not
     have to be asking git where that location is.

  2. You do need to know where the locally-installed system path is, so
     we provide an option to query that.

  3. You don't need to ask git where the per-user path is. It will be at
     some well-known location like $HOME/.gitplugins.

That being said, I think you are more concerned with not presenting
too many choices to the user. And that still leaves one choice: system
(2) versus user (3) install.

But I don't think there's a way around that. You are going to have users
of both types, and you will want to serve them. You can make a guess
based on something like writability of the system directory, I suppose,
and let them override via the command-line if they want to.

That strikes me as somewhat flaky and unpredictable, but I am perhaps
not a representative sample. I have always thought the Windows "if you
install as Administrator, it is available to everybody, but otherwise it
is available only to you" behavior was confusing, and this seems like
the same thing.

> Sure a decision procedure can be arrived at to choose, but it might be
> hard for the user to predict what decision the procedure will reach.
> In that case, the user must be prompted for a selection.

I think this is in agreement with what I just wrote above.

> Again, part of the idea is to give the extension installer some degree
> of confidence that the selected prefix/bin is, in fact, in the path
> and to give the user an override in case the compiled in default isn't
> workable for some reason (for example, due to a permissions issue).

So from this, I gather you would like to see something like:

  $ cd git-work && ./install
  fatal: unable to write to /usr/local/share/git: permission denied
  You may install git-work just for the current user with:

    ./install --user

I.e., the script has a predictable and sane behavior, but you guide the
user through overriding it if need be. That doesn't seem unreasonable to
me.

Anyway, with respect to "core.preferredPluginPath", if you do want to go
in that direction, I don't think any extra git support is needed. You
can already just do "git config --path core.preferredPluginPath" to get
the value (with tilde-expansion, even).

-Peff

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Mon, May 9, 2011 at 9:24 PM, Jeff King <peff@peff.net> wrote:
> On Mon, May 09, 2011 at 09:07:31PM +1000, Jon Seymour wrote:
>
>> > I don't really see any need for git's role in this to be more than:
>> >
>> >  1. Check a set list of directories for extra paths to add to PATH and
>> >     MANPATH.
>> >
>> >  2. Tell the packager's script what that set list is, so they can be
>> >     sure to put their files in the right spot.
>> >
>>
>> The problem with presenting a list of possible directories is that
>> given a list, the choice is ambiguous.
>
> Well, yeah. Though between git and the package script, I think it ends
> up being simplified to the --system-extension-dir thing, because out of
> the three places I mentioned you might want to install something:
>
>  1. You definitely don't want to install in the distro location. If you
>     did, then you would be a distro package, and therefore would not
>     have to be asking git where that location is.
>
>  2. You do need to know where the locally-installed system path is, so
>     we provide an option to query that.
>
>  3. You don't need to ask git where the per-user path is. It will be at
>     some well-known location like $HOME/.gitplugins.
>
> That being said, I think you are more concerned with not presenting
> too many choices to the user. And that still leaves one choice: system
> (2) versus user (3) install.
>
> But I don't think there's a way around that. You are going to have users
> of both types, and you will want to serve them. You can make a guess
> based on something like writability of the system directory, I suppose,
> and let them override via the command-line if they want to.
>
> That strikes me as somewhat flaky and unpredictable, but I am perhaps
> not a representative sample. I have always thought the Windows "if you
> install as Administrator, it is available to everybody, but otherwise it
> is available only to you" behavior was confusing, and this seems like
> the same thing.
>
>> Sure a decision procedure can be arrived at to choose, but it might be
>> hard for the user to predict what decision the procedure will reach.
>> In that case, the user must be prompted for a selection.
>
> I think this is in agreement with what I just wrote above.
>
>> Again, part of the idea is to give the extension installer some degree
>> of confidence that the selected prefix/bin is, in fact, in the path
>> and to give the user an override in case the compiled in default isn't
>> workable for some reason (for example, due to a permissions issue).
>
> So from this, I gather you would like to see something like:
>
>  $ cd git-work && ./install
>  fatal: unable to write to /usr/local/share/git: permission denied
>  You may install git-work just for the current user with:
>
>    ./install --user
>
> I.e., the script has a predictable and sane behavior, but you guide the
> user through overriding it if need be. That doesn't seem unreasonable to
> me.
>
> Anyway, with respect to "core.preferredPluginPath", if you do want to go
> in that direction, I don't think any extra git support is needed. You
> can already just do "git config --path core.preferredPluginPath" to get
> the value (with tilde-expansion, even).
>
> -Peff
>

I still think it would be useful for the git wrapper to add
core.preferredPluginPrefix/bin to
the PATH, so that there is no requirement for the user to do this
separately via mechanisms
that will differ according to platform, shell etc.

jon.

Re: RFC: a plugin architecture for git extensions?

[Jeff King]

On Mon, May 09, 2011 at 09:28:26PM +1000, Jon Seymour wrote:

> > Anyway, with respect to "core.preferredPluginPath", if you do want to go
> > in that direction, I don't think any extra git support is needed. You
> > can already just do "git config --path core.preferredPluginPath" to get
> > the value (with tilde-expansion, even).
> 
> I still think it would be useful for the git wrapper to add
> core.preferredPluginPrefix/bin to the PATH, so that there is no
> requirement for the user to do this separately via mechanisms that
> will differ according to platform, shell etc.

Ah, yeah, that probably does make sense.

My original conception was that it would be more like
"preferredExtensionType" and would be either "user" or "system", from
which the installer would select either "$HOME/.gitplugins" or "git
--system-extension-dir" respectively. And I was still thinking in those
terms, even though the example you showed was obviously an arbitrary
path.

If you want to go the arbitrary path route, then yeah, it would need to
be added to git's expansion list.

There is one drawback with that, though. Consider something like this:

  $ git config core.preferredPluginPrefix /opt/git-plugins
  $ cd git-foo && ./install
  [installs in /opt/git-plugins/git-foo]
  $ git foo ;# works fine

  [time passes; now you decide you want to install new plugins in your
   home directory]
  $ git config core.preferredPluginPrefix $HOME/.gitplugins
  $ cd git-bar && ./install
  [installs in $HOME/.gitplugins]
  $ git bar ;# works fine
  $ git foo ;# now broken!

So there is some value to separating the concept of "these are the paths
git looks in" and "this is the path we install into" and enforcing that
the latter points to one of the former.

-Peff

Re: RFC: a plugin architecture for git extensions?

[Jon Seymour]

On Mon, May 9, 2011 at 10:21 PM, Jeff King <peff@peff.net> wrote:
> On Mon, May 09, 2011 at 09:28:26PM +1000, Jon Seymour wrote:
>
>> > Anyway, with respect to "core.preferredPluginPath", if you do want to go
>> > in that direction, I don't think any extra git support is needed. You
>> > can already just do "git config --path core.preferredPluginPath" to get
>> > the value (with tilde-expansion, even).
>>
>> I still think it would be useful for the git wrapper to add
>> core.preferredPluginPrefix/bin to the PATH, so that there is no
>> requirement for the user to do this separately via mechanisms that
>> will differ according to platform, shell etc.
>
> Ah, yeah, that probably does make sense.
>
> My original conception was that it would be more like
> "preferredExtensionType" and would be either "user" or "system", from
> which the installer would select either "$HOME/.gitplugins" or "git
> --system-extension-dir" respectively. And I was still thinking in those
> terms, even though the example you showed was obviously an arbitrary
> path.
>
> If you want to go the arbitrary path route, then yeah, it would need to
> be added to git's expansion list.
>
> There is one drawback with that, though. Consider something like this:
>
>  $ git config core.preferredPluginPrefix /opt/git-plugins
>  $ cd git-foo && ./install
>  [installs in /opt/git-plugins/git-foo]
>  $ git foo ;# works fine
>
>  [time passes; now you decide you want to install new plugins in your
>   home directory]
>  $ git config core.preferredPluginPrefix $HOME/.gitplugins
>  $ cd git-bar && ./install
>  [installs in $HOME/.gitplugins]
>  $ git bar ;# works fine
>  $ git foo ;# now broken!
>
> So there is some value to separating the concept of "these are the paths
> git looks in" and "this is the path we install into" and enforcing that
> the latter points to one of the former.

Fair point. I'll ponder this some more.

Thanks,

jon.

Re: RFC: a plugin architecture for git extensions?

[Enrico Weigelt]

* Jon Seymour <jon.seymour@gmail.com> wrote:

> No. As I explained in the posts
>  that you chose not to read, such concerns would be dealt with by real
> package managers.

Okay, you're essentially aliasing git-pm to the distro's existing
package manager. In the end, these git extensions still have to
be packaged for your actual distro. So why not just using the
distro's package manager directly ?


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: RFC: a plugin architecture for git extensions?

[David Aguilar]

On Mon, May 09, 2011 at 01:36:16PM +0900, Miles Bader wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> > Do you or do you not run "apt-get install git-superadd"?
> >
> > One possible answer may be to run "apt-get install git-superadd", and then
> > the users who want "git add" to behave in a new way to opt-in to use the
> > "plug-in".  I think that is what Jon is getting at.
> 
> If aliases could override built-in command names, it'd be easy enough
> ("alias add=superadd") ... [with some feature to allow suppressing the
> alias to prevent recursion, e.g. an environment variable or something.]

I always thought this restriction was a good thing.

The usability argument usually goes that it's generally
bad for "git frotz" somewhere to behave different from
"git frotz" somewhere else.

Keeping this "limitation" is good for the sake of consistency.

This topic is orthogonal to the plugin rfc, though.
-- 
					David