[Proposal] Clonable scripts

[Proposal] Clonable scripts

[Niels Basjes]

Hi,

As we all know the hooks ( in .git/hooks ) are not cloned along with
the code of a project.
Now this is a correct approach for the scripts that do stuff like
emailing the people responsible for releases or submitting the commit
to a CI system.

For several other things it makes a lot of sense to give the developer
immediate feedback. Things like the format of the commit message (i.e.
it must start with an issue tracker id) or compliance with a coding
standard.

Initially I wanted to propose introducing fully clonable (pre-commit)
hook scripts.
However I can imagine that a malicious opensource coder can create a
github repo and try to hack the computer of a contributer via those
scripts. So having such scripts is a 'bad idea'.

If those scripts were how ever written in a language that is build
into the git program and the script are run in such a way that they
can only interact with the files in the local git (and _nothing_
outside of that) this would be solved.

Also have a builtin scripting language also means that this would run
on all operating systems (yes, even Windows).

So I propose the following new feature:

1) A scripting language is put inside git. Perhaps a version of python
or ruby or go or ... (no need for a 'new' language)

2) If a project contains a folder called .githooks in the root of the
code base then the rules/scripts that are present there are executed
ONLY on the system doing the actual commit. These scripts are run in
such a limited way that they can only read the files in the
repository, they cannot do any networking/write to disk/etc and they
can only do a limited set op actions against the current operation at
hand (i.e. do checks, parse messages, etc).

3) For the regular hooks this language is also support and when
located in the (not cloned!) .git/hooks directory they are just as
powerful as a normal script (i.e. can control CI, send emails, etc.).

Like I said, this is just a proposal and I would like to know what you
guys think.

-- 
Best regards / Met vriendelijke groeten,

Niels Basjes

Re: [Proposal] Clonable scripts

[Hilco Wijbenga]

On 9 September 2013 13:48, Niels Basjes <Niels@basjes.nl> wrote:
> If those scripts were how ever written in a language that is build
> into the git program and the script are run in such a way that they
> can only interact with the files in the local git (and _nothing_
> outside of that) this would be solved.

That sounds interesting.

> Also have a builtin scripting language also means that this would run
> on all operating systems (yes, even Windows).

This would be *very* helpful. It's a total pain trying to get hooks
working across different OSes.

> So I propose the following new feature:
>
> 1) A scripting language is put inside git. Perhaps a version of python
> or ruby or go or ... (no need for a 'new' language)

That sounds nice but ...

> 2) If a project contains a folder called .githooks in the root of the
> code base then the rules/scripts that are present there are executed
> ONLY on the system doing the actual commit. These scripts are run in
> such a limited way that they can only read the files in the
> repository, they cannot do any networking/write to disk/etc and they
> can only do a limited set op actions against the current operation at
> hand (i.e. do checks, parse messages, etc).

... how would you prevent Ruby/Python/Go/$GeneralProgLang from
executing arbitrary code?

> Like I said, this is just a proposal and I would like to know what you
> guys think.

I love the idea but I'm not sure how feasible it is. I think you would
be forced to copy an existing language and somehow "make it secure"
(seems like a maintenance nightmare) or to create your own language
(potentially a lot of work). But perhaps something more declarative
might be usable?

Re: [Proposal] Clonable scripts

[Niels Basjes]

On Mon, Sep 9, 2013 at 11:13 PM, Hilco Wijbenga
<hilco.wijbenga@gmail.com> wrote:
> On 9 September 2013 13:48, Niels Basjes <Niels@basjes.nl> wrote:
>> So I propose the following new feature:
>>
>> 1) A scripting language is put inside git. Perhaps a version of python
>> or ruby or go or ... (no need for a 'new' language)
>
> That sounds nice but ...

>> 2) If a project contains a folder called .githooks in the root of the
>> code base then the rules/scripts that are present there are executed
>> ONLY on the system doing the actual commit. These scripts are run in
>> such a limited way that they can only read the files in the
>> repository, they cannot do any networking/write to disk/etc and they
>> can only do a limited set op actions against the current operation at
>> hand (i.e. do checks, parse messages, etc).

> ... how would you prevent Ruby/Python/Go/$GeneralProgLang from
> executing arbitrary code?

Some kind of sandbox?

>> Like I said, this is just a proposal and I would like to know what you
>> guys think.
>
> I love the idea but I'm not sure how feasible it is. I think you would
> be forced to copy an existing language and somehow "make it secure"
> (seems like a maintenance nightmare) or to create your own language
> (potentially a lot of work). But perhaps something more declarative
> might be usable?

As far as I'm concerned it should be the 'best suitable' language for
the task at hand.

-- 
Best regards / Met vriendelijke groeten,

Niels Basjes

Re: [Proposal] Clonable scripts

[Ramkumar Ramachandra]

Niels Basjes wrote:
> As we all know the hooks ( in .git/hooks ) are not cloned along with
> the code of a project.
> Now this is a correct approach for the scripts that do stuff like
> emailing the people responsible for releases or submitting the commit
> to a CI system.

More often than not, maintainers come with these hooks and they keep
them private.

> For several other things it makes a lot of sense to give the developer
> immediate feedback. Things like the format of the commit message (i.e.
> it must start with an issue tracker id) or compliance with a coding
> standard.

i.e. tracker ID. Compliance is simply a request. The developer must be
able to pick it up from surrounding style.

> Initially I wanted to propose introducing fully clonable (pre-commit)
> hook scripts.
> However I can imagine that a malicious opensource coder can create a
> github repo and try to hack the computer of a contributer via those
> scripts. So having such scripts is a 'bad idea'.

I think it's a good idea, since the contributer can look through the scripts.

> If those scripts were how ever written in a language that is build
> into the git program and the script are run in such a way that they
> can only interact with the files in the local git (and _nothing_
> outside of that) this would be solved.

GNU make.

> Also have a builtin scripting language also means that this would run
> on all operating systems (yes, even Windows).

kbuild tends to get complicated.

> So I propose the following new feature:
>
> 1) A scripting language is put inside git. Perhaps a version of python
> or ruby or go or ... (no need for a 'new' language)

make + go sounds like a good alternative.

> 2) If a project contains a folder called .githooks in the root of the
> code base then the rules/scripts that are present there are executed
> ONLY on the system doing the actual commit. These scripts are run in
> such a limited way that they can only read the files in the
> repository, they cannot do any networking/write to disk/etc and they
> can only do a limited set op actions against the current operation at
> hand (i.e. do checks, parse messages, etc).

Submodules and url.<url>.insteadOf come in handy here.

> 3) For the regular hooks this language is also support and when
> located in the (not cloned!) .git/hooks directory they are just as
> powerful as a normal script (i.e. can control CI, send emails, etc.).

I'm confused now; how can .git/hooks be as powerful as .githooks? The
former users should consider uploading their code on GitHub.

> Like I said, this is just a proposal and I would like to know what you
> guys think.

> Best regards / Met vriendelijke groeten,

Which reminds me that we need to have GitTogethers. Thanks for this!

Re: [Proposal] Clonable scripts

[Niels Basjes]

Hi,

On Tue, Sep 10, 2013 at 12:18 AM, Ramkumar Ramachandra
<artagnon@gmail.com> wrote:
> Niels Basjes wrote:
>> As we all know the hooks ( in .git/hooks ) are not cloned along with
>> the code of a project.
>> Now this is a correct approach for the scripts that do stuff like
>> emailing the people responsible for releases or submitting the commit
>> to a CI system.
>
> More often than not, maintainers come with these hooks and they keep
> them private.

Yes.

>> Initially I wanted to propose introducing fully clonable (pre-commit)
>> hook scripts.
>> However I can imagine that a malicious opensource coder can create a
>> github repo and try to hack the computer of a contributer via those
>> scripts. So having such scripts is a 'bad idea'.
>
> I think it's a good idea, since the contributer can look through the scripts.

What I meant to say is that having "fully functional unrestricted
scripts" that are cloned is a "bad idea".
Having "restricted cloned scripts" to me is a "goog idea" (or atleast,
that is what I propose here).


>> 3) For the regular hooks this language is also support and when
>> located in the (not cloned!) .git/hooks directory they are just as
>> powerful as a normal script (i.e. can control CI, send emails, etc.).
>
> I'm confused now; how can .git/hooks be as powerful as .githooks? The
> former users should consider uploading their code on GitHub.

The way I envisioned is is that the scripting language in .git/hooks
is "pick any language you like" with the builtin language as a new
addition.
In the .githooks (which is under version control in the code base and
cloned) is a the same builtin language, yet constrained in a sandbox.

> Which reminds me that we need to have GitTogethers. Thanks for this!

You're welcome.

-- 
Best regards / Met vriendelijke groeten,

Niels Basjes

Re: [Proposal] Clonable scripts

[Sitaram Chamarty]

On 09/10/2013 02:18 AM, Niels Basjes wrote:

> As we all know the hooks ( in .git/hooks ) are not cloned along with
> the code of a project.
> Now this is a correct approach for the scripts that do stuff like
> emailing the people responsible for releases or submitting the commit
> to a CI system.
> 
> For several other things it makes a lot of sense to give the developer
> immediate feedback. Things like the format of the commit message (i.e.
> it must start with an issue tracker id) or compliance with a coding
> standard.
> 
> Initially I wanted to propose introducing fully clonable (pre-commit)
> hook scripts.
> However I can imagine that a malicious opensource coder can create a
> github repo and try to hack the computer of a contributer via those
> scripts. So having such scripts is a 'bad idea'.
> 
> If those scripts were how ever written in a language that is build
> into the git program and the script are run in such a way that they
> can only interact with the files in the local git (and _nothing_
> outside of that) this would be solved.
> 
> Also have a builtin scripting language also means that this would run
> on all operating systems (yes, even Windows).
> 
> So I propose the following new feature:
> 
> 1) A scripting language is put inside git. Perhaps a version of python
> or ruby or go or ... (no need for a 'new' language)
> 
> 2) If a project contains a folder called .githooks in the root of the
> code base then the rules/scripts that are present there are executed
> ONLY on the system doing the actual commit. These scripts are run in
> such a limited way that they can only read the files in the
> repository, they cannot do any networking/write to disk/etc and they
> can only do a limited set op actions against the current operation at
> hand (i.e. do checks, parse messages, etc).
> 
> 3) For the regular hooks this language is also support and when
> located in the (not cloned!) .git/hooks directory they are just as
> powerful as a normal script (i.e. can control CI, send emails, etc.).
> 
> Like I said, this is just a proposal and I would like to know what you
> guys think.

I am not in favour of any idea like this.  It will end in some sort of
compromise (in both sense of the word!)

It has to be voluntary, but we can make it easier.  I suggest something
like this:

  - some special directory can have normal hook files, but it's just a
    place holder.

  - each hook code file comes with some meta data at the top, say
    githook name, hook name, version, remote-name.  I'll use these
    examples:

        pre-commit  crlf-check  1.1     origin

  - on a clone/pull, if there is a change to any of these code files
    when compared to the previous HEAD, and if the program is running
    interactively, then you can ask and setup these hooks.

    The purpose of the remote name in the stored metadata is that we
    don't want to bother updating when we pull from some other repo,
    like when merging a feature branch.

    The purpose of the version number is so you can do some intelligent
    things, even silently upgrade under certain conditions.

All we're doing is making things easier compared to what you can already
do even now (which is completely manual and instructions based).

I don't think anything more intrusive or forced is wise.

And people who say it is OK, I'm going to seriously wonder if you work
for the NSA (directly or indirectly).  Sadly, that is not meant to be a
joke question; such is life now.

Re: [Proposal] Clonable scripts

[Andreas Krey]

On Mon, 09 Sep 2013 22:48:42 +0000, Niels Basjes wrote:
...
> However I can imagine that a malicious opensource coder can create a
> github repo and try to hack the computer of a contributer via those
> scripts. So having such scripts is a 'bad idea'.

Given that half the repos out there are cloned to 'make install' in
them...it's still a bad idea.

> If those scripts were how ever written in a language that is build
> into the git program and the script are run in such a way that they
> can only interact with the files in the local git (and _nothing_
> outside of that) this would be solved.

I still think this is a nightmare of maintenance. You'd need a restricted
version of a language that doesn't allow access outside the repo (and
no TCP either), and someone will always miss some module...

Not that it wouldn't be cool, yet.

...
> Like I said, this is just a proposal and I would like to know what you
> guys think.

I think there are generally two use cases:

- Many people working on repos in an organization. Give them a wrapper
  script that does the clone (and also knows the clone URL already),
  that will set up hooks and configuration as needed.

- github-style cooperation. Add a make hooks to your Makefile that sets
  up the hooks your project seems to want. After all, this is for the
  developers to pre-check what they will submit, so it is in their own
  interest to have (and cross-read) the hooks.

Andreas

-- 
"Totally trivial. Famous last words."
From: Linus Torvalds <torvalds@*.org>
Date: Fri, 22 Jan 2010 07:29:21 -0800