abstracting commit signing/verify to support other signing schemes

abstracting commit signing/verify to support other signing schemes

[Tacitus Aedifex]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=unknown-8bit, Size: 2105 bytes --]

I'm looking at the existing commit signing and verification
integration and it is all GPG specific. I'm interested in refactoring
the code to have a generic signing/verifying interface so that "drivers"
for other signing tools can be created and other signing tools can be
used (e.g. OpenBSD signify).

The existing interface defined in gpg-interface.h is already fairly
generic. It looks like the only things that would need to be fixed
are the names of some members in the signature_check struct and the GPG
specific constants.

I propose to rename the gpg-interface.h file to signature-interface.h.
There are several different ways to do the "polymorphism" needed to have
a base signature_check struct with a tool-specific part for storing the
tool-specific data (e.g. gpg_output, gpg_status, result). I'm looking
for suggestions on the way this has been done in other places in the Git
code so I can do it the same way. My initial impulse it to have a union
of tool-specific structs inside of the signature_check struct.

The plan for changing the signing behavior is to change the code looking
for commit.gpgsign in sequencer.c to instead look for commit.signtool.
The string value will define which signing tool to use. The default will
be null which is the equivilent to gpgsign=false. To get GPG
signing the user would set it to "gpg". To maintain backwards
compatibility, the code will continue to check for commit.gpgsign and
translate that to commit.signtool=gpg and output a warning.

I also think that it makes sense to move the user.signingkey to be
gpg.signingkey since that only makes sense in the context of GPG.

The real trick here is how to handle signatures from different tools in
a given project. I think the answer is to store the value of
commit.signtool along with the signature blob associted with each signed
commit. That way the signature verification code can know which tool to
use to verify the signature. If a commit has a signture but no tool
selector, the default will be to assume GPG to preserve backwards
compatibility.

Any other thoughts and/or suggestions?

//tæ

Re: abstracting commit signing/verify to support other signing schemes

[Jeff King]

On Fri, Aug 03, 2018 at 09:38:34PM +0000, Tacitus Aedifex wrote:

> I'm looking at the existing commit signing and verification
> integration and it is all GPG specific. I'm interested in refactoring
> the code to have a generic signing/verifying interface so that "drivers"
> for other signing tools can be created and other signing tools can be
> used (e.g. OpenBSD signify).
> [...]
> Any other thoughts and/or suggestions?

There's been some work on this lately. See this patch and the response
thread:

  https://public-inbox.org/git/20180409204129.43537-9-mastahyeti@gmail.com/

One of the main complaints there was that it was doing just enough to
make gpgsm work, and it was unclear if some of the abstractions would be
insufficient for something like signify.

The more recent work focused on just doing the minimum to provide
gpg/gpgsm variants:

  https://public-inbox.org/git/cover.1531831244.git.henning.schild@siemens.com/

That replaces the earlier patch series, and is currently merged to the
'next' branch and is on track to get merged to 'master' before Git 2.19
is released.

One of the downsides there is that if we eventually move to a generic
signing-tool config, we'd have to support two layers of historical
abstraction (the original "gpg.program" config, and the new
"gpg.<tool>.*" config).

So _if_ we knew what it would take to support signify, we could
potentially adjust what's going into 2.19 in order to skip straight to
the more generic interface. But on the OTOH, it may not be worth
rushing, and there is already a vague plan of how the gpg.<tool>.*
config would interact with a more generic config.

Anyway. Hopefully that gives you a sense of what the current state is,
and that work should answer the questions you asked about how to
approach the code changes.

-Peff

RE: abstracting commit signing/verify to support other signing schemes

[Randall S. Becker]

On August 3, 2018 5:39 PM, Tacitus Aedifex wrote:
> I'm looking at the existing commit signing and verification integration and it is
> all GPG specific. I'm interested in refactoring the code to have a generic
> signing/verifying interface so that "drivers"
> for other signing tools can be created and other signing tools can be used
> (e.g. OpenBSD signify).
> 
> The existing interface defined in gpg-interface.h is already fairly generic. It
> looks like the only things that would need to be fixed are the names of some
> members in the signature_check struct and the GPG specific constants.
> 
> I propose to rename the gpg-interface.h file to signature-interface.h.
> There are several different ways to do the "polymorphism" needed to have a
> base signature_check struct with a tool-specific part for storing the tool-
> specific data (e.g. gpg_output, gpg_status, result). I'm looking for
> suggestions on the way this has been done in other places in the Git code so I
> can do it the same way. My initial impulse it to have a union of tool-specific
> structs inside of the signature_check struct.
> 
> The plan for changing the signing behavior is to change the code looking for
> commit.gpgsign in sequencer.c to instead look for commit.signtool.
> The string value will define which signing tool to use. The default will be null
> which is the equivilent to gpgsign=false. To get GPG signing the user would
> set it to "gpg". To maintain backwards compatibility, the code will continue to
> check for commit.gpgsign and translate that to commit.signtool=gpg and
> output a warning.
> 
> I also think that it makes sense to move the user.signingkey to be
> gpg.signingkey since that only makes sense in the context of GPG.
> 
> The real trick here is how to handle signatures from different tools in a given
> project. I think the answer is to store the value of commit.signtool along with
> the signature blob associted with each signed commit. That way the
> signature verification code can know which tool to use to verify the
> signature. If a commit has a signture but no tool selector, the default will be
> to assume GPG to preserve backwards compatibility.

If I may suggest something a little off the wall... the abstraction needs to go beyond just the signing tool, but the whole signing infrastructure. I would like to see something along the lines of introducing a signing authority into the mix, so that not only the tool of signing is abstracted, but also the interface to who, if anyone, is responsible for signing. If I had my dream, it would be that one (or more) signing authorities would have potentially overlapping responsibilities for signing parts of the tree either on demand or by requirement.

So when a commit occurs, at least on master, or other designated branches, it may be the repository requires a signature from a particular authority, regardless of whether the committer has requested one. And there may be more than one authority or notary involved. Or, the repository could accept the signature of the committer as abstracted.

Where I'm going is that I would like to see a tighter integration with block-chain concepts in git. My customer base has very tight requirements for this type of software certification. Signatures, GPG or other, may only go so far. I am even considering whether particular parts of the tree are even visible (remember the Islands of Sparceness discussion?).

I expect to be able to contribute more to this conversation in a few months (current $NDA prohibition), if this goes anywhere.

My feature time machine window doesn't see this any time soon, if ever, but one never knows. I have my delusional hopes. 😉

Please take this as simply a suggestion for the long-term.

Cheers,
Randall

-- Brief whoami:
 NonStop developer since approximately 211288444200000000
 UNIX developer since approximately 421664400
-- In my real life, I talk too much.

Re: abstracting commit signing/verify to support other signing schemes

[Tacitus Aedifex]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=unknown-8bit; format=flowed, Size: 3550 bytes --]

On Fri, Aug 03, 2018 at 06:07:46PM -0400, Jeff King wrote:
>There's been some work on this lately. See this patch and the response
>thread:
>
>  https://public-inbox.org/git/20180409204129.43537-9-mastahyeti@gmail.com/
>
>The more recent work focused on just doing the minimum to provide
>gpg/gpgsm variants:
>
>  https://public-inbox.org/git/cover.1531831244.git.henning.schild@siemens.com/
>
>That replaces the earlier patch series, and is currently merged to the
>'next' branch and is on track to get merged to 'master' before Git 2.19
>is released.

thank you for setting the context. it looks like both patch sets are going in 
the same direction that i suggested, at least with the config variables.  
personally, i prefer the 'signingtool.<tool>' approach in the older patch set 
over the 'gpg.<tool>' approach in the newer patch set since my goal is to get 
away from assuming gpg.

the older patch set suggested the idea of using PEM strings to match up the 
signature payload with a certain signing tool.  i can't tell if they mean the 
'pre-ecapsulation boundary' (e.g. '-----BEGIN FOO-----') or if they mean the 
encapsulated headers; both as defined in RFC 1421 [0].

the newer patch set looks specifically at the pre-encapsulation boundary to 
switch behaviors. that works assuming that the signing tools all understand 
PEM. in the case of signify, it doesn't, so the driver code in git will have to 
translate to/from PEM.

i suggest that we switch to a standard format for all signatures that is 
similar to the commit message format with colon ':' separated fields followed 
by a payload.  the colon separated fields would specify the signing tool used 
to generate the signature and the tool specific data followed by the signature 
blob like so:

  signing-tool: gpg
  gpg-keyid: 0123456789ABCDEF
  
  -----BEGIN PGP SIGNATURE-----
  <base64 encoded signature>
  -----END PGP SIGNATURE-----

by adopting this format, git will be fully abstracted from the underlying 
signing tool and the user can specify multiple signing tools in their config 
and git will be able to map the signature to the tool when verifying (e.g. git 
log --show-signature).

a signify signature would look something like this:

  signing-tool: signify
  signify-public-key: <base64 encoded public key>
  
  <base64 encoded signature>

i hope we adopt a more generic approach like this.

>One of the downsides there is that if we eventually move to a generic
>signing-tool config, we'd have to support two layers of historical
>abstraction (the original "gpg.program" config, and the new
>"gpg.<tool>.*" config).

i like the idea of aliasing all of the old config variables to their equivilent 
and outputting a deprecation warning when we get plan on removing the aliases 
altogether in the future.

>So _if_ we knew what it would take to support signify, we could
>potentially adjust what's going into 2.19 in order to skip straight to
>the more generic interface. But on the OTOH, it may not be worth
>rushing, and there is already a vague plan of how the gpg.<tool>.*
>config would interact with a more generic config.

there's no rush, but i would prefer that the newer patch set get changed to use 
the more generic 'signingtool.<tool>.*' instead of 'gpg.<tool>.*'. if you all 
agree, i'll follow up with a patch to change that part of what is going into 
2.19.

then round two will be to experiment with a new, standard signature format that 
doesn't assume anything about the underlying signing tool.

//tæ

[0] https://tools.ietf.org/html/rfc1421

Re: abstracting commit signing/verify to support other signing schemes

[Jeff King]

On Mon, Aug 06, 2018 at 08:24:25PM +0000, Tacitus Aedifex wrote:

> the older patch set suggested the idea of using PEM strings to match up the
> signature payload with a certain signing tool.  i can't tell if they mean
> the 'pre-ecapsulation boundary' (e.g. '-----BEGIN FOO-----') or if they mean
> the encapsulated headers; both as defined in RFC 1421 [0].

It was the pre-encapsulation boundary (we didn't use that word, but it
was definitely the "-----BEGIN" line ;) ).

And that was the sticking point: there was an open question of what
support for something like signify would look like exactly, and what the
matching rules would need to be. My thought was to allow multiple
matching types, and "PEM type" (by which I meant that pre-encapsulation
boundary) would be the first such type.

But that got punted on, since we didn't have a real-world example to
look at, and we really only cared about gpgsm in the near-term anyway.
And that obviously does PEM. So the gpg.* tools all require PEM, but if
we add a generic signingtool config, it doesn't have to.

> the newer patch set looks specifically at the pre-encapsulation boundary to
> switch behaviors. that works assuming that the signing tools all understand
> PEM. in the case of signify, it doesn't, so the driver code in git will have
> to translate to/from PEM.

Right. It might be fine to encapsulate it, though I prefer not inventing
new micro-formats if we can avoid it.

There actually _is_ an interesting micro-format already in Git, which is
that commit signatures (not tags) are held in the "gpgsig" header of the
commit. Which would be an awkward name for storing a non-gpg tool. We
may want to live with that for historical reasons, or we could switch to
a more generic name.

The actual PEM detection is for tags, where the signature is in the tag
body itself.

In either case, we could use some object header to indicate the
signature type (on the other hand, it could be possible to have multiple
signatures of different types).

> i suggest that we switch to a standard format for all signatures that is
> similar to the commit message format with colon ':' separated fields
> followed by a payload.  the colon separated fields would specify the signing
> tool used to generate the signature and the tool specific data followed by
> the signature blob like so:
> 
>  signing-tool: gpg
>  gpg-keyid: 0123456789ABCDEF
>  -----BEGIN PGP SIGNATURE-----
>  <base64 encoded signature>
>  -----END PGP SIGNATURE-----
> 
> by adopting this format, git will be fully abstracted from the underlying
> signing tool and the user can specify multiple signing tools in their config
> and git will be able to map the signature to the tool when verifying (e.g.
> git log --show-signature).

One problem with that for the signatures in tag bodies is that
"signing-tool: gpg" looks like something a human might right (as opposed
to the PEM boundary, which is a bit more obvious).

If we're going to make up a micro-format, it may be simpler to just have
something PEM-like in the first place, and shove signify into that.

> > So _if_ we knew what it would take to support signify, we could
> > potentially adjust what's going into 2.19 in order to skip straight to
> > the more generic interface. But on the OTOH, it may not be worth
> > rushing, and there is already a vague plan of how the gpg.<tool>.*
> > config would interact with a more generic config.
> 
> there's no rush, but i would prefer that the newer patch set get changed to
> use the more generic 'signingtool.<tool>.*' instead of 'gpg.<tool>.*'. if
> you all agree, i'll follow up with a patch to change that part of what is
> going into 2.19.

I'm on the fence for that myself. The best way to get people to comment
would be to make a patch, and cc people involved in the earlier
discussions.

-Peff