SSL Certificate management proposal.

SSL Certificate management proposal.

[Jayanth Othayoth]

[-- Attachment #1: Type: text/plain, Size: 2456 bytes --]

This proposal provides a mechanism to replace the self signed certificate
with  CA signed certificate based on BMC generated CSR.

What is CSR: Certificate Signing Request is a digital request  sent to
certificate authority(CA) to apply for a digital identity certificates,
which includes the  information about the organization info and a unique
identifying key( public key).

OpenSSL tools can be used to generate CSR and corresponding private key.

How to update a signed certificate on the BMC ?

The workflow for updating a signed certificate on the BMC consists of:
1. Generating a CSR on the BMC
2. Exporting the CSR from the BMC onto the user’s storage device
3. Obtaining a singed certificate corresponding to the CSR from a CA
4. Importing the signed certificate on the BMC

The user shall have the ability to export the generated CSR any number of
times until the signed certificate is imported.

In order to support the above workflow, the BMC shall provide the following
REST APIs:

   - Generate CSR
   - Renew CSR
   - Download CSR
   - Upload digitally signed certificate.
   - Activate digitally signed certificates
   - Download digitally signed certificate

Additional Requirements:

   - BMC should store the Signed certificate and private key in a
   persistent secured storage location..
   - Activate process shall validate the new certificate against the
   private key and information in the CSR.
   - Successful certificate activation shall replace the existing
   certificate and private key.
   - A new CSR generate request will overwrite the previous CSR. User
   should take caution to not repeat CSR requests to prevent overwriting
   pending CSRs.
   - Certificate management operation should be restricted for certain
   privilege levels.
   - As part of Boot process,  a new self-signed certificate is generated
   if:


   - No CA signed certificate is present or corrupted.
   - A self-signed certificate is not present or corrupted.
   - The certificate has expired.


   - Validity of the certificate must be ensured by the user or client
   application by periodically checking the expiry date. Certificate
   management feature may be be enhanced in future to generate events when the
   certificate is about to expire.


   - Existing REST session will be tossed during certificate activate
   process.


Note: REST/D-bus details not included here.

[-- Attachment #2: Type: text/html, Size: 2585 bytes --]

Re: SSL Certificate management proposal.

[Ed Tanous]

On 07/31/2018 07:49 AM, Jayanth Othayoth wrote:
> 
> The workflow for updating a signed certificate on the BMC consists of:
> 1. Generating a CSR on the BMC
> 2. Exporting the CSR from the BMC onto the user’s storage device
> 3. Obtaining a singed certificate corresponding to the CSR from a CA
> 4. Importing the signed certificate on the BMC
> 

This workflow is somewhat of a non starter for a lot of organizations 
and IT departments.  First, it requires that every BMC have its own 
private key, which isn't always desired.
Second, it means that you either have to be able to specify all 
parameters that a user would want to specify about his key (number of 
bits, EC vs RSA, ect).
Third, it requires that every BMC be provisioned by the intermediate 
authority, which can be difficult if keys either have a cost associated 
with them, or are difficult to acquire.

I would vote that we should start with the key and certificate upload 
flows, as they are far simpler to implement, and better understood in 
practice then try to go after the BMC CSR implementation if it's still 
required.

> 
> Additional Requirements:
> 
>   * BMC should store the Signed certificate and private key in a
>     persistent secured storage location..

What does this mean exactly?  Encrypted disk backed by hardware 
security?  Is it just implying file system permissions?

>   * Activate process shall validate the new certificate against the
>     private key and information in the CSR.
You don't really define what the "activate process" is in this context. 
I suspect you're talking about a process designed for cert upload, but 
I'm not really sure given the context.

>   * Successful certificate activation shall replace the existing
>     certificate and private key.
>   * A new CSR generate request will overwrite the previous CSR. User
>     should take caution to not repeat CSR requests to prevent
>     overwriting pending CSRs.

If you're going after a CSR flow, I don't think you can do this to a 
user.  Having a CSR immediately overwrite would mean that it invalidated 
the old one, and you have a period of time where your BMC interfaces are 
down while you request a new certificate from your intermediate authority.

>   * Certificate management operation should be restricted for certain
>     privilege levels.

Which privilege levels?  This requirement should be more specific.

>   * As part of Boot process,  a new self-signed certificate is generated if:
> 
>   * No CA signed certificate is present or corrupted.
>   * A self-signed certificate is not present or corrupted.

Agree with these two, and I believe this is how all the web clients 
currently work.

>   * The certificate has expired.
Disagree with this one.  If the certificate has expired AND it was a 
self signed cert, it could be replaced, but it doesn't happen much in 
practice, as self signed certs usually have a very long validity period 
(10 years or more).
If a certificate that a user has uploaded expired, it should be left 
expired until the user decides to replace it.  We shouldn't be changing 
keys and certificates seemingly randomly or automatically.
There's also a question of how you determine if a certificate is 
"expired".  There are no guarantees that the BMC clock is accurate at 
all times, and the potential race conditions in supporting this outweigh 
the benefit.

> 
>   * Validity of the certificate must be ensured by the user or client
>     application by periodically checking the expiry date. Certificate
>     management feature may be be enhanced in future to generate events
>     when the certificate is about to expire.
> 
>   * Existing REST session will be tossed during certificate activate
>     process.
I assume you mean restarted?  I would word this requirement a little 
differently.

* While the web services may restart to begin using the new certificate, 
it shall not interrupt any update flows, and return codes returned by 
all interfaces shall be correct.

> 
> 
> Note: REST/D-bus details not included here.

Re: SSL Certificate management proposal.

[Deepak Kodihalli]

On 31/07/18 11:21 PM, Ed Tanous wrote:
> On 07/31/2018 07:49 AM, Jayanth Othayoth wrote:
>>
>> The workflow for updating a signed certificate on the BMC consists of:
>> 1. Generating a CSR on the BMC
>> 2. Exporting the CSR from the BMC onto the user’s storage device
>> 3. Obtaining a singed certificate corresponding to the CSR from a CA
>> 4. Importing the signed certificate on the BMC
>>
> 
> This workflow is somewhat of a non starter for a lot of organizations 
> and IT departments.  First, it requires that every BMC have its own 
> private key, which isn't always desired.
> Second, it means that you either have to be able to specify all 
> parameters that a user would want to specify about his key (number of 
> bits, EC vs RSA, ect).
> Third, it requires that every BMC be provisioned by the intermediate 
> authority, which can be difficult if keys either have a cost associated 
> with them, or are difficult to acquire.
> 
> I would vote that we should start with the key and certificate upload 
> flows, as they are far simpler to implement, and better understood in 
> practice then try to go after the BMC CSR implementation if it's still 
> required.

We might have to do both (the CSR option if customers prefer the the 
private key never leaves the BMC), but agree with starting with a simple 
certificate and key upload approach. I wanted to discuss the following 
high-level design for the same :

1) OpenBMC Rest/Redfish API to upload certificate and private key, and 
also specify target protocol (see point 2). Synchronous API that lets 
caller know if the upload and installation was successful.
2) Have a json config that lets you know certificate install location 
based on different protocols (these would still be SSL/TLS certificates, 
but based on the protocol, say https or ldap) and/or web-servers being 
used, the certificate may have to be installed at different locations on 
the BMC flash.
3) Rest server copies the certificate to a common location and invokes 
an app via (internal) D-Bus api for certificate activation (lets app 
know the certificate path and the protocol type). The app consumes the 
config from 2).
4) The certificate "activation" would essentially be validating the 
certificate content (via openssl api) and then restarting specific 
systemd units. The app from 3) would perform this, and let the rest 
server know about the status. The app could again be configured based on 
the protocol and the web-server, to determine what service to restart 
(for eg restart nginx).

Regards,
Deepak

Re: SSL Certificate management proposal.

[Jayanth Othayoth]

[-- Attachment #1: Type: text/plain, Size: 5092 bytes --]

On Tue, Jul 31, 2018 at 11:21 PM Ed Tanous <ed.tanous@intel.com> wrote:
>
> On 07/31/2018 07:49 AM, Jayanth Othayoth wrote:
> >
> > The workflow for updating a signed certificate on the BMC consists of:
> > 1. Generating a CSR on the BMC
> > 2. Exporting the CSR from the BMC onto the user’s storage device
> > 3. Obtaining a singed certificate corresponding to the CSR from a CA
> > 4. Importing the signed certificate on the BMC
> >
>
> This workflow is somewhat of a non starter for a lot of organizations
> and IT departments.  First, it requires that every BMC have its own
> private key, which isn't always desired.
> Second, it means that you either have to be able to specify all
> parameters that a user would want to specify about his key (number of
> bits, EC vs RSA, ect).
> Third, it requires that every BMC be provisioned by the intermediate
> authority, which can be difficult if keys either have a cost associated
> with them, or are difficult to acquire.
>

> I would vote that we should start with the key and certificate upload
> flows, as they are far simpler to implement, and better understood in
> practice then try to go after the BMC CSR implementation if it's still
> required.
>
Agree with your concern, but from the security standpoint transmitting
private key over network is not recommended.  Additionally a simplified
method also can be provided for customers who wants that.
> >
> > Additional Requirements:
> >
> >   * BMC should store the Signed certificate and private key in a
> >     persistent secured storage location..
>
> What does this mean exactly?  Encrypted disk backed by hardware
> security?  Is it just implying file system permissions?
Storage method can be varied based on the implementation. Curenlty looking
at file system permissions.
>
> >   * Activate process shall validate the new certificate against the
> >     private key and information in the CSR.
> You don't really define what the "activate process" is in this context.
> I suspect you're talking about a process designed for cert upload, but
> I'm not really sure given the context.
>
Upload process just save the Signed certificate in a  BMC location and
"activation" will replace the curently used certificate in the secured
location and restart all the impacted services.
> >   * Successful certificate activation shall replace the existing
> >     certificate and private key.
> >   * A new CSR generate request will overwrite the previous CSR. User
> >     should take caution to not repeat CSR requests to prevent
> >     overwriting pending CSRs.
>
> If you're going after a CSR flow, I don't think you can do this to a
> user.  Having a CSR immediately overwrite would mean that it invalidated
> the old one, and you have a period of time where your BMC interfaces are
> down while you request a new certificate from your intermediate authority.
>
As mentioned earlier  curently used certificate will be replaced only after
the successful activation. So there wont be any downtime during CSR
creation process.
> >   * Certificate management operation should be restricted for certain
> >     privilege levels.
>
> Which privilege levels?  This requirement should be more specific.
>
User privilege level need to be discussed more.
> >   * As part of Boot process,  a new self-signed certificate is
generated if:
> >
> >   * No CA signed certificate is present or corrupted.
> >   * A self-signed certificate is not present or corrupted.
>
> Agree with these two, and I believe this is how all the web clients
> currently work.
>
> >   * The certificate has expired.
> Disagree with this one.  If the certificate has expired AND it was a
> self signed cert, it could be replaced, but it doesn't happen much in
> practice, as self signed certs usually have a very long validity period
> (10 years or more).
> If a certificate that a user has uploaded expired, it should be left
> expired until the user decides to replace it.  We shouldn't be changing
> keys and certificates seemingly randomly or automatically.
> There's also a question of how you determine if a certificate is
> "expired".  There are no guarantees that the BMC clock is accurate at
> all times, and the potential race conditions in supporting this outweigh
> the benefit.

Agreed.
> >
> >   * Validity of the certificate must be ensured by the user or client
> >     application by periodically checking the expiry date. Certificate
> >     management feature may be be enhanced in future to generate events
> >     when the certificate is about to expire.
> >
> >   * Existing REST session will be tossed during certificate activate
> >     process.
> I assume you mean restarted?  I would word this requirement a little
> differently.
>
> * While the web services may restart to begin using the new certificate,
> it shall not interrupt any update flows, and return codes returned by
> all interfaces shall be correct.
>
Agreed.
> >
> >
> > Note: REST/D-bus details not included here.

[-- Attachment #2: Type: text/html, Size: 6172 bytes --]

Re: SSL Certificate management proposal.

[Hariharasubramanian Ramasubramanian]

[-- Attachment #1: Type: text/plain, Size: 1934 bytes --]

"openbmc" <openbmc-bounces+hramasub=in.ibm.com@lists.ozlabs.org> wrote on 
07/31/2018 11:21:27 PM:

> From: Ed Tanous <ed.tanous@intel.com>
> To: Jayanth Othayoth <ojayanth@gmail.com>, openbmc@lists.ozlabs.org,
> bradleyb@fuzziesquirrel.com
> Date: 07/31/2018 11:21 PM
> Subject: Re: SSL Certificate management proposal.
> Sent by: "openbmc" 
<openbmc-bounces+hramasub=in.ibm.com@lists.ozlabs.org>
> 
> On 07/31/2018 07:49 AM, Jayanth Othayoth wrote:
> > 
> > The workflow for updating a signed certificate on the BMC consists of:
> > 1. Generating a CSR on the BMC
> > 2. Exporting the CSR from the BMC onto the user?s storage device
> > 3. Obtaining a singed certificate corresponding to the CSR from a CA
> > 4. Importing the signed certificate on the BMC
> > 
> 
> This workflow is somewhat of a non starter for a lot of organizations 
> and IT departments. 
Could you please share your thoughts on why this is so ?

> First, it requires that every BMC have its own 
> private key, which isn't always desired.
_Sharing_ the private is a potential security risk. If the private key 
were to be compromised, then _all_ systems sharing the key are exposed. 
Hence I'm inclined to argue that private key should not be shared.

> Second, it means that you either have to be able to specify all 
> parameters that a user would want to specify about his key (number of 
> bits, EC vs RSA, ect).
What really is the concern here ? What's the challenge in invoking the 
REST API with a json body describing the key type & size ?

> Third, it requires that every BMC be provisioned by the intermediate 
> authority, which can be difficult if keys either have a cost associated 
> with them, or are difficult to acquire.
The cost considerations could potentially be different for Enterprise 
class servers and servers deployed on the cloud ? I presume enterprises 
would favour security to cost.



[-- Attachment #2: Type: text/html, Size: 2547 bytes --]

Re: SSL Certificate management proposal.

[Ed Tanous]

>  >
> Agree with your concern, but from the security standpoint transmitting 
> private key over network is not recommended.  Additionally a simplified 
> method also can be provided for customers who wants that.
I'm happy with both, and I understand the desire to not transmit private 
keys over the network, but this is how a lot of organizations are set up 
today.  In general, the private key is sent over TLS, during an initial 
provisioning phase, on an isolated subnet to reduce the risk.  I'm not 
saying the CSR approach isn't an improvement, but I think we will need 
to support both approaches, and I don't know of any BMC that supports 
the CSR approach, although I'm not very familiar with some of the high 
end enterprise BMCs feature set.  It might be supported.

>  >
>  > What does this mean exactly?  Encrypted disk backed by hardware
>  > security?  Is it just implying file system permissions?
> Storage method can be varied based on the implementation. Curenlty 
> looking at file system permissions.
Got it.

>  >
>  > >   * Activate process shall validate the new certificate against the
>  > >     private key and information in the CSR.
>  > You don't really define what the "activate process" is in this context.
>  > I suspect you're talking about a process designed for cert upload, but
>  > I'm not really sure given the context.
>  >
> Upload process just save the Signed certificate in a  BMC location and 
> "activation" will replace the curently used certificate in the secured 
> location and restart all the impacted services.
Why have these as two steps?  Why not just upload the key and apply it 
in one atomic action?  It seems like it would simplify the code quite a 
bit, and reduce the possibility of issues.

>  >
> As mentioned earlier  curently used certificate will be replaced only 
> after the successful activation. So there wont be any downtime during 
> CSR creation process.
If I'm following correctly, the CSR gets replaced, but not the 
certificate?  If so, I misunderstood your initial requirement.  This 
should be fine.

>  >
> User privilege level need to be discussed more.
Agreed.



All in all, sounds like a good start to me.

Re: SSL Certificate management proposal.

[Ed Tanous]

On 08/01/2018 06:05 AM, Hariharasubramanian Ramasubramanian wrote:
> 
>  > Third, it requires that every BMC be provisioned by the intermediate
>  > authority, which can be difficult if keys either have a cost associated
>  > with them, or are difficult to acquire.
> The cost considerations could potentially be different for Enterprise 
> class servers and servers deployed on the cloud ? I presume enterprises 
> would favour security to cost.
> 

Varies greatly between organizations, and how they manage their threat 
model.  I think that we should support both.

Re: SSL Certificate management proposal.

[Jayanth Othayoth]

[-- Attachment #1: Type: text/plain, Size: 870 bytes --]

Initial version of certificate management design document is available for
review.
https://gerrit.openbmc-project.xyz/#/c/openbmc/phosphor-dbus-interfaces/+/11840/1/xyz/openbmc_project/Certificate/README.md

On Wed, Aug 1, 2018 at 10:19 PM Ed Tanous <ed.tanous@intel.com> wrote:

> On 08/01/2018 06:05 AM, Hariharasubramanian Ramasubramanian wrote:
> >
> >  > Third, it requires that every BMC be provisioned by the intermediate
> >  > authority, which can be difficult if keys either have a cost
> associated
> >  > with them, or are difficult to acquire.
> > The cost considerations could potentially be different for Enterprise
> > class servers and servers deployed on the cloud ? I presume enterprises
> > would favour security to cost.
> >
>
> Varies greatly between organizations, and how they manage their threat
> model.  I think that we should support both.
>

[-- Attachment #2: Type: text/html, Size: 1336 bytes --]