All of lore.kernel.org
 help / color / mirror / Atom feed
* Security Working Group meeting - Wednesday August 4
@ 2021-08-03 22:57 Joseph Reynolds
  2021-08-04  3:04 ` Patrick Williams
                   ` (3 more replies)
  0 siblings, 4 replies; 17+ messages in thread
From: Joseph Reynolds @ 2021-08-03 22:57 UTC (permalink / raw)
  To: openbmc

This is a reminder of the OpenBMC Security Working Group meeting 
scheduled for this Wednesday August 4 at 10:00am PDT.

We'll discuss the following items on the agenda 
<https://docs.google.com/document/d/1b7x9BaxsfcukQDqbvZsU2ehMq4xoJRQvLxxsDUWmAOI/edit>, 
and anything else that comes up:

 1. (Joseph): IBM ACF design (2FA authentication for the special IBM
    service account) is in review -
    https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/45201
    <https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/45201>
 2. (Joseph): Updated password hash algorithm from MD5 to SHA512 (while
    keeping the same cleartext password)
    https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/45214
    <https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/45214>
 3. (Joseph): Change the SSH server per-session idle timeout to an hour
    (was unlimited)?  (Sent idea to upstream project
    yocto-security@yoctoproject.org
    <mailto:yocto-security@yoctoproject.org>.)  Alternatively, update
    both SSH and BMCWeb to 30 minutes.
     1. Guidelines:
         1. NIST SP800-63B requires a timeout of 30 minutes for
            "assurance level 2" (high confidence that the authentication
            is still valid), or 15 minutes for "assurance level 2" (very
            high confidence).
            https://pages.nist.gov/800-63-3/sp800-63b.html
            <https://pages.nist.gov/800-63-3/sp800-63b.html>
         2. OWASP suggests idle timeouts of 15-30 minutes.
            https://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.html#session-expiration
            <https://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.html#session-expiration>
     2. Alternatively, use the bash shell’s TMOUT variable?
     3. See Yocto discussion (representative archived email):
        https://lists.yoctoproject.org/g/yocto-security/message/381
        <https://lists.yoctoproject.org/g/yocto-security/message/381>



Access, agenda and notes are in the wiki:
https://github.com/openbmc/openbmc/wiki/Security-working-group 
<https://github.com/openbmc/openbmc/wiki/Security-working-group>

- Joseph

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4
  2021-08-03 22:57 Security Working Group meeting - Wednesday August 4 Joseph Reynolds
@ 2021-08-04  3:04 ` Patrick Williams
  2021-08-04  3:22 ` Patrick Williams
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 17+ messages in thread
From: Patrick Williams @ 2021-08-04  3:04 UTC (permalink / raw)
  To: Joseph Reynolds; +Cc: openbmc

[-- Attachment #1: Type: text/plain, Size: 2497 bytes --]

On Tue, Aug 03, 2021 at 05:57:52PM -0500, Joseph Reynolds wrote:

>  3. (Joseph): Change the SSH server per-session idle timeout to an hour
>     (was unlimited)?  (Sent idea to upstream project
>     yocto-security@yoctoproject.org
>     <mailto:yocto-security@yoctoproject.org>.)  Alternatively, update
>     both SSH and BMCWeb to 30 minutes.

Facebook has had this implemented in our BMC for a long time.  We use to
have to patch SSH but that stopped working and we ended up using the TMOUT
variable.  Relevant commits are [1,2].

>      1. Guidelines:
>          1. NIST SP800-63B requires a timeout of 30 minutes for
>             "assurance level 2" (high confidence that the authentication
>             is still valid), or 15 minutes for "assurance level 2" (very
>             high confidence).
>             https://pages.nist.gov/800-63-3/sp800-63b.html
>             <https://pages.nist.gov/800-63-3/sp800-63b.html>
>          2. OWASP suggests idle timeouts of 15-30 minutes.
>             https://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.html#session-expiration
>             <https://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.html#session-expiration>

15 minutes seems more than enough to me.  We have ours set to 5 minutes on the
console and 30 minutes on the SSH, but I think those are relatively arbitrary.
Ideally whatever you implement can be configured with a Yocto variable so if
someone feels your choice is "wrong" they can easily override it in their own
machine.

>      2. Alternatively, use the bash shell’s TMOUT variable?

Whatever you do, I think you need to take into account the serial console as
well.  Not just SSH.

>      3. See Yocto discussion (representative archived email):
>         https://lists.yoctoproject.org/g/yocto-security/message/381
>         <https://lists.yoctoproject.org/g/yocto-security/message/381>

I agree with Richard even in the context of OpenBMC itself:

   > There is never going to be one "right" solution for everyone but 
   > making it easy/clear for users to do it would be ideal (which includes
   > making it easy for OpenBMC to configure what they need).

Whatever you pick someone is going to argue it is wrong.

1. https://github.com/facebook/openbmc/commit/8171ad7183269e3050f7f37b9b3956ce54b0ee87
2. https://github.com/facebook/openbmc/commit/59d7b23a9c2aa08efde19f913df446a82e1f6804

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4
  2021-08-03 22:57 Security Working Group meeting - Wednesday August 4 Joseph Reynolds
  2021-08-04  3:04 ` Patrick Williams
@ 2021-08-04  3:22 ` Patrick Williams
  2021-08-09 14:09   ` Security Working Group meeting - Wednesday August 4 - ibm-acf repo Joseph Reynolds
  2021-08-04  3:28 ` Security Working Group meeting - Wednesday August 4 Patrick Williams
  2021-08-04 18:47 ` Security Working Group meeting - Wednesday August 4 - results Joseph Reynolds
  3 siblings, 1 reply; 17+ messages in thread
From: Patrick Williams @ 2021-08-04  3:22 UTC (permalink / raw)
  To: Joseph Reynolds; +Cc: openbmc

[-- Attachment #1: Type: text/plain, Size: 1776 bytes --]

On Tue, Aug 03, 2021 at 05:57:52PM -0500, Joseph Reynolds wrote:
> This is a reminder of the OpenBMC Security Working Group meeting 
> scheduled for this Wednesday August 4 at 10:00am PDT.
> 
> We'll discuss the following items on the agenda 
> <https://docs.google.com/document/d/1b7x9BaxsfcukQDqbvZsU2ehMq4xoJRQvLxxsDUWmAOI/edit>, 
> and anything else that comes up:
> 
>  1. (Joseph): IBM ACF design (2FA authentication for the special IBM
>     service account) is in review -
>     https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/45201
>     <https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/45201>

I still feel like the "Alternatives considered" are pretty weak in this
document.  Rather than paint broad brushes ("Other were considered.  They were
not suitable.") I think you should enumerate _which_ alternatives were
considered and _why_ they don't fit the problem at hand.

```
- Takes four parameters: machine serial number, expiration date, password, and
  private key.
- Algorithm:
   - Hashes the password.
   - Creates the ACF from the hashed password, serial number, and expiration date.
   - Digitally signs the ACF using the private key.
   - Returns the ACF to the caller.
```

This sounds a lot like U2F.  The "4 parameters" are the challenge, IBM's key
signing server is the U2F device, and PAM is the "Relying Party".  There are
already PAM modules for some aspects of U2F and the token you need to exchange
is reasonably short (my Yubikey output is 33 characters).

https://developers.yubico.com/U2F/Protocol_details/Overview.html

The nice aspect if you can reuse portions of the U2F protocol is that you go a
long way towards enabling others to add more typical 2FA like Yubikeys.

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4
  2021-08-03 22:57 Security Working Group meeting - Wednesday August 4 Joseph Reynolds
  2021-08-04  3:04 ` Patrick Williams
  2021-08-04  3:22 ` Patrick Williams
@ 2021-08-04  3:28 ` Patrick Williams
  2021-08-04 18:43   ` Security Working Group meeting - Wednesday August 4 - all distro owners please review Joseph Reynolds
  2021-08-04 18:47 ` Security Working Group meeting - Wednesday August 4 - results Joseph Reynolds
  3 siblings, 1 reply; 17+ messages in thread
From: Patrick Williams @ 2021-08-04  3:28 UTC (permalink / raw)
  To: Joseph Reynolds; +Cc: openbmc

[-- Attachment #1: Type: text/plain, Size: 5187 bytes --]

On Tue, Aug 03, 2021 at 05:57:52PM -0500, Joseph Reynolds wrote:
>  2. (Joseph): Updated password hash algorithm from MD5 to SHA512 (while
>     keeping the same cleartext password)
>     https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/45214
>     <https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/45214>

The big hangup on this one right now is that there are tons of overrides of
EXTRA_USERS_PARAMS in a bunch of `local.conf.sample` files.  Does anyone know
why these are there rather than relying on the ones we get for free from the
meta-phosphor layer?  Does anyone have a problem if they are removed?

```
$ find -name local.conf.sample | xargs grep usermod -B1
./meta-evb/meta-evb-aspeed/meta-evb-ast2500/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-evb/meta-evb-aspeed/meta-evb-ast2500/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-evb/meta-evb-aspeed/meta-evb-ast2600/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-evb/meta-evb-aspeed/meta-evb-ast2600/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-evb/meta-evb-enclustra/meta-evb-zx3-pm3/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-evb/meta-evb-enclustra/meta-evb-zx3-pm3/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-evb/meta-evb-nuvoton/meta-evb-npcm750/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-evb/meta-evb-nuvoton/meta-evb-npcm750/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-evb/meta-evb-raspberrypi/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-evb/meta-evb-raspberrypi/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-facebook/meta-tiogapass/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-facebook/meta-tiogapass/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-facebook/meta-yosemitev2/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-facebook/meta-yosemitev2/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-facebook/meta-bletchley/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-facebook/meta-bletchley/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-hxt/meta-stardragon4800-rep2/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-hxt/meta-stardragon4800-rep2/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-inspur/meta-on5263m5/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-inspur/meta-on5263m5/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-inventec/meta-lanyang/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-inventec/meta-lanyang/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-portwell/meta-neptune/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-portwell/meta-neptune/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-qualcomm/meta-centriq2400-rep/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-qualcomm/meta-centriq2400-rep/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-quanta/meta-q71l/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-quanta/meta-q71l/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-lenovo/meta-hr630/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-lenovo/meta-hr630/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-lenovo/meta-hr855xg2/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-lenovo/meta-hr855xg2/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-microsoft/meta-olympus/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-microsoft/meta-olympus/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-hpe/meta-dl360poc/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-hpe/meta-dl360poc/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-supermicro/meta-x11spi/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-supermicro/meta-x11spi/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
--
./meta-asrock/meta-e3c246d4i/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
./meta-asrock/meta-e3c246d4i/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
```

My feeling is that the majority of them can be removed and were probably just
copy/paste jobs from somewhere else.  The only ones that we may need to
reconsider are the ones in meta-evb because I'm not 100% convinced that those
machines always use meta-phosphor.

I haven't had time to yet, but I would suggest just making a commit to delete
all of these and see who is outraged by it.

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4 - all distro owners please review
  2021-08-04  3:28 ` Security Working Group meeting - Wednesday August 4 Patrick Williams
@ 2021-08-04 18:43   ` Joseph Reynolds
  0 siblings, 0 replies; 17+ messages in thread
From: Joseph Reynolds @ 2021-08-04 18:43 UTC (permalink / raw)
  To: Patrick Williams; +Cc: openbmc



On 8/3/21 10:28 PM, Patrick Williams wrote:
> On Tue, Aug 03, 2021 at 05:57:52PM -0500, Joseph Reynolds wrote:
>>   2. (Joseph): Updated password hash algorithm from MD5 to SHA512 (while
>>      keeping the same cleartext password)
>>      https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/45214
>>      <https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/45214>
> The big hangup on this one right now is that there are tons of overrides of
> EXTRA_USERS_PARAMS in a bunch of `local.conf.sample` files.  Does anyone know
> why these are there rather than relying on the ones we get for free from the
> meta-phosphor layer?  Does anyone have a problem if they are removed?
>
> ```
> $ find -name local.conf.sample | xargs grep usermod -B1
> ./meta-evb/meta-evb-aspeed/meta-evb-ast2500/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
> ./meta-evb/meta-evb-aspeed/meta-evb-ast2500/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
> --
...snip...
> --
> ./meta-asrock/meta-e3c246d4i/conf/local.conf.sample-EXTRA_USERS_PARAMS = " \
> ./meta-asrock/meta-e3c246d4i/conf/local.conf.sample:  usermod -p '\$1\$UGMqyqdG\$FZiylVFmRRfl9Z0Ue8G7e/' root; \
> ```
>
> My feeling is that the majority of them can be removed and were probably just
> copy/paste jobs from somewhere else.  The only ones that we may need to
> reconsider are the ones in meta-evb because I'm not 100% convinced that those
> machines always use meta-phosphor.
>
> I haven't had time to yet, but I would suggest just making a commit to delete
> all of these and see who is outraged by it.

Although I don't see my role as to cause outrage, I created the commit here:
Please review https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/45614

- Joseph


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4 - results
  2021-08-03 22:57 Security Working Group meeting - Wednesday August 4 Joseph Reynolds
                   ` (2 preceding siblings ...)
  2021-08-04  3:28 ` Security Working Group meeting - Wednesday August 4 Patrick Williams
@ 2021-08-04 18:47 ` Joseph Reynolds
  2021-08-04 20:09   ` Patrick Williams
  2021-08-05 15:54   ` Brad Bishop
  3 siblings, 2 replies; 17+ messages in thread
From: Joseph Reynolds @ 2021-08-04 18:47 UTC (permalink / raw)
  To: openbmc

On 8/3/21 5:57 PM, Joseph Reynolds wrote:
> This is a reminder of the OpenBMC Security Working Group meeting 
> scheduled for this Wednesday August 4 at 10:00am PDT.
>
> We'll discuss the following items on the agenda 
> <https://docs.google.com/document/d/1b7x9BaxsfcukQDqbvZsU2ehMq4xoJRQvLxxsDUWmAOI/edit>, 
> and anything else that comes up:
>
> 1. (Joseph): IBM ACF design (2FA authentication for the special IBM
>   service account) is in review -
>   https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/45201
> <https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/45201>

DISCUSSION: Joseph gave a brief overview with Q&A.


> 2. (Joseph): Updated password hash algorithm from MD5 to SHA512 (while
>   keeping the same cleartext password)
>   https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/45214
> <https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/45214>

DISCUSSION: Joseph gave a brief overview and mentioned the pre-requisite 
patch https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/45614 
<https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/45614>.  Please 
review!

(Note there is a related email thread for this.)

> 3. (Joseph): Change the SSH server per-session idle timeout to an hour
>   (was unlimited)?  (Sent idea to upstream project
>   yocto-security@yoctoproject.org
>   <mailto:yocto-security@yoctoproject.org>.)  Alternatively, update
>   both SSH and BMCWeb to 30 minutes.
>    1. Guidelines:
>        1. NIST SP800-63B requires a timeout of 30 minutes for
>           "assurance level 2" (high confidence that the authentication
>           is still valid), or 15 minutes for "assurance level 2" (very
>           high confidence).
>           https://pages.nist.gov/800-63-3/sp800-63b.html
>           <https://pages.nist.gov/800-63-3/sp800-63b.html>
>        2. OWASP suggests idle timeouts of 15-30 minutes.
> https://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.html#session-expiration
> <https://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.html#session-expiration>
>    2. Alternatively, use the bash shell’s TMOUT variable?
>    3. See Yocto discussion (representative archived email):
>       https://lists.yoctoproject.org/g/yocto-security/message/381
> <https://lists.yoctoproject.org/g/yocto-security/message/381>

DISCUSSION:

There was general agreement that OpenBMC should set a default idle timeout:

  *

    Must be able to configure each interface separately: SSH port 22
    (BMC command shell), SSH port 2200 (host console).

  *

    30 minutes was suggested for the command shell.

  *

    The BMC admin should be able to configure the timeout.  Need to
    check if there is a Redfish API or property for this.

  *

    The technology to have a timeout may be present in the SSH server,
    the underlying application (command shell, host console, etc.), or
    provided by an intervening program such as “screen”.

Joseph to follow up via email.

We also discussed the risks of allowing SSH at all.


Bonus topics:

4 Surya set up a bugzilla within Intel and will administer it.  Demo’d 
the database. We briefly examined the database fields and agreed it 
looks like a good start.

Who has access?: The security response team (see Joseph as admin).  Also 
the bug submitter and the bug fixer will have access to each of their bugs.


https://github.com/openbmc/docs/blob/master/security/obmc-security-response-team.md 
<https://github.com/openbmc/docs/blob/master/security/obmc-security-response-team.md>

Side discussion: Can we add a security responder from Nvidia?  Yes, 
first review See 
https://github.com/openbmc/docs/blob/master/security/obmc-security-response-team-guidelines.md#team-composition-and-email-maintenance 
<https://github.com/openbmc/docs/blob/master/security/obmc-security-response-team-guidelines.md#team-composition-and-email-maintenance>

  And then petition the TSC via email: 
https://github.com/openbmc/openbmc#technical-steering-committee 
<https://github.com/openbmc/openbmc#technical-steering-committee>.


5 How to escalate bugs reported to the security response team?

DISCUSSION: We briefly discussed this as the meeting time was past the 
end.  It is hard to make people fix bugs.  Ideas: keep sending reminder 
emails, and try to get someone to fix the bug.



>
>
>
> Access, agenda and notes are in the wiki:
> https://github.com/openbmc/openbmc/wiki/Security-working-group 
> <https://github.com/openbmc/openbmc/wiki/Security-working-group>
>
> - Joseph


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4 - results
  2021-08-04 18:47 ` Security Working Group meeting - Wednesday August 4 - results Joseph Reynolds
@ 2021-08-04 20:09   ` Patrick Williams
  2021-08-04 20:39     ` Joseph Reynolds
  2021-08-05 15:54   ` Brad Bishop
  1 sibling, 1 reply; 17+ messages in thread
From: Patrick Williams @ 2021-08-04 20:09 UTC (permalink / raw)
  To: Joseph Reynolds; +Cc: openbmc, Brad Bishop

[-- Attachment #1: Type: text/plain, Size: 516 bytes --]

On Wed, Aug 04, 2021 at 01:47:31PM -0500, Joseph Reynolds wrote:
 
> 4 Surya set up a bugzilla within Intel and will administer it.  Demo’d 
> the database. We briefly examined the database fields and agreed it 
> looks like a good start.
> 

Once again I'll ask ***WHY***??!?

https://lore.kernel.org/openbmc/YNzsE1ipYQR7yfDq@heinlein/
https://lore.kernel.org/openbmc/YPiK8xqFPJFZDa1+@heinlein/

Can we please create a private Github repository and be done with this topic?

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4 - results
  2021-08-04 20:09   ` Patrick Williams
@ 2021-08-04 20:39     ` Joseph Reynolds
  2021-08-04 20:49       ` Patrick Williams
  2021-08-05 13:43       ` Brad Bishop
  0 siblings, 2 replies; 17+ messages in thread
From: Joseph Reynolds @ 2021-08-04 20:39 UTC (permalink / raw)
  To: Patrick Williams; +Cc: openbmc, Brad Bishop

On 8/4/21 3:09 PM, Patrick Williams wrote:
> On Wed, Aug 04, 2021 at 01:47:31PM -0500, Joseph Reynolds wrote:
>   
>> 4 Surya set up a bugzilla within Intel and will administer it.  Demo’d
>> the database. We briefly examined the database fields and agreed it
>> looks like a good start.
>>
> Once again I'll ask ***WHY***??!?
>
> https://lore.kernel.org/openbmc/YNzsE1ipYQR7yfDq@heinlein/
> https://lore.kernel.org/openbmc/YPiK8xqFPJFZDa1+@heinlein/
>
> Can we please create a private Github repository and be done with this topic?

I don't have any insight into how to resolve this question.

 From today's meeting: using bugzilla has advantages over github issues:
- lets us define the fields we need: fix commitID, CVSS score, etc.
- has desirable access controls, specifically acess by the security 
respone tram plus we can add access for the problem submitter and the 
problem fixer

- Joseph

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4 - results
  2021-08-04 20:39     ` Joseph Reynolds
@ 2021-08-04 20:49       ` Patrick Williams
  2021-08-04 23:23         ` Patrick Williams
  2021-08-04 23:47         ` Andrew Jeffery
  2021-08-05 13:43       ` Brad Bishop
  1 sibling, 2 replies; 17+ messages in thread
From: Patrick Williams @ 2021-08-04 20:49 UTC (permalink / raw)
  To: Joseph Reynolds; +Cc: openbmc, Brad Bishop

[-- Attachment #1: Type: text/plain, Size: 1386 bytes --]

On Wed, Aug 04, 2021 at 03:39:45PM -0500, Joseph Reynolds wrote:
> On 8/4/21 3:09 PM, Patrick Williams wrote:
> > On Wed, Aug 04, 2021 at 01:47:31PM -0500, Joseph Reynolds wrote:
> >   
> >> 4 Surya set up a bugzilla within Intel and will administer it.  Demo’d
> >> the database. We briefly examined the database fields and agreed it
> >> looks like a good start.
> >>
> > Once again I'll ask ***WHY***??!?
> >
> > https://lore.kernel.org/openbmc/YNzsE1ipYQR7yfDq@heinlein/
> > https://lore.kernel.org/openbmc/YPiK8xqFPJFZDa1+@heinlein/
> >
> > Can we please create a private Github repository and be done with this topic?
> 
> I don't have any insight into how to resolve this question.
> 
>  From today's meeting: using bugzilla has advantages over github issues:
> - lets us define the fields we need: fix commitID, CVSS score, etc.

These are pretty minor when you could just add a comment template with this
information.

> - has desirable access controls, specifically acess by the security 
> respone tram plus we can add access for the problem submitter and the 
> problem fixer

So does Github.

----

I really don't think that some subset of the community should go off on their
own bug tracking system.  This is a waste of time to maintain and just further
segments this "Security Team" off in their own bubble.

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4 - results
  2021-08-04 20:49       ` Patrick Williams
@ 2021-08-04 23:23         ` Patrick Williams
  2021-08-06 17:10           ` Mihm, James
  2021-08-04 23:47         ` Andrew Jeffery
  1 sibling, 1 reply; 17+ messages in thread
From: Patrick Williams @ 2021-08-04 23:23 UTC (permalink / raw)
  To: Joseph Reynolds; +Cc: openbmc, Brad Bishop

[-- Attachment #1: Type: text/plain, Size: 1638 bytes --]

Has this been read through?

https://docs.github.com/en/code-security/security-advisories/about-github-security-advisories

> On Aug 4, 2021, at 3:49 PM, Patrick Williams <patrick@stwcx.xyz> wrote:
> 
> On Wed, Aug 04, 2021 at 03:39:45PM -0500, Joseph Reynolds wrote:
>>> On 8/4/21 3:09 PM, Patrick Williams wrote:
>>>> On Wed, Aug 04, 2021 at 01:47:31PM -0500, Joseph Reynolds wrote:
>>> 
>>>> 4 Surya set up a bugzilla within Intel and will administer it.  Demo’d
>>>> the database. We briefly examined the database fields and agreed it
>>>> looks like a good start.
>>>> 
>>> Once again I'll ask ***WHY***??!?
>>> 
>>> https://lore.kernel.org/openbmc/YNzsE1ipYQR7yfDq@heinlein/
>>> https://lore.kernel.org/openbmc/YPiK8xqFPJFZDa1+@heinlein/
>>> 
>>> Can we please create a private Github repository and be done with this topic?
>> 
>> I don't have any insight into how to resolve this question.
>> 
>> From today's meeting: using bugzilla has advantages over github issues:
>> - lets us define the fields we need: fix commitID, CVSS score, etc.
> 
> These are pretty minor when you could just add a comment template with this
> information.
> 
>> - has desirable access controls, specifically acess by the security 
>> respone tram plus we can add access for the problem submitter and the 
>> problem fixer
> 
> So does Github.
> 
> ----
> 
> I really don't think that some subset of the community should go off on their
> own bug tracking system.  This is a waste of time to maintain and just further
> segments this "Security Team" off in their own bubble.
> 
> -- 
> Patrick Williams

[-- Attachment #2: Type: text/html, Size: 3833 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4 - results
  2021-08-04 20:49       ` Patrick Williams
  2021-08-04 23:23         ` Patrick Williams
@ 2021-08-04 23:47         ` Andrew Jeffery
  2021-08-04 23:57           ` Ed Tanous
  1 sibling, 1 reply; 17+ messages in thread
From: Andrew Jeffery @ 2021-08-04 23:47 UTC (permalink / raw)
  To: Patrick Williams, Joseph Reynolds; +Cc: openbmc, Brad Bishop



On Thu, 5 Aug 2021, at 06:19, Patrick Williams wrote:
> On Wed, Aug 04, 2021 at 03:39:45PM -0500, Joseph Reynolds wrote:
> > On 8/4/21 3:09 PM, Patrick Williams wrote:
> > > On Wed, Aug 04, 2021 at 01:47:31PM -0500, Joseph Reynolds wrote:
> > >   
> > >> 4 Surya set up a bugzilla within Intel and will administer it.  Demo’d
> > >> the database. We briefly examined the database fields and agreed it
> > >> looks like a good start.
> > >>
> > > Once again I'll ask ***WHY***??!?
> > >
> > > https://lore.kernel.org/openbmc/YNzsE1ipYQR7yfDq@heinlein/
> > > https://lore.kernel.org/openbmc/YPiK8xqFPJFZDa1+@heinlein/
> > >
> > > Can we please create a private Github repository and be done with this topic?
> > 
> > I don't have any insight into how to resolve this question.
> > 
> >  From today's meeting: using bugzilla has advantages over github issues:
> > - lets us define the fields we need: fix commitID, CVSS score, etc.
> 
> These are pretty minor when you could just add a comment template with this
> information.
> 
> > - has desirable access controls, specifically acess by the security 
> > respone tram plus we can add access for the problem submitter and the 
> > problem fixer
> 
> So does Github.
> 
> ----
> 
> I really don't think that some subset of the community should go off on their
> own bug tracking system. 

+1

I'm not aware of any effort to use Github security advisories so far. I 
think we should try that before burdening ourselves and any bug 
reporters with yet more disjoint bits of infrastructure (we already
have the two mailing lists, discord, gerrit, and github).

Andrew

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4 - results
  2021-08-04 23:47         ` Andrew Jeffery
@ 2021-08-04 23:57           ` Ed Tanous
  2021-08-05 13:55             ` Brad Bishop
  0 siblings, 1 reply; 17+ messages in thread
From: Ed Tanous @ 2021-08-04 23:57 UTC (permalink / raw)
  To: Andrew Jeffery; +Cc: openbmc, Brad Bishop, Joseph Reynolds

On Wed, Aug 4, 2021 at 4:48 PM Andrew Jeffery <andrew@aj.id.au> wrote:
>
>
>
> On Thu, 5 Aug 2021, at 06:19, Patrick Williams wrote:
> > On Wed, Aug 04, 2021 at 03:39:45PM -0500, Joseph Reynolds wrote:
> > > On 8/4/21 3:09 PM, Patrick Williams wrote:
> > > > On Wed, Aug 04, 2021 at 01:47:31PM -0500, Joseph Reynolds wrote:
> > > >
> > > >> 4 Surya set up a bugzilla within Intel and will administer it.  Demo’d
> > > >> the database. We briefly examined the database fields and agreed it
> > > >> looks like a good start.
> > > >>
> > > > Once again I'll ask ***WHY***??!?
> > > >
> > > > https://lore.kernel.org/openbmc/YNzsE1ipYQR7yfDq@heinlein/
> > > > https://lore.kernel.org/openbmc/YPiK8xqFPJFZDa1+@heinlein/
> > > >
> > > > Can we please create a private Github repository and be done with this topic?
> > >
> > > I don't have any insight into how to resolve this question.
> > >
> > >  From today's meeting: using bugzilla has advantages over github issues:
> > > - lets us define the fields we need: fix commitID, CVSS score, etc.

I'm also really not following this as a rationale for starting a
completely new server.  This is easy enough to add on github in the
bug template.

> >
> > These are pretty minor when you could just add a comment template with this
> > information.
> >
> > > - has desirable access controls, specifically acess by the security
> > > respone tram plus we can add access for the problem submitter and the
> > > problem fixer
> >
> > So does Github.

+1

> >
> > ----
> >
> > I really don't think that some subset of the community should go off on their
> > own bug tracking system.
>
> +1
>
> I'm not aware of any effort to use Github security advisories so far. I
> think we should try that before burdening ourselves and any bug
> reporters with yet more disjoint bits of infrastructure (we already
> have the two mailing lists, discord, gerrit, and github).

+1

>
> Andrew

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4 - results
  2021-08-04 20:39     ` Joseph Reynolds
  2021-08-04 20:49       ` Patrick Williams
@ 2021-08-05 13:43       ` Brad Bishop
  1 sibling, 0 replies; 17+ messages in thread
From: Brad Bishop @ 2021-08-05 13:43 UTC (permalink / raw)
  To: Joseph Reynolds, James Mihm, Sekar, Suryakanth; +Cc: openbmc

On Wed, 2021-08-04 at 15:39 -0500, Joseph Reynolds wrote:
> On 8/4/21 3:09 PM, Patrick Williams wrote:
> > 
> > Can we please create a private Github repository and be done with
> > this topic?
> 
> I don't have any insight into how to resolve this question.

I think the first step is to bring the people actually doing the work
into the discussion.

James, Surya

First - I appreciate you setting up a security database on behalf of the
project.

> From today's meeting: using bugzilla has advantages over github issues

Ok, but now several community members have voiced concerns and
alternatives that address those concerns.  Can you please engage with
them and either consider the alternatives and/or explain why you don't
think the concerns are relevant?

-brad


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4 - results
  2021-08-04 23:57           ` Ed Tanous
@ 2021-08-05 13:55             ` Brad Bishop
  0 siblings, 0 replies; 17+ messages in thread
From: Brad Bishop @ 2021-08-05 13:55 UTC (permalink / raw)
  To: Joseph Reynolds, James Mihm, Sekar, Suryakanth
  Cc: Andrew Jeffery, Ed Tanous, openbmc

On Wed, 2021-08-04 at 16:57 -0700, Ed Tanous wrote:
> On Wed, Aug 4, 2021 at 4:48 PM Andrew Jeffery <andrew@aj.id.au> wrote:
> > 
> > 
> > 
> > On Thu, 5 Aug 2021, at 06:19, Patrick Williams wrote:
> > > On Wed, Aug 04, 2021 at 03:39:45PM -0500, Joseph Reynolds wrote:
> > > > On 8/4/21 3:09 PM, Patrick Williams wrote:
> > > > > On Wed, Aug 04, 2021 at 01:47:31PM -0500, Joseph Reynolds
> > > > > wrote:
> > > > > 
> > > > > > 4 Surya set up a bugzilla within Intel and will administer
> > > > > > it.  Demo’d
> > > > > > the database. We briefly examined the database fields and
> > > > > > agreed it
> > > > > > looks like a good start.
> > > > > > 
> > > > > Once again I'll ask ***WHY***??!?
> > > > > 
> > > > > https://lore.kernel.org/openbmc/YNzsE1ipYQR7yfDq@heinlein/
> > > > > https://lore.kernel.org/openbmc/YPiK8xqFPJFZDa1+@heinlein/
> > > > > 
> > > > > Can we please create a private Github repository and be done
> > > > > with this topic?
> > > > 
> > > > I don't have any insight into how to resolve this question.
> > > > 
> > > >  From today's meeting: using bugzilla has advantages over github
> > > > issues:
> > > > - lets us define the fields we need: fix commitID, CVSS score,
> > > > etc.
> 
> I'm also really not following this as a rationale for starting a
> completely new server.  This is easy enough to add on github in the
> bug template.
> 
> > > 
> > > These are pretty minor when you could just add a comment template
> > > with this
> > > information.
> > > 
> > > > - has desirable access controls, specifically acess by the
> > > > security
> > > > respone tram plus we can add access for the problem submitter
> > > > and the
> > > > problem fixer
> > > 
> > > So does Github.
> 
> +1
> 
> > > 
> > > ----
> > > 
> > > I really don't think that some subset of the community should go
> > > off on their
> > > own bug tracking system.
> > 
> > +1
> > 
> > I'm not aware of any effort to use Github security advisories so
> > far. I
> > think we should try that before burdening ourselves and any bug
> > reporters with yet more disjoint bits of infrastructure (we already
> > have the two mailing lists, discord, gerrit, and github).
> 
> +1
> 
> > 
> > Andrew


+1 from me as well on all points.  We already use GitHub bug templates
and GitHub security controls in this project with great success and the
community is here to support the security working group if you don't
know how to make use of those.

Additional benefits to GitHub that were pointed out to me is that GitHub
checks passwords against leak databases, supports 2FA, can be configured
to require 2FA for those that have access.  GitHub is also more likely
to keep their software updated than a privately maintained bugzilla
instance.  In 2021 it is just too hard to compete with the scale of
support that services like GitHub provide.

I think my proposal would be that the security working group create a
new GitHub organization here:

https://github.com/openbmc-security

The security WG leadership could own the organization and create any
number of groups, acls, repositories (public or private), wikis, etc..

thx - brad


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4 - results
  2021-08-04 18:47 ` Security Working Group meeting - Wednesday August 4 - results Joseph Reynolds
  2021-08-04 20:09   ` Patrick Williams
@ 2021-08-05 15:54   ` Brad Bishop
  1 sibling, 0 replies; 17+ messages in thread
From: Brad Bishop @ 2021-08-05 15:54 UTC (permalink / raw)
  To: Joseph Reynolds; +Cc: openbmc

On Wed, 2021-08-04 at 13:47 -0500, Joseph Reynolds wrote:
> 
> Bonus topics:
> 
> 4 Surya set up a bugzilla within Intel and will administer it.  Demo’d
> the database. We briefly examined the database fields and agreed it 
> looks like a good start.

Hi Joseph

Another idea I've just had that might help here - I wonder/suspect there
are security working group participants that aren't following the
mailing list (which is fine).  If that is the case, they simply aren't
aware of the feedback or questions coming in from the community in
response to your minutes.

Would it help if you Joseph, as the security working group chair who has
the best understanding of who from the meetings can best engage with the
community to directly CC: those meeting participants on the minutes in
addition to the list?  That way they will get the feedback/questions in
their inbox and have the opportunity to engage?

thx - brad


^ permalink raw reply	[flat|nested] 17+ messages in thread

* RE: Security Working Group meeting - Wednesday August 4 - results
  2021-08-04 23:23         ` Patrick Williams
@ 2021-08-06 17:10           ` Mihm, James
  0 siblings, 0 replies; 17+ messages in thread
From: Mihm, James @ 2021-08-06 17:10 UTC (permalink / raw)
  To: Patrick Williams, Joseph Reynolds; +Cc: openbmc, Brad Bishop

[-- Attachment #1: Type: text/plain, Size: 2441 bytes --]

I’ve been pushing for a database of some sort to track the security issues that I’ve submitted so far. My initial impression was that the github security advisories was targeted more for disclosures and not necessarily management. I’ll look into the github security advisories further. What I’m looking for is a tool that will help us track the progress of mitigations or the lack thereof.
I’d also like to track all of the issues from upstream projects that impact openbmc, and a database seems like a good option for that.

Regards, James.

From: openbmc <openbmc-bounces+james.mihm=intel.com@lists.ozlabs.org> On Behalf Of Patrick Williams
Sent: Wednesday, August 4, 2021 4:24 PM
To: Joseph Reynolds <jrey@linux.ibm.com>
Cc: openbmc@lists.ozlabs.org; Brad Bishop <bradleyb@fuzziesquirrel.com>
Subject: Re: Security Working Group meeting - Wednesday August 4 - results

Has this been read through?

https://docs.github.com/en/code-security/security-advisories/about-github-security-advisories


On Aug 4, 2021, at 3:49 PM, Patrick Williams <patrick@stwcx.xyz<mailto:patrick@stwcx.xyz>> wrote:
On Wed, Aug 04, 2021 at 03:39:45PM -0500, Joseph Reynolds wrote:

On 8/4/21 3:09 PM, Patrick Williams wrote:
On Wed, Aug 04, 2021 at 01:47:31PM -0500, Joseph Reynolds wrote:

4 Surya set up a bugzilla within Intel and will administer it.  Demo’d
the database. We briefly examined the database fields and agreed it
looks like a good start.

Once again I'll ask ***WHY***??!?

https://lore.kernel.org/openbmc/YNzsE1ipYQR7yfDq@heinlein/
https://lore.kernel.org/openbmc/YPiK8xqFPJFZDa1+@heinlein/

Can we please create a private Github repository and be done with this topic?

I don't have any insight into how to resolve this question.

From today's meeting: using bugzilla has advantages over github issues:
- lets us define the fields we need: fix commitID, CVSS score, etc.

These are pretty minor when you could just add a comment template with this
information.


- has desirable access controls, specifically acess by the security
respone tram plus we can add access for the problem submitter and the
problem fixer

So does Github.

----

I really don't think that some subset of the community should go off on their
own bug tracking system.  This is a waste of time to maintain and just further
segments this "Security Team" off in their own bubble.

--
Patrick Williams

[-- Attachment #2: Type: text/html, Size: 8954 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Security Working Group meeting - Wednesday August 4 - ibm-acf repo
  2021-08-04  3:22 ` Patrick Williams
@ 2021-08-09 14:09   ` Joseph Reynolds
  0 siblings, 0 replies; 17+ messages in thread
From: Joseph Reynolds @ 2021-08-09 14:09 UTC (permalink / raw)
  To: Patrick Williams; +Cc: openbmc

On 8/3/21 10:22 PM, Patrick Williams wrote:
> On Tue, Aug 03, 2021 at 05:57:52PM -0500, Joseph Reynolds wrote:
>> This is a reminder of the OpenBMC Security Working Group meeting
>> scheduled for this Wednesday August 4 at 10:00am PDT.
>>
>> We'll discuss the following items on the agenda
>> <https://docs.google.com/document/d/1b7x9BaxsfcukQDqbvZsU2ehMq4xoJRQvLxxsDUWmAOI/edit>,
>> and anything else that comes up:
>>
>>   1. (Joseph): IBM ACF design (2FA authentication for the special IBM
>>      service account) is in review -
>>      https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/45201
>>      <https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/45201>
> I still feel like the "Alternatives considered" are pretty weak in this
> document.  Rather than paint broad brushes ("Other were considered.  They were
> not suitable.") I think you should enumerate _which_ alternatives were
> considered and _why_ they don't fit the problem at hand.

You're right, I rushed that section.  I will fill in some details. Thanks!

> ```
> - Takes four parameters: machine serial number, expiration date, password, and
>    private key.
> - Algorithm:
>     - Hashes the password.
>     - Creates the ACF from the hashed password, serial number, and expiration date.
>     - Digitally signs the ACF using the private key.
>     - Returns the ACF to the caller.
> ```
>
> This sounds a lot like U2F.  The "4 parameters" are the challenge, IBM's key
> signing server is the U2F device, and PAM is the "Relying Party".  There are
> already PAM modules for some aspects of U2F and the token you need to exchange
> is reasonably short (my Yubikey output is 33 characters).
>
> https://developers.yubico.com/U2F/Protocol_details/Overview.html
>
> The nice aspect if you can reuse portions of the U2F protocol is that you go a
> long way towards enabling others to add more typical 2FA like Yubikeys.

We discussed this an email exchange ending 2021 March 9 "Request new 
repo for IBM-specific code - pam_2fa discussion".  Although I could use 
the U2F module, it is not a good fit because the ACF credential flow 
does not fit well under the U2F protocol.  I'll excerpt that 
conversation and add details to the design.

Joseph


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2021-08-09 14:10 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-03 22:57 Security Working Group meeting - Wednesday August 4 Joseph Reynolds
2021-08-04  3:04 ` Patrick Williams
2021-08-04  3:22 ` Patrick Williams
2021-08-09 14:09   ` Security Working Group meeting - Wednesday August 4 - ibm-acf repo Joseph Reynolds
2021-08-04  3:28 ` Security Working Group meeting - Wednesday August 4 Patrick Williams
2021-08-04 18:43   ` Security Working Group meeting - Wednesday August 4 - all distro owners please review Joseph Reynolds
2021-08-04 18:47 ` Security Working Group meeting - Wednesday August 4 - results Joseph Reynolds
2021-08-04 20:09   ` Patrick Williams
2021-08-04 20:39     ` Joseph Reynolds
2021-08-04 20:49       ` Patrick Williams
2021-08-04 23:23         ` Patrick Williams
2021-08-06 17:10           ` Mihm, James
2021-08-04 23:47         ` Andrew Jeffery
2021-08-04 23:57           ` Ed Tanous
2021-08-05 13:55             ` Brad Bishop
2021-08-05 13:43       ` Brad Bishop
2021-08-05 15:54   ` Brad Bishop

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.