How to comprehensively search the OpenBMC Archives

How to comprehensively search the OpenBMC Archives

[Muggeridge, Matt]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

Is there a more comprehensive/reliable way to search through these mamil archives?

I have been using google to search the archives, but have noticed the results are not complete.

E.g. if I wanted to find my old posts, I would enter a search term like this into google:

muggeridge site:https://lists.ozlabs.org/pipermail/openbmc/

Though, that returns no results (and similarly for other search terms for which I know there are matching archives).

Thanks,
Matt.


[-- Attachment #2: Type: text/html, Size: 2508 bytes --]

Re: How to comprehensively search the OpenBMC Archives

[Gunnar Mills]

[-- Attachment #1: Type: text/plain, Size: 356 bytes --]

On 9/3/2020 6:14 PM, Muggeridge, Matt wrote:
>
> Is there a more comprehensive/reliable way to search through these 
> mamil archives?
>
+1. Could we look at moving to Mailman v3 and like 
https://docs.mailman3.org/projects/hyperkitty/en/latest/ ?


> I have been using google to search the archives, but have noticed the 
> results are not complete.
>
>


[-- Attachment #2: Type: text/html, Size: 2409 bytes --]

Re: How to comprehensively search the OpenBMC Archives

[Joseph Reynolds]

On 9/8/20 1:51 PM, Gunnar Mills wrote:
> On 9/3/2020 6:14 PM, Muggeridge, Matt wrote:
>>
>> Is there a more comprehensive/reliable way to search through these 
>> mamil archives?
>>

I used the following (bash) script to download the entire openbmc email 
archive to my workstation's storage.  (And then I use my local search 
capability (grep,  spotlight search, Windows-f, etc.) to locate the 
desired content.

- Joseph

wget --no-check-certificate https://lists.ozlabs.org/pipermail/openbmc
set -x
for f in $(grep '<td>.*.txt.gz' index.html | cut -d\" -f2); do
   wget --no-check-certificate https://lists.ozlabs.org/pipermail/openbmc/$f
   gunzip "$f"
done


> +1. Could we look at moving to Mailman v3 and like 
> https://docs.mailman3.org/projects/hyperkitty/en/latest/ ?
>
>
>> I have been using google to search the archives, but have noticed the 
>> results are not complete.
>>
>>
>

RE: How to comprehensively search the OpenBMC Archives

[Muggeridge, Matt]

I am impressed Joseph!  Thank you.  That certainly highlights the lengths others have gone to for a comprehensive search feature.

There is a lot of very valuable content locked up in these archives.  I'm hopeful admins will agree that comprehensive searching of the archives needs to be made easier and more accessible to the masses.

Matt.

> -----Original Message-----
> From: Joseph Reynolds <jrey@linux.ibm.com>
> Sent: Wednesday, 9 September 2020 7:25 AM
> To: Gunnar Mills <gmills@linux.vnet.ibm.com>; Muggeridge, Matt
> <matt.muggeridge2@hpe.com>; OpenBMC Maillist
> <openbmc@lists.ozlabs.org>
> Subject: Re: How to comprehensively search the OpenBMC Archives
> 
> On 9/8/20 1:51 PM, Gunnar Mills wrote:
> > On 9/3/2020 6:14 PM, Muggeridge, Matt wrote:
> >>
> >> Is there a more comprehensive/reliable way to search through these
> >> mamil archives?
> >>
> 
> I used the following (bash) script to download the entire openbmc email
> archive to my workstation's storage.  (And then I use my local search
> capability (grep,  spotlight search, Windows-f, etc.) to locate the desired
> content.
> 
> - Joseph
> 
> wget --no-check-certificate
> INVALID URI REMOVED
> 3A__lists.ozlabs.org_pipermail_openbmc&d=DwID-
> g&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647cJDeqUETccV2yvBRCeNJXBtz6
> 14MxJzMR9PZk&m=tjRnWO2f8md0hwPIF4dbkqaJbj-
> OhYZN2fNFoROxHeM&s=oUgen4rG7u90Uby5812hA_y0dDnG41oD1ZuA4K7K
> NiU&e=
> set -x
> for f in $(grep '<td>.*.txt.gz' index.html | cut -d\" -f2); do
>    wget --no-check-certificate
> INVALID URI REMOVED
> 3A__lists.ozlabs.org_pipermail_openbmc_-24f&d=DwID-
> g&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647cJDeqUETccV2yvBRCeNJXBtz6
> 14MxJzMR9PZk&m=tjRnWO2f8md0hwPIF4dbkqaJbj-
> OhYZN2fNFoROxHeM&s=AVV8sFghgocf1nC9Vf7UGQtg9m1A63Yeux7DAywt
> cRs&e=
>    gunzip "$f"
> done
> 
> 
> > +1. Could we look at moving to Mailman v3 and like
> > INVALID URI REMOVED
> 3A__docs.mailman3.org_projects_hyperkitty_en_latest_&d=DwID-
> g&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647cJDeqUETccV2yvBRCeNJXBtz6
> 14MxJzMR9PZk&m=tjRnWO2f8md0hwPIF4dbkqaJbj-
> OhYZN2fNFoROxHeM&s=HWwrQzEzVNtANfa8DBfvdGnK_2nRlSp-
> peNmEl1y6n0&e=  ?
> >
> >
> >> I have been using google to search the archives, but have noticed the
> >> results are not complete.
> >>
> >>
> >

Re: How to comprehensively search the OpenBMC Archives

[Andrew Jeffery]

On Wed, 9 Sep 2020, at 07:52, Muggeridge, Matt wrote:
> I am impressed Joseph!  Thank you.  That certainly highlights the 
> lengths others have gone to for a comprehensive search feature.
> 
> There is a lot of very valuable content locked up in these archives.  
> I'm hopeful admins will agree that comprehensive searching of the 
> archives needs to be made easier and more accessible to the masses.

Maybe we should set up a public-inbox instance?

https://public-inbox.org/README.html

lore.openbmc.org?

Andrew

> 
> Matt.
> 
> > -----Original Message-----
> > From: Joseph Reynolds <jrey@linux.ibm.com>
> > Sent: Wednesday, 9 September 2020 7:25 AM
> > To: Gunnar Mills <gmills@linux.vnet.ibm.com>; Muggeridge, Matt
> > <matt.muggeridge2@hpe.com>; OpenBMC Maillist
> > <openbmc@lists.ozlabs.org>
> > Subject: Re: How to comprehensively search the OpenBMC Archives
> > 
> > On 9/8/20 1:51 PM, Gunnar Mills wrote:
> > > On 9/3/2020 6:14 PM, Muggeridge, Matt wrote:
> > >>
> > >> Is there a more comprehensive/reliable way to search through these
> > >> mamil archives?
> > >>
> > 
> > I used the following (bash) script to download the entire openbmc email
> > archive to my workstation's storage.  (And then I use my local search
> > capability (grep,  spotlight search, Windows-f, etc.) to locate the desired
> > content.
> > 
> > - Joseph
> > 
> > wget --no-check-certificate
> > INVALID URI REMOVED
> > 3A__lists.ozlabs.org_pipermail_openbmc&d=DwID-
> > g&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647cJDeqUETccV2yvBRCeNJXBtz6
> > 14MxJzMR9PZk&m=tjRnWO2f8md0hwPIF4dbkqaJbj-
> > OhYZN2fNFoROxHeM&s=oUgen4rG7u90Uby5812hA_y0dDnG41oD1ZuA4K7K
> > NiU&e=
> > set -x
> > for f in $(grep '<td>.*.txt.gz' index.html | cut -d\" -f2); do
> >    wget --no-check-certificate
> > INVALID URI REMOVED
> > 3A__lists.ozlabs.org_pipermail_openbmc_-24f&d=DwID-
> > g&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647cJDeqUETccV2yvBRCeNJXBtz6
> > 14MxJzMR9PZk&m=tjRnWO2f8md0hwPIF4dbkqaJbj-
> > OhYZN2fNFoROxHeM&s=AVV8sFghgocf1nC9Vf7UGQtg9m1A63Yeux7DAywt
> > cRs&e=
> >    gunzip "$f"
> > done
> > 
> > 
> > > +1. Could we look at moving to Mailman v3 and like
> > > INVALID URI REMOVED
> > 3A__docs.mailman3.org_projects_hyperkitty_en_latest_&d=DwID-
> > g&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647cJDeqUETccV2yvBRCeNJXBtz6
> > 14MxJzMR9PZk&m=tjRnWO2f8md0hwPIF4dbkqaJbj-
> > OhYZN2fNFoROxHeM&s=HWwrQzEzVNtANfa8DBfvdGnK_2nRlSp-
> > peNmEl1y6n0&e=  ?
> > >
> > >
> > >> I have been using google to search the archives, but have noticed the
> > >> results are not complete.
> > >>
> > >>
> > >
> 
>

Re: How to comprehensively search the OpenBMC Archives

[Joel Stanley]

On Thu, 10 Sep 2020 at 03:36, Andrew Jeffery <andrew@aj.id.au> wrote:
>
>
>
> On Wed, 9 Sep 2020, at 07:52, Muggeridge, Matt wrote:
> > I am impressed Joseph!  Thank you.  That certainly highlights the
> > lengths others have gone to for a comprehensive search feature.
> >
> > There is a lot of very valuable content locked up in these archives.
> > I'm hopeful admins will agree that comprehensive searching of the
> > archives needs to be made easier and more accessible to the masses.
>
> Maybe we should set up a public-inbox instance?
>
> https://public-inbox.org/README.html
>
> lore.openbmc.org?

I was thinking about this the other day. Given qemu is hosted on
lore.kernel.org, I think we could do the same.

I'll contact the lore-man about it.

Cheers,

Joel

>
> Andrew
>
> >
> > Matt.
> >
> > > -----Original Message-----
> > > From: Joseph Reynolds <jrey@linux.ibm.com>
> > > Sent: Wednesday, 9 September 2020 7:25 AM
> > > To: Gunnar Mills <gmills@linux.vnet.ibm.com>; Muggeridge, Matt
> > > <matt.muggeridge2@hpe.com>; OpenBMC Maillist
> > > <openbmc@lists.ozlabs.org>
> > > Subject: Re: How to comprehensively search the OpenBMC Archives
> > >
> > > On 9/8/20 1:51 PM, Gunnar Mills wrote:
> > > > On 9/3/2020 6:14 PM, Muggeridge, Matt wrote:
> > > >>
> > > >> Is there a more comprehensive/reliable way to search through these
> > > >> mamil archives?
> > > >>
> > >
> > > I used the following (bash) script to download the entire openbmc email
> > > archive to my workstation's storage.  (And then I use my local search
> > > capability (grep,  spotlight search, Windows-f, etc.) to locate the desired
> > > content.
> > >
> > > - Joseph
> > >
> > > wget --no-check-certificate
> > > INVALID URI REMOVED
> > > 3A__lists.ozlabs.org_pipermail_openbmc&d=DwID-
> > > g&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647cJDeqUETccV2yvBRCeNJXBtz6
> > > 14MxJzMR9PZk&m=tjRnWO2f8md0hwPIF4dbkqaJbj-
> > > OhYZN2fNFoROxHeM&s=oUgen4rG7u90Uby5812hA_y0dDnG41oD1ZuA4K7K
> > > NiU&e=
> > > set -x
> > > for f in $(grep '<td>.*.txt.gz' index.html | cut -d\" -f2); do
> > >    wget --no-check-certificate
> > > INVALID URI REMOVED
> > > 3A__lists.ozlabs.org_pipermail_openbmc_-24f&d=DwID-
> > > g&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647cJDeqUETccV2yvBRCeNJXBtz6
> > > 14MxJzMR9PZk&m=tjRnWO2f8md0hwPIF4dbkqaJbj-
> > > OhYZN2fNFoROxHeM&s=AVV8sFghgocf1nC9Vf7UGQtg9m1A63Yeux7DAywt
> > > cRs&e=
> > >    gunzip "$f"
> > > done
> > >
> > >
> > > > +1. Could we look at moving to Mailman v3 and like
> > > > INVALID URI REMOVED
> > > 3A__docs.mailman3.org_projects_hyperkitty_en_latest_&d=DwID-
> > > g&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647cJDeqUETccV2yvBRCeNJXBtz6
> > > 14MxJzMR9PZk&m=tjRnWO2f8md0hwPIF4dbkqaJbj-
> > > OhYZN2fNFoROxHeM&s=HWwrQzEzVNtANfa8DBfvdGnK_2nRlSp-
> > > peNmEl1y6n0&e=  ?
> > > >
> > > >
> > > >> I have been using google to search the archives, but have noticed the
> > > >> results are not complete.
> > > >>
> > > >>
> > > >
> >
> >

Mailing list archive on lore.kernel.org/openbmc

[Joel Stanley]

On Fri, 11 Sep 2020 at 03:47, Joel Stanley <joel@jms.id.au> wrote:
> > Maybe we should set up a public-inbox instance?
> >
> > https://public-inbox.org/README.html
> >
> > lore.openbmc.org?
>
> I was thinking about this the other day. Given qemu is hosted on
> lore.kernel.org, I think we could do the same.
>
> I'll contact the lore-man about it.

We now have the openbmc list archived at
https://lore.kernel.org/openbmc. This contains all of the history, and
will receive all future email.

Have a read about lore to learn about the features it offers. Most
importantly, it has a super fast web interface for searching.

 https://www.kernel.org/lore.html

Cheers,

Joel

RE: Mailing list archive on lore.kernel.org/openbmc

[Muggeridge, Matt]

Thank you, Joel and others involved.  Indeed, the searches that previously returned no results are now finding all the posts.

Really nice work!

Matt.

> -----Original Message-----
> From: Joel Stanley <joel@jms.id.au>
> Sent: Tuesday, 22 September 2020 4:02 PM
> To: OpenBMC Maillist <openbmc@lists.ozlabs.org>
> Cc: Muggeridge, Matt <matt.muggeridge2@hpe.com>; Joseph Reynolds
> <jrey@linux.ibm.com>; Gunnar Mills <gmills@linux.vnet.ibm.com>; Andrew
> Jeffery <andrew@aj.id.au>
> Subject: Mailing list archive on lore.kernel.org/openbmc
> 
> On Fri, 11 Sep 2020 at 03:47, Joel Stanley <joel@jms.id.au> wrote:
> > > Maybe we should set up a public-inbox instance?
> > >
> > > INVALID URI REMOVED.
> > >
> org_README.html&d=DwIBaQ&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647c
> JDeqUETcc
> > >
> V2yvBRCeNJXBtz614MxJzMR9PZk&m=3xP2b8yi2nP6IfbjtR9B3CQd13W6gLVjx
> RIf_V
> > > VKYYc&s=lV5UvlqSorE6DU5hR643yXywovGR3zcAEVKbGNu6jiU&e=
> > >
> > > lore.openbmc.org?
> >
> > I was thinking about this the other day. Given qemu is hosted on
> > lore.kernel.org, I think we could do the same.
> >
> > I'll contact the lore-man about it.
> 
> We now have the openbmc list archived at https://lore.kernel.org/openbmc.
> This contains all of the history, and will receive all future email.
> 
> Have a read about lore to learn about the features it offers. Most
> importantly, it has a super fast web interface for searching.
> 
>  https://www.kernel.org/lore.html
> 
> Cheers,
> 
> Joel

Re: Mailing list archive on lore.kernel.org/openbmc

[krtaylor]

On 9/22/20 1:01 AM, Joel Stanley wrote:
> On Fri, 11 Sep 2020 at 03:47, Joel Stanley <joel@jms.id.au> wrote:
>>> Maybe we should set up a public-inbox instance?
>>>
>>> https://public-inbox.org/README.html
>>>
>>> lore.openbmc.org?
>>
>> I was thinking about this the other day. Given qemu is hosted on
>> lore.kernel.org, I think we could do the same.
>>
>> I'll contact the lore-man about it.
> 
> We now have the openbmc list archived at
> https://lore.kernel.org/openbmc. This contains all of the history, and
> will receive all future email.

Thanks Joel!

Kurt Taylor (krtaylor)

> Have a read about lore to learn about the features it offers. Most
> importantly, it has a super fast web interface for searching.
> 
>   https://www.kernel.org/lore.html
> 
> Cheers,
> 
> Joel
> 

Re: How to comprehensively search the OpenBMC Archives

[krtaylor]

On 9/8/20 5:22 PM, Muggeridge, Matt wrote:
> I am impressed Joseph!  Thank you.  That certainly highlights the lengths others have gone to for a comprehensive search feature.
> 
> There is a lot of very valuable content locked up in these archives.  I'm hopeful admins will agree that comprehensive searching of the archives needs to be made easier and more accessible to the masses.

As always, feel free to add wish list items for future services to this 
page, we can vet potential services there. (I have added this one)

https://docs.google.com/document/d/1QCFRGCRofcR3K8clSLtJHw10-Cu9zkp0dvwXPWzQSCc/edit?usp=sharing

Kurt Taylor (krtaylor)

> Matt.
> 
>> -----Original Message-----
>> From: Joseph Reynolds <jrey@linux.ibm.com>
>> Sent: Wednesday, 9 September 2020 7:25 AM
>> To: Gunnar Mills <gmills@linux.vnet.ibm.com>; Muggeridge, Matt
>> <matt.muggeridge2@hpe.com>; OpenBMC Maillist
>> <openbmc@lists.ozlabs.org>
>> Subject: Re: How to comprehensively search the OpenBMC Archives
>>
>> On 9/8/20 1:51 PM, Gunnar Mills wrote:
>>> On 9/3/2020 6:14 PM, Muggeridge, Matt wrote:
>>>>
>>>> Is there a more comprehensive/reliable way to search through these
>>>> mamil archives?
>>>>
>>
>> I used the following (bash) script to download the entire openbmc email
>> archive to my workstation's storage.  (And then I use my local search
>> capability (grep,  spotlight search, Windows-f, etc.) to locate the desired
>> content.
>>
>> - Joseph
>>
>> wget --no-check-certificate
>> INVALID URI REMOVED
>> 3A__lists.ozlabs.org_pipermail_openbmc&d=DwID-
>> g&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647cJDeqUETccV2yvBRCeNJXBtz6
>> 14MxJzMR9PZk&m=tjRnWO2f8md0hwPIF4dbkqaJbj-
>> OhYZN2fNFoROxHeM&s=oUgen4rG7u90Uby5812hA_y0dDnG41oD1ZuA4K7K
>> NiU&e=
>> set -x
>> for f in $(grep '<td>.*.txt.gz' index.html | cut -d\" -f2); do
>>     wget --no-check-certificate
>> INVALID URI REMOVED
>> 3A__lists.ozlabs.org_pipermail_openbmc_-24f&d=DwID-
>> g&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647cJDeqUETccV2yvBRCeNJXBtz6
>> 14MxJzMR9PZk&m=tjRnWO2f8md0hwPIF4dbkqaJbj-
>> OhYZN2fNFoROxHeM&s=AVV8sFghgocf1nC9Vf7UGQtg9m1A63Yeux7DAywt
>> cRs&e=
>>     gunzip "$f"
>> done
>>
>>
>>> +1. Could we look at moving to Mailman v3 and like
>>> INVALID URI REMOVED
>> 3A__docs.mailman3.org_projects_hyperkitty_en_latest_&d=DwID-
>> g&c=C5b8zRQO1miGmBeVZ2LFWg&r=Xhm647cJDeqUETccV2yvBRCeNJXBtz6
>> 14MxJzMR9PZk&m=tjRnWO2f8md0hwPIF4dbkqaJbj-
>> OhYZN2fNFoROxHeM&s=HWwrQzEzVNtANfa8DBfvdGnK_2nRlSp-
>> peNmEl1y6n0&e=  ?
>>>
>>>
>>>> I have been using google to search the archives, but have noticed the
>>>> results are not complete.
>>>>
>>>>
>>>
>