[Buildroot] [POC] buildroot packages site

[Buildroot] [POC] buildroot packages site

[Heiko Thiery]

Hi All,

I invested some spare time to create a buildroot packages/stats site.
I used python-flask for processing the json output of the pkg-stats
script. To have more information in the output file I did some
modifications on the pkg-stats script as well.

A running installation as fcgi is available here:
http://packages.buildroot.thiery.it

The current sources of the implementation is available under:
https://github.com/hthiery/buildroot-stats
https://github.com/hthiery/buildroot/tree/feature-pkg-stats

Note that this is just a proof of concept with a lot of flaws ... ;-)
Maybe it is possible for you to take a look on it and you can give me
some feedback.

BR
--
Heiko

[Buildroot] [POC] buildroot packages site

[Giulio Benetti]

Hi Heiko,

On 1/3/20 11:48 AM, Heiko Thiery wrote:
> Hi All,
> 
> I invested some spare time to create a buildroot packages/stats site.
> I used python-flask for processing the json output of the pkg-stats
> script. To have more information in the output file I did some
> modifications on the pkg-stats script as well.
> 
> A running installation as fcgi is available here:
> http://packages.buildroot.thiery.it

Wow! This is very useful!
If I try to filter for developers it doesn't show Developers Column, it 
only shows this on top:
1118 packages with error in developers

But I think it could be useful having a row with ORPHANED(y/n), what 
about it?

Best regards
-- 
Giulio Benetti
Benetti Engineering sas

> The current sources of the implementation is available under:
> https://github.com/hthiery/buildroot-stats
> https://github.com/hthiery/buildroot/tree/feature-pkg-stats
> 
> Note that this is just a proof of concept with a lot of flaws ... ;-)
> Maybe it is possible for you to take a look on it and you can give me
> some feedback.
> 
> BR
> --
> Heiko
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 

[Buildroot] [POC] buildroot packages site

[Heiko Thiery]

> Wow! This is very useful!
> If I try to filter for developers it doesn't show Developers Column, it
> only shows this on top:
> 1118 packages with error in developers
>
> But I think it could be useful having a row with ORPHANED(y/n), what
> about it?

I show "filterd" packages list in the same manner like the all
packages site. So I can generally add a developers count column. What
do you think?


> Best regards
> --
> Giulio Benetti
> Benetti Engineering sas

[Buildroot] [POC] buildroot packages site

[Thomas Petazzoni]

Hello Heiko,

On Fri, 3 Jan 2020 11:48:53 +0100
Heiko Thiery <heiko.thiery@gmail.com> wrote:

> I invested some spare time to create a buildroot packages/stats site.
> I used python-flask for processing the json output of the pkg-stats
> script. To have more information in the output file I did some
> modifications on the pkg-stats script as well.
> 
> A running installation as fcgi is available here:
> http://packages.buildroot.thiery.it

Wow, that's really nice. We definitely want to deploy this as a
replacement to http://autobuild.buildroot.org/stats/. Perhaps we should
even promote this to http://stats.buildroot.org/ or something like that.

One question: how did you collect the stats history at
http://packages.buildroot.thiery.it/packages/stats?commit=198bdaadd03f75fe959c21089c354d36c90069bc ?

Thanks for this development, looks very promising!

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [POC] buildroot packages site

[Heiko Thiery]

Hi,

Am Fr., 3. Jan. 2020 um 13:26 Uhr schrieb Thomas Petazzoni
<thomas.petazzoni@bootlin.com>:
>
> Hello Heiko,
>
> On Fri, 3 Jan 2020 11:48:53 +0100
> Heiko Thiery <heiko.thiery@gmail.com> wrote:
>
> > I invested some spare time to create a buildroot packages/stats site.
> > I used python-flask for processing the json output of the pkg-stats
> > script. To have more information in the output file I did some
> > modifications on the pkg-stats script as well.
> >
> > A running installation as fcgi is available here:
> > http://packages.buildroot.thiery.it
>
> Wow, that's really nice. We definitely want to deploy this as a
> replacement to http://autobuild.buildroot.org/stats/. Perhaps we should
> even promote this to http://stats.buildroot.org/ or something like that.

This sounds good  :-)

> One question: how did you collect the stats history at
> http://packages.buildroot.thiery.it/packages/stats?commit=198bdaadd03f75fe959c21089c354d36c90069bc ?

Fist I have a cron job that does updates the latest.json in a fixed
interval (e.g. every hour). In addition I use that output and strip
the packages stuff from the json with jq and only store the stats part
of the output for every day.
For having the stats back in the past I have a script that does a
checkout for every day in the past and generates this stats output.

> Thanks for this development, looks very promising!
>
> Best regards,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

--
Heiko

[Buildroot] [POC] buildroot packages site

[Thomas Petazzoni]

Hello,

On Fri, 3 Jan 2020 13:33:28 +0100
Heiko Thiery <heiko.thiery@gmail.com> wrote:

> > Wow, that's really nice. We definitely want to deploy this as a
> > replacement to http://autobuild.buildroot.org/stats/. Perhaps we should
> > even promote this to http://stats.buildroot.org/ or something like that.  
> 
> This sounds good  :-)

OK, let's work on getting this in place. We need to sync with Peter
Korsgaard to see how it can be deployed on the existing infrastructure.

> > One question: how did you collect the stats history at
> > http://packages.buildroot.thiery.it/packages/stats?commit=198bdaadd03f75fe959c21089c354d36c90069bc ?  
> 
> Fist I have a cron job that does updates the latest.json in a fixed
> interval (e.g. every hour). In addition I use that output and strip
> the packages stuff from the json with jq and only store the stats part
> of the output for every day.
> For having the stats back in the past I have a script that does a
> checkout for every day in the past and generates this stats output.

OK, sounds good. Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [POC] buildroot packages site

[Giulio Benetti]

Hi Heiko,

On 1/3/20 12:08 PM, Heiko Thiery wrote:
>> Wow! This is very useful!
>> If I try to filter for developers it doesn't show Developers Column, it
>> only shows this on top:
>> 1118 packages with error in developers
>>
>> But I think it could be useful having a row with ORPHANED(y/n), what
>> about it?
> 
> I show "filterd" packages list in the same manner like the all
> packages site. So I can generally add a developers count column. What
> do you think?

That's a good idea, I've already seen you've implemented.

Best regards
-- 
Giulio Benetti
Benetti Engineering sas
> 
>> Best regards
>> --
>> Giulio Benetti
>> Benetti Engineering sas
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 

[Buildroot] [POC] buildroot packages site

[Thomas Petazzoni]

Hello,

On Fri, 3 Jan 2020 11:48:53 +0100
Heiko Thiery <heiko.thiery@gmail.com> wrote:

> I invested some spare time to create a buildroot packages/stats site.
> I used python-flask for processing the json output of the pkg-stats
> script. To have more information in the output file I did some
> modifications on the pkg-stats script as well.
> 
> A running installation as fcgi is available here:
> http://packages.buildroot.thiery.it
> 
> The current sources of the implementation is available under:
> https://github.com/hthiery/buildroot-stats
> https://github.com/hthiery/buildroot/tree/feature-pkg-stats

Perhaps on the per-developer page, such as
http://packages.buildroot.thiery.it/packages/developer/Thomas%20Petazzoni%20%3Cthomas.petazzoni%40bootlin.com%3E
it would make sense to sort the list alphabetically by package name, at
least by default.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [POC] buildroot packages site

[Heiko Thiery]

Am Fr., 3. Jan. 2020 um 15:46 Uhr schrieb Thomas Petazzoni
<thomas.petazzoni@bootlin.com>:
>
> Hello,
>
> On Fri, 3 Jan 2020 11:48:53 +0100
> Heiko Thiery <heiko.thiery@gmail.com> wrote:
>
> > I invested some spare time to create a buildroot packages/stats site.
> > I used python-flask for processing the json output of the pkg-stats
> > script. To have more information in the output file I did some
> > modifications on the pkg-stats script as well.
> >
> > A running installation as fcgi is available here:
> > http://packages.buildroot.thiery.it
> >
> > The current sources of the implementation is available under:
> > https://github.com/hthiery/buildroot-stats
> > https://github.com/hthiery/buildroot/tree/feature-pkg-stats
>
> Perhaps on the per-developer page, such as
> http://packages.buildroot.thiery.it/packages/developer/Thomas%20Petazzoni%20%3Cthomas.petazzoni%40bootlin.com%3E
> it would make sense to sort the list alphabetically by package name, at
> least by default.

For sure .. it is done.

> Thanks!
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

[Buildroot] [POC] buildroot packages site

[Heiko Thiery]

Hi,

Am Fr., 3. Jan. 2020 um 13:46 Uhr schrieb Thomas Petazzoni
<thomas.petazzoni@bootlin.com>:
>
> Hello,
>
> On Fri, 3 Jan 2020 13:33:28 +0100
> Heiko Thiery <heiko.thiery@gmail.com> wrote:
>
> > > Wow, that's really nice. We definitely want to deploy this as a
> > > replacement to http://autobuild.buildroot.org/stats/. Perhaps we should
> > > even promote this to http://stats.buildroot.org/ or something like that.
> >
> > This sounds good  :-)
>
> OK, let's work on getting this in place. We need to sync with Peter
> Korsgaard to see how it can be deployed on the existing infrastructure.

Is that something we can discuss at Brussel on the Buildroot Developers Meeting?

> > > One question: how did you collect the stats history at
> > > http://packages.buildroot.thiery.it/packages/stats?commit=198bdaadd03f75fe959c21089c354d36c90069bc ?
> >
> > Fist I have a cron job that does updates the latest.json in a fixed
> > interval (e.g. every hour). In addition I use that output and strip
> > the packages stuff from the json with jq and only store the stats part
> > of the output for every day.
> > For having the stats back in the past I have a script that does a
> > checkout for every day in the past and generates this stats output.
>
> OK, sounds good. Thanks!
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

Heiko

[Buildroot] [POC] buildroot packages site

[Thomas Petazzoni]

On Thu, 9 Jan 2020 15:58:35 +0100
Heiko Thiery <heiko.thiery@gmail.com> wrote:

> > OK, let's work on getting this in place. We need to sync with Peter
> > Korsgaard to see how it can be deployed on the existing infrastructure.  
> 
> Is that something we can discuss at Brussel on the Buildroot Developers Meeting?

Of course!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com