[PATCH 0/1] [RESEND]Create a script for SUMMARY audit in recipes

[PATCH 0/1] [RESEND]Create a script for SUMMARY audit in recipes

[Shane Wang]

Here is the script to check which recipe provides SUMMARY and which doesnot.
For those which do not, maintainer should add or update to a meaningful summary for HOB to display in description.

The following changes since commit b281fd127bac9ba77ab63a5c2b812ddd5d56df37:

  documentation/poky-ref-manual/technical-details.xml: edits per Richard Purdie (2011-12-16 16:58:41 +0000)

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib shane/summary-audit
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=shane/summary-audit

Shane Wang (1):
  Create a script for SUMMARY audit in recipes

 scripts/contrib/summary-audit.sh |   55 ++++++++++++++++++++++++++++++++++++++
 1 files changed, 55 insertions(+), 0 deletions(-)
 create mode 100755 scripts/contrib/summary-audit.sh

-- 
1.7.6

[PATCH 1/1] Create a script for SUMMARY audit in recipes

[Shane Wang]

Some recipes don't contain SUMMARY, which HOB will use for descriptions.
If the summary is missing, bitbake will create a default value for summary.
That is PN plus string " version " plus its version. Every maintainer should add and update the summary fields according to audit results.

[YOCTO #1804] got fixed, and maintainers should follow up.

Signed-off-by: Shane Wang <shane.wang@intel.com>
---
 scripts/contrib/summary-audit.sh |   55 ++++++++++++++++++++++++++++++++++++++
 1 files changed, 55 insertions(+), 0 deletions(-)
 create mode 100755 scripts/contrib/summary-audit.sh

diff --git a/scripts/contrib/summary-audit.sh b/scripts/contrib/summary-audit.sh
new file mode 100755
index 0000000..63b43cb
--- /dev/null
+++ b/scripts/contrib/summary-audit.sh
@@ -0,0 +1,55 @@
+#!/bin/sh
+#
+# Perform an audit of which recipes provide summary and which
+# are missing for HOB2.
+#
+# Setup requirements: Run this script after source'ing the build
+# environment script, so you're running it from build/ directory.
+#
+# Maintainer: Shane Wang <shane.wang@intel.com>
+
+DISTRO_TRACKING_FILE="../meta/conf/distro/include/distro_tracking_fields.inc"
+REPORT_MISSING_SUMMARY="summary.txt"
+REPORT_LOG="summary.log"
+
+rm -rf $REPORT_MISSING_SUMMARY $REPORT_LOG
+
+BITBAKE=`which bitbake`
+if [ -z "$BITBAKE" ]; then
+	echo "Error: bitbake command not found."
+	echo "Did you forget to source the build environment script?"
+	exit 1
+fi
+
+packages=(`bitbake -s | awk '{ print \$1 }'`)
+versions=(`bitbake -s | awk '{ print \$2 }'`)
+
+echo "Package Num: ${#packages[*]} Version Num: ${#versions[*]}" >> "$REPORT_LOG"
+len=${#packages[*]}
+
+for (( i=0; i < "$len"; i++ )); do
+	pkg=${packages[$i]}
+
+	if [[ "$pkg" == "Loading" || "$pkg" == "Loaded" ||
+          "$pkg" == "Parsing" || "$pkg" == "Package" ||
+          "$pkg" == "NOTE:"   || "$pkg" == "WARNING:" ||
+          "$pkg" == "done."   || "$pkg" == "============" ||
+          "$pkg" == "ERROR:"  || "$pkg" == "Traceback" ||
+          "$pkg" == "File"    || "$pkg" == "IndexError:" ]]
+	then
+		# Skip initial bitbake output
+		continue
+	fi
+
+	SUMMARY=`bitbake -e $pkg | grep -e "^SUMMARY *=" | awk -F '=' '{ print \$2 }' | awk -F '"' '{ print \$2 }'`
+	ver=${versions[$i]#*:} # remove ':' in the version
+	echo "Handling package $pkg ($i out of $len) ..."
+	echo "HANDLING SUMMARY of package $pkg ... $SUMMARY <---NOT---> $pkg version $ver" >> "$REPORT_LOG"
+
+	if [[ "$SUMMARY" =~ "$pkg"" version ""$ver" ]]; then
+		# find which summary is missing and report
+		maintainer=`cat ../meta/conf/distro/include/distro_tracking_fields.inc | grep -e "^RECIPE_MAINTAINER_pn-"$pkg" *=" | cut -d= -f2`
+		echo "$pkg"" = ""$maintainer" >> "$REPORT_MISSING_SUMMARY"
+	fi
+done
+echo "DONE!" >> "$REPORT_LOG"
-- 
1.7.6

Re: [PATCH 0/1] [RESEND]Create a script for SUMMARY audit in recipes

[Saul Wold]

On 12/20/2011 08:32 PM, Shane Wang wrote:
> Here is the script to check which recipe provides SUMMARY and which doesnot.
> For those which do not, maintainer should add or update to a meaningful summary for HOB to display in description.
>
Shane,

I guess I am not understanding why this is needed.  Why can't we use 
DESCRIPTION which is a required entry?  Use the first X Characters of 
DESCRIPTION?

Sau!


> The following changes since commit b281fd127bac9ba77ab63a5c2b812ddd5d56df37:
>
>    documentation/poky-ref-manual/technical-details.xml: edits per Richard Purdie (2011-12-16 16:58:41 +0000)
>
> are available in the git repository at:
>    git://git.pokylinux.org/poky-contrib shane/summary-audit
>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=shane/summary-audit
>
> Shane Wang (1):
>    Create a script for SUMMARY audit in recipes
>
>   scripts/contrib/summary-audit.sh |   55 ++++++++++++++++++++++++++++++++++++++
>   1 files changed, 55 insertions(+), 0 deletions(-)
>   create mode 100755 scripts/contrib/summary-audit.sh
>

Re: [PATCH 0/1] [RESEND]Create a script for SUMMARY audit in recipes

[Wang, Shane]

Saul Wold wrote on 2011-12-21:

> On 12/20/2011 08:32 PM, Shane Wang wrote:
>> Here is the script to check which recipe provides SUMMARY and which
>> doesnot. For those which do not, maintainer should add or update to a
>> meaningful summary for HOB to display in description.
>> 
> Shane,
> 
> I guess I am not understanding why this is needed.  Why can't we use
> DESCRIPTION which is a required entry?  Use the first X Characters of
> DESCRIPTION?
> 
> Sau!

OK, then HOB has a bug. To use DESCRIPTION instead of SUMMARY. I am OK with that.
Josh, Dongxiao, did you see any problem if I change that?

--
Shane

Re: [PATCH 0/1] [RESEND]Create a script for SUMMARY audit in recipes

[Joshua Lock]

On 20/12/11 22:10, Wang, Shane wrote:
> Saul Wold wrote on 2011-12-21:
> 
>> On 12/20/2011 08:32 PM, Shane Wang wrote:
>>> Here is the script to check which recipe provides SUMMARY and which
>>> doesnot. For those which do not, maintainer should add or update to a
>>> meaningful summary for HOB to display in description.
>>>
>> Shane,
>>
>> I guess I am not understanding why this is needed.  Why can't we use
>> DESCRIPTION which is a required entry?  Use the first X Characters of
>> DESCRIPTION?

There's a huge difference between a purposefully crafted 72 character
summary and a free-for all description field that will have to be
chopped to be displayed in the GUI. I originally chose summary as a
succinct 72 characters would fit much better in the available UI.

Aside: according to the Yocto docs the SUMMARY field should fall back to
DESCRIPTION anyway. It's just that right now we do that at the package
back-end level for each package back-end.


> OK, then HOB has a bug. To use DESCRIPTION instead of SUMMARY. I am OK with that.
> Josh, Dongxiao, did you see any problem if I change that?

I think it's the wrong solution.

Joshua
-- 
Joshua Lock
        Yocto Project "Johannes factotum"
        Intel Open Source Technology Centre

Re: [PATCH 0/1] [RESEND]Create a script for SUMMARY audit in recipes

[Saul Wold]

On 12/21/2011 10:16 AM, Joshua Lock wrote:
> On 20/12/11 22:10, Wang, Shane wrote:
>> Saul Wold wrote on 2011-12-21:
>>
>>> On 12/20/2011 08:32 PM, Shane Wang wrote:
>>>> Here is the script to check which recipe provides SUMMARY and which
>>>> doesnot. For those which do not, maintainer should add or update to a
>>>> meaningful summary for HOB to display in description.
>>>>
>>> Shane,
>>>
>>> I guess I am not understanding why this is needed.  Why can't we use
>>> DESCRIPTION which is a required entry?  Use the first X Characters of
>>> DESCRIPTION?
>
> There's a huge difference between a purposefully crafted 72 character
> summary and a free-for all description field that will have to be
> chopped to be displayed in the GUI. I originally chose summary as a
> succinct 72 characters would fit much better in the available UI.
>
> Aside: according to the Yocto docs the SUMMARY field should fall back to
> DESCRIPTION anyway. It's just that right now we do that at the package
> back-end level for each package back-end.
>
Seems that's the other way around as coded, DESCRIPTION falls back to 
SUMMARY

meta/conf/bitbake.conf:DESCRIPTION ?= "${SUMMARY}"

Which is why I think this issue is cropping up. So, then the proposal
should really be to add SUMMARY to all recipes and initially make it a 
warning for now if non-existent SUMMARY as with DESCRIPTION and then 
remove the existing SUMMARY = ${PN}-${PV} ...??

The audit would then be the list of warnings which later becomes error

Sau!

>
>> OK, then HOB has a bug. To use DESCRIPTION instead of SUMMARY. I am OK with that.
>> Josh, Dongxiao, did you see any problem if I change that?
>
> I think it's the wrong solution.
>
> Joshua

Re: [PATCH 0/1] [RESEND]Create a script for SUMMARY audit in recipes

[Mark Hatle]

On 12/21/11 12:43 PM, Saul Wold wrote:
> On 12/21/2011 10:16 AM, Joshua Lock wrote:
>> On 20/12/11 22:10, Wang, Shane wrote:
>>> Saul Wold wrote on 2011-12-21:
>>>
>>>> On 12/20/2011 08:32 PM, Shane Wang wrote:
>>>>> Here is the script to check which recipe provides SUMMARY and which
>>>>> doesnot. For those which do not, maintainer should add or update to a
>>>>> meaningful summary for HOB to display in description.
>>>>>
>>>> Shane,
>>>>
>>>> I guess I am not understanding why this is needed.  Why can't we use
>>>> DESCRIPTION which is a required entry?  Use the first X Characters of
>>>> DESCRIPTION?
>>
>> There's a huge difference between a purposefully crafted 72 character
>> summary and a free-for all description field that will have to be
>> chopped to be displayed in the GUI. I originally chose summary as a
>> succinct 72 characters would fit much better in the available UI.
>>
>> Aside: according to the Yocto docs the SUMMARY field should fall back to
>> DESCRIPTION anyway. It's just that right now we do that at the package
>> back-end level for each package back-end.
>>
> Seems that's the other way around as coded, DESCRIPTION falls back to
> SUMMARY
>
> meta/conf/bitbake.conf:DESCRIPTION ?= "${SUMMARY}"

Ya, I believe the original work was the reverse.. SUMMARY was the first X number 
of characters from the DESCRIPTION, but that soon changed..

Then the next iteration, all of the descriptions become summary fields and 
DESCRIPTION inherited SUMMARY.

And then a default SUMMARY was added to maintain compatibility with older OE 
recipes.

> Which is why I think this issue is cropping up. So, then the proposal
> should really be to add SUMMARY to all recipes and initially make it a
> warning for now if non-existent SUMMARY as with DESCRIPTION and then
> remove the existing SUMMARY = ${PN}-${PV} ...??
>
> The audit would then be the list of warnings which later becomes error

I agree, this should become some kind of an audit warning, so we can improve the 
quality of the recipe information.  I'm not sure I want it to be an error though 
as it may impact folks with existing OE recipes...

--Mark

> Sau!
>
>>
>>> OK, then HOB has a bug. To use DESCRIPTION instead of SUMMARY. I am OK with that.
>>> Josh, Dongxiao, did you see any problem if I change that?
>>
>> I think it's the wrong solution.
>>
>> Joshua
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: [PATCH 0/1] [RESEND]Create a script for SUMMARY audit in recipes

[Paul Eggleton]

On Wednesday 21 December 2011 12:59:05 Mark Hatle wrote:
> On 12/21/11 12:43 PM, Saul Wold wrote:
> > Which is why I think this issue is cropping up. So, then the proposal
> > should really be to add SUMMARY to all recipes and initially make it a
> > warning for now if non-existent SUMMARY as with DESCRIPTION and then
> > remove the existing SUMMARY = ${PN}-${PV} ...??
> > 
> > The audit would then be the list of warnings which later becomes error
> 
> I agree, this should become some kind of an audit warning, so we can improve
> the quality of the recipe information.  I'm not sure I want it to be an
> error though as it may impact folks with existing OE recipes...

Do we want to be showing this warning to everyone or should it just be part of 
a class you can enable (e.g. oelint.bbclass - which needs some work by the 
looks of it)?

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre