* [OpenBMC]: Unit test coverage analysis on openbmc repositories
@ 2020-02-28 15:35 Lakshminarayana R Kammath
2020-02-28 16:18 ` Brad Bishop
` (4 more replies)
0 siblings, 5 replies; 16+ messages in thread
From: Lakshminarayana R Kammath @ 2020-02-28 15:35 UTC (permalink / raw)
To: openbmc
[-- Attachment #1: Type: text/html, Size: 22121 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-02-28 15:35 [OpenBMC]: Unit test coverage analysis on openbmc repositories Lakshminarayana R Kammath @ 2020-02-28 16:18 ` Brad Bishop 2020-02-28 16:30 ` Lakshminarayana R Kammath ` (3 subsequent siblings) 4 siblings, 0 replies; 16+ messages in thread From: Brad Bishop @ 2020-02-28 16:18 UTC (permalink / raw) To: Lakshminarayana R Kammath; +Cc: openbmc at 10:35 AM, Lakshminarayana R Kammath <lkammath@in.ibm.com> wrote: > Hi All, > > Recently we worked on pulling the UT coverage analysis from all the > available OpenBMC repositories > > https://github.com/openbmc/openbmc-build-scripts/commit/229b76a95f87af60c976a0c0dfe84716c9ce5318 > > This script does following > • Clone the repo > • Use the CI build environment to build code > • Publish the result in the following format > Latest report: > *************************UNIT TEST COVERAGE REPORT************************* > https://github.com/openbmc/bmcweb.git NO > https://github.com/openbmc/boost-dbus.git ERROR > https://github.com/openbmc/btbridge.git NO > https://github.com/openbmc/dbus-sensors.git NO > https://github.com/openbmc/entity-manager.git NO > https://github.com/openbmc/fb-ipmi-oem.git NO > https://github.com/openbmc/google-ipmi-i2c.git ERROR > https://github.com/openbmc/google-ipmi-sys.git YES, UNIT TEST > https://github.com/openbmc/gpioplus.git YES, COVERAGE > https://github.com/openbmc/hiomapd.git YES, UNIT TEST > https://github.com/openbmc/ibm-dbus-interfaces.git NO > https://github.com/openbmc/ibm-logging.git YES, UNIT TEST > https://github.com/openbmc/ibm-pldm-oem.git ERROR > https://github.com/openbmc/inarp.git ERROR > https://github.com/openbmc/intel-dbus-interfaces.git NO > https://github.com/openbmc/intel-ipmi-oem.git NO > https://github.com/openbmc/ipmbbridge.git NO > https://github.com/openbmc/ipmi-blob-tool.git YES, UNIT TEST > https://github.com/openbmc/ipmi-fru-parser.git NO > https://github.com/openbmc/ipmitool.git ERROR > https://github.com/openbmc/jsnbd.git ERROR > https://github.com/openbmc/kcsbridge.git NO > https://github.com/openbmc/lenovo-ipmi-oem.git NO > https://github.com/openbmc/libmctp.git YES, UNIT TEST > https://github.com/openbmc/librrdplus.git NO > https://github.com/openbmc/obmc-console.git YES, UNIT TEST > https://github.com/openbmc/obmc-ikvm.git NO > https://github.com/openbmc/openbmc.git NO > https://github.com/openbmc/openbmc-tools.git NO > https://github.com/openbmc/openbmc.github.io.git NO > https://github.com/openbmc/openpower-dbus-interfaces.git NO > https://github.com/openbmc/openpower-debug-collector.git NO > https://github.com/openbmc/openpower-host-ipmi-flash.git NO > https://github.com/openbmc/openpower-host-ipmi-oem.git NO > https://github.com/openbmc/openpower-hw-diags.git YES, COVERAGE > https://github.com/openbmc/openpower-inventory-upload.git NO > https://github.com/openbmc/openpower-libhei.git YES, COVERAGE > https://github.com/openbmc/openpower-logging.git NO > https://github.com/openbmc/openpower-occ-control.git YES, UNIT TEST > https://github.com/openbmc/openpower-pnor-code-mgmt.git YES, UNIT TEST > https://github.com/openbmc/openpower-power-control.git NO > https://github.com/openbmc/openpower-proc-control.git YES, UNIT TEST > https://github.com/openbmc/openpower-sbe-interface.git NO > https://github.com/openbmc/openpower-vpd-parser.git YES, UNIT TEST > https://github.com/openbmc/pam-ipmi.git NO > https://github.com/openbmc/phosphor-bmc-code-mgmt.git YES, UNIT TEST > https://github.com/openbmc/phosphor-buttons.git NO > https://github.com/openbmc/phosphor-certificate-manager.git YES, UNIT TEST > https://github.com/openbmc/phosphor-dbus-interfaces.git NO > https://github.com/openbmc/phosphor-dbus-monitor.git YES, UNIT TEST > https://github.com/openbmc/phosphor-debug-collector.git NO > https://github.com/openbmc/phosphor-ecc.git NO > https://github.com/openbmc/phosphor-event.git YES, UNIT TEST > https://github.com/openbmc/phosphor-fan-presence.git YES, UNIT TEST > https://github.com/openbmc/phosphor-gpio-monitor.git YES, COVERAGE > https://github.com/openbmc/phosphor-host-ipmid.git ERROR > https://github.com/openbmc/phosphor-host-postd.git YES, UNIT TEST > https://github.com/openbmc/phosphor-hostlogger.git NO > https://github.com/openbmc/phosphor-hwmon.git NO > https://github.com/openbmc/phosphor-inventory-manager.git YES, UNIT TEST > https://github.com/openbmc/phosphor-ipmi-blobs.git YES, UNIT TEST > https://github.com/openbmc/phosphor-ipmi-blobs-binarystore.git YES, UNIT > TEST > https://github.com/openbmc/phosphor-ipmi-ethstats.git YES, UNIT TEST > https://github.com/openbmc/phosphor-ipmi-flash.git YES, UNIT TEST > https://github.com/openbmc/phosphor-led-manager.git YES, UNIT TEST > https://github.com/openbmc/phosphor-led-sysfs.git YES, UNIT TEST > https://github.com/openbmc/phosphor-logging.git YES, UNIT TEST > https://github.com/openbmc/phosphor-mboxd.git ERROR > https://github.com/openbmc/phosphor-misc.git NO > https://github.com/openbmc/phosphor-mrw-tools.git NO > https://github.com/openbmc/phosphor-net-ipmid.git YES, UNIT TEST > https://github.com/openbmc/phosphor-networkd.git YES, UNIT TEST > https://github.com/openbmc/phosphor-nvme.git NO > https://github.com/openbmc/phosphor-objmgr.git YES, UNIT TEST > https://github.com/openbmc/phosphor-pcie-presence.git NO > https://github.com/openbmc/phosphor-pid-control.git YES, UNIT TEST > https://github.com/openbmc/phosphor-post-code-manager.git NO > https://github.com/openbmc/phosphor-power.git YES, COVERAGE > https://github.com/openbmc/phosphor-power-control.git NO > https://github.com/openbmc/phosphor-power-monitor.git NO > https://github.com/openbmc/phosphor-psu-code-mgmt.git YES, COVERAGE > https://github.com/openbmc/phosphor-rest-server.git NO > https://github.com/openbmc/phosphor-sel-logger.git NO > https://github.com/openbmc/phosphor-settingsd.git NO > https://github.com/openbmc/phosphor-snmp.git ERROR > https://github.com/openbmc/phosphor-state-manager.git YES, COVERAGE > https://github.com/openbmc/phosphor-time-manager.git YES, UNIT TEST > https://github.com/openbmc/phosphor-u-boot-env-mgr.git NO > https://github.com/openbmc/phosphor-user-manager.git YES, UNIT TEST > https://github.com/openbmc/phosphor-watchdog.git YES, COVERAGE > https://github.com/openbmc/phosphor-webui.git NO > https://github.com/openbmc/pldm.git YES, COVERAGE > https://github.com/openbmc/pyphosphor.git NO > https://github.com/openbmc/qualcomm-ipmi-oem.git NO > https://github.com/openbmc/qualcomm-misc.git NO > https://github.com/openbmc/quanta-ipmi-oem.git NO > https://github.com/openbmc/rest-dbus.git NO > https://github.com/openbmc/rrd-ipmi-blobs.git NO > https://github.com/openbmc/s2600wf-misc.git NO > https://github.com/openbmc/sdbusplus.git YES, UNIT TEST > https://github.com/openbmc/sdeventplus.git YES, COVERAGE > https://github.com/openbmc/skeleton.git ERROR > https://github.com/openbmc/slpd-lite.git NO > https://github.com/openbmc/smbios-mdr.git ERROR > https://github.com/openbmc/stdplus.git YES, COVERAGE > https://github.com/openbmc/telemetry.git NO > https://github.com/openbmc/u-boot NO > https://github.com/openbmc/webui-vue.git NO > https://github.com/openbmc/witherspoon-pfault-analysis.git YES, COVERAGE > https://github.com/openbmc/x86-power-control.git NO > *************************UNIT TEST COVERAGE REPORT************************* > SUMMARY: > TOTAL REPOSITORIES : 110 > TESTED REPOSITORIES : 43 > ERROR : 11 > COVERAGE REPORT : 12 > UNIT TEST REPORT : 31 > NO REPORT : 56 > *************************************************************************** Very nice Lakshmi! Some of the repos have been archived. Can the script be updated to check for that and skip it if so? boost-dbus, inarp are a couple - there are likely more. ^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-02-28 15:35 [OpenBMC]: Unit test coverage analysis on openbmc repositories Lakshminarayana R Kammath 2020-02-28 16:18 ` Brad Bishop @ 2020-02-28 16:30 ` Lakshminarayana R Kammath 2020-02-28 16:31 ` Brad Bishop 2020-02-28 18:20 ` Patrick Venture ` (2 subsequent siblings) 4 siblings, 1 reply; 16+ messages in thread From: Lakshminarayana R Kammath @ 2020-02-28 16:30 UTC (permalink / raw) To: bradleyb; +Cc: openbmc [-- Attachment #1: Type: text/html, Size: 19589 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-02-28 16:30 ` Lakshminarayana R Kammath @ 2020-02-28 16:31 ` Brad Bishop 2020-02-28 16:36 ` Brad Bishop 2020-02-28 16:39 ` Lakshminarayana R Kammath 0 siblings, 2 replies; 16+ messages in thread From: Brad Bishop @ 2020-02-28 16:31 UTC (permalink / raw) To: Lakshminarayana R Kammath; +Cc: openbmc at 11:30 AM, Lakshminarayana R Kammath <lkammath@in.ibm.com> wrote: > sure Brad! > > If Some one can confirm the list of repo's that can be ignored we can put > them to blacklist Is it possible to use the GitHub api to query this? Then we don’t need to maintain a blacklist. How are you building the list of repos? ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-02-28 16:31 ` Brad Bishop @ 2020-02-28 16:36 ` Brad Bishop 2020-03-04 6:42 ` Lakshminarayana R Kammath 2020-02-28 16:39 ` Lakshminarayana R Kammath 1 sibling, 1 reply; 16+ messages in thread From: Brad Bishop @ 2020-02-28 16:36 UTC (permalink / raw) To: Lakshminarayana R Kammath; +Cc: openbmc at 11:31 AM, Brad Bishop <bradleyb@fuzziesquirrel.com> wrote: > at 11:30 AM, Lakshminarayana R Kammath <lkammath@in.ibm.com> wrote: > >> sure Brad! >> >> If Some one can confirm the list of repo's that can be ignored we can >> put them to blacklist > > Is it possible to use the GitHub api to query this? Then we don’t need > to maintain a blacklist. How are you building the list of repos? https://developer.github.com/v3/repos/#list-organization-repositories there is an “archived” property in the response to this list-org-repos api that seems like it would work? This looks like an interesting python module that might make the API interaction easier... https://pygithub.readthedocs.io/en/latest/ ^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-02-28 16:36 ` Brad Bishop @ 2020-03-04 6:42 ` Lakshminarayana R Kammath 0 siblings, 0 replies; 16+ messages in thread From: Lakshminarayana R Kammath @ 2020-03-04 6:42 UTC (permalink / raw) To: bradleyb; +Cc: openbmc [-- Attachment #1: Type: text/html, Size: 4411 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-02-28 16:31 ` Brad Bishop 2020-02-28 16:36 ` Brad Bishop @ 2020-02-28 16:39 ` Lakshminarayana R Kammath 2020-02-28 18:12 ` Nancy Yuen 1 sibling, 1 reply; 16+ messages in thread From: Lakshminarayana R Kammath @ 2020-02-28 16:39 UTC (permalink / raw) To: bradleyb; +Cc: openbmc [-- Attachment #1: Type: text/html, Size: 1735 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-02-28 16:39 ` Lakshminarayana R Kammath @ 2020-02-28 18:12 ` Nancy Yuen 0 siblings, 0 replies; 16+ messages in thread From: Nancy Yuen @ 2020-02-28 18:12 UTC (permalink / raw) To: Lakshminarayana R Kammath; +Cc: Brad Bishop, OpenBMC Maillist [-- Attachment #1: Type: text/plain, Size: 935 bytes --] This is great information! Thank ---------- Nancy On Fri, Feb 28, 2020 at 8:40 AM Lakshminarayana R Kammath < lkammath@in.ibm.com> wrote: > Yes, We can do that. Just need to figure out the API's > > Thanks & Regards, > Lakshminarayana Kamath > > > > ----- Original message ----- > From: Brad Bishop <bradleyb@fuzziesquirrel.com> > To: Lakshminarayana R Kammath <lkammath@in.ibm.com> > Cc: openbmc@lists.ozlabs.org > Subject: [EXTERNAL] Re: [OpenBMC]: Unit test coverage analysis on openbmc > repositories > Date: Fri, Feb 28, 2020 10:01 PM > > at 11:30 AM, Lakshminarayana R Kammath <lkammath@in.ibm.com> wrote: > > > sure Brad! > > > > If Some one can confirm the list of repo's that can be ignored we can > put > > them to blacklist > > Is it possible to use the GitHub api to query this? Then we don’t need to > > maintain a blacklist. How are you building the list of repos? > > > > > [-- Attachment #2: Type: text/html, Size: 2395 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-02-28 15:35 [OpenBMC]: Unit test coverage analysis on openbmc repositories Lakshminarayana R Kammath 2020-02-28 16:18 ` Brad Bishop 2020-02-28 16:30 ` Lakshminarayana R Kammath @ 2020-02-28 18:20 ` Patrick Venture 2020-03-05 1:25 ` James Feist 2020-03-03 4:48 ` Andrew Jeffery 2020-03-04 18:43 ` Andrew Geissler 4 siblings, 1 reply; 16+ messages in thread From: Patrick Venture @ 2020-02-28 18:20 UTC (permalink / raw) To: Lakshminarayana R Kammath; +Cc: OpenBMC Maillist On Fri, Feb 28, 2020 at 7:36 AM Lakshminarayana R Kammath <lkammath@in.ibm.com> wrote: > > Hi All, > > Recently we worked on pulling the UT coverage analysis from all the available OpenBMC repositories > > https://github.com/openbmc/openbmc-build-scripts/commit/229b76a95f87af60c976a0c0dfe84716c9ce5318 > > This script does following > > Clone the repo > Use the CI build environment to build code > Publish the result in the following format > > Latest report: > *************************UNIT TEST COVERAGE REPORT************************* > https://github.com/openbmc/bmcweb.git NO > https://github.com/openbmc/boost-dbus.git ERROR > https://github.com/openbmc/btbridge.git NO > https://github.com/openbmc/dbus-sensors.git NO > https://github.com/openbmc/entity-manager.git NO > https://github.com/openbmc/fb-ipmi-oem.git NO > https://github.com/openbmc/google-ipmi-i2c.git ERROR > https://github.com/openbmc/google-ipmi-sys.git YES, UNIT TEST > https://github.com/openbmc/gpioplus.git YES, COVERAGE > https://github.com/openbmc/hiomapd.git YES, UNIT TEST > https://github.com/openbmc/ibm-dbus-interfaces.git NO > https://github.com/openbmc/ibm-logging.git YES, UNIT TEST > https://github.com/openbmc/ibm-pldm-oem.git ERROR > https://github.com/openbmc/inarp.git ERROR > https://github.com/openbmc/intel-dbus-interfaces.git NO > https://github.com/openbmc/intel-ipmi-oem.git NO > https://github.com/openbmc/ipmbbridge.git NO > https://github.com/openbmc/ipmi-blob-tool.git YES, UNIT TEST > https://github.com/openbmc/ipmi-fru-parser.git NO > https://github.com/openbmc/ipmitool.git ERROR > https://github.com/openbmc/jsnbd.git ERROR > https://github.com/openbmc/kcsbridge.git NO > https://github.com/openbmc/lenovo-ipmi-oem.git NO > https://github.com/openbmc/libmctp.git YES, UNIT TEST > https://github.com/openbmc/librrdplus.git NO > https://github.com/openbmc/obmc-console.git YES, UNIT TEST > https://github.com/openbmc/obmc-ikvm.git NO > https://github.com/openbmc/openbmc.git NO > https://github.com/openbmc/openbmc-tools.git NO > https://github.com/openbmc/openbmc.github.io.git NO > https://github.com/openbmc/openpower-dbus-interfaces.git NO > https://github.com/openbmc/openpower-debug-collector.git NO > https://github.com/openbmc/openpower-host-ipmi-flash.git NO > https://github.com/openbmc/openpower-host-ipmi-oem.git NO > https://github.com/openbmc/openpower-hw-diags.git YES, COVERAGE > https://github.com/openbmc/openpower-inventory-upload.git NO > https://github.com/openbmc/openpower-libhei.git YES, COVERAGE > https://github.com/openbmc/openpower-logging.git NO > https://github.com/openbmc/openpower-occ-control.git YES, UNIT TEST > https://github.com/openbmc/openpower-pnor-code-mgmt.git YES, UNIT TEST > https://github.com/openbmc/openpower-power-control.git NO > https://github.com/openbmc/openpower-proc-control.git YES, UNIT TEST > https://github.com/openbmc/openpower-sbe-interface.git NO > https://github.com/openbmc/openpower-vpd-parser.git YES, UNIT TEST > https://github.com/openbmc/pam-ipmi.git NO > https://github.com/openbmc/phosphor-bmc-code-mgmt.git YES, UNIT TEST > https://github.com/openbmc/phosphor-buttons.git NO > https://github.com/openbmc/phosphor-certificate-manager.git YES, UNIT TEST > https://github.com/openbmc/phosphor-dbus-interfaces.git NO > https://github.com/openbmc/phosphor-dbus-monitor.git YES, UNIT TEST > https://github.com/openbmc/phosphor-debug-collector.git NO > https://github.com/openbmc/phosphor-ecc.git NO > https://github.com/openbmc/phosphor-event.git YES, UNIT TEST > https://github.com/openbmc/phosphor-fan-presence.git YES, UNIT TEST > https://github.com/openbmc/phosphor-gpio-monitor.git YES, COVERAGE > https://github.com/openbmc/phosphor-host-ipmid.git ERROR > https://github.com/openbmc/phosphor-host-postd.git YES, UNIT TEST > https://github.com/openbmc/phosphor-hostlogger.git NO > https://github.com/openbmc/phosphor-hwmon.git NO ^---- This one should be YES, UNIT TEST. Thanks for working on identifying missing coverage. I wanted to point this out so you can take another swing at this, -- Thanks! > https://github.com/openbmc/phosphor-inventory-manager.git YES, UNIT TEST > https://github.com/openbmc/phosphor-ipmi-blobs.git YES, UNIT TEST > https://github.com/openbmc/phosphor-ipmi-blobs-binarystore.git YES, UNIT TEST > https://github.com/openbmc/phosphor-ipmi-ethstats.git YES, UNIT TEST > https://github.com/openbmc/phosphor-ipmi-flash.git YES, UNIT TEST > https://github.com/openbmc/phosphor-led-manager.git YES, UNIT TEST > https://github.com/openbmc/phosphor-led-sysfs.git YES, UNIT TEST > https://github.com/openbmc/phosphor-logging.git YES, UNIT TEST > https://github.com/openbmc/phosphor-mboxd.git ERROR > https://github.com/openbmc/phosphor-misc.git NO > https://github.com/openbmc/phosphor-mrw-tools.git NO > https://github.com/openbmc/phosphor-net-ipmid.git YES, UNIT TEST > https://github.com/openbmc/phosphor-networkd.git YES, UNIT TEST > https://github.com/openbmc/phosphor-nvme.git NO > https://github.com/openbmc/phosphor-objmgr.git YES, UNIT TEST > https://github.com/openbmc/phosphor-pcie-presence.git NO > https://github.com/openbmc/phosphor-pid-control.git YES, UNIT TEST > https://github.com/openbmc/phosphor-post-code-manager.git NO > https://github.com/openbmc/phosphor-power.git YES, COVERAGE > https://github.com/openbmc/phosphor-power-control.git NO > https://github.com/openbmc/phosphor-power-monitor.git NO > https://github.com/openbmc/phosphor-psu-code-mgmt.git YES, COVERAGE > https://github.com/openbmc/phosphor-rest-server.git NO > https://github.com/openbmc/phosphor-sel-logger.git NO > https://github.com/openbmc/phosphor-settingsd.git NO > https://github.com/openbmc/phosphor-snmp.git ERROR > https://github.com/openbmc/phosphor-state-manager.git YES, COVERAGE > https://github.com/openbmc/phosphor-time-manager.git YES, UNIT TEST > https://github.com/openbmc/phosphor-u-boot-env-mgr.git NO > https://github.com/openbmc/phosphor-user-manager.git YES, UNIT TEST > https://github.com/openbmc/phosphor-watchdog.git YES, COVERAGE > https://github.com/openbmc/phosphor-webui.git NO > https://github.com/openbmc/pldm.git YES, COVERAGE > https://github.com/openbmc/pyphosphor.git NO > https://github.com/openbmc/qualcomm-ipmi-oem.git NO > https://github.com/openbmc/qualcomm-misc.git NO > https://github.com/openbmc/quanta-ipmi-oem.git NO > https://github.com/openbmc/rest-dbus.git NO > https://github.com/openbmc/rrd-ipmi-blobs.git NO > https://github.com/openbmc/s2600wf-misc.git NO > https://github.com/openbmc/sdbusplus.git YES, UNIT TEST > https://github.com/openbmc/sdeventplus.git YES, COVERAGE > https://github.com/openbmc/skeleton.git ERROR > https://github.com/openbmc/slpd-lite.git NO > https://github.com/openbmc/smbios-mdr.git ERROR > https://github.com/openbmc/stdplus.git YES, COVERAGE > https://github.com/openbmc/telemetry.git NO > https://github.com/openbmc/u-boot NO > https://github.com/openbmc/webui-vue.git NO > https://github.com/openbmc/witherspoon-pfault-analysis.git YES, COVERAGE > https://github.com/openbmc/x86-power-control.git NO > *************************UNIT TEST COVERAGE REPORT************************* > SUMMARY: > TOTAL REPOSITORIES : 110 > TESTED REPOSITORIES : 43 > ERROR : 11 > COVERAGE REPORT : 12 > UNIT TEST REPORT : 31 > NO REPORT : 56 > *************************************************************************** > > > Following actions, we can draw from above > > There are 12 repo's that have GCOV/LCOV plugged in - We need to maintain which has good coverage and try to improve those which has less coverage > There are 31 repo's that have unit test cases added but no coverage analysis because they aren't plugged in yet to GCOV/LCOV - Is there a plan for plugging in GCOV > There are 11 repo's which fail to compile and errors out - Fix compilation issue. Some of these maybe not used currently? > There are 56 repo's for which there is no UT tests available - This is a major issue we need to tackle and improve upon > > Feedback's , views highly appreciated > > Thanks & Regards, > Lakshminarayana Kamath > -------------------------------------------------------------------------------------------------- > Senior Engineer, > IBM India Software Labs > Bangalore > github:lkammath > --------------------------------------------------------------------------------------------------- > > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-02-28 18:20 ` Patrick Venture @ 2020-03-05 1:25 ` James Feist 0 siblings, 0 replies; 16+ messages in thread From: James Feist @ 2020-03-05 1:25 UTC (permalink / raw) To: Patrick Venture, Lakshminarayana R Kammath; +Cc: OpenBMC Maillist On 2/28/20 10:20 AM, Patrick Venture wrote: > On Fri, Feb 28, 2020 at 7:36 AM Lakshminarayana R Kammath > <lkammath@in.ibm.com> wrote: >> >> Hi All, >> >> Recently we worked on pulling the UT coverage analysis from all the available OpenBMC repositories >> >> https://github.com/openbmc/openbmc-build-scripts/commit/229b76a95f87af60c976a0c0dfe84716c9ce5318 >> >> This script does following >> >> Clone the repo >> Use the CI build environment to build code >> Publish the result in the following format >> >> Latest report: >> *************************UNIT TEST COVERAGE REPORT************************* >> https://github.com/openbmc/bmcweb.git NO >> https://github.com/openbmc/boost-dbus.git ERROR >> https://github.com/openbmc/btbridge.git NO >> https://github.com/openbmc/dbus-sensors.git NO >> https://github.com/openbmc/entity-manager.git NO This has some tests: https://github.com/openbmc/entity-manager/blob/master/test/test_entity-manager.cpp >> https://github.com/openbmc/fb-ipmi-oem.git NO >> https://github.com/openbmc/google-ipmi-i2c.git ERROR >> https://github.com/openbmc/google-ipmi-sys.git YES, UNIT TEST >> https://github.com/openbmc/gpioplus.git YES, COVERAGE >> https://github.com/openbmc/hiomapd.git YES, UNIT TEST >> https://github.com/openbmc/ibm-dbus-interfaces.git NO >> https://github.com/openbmc/ibm-logging.git YES, UNIT TEST >> https://github.com/openbmc/ibm-pldm-oem.git ERROR >> https://github.com/openbmc/inarp.git ERROR >> https://github.com/openbmc/intel-dbus-interfaces.git NO >> https://github.com/openbmc/intel-ipmi-oem.git NO So does this: https://github.com/openbmc/intel-ipmi-oem/blob/master/tests/test_sensorcommands.cpp >> https://github.com/openbmc/ipmbbridge.git NO >> https://github.com/openbmc/ipmi-blob-tool.git YES, UNIT TEST >> https://github.com/openbmc/ipmi-fru-parser.git NO >> https://github.com/openbmc/ipmitool.git ERROR >> https://github.com/openbmc/jsnbd.git ERROR >> https://github.com/openbmc/kcsbridge.git NO >> https://github.com/openbmc/lenovo-ipmi-oem.git NO >> https://github.com/openbmc/libmctp.git YES, UNIT TEST >> https://github.com/openbmc/librrdplus.git NO >> https://github.com/openbmc/obmc-console.git YES, UNIT TEST >> https://github.com/openbmc/obmc-ikvm.git NO >> https://github.com/openbmc/openbmc.git NO >> https://github.com/openbmc/openbmc-tools.git NO >> https://github.com/openbmc/openbmc.github.io.git NO >> https://github.com/openbmc/openpower-dbus-interfaces.git NO >> https://github.com/openbmc/openpower-debug-collector.git NO >> https://github.com/openbmc/openpower-host-ipmi-flash.git NO >> https://github.com/openbmc/openpower-host-ipmi-oem.git NO >> https://github.com/openbmc/openpower-hw-diags.git YES, COVERAGE >> https://github.com/openbmc/openpower-inventory-upload.git NO >> https://github.com/openbmc/openpower-libhei.git YES, COVERAGE >> https://github.com/openbmc/openpower-logging.git NO >> https://github.com/openbmc/openpower-occ-control.git YES, UNIT TEST >> https://github.com/openbmc/openpower-pnor-code-mgmt.git YES, UNIT TEST >> https://github.com/openbmc/openpower-power-control.git NO >> https://github.com/openbmc/openpower-proc-control.git YES, UNIT TEST >> https://github.com/openbmc/openpower-sbe-interface.git NO >> https://github.com/openbmc/openpower-vpd-parser.git YES, UNIT TEST >> https://github.com/openbmc/pam-ipmi.git NO >> https://github.com/openbmc/phosphor-bmc-code-mgmt.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-buttons.git NO >> https://github.com/openbmc/phosphor-certificate-manager.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-dbus-interfaces.git NO >> https://github.com/openbmc/phosphor-dbus-monitor.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-debug-collector.git NO >> https://github.com/openbmc/phosphor-ecc.git NO >> https://github.com/openbmc/phosphor-event.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-fan-presence.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-gpio-monitor.git YES, COVERAGE >> https://github.com/openbmc/phosphor-host-ipmid.git ERROR >> https://github.com/openbmc/phosphor-host-postd.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-hostlogger.git NO >> https://github.com/openbmc/phosphor-hwmon.git NO > > ^---- This one should be YES, UNIT TEST. > > Thanks for working on identifying missing coverage. I wanted to point > this out so you can take another swing at this, -- Thanks! > >> https://github.com/openbmc/phosphor-inventory-manager.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-ipmi-blobs.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-ipmi-blobs-binarystore.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-ipmi-ethstats.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-ipmi-flash.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-led-manager.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-led-sysfs.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-logging.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-mboxd.git ERROR >> https://github.com/openbmc/phosphor-misc.git NO >> https://github.com/openbmc/phosphor-mrw-tools.git NO >> https://github.com/openbmc/phosphor-net-ipmid.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-networkd.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-nvme.git NO >> https://github.com/openbmc/phosphor-objmgr.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-pcie-presence.git NO >> https://github.com/openbmc/phosphor-pid-control.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-post-code-manager.git NO >> https://github.com/openbmc/phosphor-power.git YES, COVERAGE >> https://github.com/openbmc/phosphor-power-control.git NO >> https://github.com/openbmc/phosphor-power-monitor.git NO >> https://github.com/openbmc/phosphor-psu-code-mgmt.git YES, COVERAGE >> https://github.com/openbmc/phosphor-rest-server.git NO >> https://github.com/openbmc/phosphor-sel-logger.git NO >> https://github.com/openbmc/phosphor-settingsd.git NO >> https://github.com/openbmc/phosphor-snmp.git ERROR >> https://github.com/openbmc/phosphor-state-manager.git YES, COVERAGE >> https://github.com/openbmc/phosphor-time-manager.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-u-boot-env-mgr.git NO >> https://github.com/openbmc/phosphor-user-manager.git YES, UNIT TEST >> https://github.com/openbmc/phosphor-watchdog.git YES, COVERAGE >> https://github.com/openbmc/phosphor-webui.git NO >> https://github.com/openbmc/pldm.git YES, COVERAGE >> https://github.com/openbmc/pyphosphor.git NO >> https://github.com/openbmc/qualcomm-ipmi-oem.git NO >> https://github.com/openbmc/qualcomm-misc.git NO >> https://github.com/openbmc/quanta-ipmi-oem.git NO >> https://github.com/openbmc/rest-dbus.git NO >> https://github.com/openbmc/rrd-ipmi-blobs.git NO >> https://github.com/openbmc/s2600wf-misc.git NO >> https://github.com/openbmc/sdbusplus.git YES, UNIT TEST >> https://github.com/openbmc/sdeventplus.git YES, COVERAGE >> https://github.com/openbmc/skeleton.git ERROR >> https://github.com/openbmc/slpd-lite.git NO >> https://github.com/openbmc/smbios-mdr.git ERROR >> https://github.com/openbmc/stdplus.git YES, COVERAGE >> https://github.com/openbmc/telemetry.git NO >> https://github.com/openbmc/u-boot NO >> https://github.com/openbmc/webui-vue.git NO >> https://github.com/openbmc/witherspoon-pfault-analysis.git YES, COVERAGE >> https://github.com/openbmc/x86-power-control.git NO >> *************************UNIT TEST COVERAGE REPORT************************* >> SUMMARY: >> TOTAL REPOSITORIES : 110 >> TESTED REPOSITORIES : 43 >> ERROR : 11 >> COVERAGE REPORT : 12 >> UNIT TEST REPORT : 31 >> NO REPORT : 56 >> *************************************************************************** >> >> >> Following actions, we can draw from above >> >> There are 12 repo's that have GCOV/LCOV plugged in - We need to maintain which has good coverage and try to improve those which has less coverage >> There are 31 repo's that have unit test cases added but no coverage analysis because they aren't plugged in yet to GCOV/LCOV - Is there a plan for plugging in GCOV >> There are 11 repo's which fail to compile and errors out - Fix compilation issue. Some of these maybe not used currently? >> There are 56 repo's for which there is no UT tests available - This is a major issue we need to tackle and improve upon >> >> Feedback's , views highly appreciated >> >> Thanks & Regards, >> Lakshminarayana Kamath >> -------------------------------------------------------------------------------------------------- >> Senior Engineer, >> IBM India Software Labs >> Bangalore >> github:lkammath >> --------------------------------------------------------------------------------------------------- >> >> ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-02-28 15:35 [OpenBMC]: Unit test coverage analysis on openbmc repositories Lakshminarayana R Kammath ` (2 preceding siblings ...) 2020-02-28 18:20 ` Patrick Venture @ 2020-03-03 4:48 ` Andrew Jeffery 2020-03-03 6:16 ` Lakshminarayana R Kammath 2020-03-05 7:57 ` Lakshminarayana R Kammath 2020-03-04 18:43 ` Andrew Geissler 4 siblings, 2 replies; 16+ messages in thread From: Andrew Jeffery @ 2020-03-03 4:48 UTC (permalink / raw) To: Lakshminarayana R Kammath, openbmc On Sat, 29 Feb 2020, at 02:05, Lakshminarayana R Kammath wrote: > https://github.com/openbmc/obmc-console.git YES, UNIT TEST I need to add coverage support to obmc-console. > https://github.com/openbmc/openbmc-tools.git NO Not sure we can expect anything for openbmc-tools - it's a collection of random scripts people have written to make their life developing openbmc easier. It's not something that should have strict QA processes. > https://github.com/openbmc/openpower-host-ipmi-flash.git NO I should add some tests here where I can. > https://github.com/openbmc/hiomapd.git YES, UNIT TEST > https://github.com/openbmc/libmctp.git YES, UNIT TEST > https://github.com/openbmc/phosphor-led-sysfs.git YES, UNIT TEST These three do in-fact have GCOV/LCOV support, you need to do the usual autotools invocations to get it: ``` ./configure --enable-code-coverage && make check-code-coverage ``` > https://github.com/openbmc/phosphor-mboxd.git ERROR This is archived. ... > https://github.com/openbmc/u-boot NO This uses Kbuild as a build system. I notice openbmc/linux and openbmc/qemu aren't included in the list. > https://github.com/openbmc/webui-vue.git NO > https://github.com/openbmc/witherspoon-pfault-analysis.git YES, COVERAGE > https://github.com/openbmc/x86-power-control.git NO > *************************UNIT TEST COVERAGE > REPORT************************* > SUMMARY: > TOTAL REPOSITORIES : 110 > TESTED REPOSITORIES : 43 > ERROR : 11 > COVERAGE REPORT : 12 > UNIT TEST REPORT : 31 > NO REPORT : 56 > *************************************************************************** > Following actions, we can draw from above > > 1. *There are 12 repo's that have GCOV/LCOV plugged in* - We need to > maintain which has good coverage and try to improve those which has > less coverage > 2. *There are 31 repo's that have unit test cases added but no > coverage analysis because they aren't plugged in yet to GCOV/LCOV* - Is > there a plan for plugging in GCOV As above at least three of these are false-negatives for test coverage. Andrew ^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-03-03 4:48 ` Andrew Jeffery @ 2020-03-03 6:16 ` Lakshminarayana R Kammath 2020-03-05 7:57 ` Lakshminarayana R Kammath 1 sibling, 0 replies; 16+ messages in thread From: Lakshminarayana R Kammath @ 2020-03-03 6:16 UTC (permalink / raw) To: andrew; +Cc: openbmc [-- Attachment #1: Type: text/html, Size: 5325 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-03-03 4:48 ` Andrew Jeffery 2020-03-03 6:16 ` Lakshminarayana R Kammath @ 2020-03-05 7:57 ` Lakshminarayana R Kammath 2020-03-06 9:27 ` Lei YU 1 sibling, 1 reply; 16+ messages in thread From: Lakshminarayana R Kammath @ 2020-03-05 7:57 UTC (permalink / raw) To: andrew; +Cc: openbmc [-- Attachment #1: Type: text/html, Size: 8614 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-03-05 7:57 ` Lakshminarayana R Kammath @ 2020-03-06 9:27 ` Lei YU 0 siblings, 0 replies; 16+ messages in thread From: Lei YU @ 2020-03-06 9:27 UTC (permalink / raw) To: Lakshminarayana R Kammath; +Cc: Andrew Jeffery, OpenBMC Maillist It's found that the test coverage result is not correct for some repos. E.g. for phosphor-psu-code-mgmt, there are quite a few tests for code in `src`, but it only shows 2.5%: https://openpower.xyz/job/openbmc-unit-test-coverage/2/artifact/openbmc-build-scripts/scripts/unit-test-meta-data/phosphor-psu-code-mgmt/build/meson-logs/coveragereport/index.html It looks like `lcov` is not handling such case well, and there is a similar question on StackOverflow: https://stackoverflow.com/questions/27556872/lcov-doesnt-match-gcov-coverage-results-for-a-googletest-unit-test I tried to remove `lcov` and use `gcovr` in local CI, and the result looks correct. `lcov` is required for autotools' coverage, so we can not just remove it. If I add `gcovr` into the docker image without removing `lcov`, unfortunately, meson still uses `lcov` for `coverage-html` when both tools are available. An issue is created at https://github.com/mesonbuild/meson/issues/6747, hopefully we could get some feedback from meson. In OpenBMC, do you have any ideas on how to "fix" the coverage result? On Thu, Mar 5, 2020 at 3:59 PM Lakshminarayana R Kammath <lkammath@in.ibm.com> wrote: > > Hi All, > > please confirm on following > > Boost-bus - compilation fails . Is it used? Should be archived @andrewj? > > Google-ipmi-i2c - code compiles well, complain in formatting script @pstrinkle > subsys_initcall(i2c_via_ipmi_init);^[[m^M > Traceback (most recent call last):^M > File "/home/lkammath/final_data/unit-test.py", line 826, in <module>^M > check_call_cmd("./format-code.sh", CODE_SCAN_DIR)^M > File "/home/lkammath/final_data/unit-test.py", line 229, in check_call_cmd^M > check_call(cmd)^M > File "/usr/lib/python2.7/subprocess.py", line 190, in check_call^M > raise CalledProcessError(retcode, cmd)^M > subprocess.CalledProcessError: Command '('./format-code.sh', '/home/lkammath/final_data/google-ipmi-i2c')' returned non-zero exit status 1^M > Traceback (most recent call last):^M > File "/home/lkammath/final_data/dbus-unit-test.py", line 91, in <module>^M > > Inarp.git - Should be archived @andrewj? > Compilation issue > inarp.c: In function ‘arp_recv’: > inarp.c:433:4: error: taking address of packed member of ‘struct arp_packet’ may result in an unaligned pointer value [-Werror=address-of-packed-member] > 433 | &inarp_req.src_ip); > | ^~~~~~~~~~~~~~~~~ > In function ‘do_ifreq’, > inlined from ‘get_local_ipaddr’ at inarp.c:182:7, > inlined from ‘arp_recv’ at inarp.c:419:7, > inlined from ‘main’ at inarp.c:480:4: > inarp.c:170:2: error: ‘strncpy’ output may be truncated copying 16 bytes from a string of length 16 [-Werror=stringop-truncation] > 170 | strncpy(ifreq->ifr_name, ifname, sizeof(ifreq->ifr_name)); > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > cc1: all warnings being treated as errors > make[1]: *** [inarp.o] Error 1 > make[1]: Leaving directory `/esw/san5/lkammath/boost/inarp' > make: *** [all] Error 2 > In CI environment > ool.la -lncurses -ltinfo -lcrypto^M > copying selected object files to avoid basename conflicts...^M > ar: `u' modifier ignored since `D' is the default (see `U')^M > make[4]: Leaving directory '/home/lkammath/final_data/ipmitool/src/plugins/lan'^M > Making all in lanplus^M > make[4]: Entering directory '/home/lkammath/final_data/ipmitool/src/plugins/lanplus'^M > /bin/bash ../../../libtool --silent --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../include -g -O2 -Wall -Wextra -std=c99 -pedantic -Wformat -Wformat-nonliteral -MT lanplus_crypt_impl.lo -MD -MP -MF .deps/lanplus_crypt_impl.Tpo -c -o lanplus_crypt_impl.lo lanplus_crypt_impl.c^M > lanplus_crypt_impl.c: In function 'lanplus_encrypt_aes_cbc_128':^M > lanplus_crypt_impl.c:158:17: error: storage size of 'ctx' isn't known^M > 158 | EVP_CIPHER_CTX ctx;^M > | ^~~^M > lanplus_crypt_impl.c:158:17: warning: unused variable 'ctx' [-Wunused-variable]^M > lanplus_crypt_impl.c: In function 'lanplus_decrypt_aes_cbc_128':^M > lanplus_crypt_impl.c:233:17: error: storage size of 'ctx' isn't known^M > 233 | EVP_CIPHER_CTX ctx;^M > | ^~~^M > > Smbios-mdr - @brad. Please confirm if used? > No automake env (make) > > Thanks & Regards, > Lakshminarayana Kamath > -------------------------------------------------------------------------------------------------- > > > > ----- Original message ----- > From: "Andrew Jeffery" <andrew@aj.id.au> > To: "Lakshminarayana R Kammath" <lkammath@in.ibm.com>, openbmc@lists.ozlabs.org > Cc: > Subject: [EXTERNAL] Re: [OpenBMC]: Unit test coverage analysis on openbmc repositories > Date: Tue, Mar 3, 2020 10:18 AM > > > On Sat, 29 Feb 2020, at 02:05, Lakshminarayana R Kammath wrote: > > > https://github.com/openbmc/obmc-console.git YES, UNIT TEST > > I need to add coverage support to obmc-console. > > > https://github.com/openbmc/openbmc-tools.git NO > > Not sure we can expect anything for openbmc-tools - it's a collection > of random scripts people have written to make their life developing > openbmc easier. It's not something that should have strict QA > processes. > > > https://github.com/openbmc/openpower-host-ipmi-flash.git NO > > I should add some tests here where I can. > > > https://github.com/openbmc/hiomapd.git YES, UNIT TEST > > https://github.com/openbmc/libmctp.git YES, UNIT TEST > > https://github.com/openbmc/phosphor-led-sysfs.git YES, UNIT TEST > > These three do in-fact have GCOV/LCOV support, you need to do the usual autotools invocations to get it: > > ``` > ./configure --enable-code-coverage && make check-code-coverage > ``` > > > https://github.com/openbmc/phosphor-mboxd.git ERROR > > This is archived. > > ... > > > https://github.com/openbmc/u-boot NO > > This uses Kbuild as a build system. I notice openbmc/linux and openbmc/qemu aren't included in the list. > > > https://github.com/openbmc/webui-vue.git NO > > https://github.com/openbmc/witherspoon-pfault-analysis.git YES, COVERAGE > > https://github.com/openbmc/x86-power-control.git NO > > *************************UNIT TEST COVERAGE > > REPORT************************* > > SUMMARY: > > TOTAL REPOSITORIES : 110 > > TESTED REPOSITORIES : 43 > > ERROR : 11 > > COVERAGE REPORT : 12 > > UNIT TEST REPORT : 31 > > NO REPORT : 56 > > *************************************************************************** > > Following actions, we can draw from above > > > > 1. *There are 12 repo's that have GCOV/LCOV plugged in* - We need to > > maintain which has good coverage and try to improve those which has > > less coverage > > 2. *There are 31 repo's that have unit test cases added but no > > coverage analysis because they aren't plugged in yet to GCOV/LCOV* - Is > > there a plan for plugging in GCOV > > As above at least three of these are false-negatives for test coverage. > > Andrew > > > > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [OpenBMC]: Unit test coverage analysis on openbmc repositories 2020-02-28 15:35 [OpenBMC]: Unit test coverage analysis on openbmc repositories Lakshminarayana R Kammath ` (3 preceding siblings ...) 2020-03-03 4:48 ` Andrew Jeffery @ 2020-03-04 18:43 ` Andrew Geissler 4 siblings, 0 replies; 16+ messages in thread From: Andrew Geissler @ 2020-03-04 18:43 UTC (permalink / raw) To: Lakshminarayana R Kammath; +Cc: openbmc > On Feb 28, 2020, at 9:35 AM, Lakshminarayana R Kammath <lkammath@in.ibm.com> wrote: > > Hi All, > > Recently we worked on pulling the UT coverage analysis from all the available OpenBMC repositories > > https://github.com/openbmc/openbmc-build-scripts/commit/229b76a95f87af60c976a0c0dfe84716c9ce5318 > > This script does following > • Clone the repo > • Use the CI build environment to build code > • Publish the result in the following format > Feedback's , views highly appreciated Good stuff. I added this to a jenkins job which will run each day. https://openpower.xyz/job/openbmc-unit-test-coverage/ You can then look at coverage info for your repo of interest by clicking through the artifacts and getting to your coverage info. For example: https://openpower.xyz/job/openbmc-unit-test-coverage/lastSuccessfulBuild/artifact/openbmc-build-scripts/scripts/unit-test-meta-data/pldm/build/meson-logs/coveragereport/index.html The job uses the following to know which repositories to check so feel free to throw up some commits if there are missing or invalid repos: https://github.com/openbmc/openbmc-build-scripts/blob/master/scripts/repositories.txt > > Thanks & Regards, > Lakshminarayana Kamath > -------------------------------------------------------------------------------------------------- > Senior Engineer, > IBM India Software Labs > Bangalore > github:lkammath > --------------------------------------------------------------------------------------------------- > > ^ permalink raw reply [flat|nested] 16+ messages in thread
* [OpenBMC]: Unit test coverage analysis on openbmc repositories @ 2020-02-28 16:08 lakshminarayan kamath 0 siblings, 0 replies; 16+ messages in thread From: lakshminarayan kamath @ 2020-02-28 16:08 UTC (permalink / raw) To: openbmc [-- Attachment #1: Type: text/plain, Size: 7920 bytes --] Hi All, Somehow my earlier mail did not get displayed properly Recently we worked on pulling the UT coverage analysis from all the available OpenBMC repositories https://github.com/openbmc/openbmc-build-scripts/commit/229b76a95f87af60c976a0c0dfe84716c9ce5318 This script does following 1. Clone the repo 2. Use the CI build environment to build code 3. Publish the result in the following format Latest report: *************************UNIT TEST COVERAGE REPORT************************* https://github.com/openbmc/bmcweb.git NO https://github.com/openbmc/boost-dbus.git ERROR https://github.com/openbmc/btbridge.git NO https://github.com/openbmc/dbus-sensors.git NO https://github.com/openbmc/entity-manager.git NO https://github.com/openbmc/fb-ipmi-oem.git NO https://github.com/openbmc/google-ipmi-i2c.git ERROR https://github.com/openbmc/google-ipmi-sys.git YES, UNIT TEST https://github.com/openbmc/gpioplus.git YES, COVERAGE https://github.com/openbmc/hiomapd.git YES, UNIT TEST https://github.com/openbmc/ibm-dbus-interfaces.git NO https://github.com/openbmc/ibm-logging.git YES, UNIT TEST https://github.com/openbmc/ibm-pldm-oem.git ERROR https://github.com/openbmc/inarp.git ERROR https://github.com/openbmc/intel-dbus-interfaces.git NO https://github.com/openbmc/intel-ipmi-oem.git NO https://github.com/openbmc/ipmbbridge.git NO https://github.com/openbmc/ipmi-blob-tool.git YES, UNIT TEST https://github.com/openbmc/ipmi-fru-parser.git NO https://github.com/openbmc/ipmitool.git ERROR https://github.com/openbmc/jsnbd.git ERROR https://github.com/openbmc/kcsbridge.git NO https://github.com/openbmc/lenovo-ipmi-oem.git NO https://github.com/openbmc/libmctp.git YES, UNIT TEST https://github.com/openbmc/librrdplus.git NO https://github.com/openbmc/obmc-console.git YES, UNIT TEST https://github.com/openbmc/obmc-ikvm.git NO https://github.com/openbmc/openbmc.git NO https://github.com/openbmc/openbmc-tools.git NO https://github.com/openbmc/openbmc.github.io.git NO https://github.com/openbmc/openpower-dbus-interfaces.git NO https://github.com/openbmc/openpower-debug-collector.git NO https://github.com/openbmc/openpower-host-ipmi-flash.git NO https://github.com/openbmc/openpower-host-ipmi-oem.git NO https://github.com/openbmc/openpower-hw-diags.git YES, COVERAGE https://github.com/openbmc/openpower-inventory-upload.git NO https://github.com/openbmc/openpower-libhei.git YES, COVERAGE https://github.com/openbmc/openpower-logging.git NO https://github.com/openbmc/openpower-occ-control.git YES, UNIT TEST https://github.com/openbmc/openpower-pnor-code-mgmt.git YES, UNIT TEST https://github.com/openbmc/openpower-power-control.git NO https://github.com/openbmc/openpower-proc-control.git YES, UNIT TEST https://github.com/openbmc/openpower-sbe-interface.git NO https://github.com/openbmc/openpower-vpd-parser.git YES, UNIT TEST https://github.com/openbmc/pam-ipmi.git NO https://github.com/openbmc/phosphor-bmc-code-mgmt.git YES, UNIT TEST https://github.com/openbmc/phosphor-buttons.git NO https://github.com/openbmc/phosphor-certificate-manager.git YES, UNIT TEST https://github.com/openbmc/phosphor-dbus-interfaces.git NO https://github.com/openbmc/phosphor-dbus-monitor.git YES, UNIT TEST https://github.com/openbmc/phosphor-debug-collector.git NO https://github.com/openbmc/phosphor-ecc.git NO https://github.com/openbmc/phosphor-event.git YES, UNIT TEST https://github.com/openbmc/phosphor-fan-presence.git YES, UNIT TEST https://github.com/openbmc/phosphor-gpio-monitor.git YES, COVERAGE https://github.com/openbmc/phosphor-host-ipmid.git ERROR https://github.com/openbmc/phosphor-host-postd.git YES, UNIT TEST https://github.com/openbmc/phosphor-hostlogger.git NO https://github.com/openbmc/phosphor-hwmon.git NO https://github.com/openbmc/phosphor-inventory-manager.git YES, UNIT TEST https://github.com/openbmc/phosphor-ipmi-blobs.git YES, UNIT TEST https://github.com/openbmc/phosphor-ipmi-blobs-binarystore.git YES, UNIT TEST https://github.com/openbmc/phosphor-ipmi-ethstats.git YES, UNIT TEST https://github.com/openbmc/phosphor-ipmi-flash.git YES, UNIT TEST https://github.com/openbmc/phosphor-led-manager.git YES, UNIT TEST https://github.com/openbmc/phosphor-led-sysfs.git YES, UNIT TEST https://github.com/openbmc/phosphor-logging.git YES, UNIT TEST https://github.com/openbmc/phosphor-mboxd.git ERROR https://github.com/openbmc/phosphor-misc.git NO https://github.com/openbmc/phosphor-mrw-tools.git NO https://github.com/openbmc/phosphor-net-ipmid.git YES, UNIT TEST https://github.com/openbmc/phosphor-networkd.git YES, UNIT TEST https://github.com/openbmc/phosphor-nvme.git NO https://github.com/openbmc/phosphor-objmgr.git YES, UNIT TEST https://github.com/openbmc/phosphor-pcie-presence.git NO https://github.com/openbmc/phosphor-pid-control.git YES, UNIT TEST https://github.com/openbmc/phosphor-post-code-manager.git NO https://github.com/openbmc/phosphor-power.git YES, COVERAGE https://github.com/openbmc/phosphor-power-control.git NO https://github.com/openbmc/phosphor-power-monitor.git NO https://github.com/openbmc/phosphor-psu-code-mgmt.git YES, COVERAGE https://github.com/openbmc/phosphor-rest-server.git NO https://github.com/openbmc/phosphor-sel-logger.git NO https://github.com/openbmc/phosphor-settingsd.git NO https://github.com/openbmc/phosphor-snmp.git ERROR https://github.com/openbmc/phosphor-state-manager.git YES, COVERAGE https://github.com/openbmc/phosphor-time-manager.git YES, UNIT TEST https://github.com/openbmc/phosphor-u-boot-env-mgr.git NO https://github.com/openbmc/phosphor-user-manager.git YES, UNIT TEST https://github.com/openbmc/phosphor-watchdog.git YES, COVERAGE https://github.com/openbmc/phosphor-webui.git NO https://github.com/openbmc/pldm.git YES, COVERAGE https://github.com/openbmc/pyphosphor.git NO https://github.com/openbmc/qualcomm-ipmi-oem.git NO https://github.com/openbmc/qualcomm-misc.git NO https://github.com/openbmc/quanta-ipmi-oem.git NO https://github.com/openbmc/rest-dbus.git NO https://github.com/openbmc/rrd-ipmi-blobs.git NO https://github.com/openbmc/s2600wf-misc.git NO https://github.com/openbmc/sdbusplus.git YES, UNIT TEST https://github.com/openbmc/sdeventplus.git YES, COVERAGE https://github.com/openbmc/skeleton.git ERROR https://github.com/openbmc/slpd-lite.git NO https://github.com/openbmc/smbios-mdr.git ERROR https://github.com/openbmc/stdplus.git YES, COVERAGE https://github.com/openbmc/telemetry.git NO https://github.com/openbmc/u-boot NO https://github.com/openbmc/webui-vue.git NO https://github.com/openbmc/witherspoon-pfault-analysis.git YES, COVERAGE https://github.com/openbmc/x86-power-control.git NO *************************UNIT TEST COVERAGE REPORT************************* SUMMARY: TOTAL REPOSITORIES : 110 TESTED REPOSITORIES : 43 ERROR : 11 COVERAGE REPORT : 12 UNIT TEST REPORT : 31 NO REPORT : 56 *************************************************************************** Following actions, we can draw from above 1. *There are 12 repo's that have GCOV/LCOV plugged in* - We need to maintain which has good coverage and try to improve those which has less coverage 2. *There are 31 repo's that have unit test cases added but no coverage analysis because they aren't plugged in yet to GCOV/LCOV* - Is there a plan for plugging in GCOV 3. *There are 11 repo's which fail to compile and errors out* - Fix compilation issue. Some of these maybe not used currently? 4. *There are 56 repo's for which there is no UT tests availabl*e - This is a major issue we need to tackle and improve upon Feedback's, views highly appreciated Thanks & Regards, Lakshminarayana Kamath -------------------------------------------------------------------------------------------------- Senior Engineer, IBM India Software Labs Bangalore github:lkammath --------------------------------------------------------------------------------------------------- [-- Attachment #2: Type: text/html, Size: 22009 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2020-03-06 9:27 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-02-28 15:35 [OpenBMC]: Unit test coverage analysis on openbmc repositories Lakshminarayana R Kammath 2020-02-28 16:18 ` Brad Bishop 2020-02-28 16:30 ` Lakshminarayana R Kammath 2020-02-28 16:31 ` Brad Bishop 2020-02-28 16:36 ` Brad Bishop 2020-03-04 6:42 ` Lakshminarayana R Kammath 2020-02-28 16:39 ` Lakshminarayana R Kammath 2020-02-28 18:12 ` Nancy Yuen 2020-02-28 18:20 ` Patrick Venture 2020-03-05 1:25 ` James Feist 2020-03-03 4:48 ` Andrew Jeffery 2020-03-03 6:16 ` Lakshminarayana R Kammath 2020-03-05 7:57 ` Lakshminarayana R Kammath 2020-03-06 9:27 ` Lei YU 2020-03-04 18:43 ` Andrew Geissler 2020-02-28 16:08 lakshminarayan kamath
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.