Re: [PATCH RFC CFH][sumo 00/47] CVE check backport

[akuster808 <akuster808@gmail.com>]

Hello Mikko;

On 11/6/19 7:37 AM, Mikko Rapeli wrote:
> Hi,
>
> Request for comments, call for help, LTS too?
>
> Yocto 2.5 sumo isn't actively maintained by the Yocto Project
> anymore. But that does not mean that support for it
> needs to stop.
>
> I use sumo and due to various reasons like BSP layers, binary
> compatibility, contracts etc can't update to newer release
> or to master branch. I suspect I'm not alone.
>
> sumo CVE checking machinery is broken due to changes in
> NIST and NVD (see
> https://nvd.nist.gov/general/news/XML-Vulnerability-Feed-Retirement and
> https://nvd.nist.gov/General/News/JSON-1-1-Vulnerability-Feed-Release )
> so some backports from poky master/zeus are needed to fix the
> tooling. Thanks to Anuj, Chen, Chin, Pierre, Ross and others
> who fixed these on master branch!
>
> The tooling will expose that sumo is severely lacking in security
> patches, but the tooling is a start for anyone interested, like me,
> to fill the gaps and publish patches for bitbake recipes we care
> about.
>
> Could sumo be an LTS? 
No, IMHO. We don't have the appropriate system in place to do so.

> Well I hope so. The LTS proposal
> http://lists.openembedded.org/pipermail/openembedded-architecture/2019-October/001665.html
> https://docs.google.com/document/d/1AwAFDf52f_FoXksbHEVUMlu4hpcI0JMGVG-Kj_sUkyc/edit
> from Yocto Project is great. Maybe as part of that work, someone could
> setup a really minimal set of QA on Yocto Project side to also test

QA resources have been a donation from Intel and Windriver above their
membership fees.  I don't fee right asking them to run QA.
> patches aiming at yocto 2.5 sumo. If not, would be really nice if
> someone could collect patches into sumo-next or sumo-contrib branch where us
> users could be in charge of all Quality Assurance.
I have collected other patches for sumo and built them locally but I
have no way to inform Richard they pass an AB  builds or automated
testing for them to get  into mainline sumo.

I am placing them into
https://git.openembedded.org/openembedded-core-contrib/log/?h=stable/sumo-community


>
> So, comments and review are welcome. Patches even more so!

i will look at them when i get back from vacation.

Thanks for sending them,

regards,
Armin
>
> Patches were tested on an x86 product tree where full stack CVE
> analysis produces good results. Then I ported them to pure poky sumo
> and ran core-image-minimal build. Tried running "bitbake world" build
> which also succeeds. The results show following bitbake target
> recipes from poky with unpatched CVEs (ignored native, SDK and cross
> tools for now):
>
> build/tmp/deploy/cve$ grep -l "Unpatched" * | egrep -v -- "-native|nativesdk-|-cross" | sort
> apt
> aspell
> binutils
> bluez5
> busybox
> bzip2
> cairo
> cups
> curl
> db
> dropbear
> elfutils
> epiphany
> expat
> file
> gcc
> gcc-runtime
> gcc-sanitizers
> gcc-source-7.3.0
> ghostscript
> git
> glib-2.0
> glibc
> gnupg
> gnutls
> go
> gstreamer1.0
> libarchive
> libcomps
> libcroco
> libexif
> libgcc
> libgcrypt
> libid3tag
> libjpeg-turbo
> libpcap
> libpcre
> libpng
> librsvg
> libsndfile1
> libsolv
> libvorbis
> libx11
> libxkbcommon
> libxslt
> lighttpd
> lz4
> nasm
> ncurses
> openssh
> openssl
> pango
> patch
> pcmanfm
> perl
> python
> python3
> qemu
> shadow
> sqlite3
> sudo
> sysstat
> systemd
> tar
> tiff
> unzip
> webkitgtk
> wget
> wpa-supplicant
> xdg-utils
> xserver-xorg
> zip
>
> Sampling on the data shows that
>
>  * openssl 1.0.2p is missing patch for CVE-2019-1559
>  * openssh 7.6p1 is missing a lot more patches
>  * gcc is missing patches for CVE-2018-12886 on ARM
>    and CVE-2019-15847 on POWER9
>  * libpng is missing patch for CVE-2018-14048
>  * libjpeg-turbo is missing patch for CVE-2018-14498
>  * libgcrypt is missing patch for CVE-2018-6829
> etc.
>
> About CVE checking in yocto:
>
>  * enable with 'INHERIT += "cve-check"' in conf/local.conf
>  * see the resulting reports in tmp/deploy/cve/ directory for
>    all compiled recipes
>  * there is also an image specific summary but I saw it included
>    native and nativesdk recipe data too
>  * for applying CVE patches, white listing, setting product names
>    etc see the meta/classes/cve-check.bbclass and examples in this patchset
>    and in master branch
>  * note that only recompiled recipes will be analyzed for CVEs
>    so things from sstate cache will be ignored, a clean build without
>    cache may be needed when enabling the check
>
> ps. sumo still comes with gcc 7.3 and my patch to update to 7.4
> with lots of bug fixes has not been applied from
> http://lists.openembedded.org/pipermail/openembedded-core/2019-January/278049.html
> I've been using gcc 7.4 in several x86 and arm64 projects so I would also
> apply this update to any sumo tree out there.
>
> Cheers,
>
> -Mikko
>
> Anuj Mittal (2):
>   openssl: set CVE vendor to openssl
>   rsync: fix CVEs for included zlib
>
> Chen Qi (9):
>   flac: also add flac to CVE_PRODUCT
>   xserver-xorg: set CVE_PRODUCT
>   nasm: add CVE_PRODUCT
>   dropbear: set CVE_PRODUCT
>   libsdl: set CVE_PRODUCT
>   ghostscript: set CVE_PRODUCT
>   squashfs-tools: set CVE_PRODUCT
>   libxfont2: set CVE_PRODUCT
>   webkitgtk: set CVE_PRODUCT
>
> Chin Huat Ang (1):
>   cve-update-db-native: fix https proxy issues
>
> Mikko Rapeli (1):
>   cve-check.bbclass: initialize to_append
>
> Pierre Le Magourou (13):
>   cve-update-db: New recipe to update CVE database
>   cve-check: Remove dependency to cve-check-tool-native
>   cve-check: Manage CVE_PRODUCT with more than one name
>   cve-check: Consider CVE that affects versions with less than operator
>   cve-update-db: Use std library instead of urllib3
>   cve-update-db: Manage proxy if needed.
>   cve-update-db: do_populate_cve_db depends on do_fetch
>   cve-update-db: Catch request.urlopen errors.
>   cve-check: Depends on cve-update-db-native
>   cve-check: Update unpatched CVE matching
>   cve-check: Replace CVE_CHECK_CVE_WHITELIST by CVE_CHECK_WHITELIST
>   cve-update-db: Use NVD CPE data to populate PRODUCTS table
>   cve-update-db-native: Remove hash column from database.
>
> Ross Burton (21):
>   cve-check: be idiomatic
>   cve-check: remove redundant readline CVE whitelisting
>   cve-check-tool: remove
>   glibc: exclude child recipes from CVE scanning
>   cve-check: allow comparison of Vendor as well as Product
>   cve-update-db-native: use SQL placeholders instead of format strings
>   cve-update-db-native: use os.path.join instead of +
>   cve-update-db: actually inherit native
>   cve-update-db-native: use executemany() to optimise CPE insertion
>   cve-update-db-native: improve metadata parsing
>   cve-update-db-native: clean up JSON fetching
>   cve-check: ensure all known CVEs are in the report
>   cve-check: failure to parse versions should be more visible
>   flex: set CVE_PRODUCT to include vendor
>   libpam: set CVE_PRODUCT
>   procps: whitelist CVE-2018-1121
>   libpng: whitelist CVE-2019-17371
>   ed: set CVE vendor to avoid false positives
>   boost: set CVE vendor to Boost
>   subversion: set CVE vendor to Apache
>   git: set CVE vendor to git-scm
>
>  meta/classes/cve-check.bbclass                     | 147 ++++++++-----
>  meta/conf/distro/include/maintainers.inc           |   2 +
>  .../recipes-connectivity/openssl/openssl_1.0.2p.bb |   2 +
>  .../recipes-connectivity/openssl/openssl_1.1.0i.bb |   2 +
>  meta/recipes-core/dropbear/dropbear.inc            |   2 +
>  meta/recipes-core/glibc/glibc-locale.inc           |   3 +
>  meta/recipes-core/glibc/glibc-mtrace.inc           |   3 +
>  meta/recipes-core/glibc/glibc-scripts.inc          |   3 +
>  meta/recipes-core/meta/cve-update-db-native.bb     | 190 +++++++++++++++++
>  .../cve-check-tool/cve-check-tool_5.6.4.bb         |  62 ------
>  ...01-Fix-freeing-memory-allocated-by-sqlite.patch |  50 -----
>  ...ow-overriding-default-CA-certificate-file.patch | 215 -------------------
>  ...ogress-in-percent-when-downloading-CVE-db.patch | 135 ------------
>  ...are-computed-vs-expected-sha256-digit-str.patch |  52 -----
>  .../check-for-malloc_trim-before-using-it.patch    |  51 -----
>  meta/recipes-devtools/flex/flex_2.6.0.bb           |   3 +
>  meta/recipes-devtools/git/git.inc                  |   2 +
>  meta/recipes-devtools/nasm/nasm_2.13.03.bb         |   2 +
>  .../rsync/files/CVE-2016-9840.patch                |  75 +++++++
>  .../rsync/files/CVE-2016-9841.patch                | 228 +++++++++++++++++++++
>  .../rsync/files/CVE-2016-9842.patch                |  33 +++
>  .../rsync/files/CVE-2016-9843.patch                |  53 +++++
>  meta/recipes-devtools/rsync/rsync_3.1.3.bb         |   7 +-
>  .../squashfs-tools/squashfs-tools_git.bb           |   2 +
>  .../subversion/subversion_1.9.7.bb                 |   2 +
>  meta/recipes-extended/ed/ed_1.14.2.bb              |   2 +
>  .../ghostscript/ghostscript_9.21.bb                |   3 +
>  meta/recipes-extended/pam/libpam_1.3.0.bb          |   2 +
>  meta/recipes-extended/procps/procps_3.3.12.bb      |   3 +
>  meta/recipes-graphics/libsdl/libsdl_1.2.15.bb      |   2 +
>  meta/recipes-graphics/libsdl2/libsdl2_2.0.8.bb     |   2 +
>  meta/recipes-graphics/xorg-lib/libxfont2_2.0.3.bb  |   2 +
>  .../recipes-graphics/xorg-xserver/xserver-xorg.inc |   2 +
>  meta/recipes-multimedia/flac/flac_1.3.2.bb         |   2 +-
>  meta/recipes-multimedia/libpng/libpng_1.6.34.bb    |   3 +
>  meta/recipes-sato/webkit/webkitgtk_2.18.6.bb       |   2 +
>  meta/recipes-support/boost/boost.inc               |   2 +
>  37 files changed, 731 insertions(+), 622 deletions(-)
>  create mode 100644 meta/recipes-core/meta/cve-update-db-native.bb
>  delete mode 100644 meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb
>  delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-Fix-freeing-memory-allocated-by-sqlite.patch
>  delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-curl-allow-overriding-default-CA-certificate-file.patch
>  delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-print-progress-in-percent-when-downloading-CVE-db.patch
>  delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-update-Compare-computed-vs-expected-sha256-digit-str.patch
>  delete mode 100644 meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch
>  create mode 100644 meta/recipes-devtools/rsync/files/CVE-2016-9840.patch
>  create mode 100644 meta/recipes-devtools/rsync/files/CVE-2016-9841.patch
>  create mode 100644 meta/recipes-devtools/rsync/files/CVE-2016-9842.patch
>  create mode 100644 meta/recipes-devtools/rsync/files/CVE-2016-9843.patch
>