All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
@ 2022-03-29 13:07 Ross Burton
  2022-03-29 19:49 ` [OE-core] " Steve Sakoman
  2022-04-05 19:04 ` Jeroen Hofstee
  0 siblings, 2 replies; 38+ messages in thread
From: Ross Burton @ 2022-03-29 13:07 UTC (permalink / raw)
  To: openembedded-core

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 .../zlib/zlib/CVE-2018-25032.patch            | 347 ++++++++++++++++++
 meta/recipes-core/zlib/zlib_1.2.11.bb         |   1 +
 2 files changed, 348 insertions(+)
 create mode 100644 meta/recipes-core/zlib/zlib/CVE-2018-25032.patch

diff --git a/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
new file mode 100644
index 00000000000..5cb61836419
--- /dev/null
+++ b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
@@ -0,0 +1,347 @@
+CVE: CVE-2018-25032
+Upstream-Status: Backport
+Signed-off-by: Ross Burton <ross.burton@arm.com>
+
+From 5c44459c3b28a9bd3283aaceab7c615f8020c531 Mon Sep 17 00:00:00 2001
+From: Mark Adler <madler@alumni.caltech.edu>
+Date: Tue, 17 Apr 2018 22:09:22 -0700
+Subject: [PATCH] Fix a bug that can crash deflate on some input when using
+ Z_FIXED.
+
+This bug was reported by Danilo Ramos of Eideticom, Inc. It has
+lain in wait 13 years before being found! The bug was introduced
+in zlib 1.2.2.2, with the addition of the Z_FIXED option. That
+option forces the use of fixed Huffman codes. For rare inputs with
+a large number of distant matches, the pending buffer into which
+the compressed data is written can overwrite the distance symbol
+table which it overlays. That results in corrupted output due to
+invalid distances, and can result in out-of-bound accesses,
+crashing the application.
+
+The fix here combines the distance buffer and literal/length
+buffers into a single symbol buffer. Now three bytes of pending
+buffer space are opened up for each literal or length/distance
+pair consumed, instead of the previous two bytes. This assures
+that the pending buffer cannot overwrite the symbol table, since
+the maximum fixed code compressed length/distance is 31 bits, and
+since there are four bytes of pending space for every three bytes
+of symbol space.
+---
+ deflate.c | 74 ++++++++++++++++++++++++++++++++++++++++---------------
+ deflate.h | 25 +++++++++----------
+ trees.c   | 50 +++++++++++--------------------------
+ 3 files changed, 79 insertions(+), 70 deletions(-)
+
+diff --git a/deflate.c b/deflate.c
+index 425babc00..19cba873a 100644
+--- a/deflate.c
++++ b/deflate.c
+@@ -255,11 +255,6 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
+     int wrap = 1;
+     static const char my_version[] = ZLIB_VERSION;
+ 
+-    ushf *overlay;
+-    /* We overlay pending_buf and d_buf+l_buf. This works since the average
+-     * output size for (length,distance) codes is <= 24 bits.
+-     */
+-
+     if (version == Z_NULL || version[0] != my_version[0] ||
+         stream_size != sizeof(z_stream)) {
+         return Z_VERSION_ERROR;
+@@ -329,9 +324,47 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
+ 
+     s->lit_bufsize = 1 << (memLevel + 6); /* 16K elements by default */
+ 
+-    overlay = (ushf *) ZALLOC(strm, s->lit_bufsize, sizeof(ush)+2);
+-    s->pending_buf = (uchf *) overlay;
+-    s->pending_buf_size = (ulg)s->lit_bufsize * (sizeof(ush)+2L);
++    /* We overlay pending_buf and sym_buf. This works since the average size
++     * for length/distance pairs over any compressed block is assured to be 31
++     * bits or less.
++     *
++     * Analysis: The longest fixed codes are a length code of 8 bits plus 5
++     * extra bits, for lengths 131 to 257. The longest fixed distance codes are
++     * 5 bits plus 13 extra bits, for distances 16385 to 32768. The longest
++     * possible fixed-codes length/distance pair is then 31 bits total.
++     *
++     * sym_buf starts one-fourth of the way into pending_buf. So there are
++     * three bytes in sym_buf for every four bytes in pending_buf. Each symbol
++     * in sym_buf is three bytes -- two for the distance and one for the
++     * literal/length. As each symbol is consumed, the pointer to the next
++     * sym_buf value to read moves forward three bytes. From that symbol, up to
++     * 31 bits are written to pending_buf. The closest the written pending_buf
++     * bits gets to the next sym_buf symbol to read is just before the last
++     * code is written. At that time, 31*(n-2) bits have been written, just
++     * after 24*(n-2) bits have been consumed from sym_buf. sym_buf starts at
++     * 8*n bits into pending_buf. (Note that the symbol buffer fills when n-1
++     * symbols are written.) The closest the writing gets to what is unread is
++     * then n+14 bits. Here n is lit_bufsize, which is 16384 by default, and
++     * can range from 128 to 32768.
++     *
++     * Therefore, at a minimum, there are 142 bits of space between what is
++     * written and what is read in the overlain buffers, so the symbols cannot
++     * be overwritten by the compressed data. That space is actually 139 bits,
++     * due to the three-bit fixed-code block header.
++     *
++     * That covers the case where either Z_FIXED is specified, forcing fixed
++     * codes, or when the use of fixed codes is chosen, because that choice
++     * results in a smaller compressed block than dynamic codes. That latter
++     * condition then assures that the above analysis also covers all dynamic
++     * blocks. A dynamic-code block will only be chosen to be emitted if it has
++     * fewer bits than a fixed-code block would for the same set of symbols.
++     * Therefore its average symbol length is assured to be less than 31. So
++     * the compressed data for a dynamic block also cannot overwrite the
++     * symbols from which it is being constructed.
++     */
++
++    s->pending_buf = (uchf *) ZALLOC(strm, s->lit_bufsize, 4);
++    s->pending_buf_size = (ulg)s->lit_bufsize * 4;
+ 
+     if (s->window == Z_NULL || s->prev == Z_NULL || s->head == Z_NULL ||
+         s->pending_buf == Z_NULL) {
+@@ -340,8 +373,12 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
+         deflateEnd (strm);
+         return Z_MEM_ERROR;
+     }
+-    s->d_buf = overlay + s->lit_bufsize/sizeof(ush);
+-    s->l_buf = s->pending_buf + (1+sizeof(ush))*s->lit_bufsize;
++    s->sym_buf = s->pending_buf + s->lit_bufsize;
++    s->sym_end = (s->lit_bufsize - 1) * 3;
++    /* We avoid equality with lit_bufsize*3 because of wraparound at 64K
++     * on 16 bit machines and because stored blocks are restricted to
++     * 64K-1 bytes.
++     */
+ 
+     s->level = level;
+     s->strategy = strategy;
+@@ -552,7 +589,7 @@ int ZEXPORT deflatePrime (strm, bits, value)
+ 
+     if (deflateStateCheck(strm)) return Z_STREAM_ERROR;
+     s = strm->state;
+-    if ((Bytef *)(s->d_buf) < s->pending_out + ((Buf_size + 7) >> 3))
++    if (s->sym_buf < s->pending_out + ((Buf_size + 7) >> 3))
+         return Z_BUF_ERROR;
+     do {
+         put = Buf_size - s->bi_valid;
+@@ -1113,7 +1150,6 @@ int ZEXPORT deflateCopy (dest, source)
+ #else
+     deflate_state *ds;
+     deflate_state *ss;
+-    ushf *overlay;
+ 
+ 
+     if (deflateStateCheck(source) || dest == Z_NULL) {
+@@ -1133,8 +1169,7 @@ int ZEXPORT deflateCopy (dest, source)
+     ds->window = (Bytef *) ZALLOC(dest, ds->w_size, 2*sizeof(Byte));
+     ds->prev   = (Posf *)  ZALLOC(dest, ds->w_size, sizeof(Pos));
+     ds->head   = (Posf *)  ZALLOC(dest, ds->hash_size, sizeof(Pos));
+-    overlay = (ushf *) ZALLOC(dest, ds->lit_bufsize, sizeof(ush)+2);
+-    ds->pending_buf = (uchf *) overlay;
++    ds->pending_buf = (uchf *) ZALLOC(dest, ds->lit_bufsize, 4);
+ 
+     if (ds->window == Z_NULL || ds->prev == Z_NULL || ds->head == Z_NULL ||
+         ds->pending_buf == Z_NULL) {
+@@ -1148,8 +1183,7 @@ int ZEXPORT deflateCopy (dest, source)
+     zmemcpy(ds->pending_buf, ss->pending_buf, (uInt)ds->pending_buf_size);
+ 
+     ds->pending_out = ds->pending_buf + (ss->pending_out - ss->pending_buf);
+-    ds->d_buf = overlay + ds->lit_bufsize/sizeof(ush);
+-    ds->l_buf = ds->pending_buf + (1+sizeof(ush))*ds->lit_bufsize;
++    ds->sym_buf = ds->pending_buf + ds->lit_bufsize;
+ 
+     ds->l_desc.dyn_tree = ds->dyn_ltree;
+     ds->d_desc.dyn_tree = ds->dyn_dtree;
+@@ -1925,7 +1959,7 @@ local block_state deflate_fast(s, flush)
+         FLUSH_BLOCK(s, 1);
+         return finish_done;
+     }
+-    if (s->last_lit)
++    if (s->sym_next)
+         FLUSH_BLOCK(s, 0);
+     return block_done;
+ }
+@@ -2056,7 +2090,7 @@ local block_state deflate_slow(s, flush)
+         FLUSH_BLOCK(s, 1);
+         return finish_done;
+     }
+-    if (s->last_lit)
++    if (s->sym_next)
+         FLUSH_BLOCK(s, 0);
+     return block_done;
+ }
+@@ -2131,7 +2165,7 @@ local block_state deflate_rle(s, flush)
+         FLUSH_BLOCK(s, 1);
+         return finish_done;
+     }
+-    if (s->last_lit)
++    if (s->sym_next)
+         FLUSH_BLOCK(s, 0);
+     return block_done;
+ }
+@@ -2170,7 +2204,7 @@ local block_state deflate_huff(s, flush)
+         FLUSH_BLOCK(s, 1);
+         return finish_done;
+     }
+-    if (s->last_lit)
++    if (s->sym_next)
+         FLUSH_BLOCK(s, 0);
+     return block_done;
+ }
+diff --git a/deflate.h b/deflate.h
+index 23ecdd312..d4cf1a98b 100644
+--- a/deflate.h
++++ b/deflate.h
+@@ -217,7 +217,7 @@ typedef struct internal_state {
+     /* Depth of each subtree used as tie breaker for trees of equal frequency
+      */
+ 
+-    uchf *l_buf;          /* buffer for literals or lengths */
++    uchf *sym_buf;        /* buffer for distances and literals/lengths */
+ 
+     uInt  lit_bufsize;
+     /* Size of match buffer for literals/lengths.  There are 4 reasons for
+@@ -239,13 +239,8 @@ typedef struct internal_state {
+      *   - I can't count above 4
+      */
+ 
+-    uInt last_lit;      /* running index in l_buf */
+-
+-    ushf *d_buf;
+-    /* Buffer for distances. To simplify the code, d_buf and l_buf have
+-     * the same number of elements. To use different lengths, an extra flag
+-     * array would be necessary.
+-     */
++    uInt sym_next;      /* running index in sym_buf */
++    uInt sym_end;       /* symbol table full when sym_next reaches this */
+ 
+     ulg opt_len;        /* bit length of current block with optimal trees */
+     ulg static_len;     /* bit length of current block with static trees */
+@@ -325,20 +320,22 @@ void ZLIB_INTERNAL _tr_stored_block OF((deflate_state *s, charf *buf,
+ 
+ # define _tr_tally_lit(s, c, flush) \
+   { uch cc = (c); \
+-    s->d_buf[s->last_lit] = 0; \
+-    s->l_buf[s->last_lit++] = cc; \
++    s->sym_buf[s->sym_next++] = 0; \
++    s->sym_buf[s->sym_next++] = 0; \
++    s->sym_buf[s->sym_next++] = cc; \
+     s->dyn_ltree[cc].Freq++; \
+-    flush = (s->last_lit == s->lit_bufsize-1); \
++    flush = (s->sym_next == s->sym_end); \
+    }
+ # define _tr_tally_dist(s, distance, length, flush) \
+   { uch len = (uch)(length); \
+     ush dist = (ush)(distance); \
+-    s->d_buf[s->last_lit] = dist; \
+-    s->l_buf[s->last_lit++] = len; \
++    s->sym_buf[s->sym_next++] = dist; \
++    s->sym_buf[s->sym_next++] = dist >> 8; \
++    s->sym_buf[s->sym_next++] = len; \
+     dist--; \
+     s->dyn_ltree[_length_code[len]+LITERALS+1].Freq++; \
+     s->dyn_dtree[d_code(dist)].Freq++; \
+-    flush = (s->last_lit == s->lit_bufsize-1); \
++    flush = (s->sym_next == s->sym_end); \
+   }
+ #else
+ # define _tr_tally_lit(s, c, flush) flush = _tr_tally(s, 0, c)
+diff --git a/trees.c b/trees.c
+index 4f4a65011..decaeb7c3 100644
+--- a/trees.c
++++ b/trees.c
+@@ -416,7 +416,7 @@ local void init_block(s)
+ 
+     s->dyn_ltree[END_BLOCK].Freq = 1;
+     s->opt_len = s->static_len = 0L;
+-    s->last_lit = s->matches = 0;
++    s->sym_next = s->matches = 0;
+ }
+ 
+ #define SMALLEST 1
+@@ -948,7 +948,7 @@ void ZLIB_INTERNAL _tr_flush_block(s, buf, stored_len, last)
+ 
+         Tracev((stderr, "\nopt %lu(%lu) stat %lu(%lu) stored %lu lit %u ",
+                 opt_lenb, s->opt_len, static_lenb, s->static_len, stored_len,
+-                s->last_lit));
++                s->sym_next / 3));
+ 
+         if (static_lenb <= opt_lenb) opt_lenb = static_lenb;
+ 
+@@ -1017,8 +1017,9 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
+     unsigned dist;  /* distance of matched string */
+     unsigned lc;    /* match length-MIN_MATCH or unmatched char (if dist==0) */
+ {
+-    s->d_buf[s->last_lit] = (ush)dist;
+-    s->l_buf[s->last_lit++] = (uch)lc;
++    s->sym_buf[s->sym_next++] = dist;
++    s->sym_buf[s->sym_next++] = dist >> 8;
++    s->sym_buf[s->sym_next++] = lc;
+     if (dist == 0) {
+         /* lc is the unmatched char */
+         s->dyn_ltree[lc].Freq++;
+@@ -1033,30 +1034,7 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
+         s->dyn_ltree[_length_code[lc]+LITERALS+1].Freq++;
+         s->dyn_dtree[d_code(dist)].Freq++;
+     }
+-
+-#ifdef TRUNCATE_BLOCK
+-    /* Try to guess if it is profitable to stop the current block here */
+-    if ((s->last_lit & 0x1fff) == 0 && s->level > 2) {
+-        /* Compute an upper bound for the compressed length */
+-        ulg out_length = (ulg)s->last_lit*8L;
+-        ulg in_length = (ulg)((long)s->strstart - s->block_start);
+-        int dcode;
+-        for (dcode = 0; dcode < D_CODES; dcode++) {
+-            out_length += (ulg)s->dyn_dtree[dcode].Freq *
+-                (5L+extra_dbits[dcode]);
+-        }
+-        out_length >>= 3;
+-        Tracev((stderr,"\nlast_lit %u, in %ld, out ~%ld(%ld%%) ",
+-               s->last_lit, in_length, out_length,
+-               100L - out_length*100L/in_length));
+-        if (s->matches < s->last_lit/2 && out_length < in_length/2) return 1;
+-    }
+-#endif
+-    return (s->last_lit == s->lit_bufsize-1);
+-    /* We avoid equality with lit_bufsize because of wraparound at 64K
+-     * on 16 bit machines and because stored blocks are restricted to
+-     * 64K-1 bytes.
+-     */
++    return (s->sym_next == s->sym_end);
+ }
+ 
+ /* ===========================================================================
+@@ -1069,13 +1047,14 @@ local void compress_block(s, ltree, dtree)
+ {
+     unsigned dist;      /* distance of matched string */
+     int lc;             /* match length or unmatched char (if dist == 0) */
+-    unsigned lx = 0;    /* running index in l_buf */
++    unsigned sx = 0;    /* running index in sym_buf */
+     unsigned code;      /* the code to send */
+     int extra;          /* number of extra bits to send */
+ 
+-    if (s->last_lit != 0) do {
+-        dist = s->d_buf[lx];
+-        lc = s->l_buf[lx++];
++    if (s->sym_next != 0) do {
++        dist = s->sym_buf[sx++] & 0xff;
++        dist += (unsigned)(s->sym_buf[sx++] & 0xff) << 8;
++        lc = s->sym_buf[sx++];
+         if (dist == 0) {
+             send_code(s, lc, ltree); /* send a literal byte */
+             Tracecv(isgraph(lc), (stderr," '%c' ", lc));
+@@ -1100,11 +1079,10 @@ local void compress_block(s, ltree, dtree)
+             }
+         } /* literal or match pair ? */
+ 
+-        /* Check that the overlay between pending_buf and d_buf+l_buf is ok: */
+-        Assert((uInt)(s->pending) < s->lit_bufsize + 2*lx,
+-               "pendingBuf overflow");
++        /* Check that the overlay between pending_buf and sym_buf is ok: */
++        Assert(s->pending < s->lit_bufsize + sx, "pendingBuf overflow");
+ 
+-    } while (lx < s->last_lit);
++    } while (sx < s->sym_next);
+ 
+     send_code(s, END_BLOCK, ltree);
+ }
diff --git a/meta/recipes-core/zlib/zlib_1.2.11.bb b/meta/recipes-core/zlib/zlib_1.2.11.bb
index ef9431ae475..bc42cd64e9d 100644
--- a/meta/recipes-core/zlib/zlib_1.2.11.bb
+++ b/meta/recipes-core/zlib/zlib_1.2.11.bb
@@ -8,6 +8,7 @@ LIC_FILES_CHKSUM = "file://zlib.h;beginline=6;endline=23;md5=5377232268e952e9ef6
 
 SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/${BPN}/${PV}/${BPN}-${PV}.tar.xz \
            file://ldflags-tests.patch \
+           file://CVE-2018-25032.patch \
            file://run-ptest \
            "
 UPSTREAM_CHECK_URI = "http://zlib.net/"
-- 
2.25.1



^ permalink raw reply related	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-03-29 13:07 [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032 Ross Burton
@ 2022-03-29 19:49 ` Steve Sakoman
  2022-03-30 20:40   ` Ross Burton
  2022-04-05 19:04 ` Jeroen Hofstee
  1 sibling, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-03-29 19:49 UTC (permalink / raw)
  To: Ross Burton, Mittal, Anuj; +Cc: openembedded-core

On Tue, Mar 29, 2022 at 3:07 AM Ross Burton <ross@burtonini.com> wrote:
>
> Signed-off-by: Ross Burton <ross.burton@arm.com>
> ---
>  .../zlib/zlib/CVE-2018-25032.patch            | 347 ++++++++++++++++++

This breaks dunfell meta-intel:

ERROR: Applying patch 'CVE-2018-25032.patch' on target directory
'TOPDIR/tmp/work/corei7-64-poky-linux/zlib-intel/1.2.11.1.jtkv6.3-r0/git'
Command Error: 'quilt --quiltrc
TOPDIR/tmp/work/corei7-64-poky-linux/zlib-intel/1.2.11.1.jtkv6.3-r0/recipe-sysroot-native/etc/quiltrc
push' exited with 0  Output:
Applying patch CVE-2018-25032.patch
patching file deflate.c
Hunk #1 succeeded at 237 (offset -18 lines).
Hunk #2 succeeded at 319 (offset -5 lines).
Hunk #3 succeeded at 368 (offset -5 lines).
Hunk #4 succeeded at 590 (offset 1 line).
Hunk #5 succeeded at 1162 (offset 12 lines).
Hunk #6 succeeded at 1181 (offset 12 lines).
Hunk #7 succeeded at 1195 (offset 12 lines).
Hunk #8 succeeded at 1957 (offset -2 lines).
Hunk #9 succeeded at 1996 (offset -94 lines).
Hunk #10 FAILED at 2165.
Hunk #11 FAILED at 2204.
2 out of 11 hunks FAILED -- rejects in file deflate.c
patching file deflate.h
Hunk #1 succeeded at 230 (offset 13 lines).
Hunk #2 succeeded at 252 (offset 13 lines).
Hunk #3 succeeded at 339 (offset 19 lines).
patching file trees.c
Hunk #1 succeeded at 343 (offset -73 lines).
Hunk #2 succeeded at 875 (offset -73 lines).
Hunk #3 succeeded at 944 (offset -73 lines).
Hunk #4 succeeded at 961 (offset -73 lines).
Hunk #5 succeeded at 974 (offset -73 lines).
Hunk #6 succeeded at 1006 (offset -73 lines).
Patch CVE-2018-25032.patch does not apply (enforce with -f)

Steve

>  meta/recipes-core/zlib/zlib_1.2.11.bb         |   1 +
>  2 files changed, 348 insertions(+)
>  create mode 100644 meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
>
> diff --git a/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> new file mode 100644
> index 00000000000..5cb61836419
> --- /dev/null
> +++ b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> @@ -0,0 +1,347 @@
> +CVE: CVE-2018-25032
> +Upstream-Status: Backport
> +Signed-off-by: Ross Burton <ross.burton@arm.com>
> +
> +From 5c44459c3b28a9bd3283aaceab7c615f8020c531 Mon Sep 17 00:00:00 2001
> +From: Mark Adler <madler@alumni.caltech.edu>
> +Date: Tue, 17 Apr 2018 22:09:22 -0700
> +Subject: [PATCH] Fix a bug that can crash deflate on some input when using
> + Z_FIXED.
> +
> +This bug was reported by Danilo Ramos of Eideticom, Inc. It has
> +lain in wait 13 years before being found! The bug was introduced
> +in zlib 1.2.2.2, with the addition of the Z_FIXED option. That
> +option forces the use of fixed Huffman codes. For rare inputs with
> +a large number of distant matches, the pending buffer into which
> +the compressed data is written can overwrite the distance symbol
> +table which it overlays. That results in corrupted output due to
> +invalid distances, and can result in out-of-bound accesses,
> +crashing the application.
> +
> +The fix here combines the distance buffer and literal/length
> +buffers into a single symbol buffer. Now three bytes of pending
> +buffer space are opened up for each literal or length/distance
> +pair consumed, instead of the previous two bytes. This assures
> +that the pending buffer cannot overwrite the symbol table, since
> +the maximum fixed code compressed length/distance is 31 bits, and
> +since there are four bytes of pending space for every three bytes
> +of symbol space.
> +---
> + deflate.c | 74 ++++++++++++++++++++++++++++++++++++++++---------------
> + deflate.h | 25 +++++++++----------
> + trees.c   | 50 +++++++++++--------------------------
> + 3 files changed, 79 insertions(+), 70 deletions(-)
> +
> +diff --git a/deflate.c b/deflate.c
> +index 425babc00..19cba873a 100644
> +--- a/deflate.c
> ++++ b/deflate.c
> +@@ -255,11 +255,6 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
> +     int wrap = 1;
> +     static const char my_version[] = ZLIB_VERSION;
> +
> +-    ushf *overlay;
> +-    /* We overlay pending_buf and d_buf+l_buf. This works since the average
> +-     * output size for (length,distance) codes is <= 24 bits.
> +-     */
> +-
> +     if (version == Z_NULL || version[0] != my_version[0] ||
> +         stream_size != sizeof(z_stream)) {
> +         return Z_VERSION_ERROR;
> +@@ -329,9 +324,47 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
> +
> +     s->lit_bufsize = 1 << (memLevel + 6); /* 16K elements by default */
> +
> +-    overlay = (ushf *) ZALLOC(strm, s->lit_bufsize, sizeof(ush)+2);
> +-    s->pending_buf = (uchf *) overlay;
> +-    s->pending_buf_size = (ulg)s->lit_bufsize * (sizeof(ush)+2L);
> ++    /* We overlay pending_buf and sym_buf. This works since the average size
> ++     * for length/distance pairs over any compressed block is assured to be 31
> ++     * bits or less.
> ++     *
> ++     * Analysis: The longest fixed codes are a length code of 8 bits plus 5
> ++     * extra bits, for lengths 131 to 257. The longest fixed distance codes are
> ++     * 5 bits plus 13 extra bits, for distances 16385 to 32768. The longest
> ++     * possible fixed-codes length/distance pair is then 31 bits total.
> ++     *
> ++     * sym_buf starts one-fourth of the way into pending_buf. So there are
> ++     * three bytes in sym_buf for every four bytes in pending_buf. Each symbol
> ++     * in sym_buf is three bytes -- two for the distance and one for the
> ++     * literal/length. As each symbol is consumed, the pointer to the next
> ++     * sym_buf value to read moves forward three bytes. From that symbol, up to
> ++     * 31 bits are written to pending_buf. The closest the written pending_buf
> ++     * bits gets to the next sym_buf symbol to read is just before the last
> ++     * code is written. At that time, 31*(n-2) bits have been written, just
> ++     * after 24*(n-2) bits have been consumed from sym_buf. sym_buf starts at
> ++     * 8*n bits into pending_buf. (Note that the symbol buffer fills when n-1
> ++     * symbols are written.) The closest the writing gets to what is unread is
> ++     * then n+14 bits. Here n is lit_bufsize, which is 16384 by default, and
> ++     * can range from 128 to 32768.
> ++     *
> ++     * Therefore, at a minimum, there are 142 bits of space between what is
> ++     * written and what is read in the overlain buffers, so the symbols cannot
> ++     * be overwritten by the compressed data. That space is actually 139 bits,
> ++     * due to the three-bit fixed-code block header.
> ++     *
> ++     * That covers the case where either Z_FIXED is specified, forcing fixed
> ++     * codes, or when the use of fixed codes is chosen, because that choice
> ++     * results in a smaller compressed block than dynamic codes. That latter
> ++     * condition then assures that the above analysis also covers all dynamic
> ++     * blocks. A dynamic-code block will only be chosen to be emitted if it has
> ++     * fewer bits than a fixed-code block would for the same set of symbols.
> ++     * Therefore its average symbol length is assured to be less than 31. So
> ++     * the compressed data for a dynamic block also cannot overwrite the
> ++     * symbols from which it is being constructed.
> ++     */
> ++
> ++    s->pending_buf = (uchf *) ZALLOC(strm, s->lit_bufsize, 4);
> ++    s->pending_buf_size = (ulg)s->lit_bufsize * 4;
> +
> +     if (s->window == Z_NULL || s->prev == Z_NULL || s->head == Z_NULL ||
> +         s->pending_buf == Z_NULL) {
> +@@ -340,8 +373,12 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
> +         deflateEnd (strm);
> +         return Z_MEM_ERROR;
> +     }
> +-    s->d_buf = overlay + s->lit_bufsize/sizeof(ush);
> +-    s->l_buf = s->pending_buf + (1+sizeof(ush))*s->lit_bufsize;
> ++    s->sym_buf = s->pending_buf + s->lit_bufsize;
> ++    s->sym_end = (s->lit_bufsize - 1) * 3;
> ++    /* We avoid equality with lit_bufsize*3 because of wraparound at 64K
> ++     * on 16 bit machines and because stored blocks are restricted to
> ++     * 64K-1 bytes.
> ++     */
> +
> +     s->level = level;
> +     s->strategy = strategy;
> +@@ -552,7 +589,7 @@ int ZEXPORT deflatePrime (strm, bits, value)
> +
> +     if (deflateStateCheck(strm)) return Z_STREAM_ERROR;
> +     s = strm->state;
> +-    if ((Bytef *)(s->d_buf) < s->pending_out + ((Buf_size + 7) >> 3))
> ++    if (s->sym_buf < s->pending_out + ((Buf_size + 7) >> 3))
> +         return Z_BUF_ERROR;
> +     do {
> +         put = Buf_size - s->bi_valid;
> +@@ -1113,7 +1150,6 @@ int ZEXPORT deflateCopy (dest, source)
> + #else
> +     deflate_state *ds;
> +     deflate_state *ss;
> +-    ushf *overlay;
> +
> +
> +     if (deflateStateCheck(source) || dest == Z_NULL) {
> +@@ -1133,8 +1169,7 @@ int ZEXPORT deflateCopy (dest, source)
> +     ds->window = (Bytef *) ZALLOC(dest, ds->w_size, 2*sizeof(Byte));
> +     ds->prev   = (Posf *)  ZALLOC(dest, ds->w_size, sizeof(Pos));
> +     ds->head   = (Posf *)  ZALLOC(dest, ds->hash_size, sizeof(Pos));
> +-    overlay = (ushf *) ZALLOC(dest, ds->lit_bufsize, sizeof(ush)+2);
> +-    ds->pending_buf = (uchf *) overlay;
> ++    ds->pending_buf = (uchf *) ZALLOC(dest, ds->lit_bufsize, 4);
> +
> +     if (ds->window == Z_NULL || ds->prev == Z_NULL || ds->head == Z_NULL ||
> +         ds->pending_buf == Z_NULL) {
> +@@ -1148,8 +1183,7 @@ int ZEXPORT deflateCopy (dest, source)
> +     zmemcpy(ds->pending_buf, ss->pending_buf, (uInt)ds->pending_buf_size);
> +
> +     ds->pending_out = ds->pending_buf + (ss->pending_out - ss->pending_buf);
> +-    ds->d_buf = overlay + ds->lit_bufsize/sizeof(ush);
> +-    ds->l_buf = ds->pending_buf + (1+sizeof(ush))*ds->lit_bufsize;
> ++    ds->sym_buf = ds->pending_buf + ds->lit_bufsize;
> +
> +     ds->l_desc.dyn_tree = ds->dyn_ltree;
> +     ds->d_desc.dyn_tree = ds->dyn_dtree;
> +@@ -1925,7 +1959,7 @@ local block_state deflate_fast(s, flush)
> +         FLUSH_BLOCK(s, 1);
> +         return finish_done;
> +     }
> +-    if (s->last_lit)
> ++    if (s->sym_next)
> +         FLUSH_BLOCK(s, 0);
> +     return block_done;
> + }
> +@@ -2056,7 +2090,7 @@ local block_state deflate_slow(s, flush)
> +         FLUSH_BLOCK(s, 1);
> +         return finish_done;
> +     }
> +-    if (s->last_lit)
> ++    if (s->sym_next)
> +         FLUSH_BLOCK(s, 0);
> +     return block_done;
> + }
> +@@ -2131,7 +2165,7 @@ local block_state deflate_rle(s, flush)
> +         FLUSH_BLOCK(s, 1);
> +         return finish_done;
> +     }
> +-    if (s->last_lit)
> ++    if (s->sym_next)
> +         FLUSH_BLOCK(s, 0);
> +     return block_done;
> + }
> +@@ -2170,7 +2204,7 @@ local block_state deflate_huff(s, flush)
> +         FLUSH_BLOCK(s, 1);
> +         return finish_done;
> +     }
> +-    if (s->last_lit)
> ++    if (s->sym_next)
> +         FLUSH_BLOCK(s, 0);
> +     return block_done;
> + }
> +diff --git a/deflate.h b/deflate.h
> +index 23ecdd312..d4cf1a98b 100644
> +--- a/deflate.h
> ++++ b/deflate.h
> +@@ -217,7 +217,7 @@ typedef struct internal_state {
> +     /* Depth of each subtree used as tie breaker for trees of equal frequency
> +      */
> +
> +-    uchf *l_buf;          /* buffer for literals or lengths */
> ++    uchf *sym_buf;        /* buffer for distances and literals/lengths */
> +
> +     uInt  lit_bufsize;
> +     /* Size of match buffer for literals/lengths.  There are 4 reasons for
> +@@ -239,13 +239,8 @@ typedef struct internal_state {
> +      *   - I can't count above 4
> +      */
> +
> +-    uInt last_lit;      /* running index in l_buf */
> +-
> +-    ushf *d_buf;
> +-    /* Buffer for distances. To simplify the code, d_buf and l_buf have
> +-     * the same number of elements. To use different lengths, an extra flag
> +-     * array would be necessary.
> +-     */
> ++    uInt sym_next;      /* running index in sym_buf */
> ++    uInt sym_end;       /* symbol table full when sym_next reaches this */
> +
> +     ulg opt_len;        /* bit length of current block with optimal trees */
> +     ulg static_len;     /* bit length of current block with static trees */
> +@@ -325,20 +320,22 @@ void ZLIB_INTERNAL _tr_stored_block OF((deflate_state *s, charf *buf,
> +
> + # define _tr_tally_lit(s, c, flush) \
> +   { uch cc = (c); \
> +-    s->d_buf[s->last_lit] = 0; \
> +-    s->l_buf[s->last_lit++] = cc; \
> ++    s->sym_buf[s->sym_next++] = 0; \
> ++    s->sym_buf[s->sym_next++] = 0; \
> ++    s->sym_buf[s->sym_next++] = cc; \
> +     s->dyn_ltree[cc].Freq++; \
> +-    flush = (s->last_lit == s->lit_bufsize-1); \
> ++    flush = (s->sym_next == s->sym_end); \
> +    }
> + # define _tr_tally_dist(s, distance, length, flush) \
> +   { uch len = (uch)(length); \
> +     ush dist = (ush)(distance); \
> +-    s->d_buf[s->last_lit] = dist; \
> +-    s->l_buf[s->last_lit++] = len; \
> ++    s->sym_buf[s->sym_next++] = dist; \
> ++    s->sym_buf[s->sym_next++] = dist >> 8; \
> ++    s->sym_buf[s->sym_next++] = len; \
> +     dist--; \
> +     s->dyn_ltree[_length_code[len]+LITERALS+1].Freq++; \
> +     s->dyn_dtree[d_code(dist)].Freq++; \
> +-    flush = (s->last_lit == s->lit_bufsize-1); \
> ++    flush = (s->sym_next == s->sym_end); \
> +   }
> + #else
> + # define _tr_tally_lit(s, c, flush) flush = _tr_tally(s, 0, c)
> +diff --git a/trees.c b/trees.c
> +index 4f4a65011..decaeb7c3 100644
> +--- a/trees.c
> ++++ b/trees.c
> +@@ -416,7 +416,7 @@ local void init_block(s)
> +
> +     s->dyn_ltree[END_BLOCK].Freq = 1;
> +     s->opt_len = s->static_len = 0L;
> +-    s->last_lit = s->matches = 0;
> ++    s->sym_next = s->matches = 0;
> + }
> +
> + #define SMALLEST 1
> +@@ -948,7 +948,7 @@ void ZLIB_INTERNAL _tr_flush_block(s, buf, stored_len, last)
> +
> +         Tracev((stderr, "\nopt %lu(%lu) stat %lu(%lu) stored %lu lit %u ",
> +                 opt_lenb, s->opt_len, static_lenb, s->static_len, stored_len,
> +-                s->last_lit));
> ++                s->sym_next / 3));
> +
> +         if (static_lenb <= opt_lenb) opt_lenb = static_lenb;
> +
> +@@ -1017,8 +1017,9 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
> +     unsigned dist;  /* distance of matched string */
> +     unsigned lc;    /* match length-MIN_MATCH or unmatched char (if dist==0) */
> + {
> +-    s->d_buf[s->last_lit] = (ush)dist;
> +-    s->l_buf[s->last_lit++] = (uch)lc;
> ++    s->sym_buf[s->sym_next++] = dist;
> ++    s->sym_buf[s->sym_next++] = dist >> 8;
> ++    s->sym_buf[s->sym_next++] = lc;
> +     if (dist == 0) {
> +         /* lc is the unmatched char */
> +         s->dyn_ltree[lc].Freq++;
> +@@ -1033,30 +1034,7 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
> +         s->dyn_ltree[_length_code[lc]+LITERALS+1].Freq++;
> +         s->dyn_dtree[d_code(dist)].Freq++;
> +     }
> +-
> +-#ifdef TRUNCATE_BLOCK
> +-    /* Try to guess if it is profitable to stop the current block here */
> +-    if ((s->last_lit & 0x1fff) == 0 && s->level > 2) {
> +-        /* Compute an upper bound for the compressed length */
> +-        ulg out_length = (ulg)s->last_lit*8L;
> +-        ulg in_length = (ulg)((long)s->strstart - s->block_start);
> +-        int dcode;
> +-        for (dcode = 0; dcode < D_CODES; dcode++) {
> +-            out_length += (ulg)s->dyn_dtree[dcode].Freq *
> +-                (5L+extra_dbits[dcode]);
> +-        }
> +-        out_length >>= 3;
> +-        Tracev((stderr,"\nlast_lit %u, in %ld, out ~%ld(%ld%%) ",
> +-               s->last_lit, in_length, out_length,
> +-               100L - out_length*100L/in_length));
> +-        if (s->matches < s->last_lit/2 && out_length < in_length/2) return 1;
> +-    }
> +-#endif
> +-    return (s->last_lit == s->lit_bufsize-1);
> +-    /* We avoid equality with lit_bufsize because of wraparound at 64K
> +-     * on 16 bit machines and because stored blocks are restricted to
> +-     * 64K-1 bytes.
> +-     */
> ++    return (s->sym_next == s->sym_end);
> + }
> +
> + /* ===========================================================================
> +@@ -1069,13 +1047,14 @@ local void compress_block(s, ltree, dtree)
> + {
> +     unsigned dist;      /* distance of matched string */
> +     int lc;             /* match length or unmatched char (if dist == 0) */
> +-    unsigned lx = 0;    /* running index in l_buf */
> ++    unsigned sx = 0;    /* running index in sym_buf */
> +     unsigned code;      /* the code to send */
> +     int extra;          /* number of extra bits to send */
> +
> +-    if (s->last_lit != 0) do {
> +-        dist = s->d_buf[lx];
> +-        lc = s->l_buf[lx++];
> ++    if (s->sym_next != 0) do {
> ++        dist = s->sym_buf[sx++] & 0xff;
> ++        dist += (unsigned)(s->sym_buf[sx++] & 0xff) << 8;
> ++        lc = s->sym_buf[sx++];
> +         if (dist == 0) {
> +             send_code(s, lc, ltree); /* send a literal byte */
> +             Tracecv(isgraph(lc), (stderr," '%c' ", lc));
> +@@ -1100,11 +1079,10 @@ local void compress_block(s, ltree, dtree)
> +             }
> +         } /* literal or match pair ? */
> +
> +-        /* Check that the overlay between pending_buf and d_buf+l_buf is ok: */
> +-        Assert((uInt)(s->pending) < s->lit_bufsize + 2*lx,
> +-               "pendingBuf overflow");
> ++        /* Check that the overlay between pending_buf and sym_buf is ok: */
> ++        Assert(s->pending < s->lit_bufsize + sx, "pendingBuf overflow");
> +
> +-    } while (lx < s->last_lit);
> ++    } while (sx < s->sym_next);
> +
> +     send_code(s, END_BLOCK, ltree);
> + }
> diff --git a/meta/recipes-core/zlib/zlib_1.2.11.bb b/meta/recipes-core/zlib/zlib_1.2.11.bb
> index ef9431ae475..bc42cd64e9d 100644
> --- a/meta/recipes-core/zlib/zlib_1.2.11.bb
> +++ b/meta/recipes-core/zlib/zlib_1.2.11.bb
> @@ -8,6 +8,7 @@ LIC_FILES_CHKSUM = "file://zlib.h;beginline=6;endline=23;md5=5377232268e952e9ef6
>
>  SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/${BPN}/${PV}/${BPN}-${PV}.tar.xz \
>             file://ldflags-tests.patch \
> +           file://CVE-2018-25032.patch \
>             file://run-ptest \
>             "
>  UPSTREAM_CHECK_URI = "http://zlib.net/"
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#163750): https://lists.openembedded.org/g/openembedded-core/message/163750
> Mute This Topic: https://lists.openembedded.org/mt/90107518/3620601
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-03-29 19:49 ` [OE-core] " Steve Sakoman
@ 2022-03-30 20:40   ` Ross Burton
  2022-03-30 22:57     ` Steve Sakoman
  2022-03-31  0:24     ` Mittal, Anuj
  0 siblings, 2 replies; 38+ messages in thread
From: Ross Burton @ 2022-03-30 20:40 UTC (permalink / raw)
  To: Steve Sakoman; +Cc: Mittal, Anuj, openembedded-core

Hm, turns out I was being too clever back in 2019.

Anuj: the quick fix is to not use :prepend/:remove to manipulate
SRC_URI, but just override it entirely.  You won't get the CVE but
that will be flagged in scans and you (or even better,the fork
maintainer) can rebase the CVE patch.

As this is a fairly important CVE I'm tempted to say we merge to
oe-core dunfell now and let meta-intel catch up...

Ross

On Tue, 29 Mar 2022 at 20:49, Steve Sakoman <steve@sakoman.com> wrote:
>
> On Tue, Mar 29, 2022 at 3:07 AM Ross Burton <ross@burtonini.com> wrote:
> >
> > Signed-off-by: Ross Burton <ross.burton@arm.com>
> > ---
> >  .../zlib/zlib/CVE-2018-25032.patch            | 347 ++++++++++++++++++
>
> This breaks dunfell meta-intel:
>
> ERROR: Applying patch 'CVE-2018-25032.patch' on target directory
> 'TOPDIR/tmp/work/corei7-64-poky-linux/zlib-intel/1.2.11.1.jtkv6.3-r0/git'
> Command Error: 'quilt --quiltrc
> TOPDIR/tmp/work/corei7-64-poky-linux/zlib-intel/1.2.11.1.jtkv6.3-r0/recipe-sysroot-native/etc/quiltrc
> push' exited with 0  Output:
> Applying patch CVE-2018-25032.patch
> patching file deflate.c
> Hunk #1 succeeded at 237 (offset -18 lines).
> Hunk #2 succeeded at 319 (offset -5 lines).
> Hunk #3 succeeded at 368 (offset -5 lines).
> Hunk #4 succeeded at 590 (offset 1 line).
> Hunk #5 succeeded at 1162 (offset 12 lines).
> Hunk #6 succeeded at 1181 (offset 12 lines).
> Hunk #7 succeeded at 1195 (offset 12 lines).
> Hunk #8 succeeded at 1957 (offset -2 lines).
> Hunk #9 succeeded at 1996 (offset -94 lines).
> Hunk #10 FAILED at 2165.
> Hunk #11 FAILED at 2204.
> 2 out of 11 hunks FAILED -- rejects in file deflate.c
> patching file deflate.h
> Hunk #1 succeeded at 230 (offset 13 lines).
> Hunk #2 succeeded at 252 (offset 13 lines).
> Hunk #3 succeeded at 339 (offset 19 lines).
> patching file trees.c
> Hunk #1 succeeded at 343 (offset -73 lines).
> Hunk #2 succeeded at 875 (offset -73 lines).
> Hunk #3 succeeded at 944 (offset -73 lines).
> Hunk #4 succeeded at 961 (offset -73 lines).
> Hunk #5 succeeded at 974 (offset -73 lines).
> Hunk #6 succeeded at 1006 (offset -73 lines).
> Patch CVE-2018-25032.patch does not apply (enforce with -f)
>
> Steve
>
> >  meta/recipes-core/zlib/zlib_1.2.11.bb         |   1 +
> >  2 files changed, 348 insertions(+)
> >  create mode 100644 meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> >
> > diff --git a/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > new file mode 100644
> > index 00000000000..5cb61836419
> > --- /dev/null
> > +++ b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > @@ -0,0 +1,347 @@
> > +CVE: CVE-2018-25032
> > +Upstream-Status: Backport
> > +Signed-off-by: Ross Burton <ross.burton@arm.com>
> > +
> > +From 5c44459c3b28a9bd3283aaceab7c615f8020c531 Mon Sep 17 00:00:00 2001
> > +From: Mark Adler <madler@alumni.caltech.edu>
> > +Date: Tue, 17 Apr 2018 22:09:22 -0700
> > +Subject: [PATCH] Fix a bug that can crash deflate on some input when using
> > + Z_FIXED.
> > +
> > +This bug was reported by Danilo Ramos of Eideticom, Inc. It has
> > +lain in wait 13 years before being found! The bug was introduced
> > +in zlib 1.2.2.2, with the addition of the Z_FIXED option. That
> > +option forces the use of fixed Huffman codes. For rare inputs with
> > +a large number of distant matches, the pending buffer into which
> > +the compressed data is written can overwrite the distance symbol
> > +table which it overlays. That results in corrupted output due to
> > +invalid distances, and can result in out-of-bound accesses,
> > +crashing the application.
> > +
> > +The fix here combines the distance buffer and literal/length
> > +buffers into a single symbol buffer. Now three bytes of pending
> > +buffer space are opened up for each literal or length/distance
> > +pair consumed, instead of the previous two bytes. This assures
> > +that the pending buffer cannot overwrite the symbol table, since
> > +the maximum fixed code compressed length/distance is 31 bits, and
> > +since there are four bytes of pending space for every three bytes
> > +of symbol space.
> > +---
> > + deflate.c | 74 ++++++++++++++++++++++++++++++++++++++++---------------
> > + deflate.h | 25 +++++++++----------
> > + trees.c   | 50 +++++++++++--------------------------
> > + 3 files changed, 79 insertions(+), 70 deletions(-)
> > +
> > +diff --git a/deflate.c b/deflate.c
> > +index 425babc00..19cba873a 100644
> > +--- a/deflate.c
> > ++++ b/deflate.c
> > +@@ -255,11 +255,6 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
> > +     int wrap = 1;
> > +     static const char my_version[] = ZLIB_VERSION;
> > +
> > +-    ushf *overlay;
> > +-    /* We overlay pending_buf and d_buf+l_buf. This works since the average
> > +-     * output size for (length,distance) codes is <= 24 bits.
> > +-     */
> > +-
> > +     if (version == Z_NULL || version[0] != my_version[0] ||
> > +         stream_size != sizeof(z_stream)) {
> > +         return Z_VERSION_ERROR;
> > +@@ -329,9 +324,47 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
> > +
> > +     s->lit_bufsize = 1 << (memLevel + 6); /* 16K elements by default */
> > +
> > +-    overlay = (ushf *) ZALLOC(strm, s->lit_bufsize, sizeof(ush)+2);
> > +-    s->pending_buf = (uchf *) overlay;
> > +-    s->pending_buf_size = (ulg)s->lit_bufsize * (sizeof(ush)+2L);
> > ++    /* We overlay pending_buf and sym_buf. This works since the average size
> > ++     * for length/distance pairs over any compressed block is assured to be 31
> > ++     * bits or less.
> > ++     *
> > ++     * Analysis: The longest fixed codes are a length code of 8 bits plus 5
> > ++     * extra bits, for lengths 131 to 257. The longest fixed distance codes are
> > ++     * 5 bits plus 13 extra bits, for distances 16385 to 32768. The longest
> > ++     * possible fixed-codes length/distance pair is then 31 bits total.
> > ++     *
> > ++     * sym_buf starts one-fourth of the way into pending_buf. So there are
> > ++     * three bytes in sym_buf for every four bytes in pending_buf. Each symbol
> > ++     * in sym_buf is three bytes -- two for the distance and one for the
> > ++     * literal/length. As each symbol is consumed, the pointer to the next
> > ++     * sym_buf value to read moves forward three bytes. From that symbol, up to
> > ++     * 31 bits are written to pending_buf. The closest the written pending_buf
> > ++     * bits gets to the next sym_buf symbol to read is just before the last
> > ++     * code is written. At that time, 31*(n-2) bits have been written, just
> > ++     * after 24*(n-2) bits have been consumed from sym_buf. sym_buf starts at
> > ++     * 8*n bits into pending_buf. (Note that the symbol buffer fills when n-1
> > ++     * symbols are written.) The closest the writing gets to what is unread is
> > ++     * then n+14 bits. Here n is lit_bufsize, which is 16384 by default, and
> > ++     * can range from 128 to 32768.
> > ++     *
> > ++     * Therefore, at a minimum, there are 142 bits of space between what is
> > ++     * written and what is read in the overlain buffers, so the symbols cannot
> > ++     * be overwritten by the compressed data. That space is actually 139 bits,
> > ++     * due to the three-bit fixed-code block header.
> > ++     *
> > ++     * That covers the case where either Z_FIXED is specified, forcing fixed
> > ++     * codes, or when the use of fixed codes is chosen, because that choice
> > ++     * results in a smaller compressed block than dynamic codes. That latter
> > ++     * condition then assures that the above analysis also covers all dynamic
> > ++     * blocks. A dynamic-code block will only be chosen to be emitted if it has
> > ++     * fewer bits than a fixed-code block would for the same set of symbols.
> > ++     * Therefore its average symbol length is assured to be less than 31. So
> > ++     * the compressed data for a dynamic block also cannot overwrite the
> > ++     * symbols from which it is being constructed.
> > ++     */
> > ++
> > ++    s->pending_buf = (uchf *) ZALLOC(strm, s->lit_bufsize, 4);
> > ++    s->pending_buf_size = (ulg)s->lit_bufsize * 4;
> > +
> > +     if (s->window == Z_NULL || s->prev == Z_NULL || s->head == Z_NULL ||
> > +         s->pending_buf == Z_NULL) {
> > +@@ -340,8 +373,12 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
> > +         deflateEnd (strm);
> > +         return Z_MEM_ERROR;
> > +     }
> > +-    s->d_buf = overlay + s->lit_bufsize/sizeof(ush);
> > +-    s->l_buf = s->pending_buf + (1+sizeof(ush))*s->lit_bufsize;
> > ++    s->sym_buf = s->pending_buf + s->lit_bufsize;
> > ++    s->sym_end = (s->lit_bufsize - 1) * 3;
> > ++    /* We avoid equality with lit_bufsize*3 because of wraparound at 64K
> > ++     * on 16 bit machines and because stored blocks are restricted to
> > ++     * 64K-1 bytes.
> > ++     */
> > +
> > +     s->level = level;
> > +     s->strategy = strategy;
> > +@@ -552,7 +589,7 @@ int ZEXPORT deflatePrime (strm, bits, value)
> > +
> > +     if (deflateStateCheck(strm)) return Z_STREAM_ERROR;
> > +     s = strm->state;
> > +-    if ((Bytef *)(s->d_buf) < s->pending_out + ((Buf_size + 7) >> 3))
> > ++    if (s->sym_buf < s->pending_out + ((Buf_size + 7) >> 3))
> > +         return Z_BUF_ERROR;
> > +     do {
> > +         put = Buf_size - s->bi_valid;
> > +@@ -1113,7 +1150,6 @@ int ZEXPORT deflateCopy (dest, source)
> > + #else
> > +     deflate_state *ds;
> > +     deflate_state *ss;
> > +-    ushf *overlay;
> > +
> > +
> > +     if (deflateStateCheck(source) || dest == Z_NULL) {
> > +@@ -1133,8 +1169,7 @@ int ZEXPORT deflateCopy (dest, source)
> > +     ds->window = (Bytef *) ZALLOC(dest, ds->w_size, 2*sizeof(Byte));
> > +     ds->prev   = (Posf *)  ZALLOC(dest, ds->w_size, sizeof(Pos));
> > +     ds->head   = (Posf *)  ZALLOC(dest, ds->hash_size, sizeof(Pos));
> > +-    overlay = (ushf *) ZALLOC(dest, ds->lit_bufsize, sizeof(ush)+2);
> > +-    ds->pending_buf = (uchf *) overlay;
> > ++    ds->pending_buf = (uchf *) ZALLOC(dest, ds->lit_bufsize, 4);
> > +
> > +     if (ds->window == Z_NULL || ds->prev == Z_NULL || ds->head == Z_NULL ||
> > +         ds->pending_buf == Z_NULL) {
> > +@@ -1148,8 +1183,7 @@ int ZEXPORT deflateCopy (dest, source)
> > +     zmemcpy(ds->pending_buf, ss->pending_buf, (uInt)ds->pending_buf_size);
> > +
> > +     ds->pending_out = ds->pending_buf + (ss->pending_out - ss->pending_buf);
> > +-    ds->d_buf = overlay + ds->lit_bufsize/sizeof(ush);
> > +-    ds->l_buf = ds->pending_buf + (1+sizeof(ush))*ds->lit_bufsize;
> > ++    ds->sym_buf = ds->pending_buf + ds->lit_bufsize;
> > +
> > +     ds->l_desc.dyn_tree = ds->dyn_ltree;
> > +     ds->d_desc.dyn_tree = ds->dyn_dtree;
> > +@@ -1925,7 +1959,7 @@ local block_state deflate_fast(s, flush)
> > +         FLUSH_BLOCK(s, 1);
> > +         return finish_done;
> > +     }
> > +-    if (s->last_lit)
> > ++    if (s->sym_next)
> > +         FLUSH_BLOCK(s, 0);
> > +     return block_done;
> > + }
> > +@@ -2056,7 +2090,7 @@ local block_state deflate_slow(s, flush)
> > +         FLUSH_BLOCK(s, 1);
> > +         return finish_done;
> > +     }
> > +-    if (s->last_lit)
> > ++    if (s->sym_next)
> > +         FLUSH_BLOCK(s, 0);
> > +     return block_done;
> > + }
> > +@@ -2131,7 +2165,7 @@ local block_state deflate_rle(s, flush)
> > +         FLUSH_BLOCK(s, 1);
> > +         return finish_done;
> > +     }
> > +-    if (s->last_lit)
> > ++    if (s->sym_next)
> > +         FLUSH_BLOCK(s, 0);
> > +     return block_done;
> > + }
> > +@@ -2170,7 +2204,7 @@ local block_state deflate_huff(s, flush)
> > +         FLUSH_BLOCK(s, 1);
> > +         return finish_done;
> > +     }
> > +-    if (s->last_lit)
> > ++    if (s->sym_next)
> > +         FLUSH_BLOCK(s, 0);
> > +     return block_done;
> > + }
> > +diff --git a/deflate.h b/deflate.h
> > +index 23ecdd312..d4cf1a98b 100644
> > +--- a/deflate.h
> > ++++ b/deflate.h
> > +@@ -217,7 +217,7 @@ typedef struct internal_state {
> > +     /* Depth of each subtree used as tie breaker for trees of equal frequency
> > +      */
> > +
> > +-    uchf *l_buf;          /* buffer for literals or lengths */
> > ++    uchf *sym_buf;        /* buffer for distances and literals/lengths */
> > +
> > +     uInt  lit_bufsize;
> > +     /* Size of match buffer for literals/lengths.  There are 4 reasons for
> > +@@ -239,13 +239,8 @@ typedef struct internal_state {
> > +      *   - I can't count above 4
> > +      */
> > +
> > +-    uInt last_lit;      /* running index in l_buf */
> > +-
> > +-    ushf *d_buf;
> > +-    /* Buffer for distances. To simplify the code, d_buf and l_buf have
> > +-     * the same number of elements. To use different lengths, an extra flag
> > +-     * array would be necessary.
> > +-     */
> > ++    uInt sym_next;      /* running index in sym_buf */
> > ++    uInt sym_end;       /* symbol table full when sym_next reaches this */
> > +
> > +     ulg opt_len;        /* bit length of current block with optimal trees */
> > +     ulg static_len;     /* bit length of current block with static trees */
> > +@@ -325,20 +320,22 @@ void ZLIB_INTERNAL _tr_stored_block OF((deflate_state *s, charf *buf,
> > +
> > + # define _tr_tally_lit(s, c, flush) \
> > +   { uch cc = (c); \
> > +-    s->d_buf[s->last_lit] = 0; \
> > +-    s->l_buf[s->last_lit++] = cc; \
> > ++    s->sym_buf[s->sym_next++] = 0; \
> > ++    s->sym_buf[s->sym_next++] = 0; \
> > ++    s->sym_buf[s->sym_next++] = cc; \
> > +     s->dyn_ltree[cc].Freq++; \
> > +-    flush = (s->last_lit == s->lit_bufsize-1); \
> > ++    flush = (s->sym_next == s->sym_end); \
> > +    }
> > + # define _tr_tally_dist(s, distance, length, flush) \
> > +   { uch len = (uch)(length); \
> > +     ush dist = (ush)(distance); \
> > +-    s->d_buf[s->last_lit] = dist; \
> > +-    s->l_buf[s->last_lit++] = len; \
> > ++    s->sym_buf[s->sym_next++] = dist; \
> > ++    s->sym_buf[s->sym_next++] = dist >> 8; \
> > ++    s->sym_buf[s->sym_next++] = len; \
> > +     dist--; \
> > +     s->dyn_ltree[_length_code[len]+LITERALS+1].Freq++; \
> > +     s->dyn_dtree[d_code(dist)].Freq++; \
> > +-    flush = (s->last_lit == s->lit_bufsize-1); \
> > ++    flush = (s->sym_next == s->sym_end); \
> > +   }
> > + #else
> > + # define _tr_tally_lit(s, c, flush) flush = _tr_tally(s, 0, c)
> > +diff --git a/trees.c b/trees.c
> > +index 4f4a65011..decaeb7c3 100644
> > +--- a/trees.c
> > ++++ b/trees.c
> > +@@ -416,7 +416,7 @@ local void init_block(s)
> > +
> > +     s->dyn_ltree[END_BLOCK].Freq = 1;
> > +     s->opt_len = s->static_len = 0L;
> > +-    s->last_lit = s->matches = 0;
> > ++    s->sym_next = s->matches = 0;
> > + }
> > +
> > + #define SMALLEST 1
> > +@@ -948,7 +948,7 @@ void ZLIB_INTERNAL _tr_flush_block(s, buf, stored_len, last)
> > +
> > +         Tracev((stderr, "\nopt %lu(%lu) stat %lu(%lu) stored %lu lit %u ",
> > +                 opt_lenb, s->opt_len, static_lenb, s->static_len, stored_len,
> > +-                s->last_lit));
> > ++                s->sym_next / 3));
> > +
> > +         if (static_lenb <= opt_lenb) opt_lenb = static_lenb;
> > +
> > +@@ -1017,8 +1017,9 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
> > +     unsigned dist;  /* distance of matched string */
> > +     unsigned lc;    /* match length-MIN_MATCH or unmatched char (if dist==0) */
> > + {
> > +-    s->d_buf[s->last_lit] = (ush)dist;
> > +-    s->l_buf[s->last_lit++] = (uch)lc;
> > ++    s->sym_buf[s->sym_next++] = dist;
> > ++    s->sym_buf[s->sym_next++] = dist >> 8;
> > ++    s->sym_buf[s->sym_next++] = lc;
> > +     if (dist == 0) {
> > +         /* lc is the unmatched char */
> > +         s->dyn_ltree[lc].Freq++;
> > +@@ -1033,30 +1034,7 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
> > +         s->dyn_ltree[_length_code[lc]+LITERALS+1].Freq++;
> > +         s->dyn_dtree[d_code(dist)].Freq++;
> > +     }
> > +-
> > +-#ifdef TRUNCATE_BLOCK
> > +-    /* Try to guess if it is profitable to stop the current block here */
> > +-    if ((s->last_lit & 0x1fff) == 0 && s->level > 2) {
> > +-        /* Compute an upper bound for the compressed length */
> > +-        ulg out_length = (ulg)s->last_lit*8L;
> > +-        ulg in_length = (ulg)((long)s->strstart - s->block_start);
> > +-        int dcode;
> > +-        for (dcode = 0; dcode < D_CODES; dcode++) {
> > +-            out_length += (ulg)s->dyn_dtree[dcode].Freq *
> > +-                (5L+extra_dbits[dcode]);
> > +-        }
> > +-        out_length >>= 3;
> > +-        Tracev((stderr,"\nlast_lit %u, in %ld, out ~%ld(%ld%%) ",
> > +-               s->last_lit, in_length, out_length,
> > +-               100L - out_length*100L/in_length));
> > +-        if (s->matches < s->last_lit/2 && out_length < in_length/2) return 1;
> > +-    }
> > +-#endif
> > +-    return (s->last_lit == s->lit_bufsize-1);
> > +-    /* We avoid equality with lit_bufsize because of wraparound at 64K
> > +-     * on 16 bit machines and because stored blocks are restricted to
> > +-     * 64K-1 bytes.
> > +-     */
> > ++    return (s->sym_next == s->sym_end);
> > + }
> > +
> > + /* ===========================================================================
> > +@@ -1069,13 +1047,14 @@ local void compress_block(s, ltree, dtree)
> > + {
> > +     unsigned dist;      /* distance of matched string */
> > +     int lc;             /* match length or unmatched char (if dist == 0) */
> > +-    unsigned lx = 0;    /* running index in l_buf */
> > ++    unsigned sx = 0;    /* running index in sym_buf */
> > +     unsigned code;      /* the code to send */
> > +     int extra;          /* number of extra bits to send */
> > +
> > +-    if (s->last_lit != 0) do {
> > +-        dist = s->d_buf[lx];
> > +-        lc = s->l_buf[lx++];
> > ++    if (s->sym_next != 0) do {
> > ++        dist = s->sym_buf[sx++] & 0xff;
> > ++        dist += (unsigned)(s->sym_buf[sx++] & 0xff) << 8;
> > ++        lc = s->sym_buf[sx++];
> > +         if (dist == 0) {
> > +             send_code(s, lc, ltree); /* send a literal byte */
> > +             Tracecv(isgraph(lc), (stderr," '%c' ", lc));
> > +@@ -1100,11 +1079,10 @@ local void compress_block(s, ltree, dtree)
> > +             }
> > +         } /* literal or match pair ? */
> > +
> > +-        /* Check that the overlay between pending_buf and d_buf+l_buf is ok: */
> > +-        Assert((uInt)(s->pending) < s->lit_bufsize + 2*lx,
> > +-               "pendingBuf overflow");
> > ++        /* Check that the overlay between pending_buf and sym_buf is ok: */
> > ++        Assert(s->pending < s->lit_bufsize + sx, "pendingBuf overflow");
> > +
> > +-    } while (lx < s->last_lit);
> > ++    } while (sx < s->sym_next);
> > +
> > +     send_code(s, END_BLOCK, ltree);
> > + }
> > diff --git a/meta/recipes-core/zlib/zlib_1.2.11.bb b/meta/recipes-core/zlib/zlib_1.2.11.bb
> > index ef9431ae475..bc42cd64e9d 100644
> > --- a/meta/recipes-core/zlib/zlib_1.2.11.bb
> > +++ b/meta/recipes-core/zlib/zlib_1.2.11.bb
> > @@ -8,6 +8,7 @@ LIC_FILES_CHKSUM = "file://zlib.h;beginline=6;endline=23;md5=5377232268e952e9ef6
> >
> >  SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/${BPN}/${PV}/${BPN}-${PV}.tar.xz \
> >             file://ldflags-tests.patch \
> > +           file://CVE-2018-25032.patch \
> >             file://run-ptest \
> >             "
> >  UPSTREAM_CHECK_URI = "http://zlib.net/"
> > --
> > 2.25.1
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#163750): https://lists.openembedded.org/g/openembedded-core/message/163750
> > Mute This Topic: https://lists.openembedded.org/mt/90107518/3620601
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-03-30 20:40   ` Ross Burton
@ 2022-03-30 22:57     ` Steve Sakoman
  2022-04-04 15:22       ` Steve Sakoman
  2022-03-31  0:24     ` Mittal, Anuj
  1 sibling, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-03-30 22:57 UTC (permalink / raw)
  To: Ross Burton; +Cc: Mittal, Anuj, openembedded-core

On Wed, Mar 30, 2022 at 10:40 AM Ross Burton <ross@burtonini.com> wrote:
>
> Hm, turns out I was being too clever back in 2019.
>
> Anuj: the quick fix is to not use :prepend/:remove to manipulate
> SRC_URI, but just override it entirely.  You won't get the CVE but
> that will be flagged in scans and you (or even better,the fork
> maintainer) can rebase the CVE patch.
>
> As this is a fairly important CVE I'm tempted to say we merge to
> oe-core dunfell now and let meta-intel catch up...

Unfortunately this triggers other errors too :-(

https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/4987
https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4949

Steve


>
> Ross
>
> On Tue, 29 Mar 2022 at 20:49, Steve Sakoman <steve@sakoman.com> wrote:
> >
> > On Tue, Mar 29, 2022 at 3:07 AM Ross Burton <ross@burtonini.com> wrote:
> > >
> > > Signed-off-by: Ross Burton <ross.burton@arm.com>
> > > ---
> > >  .../zlib/zlib/CVE-2018-25032.patch            | 347 ++++++++++++++++++
> >
> > This breaks dunfell meta-intel:
> >
> > ERROR: Applying patch 'CVE-2018-25032.patch' on target directory
> > 'TOPDIR/tmp/work/corei7-64-poky-linux/zlib-intel/1.2.11.1.jtkv6.3-r0/git'
> > Command Error: 'quilt --quiltrc
> > TOPDIR/tmp/work/corei7-64-poky-linux/zlib-intel/1.2.11.1.jtkv6.3-r0/recipe-sysroot-native/etc/quiltrc
> > push' exited with 0  Output:
> > Applying patch CVE-2018-25032.patch
> > patching file deflate.c
> > Hunk #1 succeeded at 237 (offset -18 lines).
> > Hunk #2 succeeded at 319 (offset -5 lines).
> > Hunk #3 succeeded at 368 (offset -5 lines).
> > Hunk #4 succeeded at 590 (offset 1 line).
> > Hunk #5 succeeded at 1162 (offset 12 lines).
> > Hunk #6 succeeded at 1181 (offset 12 lines).
> > Hunk #7 succeeded at 1195 (offset 12 lines).
> > Hunk #8 succeeded at 1957 (offset -2 lines).
> > Hunk #9 succeeded at 1996 (offset -94 lines).
> > Hunk #10 FAILED at 2165.
> > Hunk #11 FAILED at 2204.
> > 2 out of 11 hunks FAILED -- rejects in file deflate.c
> > patching file deflate.h
> > Hunk #1 succeeded at 230 (offset 13 lines).
> > Hunk #2 succeeded at 252 (offset 13 lines).
> > Hunk #3 succeeded at 339 (offset 19 lines).
> > patching file trees.c
> > Hunk #1 succeeded at 343 (offset -73 lines).
> > Hunk #2 succeeded at 875 (offset -73 lines).
> > Hunk #3 succeeded at 944 (offset -73 lines).
> > Hunk #4 succeeded at 961 (offset -73 lines).
> > Hunk #5 succeeded at 974 (offset -73 lines).
> > Hunk #6 succeeded at 1006 (offset -73 lines).
> > Patch CVE-2018-25032.patch does not apply (enforce with -f)
> >
> > Steve
> >
> > >  meta/recipes-core/zlib/zlib_1.2.11.bb         |   1 +
> > >  2 files changed, 348 insertions(+)
> > >  create mode 100644 meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > >
> > > diff --git a/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > > new file mode 100644
> > > index 00000000000..5cb61836419
> > > --- /dev/null
> > > +++ b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > > @@ -0,0 +1,347 @@
> > > +CVE: CVE-2018-25032
> > > +Upstream-Status: Backport
> > > +Signed-off-by: Ross Burton <ross.burton@arm.com>
> > > +
> > > +From 5c44459c3b28a9bd3283aaceab7c615f8020c531 Mon Sep 17 00:00:00 2001
> > > +From: Mark Adler <madler@alumni.caltech.edu>
> > > +Date: Tue, 17 Apr 2018 22:09:22 -0700
> > > +Subject: [PATCH] Fix a bug that can crash deflate on some input when using
> > > + Z_FIXED.
> > > +
> > > +This bug was reported by Danilo Ramos of Eideticom, Inc. It has
> > > +lain in wait 13 years before being found! The bug was introduced
> > > +in zlib 1.2.2.2, with the addition of the Z_FIXED option. That
> > > +option forces the use of fixed Huffman codes. For rare inputs with
> > > +a large number of distant matches, the pending buffer into which
> > > +the compressed data is written can overwrite the distance symbol
> > > +table which it overlays. That results in corrupted output due to
> > > +invalid distances, and can result in out-of-bound accesses,
> > > +crashing the application.
> > > +
> > > +The fix here combines the distance buffer and literal/length
> > > +buffers into a single symbol buffer. Now three bytes of pending
> > > +buffer space are opened up for each literal or length/distance
> > > +pair consumed, instead of the previous two bytes. This assures
> > > +that the pending buffer cannot overwrite the symbol table, since
> > > +the maximum fixed code compressed length/distance is 31 bits, and
> > > +since there are four bytes of pending space for every three bytes
> > > +of symbol space.
> > > +---
> > > + deflate.c | 74 ++++++++++++++++++++++++++++++++++++++++---------------
> > > + deflate.h | 25 +++++++++----------
> > > + trees.c   | 50 +++++++++++--------------------------
> > > + 3 files changed, 79 insertions(+), 70 deletions(-)
> > > +
> > > +diff --git a/deflate.c b/deflate.c
> > > +index 425babc00..19cba873a 100644
> > > +--- a/deflate.c
> > > ++++ b/deflate.c
> > > +@@ -255,11 +255,6 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
> > > +     int wrap = 1;
> > > +     static const char my_version[] = ZLIB_VERSION;
> > > +
> > > +-    ushf *overlay;
> > > +-    /* We overlay pending_buf and d_buf+l_buf. This works since the average
> > > +-     * output size for (length,distance) codes is <= 24 bits.
> > > +-     */
> > > +-
> > > +     if (version == Z_NULL || version[0] != my_version[0] ||
> > > +         stream_size != sizeof(z_stream)) {
> > > +         return Z_VERSION_ERROR;
> > > +@@ -329,9 +324,47 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
> > > +
> > > +     s->lit_bufsize = 1 << (memLevel + 6); /* 16K elements by default */
> > > +
> > > +-    overlay = (ushf *) ZALLOC(strm, s->lit_bufsize, sizeof(ush)+2);
> > > +-    s->pending_buf = (uchf *) overlay;
> > > +-    s->pending_buf_size = (ulg)s->lit_bufsize * (sizeof(ush)+2L);
> > > ++    /* We overlay pending_buf and sym_buf. This works since the average size
> > > ++     * for length/distance pairs over any compressed block is assured to be 31
> > > ++     * bits or less.
> > > ++     *
> > > ++     * Analysis: The longest fixed codes are a length code of 8 bits plus 5
> > > ++     * extra bits, for lengths 131 to 257. The longest fixed distance codes are
> > > ++     * 5 bits plus 13 extra bits, for distances 16385 to 32768. The longest
> > > ++     * possible fixed-codes length/distance pair is then 31 bits total.
> > > ++     *
> > > ++     * sym_buf starts one-fourth of the way into pending_buf. So there are
> > > ++     * three bytes in sym_buf for every four bytes in pending_buf. Each symbol
> > > ++     * in sym_buf is three bytes -- two for the distance and one for the
> > > ++     * literal/length. As each symbol is consumed, the pointer to the next
> > > ++     * sym_buf value to read moves forward three bytes. From that symbol, up to
> > > ++     * 31 bits are written to pending_buf. The closest the written pending_buf
> > > ++     * bits gets to the next sym_buf symbol to read is just before the last
> > > ++     * code is written. At that time, 31*(n-2) bits have been written, just
> > > ++     * after 24*(n-2) bits have been consumed from sym_buf. sym_buf starts at
> > > ++     * 8*n bits into pending_buf. (Note that the symbol buffer fills when n-1
> > > ++     * symbols are written.) The closest the writing gets to what is unread is
> > > ++     * then n+14 bits. Here n is lit_bufsize, which is 16384 by default, and
> > > ++     * can range from 128 to 32768.
> > > ++     *
> > > ++     * Therefore, at a minimum, there are 142 bits of space between what is
> > > ++     * written and what is read in the overlain buffers, so the symbols cannot
> > > ++     * be overwritten by the compressed data. That space is actually 139 bits,
> > > ++     * due to the three-bit fixed-code block header.
> > > ++     *
> > > ++     * That covers the case where either Z_FIXED is specified, forcing fixed
> > > ++     * codes, or when the use of fixed codes is chosen, because that choice
> > > ++     * results in a smaller compressed block than dynamic codes. That latter
> > > ++     * condition then assures that the above analysis also covers all dynamic
> > > ++     * blocks. A dynamic-code block will only be chosen to be emitted if it has
> > > ++     * fewer bits than a fixed-code block would for the same set of symbols.
> > > ++     * Therefore its average symbol length is assured to be less than 31. So
> > > ++     * the compressed data for a dynamic block also cannot overwrite the
> > > ++     * symbols from which it is being constructed.
> > > ++     */
> > > ++
> > > ++    s->pending_buf = (uchf *) ZALLOC(strm, s->lit_bufsize, 4);
> > > ++    s->pending_buf_size = (ulg)s->lit_bufsize * 4;
> > > +
> > > +     if (s->window == Z_NULL || s->prev == Z_NULL || s->head == Z_NULL ||
> > > +         s->pending_buf == Z_NULL) {
> > > +@@ -340,8 +373,12 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
> > > +         deflateEnd (strm);
> > > +         return Z_MEM_ERROR;
> > > +     }
> > > +-    s->d_buf = overlay + s->lit_bufsize/sizeof(ush);
> > > +-    s->l_buf = s->pending_buf + (1+sizeof(ush))*s->lit_bufsize;
> > > ++    s->sym_buf = s->pending_buf + s->lit_bufsize;
> > > ++    s->sym_end = (s->lit_bufsize - 1) * 3;
> > > ++    /* We avoid equality with lit_bufsize*3 because of wraparound at 64K
> > > ++     * on 16 bit machines and because stored blocks are restricted to
> > > ++     * 64K-1 bytes.
> > > ++     */
> > > +
> > > +     s->level = level;
> > > +     s->strategy = strategy;
> > > +@@ -552,7 +589,7 @@ int ZEXPORT deflatePrime (strm, bits, value)
> > > +
> > > +     if (deflateStateCheck(strm)) return Z_STREAM_ERROR;
> > > +     s = strm->state;
> > > +-    if ((Bytef *)(s->d_buf) < s->pending_out + ((Buf_size + 7) >> 3))
> > > ++    if (s->sym_buf < s->pending_out + ((Buf_size + 7) >> 3))
> > > +         return Z_BUF_ERROR;
> > > +     do {
> > > +         put = Buf_size - s->bi_valid;
> > > +@@ -1113,7 +1150,6 @@ int ZEXPORT deflateCopy (dest, source)
> > > + #else
> > > +     deflate_state *ds;
> > > +     deflate_state *ss;
> > > +-    ushf *overlay;
> > > +
> > > +
> > > +     if (deflateStateCheck(source) || dest == Z_NULL) {
> > > +@@ -1133,8 +1169,7 @@ int ZEXPORT deflateCopy (dest, source)
> > > +     ds->window = (Bytef *) ZALLOC(dest, ds->w_size, 2*sizeof(Byte));
> > > +     ds->prev   = (Posf *)  ZALLOC(dest, ds->w_size, sizeof(Pos));
> > > +     ds->head   = (Posf *)  ZALLOC(dest, ds->hash_size, sizeof(Pos));
> > > +-    overlay = (ushf *) ZALLOC(dest, ds->lit_bufsize, sizeof(ush)+2);
> > > +-    ds->pending_buf = (uchf *) overlay;
> > > ++    ds->pending_buf = (uchf *) ZALLOC(dest, ds->lit_bufsize, 4);
> > > +
> > > +     if (ds->window == Z_NULL || ds->prev == Z_NULL || ds->head == Z_NULL ||
> > > +         ds->pending_buf == Z_NULL) {
> > > +@@ -1148,8 +1183,7 @@ int ZEXPORT deflateCopy (dest, source)
> > > +     zmemcpy(ds->pending_buf, ss->pending_buf, (uInt)ds->pending_buf_size);
> > > +
> > > +     ds->pending_out = ds->pending_buf + (ss->pending_out - ss->pending_buf);
> > > +-    ds->d_buf = overlay + ds->lit_bufsize/sizeof(ush);
> > > +-    ds->l_buf = ds->pending_buf + (1+sizeof(ush))*ds->lit_bufsize;
> > > ++    ds->sym_buf = ds->pending_buf + ds->lit_bufsize;
> > > +
> > > +     ds->l_desc.dyn_tree = ds->dyn_ltree;
> > > +     ds->d_desc.dyn_tree = ds->dyn_dtree;
> > > +@@ -1925,7 +1959,7 @@ local block_state deflate_fast(s, flush)
> > > +         FLUSH_BLOCK(s, 1);
> > > +         return finish_done;
> > > +     }
> > > +-    if (s->last_lit)
> > > ++    if (s->sym_next)
> > > +         FLUSH_BLOCK(s, 0);
> > > +     return block_done;
> > > + }
> > > +@@ -2056,7 +2090,7 @@ local block_state deflate_slow(s, flush)
> > > +         FLUSH_BLOCK(s, 1);
> > > +         return finish_done;
> > > +     }
> > > +-    if (s->last_lit)
> > > ++    if (s->sym_next)
> > > +         FLUSH_BLOCK(s, 0);
> > > +     return block_done;
> > > + }
> > > +@@ -2131,7 +2165,7 @@ local block_state deflate_rle(s, flush)
> > > +         FLUSH_BLOCK(s, 1);
> > > +         return finish_done;
> > > +     }
> > > +-    if (s->last_lit)
> > > ++    if (s->sym_next)
> > > +         FLUSH_BLOCK(s, 0);
> > > +     return block_done;
> > > + }
> > > +@@ -2170,7 +2204,7 @@ local block_state deflate_huff(s, flush)
> > > +         FLUSH_BLOCK(s, 1);
> > > +         return finish_done;
> > > +     }
> > > +-    if (s->last_lit)
> > > ++    if (s->sym_next)
> > > +         FLUSH_BLOCK(s, 0);
> > > +     return block_done;
> > > + }
> > > +diff --git a/deflate.h b/deflate.h
> > > +index 23ecdd312..d4cf1a98b 100644
> > > +--- a/deflate.h
> > > ++++ b/deflate.h
> > > +@@ -217,7 +217,7 @@ typedef struct internal_state {
> > > +     /* Depth of each subtree used as tie breaker for trees of equal frequency
> > > +      */
> > > +
> > > +-    uchf *l_buf;          /* buffer for literals or lengths */
> > > ++    uchf *sym_buf;        /* buffer for distances and literals/lengths */
> > > +
> > > +     uInt  lit_bufsize;
> > > +     /* Size of match buffer for literals/lengths.  There are 4 reasons for
> > > +@@ -239,13 +239,8 @@ typedef struct internal_state {
> > > +      *   - I can't count above 4
> > > +      */
> > > +
> > > +-    uInt last_lit;      /* running index in l_buf */
> > > +-
> > > +-    ushf *d_buf;
> > > +-    /* Buffer for distances. To simplify the code, d_buf and l_buf have
> > > +-     * the same number of elements. To use different lengths, an extra flag
> > > +-     * array would be necessary.
> > > +-     */
> > > ++    uInt sym_next;      /* running index in sym_buf */
> > > ++    uInt sym_end;       /* symbol table full when sym_next reaches this */
> > > +
> > > +     ulg opt_len;        /* bit length of current block with optimal trees */
> > > +     ulg static_len;     /* bit length of current block with static trees */
> > > +@@ -325,20 +320,22 @@ void ZLIB_INTERNAL _tr_stored_block OF((deflate_state *s, charf *buf,
> > > +
> > > + # define _tr_tally_lit(s, c, flush) \
> > > +   { uch cc = (c); \
> > > +-    s->d_buf[s->last_lit] = 0; \
> > > +-    s->l_buf[s->last_lit++] = cc; \
> > > ++    s->sym_buf[s->sym_next++] = 0; \
> > > ++    s->sym_buf[s->sym_next++] = 0; \
> > > ++    s->sym_buf[s->sym_next++] = cc; \
> > > +     s->dyn_ltree[cc].Freq++; \
> > > +-    flush = (s->last_lit == s->lit_bufsize-1); \
> > > ++    flush = (s->sym_next == s->sym_end); \
> > > +    }
> > > + # define _tr_tally_dist(s, distance, length, flush) \
> > > +   { uch len = (uch)(length); \
> > > +     ush dist = (ush)(distance); \
> > > +-    s->d_buf[s->last_lit] = dist; \
> > > +-    s->l_buf[s->last_lit++] = len; \
> > > ++    s->sym_buf[s->sym_next++] = dist; \
> > > ++    s->sym_buf[s->sym_next++] = dist >> 8; \
> > > ++    s->sym_buf[s->sym_next++] = len; \
> > > +     dist--; \
> > > +     s->dyn_ltree[_length_code[len]+LITERALS+1].Freq++; \
> > > +     s->dyn_dtree[d_code(dist)].Freq++; \
> > > +-    flush = (s->last_lit == s->lit_bufsize-1); \
> > > ++    flush = (s->sym_next == s->sym_end); \
> > > +   }
> > > + #else
> > > + # define _tr_tally_lit(s, c, flush) flush = _tr_tally(s, 0, c)
> > > +diff --git a/trees.c b/trees.c
> > > +index 4f4a65011..decaeb7c3 100644
> > > +--- a/trees.c
> > > ++++ b/trees.c
> > > +@@ -416,7 +416,7 @@ local void init_block(s)
> > > +
> > > +     s->dyn_ltree[END_BLOCK].Freq = 1;
> > > +     s->opt_len = s->static_len = 0L;
> > > +-    s->last_lit = s->matches = 0;
> > > ++    s->sym_next = s->matches = 0;
> > > + }
> > > +
> > > + #define SMALLEST 1
> > > +@@ -948,7 +948,7 @@ void ZLIB_INTERNAL _tr_flush_block(s, buf, stored_len, last)
> > > +
> > > +         Tracev((stderr, "\nopt %lu(%lu) stat %lu(%lu) stored %lu lit %u ",
> > > +                 opt_lenb, s->opt_len, static_lenb, s->static_len, stored_len,
> > > +-                s->last_lit));
> > > ++                s->sym_next / 3));
> > > +
> > > +         if (static_lenb <= opt_lenb) opt_lenb = static_lenb;
> > > +
> > > +@@ -1017,8 +1017,9 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
> > > +     unsigned dist;  /* distance of matched string */
> > > +     unsigned lc;    /* match length-MIN_MATCH or unmatched char (if dist==0) */
> > > + {
> > > +-    s->d_buf[s->last_lit] = (ush)dist;
> > > +-    s->l_buf[s->last_lit++] = (uch)lc;
> > > ++    s->sym_buf[s->sym_next++] = dist;
> > > ++    s->sym_buf[s->sym_next++] = dist >> 8;
> > > ++    s->sym_buf[s->sym_next++] = lc;
> > > +     if (dist == 0) {
> > > +         /* lc is the unmatched char */
> > > +         s->dyn_ltree[lc].Freq++;
> > > +@@ -1033,30 +1034,7 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
> > > +         s->dyn_ltree[_length_code[lc]+LITERALS+1].Freq++;
> > > +         s->dyn_dtree[d_code(dist)].Freq++;
> > > +     }
> > > +-
> > > +-#ifdef TRUNCATE_BLOCK
> > > +-    /* Try to guess if it is profitable to stop the current block here */
> > > +-    if ((s->last_lit & 0x1fff) == 0 && s->level > 2) {
> > > +-        /* Compute an upper bound for the compressed length */
> > > +-        ulg out_length = (ulg)s->last_lit*8L;
> > > +-        ulg in_length = (ulg)((long)s->strstart - s->block_start);
> > > +-        int dcode;
> > > +-        for (dcode = 0; dcode < D_CODES; dcode++) {
> > > +-            out_length += (ulg)s->dyn_dtree[dcode].Freq *
> > > +-                (5L+extra_dbits[dcode]);
> > > +-        }
> > > +-        out_length >>= 3;
> > > +-        Tracev((stderr,"\nlast_lit %u, in %ld, out ~%ld(%ld%%) ",
> > > +-               s->last_lit, in_length, out_length,
> > > +-               100L - out_length*100L/in_length));
> > > +-        if (s->matches < s->last_lit/2 && out_length < in_length/2) return 1;
> > > +-    }
> > > +-#endif
> > > +-    return (s->last_lit == s->lit_bufsize-1);
> > > +-    /* We avoid equality with lit_bufsize because of wraparound at 64K
> > > +-     * on 16 bit machines and because stored blocks are restricted to
> > > +-     * 64K-1 bytes.
> > > +-     */
> > > ++    return (s->sym_next == s->sym_end);
> > > + }
> > > +
> > > + /* ===========================================================================
> > > +@@ -1069,13 +1047,14 @@ local void compress_block(s, ltree, dtree)
> > > + {
> > > +     unsigned dist;      /* distance of matched string */
> > > +     int lc;             /* match length or unmatched char (if dist == 0) */
> > > +-    unsigned lx = 0;    /* running index in l_buf */
> > > ++    unsigned sx = 0;    /* running index in sym_buf */
> > > +     unsigned code;      /* the code to send */
> > > +     int extra;          /* number of extra bits to send */
> > > +
> > > +-    if (s->last_lit != 0) do {
> > > +-        dist = s->d_buf[lx];
> > > +-        lc = s->l_buf[lx++];
> > > ++    if (s->sym_next != 0) do {
> > > ++        dist = s->sym_buf[sx++] & 0xff;
> > > ++        dist += (unsigned)(s->sym_buf[sx++] & 0xff) << 8;
> > > ++        lc = s->sym_buf[sx++];
> > > +         if (dist == 0) {
> > > +             send_code(s, lc, ltree); /* send a literal byte */
> > > +             Tracecv(isgraph(lc), (stderr," '%c' ", lc));
> > > +@@ -1100,11 +1079,10 @@ local void compress_block(s, ltree, dtree)
> > > +             }
> > > +         } /* literal or match pair ? */
> > > +
> > > +-        /* Check that the overlay between pending_buf and d_buf+l_buf is ok: */
> > > +-        Assert((uInt)(s->pending) < s->lit_bufsize + 2*lx,
> > > +-               "pendingBuf overflow");
> > > ++        /* Check that the overlay between pending_buf and sym_buf is ok: */
> > > ++        Assert(s->pending < s->lit_bufsize + sx, "pendingBuf overflow");
> > > +
> > > +-    } while (lx < s->last_lit);
> > > ++    } while (sx < s->sym_next);
> > > +
> > > +     send_code(s, END_BLOCK, ltree);
> > > + }
> > > diff --git a/meta/recipes-core/zlib/zlib_1.2.11.bb b/meta/recipes-core/zlib/zlib_1.2.11.bb
> > > index ef9431ae475..bc42cd64e9d 100644
> > > --- a/meta/recipes-core/zlib/zlib_1.2.11.bb
> > > +++ b/meta/recipes-core/zlib/zlib_1.2.11.bb
> > > @@ -8,6 +8,7 @@ LIC_FILES_CHKSUM = "file://zlib.h;beginline=6;endline=23;md5=5377232268e952e9ef6
> > >
> > >  SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/${BPN}/${PV}/${BPN}-${PV}.tar.xz \
> > >             file://ldflags-tests.patch \
> > > +           file://CVE-2018-25032.patch \
> > >             file://run-ptest \
> > >             "
> > >  UPSTREAM_CHECK_URI = "http://zlib.net/"
> > > --
> > > 2.25.1
> > >
> > >
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > > Links: You receive all messages sent to this group.
> > > View/Reply Online (#163750): https://lists.openembedded.org/g/openembedded-core/message/163750
> > > Mute This Topic: https://lists.openembedded.org/mt/90107518/3620601
> > > Group Owner: openembedded-core+owner@lists.openembedded.org
> > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > >


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-03-30 20:40   ` Ross Burton
  2022-03-30 22:57     ` Steve Sakoman
@ 2022-03-31  0:24     ` Mittal, Anuj
  1 sibling, 0 replies; 38+ messages in thread
From: Mittal, Anuj @ 2022-03-31  0:24 UTC (permalink / raw)
  To: ross, steve; +Cc: openembedded-core

On Wed, 2022-03-30 at 21:40 +0100, Ross Burton wrote:
> Hm, turns out I was being too clever back in 2019.
> 
> Anuj: the quick fix is to not use :prepend/:remove to manipulate
> SRC_URI, but just override it entirely.  You won't get the CVE but
> that will be flagged in scans and you (or even better,the fork
> maintainer) can rebase the CVE patch.
> 
> As this is a fairly important CVE I'm tempted to say we merge to
> oe-core dunfell now and let meta-intel catch up...

Yes, I think we should merge this if there are no other issues and I
will take care of the meta-intel failure.

Thanks,

Anuj

> 
> Ross
> 
> On Tue, 29 Mar 2022 at 20:49, Steve Sakoman <steve@sakoman.com>
> wrote:
> > 
> > On Tue, Mar 29, 2022 at 3:07 AM Ross Burton <ross@burtonini.com>
> > wrote:
> > > 
> > > Signed-off-by: Ross Burton <ross.burton@arm.com>
> > > ---
> > >  .../zlib/zlib/CVE-2018-25032.patch            | 347
> > > ++++++++++++++++++
> > 
> > This breaks dunfell meta-intel:
> > 
> > ERROR: Applying patch 'CVE-2018-25032.patch' on target directory
> > 'TOPDIR/tmp/work/corei7-64-poky-linux/zlib-intel/1.2.11.1.jtkv6.3-
> > r0/git'
> > Command Error: 'quilt --quiltrc
> > TOPDIR/tmp/work/corei7-64-poky-linux/zlib-intel/1.2.11.1.jtkv6.3-
> > r0/recipe-sysroot-native/etc/quiltrc
> > push' exited with 0  Output:
> > Applying patch CVE-2018-25032.patch
> > patching file deflate.c
> > Hunk #1 succeeded at 237 (offset -18 lines).
> > Hunk #2 succeeded at 319 (offset -5 lines).
> > Hunk #3 succeeded at 368 (offset -5 lines).
> > Hunk #4 succeeded at 590 (offset 1 line).
> > Hunk #5 succeeded at 1162 (offset 12 lines).
> > Hunk #6 succeeded at 1181 (offset 12 lines).
> > Hunk #7 succeeded at 1195 (offset 12 lines).
> > Hunk #8 succeeded at 1957 (offset -2 lines).
> > Hunk #9 succeeded at 1996 (offset -94 lines).
> > Hunk #10 FAILED at 2165.
> > Hunk #11 FAILED at 2204.
> > 2 out of 11 hunks FAILED -- rejects in file deflate.c
> > patching file deflate.h
> > Hunk #1 succeeded at 230 (offset 13 lines).
> > Hunk #2 succeeded at 252 (offset 13 lines).
> > Hunk #3 succeeded at 339 (offset 19 lines).
> > patching file trees.c
> > Hunk #1 succeeded at 343 (offset -73 lines).
> > Hunk #2 succeeded at 875 (offset -73 lines).
> > Hunk #3 succeeded at 944 (offset -73 lines).
> > Hunk #4 succeeded at 961 (offset -73 lines).
> > Hunk #5 succeeded at 974 (offset -73 lines).
> > Hunk #6 succeeded at 1006 (offset -73 lines).
> > Patch CVE-2018-25032.patch does not apply (enforce with -f)
> > 
> > Steve
> > 
> > >  meta/recipes-core/zlib/zlib_1.2.11.bb         |   1 +
> > >  2 files changed, 348 insertions(+)
> > >  create mode 100644 meta/recipes-core/zlib/zlib/CVE-2018-
> > > 25032.patch
> > > 
> > > diff --git a/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > > b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > > new file mode 100644
> > > index 00000000000..5cb61836419
> > > --- /dev/null
> > > +++ b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > > @@ -0,0 +1,347 @@
> > > +CVE: CVE-2018-25032
> > > +Upstream-Status: Backport
> > > +Signed-off-by: Ross Burton <ross.burton@arm.com>
> > > +
> > > +From 5c44459c3b28a9bd3283aaceab7c615f8020c531 Mon Sep 17
> > > 00:00:00 2001
> > > +From: Mark Adler <madler@alumni.caltech.edu>
> > > +Date: Tue, 17 Apr 2018 22:09:22 -0700
> > > +Subject: [PATCH] Fix a bug that can crash deflate on some input
> > > when using
> > > + Z_FIXED.
> > > +
> > > +This bug was reported by Danilo Ramos of Eideticom, Inc. It has
> > > +lain in wait 13 years before being found! The bug was introduced
> > > +in zlib 1.2.2.2, with the addition of the Z_FIXED option. That
> > > +option forces the use of fixed Huffman codes. For rare inputs
> > > with
> > > +a large number of distant matches, the pending buffer into which
> > > +the compressed data is written can overwrite the distance symbol
> > > +table which it overlays. That results in corrupted output due to
> > > +invalid distances, and can result in out-of-bound accesses,
> > > +crashing the application.
> > > +
> > > +The fix here combines the distance buffer and literal/length
> > > +buffers into a single symbol buffer. Now three bytes of pending
> > > +buffer space are opened up for each literal or length/distance
> > > +pair consumed, instead of the previous two bytes. This assures
> > > +that the pending buffer cannot overwrite the symbol table, since
> > > +the maximum fixed code compressed length/distance is 31 bits,
> > > and
> > > +since there are four bytes of pending space for every three
> > > bytes
> > > +of symbol space.
> > > +---
> > > + deflate.c | 74 ++++++++++++++++++++++++++++++++++++++++--------
> > > -------
> > > + deflate.h | 25 +++++++++----------
> > > + trees.c   | 50 +++++++++++--------------------------
> > > + 3 files changed, 79 insertions(+), 70 deletions(-)
> > > +
> > > +diff --git a/deflate.c b/deflate.c
> > > +index 425babc00..19cba873a 100644
> > > +--- a/deflate.c
> > > ++++ b/deflate.c
> > > +@@ -255,11 +255,6 @@ int ZEXPORT deflateInit2_(strm, level,
> > > method, windowBits, memLevel, strategy,
> > > +     int wrap = 1;
> > > +     static const char my_version[] = ZLIB_VERSION;
> > > +
> > > +-    ushf *overlay;
> > > +-    /* We overlay pending_buf and d_buf+l_buf. This works since
> > > the average
> > > +-     * output size for (length,distance) codes is <= 24 bits.
> > > +-     */
> > > +-
> > > +     if (version == Z_NULL || version[0] != my_version[0] ||
> > > +         stream_size != sizeof(z_stream)) {
> > > +         return Z_VERSION_ERROR;
> > > +@@ -329,9 +324,47 @@ int ZEXPORT deflateInit2_(strm, level,
> > > method, windowBits, memLevel, strategy,
> > > +
> > > +     s->lit_bufsize = 1 << (memLevel + 6); /* 16K elements by
> > > default */
> > > +
> > > +-    overlay = (ushf *) ZALLOC(strm, s->lit_bufsize,
> > > sizeof(ush)+2);
> > > +-    s->pending_buf = (uchf *) overlay;
> > > +-    s->pending_buf_size = (ulg)s->lit_bufsize *
> > > (sizeof(ush)+2L);
> > > ++    /* We overlay pending_buf and sym_buf. This works since the
> > > average size
> > > ++     * for length/distance pairs over any compressed block is
> > > assured to be 31
> > > ++     * bits or less.
> > > ++     *
> > > ++     * Analysis: The longest fixed codes are a length code of 8
> > > bits plus 5
> > > ++     * extra bits, for lengths 131 to 257. The longest fixed
> > > distance codes are
> > > ++     * 5 bits plus 13 extra bits, for distances 16385 to 32768.
> > > The longest
> > > ++     * possible fixed-codes length/distance pair is then 31
> > > bits total.
> > > ++     *
> > > ++     * sym_buf starts one-fourth of the way into pending_buf.
> > > So there are
> > > ++     * three bytes in sym_buf for every four bytes in
> > > pending_buf. Each symbol
> > > ++     * in sym_buf is three bytes -- two for the distance and
> > > one for the
> > > ++     * literal/length. As each symbol is consumed, the pointer
> > > to the next
> > > ++     * sym_buf value to read moves forward three bytes. From
> > > that symbol, up to
> > > ++     * 31 bits are written to pending_buf. The closest the
> > > written pending_buf
> > > ++     * bits gets to the next sym_buf symbol to read is just
> > > before the last
> > > ++     * code is written. At that time, 31*(n-2) bits have been
> > > written, just
> > > ++     * after 24*(n-2) bits have been consumed from sym_buf.
> > > sym_buf starts at
> > > ++     * 8*n bits into pending_buf. (Note that the symbol buffer
> > > fills when n-1
> > > ++     * symbols are written.) The closest the writing gets to
> > > what is unread is
> > > ++     * then n+14 bits. Here n is lit_bufsize, which is 16384 by
> > > default, and
> > > ++     * can range from 128 to 32768.
> > > ++     *
> > > ++     * Therefore, at a minimum, there are 142 bits of space
> > > between what is
> > > ++     * written and what is read in the overlain buffers, so the
> > > symbols cannot
> > > ++     * be overwritten by the compressed data. That space is
> > > actually 139 bits,
> > > ++     * due to the three-bit fixed-code block header.
> > > ++     *
> > > ++     * That covers the case where either Z_FIXED is specified,
> > > forcing fixed
> > > ++     * codes, or when the use of fixed codes is chosen, because
> > > that choice
> > > ++     * results in a smaller compressed block than dynamic
> > > codes. That latter
> > > ++     * condition then assures that the above analysis also
> > > covers all dynamic
> > > ++     * blocks. A dynamic-code block will only be chosen to be
> > > emitted if it has
> > > ++     * fewer bits than a fixed-code block would for the same
> > > set of symbols.
> > > ++     * Therefore its average symbol length is assured to be
> > > less than 31. So
> > > ++     * the compressed data for a dynamic block also cannot
> > > overwrite the
> > > ++     * symbols from which it is being constructed.
> > > ++     */
> > > ++
> > > ++    s->pending_buf = (uchf *) ZALLOC(strm, s->lit_bufsize, 4);
> > > ++    s->pending_buf_size = (ulg)s->lit_bufsize * 4;
> > > +
> > > +     if (s->window == Z_NULL || s->prev == Z_NULL || s->head ==
> > > Z_NULL ||
> > > +         s->pending_buf == Z_NULL) {
> > > +@@ -340,8 +373,12 @@ int ZEXPORT deflateInit2_(strm, level,
> > > method, windowBits, memLevel, strategy,
> > > +         deflateEnd (strm);
> > > +         return Z_MEM_ERROR;
> > > +     }
> > > +-    s->d_buf = overlay + s->lit_bufsize/sizeof(ush);
> > > +-    s->l_buf = s->pending_buf + (1+sizeof(ush))*s->lit_bufsize;
> > > ++    s->sym_buf = s->pending_buf + s->lit_bufsize;
> > > ++    s->sym_end = (s->lit_bufsize - 1) * 3;
> > > ++    /* We avoid equality with lit_bufsize*3 because of
> > > wraparound at 64K
> > > ++     * on 16 bit machines and because stored blocks are
> > > restricted to
> > > ++     * 64K-1 bytes.
> > > ++     */
> > > +
> > > +     s->level = level;
> > > +     s->strategy = strategy;
> > > +@@ -552,7 +589,7 @@ int ZEXPORT deflatePrime (strm, bits, value)
> > > +
> > > +     if (deflateStateCheck(strm)) return Z_STREAM_ERROR;
> > > +     s = strm->state;
> > > +-    if ((Bytef *)(s->d_buf) < s->pending_out + ((Buf_size + 7)
> > > >> 3))
> > > ++    if (s->sym_buf < s->pending_out + ((Buf_size + 7) >> 3))
> > > +         return Z_BUF_ERROR;
> > > +     do {
> > > +         put = Buf_size - s->bi_valid;
> > > +@@ -1113,7 +1150,6 @@ int ZEXPORT deflateCopy (dest, source)
> > > + #else
> > > +     deflate_state *ds;
> > > +     deflate_state *ss;
> > > +-    ushf *overlay;
> > > +
> > > +
> > > +     if (deflateStateCheck(source) || dest == Z_NULL) {
> > > +@@ -1133,8 +1169,7 @@ int ZEXPORT deflateCopy (dest, source)
> > > +     ds->window = (Bytef *) ZALLOC(dest, ds->w_size,
> > > 2*sizeof(Byte));
> > > +     ds->prev   = (Posf *)  ZALLOC(dest, ds->w_size,
> > > sizeof(Pos));
> > > +     ds->head   = (Posf *)  ZALLOC(dest, ds->hash_size,
> > > sizeof(Pos));
> > > +-    overlay = (ushf *) ZALLOC(dest, ds->lit_bufsize,
> > > sizeof(ush)+2);
> > > +-    ds->pending_buf = (uchf *) overlay;
> > > ++    ds->pending_buf = (uchf *) ZALLOC(dest, ds->lit_bufsize,
> > > 4);
> > > +
> > > +     if (ds->window == Z_NULL || ds->prev == Z_NULL || ds->head
> > > == Z_NULL ||
> > > +         ds->pending_buf == Z_NULL) {
> > > +@@ -1148,8 +1183,7 @@ int ZEXPORT deflateCopy (dest, source)
> > > +     zmemcpy(ds->pending_buf, ss->pending_buf, (uInt)ds-
> > > >pending_buf_size);
> > > +
> > > +     ds->pending_out = ds->pending_buf + (ss->pending_out - ss-
> > > >pending_buf);
> > > +-    ds->d_buf = overlay + ds->lit_bufsize/sizeof(ush);
> > > +-    ds->l_buf = ds->pending_buf + (1+sizeof(ush))*ds-
> > > >lit_bufsize;
> > > ++    ds->sym_buf = ds->pending_buf + ds->lit_bufsize;
> > > +
> > > +     ds->l_desc.dyn_tree = ds->dyn_ltree;
> > > +     ds->d_desc.dyn_tree = ds->dyn_dtree;
> > > +@@ -1925,7 +1959,7 @@ local block_state deflate_fast(s, flush)
> > > +         FLUSH_BLOCK(s, 1);
> > > +         return finish_done;
> > > +     }
> > > +-    if (s->last_lit)
> > > ++    if (s->sym_next)
> > > +         FLUSH_BLOCK(s, 0);
> > > +     return block_done;
> > > + }
> > > +@@ -2056,7 +2090,7 @@ local block_state deflate_slow(s, flush)
> > > +         FLUSH_BLOCK(s, 1);
> > > +         return finish_done;
> > > +     }
> > > +-    if (s->last_lit)
> > > ++    if (s->sym_next)
> > > +         FLUSH_BLOCK(s, 0);
> > > +     return block_done;
> > > + }
> > > +@@ -2131,7 +2165,7 @@ local block_state deflate_rle(s, flush)
> > > +         FLUSH_BLOCK(s, 1);
> > > +         return finish_done;
> > > +     }
> > > +-    if (s->last_lit)
> > > ++    if (s->sym_next)
> > > +         FLUSH_BLOCK(s, 0);
> > > +     return block_done;
> > > + }
> > > +@@ -2170,7 +2204,7 @@ local block_state deflate_huff(s, flush)
> > > +         FLUSH_BLOCK(s, 1);
> > > +         return finish_done;
> > > +     }
> > > +-    if (s->last_lit)
> > > ++    if (s->sym_next)
> > > +         FLUSH_BLOCK(s, 0);
> > > +     return block_done;
> > > + }
> > > +diff --git a/deflate.h b/deflate.h
> > > +index 23ecdd312..d4cf1a98b 100644
> > > +--- a/deflate.h
> > > ++++ b/deflate.h
> > > +@@ -217,7 +217,7 @@ typedef struct internal_state {
> > > +     /* Depth of each subtree used as tie breaker for trees of
> > > equal frequency
> > > +      */
> > > +
> > > +-    uchf *l_buf;          /* buffer for literals or lengths */
> > > ++    uchf *sym_buf;        /* buffer for distances and
> > > literals/lengths */
> > > +
> > > +     uInt  lit_bufsize;
> > > +     /* Size of match buffer for literals/lengths.  There are 4
> > > reasons for
> > > +@@ -239,13 +239,8 @@ typedef struct internal_state {
> > > +      *   - I can't count above 4
> > > +      */
> > > +
> > > +-    uInt last_lit;      /* running index in l_buf */
> > > +-
> > > +-    ushf *d_buf;
> > > +-    /* Buffer for distances. To simplify the code, d_buf and
> > > l_buf have
> > > +-     * the same number of elements. To use different lengths,
> > > an extra flag
> > > +-     * array would be necessary.
> > > +-     */
> > > ++    uInt sym_next;      /* running index in sym_buf */
> > > ++    uInt sym_end;       /* symbol table full when sym_next
> > > reaches this */
> > > +
> > > +     ulg opt_len;        /* bit length of current block with
> > > optimal trees */
> > > +     ulg static_len;     /* bit length of current block with
> > > static trees */
> > > +@@ -325,20 +320,22 @@ void ZLIB_INTERNAL _tr_stored_block
> > > OF((deflate_state *s, charf *buf,
> > > +
> > > + # define _tr_tally_lit(s, c, flush) \
> > > +   { uch cc = (c); \
> > > +-    s->d_buf[s->last_lit] = 0; \
> > > +-    s->l_buf[s->last_lit++] = cc; \
> > > ++    s->sym_buf[s->sym_next++] = 0; \
> > > ++    s->sym_buf[s->sym_next++] = 0; \
> > > ++    s->sym_buf[s->sym_next++] = cc; \
> > > +     s->dyn_ltree[cc].Freq++; \
> > > +-    flush = (s->last_lit == s->lit_bufsize-1); \
> > > ++    flush = (s->sym_next == s->sym_end); \
> > > +    }
> > > + # define _tr_tally_dist(s, distance, length, flush) \
> > > +   { uch len = (uch)(length); \
> > > +     ush dist = (ush)(distance); \
> > > +-    s->d_buf[s->last_lit] = dist; \
> > > +-    s->l_buf[s->last_lit++] = len; \
> > > ++    s->sym_buf[s->sym_next++] = dist; \
> > > ++    s->sym_buf[s->sym_next++] = dist >> 8; \
> > > ++    s->sym_buf[s->sym_next++] = len; \
> > > +     dist--; \
> > > +     s->dyn_ltree[_length_code[len]+LITERALS+1].Freq++; \
> > > +     s->dyn_dtree[d_code(dist)].Freq++; \
> > > +-    flush = (s->last_lit == s->lit_bufsize-1); \
> > > ++    flush = (s->sym_next == s->sym_end); \
> > > +   }
> > > + #else
> > > + # define _tr_tally_lit(s, c, flush) flush = _tr_tally(s, 0, c)
> > > +diff --git a/trees.c b/trees.c
> > > +index 4f4a65011..decaeb7c3 100644
> > > +--- a/trees.c
> > > ++++ b/trees.c
> > > +@@ -416,7 +416,7 @@ local void init_block(s)
> > > +
> > > +     s->dyn_ltree[END_BLOCK].Freq = 1;
> > > +     s->opt_len = s->static_len = 0L;
> > > +-    s->last_lit = s->matches = 0;
> > > ++    s->sym_next = s->matches = 0;
> > > + }
> > > +
> > > + #define SMALLEST 1
> > > +@@ -948,7 +948,7 @@ void ZLIB_INTERNAL _tr_flush_block(s, buf,
> > > stored_len, last)
> > > +
> > > +         Tracev((stderr, "\nopt %lu(%lu) stat %lu(%lu) stored
> > > %lu lit %u ",
> > > +                 opt_lenb, s->opt_len, static_lenb, s-
> > > >static_len, stored_len,
> > > +-                s->last_lit));
> > > ++                s->sym_next / 3));
> > > +
> > > +         if (static_lenb <= opt_lenb) opt_lenb = static_lenb;
> > > +
> > > +@@ -1017,8 +1017,9 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
> > > +     unsigned dist;  /* distance of matched string */
> > > +     unsigned lc;    /* match length-MIN_MATCH or unmatched char
> > > (if dist==0) */
> > > + {
> > > +-    s->d_buf[s->last_lit] = (ush)dist;
> > > +-    s->l_buf[s->last_lit++] = (uch)lc;
> > > ++    s->sym_buf[s->sym_next++] = dist;
> > > ++    s->sym_buf[s->sym_next++] = dist >> 8;
> > > ++    s->sym_buf[s->sym_next++] = lc;
> > > +     if (dist == 0) {
> > > +         /* lc is the unmatched char */
> > > +         s->dyn_ltree[lc].Freq++;
> > > +@@ -1033,30 +1034,7 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
> > > +         s->dyn_ltree[_length_code[lc]+LITERALS+1].Freq++;
> > > +         s->dyn_dtree[d_code(dist)].Freq++;
> > > +     }
> > > +-
> > > +-#ifdef TRUNCATE_BLOCK
> > > +-    /* Try to guess if it is profitable to stop the current
> > > block here */
> > > +-    if ((s->last_lit & 0x1fff) == 0 && s->level > 2) {
> > > +-        /* Compute an upper bound for the compressed length */
> > > +-        ulg out_length = (ulg)s->last_lit*8L;
> > > +-        ulg in_length = (ulg)((long)s->strstart - s-
> > > >block_start);
> > > +-        int dcode;
> > > +-        for (dcode = 0; dcode < D_CODES; dcode++) {
> > > +-            out_length += (ulg)s->dyn_dtree[dcode].Freq *
> > > +-                (5L+extra_dbits[dcode]);
> > > +-        }
> > > +-        out_length >>= 3;
> > > +-        Tracev((stderr,"\nlast_lit %u, in %ld, out ~%ld(%ld%%)
> > > ",
> > > +-               s->last_lit, in_length, out_length,
> > > +-               100L - out_length*100L/in_length));
> > > +-        if (s->matches < s->last_lit/2 && out_length <
> > > in_length/2) return 1;
> > > +-    }
> > > +-#endif
> > > +-    return (s->last_lit == s->lit_bufsize-1);
> > > +-    /* We avoid equality with lit_bufsize because of wraparound
> > > at 64K
> > > +-     * on 16 bit machines and because stored blocks are
> > > restricted to
> > > +-     * 64K-1 bytes.
> > > +-     */
> > > ++    return (s->sym_next == s->sym_end);
> > > + }
> > > +
> > > + /*
> > > =================================================================
> > > ==========
> > > +@@ -1069,13 +1047,14 @@ local void compress_block(s, ltree,
> > > dtree)
> > > + {
> > > +     unsigned dist;      /* distance of matched string */
> > > +     int lc;             /* match length or unmatched char (if
> > > dist == 0) */
> > > +-    unsigned lx = 0;    /* running index in l_buf */
> > > ++    unsigned sx = 0;    /* running index in sym_buf */
> > > +     unsigned code;      /* the code to send */
> > > +     int extra;          /* number of extra bits to send */
> > > +
> > > +-    if (s->last_lit != 0) do {
> > > +-        dist = s->d_buf[lx];
> > > +-        lc = s->l_buf[lx++];
> > > ++    if (s->sym_next != 0) do {
> > > ++        dist = s->sym_buf[sx++] & 0xff;
> > > ++        dist += (unsigned)(s->sym_buf[sx++] & 0xff) << 8;
> > > ++        lc = s->sym_buf[sx++];
> > > +         if (dist == 0) {
> > > +             send_code(s, lc, ltree); /* send a literal byte */
> > > +             Tracecv(isgraph(lc), (stderr," '%c' ", lc));
> > > +@@ -1100,11 +1079,10 @@ local void compress_block(s, ltree,
> > > dtree)
> > > +             }
> > > +         } /* literal or match pair ? */
> > > +
> > > +-        /* Check that the overlay between pending_buf and
> > > d_buf+l_buf is ok: */
> > > +-        Assert((uInt)(s->pending) < s->lit_bufsize + 2*lx,
> > > +-               "pendingBuf overflow");
> > > ++        /* Check that the overlay between pending_buf and
> > > sym_buf is ok: */
> > > ++        Assert(s->pending < s->lit_bufsize + sx, "pendingBuf
> > > overflow");
> > > +
> > > +-    } while (lx < s->last_lit);
> > > ++    } while (sx < s->sym_next);
> > > +
> > > +     send_code(s, END_BLOCK, ltree);
> > > + }
> > > diff --git a/meta/recipes-core/zlib/zlib_1.2.11.bb
> > > b/meta/recipes-core/zlib/zlib_1.2.11.bb
> > > index ef9431ae475..bc42cd64e9d 100644
> > > --- a/meta/recipes-core/zlib/zlib_1.2.11.bb
> > > +++ b/meta/recipes-core/zlib/zlib_1.2.11.bb
> > > @@ -8,6 +8,7 @@ LIC_FILES_CHKSUM =
> > > "file://zlib.h;beginline=6;endline=23;md5=5377232268e952e9ef6
> > > 
> > >  SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/${BPN}/${PV}/${BPN}-
> > > ${PV}.tar.xz \
> > >             file://ldflags-tests.patch \
> > > +           file://CVE-2018-25032.patch \
> > >             file://run-ptest \
> > >             "
> > >  UPSTREAM_CHECK_URI = "http://zlib.net/"
> > > --
> > > 2.25.1
> > > 
> > > 
> > > 
> > > 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#163793):
> https://lists.openembedded.org/g/openembedded-core/message/163793
> Mute This Topic: https://lists.openembedded.org/mt/90107518/3616702
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe:
> https://lists.openembedded.org/g/openembedded-core/unsub [
> anuj.mittal@intel.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-03-30 22:57     ` Steve Sakoman
@ 2022-04-04 15:22       ` Steve Sakoman
  2022-04-11 17:31         ` Ralph Siemsen
  0 siblings, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-04-04 15:22 UTC (permalink / raw)
  To: Ross Burton; +Cc: Mittal, Anuj, openembedded-core

On Wed, Mar 30, 2022 at 12:57 PM Steve Sakoman <steve@sakoman.com> wrote:
>
> On Wed, Mar 30, 2022 at 10:40 AM Ross Burton <ross@burtonini.com> wrote:
> >
> > Hm, turns out I was being too clever back in 2019.
> >
> > Anuj: the quick fix is to not use :prepend/:remove to manipulate
> > SRC_URI, but just override it entirely.  You won't get the CVE but
> > that will be flagged in scans and you (or even better,the fork
> > maintainer) can rebase the CVE patch.
> >
> > As this is a fairly important CVE I'm tempted to say we merge to
> > oe-core dunfell now and let meta-intel catch up...
>
> Unfortunately this triggers other errors too :-(
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/4987
> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4949

Any thoughts on the above errors, Ross?

Steve


> > On Tue, 29 Mar 2022 at 20:49, Steve Sakoman <steve@sakoman.com> wrote:
> > >
> > > On Tue, Mar 29, 2022 at 3:07 AM Ross Burton <ross@burtonini.com> wrote:
> > > >
> > > > Signed-off-by: Ross Burton <ross.burton@arm.com>
> > > > ---
> > > >  .../zlib/zlib/CVE-2018-25032.patch            | 347 ++++++++++++++++++
> > >
> > > This breaks dunfell meta-intel:
> > >
> > > ERROR: Applying patch 'CVE-2018-25032.patch' on target directory
> > > 'TOPDIR/tmp/work/corei7-64-poky-linux/zlib-intel/1.2.11.1.jtkv6.3-r0/git'
> > > Command Error: 'quilt --quiltrc
> > > TOPDIR/tmp/work/corei7-64-poky-linux/zlib-intel/1.2.11.1.jtkv6.3-r0/recipe-sysroot-native/etc/quiltrc
> > > push' exited with 0  Output:
> > > Applying patch CVE-2018-25032.patch
> > > patching file deflate.c
> > > Hunk #1 succeeded at 237 (offset -18 lines).
> > > Hunk #2 succeeded at 319 (offset -5 lines).
> > > Hunk #3 succeeded at 368 (offset -5 lines).
> > > Hunk #4 succeeded at 590 (offset 1 line).
> > > Hunk #5 succeeded at 1162 (offset 12 lines).
> > > Hunk #6 succeeded at 1181 (offset 12 lines).
> > > Hunk #7 succeeded at 1195 (offset 12 lines).
> > > Hunk #8 succeeded at 1957 (offset -2 lines).
> > > Hunk #9 succeeded at 1996 (offset -94 lines).
> > > Hunk #10 FAILED at 2165.
> > > Hunk #11 FAILED at 2204.
> > > 2 out of 11 hunks FAILED -- rejects in file deflate.c
> > > patching file deflate.h
> > > Hunk #1 succeeded at 230 (offset 13 lines).
> > > Hunk #2 succeeded at 252 (offset 13 lines).
> > > Hunk #3 succeeded at 339 (offset 19 lines).
> > > patching file trees.c
> > > Hunk #1 succeeded at 343 (offset -73 lines).
> > > Hunk #2 succeeded at 875 (offset -73 lines).
> > > Hunk #3 succeeded at 944 (offset -73 lines).
> > > Hunk #4 succeeded at 961 (offset -73 lines).
> > > Hunk #5 succeeded at 974 (offset -73 lines).
> > > Hunk #6 succeeded at 1006 (offset -73 lines).
> > > Patch CVE-2018-25032.patch does not apply (enforce with -f)
> > >
> > > Steve
> > >
> > > >  meta/recipes-core/zlib/zlib_1.2.11.bb         |   1 +
> > > >  2 files changed, 348 insertions(+)
> > > >  create mode 100644 meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > > >
> > > > diff --git a/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > > > new file mode 100644
> > > > index 00000000000..5cb61836419
> > > > --- /dev/null
> > > > +++ b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > > > @@ -0,0 +1,347 @@
> > > > +CVE: CVE-2018-25032
> > > > +Upstream-Status: Backport
> > > > +Signed-off-by: Ross Burton <ross.burton@arm.com>
> > > > +
> > > > +From 5c44459c3b28a9bd3283aaceab7c615f8020c531 Mon Sep 17 00:00:00 2001
> > > > +From: Mark Adler <madler@alumni.caltech.edu>
> > > > +Date: Tue, 17 Apr 2018 22:09:22 -0700
> > > > +Subject: [PATCH] Fix a bug that can crash deflate on some input when using
> > > > + Z_FIXED.
> > > > +
> > > > +This bug was reported by Danilo Ramos of Eideticom, Inc. It has
> > > > +lain in wait 13 years before being found! The bug was introduced
> > > > +in zlib 1.2.2.2, with the addition of the Z_FIXED option. That
> > > > +option forces the use of fixed Huffman codes. For rare inputs with
> > > > +a large number of distant matches, the pending buffer into which
> > > > +the compressed data is written can overwrite the distance symbol
> > > > +table which it overlays. That results in corrupted output due to
> > > > +invalid distances, and can result in out-of-bound accesses,
> > > > +crashing the application.
> > > > +
> > > > +The fix here combines the distance buffer and literal/length
> > > > +buffers into a single symbol buffer. Now three bytes of pending
> > > > +buffer space are opened up for each literal or length/distance
> > > > +pair consumed, instead of the previous two bytes. This assures
> > > > +that the pending buffer cannot overwrite the symbol table, since
> > > > +the maximum fixed code compressed length/distance is 31 bits, and
> > > > +since there are four bytes of pending space for every three bytes
> > > > +of symbol space.
> > > > +---
> > > > + deflate.c | 74 ++++++++++++++++++++++++++++++++++++++++---------------
> > > > + deflate.h | 25 +++++++++----------
> > > > + trees.c   | 50 +++++++++++--------------------------
> > > > + 3 files changed, 79 insertions(+), 70 deletions(-)
> > > > +
> > > > +diff --git a/deflate.c b/deflate.c
> > > > +index 425babc00..19cba873a 100644
> > > > +--- a/deflate.c
> > > > ++++ b/deflate.c
> > > > +@@ -255,11 +255,6 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
> > > > +     int wrap = 1;
> > > > +     static const char my_version[] = ZLIB_VERSION;
> > > > +
> > > > +-    ushf *overlay;
> > > > +-    /* We overlay pending_buf and d_buf+l_buf. This works since the average
> > > > +-     * output size for (length,distance) codes is <= 24 bits.
> > > > +-     */
> > > > +-
> > > > +     if (version == Z_NULL || version[0] != my_version[0] ||
> > > > +         stream_size != sizeof(z_stream)) {
> > > > +         return Z_VERSION_ERROR;
> > > > +@@ -329,9 +324,47 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
> > > > +
> > > > +     s->lit_bufsize = 1 << (memLevel + 6); /* 16K elements by default */
> > > > +
> > > > +-    overlay = (ushf *) ZALLOC(strm, s->lit_bufsize, sizeof(ush)+2);
> > > > +-    s->pending_buf = (uchf *) overlay;
> > > > +-    s->pending_buf_size = (ulg)s->lit_bufsize * (sizeof(ush)+2L);
> > > > ++    /* We overlay pending_buf and sym_buf. This works since the average size
> > > > ++     * for length/distance pairs over any compressed block is assured to be 31
> > > > ++     * bits or less.
> > > > ++     *
> > > > ++     * Analysis: The longest fixed codes are a length code of 8 bits plus 5
> > > > ++     * extra bits, for lengths 131 to 257. The longest fixed distance codes are
> > > > ++     * 5 bits plus 13 extra bits, for distances 16385 to 32768. The longest
> > > > ++     * possible fixed-codes length/distance pair is then 31 bits total.
> > > > ++     *
> > > > ++     * sym_buf starts one-fourth of the way into pending_buf. So there are
> > > > ++     * three bytes in sym_buf for every four bytes in pending_buf. Each symbol
> > > > ++     * in sym_buf is three bytes -- two for the distance and one for the
> > > > ++     * literal/length. As each symbol is consumed, the pointer to the next
> > > > ++     * sym_buf value to read moves forward three bytes. From that symbol, up to
> > > > ++     * 31 bits are written to pending_buf. The closest the written pending_buf
> > > > ++     * bits gets to the next sym_buf symbol to read is just before the last
> > > > ++     * code is written. At that time, 31*(n-2) bits have been written, just
> > > > ++     * after 24*(n-2) bits have been consumed from sym_buf. sym_buf starts at
> > > > ++     * 8*n bits into pending_buf. (Note that the symbol buffer fills when n-1
> > > > ++     * symbols are written.) The closest the writing gets to what is unread is
> > > > ++     * then n+14 bits. Here n is lit_bufsize, which is 16384 by default, and
> > > > ++     * can range from 128 to 32768.
> > > > ++     *
> > > > ++     * Therefore, at a minimum, there are 142 bits of space between what is
> > > > ++     * written and what is read in the overlain buffers, so the symbols cannot
> > > > ++     * be overwritten by the compressed data. That space is actually 139 bits,
> > > > ++     * due to the three-bit fixed-code block header.
> > > > ++     *
> > > > ++     * That covers the case where either Z_FIXED is specified, forcing fixed
> > > > ++     * codes, or when the use of fixed codes is chosen, because that choice
> > > > ++     * results in a smaller compressed block than dynamic codes. That latter
> > > > ++     * condition then assures that the above analysis also covers all dynamic
> > > > ++     * blocks. A dynamic-code block will only be chosen to be emitted if it has
> > > > ++     * fewer bits than a fixed-code block would for the same set of symbols.
> > > > ++     * Therefore its average symbol length is assured to be less than 31. So
> > > > ++     * the compressed data for a dynamic block also cannot overwrite the
> > > > ++     * symbols from which it is being constructed.
> > > > ++     */
> > > > ++
> > > > ++    s->pending_buf = (uchf *) ZALLOC(strm, s->lit_bufsize, 4);
> > > > ++    s->pending_buf_size = (ulg)s->lit_bufsize * 4;
> > > > +
> > > > +     if (s->window == Z_NULL || s->prev == Z_NULL || s->head == Z_NULL ||
> > > > +         s->pending_buf == Z_NULL) {
> > > > +@@ -340,8 +373,12 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
> > > > +         deflateEnd (strm);
> > > > +         return Z_MEM_ERROR;
> > > > +     }
> > > > +-    s->d_buf = overlay + s->lit_bufsize/sizeof(ush);
> > > > +-    s->l_buf = s->pending_buf + (1+sizeof(ush))*s->lit_bufsize;
> > > > ++    s->sym_buf = s->pending_buf + s->lit_bufsize;
> > > > ++    s->sym_end = (s->lit_bufsize - 1) * 3;
> > > > ++    /* We avoid equality with lit_bufsize*3 because of wraparound at 64K
> > > > ++     * on 16 bit machines and because stored blocks are restricted to
> > > > ++     * 64K-1 bytes.
> > > > ++     */
> > > > +
> > > > +     s->level = level;
> > > > +     s->strategy = strategy;
> > > > +@@ -552,7 +589,7 @@ int ZEXPORT deflatePrime (strm, bits, value)
> > > > +
> > > > +     if (deflateStateCheck(strm)) return Z_STREAM_ERROR;
> > > > +     s = strm->state;
> > > > +-    if ((Bytef *)(s->d_buf) < s->pending_out + ((Buf_size + 7) >> 3))
> > > > ++    if (s->sym_buf < s->pending_out + ((Buf_size + 7) >> 3))
> > > > +         return Z_BUF_ERROR;
> > > > +     do {
> > > > +         put = Buf_size - s->bi_valid;
> > > > +@@ -1113,7 +1150,6 @@ int ZEXPORT deflateCopy (dest, source)
> > > > + #else
> > > > +     deflate_state *ds;
> > > > +     deflate_state *ss;
> > > > +-    ushf *overlay;
> > > > +
> > > > +
> > > > +     if (deflateStateCheck(source) || dest == Z_NULL) {
> > > > +@@ -1133,8 +1169,7 @@ int ZEXPORT deflateCopy (dest, source)
> > > > +     ds->window = (Bytef *) ZALLOC(dest, ds->w_size, 2*sizeof(Byte));
> > > > +     ds->prev   = (Posf *)  ZALLOC(dest, ds->w_size, sizeof(Pos));
> > > > +     ds->head   = (Posf *)  ZALLOC(dest, ds->hash_size, sizeof(Pos));
> > > > +-    overlay = (ushf *) ZALLOC(dest, ds->lit_bufsize, sizeof(ush)+2);
> > > > +-    ds->pending_buf = (uchf *) overlay;
> > > > ++    ds->pending_buf = (uchf *) ZALLOC(dest, ds->lit_bufsize, 4);
> > > > +
> > > > +     if (ds->window == Z_NULL || ds->prev == Z_NULL || ds->head == Z_NULL ||
> > > > +         ds->pending_buf == Z_NULL) {
> > > > +@@ -1148,8 +1183,7 @@ int ZEXPORT deflateCopy (dest, source)
> > > > +     zmemcpy(ds->pending_buf, ss->pending_buf, (uInt)ds->pending_buf_size);
> > > > +
> > > > +     ds->pending_out = ds->pending_buf + (ss->pending_out - ss->pending_buf);
> > > > +-    ds->d_buf = overlay + ds->lit_bufsize/sizeof(ush);
> > > > +-    ds->l_buf = ds->pending_buf + (1+sizeof(ush))*ds->lit_bufsize;
> > > > ++    ds->sym_buf = ds->pending_buf + ds->lit_bufsize;
> > > > +
> > > > +     ds->l_desc.dyn_tree = ds->dyn_ltree;
> > > > +     ds->d_desc.dyn_tree = ds->dyn_dtree;
> > > > +@@ -1925,7 +1959,7 @@ local block_state deflate_fast(s, flush)
> > > > +         FLUSH_BLOCK(s, 1);
> > > > +         return finish_done;
> > > > +     }
> > > > +-    if (s->last_lit)
> > > > ++    if (s->sym_next)
> > > > +         FLUSH_BLOCK(s, 0);
> > > > +     return block_done;
> > > > + }
> > > > +@@ -2056,7 +2090,7 @@ local block_state deflate_slow(s, flush)
> > > > +         FLUSH_BLOCK(s, 1);
> > > > +         return finish_done;
> > > > +     }
> > > > +-    if (s->last_lit)
> > > > ++    if (s->sym_next)
> > > > +         FLUSH_BLOCK(s, 0);
> > > > +     return block_done;
> > > > + }
> > > > +@@ -2131,7 +2165,7 @@ local block_state deflate_rle(s, flush)
> > > > +         FLUSH_BLOCK(s, 1);
> > > > +         return finish_done;
> > > > +     }
> > > > +-    if (s->last_lit)
> > > > ++    if (s->sym_next)
> > > > +         FLUSH_BLOCK(s, 0);
> > > > +     return block_done;
> > > > + }
> > > > +@@ -2170,7 +2204,7 @@ local block_state deflate_huff(s, flush)
> > > > +         FLUSH_BLOCK(s, 1);
> > > > +         return finish_done;
> > > > +     }
> > > > +-    if (s->last_lit)
> > > > ++    if (s->sym_next)
> > > > +         FLUSH_BLOCK(s, 0);
> > > > +     return block_done;
> > > > + }
> > > > +diff --git a/deflate.h b/deflate.h
> > > > +index 23ecdd312..d4cf1a98b 100644
> > > > +--- a/deflate.h
> > > > ++++ b/deflate.h
> > > > +@@ -217,7 +217,7 @@ typedef struct internal_state {
> > > > +     /* Depth of each subtree used as tie breaker for trees of equal frequency
> > > > +      */
> > > > +
> > > > +-    uchf *l_buf;          /* buffer for literals or lengths */
> > > > ++    uchf *sym_buf;        /* buffer for distances and literals/lengths */
> > > > +
> > > > +     uInt  lit_bufsize;
> > > > +     /* Size of match buffer for literals/lengths.  There are 4 reasons for
> > > > +@@ -239,13 +239,8 @@ typedef struct internal_state {
> > > > +      *   - I can't count above 4
> > > > +      */
> > > > +
> > > > +-    uInt last_lit;      /* running index in l_buf */
> > > > +-
> > > > +-    ushf *d_buf;
> > > > +-    /* Buffer for distances. To simplify the code, d_buf and l_buf have
> > > > +-     * the same number of elements. To use different lengths, an extra flag
> > > > +-     * array would be necessary.
> > > > +-     */
> > > > ++    uInt sym_next;      /* running index in sym_buf */
> > > > ++    uInt sym_end;       /* symbol table full when sym_next reaches this */
> > > > +
> > > > +     ulg opt_len;        /* bit length of current block with optimal trees */
> > > > +     ulg static_len;     /* bit length of current block with static trees */
> > > > +@@ -325,20 +320,22 @@ void ZLIB_INTERNAL _tr_stored_block OF((deflate_state *s, charf *buf,
> > > > +
> > > > + # define _tr_tally_lit(s, c, flush) \
> > > > +   { uch cc = (c); \
> > > > +-    s->d_buf[s->last_lit] = 0; \
> > > > +-    s->l_buf[s->last_lit++] = cc; \
> > > > ++    s->sym_buf[s->sym_next++] = 0; \
> > > > ++    s->sym_buf[s->sym_next++] = 0; \
> > > > ++    s->sym_buf[s->sym_next++] = cc; \
> > > > +     s->dyn_ltree[cc].Freq++; \
> > > > +-    flush = (s->last_lit == s->lit_bufsize-1); \
> > > > ++    flush = (s->sym_next == s->sym_end); \
> > > > +    }
> > > > + # define _tr_tally_dist(s, distance, length, flush) \
> > > > +   { uch len = (uch)(length); \
> > > > +     ush dist = (ush)(distance); \
> > > > +-    s->d_buf[s->last_lit] = dist; \
> > > > +-    s->l_buf[s->last_lit++] = len; \
> > > > ++    s->sym_buf[s->sym_next++] = dist; \
> > > > ++    s->sym_buf[s->sym_next++] = dist >> 8; \
> > > > ++    s->sym_buf[s->sym_next++] = len; \
> > > > +     dist--; \
> > > > +     s->dyn_ltree[_length_code[len]+LITERALS+1].Freq++; \
> > > > +     s->dyn_dtree[d_code(dist)].Freq++; \
> > > > +-    flush = (s->last_lit == s->lit_bufsize-1); \
> > > > ++    flush = (s->sym_next == s->sym_end); \
> > > > +   }
> > > > + #else
> > > > + # define _tr_tally_lit(s, c, flush) flush = _tr_tally(s, 0, c)
> > > > +diff --git a/trees.c b/trees.c
> > > > +index 4f4a65011..decaeb7c3 100644
> > > > +--- a/trees.c
> > > > ++++ b/trees.c
> > > > +@@ -416,7 +416,7 @@ local void init_block(s)
> > > > +
> > > > +     s->dyn_ltree[END_BLOCK].Freq = 1;
> > > > +     s->opt_len = s->static_len = 0L;
> > > > +-    s->last_lit = s->matches = 0;
> > > > ++    s->sym_next = s->matches = 0;
> > > > + }
> > > > +
> > > > + #define SMALLEST 1
> > > > +@@ -948,7 +948,7 @@ void ZLIB_INTERNAL _tr_flush_block(s, buf, stored_len, last)
> > > > +
> > > > +         Tracev((stderr, "\nopt %lu(%lu) stat %lu(%lu) stored %lu lit %u ",
> > > > +                 opt_lenb, s->opt_len, static_lenb, s->static_len, stored_len,
> > > > +-                s->last_lit));
> > > > ++                s->sym_next / 3));
> > > > +
> > > > +         if (static_lenb <= opt_lenb) opt_lenb = static_lenb;
> > > > +
> > > > +@@ -1017,8 +1017,9 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
> > > > +     unsigned dist;  /* distance of matched string */
> > > > +     unsigned lc;    /* match length-MIN_MATCH or unmatched char (if dist==0) */
> > > > + {
> > > > +-    s->d_buf[s->last_lit] = (ush)dist;
> > > > +-    s->l_buf[s->last_lit++] = (uch)lc;
> > > > ++    s->sym_buf[s->sym_next++] = dist;
> > > > ++    s->sym_buf[s->sym_next++] = dist >> 8;
> > > > ++    s->sym_buf[s->sym_next++] = lc;
> > > > +     if (dist == 0) {
> > > > +         /* lc is the unmatched char */
> > > > +         s->dyn_ltree[lc].Freq++;
> > > > +@@ -1033,30 +1034,7 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
> > > > +         s->dyn_ltree[_length_code[lc]+LITERALS+1].Freq++;
> > > > +         s->dyn_dtree[d_code(dist)].Freq++;
> > > > +     }
> > > > +-
> > > > +-#ifdef TRUNCATE_BLOCK
> > > > +-    /* Try to guess if it is profitable to stop the current block here */
> > > > +-    if ((s->last_lit & 0x1fff) == 0 && s->level > 2) {
> > > > +-        /* Compute an upper bound for the compressed length */
> > > > +-        ulg out_length = (ulg)s->last_lit*8L;
> > > > +-        ulg in_length = (ulg)((long)s->strstart - s->block_start);
> > > > +-        int dcode;
> > > > +-        for (dcode = 0; dcode < D_CODES; dcode++) {
> > > > +-            out_length += (ulg)s->dyn_dtree[dcode].Freq *
> > > > +-                (5L+extra_dbits[dcode]);
> > > > +-        }
> > > > +-        out_length >>= 3;
> > > > +-        Tracev((stderr,"\nlast_lit %u, in %ld, out ~%ld(%ld%%) ",
> > > > +-               s->last_lit, in_length, out_length,
> > > > +-               100L - out_length*100L/in_length));
> > > > +-        if (s->matches < s->last_lit/2 && out_length < in_length/2) return 1;
> > > > +-    }
> > > > +-#endif
> > > > +-    return (s->last_lit == s->lit_bufsize-1);
> > > > +-    /* We avoid equality with lit_bufsize because of wraparound at 64K
> > > > +-     * on 16 bit machines and because stored blocks are restricted to
> > > > +-     * 64K-1 bytes.
> > > > +-     */
> > > > ++    return (s->sym_next == s->sym_end);
> > > > + }
> > > > +
> > > > + /* ===========================================================================
> > > > +@@ -1069,13 +1047,14 @@ local void compress_block(s, ltree, dtree)
> > > > + {
> > > > +     unsigned dist;      /* distance of matched string */
> > > > +     int lc;             /* match length or unmatched char (if dist == 0) */
> > > > +-    unsigned lx = 0;    /* running index in l_buf */
> > > > ++    unsigned sx = 0;    /* running index in sym_buf */
> > > > +     unsigned code;      /* the code to send */
> > > > +     int extra;          /* number of extra bits to send */
> > > > +
> > > > +-    if (s->last_lit != 0) do {
> > > > +-        dist = s->d_buf[lx];
> > > > +-        lc = s->l_buf[lx++];
> > > > ++    if (s->sym_next != 0) do {
> > > > ++        dist = s->sym_buf[sx++] & 0xff;
> > > > ++        dist += (unsigned)(s->sym_buf[sx++] & 0xff) << 8;
> > > > ++        lc = s->sym_buf[sx++];
> > > > +         if (dist == 0) {
> > > > +             send_code(s, lc, ltree); /* send a literal byte */
> > > > +             Tracecv(isgraph(lc), (stderr," '%c' ", lc));
> > > > +@@ -1100,11 +1079,10 @@ local void compress_block(s, ltree, dtree)
> > > > +             }
> > > > +         } /* literal or match pair ? */
> > > > +
> > > > +-        /* Check that the overlay between pending_buf and d_buf+l_buf is ok: */
> > > > +-        Assert((uInt)(s->pending) < s->lit_bufsize + 2*lx,
> > > > +-               "pendingBuf overflow");
> > > > ++        /* Check that the overlay between pending_buf and sym_buf is ok: */
> > > > ++        Assert(s->pending < s->lit_bufsize + sx, "pendingBuf overflow");
> > > > +
> > > > +-    } while (lx < s->last_lit);
> > > > ++    } while (sx < s->sym_next);
> > > > +
> > > > +     send_code(s, END_BLOCK, ltree);
> > > > + }
> > > > diff --git a/meta/recipes-core/zlib/zlib_1.2.11.bb b/meta/recipes-core/zlib/zlib_1.2.11.bb
> > > > index ef9431ae475..bc42cd64e9d 100644
> > > > --- a/meta/recipes-core/zlib/zlib_1.2.11.bb
> > > > +++ b/meta/recipes-core/zlib/zlib_1.2.11.bb
> > > > @@ -8,6 +8,7 @@ LIC_FILES_CHKSUM = "file://zlib.h;beginline=6;endline=23;md5=5377232268e952e9ef6
> > > >
> > > >  SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/${BPN}/${PV}/${BPN}-${PV}.tar.xz \
> > > >             file://ldflags-tests.patch \
> > > > +           file://CVE-2018-25032.patch \
> > > >             file://run-ptest \
> > > >             "
> > > >  UPSTREAM_CHECK_URI = "http://zlib.net/"
> > > > --
> > > > 2.25.1
> > > >
> > > >
> > > > -=-=-=-=-=-=-=-=-=-=-=-
> > > > Links: You receive all messages sent to this group.
> > > > View/Reply Online (#163750): https://lists.openembedded.org/g/openembedded-core/message/163750
> > > > Mute This Topic: https://lists.openembedded.org/mt/90107518/3620601
> > > > Group Owner: openembedded-core+owner@lists.openembedded.org
> > > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> > > > -=-=-=-=-=-=-=-=-=-=-=-
> > > >


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-03-29 13:07 [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032 Ross Burton
  2022-03-29 19:49 ` [OE-core] " Steve Sakoman
@ 2022-04-05 19:04 ` Jeroen Hofstee
  2022-04-09 18:14   ` Steve Sakoman
  1 sibling, 1 reply; 38+ messages in thread
From: Jeroen Hofstee @ 2022-04-05 19:04 UTC (permalink / raw)
  To: ross, openembedded-core

Hello Ross,

On 3/29/22 15:07, Ross Burton via lists.openembedded.org wrote:
> Signed-off-by: Ross Burton <ross.burton@arm.com>
> ---
>   .../zlib/zlib/CVE-2018-25032.patch            | 347 ++++++++++++++++++
>   meta/recipes-core/zlib/zlib_1.2.11.bb         |   1 +
>   2 files changed, 348 insertions(+)
>   create mode 100644 meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
>
> diff --git a/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> new file mode 100644
> index 00000000000..5cb61836419
> --- /dev/null
> +++ b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> @@ -0,0 +1,347 @@
> +CVE: CVE-2018-25032
> +Upstream-Status: Backport
> +Signed-off-by: Ross Burton <ross.burton@arm.com>
> +
>
It seems there _might_ be another patch needed.

https://github.com/madler/zlib/issues/605
https://github.com/madler/zlib/commit/4346a16853e19b45787ce933666026903fb8f3f8.patch

I can't judge that though :(

Regards,

Jereon



^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-05 19:04 ` Jeroen Hofstee
@ 2022-04-09 18:14   ` Steve Sakoman
  2022-04-10 21:21     ` Jeroen Hofstee
  0 siblings, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-04-09 18:14 UTC (permalink / raw)
  To: jhofstee; +Cc: ross, openembedded-core

On Tue, Apr 5, 2022 at 9:05 AM Jeroen Hofstee via
lists.openembedded.org
<jhofstee=victronenergy.com@lists.openembedded.org> wrote:
>
> Hello Ross,
>
> On 3/29/22 15:07, Ross Burton via lists.openembedded.org wrote:
> > Signed-off-by: Ross Burton <ross.burton@arm.com>
> > ---
> >   .../zlib/zlib/CVE-2018-25032.patch            | 347 ++++++++++++++++++
> >   meta/recipes-core/zlib/zlib_1.2.11.bb         |   1 +
> >   2 files changed, 348 insertions(+)
> >   create mode 100644 meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> >
> > diff --git a/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > new file mode 100644
> > index 00000000000..5cb61836419
> > --- /dev/null
> > +++ b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
> > @@ -0,0 +1,347 @@
> > +CVE: CVE-2018-25032
> > +Upstream-Status: Backport
> > +Signed-off-by: Ross Burton <ross.burton@arm.com>
> > +
> >
> It seems there _might_ be another patch needed.
>
> https://github.com/madler/zlib/issues/605
> https://github.com/madler/zlib/commit/4346a16853e19b45787ce933666026903fb8f3f8.patch

I did a dunfell autobuilder run with the second patch added, but
unfortunately still get the same failures.

So until we fix those I can't take this CVE patch :-(

Steve

> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#164052): https://lists.openembedded.org/g/openembedded-core/message/164052
> Mute This Topic: https://lists.openembedded.org/mt/90107518/3620601
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-09 18:14   ` Steve Sakoman
@ 2022-04-10 21:21     ` Jeroen Hofstee
  0 siblings, 0 replies; 38+ messages in thread
From: Jeroen Hofstee @ 2022-04-10 21:21 UTC (permalink / raw)
  To: Steve Sakoman; +Cc: ross, openembedded-core

Hello Steve,

On 4/9/22 20:14, Steve Sakoman wrote:
> On Tue, Apr 5, 2022 at 9:05 AM Jeroen Hofstee via
> lists.openembedded.org
> <jhofstee=victronenergy.com@lists.openembedded.org> wrote:
>> Hello Ross,
>>
>> On 3/29/22 15:07, Ross Burton via lists.openembedded.org wrote:
>>> Signed-off-by: Ross Burton <ross.burton@arm.com>
>>> ---
>>>    .../zlib/zlib/CVE-2018-25032.patch            | 347 ++++++++++++++++++
>>>    meta/recipes-core/zlib/zlib_1.2.11.bb         |   1 +
>>>    2 files changed, 348 insertions(+)
>>>    create mode 100644 meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
>>>
>>> diff --git a/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
>>> new file mode 100644
>>> index 00000000000..5cb61836419
>>> --- /dev/null
>>> +++ b/meta/recipes-core/zlib/zlib/CVE-2018-25032.patch
>>> @@ -0,0 +1,347 @@
>>> +CVE: CVE-2018-25032
>>> +Upstream-Status: Backport
>>> +Signed-off-by: Ross Burton <ross.burton@arm.com>
>>> +
>>>
>> It seems there _might_ be another patch needed.
>>
>> https://github.com/madler/zlib/issues/605
>> https://github.com/madler/zlib/commit/4346a16853e19b45787ce933666026903fb8f3f8.patch
> I did a dunfell autobuilder run with the second patch added, but
> unfortunately still get the same failures.
>
> So until we fix those I can't take this CVE patch :-(
>
>

Sorry if it wasn't clear, I mentioned it because another patch might be 
needed to
properly fix the CVE. I can't jugde that, simply because I am not 
familiar with the
code in question. It won't magically solve the builder failure...

I tried to reproduce that failure, but I get:

Task do_testimage does not exist for target core-image-sato-sdk 
(/home/jeroen/software/venus/sources/openembedded-core/meta/recipes-sato/images/core-image-sato-sdk.bb:do_testimage). 
Close matches: 0:00:01
   do_image

With kind regards,

Jeroen




^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-04 15:22       ` Steve Sakoman
@ 2022-04-11 17:31         ` Ralph Siemsen
  2022-04-11 17:52           ` Steve Sakoman
  0 siblings, 1 reply; 38+ messages in thread
From: Ralph Siemsen @ 2022-04-11 17:31 UTC (permalink / raw)
  To: Steve Sakoman; +Cc: Ross Burton, Mittal, Anuj, openembedded-core

On Mon, Apr 4, 2022 at 11:22 AM Steve Sakoman <steve@sakoman.com> wrote:

> > Unfortunately this triggers other errors too :-(
> >
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/4987
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4949

Both of these seem to fail in apt.AptRepoTest.test_apt_install_from_repo.
This test tries to "cd /etc/apt", then append to sources.list, and
then "apt-get update".

However, in a locally built core-image-sato, I do not have "/etc/apt"
directory. I also do not have "apt-get" or "apt" commands.
I'm not sure if this is a recent change, or a mis-configuration on my part.

But it seems the test failure is not due to the zlib CVE fix, as I am
missing '/etc/apt' even without the zilb fix applied.

Regards,
Ralph


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-11 17:31         ` Ralph Siemsen
@ 2022-04-11 17:52           ` Steve Sakoman
  2022-04-11 18:17             ` Ralph Siemsen
  0 siblings, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-04-11 17:52 UTC (permalink / raw)
  To: Ralph Siemsen; +Cc: Ross Burton, Mittal, Anuj, openembedded-core

On Mon, Apr 11, 2022 at 7:31 AM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
>
> On Mon, Apr 4, 2022 at 11:22 AM Steve Sakoman <steve@sakoman.com> wrote:
>
> > > Unfortunately this triggers other errors too :-(
> > >
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/4987
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4949
>
> Both of these seem to fail in apt.AptRepoTest.test_apt_install_from_repo.
> This test tries to "cd /etc/apt", then append to sources.list, and
> then "apt-get update".
>
> However, in a locally built core-image-sato, I do not have "/etc/apt"
> directory. I also do not have "apt-get" or "apt" commands.
> I'm not sure if this is a recent change, or a mis-configuration on my part.

I see from irc that you've discovered INHERIT += "testimage".  I think
that might help with the above issue.

> But it seems the test failure is not due to the zlib CVE fix, as I am
> missing '/etc/apt' even without the zilb fix applied.

FWIW, I'm seeing the same error with the xz CVE fix patch too.

Steve


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-11 17:52           ` Steve Sakoman
@ 2022-04-11 18:17             ` Ralph Siemsen
  2022-04-11 18:58               ` Steve Sakoman
  0 siblings, 1 reply; 38+ messages in thread
From: Ralph Siemsen @ 2022-04-11 18:17 UTC (permalink / raw)
  To: Steve Sakoman; +Cc: Ross Burton, Mittal, Anuj, openembedded-core

On Mon, Apr 11, 2022 at 1:52 PM Steve Sakoman <steve@sakoman.com> wrote:

> I see from irc that you've discovered INHERIT += "testimage".  I think
> that might help with the above issue.

Indeed, I added that, which solved the missing do_testimage problem.
However even after rebuilding core-image sato, I do not have /etc/apt
nor apt in the image.

Looking at https://git.yoctoproject.org/poky/tree/meta/lib/oeqa/runtime/cases/apt.py?h=dunfell#n43
the apt test should be skipped. Seemingly relevant bits of "bitbake -e":

IMAGE_FEATURES=" debug-tweaks"
IMAGE_PKGTYPE="rpm"
INHERIT=" cve-check testimage poky-sanity uninative reproducible_build
package_rpm buildstats image-mklibs image-prelink debian devshell
sstate license remove-l
ibtool blacklist sanity"
TESTIMAGEDEPENDS=" cpio-native:do_populate_sysroot
dnf-native:do_populate_sysroot createrepo-c-native:do_populate_sysroot
  qemu-native:do_populate_sysroot
qemu-helper-native:do_populate_sysroot
qemu-helper-native:do_addto_recipe_sysroot"

> FWIW, I'm seeing the same error with the xz CVE fix patch too.

Interesting... and without those the tests are successful (with all
other current changes)?

Ralph


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-11 18:17             ` Ralph Siemsen
@ 2022-04-11 18:58               ` Steve Sakoman
  2022-04-11 22:30                 ` Ralph Siemsen
  0 siblings, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-04-11 18:58 UTC (permalink / raw)
  To: Ralph Siemsen; +Cc: Ross Burton, Mittal, Anuj, openembedded-core

On Mon, Apr 11, 2022 at 8:17 AM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
>
> On Mon, Apr 11, 2022 at 1:52 PM Steve Sakoman <steve@sakoman.com> wrote:
>
> > I see from irc that you've discovered INHERIT += "testimage".  I think
> > that might help with the above issue.
>
> Indeed, I added that, which solved the missing do_testimage problem.
> However even after rebuilding core-image sato, I do not have /etc/apt
> nor apt in the image.
>
> Looking at https://git.yoctoproject.org/poky/tree/meta/lib/oeqa/runtime/cases/apt.py?h=dunfell#n43
> the apt test should be skipped. Seemingly relevant bits of "bitbake -e":
>
> IMAGE_FEATURES=" debug-tweaks"
> IMAGE_PKGTYPE="rpm"
> INHERIT=" cve-check testimage poky-sanity uninative reproducible_build
> package_rpm buildstats image-mklibs image-prelink debian devshell
> sstate license remove-l
> ibtool blacklist sanity"
> TESTIMAGEDEPENDS=" cpio-native:do_populate_sysroot
> dnf-native:do_populate_sysroot createrepo-c-native:do_populate_sysroot
>   qemu-native:do_populate_sysroot
> qemu-helper-native:do_populate_sysroot
> qemu-helper-native:do_addto_recipe_sysroot"

I've been busy setting up my systems to start doing the kirkstone LTS
maintenance too, so I haven't yet dug into this. But I will try to
reproduce this locally in the next day or so.  Let me know if you make
any progress!

> > FWIW, I'm seeing the same error with the xz CVE fix patch too.
>
> Interesting... and without those the tests are successful (with all
> other current changes)?

Yes, either change will trigger the error.  Without the zlib or xz
patches all is fine.  Adding just one or the other will fail the
pkgman-deb-non-deb builds (and probably the pkgman-rpm-non-rpm builds
too, though I've been concentrating on the deb version to try to
isolate the offending patches)

Steve


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-11 18:58               ` Steve Sakoman
@ 2022-04-11 22:30                 ` Ralph Siemsen
  2022-04-11 22:42                   ` Steve Sakoman
  0 siblings, 1 reply; 38+ messages in thread
From: Ralph Siemsen @ 2022-04-11 22:30 UTC (permalink / raw)
  To: Steve Sakoman; +Cc: Ross Burton, Mittal, Anuj, openembedded-core

On Mon, Apr 11, 2022 at 2:58 PM Steve Sakoman <steve@sakoman.com> wrote:

> Let me know if you make any progress!

After a clean build, and messing about with VNC, I am now able to
'bitbake core-image-sato:do_testimage'.

> Yes, either change will trigger the error.  Without the zlib or xz
> patches all is fine.  Adding just one or the other will fail the
> pkgman-deb-non-deb builds (and probably the pkgman-rpm-non-rpm builds
> too, though I've been concentrating on the deb version to try to
> isolate the offending patches)

I have both gzip and xz patches applied:
    $ git log --oneline
    52269ab7e7 (HEAD -> dunfell) gzip: fix CVE-2022-1271
    e7fa7a2a2c xz: fix CVE-2022-1271
    38c55bd388 (origin/dunfell) tzdata: update to 2022a
    86285152bd python3-jinja2: Correct HOMEPAGE

With default PACKAGE_CLASSES = package_rpm
    RESULTS - dnf.DnfBasicTest.test_dnf_help: PASSED (8.77s)
    ...
    SUMMARY:
    core-image-sato () - Ran 64 tests in 242.039s
    core-image-sato - OK - All required tests passed (successes=35,
skipped=29, failures=0, errors=0)

With PACKAGE_CLASSES = package_deb
    RESULTS - apt.AptRepoTest.test_apt_install_from_repo: PASSED (43.98s)
    ...
    SUMMARY:
    core-image-sato () - Ran 64 tests in 57.334s
    core-image-sato - OK - All required tests passed (successes=15,
skipped=49, failures=0, errors=0)

So I do not seem to reproduce the error. Any ideas what else might
differ in the autobuilder versus my local?

Ralph


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-11 22:30                 ` Ralph Siemsen
@ 2022-04-11 22:42                   ` Steve Sakoman
  2022-04-12  1:33                     ` Ralph Siemsen
  0 siblings, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-04-11 22:42 UTC (permalink / raw)
  To: Ralph Siemsen; +Cc: Ross Burton, Mittal, Anuj, openembedded-core

On Mon, Apr 11, 2022 at 12:30 PM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
>
> On Mon, Apr 11, 2022 at 2:58 PM Steve Sakoman <steve@sakoman.com> wrote:
>
> > Let me know if you make any progress!
>
> After a clean build, and messing about with VNC, I am now able to
> 'bitbake core-image-sato:do_testimage'.
>
> > Yes, either change will trigger the error.  Without the zlib or xz
> > patches all is fine.  Adding just one or the other will fail the
> > pkgman-deb-non-deb builds (and probably the pkgman-rpm-non-rpm builds
> > too, though I've been concentrating on the deb version to try to
> > isolate the offending patches)
>
> I have both gzip and xz patches applied:
>     $ git log --oneline
>     52269ab7e7 (HEAD -> dunfell) gzip: fix CVE-2022-1271
>     e7fa7a2a2c xz: fix CVE-2022-1271
>     38c55bd388 (origin/dunfell) tzdata: update to 2022a
>     86285152bd python3-jinja2: Correct HOMEPAGE
>
> With default PACKAGE_CLASSES = package_rpm
>     RESULTS - dnf.DnfBasicTest.test_dnf_help: PASSED (8.77s)
>     ...
>     SUMMARY:
>     core-image-sato () - Ran 64 tests in 242.039s
>     core-image-sato - OK - All required tests passed (successes=35,
> skipped=29, failures=0, errors=0)
>
> With PACKAGE_CLASSES = package_deb
>     RESULTS - apt.AptRepoTest.test_apt_install_from_repo: PASSED (43.98s)
>     ...
>     SUMMARY:
>     core-image-sato () - Ran 64 tests in 57.334s
>     core-image-sato - OK - All required tests passed (successes=15,
> skipped=49, failures=0, errors=0)
>
> So I do not seem to reproduce the error. Any ideas what else might
> differ in the autobuilder versus my local?

Sigh, I just ran the same experiment here with the same results - all is fine.

The setup json for the autobuilder run is:

        "pkgman-deb-non-deb" : {
            "MACHINE" : "qemux86",
            "PACKAGE_CLASSES" : "package_deb",
            "BBTARGETS" : "core-image-sato core-image-sato-sdk
core-image-minimal core-image-minimal-dev
core-image-sato:do_populate_sdk",
            "SANITYTARGETS" : "core-image-minimal:do_testimage
core-image-sato:do_testimage core-image-sato-sdk:do_testimage
core-image-sato:do_testsdk"
        },

I was using qemux86-64 for my experiment, so I guess I need to redo it
now using the correct machine!

Steve


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-11 22:42                   ` Steve Sakoman
@ 2022-04-12  1:33                     ` Ralph Siemsen
  2022-04-12  2:12                       ` Steve Sakoman
  0 siblings, 1 reply; 38+ messages in thread
From: Ralph Siemsen @ 2022-04-12  1:33 UTC (permalink / raw)
  To: Steve Sakoman; +Cc: Ross Burton, Mittal, Anuj, openembedded-core

On Mon, Apr 11, 2022 at 6:43 PM Steve Sakoman <steve@sakoman.com> wrote:
>
> I was using qemux86-64 for my experiment, so I guess I need to redo it
> now using the correct machine!

I have rebuild with MACHINE=qemux86. Same result, all tests appear to pass:

RESULTS - apt.AptRepoTest.test_apt_install_from_repo: PASSED (44.80s)
...
SUMMARY:
core-image-sato () - Ran 64 tests in 61.570s
core-image-sato - OK - All required tests passed (successes=15,
skipped=49, failures=0, errors=0)

Ralph


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-12  1:33                     ` Ralph Siemsen
@ 2022-04-12  2:12                       ` Steve Sakoman
  2022-04-12 16:07                         ` Ralph Siemsen
  0 siblings, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-04-12  2:12 UTC (permalink / raw)
  To: Ralph Siemsen
  Cc: Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 851 bytes --]

On Mon, Apr 11, 2022, 3:33 PM Ralph Siemsen <ralph.siemsen@linaro.org>
wrote:

> On Mon, Apr 11, 2022 at 6:43 PM Steve Sakoman <steve@sakoman.com> wrote:
> >
> > I was using qemux86-64 for my experiment, so I guess I need to redo it
> > now using the correct machine!
>
> I have rebuild with MACHINE=qemux86. Same result, all tests appear to pass:
>
> RESULTS - apt.AptRepoTest.test_apt_install_from_repo: PASSED (44.80s)
> ...
> SUMMARY:
> core-image-sato () - Ran 64 tests in 61.570s
> core-image-sato - OK - All required tests passed (successes=15,
> skipped=49, failures=0, errors=0)
>

Same here. But I just realized something that is different from the
autobuilder.

What distro is your build machine? On the autobuilder it is using a non
debian distro for the debian build.

My build machine is Ubuntu, so that is a major difference!

Steve

>

[-- Attachment #2: Type: text/html, Size: 1631 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-12  2:12                       ` Steve Sakoman
@ 2022-04-12 16:07                         ` Ralph Siemsen
  2022-04-12 21:49                           ` Steve Sakoman
  0 siblings, 1 reply; 38+ messages in thread
From: Ralph Siemsen @ 2022-04-12 16:07 UTC (permalink / raw)
  To: Steve Sakoman
  Cc: Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Mon, Apr 11, 2022 at 10:12 PM Steve Sakoman <steve@sakoman.com> wrote:

> What distro is your build machine? On the autobuilder it is using a non debian distro for the debian build.
>
> My build machine is Ubuntu, so that is a major difference!

My build was also done on Ubuntu.

I have just completed a build on Fedora 35, and this time, I am seeing
the same failure as the autobuilder. I'll try debugging the problem
this afternoon.

Ralph


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-12 16:07                         ` Ralph Siemsen
@ 2022-04-12 21:49                           ` Steve Sakoman
  2022-04-13  1:21                             ` Ralph Siemsen
  0 siblings, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-04-12 21:49 UTC (permalink / raw)
  To: Ralph Siemsen
  Cc: Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Tue, Apr 12, 2022 at 6:07 AM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
>
> On Mon, Apr 11, 2022 at 10:12 PM Steve Sakoman <steve@sakoman.com> wrote:
>
> > What distro is your build machine? On the autobuilder it is using a non debian distro for the debian build.
> >
> > My build machine is Ubuntu, so that is a major difference!
>
> My build was also done on Ubuntu.
>
> I have just completed a build on Fedora 35, and this time, I am seeing
> the same failure as the autobuilder. I'll try debugging the problem
> this afternoon.

I added a debug option to the failing command and did another autobuilder run.

You can see the output here:

https://errors.yoctoproject.org/Errors/Details/654608/

And the patch that adds it is the HEAD here:

https://git.yoctoproject.org/poky-contrib/log/?h=stable/dunfell-nut

So at least now we have a bit more info on the failure.

I'm off to a dentist appointment now, but maybe you can take a look
while I am gone :-)

Steve


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-12 21:49                           ` Steve Sakoman
@ 2022-04-13  1:21                             ` Ralph Siemsen
  2022-04-13 15:31                               ` Steve Sakoman
       [not found]                               ` <16E57E79FD292EFA.13992@lists.openembedded.org>
  0 siblings, 2 replies; 38+ messages in thread
From: Ralph Siemsen @ 2022-04-13  1:21 UTC (permalink / raw)
  To: Steve Sakoman
  Cc: Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Tue, Apr 12, 2022 at 5:49 PM Steve Sakoman <steve@sakoman.com> wrote:

> I added a debug option to the failing command and did another autobuilder run.
>
> You can see the output here:
>
> https://errors.yoctoproject.org/Errors/Details/654608/

Okay, same error, "Hash Sum mismatch". And if I squint between all the
URL-encoding, I can see the md5/sha1/sha256/sha512sum values.

The "apt update" command is doing the following:
- fetch the file called "Release"
- fetch the file called "Packages.gz" --> error occurs here

Looking inside the Release file, it is plain text, and contains the
md5/sha1/sha256/sha512 sums of both Packages and Packages.gz (and also
the first two lines of Release).

Manually checking each of those sums reveals an inconsistency: all the
sha256 values inside Release are incorrect, while all the other
md1/sha1/sha512 values are correct.

And when we look at the URL-encoded debug info... the sha256 value is
the correct one for Packages.gz (as computed manually). However it
does not match the (incorrect) value within the Release file. Thus it
seems apt-get is justified when it complains about "Hash Sum
mismatch".

Going back to my Ubuntu system, and looking at the generated Release
file... all the checksums are correct, including the sha256sum.

So I am now looking into how Release file gets generated... as the
problem appears to be there... and it happens on Fedora but not
Ubuntu.

One additional point to add: on the same Fedora 35 system, I did a
full rebuild *without* with xz/gzip CVE fixes, and the apt failure
still occurs. To be certain, I nuked cache, sstate-cache and tmp (so
basically the entire build directory) and the rebuild took several
hours.

Ralph


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13  1:21                             ` Ralph Siemsen
@ 2022-04-13 15:31                               ` Steve Sakoman
       [not found]                               ` <16E57E79FD292EFA.13992@lists.openembedded.org>
  1 sibling, 0 replies; 38+ messages in thread
From: Steve Sakoman @ 2022-04-13 15:31 UTC (permalink / raw)
  To: Ralph Siemsen
  Cc: Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Tue, Apr 12, 2022 at 3:21 PM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
>
> On Tue, Apr 12, 2022 at 5:49 PM Steve Sakoman <steve@sakoman.com> wrote:
>
> > I added a debug option to the failing command and did another autobuilder run.
> >
> > You can see the output here:
> >
> > https://errors.yoctoproject.org/Errors/Details/654608/
>
> Okay, same error, "Hash Sum mismatch". And if I squint between all the
> URL-encoding, I can see the md5/sha1/sha256/sha512sum values.
>
> The "apt update" command is doing the following:
> - fetch the file called "Release"
> - fetch the file called "Packages.gz" --> error occurs here
>
> Looking inside the Release file, it is plain text, and contains the
> md5/sha1/sha256/sha512 sums of both Packages and Packages.gz (and also
> the first two lines of Release).
>
> Manually checking each of those sums reveals an inconsistency: all the
> sha256 values inside Release are incorrect, while all the other
> md1/sha1/sha512 values are correct.
>
> And when we look at the URL-encoded debug info... the sha256 value is
> the correct one for Packages.gz (as computed manually). However it
> does not match the (incorrect) value within the Release file. Thus it
> seems apt-get is justified when it complains about "Hash Sum
> mismatch".
>
> Going back to my Ubuntu system, and looking at the generated Release
> file... all the checksums are correct, including the sha256sum.
>
> So I am now looking into how Release file gets generated... as the
> problem appears to be there... and it happens on Fedora but not
> Ubuntu.

As far as I can tell it is done here:

https://git.yoctoproject.org/poky/tree/meta/lib/oe/package_manager.py?h=dunfell#n301

> One additional point to add: on the same Fedora 35 system, I did a
> full rebuild *without* with xz/gzip CVE fixes, and the apt failure
> still occurs. To be certain, I nuked cache, sstate-cache and tmp (so
> basically the entire build directory) and the rebuild took several
> hours.

Now that is really strange!  In my experience it has only appeared
after adding the zlib or xz CVE fix patches.

I just started two runs on the autobuilder, with the zlib patch as the
only difference.  Both on Fedora 35.

Let's see if I can confirm what you are seeing!  I don't have a local
Fedora 35 machine, so I need to rely on the autobuilder.

Steve


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
       [not found]                               ` <16E57E79FD292EFA.13992@lists.openembedded.org>
@ 2022-04-13 16:02                                 ` Steve Sakoman
  2022-04-13 16:41                                   ` Mike Crowe
                                                     ` (2 more replies)
  0 siblings, 3 replies; 38+ messages in thread
From: Steve Sakoman @ 2022-04-13 16:02 UTC (permalink / raw)
  To: steve
  Cc: Ralph Siemsen, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, Apr 13, 2022 at 5:31 AM Steve Sakoman via
lists.openembedded.org <steve=sakoman.com@lists.openembedded.org>
wrote:
>
> On Tue, Apr 12, 2022 at 3:21 PM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
> >
> > On Tue, Apr 12, 2022 at 5:49 PM Steve Sakoman <steve@sakoman.com> wrote:
> >
> > > I added a debug option to the failing command and did another autobuilder run.
> > >
> > > You can see the output here:
> > >
> > > https://errors.yoctoproject.org/Errors/Details/654608/
> >
> > Okay, same error, "Hash Sum mismatch". And if I squint between all the
> > URL-encoding, I can see the md5/sha1/sha256/sha512sum values.
> >
> > The "apt update" command is doing the following:
> > - fetch the file called "Release"
> > - fetch the file called "Packages.gz" --> error occurs here
> >
> > Looking inside the Release file, it is plain text, and contains the
> > md5/sha1/sha256/sha512 sums of both Packages and Packages.gz (and also
> > the first two lines of Release).
> >
> > Manually checking each of those sums reveals an inconsistency: all the
> > sha256 values inside Release are incorrect, while all the other
> > md1/sha1/sha512 values are correct.
> >
> > And when we look at the URL-encoded debug info... the sha256 value is
> > the correct one for Packages.gz (as computed manually). However it
> > does not match the (incorrect) value within the Release file. Thus it
> > seems apt-get is justified when it complains about "Hash Sum
> > mismatch".
> >
> > Going back to my Ubuntu system, and looking at the generated Release
> > file... all the checksums are correct, including the sha256sum.
> >
> > So I am now looking into how Release file gets generated... as the
> > problem appears to be there... and it happens on Fedora but not
> > Ubuntu.
>
> As far as I can tell it is done here:
>
> https://git.yoctoproject.org/poky/tree/meta/lib/oe/package_manager.py?h=dunfell#n301
>
> > One additional point to add: on the same Fedora 35 system, I did a
> > full rebuild *without* with xz/gzip CVE fixes, and the apt failure
> > still occurs. To be certain, I nuked cache, sstate-cache and tmp (so
> > basically the entire build directory) and the rebuild took several
> > hours.
>
> Now that is really strange!  In my experience it has only appeared
> after adding the zlib or xz CVE fix patches.
>
> I just started two runs on the autobuilder, with the zlib patch as the
> only difference.  Both on Fedora 35.

Both runs completed and I'm still seeing success without the zlib patch:

https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5069

and failure with the patch:

https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5070

Steve


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13 16:02                                 ` Steve Sakoman
@ 2022-04-13 16:41                                   ` Mike Crowe
  2022-04-13 17:37                                     ` Steve Sakoman
  2022-04-13 18:02                                     ` Ralph Siemsen
  2022-04-13 17:11                                   ` Jose Quaresma
       [not found]                                   ` <16E583EB139C493B.16998@lists.openembedded.org>
  2 siblings, 2 replies; 38+ messages in thread
From: Mike Crowe @ 2022-04-13 16:41 UTC (permalink / raw)
  To: Steve Sakoman
  Cc: Ralph Siemsen, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wednesday 13 April 2022 at 06:02:22 -1000, Steve Sakoman wrote:
> Both runs completed and I'm still seeing success without the zlib patch:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5069
> 
> and failure with the patch:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5070

I'm certainly no expert with the autobuilder, but it looks like nothing was
actually compiled for both of those builds - everything came from the
sstate cache.

I believe that Ralph's reproduction of the test failure without the zlib
patch was from a complete rebuild without anything coming from the sstate
cache.

I suspect that if a PR bump or something similar that causes zlib and all
its reverse dependencies to be built were tested on top of the commit used
for build 5069 then the test failure would occur then as well and
exonerate the zlib patch.

Mike.


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13 16:02                                 ` Steve Sakoman
  2022-04-13 16:41                                   ` Mike Crowe
@ 2022-04-13 17:11                                   ` Jose Quaresma
       [not found]                                   ` <16E583EB139C493B.16998@lists.openembedded.org>
  2 siblings, 0 replies; 38+ messages in thread
From: Jose Quaresma @ 2022-04-13 17:11 UTC (permalink / raw)
  To: Steve Sakoman
  Cc: Ralph Siemsen, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 5275 bytes --]

Hi,

Steve Sakoman <steve@sakoman.com> escreveu no dia quarta, 13/04/2022 à(s)
17:02:

> On Wed, Apr 13, 2022 at 5:31 AM Steve Sakoman via
> lists.openembedded.org <steve=sakoman.com@lists.openembedded.org>
> wrote:
> >
> > On Tue, Apr 12, 2022 at 3:21 PM Ralph Siemsen <ralph.siemsen@linaro.org>
> wrote:
> > >
> > > On Tue, Apr 12, 2022 at 5:49 PM Steve Sakoman <steve@sakoman.com>
> wrote:
> > >
> > > > I added a debug option to the failing command and did another
> autobuilder run.
> > > >
> > > > You can see the output here:
> > > >
> > > > https://errors.yoctoproject.org/Errors/Details/654608/
> > >
> > > Okay, same error, "Hash Sum mismatch". And if I squint between all the
> > > URL-encoding, I can see the md5/sha1/sha256/sha512sum values.
> > >
> > > The "apt update" command is doing the following:
> > > - fetch the file called "Release"
> > > - fetch the file called "Packages.gz" --> error occurs here
> > >
> > > Looking inside the Release file, it is plain text, and contains the
> > > md5/sha1/sha256/sha512 sums of both Packages and Packages.gz (and also
> > > the first two lines of Release).
> > >
> > > Manually checking each of those sums reveals an inconsistency: all the
> > > sha256 values inside Release are incorrect, while all the other
> > > md1/sha1/sha512 values are correct.
> > >
> > > And when we look at the URL-encoded debug info... the sha256 value is
> > > the correct one for Packages.gz (as computed manually). However it
> > > does not match the (incorrect) value within the Release file. Thus it
> > > seems apt-get is justified when it complains about "Hash Sum
> > > mismatch".
> > >
> > > Going back to my Ubuntu system, and looking at the generated Release
> > > file... all the checksums are correct, including the sha256sum.
> > >
> > > So I am now looking into how Release file gets generated... as the
> > > problem appears to be there... and it happens on Fedora but not
> > > Ubuntu.
> >
> > As far as I can tell it is done here:
> >
> >
> https://git.yoctoproject.org/poky/tree/meta/lib/oe/package_manager.py?h=dunfell#n301
> >
> > > One additional point to add: on the same Fedora 35 system, I did a
> > > full rebuild *without* with xz/gzip CVE fixes, and the apt failure
> > > still occurs. To be certain, I nuked cache, sstate-cache and tmp (so
> > > basically the entire build directory) and the rebuild took several
> > > hours.
> >
> > Now that is really strange!  In my experience it has only appeared
> > after adding the zlib or xz CVE fix patches.
> >
> > I just started two runs on the autobuilder, with the zlib patch as the
> > only difference.  Both on Fedora 35.
>
> Both runs completed and I'm still seeing success without the zlib patch:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5069
>
> and failure with the patch:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5070


It seems the test that failed is something related with the apt.
Is this repo hosted on 192.168.7.5 shared between master and dunfell
branches?
I ask this because there are some issues with apt [1] on master and it can
be related to this.

[1] apt: add apt selftest to test signed package feeds)

Started HTTPService on 0.0.0.0:35637
Traceback (most recent call last):
File
"/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/core/decorator/__init__.py",
line 36, in wrapped_f
return func(*args, **kwargs)
File
"/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/core/decorator/__init__.py",
line 36, in wrapped_f
return func(*args, **kwargs)
File
"/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/core/decorator/__init__.py",
line 36, in wrapped_f
return func(*args, **kwargs)
File
"/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/runtime/cases/apt.py",
line 50, in test_apt_install_from_repo
self.pkg('update')
File
"/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/runtime/cases/apt.py",
line 17, in pkg
self.assertEqual(status, expected, message)
AssertionError: 100 != 0 : apt-get update
Ign:1 http://192.168.7.5:42261 ./ InRelease
Get:2 http://192.168.7.5:42261 ./ Release [1213 B]
Ign:3 http://192.168.7.5:42261 ./ Release.gpg
Get:4 http://192.168.7.5:42261 ./ Packages [59.3 kB]
Err:4 http://192.168.7.5:42261 ./ Packages
Hash Sum mismatch
Fetched 60.5 kB in 20s (3020 B/s)
Reading package lists...
W: The repository 'http://192.168.7.5:42261 ./ Release' is not signed.
E: Failed to fetch http://192.168.7.5:42261/./Packages.gz Hash Sum mismatch
E: Some index files failed to download. They have been ignored, or old ones
used instead.

Jose


>
> Steve
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#164339):
> https://lists.openembedded.org/g/openembedded-core/message/164339
> Mute This Topic: https://lists.openembedded.org/mt/90107518/5052612
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
Best regards,

José Quaresma

[-- Attachment #2: Type: text/html, Size: 16336 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13 16:41                                   ` Mike Crowe
@ 2022-04-13 17:37                                     ` Steve Sakoman
  2022-04-13 17:50                                       ` Jose Quaresma
  2022-04-13 19:16                                       ` Steve Sakoman
  2022-04-13 18:02                                     ` Ralph Siemsen
  1 sibling, 2 replies; 38+ messages in thread
From: Steve Sakoman @ 2022-04-13 17:37 UTC (permalink / raw)
  To: Mike Crowe
  Cc: Ralph Siemsen, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, Apr 13, 2022 at 6:41 AM Mike Crowe <mac@mcrowe.com> wrote:
>
> On Wednesday 13 April 2022 at 06:02:22 -1000, Steve Sakoman wrote:
> > Both runs completed and I'm still seeing success without the zlib patch:
> >
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5069
> >
> > and failure with the patch:
> >
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5070
>
> I'm certainly no expert with the autobuilder, but it looks like nothing was
> actually compiled for both of those builds - everything came from the
> sstate cache.
>
> I believe that Ralph's reproduction of the test failure without the zlib
> patch was from a complete rebuild without anything coming from the sstate
> cache.
>
> I suspect that if a PR bump or something similar that causes zlib and all
> its reverse dependencies to be built were tested on top of the commit used
> for build 5069 then the test failure would occur then as well and
> exonerate the zlib patch.

A valid point, let's see what happens with a PR bump:

https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5072

I see plenty of rebuilds in process . . .

Steve


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
       [not found]                                   ` <16E583EB139C493B.16998@lists.openembedded.org>
@ 2022-04-13 17:41                                     ` Jose Quaresma
  0 siblings, 0 replies; 38+ messages in thread
From: Jose Quaresma @ 2022-04-13 17:41 UTC (permalink / raw)
  To: Jose Quaresma
  Cc: Steve Sakoman, Ralph Siemsen, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 5800 bytes --]

Jose Quaresma via lists.openembedded.org <quaresma.jose=
gmail.com@lists.openembedded.org> escreveu no dia quarta, 13/04/2022 à(s)
18:11:

> Hi,
>
> Steve Sakoman <steve@sakoman.com> escreveu no dia quarta, 13/04/2022 à(s)
> 17:02:
>
>> On Wed, Apr 13, 2022 at 5:31 AM Steve Sakoman via
>> lists.openembedded.org <steve=sakoman.com@lists.openembedded.org>
>> wrote:
>> >
>> > On Tue, Apr 12, 2022 at 3:21 PM Ralph Siemsen <ralph.siemsen@linaro.org>
>> wrote:
>> > >
>> > > On Tue, Apr 12, 2022 at 5:49 PM Steve Sakoman <steve@sakoman.com>
>> wrote:
>> > >
>> > > > I added a debug option to the failing command and did another
>> autobuilder run.
>> > > >
>> > > > You can see the output here:
>> > > >
>> > > > https://errors.yoctoproject.org/Errors/Details/654608/
>> > >
>> > > Okay, same error, "Hash Sum mismatch". And if I squint between all the
>> > > URL-encoding, I can see the md5/sha1/sha256/sha512sum values.
>> > >
>> > > The "apt update" command is doing the following:
>> > > - fetch the file called "Release"
>> > > - fetch the file called "Packages.gz" --> error occurs here
>> > >
>> > > Looking inside the Release file, it is plain text, and contains the
>> > > md5/sha1/sha256/sha512 sums of both Packages and Packages.gz (and also
>> > > the first two lines of Release).
>> > >
>> > > Manually checking each of those sums reveals an inconsistency: all the
>> > > sha256 values inside Release are incorrect, while all the other
>> > > md1/sha1/sha512 values are correct.
>> > >
>> > > And when we look at the URL-encoded debug info... the sha256 value is
>> > > the correct one for Packages.gz (as computed manually). However it
>> > > does not match the (incorrect) value within the Release file. Thus it
>> > > seems apt-get is justified when it complains about "Hash Sum
>> > > mismatch".
>> > >
>> > > Going back to my Ubuntu system, and looking at the generated Release
>> > > file... all the checksums are correct, including the sha256sum.
>> > >
>> > > So I am now looking into how Release file gets generated... as the
>> > > problem appears to be there... and it happens on Fedora but not
>> > > Ubuntu.
>> >
>> > As far as I can tell it is done here:
>> >
>> >
>> https://git.yoctoproject.org/poky/tree/meta/lib/oe/package_manager.py?h=dunfell#n301
>> >
>> > > One additional point to add: on the same Fedora 35 system, I did a
>> > > full rebuild *without* with xz/gzip CVE fixes, and the apt failure
>> > > still occurs. To be certain, I nuked cache, sstate-cache and tmp (so
>> > > basically the entire build directory) and the rebuild took several
>> > > hours.
>> >
>> > Now that is really strange!  In my experience it has only appeared
>> > after adding the zlib or xz CVE fix patches.
>> >
>> > I just started two runs on the autobuilder, with the zlib patch as the
>> > only difference.  Both on Fedora 35.
>>
>> Both runs completed and I'm still seeing success without the zlib patch:
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5069
>>
>> and failure with the patch:
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5070
>
>
> It seems the test that failed is something related with the apt.
> Is this repo hosted on 192.168.7.5 shared between master and dunfell
> branches?
> I ask this because there are some issues with apt [1] on master and it can
> be related to this.
>

The server is started in the test.
Sorry for the noise and please discard my comment.

Started HTTPService on 0.0.0.0:42261

Jose


>
> [1] apt: add apt selftest to test signed package feeds)
>
> Started HTTPService on 0.0.0.0:35637
> Traceback (most recent call last):
> File
> "/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/core/decorator/__init__.py",
> line 36, in wrapped_f
> return func(*args, **kwargs)
> File
> "/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/core/decorator/__init__.py",
> line 36, in wrapped_f
> return func(*args, **kwargs)
> File
> "/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/core/decorator/__init__.py",
> line 36, in wrapped_f
> return func(*args, **kwargs)
> File
> "/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/runtime/cases/apt.py",
> line 50, in test_apt_install_from_repo
> self.pkg('update')
> File
> "/home/pokybuild/yocto-worker/pkgman-deb-non-deb/build/meta/lib/oeqa/runtime/cases/apt.py",
> line 17, in pkg
> self.assertEqual(status, expected, message)
> AssertionError: 100 != 0 : apt-get update
> Ign:1 http://192.168.7.5:42261 ./ InRelease
> Get:2 http://192.168.7.5:42261 ./ Release [1213 B]
> Ign:3 http://192.168.7.5:42261 ./ Release.gpg
> Get:4 http://192.168.7.5:42261 ./ Packages [59.3 kB]
> Err:4 http://192.168.7.5:42261 ./ Packages
> Hash Sum mismatch
> Fetched 60.5 kB in 20s (3020 B/s)
> Reading package lists...
> W: The repository 'http://192.168.7.5:42261 ./ Release' is not signed.
> E: Failed to fetch http://192.168.7.5:42261/./Packages.gz Hash Sum
> mismatch
> E: Some index files failed to download. They have been ignored, or old
> ones used instead.
>
> Jose
>
>
>>
>> Steve
>>
>>
>>
>>
>
> --
> Best regards,
>
> José Quaresma
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#164347):
> https://lists.openembedded.org/g/openembedded-core/message/164347
> Mute This Topic: https://lists.openembedded.org/mt/90107518/5052612
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
Best regards,

José Quaresma

[-- Attachment #2: Type: text/html, Size: 16204 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13 17:37                                     ` Steve Sakoman
@ 2022-04-13 17:50                                       ` Jose Quaresma
  2022-04-13 19:16                                       ` Steve Sakoman
  1 sibling, 0 replies; 38+ messages in thread
From: Jose Quaresma @ 2022-04-13 17:50 UTC (permalink / raw)
  To: Steve Sakoman
  Cc: Mike Crowe, Ralph Siemsen, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 2078 bytes --]

Hi Steve,

Steve Sakoman <steve@sakoman.com> escreveu no dia quarta, 13/04/2022 à(s)
18:37:

> On Wed, Apr 13, 2022 at 6:41 AM Mike Crowe <mac@mcrowe.com> wrote:
> >
> > On Wednesday 13 April 2022 at 06:02:22 -1000, Steve Sakoman wrote:
> > > Both runs completed and I'm still seeing success without the zlib
> patch:
> > >
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5069
> > >
> > > and failure with the patch:
> > >
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5070
> >
> > I'm certainly no expert with the autobuilder, but it looks like nothing
> was
> > actually compiled for both of those builds - everything came from the
> > sstate cache.
> >
> > I believe that Ralph's reproduction of the test failure without the zlib
> > patch was from a complete rebuild without anything coming from the sstate
> > cache.
> >
> > I suspect that if a PR bump or something similar that causes zlib and all
> > its reverse dependencies to be built were tested on top of the commit
> used
> > for build 5069 then the test failure would occur then as well and
> > exonerate the zlib patch.
>
> A valid point, let's see what happens with a PR bump:
>

I think that bumping the PR is not enough to rebuild a package.
The sstate cache needs to be invalidated and bumped as well
with HASHEQUIV_HASH_VERSION.

PR = "r1"
HASHEQUIV_HASH_VERSION .= ".1"

Jose


>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5072
>
> I see plenty of rebuilds in process . . .
>
> Steve
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#164349):
> https://lists.openembedded.org/g/openembedded-core/message/164349
> Mute This Topic: https://lists.openembedded.org/mt/90107518/5052612
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
Best regards,

José Quaresma

[-- Attachment #2: Type: text/html, Size: 3759 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13 16:41                                   ` Mike Crowe
  2022-04-13 17:37                                     ` Steve Sakoman
@ 2022-04-13 18:02                                     ` Ralph Siemsen
  2022-04-13 18:19                                       ` Steve Sakoman
  1 sibling, 1 reply; 38+ messages in thread
From: Ralph Siemsen @ 2022-04-13 18:02 UTC (permalink / raw)
  To: Mike Crowe
  Cc: Steve Sakoman, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, Apr 13, 2022 at 12:41 PM Mike Crowe <mac@mcrowe.com> wrote:
>
> I believe that Ralph's reproduction of the test failure without the zlib
> patch was from a complete rebuild without anything coming from the sstate
> cache.

Yes, just confirming the above. I wanted to be certain there was
nothing left-over from a previous build, so I nuked everything except
for downloads and my local.conf.

I am quite certain the problem is with apt-ftparchive command, which
is what generates the md5/sha sums in the Release file. It seems to
behave differently under Fedora versus Ubuntu. Only the SHA256 is
affected. It is worth noting that apt includes its own sha calculation
code, they are not calling out to "sha256sum" or using an existing
library.

Was busy with some other things this AM but am going to dive into
apt-ftparchive source code next (which is C++, ugh...).

Ralph


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13 18:02                                     ` Ralph Siemsen
@ 2022-04-13 18:19                                       ` Steve Sakoman
  2022-04-13 19:05                                         ` Ralph Siemsen
  0 siblings, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-04-13 18:19 UTC (permalink / raw)
  To: Ralph Siemsen
  Cc: Mike Crowe, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, Apr 13, 2022 at 8:02 AM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
>
> On Wed, Apr 13, 2022 at 12:41 PM Mike Crowe <mac@mcrowe.com> wrote:
> >
> > I believe that Ralph's reproduction of the test failure without the zlib
> > patch was from a complete rebuild without anything coming from the sstate
> > cache.
>
> Yes, just confirming the above. I wanted to be certain there was
> nothing left-over from a previous build, so I nuked everything except
> for downloads and my local.conf.
>
> I am quite certain the problem is with apt-ftparchive command, which
> is what generates the md5/sha sums in the Release file. It seems to
> behave differently under Fedora versus Ubuntu. Only the SHA256 is
> affected. It is worth noting that apt includes its own sha calculation
> code, they are not calling out to "sha256sum" or using an existing
> library.

Yes, and it appears they had a quite similar bug in the past!

https://lists.debian.org/deity/2006/07/msg00074.html

Bug#378183: apt: All SHA256 hashes generated/used by APT are wrong"

Steve


> Was busy with some other things this AM but am going to dive into
> apt-ftparchive source code next (which is C++, ugh...).
>
> Ralph


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13 18:19                                       ` Steve Sakoman
@ 2022-04-13 19:05                                         ` Ralph Siemsen
  2022-04-13 21:39                                           ` Steve Sakoman
  0 siblings, 1 reply; 38+ messages in thread
From: Ralph Siemsen @ 2022-04-13 19:05 UTC (permalink / raw)
  To: Steve Sakoman
  Cc: Mike Crowe, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, Apr 13, 2022 at 2:19 PM Steve Sakoman <steve@sakoman.com> wrote:

> Yes, and it appears they had a quite similar bug in the past!
>
> https://lists.debian.org/deity/2006/07/msg00074.html

Interesting find... though it seems that is in a different sha256
implementation than the one being used here ("Gifford").

The other factor is that it behaves on ubuntu, but not fedora. Do the
native packages get built with the host gcc (11.2 on fedora 35,
whereas 9.3 on ubuntu 20.x)? Or does yocto also build a native
toolchain to build host packages with?

Ralph


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13 17:37                                     ` Steve Sakoman
  2022-04-13 17:50                                       ` Jose Quaresma
@ 2022-04-13 19:16                                       ` Steve Sakoman
  1 sibling, 0 replies; 38+ messages in thread
From: Steve Sakoman @ 2022-04-13 19:16 UTC (permalink / raw)
  To: Mike Crowe
  Cc: Ralph Siemsen, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, Apr 13, 2022 at 7:37 AM Steve Sakoman <steve@sakoman.com> wrote:
>
> On Wed, Apr 13, 2022 at 6:41 AM Mike Crowe <mac@mcrowe.com> wrote:
> >
> > On Wednesday 13 April 2022 at 06:02:22 -1000, Steve Sakoman wrote:
> > > Both runs completed and I'm still seeing success without the zlib patch:
> > >
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5069
> > >
> > > and failure with the patch:
> > >
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5070
> >
> > I'm certainly no expert with the autobuilder, but it looks like nothing was
> > actually compiled for both of those builds - everything came from the
> > sstate cache.
> >
> > I believe that Ralph's reproduction of the test failure without the zlib
> > patch was from a complete rebuild without anything coming from the sstate
> > cache.
> >
> > I suspect that if a PR bump or something similar that causes zlib and all
> > its reverse dependencies to be built were tested on top of the commit used
> > for build 5069 then the test failure would occur then as well and
> > exonerate the zlib patch.
>
> A valid point, let's see what happens with a PR bump:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5072
>
> I see plenty of rebuilds in process . . .

As you predicted, rebuilding zlib (and all dependencies) with a PR
bump did indeed result in the same failure, exonerating the zlib CVE
patch.

So it really does appear that we are chasing a bug in the native
apt-ftparchive command on fedora-35 (and likely alma-8 since I've seen
the error there too)

Steve


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13 19:05                                         ` Ralph Siemsen
@ 2022-04-13 21:39                                           ` Steve Sakoman
  2022-04-13 21:41                                             ` Richard Purdie
  0 siblings, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-04-13 21:39 UTC (permalink / raw)
  To: Ralph Siemsen, Richard Purdie
  Cc: Mike Crowe, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, Apr 13, 2022 at 9:05 AM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
>
> On Wed, Apr 13, 2022 at 2:19 PM Steve Sakoman <steve@sakoman.com> wrote:
>
> > Yes, and it appears they had a quite similar bug in the past!
> >
> > https://lists.debian.org/deity/2006/07/msg00074.html
>
> Interesting find... though it seems that is in a different sha256
> implementation than the one being used here ("Gifford").

I did another experiment, where I disabled generation of the sha256
entries in Release (by adding --no-sha256 to the apt-ftparchive
command)

As a result we get past this first hash mismatch in Release, but then
get later hash mismatches when it tries to download .debs.

https://errors.yoctoproject.org/Errors/Details/654717/

So it really does seem that apt sha256 generation is broken on
non-debian distros.

> The other factor is that it behaves on ubuntu, but not fedora. Do the
> native packages get built with the host gcc (11.2 on fedora 35,
> whereas 9.3 on ubuntu 20.x)? Or does yocto also build a native
> toolchain to build host packages with?

Perhaps Richard can answer this question!

Steve


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13 21:39                                           ` Steve Sakoman
@ 2022-04-13 21:41                                             ` Richard Purdie
  2022-04-13 21:44                                               ` Steve Sakoman
  0 siblings, 1 reply; 38+ messages in thread
From: Richard Purdie @ 2022-04-13 21:41 UTC (permalink / raw)
  To: Steve Sakoman, Ralph Siemsen
  Cc: Mike Crowe, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, 2022-04-13 at 11:39 -1000, Steve Sakoman wrote:
> On Wed, Apr 13, 2022 at 9:05 AM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
> > 
> > On Wed, Apr 13, 2022 at 2:19 PM Steve Sakoman <steve@sakoman.com> wrote:
> > 
> > > Yes, and it appears they had a quite similar bug in the past!
> > > 
> > > https://lists.debian.org/deity/2006/07/msg00074.html
> > 
> > Interesting find... though it seems that is in a different sha256
> > implementation than the one being used here ("Gifford").
> 
> I did another experiment, where I disabled generation of the sha256
> entries in Release (by adding --no-sha256 to the apt-ftparchive
> command)
> 
> As a result we get past this first hash mismatch in Release, but then
> get later hash mismatches when it tries to download .debs.
> 
> https://errors.yoctoproject.org/Errors/Details/654717/
> 
> So it really does seem that apt sha256 generation is broken on
> non-debian distros.
> 
> > The other factor is that it behaves on ubuntu, but not fedora. Do the
> > native packages get built with the host gcc (11.2 on fedora 35,
> > whereas 9.3 on ubuntu 20.x)? Or does yocto also build a native
> > toolchain to build host packages with?
> 
> Perhaps Richard can answer this question!

Native recipes are built using the host gcc so this could be host gcc dependent.
We do use buildtools-tarball on some hosts where the gcc is too old. You can see
what is used where in config.json in autobuilder-helper near the end.

Cheers,

Richard



^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13 21:41                                             ` Richard Purdie
@ 2022-04-13 21:44                                               ` Steve Sakoman
  2022-04-14  2:47                                                 ` Ralph Siemsen
  0 siblings, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-04-13 21:44 UTC (permalink / raw)
  To: Richard Purdie
  Cc: Ralph Siemsen, Mike Crowe, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, Apr 13, 2022 at 11:41 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Wed, 2022-04-13 at 11:39 -1000, Steve Sakoman wrote:
> > On Wed, Apr 13, 2022 at 9:05 AM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
> > >
> > > On Wed, Apr 13, 2022 at 2:19 PM Steve Sakoman <steve@sakoman.com> wrote:
> > >
> > > > Yes, and it appears they had a quite similar bug in the past!
> > > >
> > > > https://lists.debian.org/deity/2006/07/msg00074.html
> > >
> > > Interesting find... though it seems that is in a different sha256
> > > implementation than the one being used here ("Gifford").
> >
> > I did another experiment, where I disabled generation of the sha256
> > entries in Release (by adding --no-sha256 to the apt-ftparchive
> > command)
> >
> > As a result we get past this first hash mismatch in Release, but then
> > get later hash mismatches when it tries to download .debs.
> >
> > https://errors.yoctoproject.org/Errors/Details/654717/
> >
> > So it really does seem that apt sha256 generation is broken on
> > non-debian distros.
> >
> > > The other factor is that it behaves on ubuntu, but not fedora. Do the
> > > native packages get built with the host gcc (11.2 on fedora 35,
> > > whereas 9.3 on ubuntu 20.x)? Or does yocto also build a native
> > > toolchain to build host packages with?
> >
> > Perhaps Richard can answer this question!
>
> Native recipes are built using the host gcc so this could be host gcc dependent.
> We do use buildtools-tarball on some hosts where the gcc is too old. You can see
> what is used where in config.json in autobuilder-helper near the end.

The issue is happening on Fedora 35 and Alma 8, so no
buildtools-tarball in this case!

Steve


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-13 21:44                                               ` Steve Sakoman
@ 2022-04-14  2:47                                                 ` Ralph Siemsen
  2022-04-14  3:00                                                   ` Steve Sakoman
       [not found]                                                   ` <16E5A41A6E4FF34A.8845@lists.openembedded.org>
  0 siblings, 2 replies; 38+ messages in thread
From: Ralph Siemsen @ 2022-04-14  2:47 UTC (permalink / raw)
  To: Steve Sakoman
  Cc: Richard Purdie, Mike Crowe, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, 2022-04-13 at 11:39 -1000, Steve Sakoman wrote:
> I did another experiment, where I disabled generation of the sha256
> entries in Release (by adding --no-sha256 to the apt-ftparchive
> command)
>
> As a result we get past this first hash mismatch in Release, but then
> get later hash mismatches when it tries to download .debs.

I am able to get past this, albeit with a hack. This fixes the sha256
sum in the Release file, as well as verification of the .deb files.
The original test then passes:

RESULTS - apt.AptRepoTest.test_apt_install_from_repo: PASSED (46.75s)

The hack to reduce the optimisation level for apt-native and apt. By
default it uses CXXFLAGS="-g -O2". Reducing this to -O1 fixes the
checksums.

> The issue is happening on Fedora 35 and Alma 8, so no
> buildtools-tarball in this case!

Fedora 35 is using gcc-11.2.1, could you check what Alma 8 uses?

Here are a few other things I checked, prior to noticing the
optimisation level issue:

1) we are using apt 1.2.31; the latest 1.2.y version is 1.2.35
- this still has the problem with bad sha256sums
- it does include several CVE fixes which we might want
- it added a new dependency on systemd

2) main branch version is 2.3.5
- it switched to CMAKE
- many new dependencies
- I got it to configure, but not compile
- custom crypto code seems to be dropped, in favour of gcrypt
- presumably this would fix the sha256 however I cannot confirm

Regards,
Ralph


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-14  2:47                                                 ` Ralph Siemsen
@ 2022-04-14  3:00                                                   ` Steve Sakoman
       [not found]                                                   ` <16E5A41A6E4FF34A.8845@lists.openembedded.org>
  1 sibling, 0 replies; 38+ messages in thread
From: Steve Sakoman @ 2022-04-14  3:00 UTC (permalink / raw)
  To: Ralph Siemsen
  Cc: Richard Purdie, Mike Crowe, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, Apr 13, 2022 at 4:47 PM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
>
> On Wed, 2022-04-13 at 11:39 -1000, Steve Sakoman wrote:
> > I did another experiment, where I disabled generation of the sha256
> > entries in Release (by adding --no-sha256 to the apt-ftparchive
> > command)
> >
> > As a result we get past this first hash mismatch in Release, but then
> > get later hash mismatches when it tries to download .debs.
>
> I am able to get past this, albeit with a hack. This fixes the sha256
> sum in the Release file, as well as verification of the .deb files.
> The original test then passes:
>
> RESULTS - apt.AptRepoTest.test_apt_install_from_repo: PASSED (46.75s)
>
> The hack to reduce the optimisation level for apt-native and apt. By
> default it uses CXXFLAGS="-g -O2". Reducing this to -O1 fixes the
> checksums.

Nice work!

> > The issue is happening on Fedora 35 and Alma 8, so no
> > buildtools-tarball in this case!

I've started a build that uses buildtools just to verify that fixes it
and there aren't any other issues.

> Fedora 35 is using gcc-11.2.1, could you check what Alma 8 uses?

[sakoman@alma8-ty-1 ~]$ gcc --version
gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-4)

>
> Here are a few other things I checked, prior to noticing the
> optimisation level issue:
>
> 1) we are using apt 1.2.31; the latest 1.2.y version is 1.2.35
> - this still has the problem with bad sha256sums
> - it does include several CVE fixes which we might want
> - it added a new dependency on systemd

Urgh . . . this last part isn't good since it would be a behavior
change which isn't OK for LTS

It may be that the best solution is to change to -O1 :-(

Steve

>
> 2) main branch version is 2.3.5
> - it switched to CMAKE
> - many new dependencies
> - I got it to configure, but not compile
> - custom crypto code seems to be dropped, in favour of gcrypt
> - presumably this would fix the sha256 however I cannot confirm
>
> Regards,
> Ralph
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#164377): https://lists.openembedded.org/g/openembedded-core/message/164377
> Mute This Topic: https://lists.openembedded.org/mt/90107518/3620601
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
       [not found]                                                   ` <16E5A41A6E4FF34A.8845@lists.openembedded.org>
@ 2022-04-14  3:02                                                     ` Steve Sakoman
  2022-04-14  3:03                                                       ` Steve Sakoman
  0 siblings, 1 reply; 38+ messages in thread
From: Steve Sakoman @ 2022-04-14  3:02 UTC (permalink / raw)
  To: Ralph Siemsen
  Cc: Richard Purdie, Mike Crowe, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, Apr 13, 2022 at 5:01 PM Steve Sakoman via
lists.openembedded.org <steve=sakoman.com@lists.openembedded.org>
wrote:
>
> On Wed, Apr 13, 2022 at 4:47 PM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
> >
> > On Wed, 2022-04-13 at 11:39 -1000, Steve Sakoman wrote:
> > > I did another experiment, where I disabled generation of the sha256
> > > entries in Release (by adding --no-sha256 to the apt-ftparchive
> > > command)
> > >
> > > As a result we get past this first hash mismatch in Release, but then
> > > get later hash mismatches when it tries to download .debs.
> >
> > I am able to get past this, albeit with a hack. This fixes the sha256
> > sum in the Release file, as well as verification of the .deb files.
> > The original test then passes:
> >
> > RESULTS - apt.AptRepoTest.test_apt_install_from_repo: PASSED (46.75s)
> >
> > The hack to reduce the optimisation level for apt-native and apt. By
> > default it uses CXXFLAGS="-g -O2". Reducing this to -O1 fixes the
> > checksums.
>
> Nice work!
>
> > > The issue is happening on Fedora 35 and Alma 8, so no
> > > buildtools-tarball in this case!
>
> I've started a build that uses buildtools just to verify that fixes it
> and there aren't any other issues.

FWIW, here is the link to that build - still underway.

Steve

>
> > Fedora 35 is using gcc-11.2.1, could you check what Alma 8 uses?
>
> [sakoman@alma8-ty-1 ~]$ gcc --version
> gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-4)
>
> >
> > Here are a few other things I checked, prior to noticing the
> > optimisation level issue:
> >
> > 1) we are using apt 1.2.31; the latest 1.2.y version is 1.2.35
> > - this still has the problem with bad sha256sums
> > - it does include several CVE fixes which we might want
> > - it added a new dependency on systemd
>
> Urgh . . . this last part isn't good since it would be a behavior
> change which isn't OK for LTS
>
> It may be that the best solution is to change to -O1 :-(
>
> Steve
>
> >
> > 2) main branch version is 2.3.5
> > - it switched to CMAKE
> > - many new dependencies
> > - I got it to configure, but not compile
> > - custom crypto code seems to be dropped, in favour of gcrypt
> > - presumably this would fix the sha256 however I cannot confirm
> >
> > Regards,
> > Ralph
> >
> >
> >
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#164384): https://lists.openembedded.org/g/openembedded-core/message/164384
> Mute This Topic: https://lists.openembedded.org/mt/90107518/3620601
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
  2022-04-14  3:02                                                     ` Steve Sakoman
@ 2022-04-14  3:03                                                       ` Steve Sakoman
  0 siblings, 0 replies; 38+ messages in thread
From: Steve Sakoman @ 2022-04-14  3:03 UTC (permalink / raw)
  To: Ralph Siemsen
  Cc: Richard Purdie, Mike Crowe, Ross Burton, Mittal, Anuj,
	Patches and discussions about the oe-core layer

On Wed, Apr 13, 2022 at 5:02 PM Steve Sakoman <steve@sakoman.com> wrote:
>
> On Wed, Apr 13, 2022 at 5:01 PM Steve Sakoman via
> lists.openembedded.org <steve=sakoman.com@lists.openembedded.org>
> wrote:
> >
> > On Wed, Apr 13, 2022 at 4:47 PM Ralph Siemsen <ralph.siemsen@linaro.org> wrote:
> > >
> > > On Wed, 2022-04-13 at 11:39 -1000, Steve Sakoman wrote:
> > > > I did another experiment, where I disabled generation of the sha256
> > > > entries in Release (by adding --no-sha256 to the apt-ftparchive
> > > > command)
> > > >
> > > > As a result we get past this first hash mismatch in Release, but then
> > > > get later hash mismatches when it tries to download .debs.
> > >
> > > I am able to get past this, albeit with a hack. This fixes the sha256
> > > sum in the Release file, as well as verification of the .deb files.
> > > The original test then passes:
> > >
> > > RESULTS - apt.AptRepoTest.test_apt_install_from_repo: PASSED (46.75s)
> > >
> > > The hack to reduce the optimisation level for apt-native and apt. By
> > > default it uses CXXFLAGS="-g -O2". Reducing this to -O1 fixes the
> > > checksums.
> >
> > Nice work!
> >
> > > > The issue is happening on Fedora 35 and Alma 8, so no
> > > > buildtools-tarball in this case!
> >
> > I've started a build that uses buildtools just to verify that fixes it
> > and there aren't any other issues.
>
> FWIW, here is the link to that build - still underway.

Sigh, here is the link:
https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5078

>
> Steve
>
> >
> > > Fedora 35 is using gcc-11.2.1, could you check what Alma 8 uses?
> >
> > [sakoman@alma8-ty-1 ~]$ gcc --version
> > gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-4)
> >
> > >
> > > Here are a few other things I checked, prior to noticing the
> > > optimisation level issue:
> > >
> > > 1) we are using apt 1.2.31; the latest 1.2.y version is 1.2.35
> > > - this still has the problem with bad sha256sums
> > > - it does include several CVE fixes which we might want
> > > - it added a new dependency on systemd
> >
> > Urgh . . . this last part isn't good since it would be a behavior
> > change which isn't OK for LTS
> >
> > It may be that the best solution is to change to -O1 :-(
> >
> > Steve
> >
> > >
> > > 2) main branch version is 2.3.5
> > > - it switched to CMAKE
> > > - many new dependencies
> > > - I got it to configure, but not compile
> > > - custom crypto code seems to be dropped, in favour of gcrypt
> > > - presumably this would fix the sha256 however I cannot confirm
> > >
> > > Regards,
> > > Ralph
> > >
> > >
> > >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#164384): https://lists.openembedded.org/g/openembedded-core/message/164384
> > Mute This Topic: https://lists.openembedded.org/mt/90107518/3620601
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >


^ permalink raw reply	[flat|nested] 38+ messages in thread

end of thread, other threads:[~2022-04-14 16:03 UTC | newest]

Thread overview: 38+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-29 13:07 [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032 Ross Burton
2022-03-29 19:49 ` [OE-core] " Steve Sakoman
2022-03-30 20:40   ` Ross Burton
2022-03-30 22:57     ` Steve Sakoman
2022-04-04 15:22       ` Steve Sakoman
2022-04-11 17:31         ` Ralph Siemsen
2022-04-11 17:52           ` Steve Sakoman
2022-04-11 18:17             ` Ralph Siemsen
2022-04-11 18:58               ` Steve Sakoman
2022-04-11 22:30                 ` Ralph Siemsen
2022-04-11 22:42                   ` Steve Sakoman
2022-04-12  1:33                     ` Ralph Siemsen
2022-04-12  2:12                       ` Steve Sakoman
2022-04-12 16:07                         ` Ralph Siemsen
2022-04-12 21:49                           ` Steve Sakoman
2022-04-13  1:21                             ` Ralph Siemsen
2022-04-13 15:31                               ` Steve Sakoman
     [not found]                               ` <16E57E79FD292EFA.13992@lists.openembedded.org>
2022-04-13 16:02                                 ` Steve Sakoman
2022-04-13 16:41                                   ` Mike Crowe
2022-04-13 17:37                                     ` Steve Sakoman
2022-04-13 17:50                                       ` Jose Quaresma
2022-04-13 19:16                                       ` Steve Sakoman
2022-04-13 18:02                                     ` Ralph Siemsen
2022-04-13 18:19                                       ` Steve Sakoman
2022-04-13 19:05                                         ` Ralph Siemsen
2022-04-13 21:39                                           ` Steve Sakoman
2022-04-13 21:41                                             ` Richard Purdie
2022-04-13 21:44                                               ` Steve Sakoman
2022-04-14  2:47                                                 ` Ralph Siemsen
2022-04-14  3:00                                                   ` Steve Sakoman
     [not found]                                                   ` <16E5A41A6E4FF34A.8845@lists.openembedded.org>
2022-04-14  3:02                                                     ` Steve Sakoman
2022-04-14  3:03                                                       ` Steve Sakoman
2022-04-13 17:11                                   ` Jose Quaresma
     [not found]                                   ` <16E583EB139C493B.16998@lists.openembedded.org>
2022-04-13 17:41                                     ` Jose Quaresma
2022-03-31  0:24     ` Mittal, Anuj
2022-04-05 19:04 ` Jeroen Hofstee
2022-04-09 18:14   ` Steve Sakoman
2022-04-10 21:21     ` Jeroen Hofstee

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.