[PATCH] tests/docker: Kludge <linux/swab.h> breakage by pinning linux-libc-dev

[PATCH] tests/docker: Kludge <linux/swab.h> breakage by pinning linux-libc-dev

[Philippe Mathieu-Daudé]

Linux kernel commit d5767057c9a [1] aimed to fix an issue with the
swab() declaration, but doing so it introduced the BITS_PER_LONG
definition, without using the kernel __ prefix, leading to odd
failures in userland code using kernel uapi headers, such:

      CC      block/file-posix.o
    In file included from include/qemu/timer.h:4,
                     from include/qemu/timed-average.h:29,
                     from include/block/accounting.h:28,
                     from include/block/block_int.h:27,
                     from block/file-posix.c:30:
    /usr/include/linux/swab.h: In function `__swab':
    include/qemu/bitops.h:20:34: error: "sizeof" is not defined, evaluates to 0 [-Werror=undef]
       20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
          |                                  ^~~~~~
    include/qemu/bitops.h:20:41: error: missing binary operator before token "("
       20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
          |                                         ^
    cc1: all warnings being treated as errors
    make: *** [rules.mak:69: block/file-posix.o] Error 1

The issue has been fixed in Linux kernel commit 467d12f5c78 [2].
Some distributions already backported the first commit, but still
lack the second one.

This is the case for Debian Buster.
The linux-libc-dev package contains the Linux kernel headers.
Kernel commit d5767057c9a has been backported in package
commit 1fb0eb7956 "Update to 4.19.116" [3], see debian/changelog:

  +linux (4.19.116-1) UNRELEASED; urgency=medium
  ...
  +    - uapi: rename ext2_swab() to swab() and share globally in swab.h

The previous released update before it is debian/4.19.98-1,
released as debian/4.19.98-1+deb10u1.

We can find this package in the Debian snapshot archives,
luckily archived on 2020-04-28 21:20:54 (see [4]).

QEMU use Debian based Docker images for most of its cross-builds,
which are tested by our Shippable CI jobs. The current broken
package makes most of our CI red. We can kludge this by using the
latest package released before the breakage. Do so by pinning
the package version (apt hold), and using the snapshot archives,
similar to commit b4048a7cd1.
We'll revert once the fix is backported on Debian.

Reference to commits:
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5767057c9a
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=467d12f5c78
[3] https://salsa.debian.org/kernel-team/linux/-/commit/1fb0eb7956
[4] http://snapshot.debian.org/package/linux/4.19.98-1%2Bdeb10u1/#linux-libc-dev_4.19.98-1:2b:deb10u1

Cc: Salvatore Bonaccorso <carnil@debian.org>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
Shippable green again:
https://app.shippable.com/github/philmd/qemu/runs/759/summary/console
---
 tests/docker/dockerfiles/debian10.docker | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker
index 0769700a41..d7084ac283 100644
--- a/tests/docker/dockerfiles/debian10.docker
+++ b/tests/docker/dockerfiles/debian10.docker
@@ -9,6 +9,15 @@
 #
 FROM debian:buster-slim
 
+ # Use a snapshot known to work (see http://snapshot.debian.org/#Usage)
+ENV DEBIAN_SNAPSHOT_DATE "20200428T212054Z"
+RUN sed -i \
+    "s%^deb \(https\?://\)deb.debian.org/debian/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT_DATE} \2%" \
+    /etc/apt/sources.list
+RUN sed -i \
+    "s%^deb \(https\?://\)security.debian.org/debian-security/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT_DATE} \2%" \
+    /etc/apt/sources.list
+
 # Duplicate deb line as deb-src
 RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
 
@@ -35,4 +44,11 @@ RUN apt update && \
         texinfo \
         $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
 
+# Install common build utilities
+RUN apt update && \
+    DEBIAN_FRONTEND=noninteractive eatmydata \
+    apt install -y --allow-downgrades \
+        linux-libc-dev=4.19.98-1+deb10u1 && \
+    apt-mark hold linux-libc-dev
+
 ENV FEATURES docs
-- 
2.21.3

Re: [PATCH] tests/docker: Kludge <linux/swab.h> breakage by pinning linux-libc-dev

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Linux kernel commit d5767057c9a [1] aimed to fix an issue with the
> swab() declaration, but doing so it introduced the BITS_PER_LONG
> definition, without using the kernel __ prefix, leading to odd
> failures in userland code using kernel uapi headers, such:
>
>       CC      block/file-posix.o
>     In file included from include/qemu/timer.h:4,
>                      from include/qemu/timed-average.h:29,
>                      from include/block/accounting.h:28,
>                      from include/block/block_int.h:27,
>                      from block/file-posix.c:30:
>     /usr/include/linux/swab.h: In function `__swab':
>     include/qemu/bitops.h:20:34: error: "sizeof" is not defined, evaluates to 0 [-Werror=undef]
>        20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
>           |                                  ^~~~~~
>     include/qemu/bitops.h:20:41: error: missing binary operator before token "("
>        20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
>           |                                         ^
>     cc1: all warnings being treated as errors
>     make: *** [rules.mak:69: block/file-posix.o] Error 1
>
> The issue has been fixed in Linux kernel commit 467d12f5c78 [2].
> Some distributions already backported the first commit, but still
> lack the second one.
>
> This is the case for Debian Buster.
> The linux-libc-dev package contains the Linux kernel headers.
> Kernel commit d5767057c9a has been backported in package
> commit 1fb0eb7956 "Update to 4.19.116" [3], see debian/changelog:
>
>   +linux (4.19.116-1) UNRELEASED; urgency=medium
>   ...
>   +    - uapi: rename ext2_swab() to swab() and share globally in swab.h
>
> The previous released update before it is debian/4.19.98-1,
> released as debian/4.19.98-1+deb10u1.
>
> We can find this package in the Debian snapshot archives,
> luckily archived on 2020-04-28 21:20:54 (see [4]).
>
> QEMU use Debian based Docker images for most of its cross-builds,
> which are tested by our Shippable CI jobs. The current broken
> package makes most of our CI red. We can kludge this by using the
> latest package released before the breakage. Do so by pinning
> the package version (apt hold), and using the snapshot archives,
> similar to commit b4048a7cd1.
> We'll revert once the fix is backported on Debian.

May as well mention:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960271

>
> Reference to commits:
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5767057c9a
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=467d12f5c78
> [3] https://salsa.debian.org/kernel-team/linux/-/commit/1fb0eb7956
> [4] http://snapshot.debian.org/package/linux/4.19.98-1%2Bdeb10u1/#linux-libc-dev_4.19.98-1:2b:deb10u1
>
> Cc: Salvatore Bonaccorso <carnil@debian.org>
> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> Shippable green again:
> https://app.shippable.com/github/philmd/qemu/runs/759/summary/console
> ---
>  tests/docker/dockerfiles/debian10.docker | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
>
> diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker
> index 0769700a41..d7084ac283 100644
> --- a/tests/docker/dockerfiles/debian10.docker
> +++ b/tests/docker/dockerfiles/debian10.docker
> @@ -9,6 +9,15 @@
>  #
>  FROM debian:buster-slim
>  
> + # Use a snapshot known to work (see http://snapshot.debian.org/#Usage)
> +ENV DEBIAN_SNAPSHOT_DATE "20200428T212054Z"
> +RUN sed -i \
> +    "s%^deb \(https\?://\)deb.debian.org/debian/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT_DATE} \2%" \
> +    /etc/apt/sources.list
> +RUN sed -i \
> +    "s%^deb \(https\?://\)security.debian.org/debian-security/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT_DATE} \2%" \
> +    /etc/apt/sources.list
> +
>  # Duplicate deb line as deb-src
>  RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
>  
> @@ -35,4 +44,11 @@ RUN apt update && \
>          texinfo \
>          $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
>  
> +# Install common build utilities
> +RUN apt update && \
> +    DEBIAN_FRONTEND=noninteractive eatmydata \
> +    apt install -y --allow-downgrades \
> +        linux-libc-dev=4.19.98-1+deb10u1 && \
> +    apt-mark hold linux-libc-dev
> +

Do we need this bit? Surely if we have a snapshot URL we don't need to
hold the package as it will never get updated?

>  ENV FEATURES docs


-- 
Alex Bennée

Re: [PATCH] tests/docker: Kludge <linux/swab.h> breakage by pinning linux-libc-dev

[Philippe Mathieu-Daudé]

On 5/13/20 5:15 PM, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> 
>> Linux kernel commit d5767057c9a [1] aimed to fix an issue with the
>> swab() declaration, but doing so it introduced the BITS_PER_LONG
>> definition, without using the kernel __ prefix, leading to odd
>> failures in userland code using kernel uapi headers, such:
>>
>>        CC      block/file-posix.o
>>      In file included from include/qemu/timer.h:4,
>>                       from include/qemu/timed-average.h:29,
>>                       from include/block/accounting.h:28,
>>                       from include/block/block_int.h:27,
>>                       from block/file-posix.c:30:
>>      /usr/include/linux/swab.h: In function `__swab':
>>      include/qemu/bitops.h:20:34: error: "sizeof" is not defined, evaluates to 0 [-Werror=undef]
>>         20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
>>            |                                  ^~~~~~
>>      include/qemu/bitops.h:20:41: error: missing binary operator before token "("
>>         20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
>>            |                                         ^
>>      cc1: all warnings being treated as errors
>>      make: *** [rules.mak:69: block/file-posix.o] Error 1
>>
>> The issue has been fixed in Linux kernel commit 467d12f5c78 [2].
>> Some distributions already backported the first commit, but still
>> lack the second one.
>>
>> This is the case for Debian Buster.
>> The linux-libc-dev package contains the Linux kernel headers.
>> Kernel commit d5767057c9a has been backported in package
>> commit 1fb0eb7956 "Update to 4.19.116" [3], see debian/changelog:
>>
>>    +linux (4.19.116-1) UNRELEASED; urgency=medium
>>    ...
>>    +    - uapi: rename ext2_swab() to swab() and share globally in swab.h
>>
>> The previous released update before it is debian/4.19.98-1,
>> released as debian/4.19.98-1+deb10u1.
>>
>> We can find this package in the Debian snapshot archives,
>> luckily archived on 2020-04-28 21:20:54 (see [4]).
>>
>> QEMU use Debian based Docker images for most of its cross-builds,
>> which are tested by our Shippable CI jobs. The current broken
>> package makes most of our CI red. We can kludge this by using the
>> latest package released before the breakage. Do so by pinning
>> the package version (apt hold), and using the snapshot archives,
>> similar to commit b4048a7cd1.
>> We'll revert once the fix is backported on Debian.
> 
> May as well mention:
> 
>    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960271

Sure (I remembered you talked about it on IRC, but was lazy to dig).

> 
>>
>> Reference to commits:
>> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5767057c9a
>> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=467d12f5c78
>> [3] https://salsa.debian.org/kernel-team/linux/-/commit/1fb0eb7956
>> [4] http://snapshot.debian.org/package/linux/4.19.98-1%2Bdeb10u1/#linux-libc-dev_4.19.98-1:2b:deb10u1
>>
>> Cc: Salvatore Bonaccorso <carnil@debian.org>
>> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>> Shippable green again:
>> https://app.shippable.com/github/philmd/qemu/runs/759/summary/console
>> ---
>>   tests/docker/dockerfiles/debian10.docker | 16 ++++++++++++++++
>>   1 file changed, 16 insertions(+)
>>
>> diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker
>> index 0769700a41..d7084ac283 100644
>> --- a/tests/docker/dockerfiles/debian10.docker
>> +++ b/tests/docker/dockerfiles/debian10.docker
>> @@ -9,6 +9,15 @@
>>   #
>>   FROM debian:buster-slim
>>   
>> + # Use a snapshot known to work (see http://snapshot.debian.org/#Usage)
>> +ENV DEBIAN_SNAPSHOT_DATE "20200428T212054Z"
>> +RUN sed -i \
>> +    "s%^deb \(https\?://\)deb.debian.org/debian/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT_DATE} \2%" \
>> +    /etc/apt/sources.list
>> +RUN sed -i \
>> +    "s%^deb \(https\?://\)security.debian.org/debian-security/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT_DATE} \2%" \
>> +    /etc/apt/sources.list
>> +
>>   # Duplicate deb line as deb-src
>>   RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
>>   
>> @@ -35,4 +44,11 @@ RUN apt update && \
>>           texinfo \
>>           $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
>>   
>> +# Install common build utilities
>> +RUN apt update && \
>> +    DEBIAN_FRONTEND=noninteractive eatmydata \
>> +    apt install -y --allow-downgrades \
>> +        linux-libc-dev=4.19.98-1+deb10u1 && \
>> +    apt-mark hold linux-libc-dev
>> +
> 
> Do we need this bit? Surely if we have a snapshot URL we don't need to
> hold the package as it will never get updated?

You are right, it is a remnant from the first approach (pin it without 
using snapshot) which didn't work. I'll remove this line and the comment 
in description.

> 
>>   ENV FEATURES docs
> 
> 

Re: [PATCH] tests/docker: Kludge <linux/swab.h> breakage by pinning linux-libc-dev

[Salvatore Bonaccorso]

Hi,

On Wed, May 13, 2020 at 04:15:48PM +0100, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> 
> > Linux kernel commit d5767057c9a [1] aimed to fix an issue with the
> > swab() declaration, but doing so it introduced the BITS_PER_LONG
> > definition, without using the kernel __ prefix, leading to odd
> > failures in userland code using kernel uapi headers, such:
> >
> >       CC      block/file-posix.o
> >     In file included from include/qemu/timer.h:4,
> >                      from include/qemu/timed-average.h:29,
> >                      from include/block/accounting.h:28,
> >                      from include/block/block_int.h:27,
> >                      from block/file-posix.c:30:
> >     /usr/include/linux/swab.h: In function `__swab':
> >     include/qemu/bitops.h:20:34: error: "sizeof" is not defined, evaluates to 0 [-Werror=undef]
> >        20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
> >           |                                  ^~~~~~
> >     include/qemu/bitops.h:20:41: error: missing binary operator before token "("
> >        20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
> >           |                                         ^
> >     cc1: all warnings being treated as errors
> >     make: *** [rules.mak:69: block/file-posix.o] Error 1
> >
> > The issue has been fixed in Linux kernel commit 467d12f5c78 [2].
> > Some distributions already backported the first commit, but still
> > lack the second one.
> >
> > This is the case for Debian Buster.
> > The linux-libc-dev package contains the Linux kernel headers.
> > Kernel commit d5767057c9a has been backported in package
> > commit 1fb0eb7956 "Update to 4.19.116" [3], see debian/changelog:
> >
> >   +linux (4.19.116-1) UNRELEASED; urgency=medium
> >   ...
> >   +    - uapi: rename ext2_swab() to swab() and share globally in swab.h
> >
> > The previous released update before it is debian/4.19.98-1,
> > released as debian/4.19.98-1+deb10u1.
> >
> > We can find this package in the Debian snapshot archives,
> > luckily archived on 2020-04-28 21:20:54 (see [4]).
> >
> > QEMU use Debian based Docker images for most of its cross-builds,
> > which are tested by our Shippable CI jobs. The current broken
> > package makes most of our CI red. We can kludge this by using the
> > latest package released before the breakage. Do so by pinning
> > the package version (apt hold), and using the snapshot archives,
> > similar to commit b4048a7cd1.
> > We'll revert once the fix is backported on Debian.
> 
> May as well mention:
> 
>   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960271

FWIW, we applied in the buster packaging branch
https://salsa.debian.org/kernel-team/linux/-/commit/a4fb2a7b7688f3a7cb36e17b9d8c661ac44a41a4

But the point release just happened last weekend, so it might take
some time until the fix will enter either via the 10.5 point release
or sooner by a needed update.

Regards,
Salvatore

Re: [PATCH] tests/docker: Kludge <linux/swab.h> breakage by pinning linux-libc-dev

[Alex Bennée]

Salvatore Bonaccorso <carnil@debian.org> writes:

> Hi,
>
> On Wed, May 13, 2020 at 04:15:48PM +0100, Alex Bennée wrote:
>> 
>> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>> 
>> > Linux kernel commit d5767057c9a [1] aimed to fix an issue with the
>> > swab() declaration, but doing so it introduced the BITS_PER_LONG
>> > definition, without using the kernel __ prefix, leading to odd
>> > failures in userland code using kernel uapi headers, such:
>> >
>> >       CC      block/file-posix.o
>> >     In file included from include/qemu/timer.h:4,
>> >                      from include/qemu/timed-average.h:29,
>> >                      from include/block/accounting.h:28,
>> >                      from include/block/block_int.h:27,
>> >                      from block/file-posix.c:30:
>> >     /usr/include/linux/swab.h: In function `__swab':
>> >     include/qemu/bitops.h:20:34: error: "sizeof" is not defined, evaluates to 0 [-Werror=undef]
>> >        20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
>> >           |                                  ^~~~~~
>> >     include/qemu/bitops.h:20:41: error: missing binary operator before token "("
>> >        20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
>> >           |                                         ^
>> >     cc1: all warnings being treated as errors
>> >     make: *** [rules.mak:69: block/file-posix.o] Error 1
>> >
>> > The issue has been fixed in Linux kernel commit 467d12f5c78 [2].
>> > Some distributions already backported the first commit, but still
>> > lack the second one.
>> >
>> > This is the case for Debian Buster.
>> > The linux-libc-dev package contains the Linux kernel headers.
>> > Kernel commit d5767057c9a has been backported in package
>> > commit 1fb0eb7956 "Update to 4.19.116" [3], see debian/changelog:
>> >
>> >   +linux (4.19.116-1) UNRELEASED; urgency=medium
>> >   ...
>> >   +    - uapi: rename ext2_swab() to swab() and share globally in swab.h
>> >
>> > The previous released update before it is debian/4.19.98-1,
>> > released as debian/4.19.98-1+deb10u1.
>> >
>> > We can find this package in the Debian snapshot archives,
>> > luckily archived on 2020-04-28 21:20:54 (see [4]).
>> >
>> > QEMU use Debian based Docker images for most of its cross-builds,
>> > which are tested by our Shippable CI jobs. The current broken
>> > package makes most of our CI red. We can kludge this by using the
>> > latest package released before the breakage. Do so by pinning
>> > the package version (apt hold), and using the snapshot archives,
>> > similar to commit b4048a7cd1.
>> > We'll revert once the fix is backported on Debian.
>> 
>> May as well mention:
>> 
>>   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960271
>
> FWIW, we applied in the buster packaging branch
> https://salsa.debian.org/kernel-team/linux/-/commit/a4fb2a7b7688f3a7cb36e17b9d8c661ac44a41a4
>
> But the point release just happened last weekend, so it might take
> some time until the fix will enter either via the 10.5 point release
> or sooner by a needed update.

Thanks for the update. So I guess we wait for a CVE to trigger an early
release ;-)

-- 
Alex Bennée

Re: [PATCH] tests/docker: Kludge <linux/swab.h> breakage by pinning linux-libc-dev

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> On 5/13/20 5:15 PM, Alex Bennée wrote:
>> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>> 
>>> Linux kernel commit d5767057c9a [1] aimed to fix an issue with the
>>> swab() declaration, but doing so it introduced the BITS_PER_LONG
>>> definition, without using the kernel __ prefix, leading to odd
>>> failures in userland code using kernel uapi headers, such:
>>>
>>>        CC      block/file-posix.o
>>>      In file included from include/qemu/timer.h:4,
>>>                       from include/qemu/timed-average.h:29,
>>>                       from include/block/accounting.h:28,
>>>                       from include/block/block_int.h:27,
>>>                       from block/file-posix.c:30:
>>>      /usr/include/linux/swab.h: In function `__swab':
>>>      include/qemu/bitops.h:20:34: error: "sizeof" is not defined, evaluates to 0 [-Werror=undef]
>>>         20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
>>>            |                                  ^~~~~~
>>>      include/qemu/bitops.h:20:41: error: missing binary operator before token "("
>>>         20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
>>>            |                                         ^
>>>      cc1: all warnings being treated as errors
>>>      make: *** [rules.mak:69: block/file-posix.o] Error 1
>>>
>>> The issue has been fixed in Linux kernel commit 467d12f5c78 [2].
>>> Some distributions already backported the first commit, but still
>>> lack the second one.
>>>
>>> This is the case for Debian Buster.
>>> The linux-libc-dev package contains the Linux kernel headers.
>>> Kernel commit d5767057c9a has been backported in package
>>> commit 1fb0eb7956 "Update to 4.19.116" [3], see debian/changelog:
>>>
>>>    +linux (4.19.116-1) UNRELEASED; urgency=medium
>>>    ...
>>>    +    - uapi: rename ext2_swab() to swab() and share globally in swab.h
>>>
>>> The previous released update before it is debian/4.19.98-1,
>>> released as debian/4.19.98-1+deb10u1.
>>>
>>> We can find this package in the Debian snapshot archives,
>>> luckily archived on 2020-04-28 21:20:54 (see [4]).
>>>
>>> QEMU use Debian based Docker images for most of its cross-builds,
>>> which are tested by our Shippable CI jobs. The current broken
>>> package makes most of our CI red. We can kludge this by using the
>>> latest package released before the breakage. Do so by pinning
>>> the package version (apt hold), and using the snapshot archives,
>>> similar to commit b4048a7cd1.
>>> We'll revert once the fix is backported on Debian.
>> May as well mention:
>>    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960271
>
> Sure (I remembered you talked about it on IRC, but was lazy to dig).
>
>> 
>>>
>>> Reference to commits:
>>> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5767057c9a
>>> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=467d12f5c78
>>> [3] https://salsa.debian.org/kernel-team/linux/-/commit/1fb0eb7956
>>> [4] http://snapshot.debian.org/package/linux/4.19.98-1%2Bdeb10u1/#linux-libc-dev_4.19.98-1:2b:deb10u1
>>>
>>> Cc: Salvatore Bonaccorso <carnil@debian.org>
>>> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>> ---
>>> Shippable green again:
>>> https://app.shippable.com/github/philmd/qemu/runs/759/summary/console
>>> ---
>>>   tests/docker/dockerfiles/debian10.docker | 16 ++++++++++++++++
>>>   1 file changed, 16 insertions(+)
>>>
>>> diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker
>>> index 0769700a41..d7084ac283 100644
>>> --- a/tests/docker/dockerfiles/debian10.docker
>>> +++ b/tests/docker/dockerfiles/debian10.docker
>>> @@ -9,6 +9,15 @@
>>>   #
>>>   FROM debian:buster-slim
>>>   + # Use a snapshot known to work (see
>>> http://snapshot.debian.org/#Usage)
>>> +ENV DEBIAN_SNAPSHOT_DATE "20200428T212054Z"
>>> +RUN sed -i \
>>> +    "s%^deb \(https\?://\)deb.debian.org/debian/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT_DATE} \2%" \
>>> +    /etc/apt/sources.list
>>> +RUN sed -i \
>>> +    "s%^deb \(https\?://\)security.debian.org/debian-security/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT_DATE} \2%" \
>>> +    /etc/apt/sources.list
>>> +
>>>   # Duplicate deb line as deb-src
>>>   RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
>>>   @@ -35,4 +44,11 @@ RUN apt update && \
>>>           texinfo \
>>>           $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
>>>   +# Install common build utilities
>>> +RUN apt update && \
>>> +    DEBIAN_FRONTEND=noninteractive eatmydata \
>>> +    apt install -y --allow-downgrades \
>>> +        linux-libc-dev=4.19.98-1+deb10u1 && \
>>> +    apt-mark hold linux-libc-dev
>>> +
>> Do we need this bit? Surely if we have a snapshot URL we don't need
>> to
>> hold the package as it will never get updated?
>
> You are right, it is a remnant from the first approach (pin it without
> using snapshot) which didn't work. I'll remove this line and the
> comment in description.

Hmm it seems this is needed - without it I see other failures:

  https://app.shippable.com/github/stsquad/qemu/runs/1377/8/console

-- 
Alex Bennée

Re: [PATCH] tests/docker: Kludge <linux/swab.h> breakage by pinning linux-libc-dev

[Philippe Mathieu-Daudé]

On 5/15/20 2:06 PM, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> 
>> On 5/13/20 5:15 PM, Alex Bennée wrote:
>>> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>>>
>>>> Linux kernel commit d5767057c9a [1] aimed to fix an issue with the
>>>> swab() declaration, but doing so it introduced the BITS_PER_LONG
>>>> definition, without using the kernel __ prefix, leading to odd
>>>> failures in userland code using kernel uapi headers, such:
>>>>
>>>>         CC      block/file-posix.o
>>>>       In file included from include/qemu/timer.h:4,
>>>>                        from include/qemu/timed-average.h:29,
>>>>                        from include/block/accounting.h:28,
>>>>                        from include/block/block_int.h:27,
>>>>                        from block/file-posix.c:30:
>>>>       /usr/include/linux/swab.h: In function `__swab':
>>>>       include/qemu/bitops.h:20:34: error: "sizeof" is not defined, evaluates to 0 [-Werror=undef]
>>>>          20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
>>>>             |                                  ^~~~~~
>>>>       include/qemu/bitops.h:20:41: error: missing binary operator before token "("
>>>>          20 | #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
>>>>             |                                         ^
>>>>       cc1: all warnings being treated as errors
>>>>       make: *** [rules.mak:69: block/file-posix.o] Error 1
>>>>
>>>> The issue has been fixed in Linux kernel commit 467d12f5c78 [2].
>>>> Some distributions already backported the first commit, but still
>>>> lack the second one.
>>>>
>>>> This is the case for Debian Buster.
>>>> The linux-libc-dev package contains the Linux kernel headers.
>>>> Kernel commit d5767057c9a has been backported in package
>>>> commit 1fb0eb7956 "Update to 4.19.116" [3], see debian/changelog:
>>>>
>>>>     +linux (4.19.116-1) UNRELEASED; urgency=medium
>>>>     ...
>>>>     +    - uapi: rename ext2_swab() to swab() and share globally in swab.h
>>>>
>>>> The previous released update before it is debian/4.19.98-1,
>>>> released as debian/4.19.98-1+deb10u1.
>>>>
>>>> We can find this package in the Debian snapshot archives,
>>>> luckily archived on 2020-04-28 21:20:54 (see [4]).
>>>>
>>>> QEMU use Debian based Docker images for most of its cross-builds,
>>>> which are tested by our Shippable CI jobs. The current broken
>>>> package makes most of our CI red. We can kludge this by using the
>>>> latest package released before the breakage. Do so by pinning
>>>> the package version (apt hold), and using the snapshot archives,
>>>> similar to commit b4048a7cd1.
>>>> We'll revert once the fix is backported on Debian.
>>> May as well mention:
>>>     https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960271
>>
>> Sure (I remembered you talked about it on IRC, but was lazy to dig).
>>
>>>
>>>>
>>>> Reference to commits:
>>>> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5767057c9a
>>>> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=467d12f5c78
>>>> [3] https://salsa.debian.org/kernel-team/linux/-/commit/1fb0eb7956
>>>> [4] http://snapshot.debian.org/package/linux/4.19.98-1%2Bdeb10u1/#linux-libc-dev_4.19.98-1:2b:deb10u1
>>>>
>>>> Cc: Salvatore Bonaccorso <carnil@debian.org>
>>>> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
>>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>>> ---
>>>> Shippable green again:
>>>> https://app.shippable.com/github/philmd/qemu/runs/759/summary/console
>>>> ---
>>>>    tests/docker/dockerfiles/debian10.docker | 16 ++++++++++++++++
>>>>    1 file changed, 16 insertions(+)
>>>>
>>>> diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker
>>>> index 0769700a41..d7084ac283 100644
>>>> --- a/tests/docker/dockerfiles/debian10.docker
>>>> +++ b/tests/docker/dockerfiles/debian10.docker
>>>> @@ -9,6 +9,15 @@
>>>>    #
>>>>    FROM debian:buster-slim
>>>>    + # Use a snapshot known to work (see
>>>> http://snapshot.debian.org/#Usage)
>>>> +ENV DEBIAN_SNAPSHOT_DATE "20200428T212054Z"
>>>> +RUN sed -i \
>>>> +    "s%^deb \(https\?://\)deb.debian.org/debian/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT_DATE} \2%" \
>>>> +    /etc/apt/sources.list
>>>> +RUN sed -i \
>>>> +    "s%^deb \(https\?://\)security.debian.org/debian-security/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT_DATE} \2%" \
>>>> +    /etc/apt/sources.list
>>>> +
>>>>    # Duplicate deb line as deb-src
>>>>    RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
>>>>    @@ -35,4 +44,11 @@ RUN apt update && \
>>>>            texinfo \
>>>>            $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
>>>>    +# Install common build utilities
>>>> +RUN apt update && \
>>>> +    DEBIAN_FRONTEND=noninteractive eatmydata \
>>>> +    apt install -y --allow-downgrades \
>>>> +        linux-libc-dev=4.19.98-1+deb10u1 && \
>>>> +    apt-mark hold linux-libc-dev
>>>> +
>>> Do we need this bit? Surely if we have a snapshot URL we don't need
>>> to
>>> hold the package as it will never get updated?
>>
>> You are right, it is a remnant from the first approach (pin it without
>> using snapshot) which didn't work. I'll remove this line and the
>> comment in description.
> 
> Hmm it seems this is needed - without it I see other failures:
> 
>    https://app.shippable.com/github/stsquad/qemu/runs/1377/8/console
> 

Sorry I haven't updated this thread.

I did the requested change, but then the snapshot server went down, so I 
couldn't finish testing.
My impression is that when 2 different versions are available (which 
happens in this snapshot date) apt will install the more recent. I 
couldn't find other snapshot dates with at least 4.19.98-1+deb10u1 and 
at most 4.19.98-1+deb10u1. Only this date but with 2 versions, so 
pinning is unfortunately required.