From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9C57C433DB for ; Fri, 19 Feb 2021 16:16:16 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3321F64D92 for ; Fri, 19 Feb 2021 16:16:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3321F64D92 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=bsdimp.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:58736 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lD8Rz-00074R-4Z for qemu-devel@archiver.kernel.org; Fri, 19 Feb 2021 11:16:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:54354) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lD8KZ-0008Ml-Nm for qemu-devel@nongnu.org; Fri, 19 Feb 2021 11:08:35 -0500 Received: from mail-qk1-x736.google.com ([2607:f8b0:4864:20::736]:35412) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lD8KW-0004Uy-L7 for qemu-devel@nongnu.org; Fri, 19 Feb 2021 11:08:35 -0500 Received: by mail-qk1-x736.google.com with SMTP id x14so6009347qkm.2 for ; Fri, 19 Feb 2021 08:08:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=mD6HBq/stT5yMnTZtFIqhMi71T/KDXTbdG2oSedVas4=; b=LMc1iyIiD7Gdp7Mm+IiOAx2jXpa0/6XL9zZ4vvuK2mMd+YrdZSSMDJa0+b7EU3/wkx NKGzDIcVMBmQfQgkUIA8jGidS+UrzZ3E26ZsWmHuNqi0jbfTyPjKsbBumMipRbZG5jU0 uGyu/M4G6otZMX7PSQM6Kb0v0TI/8opJcj2U3E53+KkuWwqzNc6SucX3t6EdvSsmwv4w H6Ah+TNz9jRv9BqaZHSa6BF9xUkzc7CWTfFhr4pHnR3aNaNJ3ykrNUOGFpbHhyXsV92t 3Ddz2Eo3aafCvgZUGGAQ+2pwKMlt8X+rVx3SqdAKghVcF3XqtyRc1POd5tgdbiHl88Ia xZoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mD6HBq/stT5yMnTZtFIqhMi71T/KDXTbdG2oSedVas4=; b=DESnyBNFXYMwfXDwflhtw3lvnqOBSIEMupOYVcKPXZ+ZOsSMbiK8r+z4BfyxnHy7fH N47tbJExSaFuD94bPv6T7CfjqGilNeTTcIiUlDDpqjEBPI3pHDrq4Ztf3tfxRUc5Praz AYh+UJYd52NqxOdoJhX4bVHZDvPnogIP74lupx/ab1OiWjk0/7QufleMuYcPYuduap6Z 0EQGMGAMBkD1+rYPtR9G9FgpJpxJVhSspP0XJlCfOhqmMm+R1gjXo4Sn6yJhWQM2EDzh Uk1rBvpumJYjIP299ao/3Ow/gAYlP+KLf6GTtTZYaW+Uvr42j+tJzxtz3CuqtBwg/a3f BWtw== X-Gm-Message-State: AOAM533kecOkEH556VErDwIKAN6392rOhbqLrO0imILMVig4EZazm/GL 3grjG5G53H2/RobdVLJyFo3XDOhxzYtJxfJmY3I1hQ== X-Google-Smtp-Source: ABdhPJwK8aFG91M9RfZA8abW2WsBOOxlboZJy+j6Wh9PnlIXsTIqMhJ/kzf8wLrkLINVkoZEcdX4AJpmLicJeuN1qZQ= X-Received: by 2002:a37:6cc6:: with SMTP id h189mr10412009qkc.195.1613750908914; Fri, 19 Feb 2021 08:08:28 -0800 (PST) MIME-Version: 1.0 References: <8735xss5q3.fsf@linaro.org> <20210219152408.34ibwagyqzgye4yd@sirius.home.kraxel.org> In-Reply-To: <20210219152408.34ibwagyqzgye4yd@sirius.home.kraxel.org> From: Warner Losh Date: Fri, 19 Feb 2021 09:08:18 -0700 Message-ID: Subject: Re: FreeBSD build regressions To: Gerd Hoffmann Content-Type: multipart/alternative; boundary="000000000000aea3d605bbb2a905" Received-SPF: none client-ip=2607:f8b0:4864:20::736; envelope-from=wlosh@bsdimp.com; helo=mail-qk1-x736.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?B?QWxleCBCZW5uw6ll?= , Peter Maydell , Ed Maste , QEMU Developers , Li-Wen Hsu Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" --000000000000aea3d605bbb2a905 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable First off, sorry for the hassles this caused... The problem is a failure to upgrade for reasons explained below... While the reasons make sense, it's still a bit of a hassle, for which I apologize. On Fri, Feb 19, 2021 at 8:51 AM Gerd Hoffmann wrote: > On Fri, Feb 19, 2021 at 10:41:44AM +0000, Peter Maydell wrote: > > On Fri, 19 Feb 2021 at 10:39, Alex Benn=C3=A9e > wrote: > > > > > > > > > Hi, > > > > > > It looks like the build has been broken on Cirrus since at least > 7b2c4c: > > > > > > https://cirrus-ci.com/github/qemu/qemu > > > > > > I did attempt to have a look but "vm-build-freebsd" seems to be faili= ng > > > with a different error > > > > FWIW the vm-build-freebsd build-and-test works for me, as I > > continue to run it as part of the merge tests. Is this something > > to do with whether you already have a freebsd image cached > > as opposed to it getting re-built from scratch (perhaps with > > a newer FreeBSD)? > > The base image should be the same no matter what (updating that needs a > tests/vm/freebsd update which in turn triggers a rebuild). The addon > package versions may differ though, so in case a broken package enters > the freebsd package repos it may happen that old, existing vm images > continue to work whereas newly created images don't ... > > Trying to rebuild the freebsd image here results in this: > > [ ... ] > ### Installing packages ... > Bootstrapping pkg from pkg+ > http://pkg.FreeBSD.org/FreeBSD:12:amd64/quarterly, please wait... > Verifying signature with trusted certificate pkg.freebsd.org.2013102301..= . > done > Installing pkg-1.16.1... > Newer FreeBSD version for package pkg: > To ignore this error set IGNORE_OSVERSION=3Dyes > - package: 1202000 <- freebsd 12.2 expected ? > - running kernel: 1201000 <- freebsd 12.1 running ? > Ignore the mismatch and continue? [y/N]: > > So it seems the freebsd 12.1 images tries to fetch 12.2 packages when > running "pkg install -y ", which would explain why they don't > work. > FreeBSD 12.1 images fetch the FreeBSD 12 packages. This works until FreeBSD 12.1 hits end of life and the packages are built on newer versions. There's no long-term support for releases once a new minor release happens. The economics of this quickly get out of hand as the testing matrix explodes and the number of machines needed to support all old versions (even on supported branches) would be prohibitively expensive. > Switching to freebsd 12.2 should solve this, at least until 12.3 is > released, but I'm wondering why the freebsd pkg utility fetches > incompatible packages in the first place and whenever there is any > way to avoid this ... > FreeBSD builds packages on the oldest supported version in the stable branch. Due to forward compatibility, that means all supported versions of FreeBSD 12.x will work. Recently, FreeBSD 12.1 became unsupported, so the build machines clicked forward to 12.2. Since there's no 'forward compatibility' guarantees, this problem was hit. While you can run binaries compiled on old versions of the software on new versions of the system, you can't necessarily do the inverse because new symbols are introduced (in this case close_range). The base FreeBSD image should be rolled when new versions are released to avoid this issue. tests/vm/freebsd should be updated. I have a full morning, but if nobody has beat me to it, I'll submit a patch this afternoon to fix this and add it to my list of things to do when FreeBSD creates a new release. Warner --000000000000aea3d605bbb2a905 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
First off, sorry for the hassles this caused... The p= roblem is a failure to upgrade for reasons explained below... While the rea= sons make sense, it's still a bit of a hassle, for which I apologize.
On= Fri, Feb 19, 2021 at 8:51 AM Gerd Hoffmann <gerd@kraxel.org> wrote:
On Fri, Feb 19, 2021 at 10:41:44AM +0000, Peter Mayd= ell wrote:
> On Fri, 19 Feb 2021 at 10:39, Alex Benn=C3=A9e <alex.bennee@linaro.org> wro= te:
> >
> >
> > Hi,
> >
> > It looks like the build has been broken on Cirrus since at least = 7b2c4c:
> >
> >=C2=A0 =C2=A0https://cirrus-ci.com/github/qemu/qemu
> >
> > I did attempt to have a look but "vm-build-freebsd" see= ms to be failing
> > with a different error
>
> FWIW the vm-build-freebsd build-and-test works for me, as I
> continue to run it as part of the merge tests. Is this something
> to do with whether you already have a freebsd image cached
> as opposed to it getting re-built from scratch (perhaps with
> a newer FreeBSD)?

The base image should be the same no matter what (updating that needs a
tests/vm/freebsd update which in turn triggers a rebuild).=C2=A0 The addon<= br> package versions may differ though, so in case a broken package enters
the freebsd package repos it may happen that old, existing vm images
continue to work whereas newly created images don't ...

Trying to rebuild the freebsd image here results in this:

[ ... ]
### Installing packages ...
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/F= reeBSD:12:amd64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... = done
Installing pkg-1.16.1...
Newer FreeBSD version for package pkg:
To ignore this error set IGNORE_OSVERSION=3Dyes
- package: 1202000=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 <- freebsd 12.2 expected ?
- running kernel: 1201000=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0<- freebsd 12.1 running ?
Ignore the mismatch and continue? [y/N]:

So it seems the freebsd 12.1 images tries to fetch 12.2 packages when
running "pkg install -y <list>", which would explain why th= ey don't
work.

FreeBSD 12.1 images fetch the Fre= eBSD 12 packages. This works until FreeBSD 12.1 hits end of life and the pa= ckages are built on newer versions. There's no long-term support for re= leases once a new minor release happens. The economics of this quickly get = out of hand as the testing matrix explodes and the number of machines neede= d to support all old versions (even on supported branches) would be prohibi= tively expensive.
=C2=A0
Switching to freebsd 12.2 should solve this, at least until 12.3 is
released, but I'm wondering why the freebsd pkg utility fetches
incompatible packages in the first place and whenever there is any
way to avoid this ...

FreeBSD builds pa= ckages on the oldest supported version in the stable branch. Due to forward= compatibility, that means all supported versions of FreeBSD 12.x will work= . Recently, FreeBSD 12.1 became unsupported, so the build machines clicked = forward to 12.2. Since there's no 'forward compatibility' guara= ntees, this problem was hit. While you can run binaries compiled on old ver= sions of the software on new versions of the system, you can't necessar= ily do the inverse because new symbols are introduced (in this case close_r= ange).

The base FreeBSD image should be rolled whe= n new versions are released to avoid this issue. tests/vm/freebsd should be= updated. I have a full morning, but if nobody has beat me to it, I'll = submit a patch this afternoon to fix this and add it to my list of things t= o do when FreeBSD creates a new release.

Warner
--000000000000aea3d605bbb2a905--