From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-vs1-f43.google.com (mail-vs1-f43.google.com [209.85.217.43]) by mx.groups.io with SMTP id smtpd.web08.1078.1610390683557076272 for ; Mon, 11 Jan 2021 10:44:43 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20161025 header.b=EBqDnQFX; spf=pass (domain: gmail.com, ip: 209.85.217.43, mailfrom: alex.kanavin@gmail.com) Received: by mail-vs1-f43.google.com with SMTP id s2so85296vsk.2 for ; Mon, 11 Jan 2021 10:44:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=QwbfbWQwLJ14BvwprxSmEQeT1d0+rtqjut2oh3x87lI=; b=EBqDnQFX3ktD6bBsC2vpWke1EzeBDcRGIyZQ9bxzqd60LVuERKS5DfnUqdsg96ZuwX LGFu2EYjP0C852bLyqj6Kz74WbMG3aXJqO4Uk6VeRj1j8iMtRK7aebVB27VfWk0odevo MtfY3aSAIbMSecPqhpKVDpAuINjN5RipTTWU9Q2ZgVsBJb9kVnTs+Sw0WAr2KurX3dLG JOtSIldc88hvcXBznWyLI8ironBRHBnFolaSQ67oqpZy0ISSKMqFv4IdMHZUEXWjnLWr JKZmDVgj3zG/AjYMKjhp7pRyXXmmml/BobQY+TYH/pvDv/+yh0GglFFBTFNF4X0V5s7S HKxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=QwbfbWQwLJ14BvwprxSmEQeT1d0+rtqjut2oh3x87lI=; b=n9iEhs/aBFQPeyQkR7SHSF0HJANnTj5vlbIPZ8b1apinlMVYicO/TqNe2B5Dtee6V2 yo1EKMvaBoIxBwqpX+U5OwtAhLibvaCE9KXsfzwEV247il4d1eP2hUlLwKk5jha5BXDq ycCCrEIL+chXBfW6Pd7G7iqd40wukfqK2kgZhRYlEUUFhZv2jFgipwN4Be6WEjYa7yT2 XDpKYsXs/wZPCmU3jNNXGuRq/eZbdUDy/79vZHQ0xWdgUOQhLH/gPOsNBx/5ZsbIIzIh F89jSDDmpV3d73/yKSJEXEHwZDZ5pRVGHWcFtu0xInoCraKxeiElnl4z8CR0EJsjrknp QRmg== X-Gm-Message-State: AOAM531WXfetfPfuV6tnK10Bq0LrQq+GGBNqvbyEuhCAOWXasxvL2lAd 6t51auaCr9bWiSA5claUCFSbfbhVPrmPhil4b6M= X-Google-Smtp-Source: ABdhPJxDNcjxFbgBN2QGfHjOCkq/cldEIlvLka9VBS5omyARU1uZH4+fp99UZ/c7RZ/3a18W8gcdSCjvnWBH1ro+ZiQ= X-Received: by 2002:a67:c89e:: with SMTP id v30mr825013vsk.54.1610390682602; Mon, 11 Jan 2021 10:44:42 -0800 (PST) MIME-Version: 1.0 References: <1610369123-220800-1-git-send-email-michael.ho@bmw.de> <48E89C79-DCF6-4AB8-B362-0846382C3208@bmw.de> In-Reply-To: <48E89C79-DCF6-4AB8-B362-0846382C3208@bmw.de> From: "Alexander Kanavin" Date: Mon, 11 Jan 2021 19:44:31 +0100 Message-ID: Subject: Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in parallel to do_rootfs To: Michael Ho Cc: OE-core Content-Type: multipart/alternative; boundary="00000000000096110f05b8a44c1d" --00000000000096110f05b8a44c1d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Right, then don't bother with deb. The level of support for deb in Yocto is significantly less than for rpm/ipk, and we do not require feature parity for all three. Thanks, Alex On Mon, 11 Jan 2021 at 17:25, wrote: > Hello Alex, > > > > Seems to not be so quick to change for the deb packages. First I had > issues building the sdk on > the master branch as it runs some postinst scripts that leak out of the > yocto sandbox and into > the host system [1]. Not sure if that=E2=80=99s a recent regression as I= don=E2=80=99t > normally use deb packages > in Yocto. > > > > Then I found that apt will fight over a lock file written into the > recipe-sysroots-native directory [2]. > I am not so familiar with apt-get to resolve that trivially. I=E2=80=99l= l try to > re-visit it another day. > > > > Kind regards, > > Michael > > > > [1] > > Setting up nativesdk-shadow (4.8.1-r0) ... > > mkdir: cannot create directory =E2=80=98/usr/lib/opkg=E2=80=99: Permissi= on denied > > dpkg: error processing package nativesdk-shadow (--configure): > > installed nativesdk-shadow package post-installation script subprocess > returned error exit status 1 > > > > [2] > > E: Could not get lock > /nvme/poky/build-deb/tmp/work/qemux86_64-poky-linux/core-image-minimal/1= .0-r0/recipe-sysroot-native/var/cache/apt/archives/lock > - open (11: Resource temporarily unavailable) > > > > -- > > *BMW Car IT GmbH* > Michael Ho > Spezialist Entwicklung =E2=80=93 Build and Release Engineering > Lise-Meitner-Stra=C3=9Fe 14 > 89081 Ulm > > Tel.: =C2=AD+49-731-37804-071 > > Mobil: +49-152-54980-471 > Fax: +49-731-37804-001 > Mail: michael.ho@bmw-carit.de > Web: http://www.bmw-carit.de > ------------------------------------------------------------------------= - > BMW Car IT GmbH > Gesch=C3=A4ftsf=C3=BChrer: Kai-Uwe Balszuweit und Michael B=C3=B6ttrich > Sitz und Registergericht: M=C3=BCnchen HRB 134810 > ------------------------------------------------------------------------= - > > > > > > *From: *Alexander Kanavin > *Date: *Monday, 11. January 2021 at 15:45 > *To: *"Ho Michael, JC-3UL" > *Cc: *OE-core > *Subject: *Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in > parallel to do_rootfs > > > > If you can fix the deb packaging easily, then please do so, but it is no= t > mandatory, and don't spend too much time if it turns out to be a time si= nk. > > > > Alex > > > > On Mon, 11 Jan 2021 at 15:01, wrote: > > Hello Alexander, > > > > Did a quick test and it seems the rpm packaging classes don=E2=80=99t ha= ve this > issue. The rootfs_deb bbclass however has the same lock files like with > rootfs_ipk. > > Should I try to patch this too to keep them aligned? > > > > Kind regards, > > Michael > > > > -- > > *BMW Car IT GmbH* > Michael Ho > Spezialist Entwicklung =E2=80=93 Build and Release Engineering > Lise-Meitner-Stra=C3=9Fe 14 > 89081 Ulm > > Tel.: =C2=AD+49-731-37804-071 > > Mobil: +49-152-54980-471 > Fax: +49-731-37804-001 > Mail: michael.ho@bmw-carit.de > Web: http://www.bmw-carit.de > ------------------------------------------------------------------------= - > BMW Car IT GmbH > Gesch=C3=A4ftsf=C3=BChrer: Kai-Uwe Balszuweit und Michael B=C3=B6ttrich > Sitz und Registergericht: M=C3=BCnchen HRB 134810 > ------------------------------------------------------------------------= - > > > > > > *From: *Alexander Kanavin > *Date: *Monday, 11. January 2021 at 13:46 > *To: *"Ho Michael, JC-3UL" > *Cc: *OE-core > *Subject: *Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in > parallel to do_rootfs > > > > Does deb/rpm packaging already have this feature? > > > > Alex > > > > On Mon, 11 Jan 2021 at 13:45, Michael Ho wrote: > > From: Michael Ho > > Switch do_populate_sdk for the ipk package manager to use a separate tar= get > opkg config file and separate the lockfiles restricting do_rootfs and > do_populate_sdk from running in parallel. > > This way if an image recipe includes a dependency to do_populate_sdk by > default then it will run in parallel to do_rootfs saving time compared t= o > the > sequential execution. > > Signed-off-by: Michael Ho > --- > meta/classes/package_ipk.bbclass | 1 + > meta/classes/rootfs_ipk.bbclass | 4 ++-- > meta/lib/oe/package_manager/ipk/sdk.py | 6 ++++++ > 3 files changed, 9 insertions(+), 2 deletions(-) > > diff --git a/meta/classes/package_ipk.bbclass > b/meta/classes/package_ipk.bbclass > index 79cb36c..d31dba0 100644 > --- a/meta/classes/package_ipk.bbclass > +++ b/meta/classes/package_ipk.bbclass > @@ -4,6 +4,7 @@ IMAGE_PKGTYPE ?=3D "ipk" > > IPKGCONF_TARGET =3D "${WORKDIR}/opkg.conf" > IPKGCONF_SDK =3D "${WORKDIR}/opkg-sdk.conf" > +IPKGCONF_SDK_TARGET =3D "${WORKDIR}/opkg-sdk-target.conf" > > PKGWRITEDIRIPK =3D "${WORKDIR}/deploy-ipks" > > diff --git a/meta/classes/rootfs_ipk.bbclass > b/meta/classes/rootfs_ipk.bbclass > index f1e0219..245c256 100644 > --- a/meta/classes/rootfs_ipk.bbclass > +++ b/meta/classes/rootfs_ipk.bbclass > @@ -14,8 +14,8 @@ do_rootfs[recrdeptask] +=3D "do_package_write_ipk > do_package_qa" > do_rootfs[vardeps] +=3D "PACKAGE_FEED_URIS PACKAGE_FEED_BASE_PATHS > PACKAGE_FEED_ARCHS" > > do_rootfs[lockfiles] +=3D "${WORKDIR}/ipk.lock" > -do_populate_sdk[lockfiles] +=3D "${WORKDIR}/ipk.lock" > -do_populate_sdk_ext[lockfiles] +=3D "${WORKDIR}/ipk.lock" > +do_populate_sdk[lockfiles] +=3D "${WORKDIR}/sdk-ipk.lock" > +do_populate_sdk_ext[lockfiles] +=3D "${WORKDIR}/sdk-ipk.lock" > > OPKG_PREPROCESS_COMMANDS =3D "" > > diff --git a/meta/lib/oe/package_manager/ipk/sdk.py > b/meta/lib/oe/package_manager/ipk/sdk.py > index 37af034..e2ca415 100644 > --- a/meta/lib/oe/package_manager/ipk/sdk.py > +++ b/meta/lib/oe/package_manager/ipk/sdk.py > @@ -14,6 +14,12 @@ class PkgSdk(Sdk): > def __init__(self, d, manifest_dir=3DNone): > super(PkgSdk, self).__init__(d, manifest_dir) > > + # In sdk_list_installed_packages the call to opkg is hardcoded = to > + # always use IPKGCONF_TARGET and there's no exposed API to chan= ge > this > + # so simply override IPKGCONF_TARGET to use this separated conf= ig > file. > + ipkgconf_sdk_target =3D d.getVar("IPKGCONF_SDK_TARGET") > + d.setVar("IPKGCONF_TARGET", ipkgconf_sdk_target) > + > self.target_conf =3D self.d.getVar("IPKGCONF_TARGET") > self.host_conf =3D self.d.getVar("IPKGCONF_SDK") > > -- > 2.7.4 > > >=20 > > --00000000000096110f05b8a44c1d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Right, then don't bother with deb. The level of s= upport for deb in Yocto is significantly less than for rpm/ipk, and we do n= ot require feature parity for all three.

Thanks,
Alex

On Mon, 11 Jan 2021 at 17:25, <Michael.Ho@bmw.de> wrote:

Hello Alex,

=C2=A0

Seems to not be so quick to ch= ange for the deb packages. First I had issues building the sdk on
the master branch as it runs some postinst scripts that leak out of the yo= cto sandbox and into
the host system [1]. Not sure if that=E2=80=99s a recent regression as I d= on=E2=80=99t normally use deb packages
in Yocto.

=C2=A0

Then I found that apt will fig= ht over a lock file written into the recipe-sysroots-native directory [2].<= br> I am not so familiar with apt-get to resolve that trivially. I=E2=80=99ll = try to re-visit it another day.

=C2=A0

Kind regards,

Michael

=C2=A0

[1]

Setting up nativesdk-shadow (4= .8.1-r0) ...

mkdir: cannot create directory= =E2=80=98/usr/lib/opkg=E2=80=99: Permission denied

dpkg: error processing package= nativesdk-shadow (--configure):

=C2=A0installed nativesdk-shad= ow package post-installation script subprocess returned error exit status 1=

=C2=A0

[2]

E: Could not get lock /nvme/po= ky/build-deb/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/recip= e-sysroot-native/var/cache/apt/archives/lock - open (11: Resource temporari= ly unavailable)

=C2=A0

--=C2=A0<= /p>

BMW Car IT Gmb= H
Michael Ho
Spezialist Entwicklung =E2=80=93 Build and Release Engineering
Lise-Meitner-Str= a=C3=9Fe 14
89081 Ulm=

Tel.:=C2=A0=C2= =AD+49-731-37804-071

Mobil: +49-152-54980-471
Fax: +49-731-378= 04-001
Mail:=C2=A0michael.ho@bmw-carit.de= =C2=A0
Web:=C2=A0http://www.bmw-carit.de
----------------= ---------------------------------------------------------
BMW Car IT GmbH<= /span>
Gesch=C3=A4ftsf= =C3=BChrer:=C2=A0Kai-Uwe Balszuweit und=C2=A0Michael B=C3=B6ttrich
Sitz und Registe= rgericht: M=C3=BCnchen HRB 134810
----------------= ---------------------------------------------------------

=C2=A0

=C2=A0

From:= Alexander Kanavin &l= t;alex.kanavin@= gmail.com>
Date: Monday, 11. January 2021 at 15:45
To: "Ho Michael, JC-3UL" <Michael.Ho@bmw.de>
Cc: OE-core <openembedded-core@lists.openembedded.org>=
Subject: Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in= parallel to do_rootfs

=C2=A0

If you can fix the deb packaging easily, then pleas= e do so, but it is not mandatory, and don't spend too much time if it t= urns out to be a time sink.

=C2=A0

Alex

=C2=A0

On Mon, 11 Jan 2021 at 15:01, <Michael.Ho@bmw.de> wrote:

Hello Alexander,=

=C2=A0

D= id a quick test and it seems the rpm packaging classes don=E2=80=99t have t= his issue. The rootfs_deb bbclass however has the same lock files like with= rootfs_ipk.

Should I try to patch this too= to keep them aligned?

=C2=A0

Kind regards,=

Michael

=C2=A0

--=C2=A0<= /p>

BMW Car IT Gmb= H
Michael Ho
Spezialist Entwicklung =E2=80=93 Build and Release Engineering
Lise-Meitner-Str= a=C3=9Fe 14
89081 Ulm=

Tel.:=C2=A0=C2= =AD+49-731-37804-071

Mobil: +49-152-54980-471
Fax: +49-731-378= 04-001
Mail:=C2=A0michael.ho@bmw-carit.de= =C2=A0
Web:=C2=A0http://www.bmw-carit.de
----------------= ---------------------------------------------------------
BMW Car IT GmbH<= /span>
Gesch=C3=A4ftsf= =C3=BChrer:=C2=A0Kai-Uwe Balszuweit und=C2=A0Michael B=C3=B6ttrich
Sitz und Registe= rgericht: M=C3=BCnchen HRB 134810
----------------= ---------------------------------------------------------

=C2=A0

=C2=A0

From: Alexander Kanavin &l= t;alex.kanavin@= gmail.com>
Date: Monday, 11. January 2021 at 13:46
To: "Ho Michael, JC-3UL" <Michael.Ho@bmw.de>
Cc: OE-core <openembedded-core@lists.openembedded.org>=
Subject: Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in= parallel to do_rootfs

=C2=A0

Does deb/rpm packaging already have this feature?

=C2=A0

Alex

=C2=A0

On Mon, 11 Jan 2021 at 13:45, Michael Ho <Michael.Ho@bmw.de> = wrote:

From: Michael Ho <<= a href=3D"mailto:Michael.Ho@bmw.de" target=3D"_blank">Michael.Ho@bmw.de= >

Switch do_populate_sdk for the ipk package manager to use a separate targe= t
opkg config file and separate the lockfiles restricting do_rootfs and
do_populate_sdk from running in parallel.

This way if an image recipe includes a dependency to do_populate_sdk by default then it will run in parallel to do_rootfs saving time compared to = the
sequential execution.

Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
---
=C2=A0meta/classes/package_ipk.bbclass=C2=A0 =C2=A0 =C2=A0 =C2=A0| 1 +
=C2=A0meta/classes/rootfs_ipk.bbclass=C2=A0 =C2=A0 =C2=A0 =C2=A0 | 4 ++--<= br> =C2=A0meta/lib/oe/package_manager/ipk/sdk.py | 6 ++++++
=C2=A03 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.b= bclass
index 79cb36c..d31dba0 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -4,6 +4,7 @@ IMAGE_PKGTYPE ?=3D "ipk"

=C2=A0IPKGCONF_TARGET =3D "${WORKDIR}/opkg.conf"
=C2=A0IPKGCONF_SDK =3D=C2=A0 "${WORKDIR}/opkg-sdk.conf"
+IPKGCONF_SDK_TARGET =3D "${WORKDIR}/opkg-sdk-target.conf"

=C2=A0PKGWRITEDIRIPK =3D "${WORKDIR}/deploy-ipks"

diff --git a/meta/classes/rootfs_ipk.bbclass b/meta/classes/rootfs_ipk.bbc= lass
index f1e0219..245c256 100644
--- a/meta/classes/rootfs_ipk.bbclass
+++ b/meta/classes/rootfs_ipk.bbclass
@@ -14,8 +14,8 @@ do_rootfs[recrdeptask] +=3D "do_package_write_ipk d= o_package_qa"
=C2=A0do_rootfs[vardeps] +=3D "PACKAGE_FEED_URIS PACKAGE_FEED_BASE_PA= THS PACKAGE_FEED_ARCHS"

=C2=A0do_rootfs[lockfiles] +=3D "${WORKDIR}/ipk.lock"
-do_populate_sdk[lockfiles] +=3D "${WORKDIR}/ipk.lock"
-do_populate_sdk_ext[lockfiles] +=3D "${WORKDIR}/ipk.lock"
+do_populate_sdk[lockfiles] +=3D "${WORKDIR}/sdk-ipk.lock"
+do_populate_sdk_ext[lockfiles] +=3D "${WORKDIR}/sdk-ipk.lock"
=C2=A0OPKG_PREPROCESS_COMMANDS =3D ""

diff --git a/meta/lib/oe/package_manager/ipk/sdk.py b/meta/lib/oe/package_= manager/ipk/sdk.py
index 37af034..e2ca415 100644
--- a/meta/lib/oe/package_manager/ipk/sdk.py
+++ b/meta/lib/oe/package_manager/ipk/sdk.py
@@ -14,6 +14,12 @@ class PkgSdk(Sdk):
=C2=A0 =C2=A0 =C2=A0def __init__(self, d, manifest_dir=3DNone):
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0super(PkgSdk, self).__init__(d, manifest= _dir)

+=C2=A0 =C2=A0 =C2=A0 =C2=A0 # In sdk_list_installed_packages the call to = opkg is hardcoded to
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 # always use IPKGCONF_TARGET and there's = no exposed API to change this
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 # so simply override IPKGCONF_TARGET to use t= his separated config file.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 ipkgconf_sdk_target =3D d.getVar("IPKGCO= NF_SDK_TARGET")
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 d.setVar("IPKGCONF_TARGET", ipkgcon= f_sdk_target)
+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.target_conf =3D self.d.getVar("= ;IPKGCONF_TARGET")
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.host_conf =3D self.d.getVar("I= PKGCONF_SDK")

--
2.7.4


--00000000000096110f05b8a44c1d--