[PATCH 0/1] autoconf-archive: inherit allarch and fix package splitting

[PATCH 0/1] autoconf-archive: inherit allarch and fix package splitting

[Chen Qi]

The following changes since commit 4db3915372228bbe045947333ec3a29785e3da94:

  dev-manual: Updated bitbake-layers overview section. (2017-11-12 06:51:20 +0000)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib ChenQi/autoconf-archive-allarch
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/autoconf-archive-allarch

Chen Qi (1):
  autoconf-archive: inherit allarch and fix package splitting

 .../autoconf-archive/autoconf-archive_2016.09.16.bb                 | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

-- 
1.9.1

[PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting

[Chen Qi]

autoconf-archive is a set of common m4 macros, it should be allarch,
just like it is on other distros.

Also fix package splitting. With current package splitting,
autoconf-archive itself contains almost nothing, the m4 macros are
contained in its dev package. However, autoconf-archive should
contain those m4 marcos itself.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 .../autoconf-archive/autoconf-archive_2016.09.16.bb                 | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
index 104dc38..89d57ac 100644
--- a/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
+++ b/meta/recipes-devtools/autoconf-archive/autoconf-archive_2016.09.16.bb
@@ -9,6 +9,10 @@ SRC_URI = "${GNU_MIRROR}/${BPN}/${BPN}-${PV}.tar.xz"
 SRC_URI[md5sum] = "bf19d4cddce260b3c3e1d51d42509071"
 SRC_URI[sha256sum] = "e8f2efd235f842bad2f6938bf4a72240a5e5fcd248e8444335e63beb60fabd82"
 
-inherit autotools
+inherit autotools allarch
+
+PACKAGES = "${PN} ${PN}-doc"
+
+FILES_${PN} += "${datadir}/aclocal"
 
 BBCLASSEXTEND = "native nativesdk"
-- 
1.9.1

Re: [PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting

[André Draszik]

Hi,

On Mon, 2017-11-13 at 13:26 +0800, Chen Qi wrote:
> autoconf-archive is a set of common m4 macros, it should be allarch,
> just like it is on other distros.

I have found previously that when changing a recipe to produce more generic
output, like in this case, and the old binary package is still in deploy/,
then the package manager would always pick the more specific version, even
if a more recent (by package version) exists in deploy. In my case I was
moving from PACKAGE_ARCH=${MACHINE_ARCH} to PACKAGE_ARCH=${TUNE_PKGARCH},
but I guess it'll be the same here.

What is the correct way to deal with that other that cleansstate?

Cheers,
Andre'

Re: [PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1330 bytes --]

Is this during do_rootfs or are you seeing this issue when upgrading on the
target as well?

I haven't found the bug for this but IIRC it's because do_rootfs is using
--prefer-arch-to-version parameter.
meta/classes/package_ipk.bbclass:OPKG_ARGS += "--force_postinstall
--prefer-arch-to-version"
which causes this undesired behavior.

On Mon, Nov 13, 2017 at 9:54 AM, André Draszik <git@andred.net> wrote:

> Hi,
>
> On Mon, 2017-11-13 at 13:26 +0800, Chen Qi wrote:
> > autoconf-archive is a set of common m4 macros, it should be allarch,
> > just like it is on other distros.
>
> I have found previously that when changing a recipe to produce more generic
> output, like in this case, and the old binary package is still in deploy/,
> then the package manager would always pick the more specific version, even
> if a more recent (by package version) exists in deploy. In my case I was
> moving from PACKAGE_ARCH=${MACHINE_ARCH} to PACKAGE_ARCH=${TUNE_PKGARCH},
> but I guess it'll be the same here.
>
> What is the correct way to deal with that other that cleansstate?
>
> Cheers,
> Andre'
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>

[-- Attachment #2: Type: text/html, Size: 2000 bytes --]

opkg --prefer-arch-to-version (was: Re: [PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting)

[André Draszik]

Only during do_rootfs indeed (I don't ever do upgrades on the target using
opkg).

Is there a reason to still specify prefer-arch-to-version? I'd actually
expect the exact opposite compared to the original commit message's
statement, unless the version numbers are identical.

Also, I would argue that the defaults during do_rootfs and when using opkg
on the target should be the same...

Cheers,
Andre'


On Mon, 2017-11-13 at 11:12 +0100, Martin Jansa wrote:
> Is this during do_rootfs or are you seeing this issue when upgrading on
> the
> target as well?
> 
> I haven't found the bug for this but IIRC it's because do_rootfs is using
> --prefer-arch-to-version parameter.
> meta/classes/package_ipk.bbclass:OPKG_ARGS += "--force_postinstall
> --prefer-arch-to-version"
> which causes this undesired behavior.
> 
> On Mon, Nov 13, 2017 at 9:54 AM, André Draszik <git@andred.net> wrote:
> 
> > Hi,
> > 
> > On Mon, 2017-11-13 at 13:26 +0800, Chen Qi wrote:
> > > autoconf-archive is a set of common m4 macros, it should be allarch,
> > > just like it is on other distros.
> > 
> > I have found previously that when changing a recipe to produce more
> > generic
> > output, like in this case, and the old binary package is still in
> > deploy/,
> > then the package manager would always pick the more specific version,
> > even
> > if a more recent (by package version) exists in deploy. In my case I was
> > moving from PACKAGE_ARCH=${MACHINE_ARCH} to
> > PACKAGE_ARCH=${TUNE_PKGARCH},
> > but I guess it'll be the same here.
> > 
> > What is the correct way to deal with that other that cleansstate?
> > 
> > Cheers,
> > Andre'
> > 
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> > 

Re: [PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting

[Robert Yang]

On 11/13/2017 04:54 PM, André Draszik wrote:
> Hi,
> 
> On Mon, 2017-11-13 at 13:26 +0800, Chen Qi wrote:
>> autoconf-archive is a set of common m4 macros, it should be allarch,
>> just like it is on other distros.
> 
> I have found previously that when changing a recipe to produce more generic
> output, like in this case, and the old binary package is still in deploy/,
> then the package manager would always pick the more specific version, even
> if a more recent (by package version) exists in deploy. In my case I was
> moving from PACKAGE_ARCH=${MACHINE_ARCH} to PACKAGE_ARCH=${TUNE_PKGARCH},
> but I guess it'll be the same here.

I think that it is because of the priority MACHINE_ARCH > TUNE_PKGARCH > allarch,
which is the right behavior.

> 
> What is the correct way to deal with that other that cleansstate?

There is no way to fix it except cleansstate AFAIK.

// Robert

> 
> Cheers,
> Andre'
> 

Re: opkg --prefer-arch-to-version (was: Re: [PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting)

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2599 bytes --]

I don't remember the details, but IIRC it was libdrm or something like that
for some MACHINE which was providing older version of libdrm with
MACHINE_ARCH with better support for that MACHINE.

So this controversial patch was implemented for opkg to support that
use-case and the default behavior was changed ignoring complains from me
and Koen that this behavior is wrong.

I've tried to find some e-mail threads about it, part of that discussion is
in:
https://patchwork.openembedded.org/patch/37461/

On Mon, Nov 13, 2017 at 11:34 AM, André Draszik <git@andred.net> wrote:

> Only during do_rootfs indeed (I don't ever do upgrades on the target using
> opkg).
>
> Is there a reason to still specify prefer-arch-to-version? I'd actually
> expect the exact opposite compared to the original commit message's
> statement, unless the version numbers are identical.
>
> Also, I would argue that the defaults during do_rootfs and when using opkg
> on the target should be the same...
>
> Cheers,
> Andre'
>
>
> On Mon, 2017-11-13 at 11:12 +0100, Martin Jansa wrote:
> > Is this during do_rootfs or are you seeing this issue when upgrading on
> > the
> > target as well?
> >
> > I haven't found the bug for this but IIRC it's because do_rootfs is using
> > --prefer-arch-to-version parameter.
> > meta/classes/package_ipk.bbclass:OPKG_ARGS += "--force_postinstall
> > --prefer-arch-to-version"
> > which causes this undesired behavior.
> >
> > On Mon, Nov 13, 2017 at 9:54 AM, André Draszik <git@andred.net> wrote:
> >
> > > Hi,
> > >
> > > On Mon, 2017-11-13 at 13:26 +0800, Chen Qi wrote:
> > > > autoconf-archive is a set of common m4 macros, it should be allarch,
> > > > just like it is on other distros.
> > >
> > > I have found previously that when changing a recipe to produce more
> > > generic
> > > output, like in this case, and the old binary package is still in
> > > deploy/,
> > > then the package manager would always pick the more specific version,
> > > even
> > > if a more recent (by package version) exists in deploy. In my case I
> was
> > > moving from PACKAGE_ARCH=${MACHINE_ARCH} to
> > > PACKAGE_ARCH=${TUNE_PKGARCH},
> > > but I guess it'll be the same here.
> > >
> > > What is the correct way to deal with that other that cleansstate?
> > >
> > > Cheers,
> > > Andre'
> > >
> > > --
> > > _______________________________________________
> > > Openembedded-core mailing list
> > > Openembedded-core@lists.openembedded.org
> > > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> > >
>

[-- Attachment #2: Type: text/html, Size: 3608 bytes --]

Re: [PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting

[André Draszik]

On Mon, 2017-11-13 at 19:54 +0800, Robert Yang wrote:
> 
> On 11/13/2017 04:54 PM, André Draszik wrote:
> > Hi,
> > 
> > On Mon, 2017-11-13 at 13:26 +0800, Chen Qi wrote:
> > > autoconf-archive is a set of common m4 macros, it should be allarch,
> > > just like it is on other distros.
> > 
> > I have found previously that when changing a recipe to produce more
> > generic
> > output, like in this case, and the old binary package is still in
> > deploy/,
> > then the package manager would always pick the more specific version,
> > even
> > if a more recent (by package version) exists in deploy. In my case I was
> > moving from PACKAGE_ARCH=${MACHINE_ARCH} to
> > PACKAGE_ARCH=${TUNE_PKGARCH},
> > but I guess it'll be the same here.
> 
> I think that it is because of the priority MACHINE_ARCH > TUNE_PKGARCH >
> allarch,
> which is the right behavior.

I'd say this patch here is a good example why it's not the right behaviour.

This patch is making autoconf-archive allarch, i.e. lower priority than
before. In other words, anybody who has the previous TUNE_PKGARCH version
already will never get an update, even if the version number is bumped.

Cheers,
Andre'

Re: [PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 1656 bytes --]

But don't we remove the old version from the feeds, or does that only
happen if the arch matches?

Ross

On 14 November 2017 at 10:16, André Draszik <git@andred.net> wrote:

> On Mon, 2017-11-13 at 19:54 +0800, Robert Yang wrote:
> >
> > On 11/13/2017 04:54 PM, André Draszik wrote:
> > > Hi,
> > >
> > > On Mon, 2017-11-13 at 13:26 +0800, Chen Qi wrote:
> > > > autoconf-archive is a set of common m4 macros, it should be allarch,
> > > > just like it is on other distros.
> > >
> > > I have found previously that when changing a recipe to produce more
> > > generic
> > > output, like in this case, and the old binary package is still in
> > > deploy/,
> > > then the package manager would always pick the more specific version,
> > > even
> > > if a more recent (by package version) exists in deploy. In my case I
> was
> > > moving from PACKAGE_ARCH=${MACHINE_ARCH} to
> > > PACKAGE_ARCH=${TUNE_PKGARCH},
> > > but I guess it'll be the same here.
> >
> > I think that it is because of the priority MACHINE_ARCH > TUNE_PKGARCH >
> > allarch,
> > which is the right behavior.
>
> I'd say this patch here is a good example why it's not the right behaviour.
>
> This patch is making autoconf-archive allarch, i.e. lower priority than
> before. In other words, anybody who has the previous TUNE_PKGARCH version
> already will never get an update, even if the version number is bumped.
>
> Cheers,
> Andre'
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>

[-- Attachment #2: Type: text/html, Size: 2446 bytes --]

Re: [PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 908 bytes --]

On 14 November 2017 at 11:34, Burton, Ross <ross.burton@intel.com> wrote:

> But don't we remove the old version from the feeds, or does that only
> happen if the arch matches?
>

Well the feed pruning is working correctly:

With master from deploy/:

$ find . -name autoconf-archive*
./rpm/corei7_64/autoconf-archive-dev-2016.09.16-r0.corei7_64.rpm
./rpm/corei7_64/autoconf-archive-2016.09.16-r0.corei7_64.rpm
./rpm/corei7_64/autoconf-archive-doc-2016.09.16-r0.corei7_64.rpm
./rpm/corei7_64/autoconf-archive-dbg-2016.09.16-r0.corei7_64.rpm

With this patch applied:

$ find . -name autoconf-archive*rpm
./rpm/noarch/autoconf-archive-2016.09.16-r0.noarch.rpm
./rpm/noarch/autoconf-archive-doc-2016.09.16-r0.noarch.rpm

So, is the allarch < tune < machine logic rootfs time only, or is it used
on target too?  Forgive my ignorance, I haven't actually used a feed for a
while.

Ross

[-- Attachment #2: Type: text/html, Size: 1493 bytes --]

Re: [PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting

[André Draszik]

On Tue, 2017-11-14 at 11:41 +0000, Burton, Ross wrote:
> On 14 November 2017 at 11:34, Burton, Ross <ross.burton@intel.com> wrote:
> 
> > But don't we remove the old version from the feeds, or does that only
> > happen if the arch matches?
> > 
> 
> Well the feed pruning is working correctly:

Not for me. I created a dummy package foo_0.1.bb, with PACKAGE_ARCH =
"${MACHINE_ARCH}"

after bitbake foo:
$ build.pyro > find tmp/deploy/ipk/ -name foo*ipk
tmp/deploy/ipk/tgm_r3/f/foo/foo-dbg_0.1-r0_tgm_r3.ipk
tmp/deploy/ipk/tgm_r3/f/foo/foo-dev_0.1-r0_tgm_r3.ipk
tmp/deploy/ipk/tgm_r3/f/foo/foo_0.1-r0_tgm_r3.ipk

Removing the PACKAGE_ARCH line from that recipe and rebuilding foo gives me:

$ build.pyro > find tmp/deploy/ipk/ -name foo*ipk
tmp/deploy/ipk/mips32r2el-24kec-nf/f/foo/foo-dbg_0.1-r0_mips32r2el-24kec-
nf.ipk
tmp/deploy/ipk/mips32r2el-24kec-nf/f/foo/foo-dev_0.1-r0_mips32r2el-24kec-
nf.ipk
tmp/deploy/ipk/mips32r2el-24kec-nf/f/foo/foo_0.1-r0_mips32r2el-24kec-nf.ipk
tmp/deploy/ipk/tgm_r3/f/foo/foo-dbg_0.1-r0_tgm_r3.ipk
tmp/deploy/ipk/tgm_r3/f/foo/foo-dev_0.1-r0_tgm_r3.ipk
tmp/deploy/ipk/tgm_r3/f/foo/foo_0.1-r0_tgm_r3.ipk


If I bump the version, I am still left with the MACHINE_ARCH packages
(doesn't matter if I bump the version before or after the change to
PACKAGE_ARCH):
$ build.pyro > find tmp/deploy/ipk/ -name foo*ipk
tmp/deploy/ipk/mips32r2el-24kec-nf/f/foo/foo-dbg_0.2-r0_mips32r2el-24kec-
nf.ipk
tmp/deploy/ipk/mips32r2el-24kec-nf/f/foo/foo-dev_0.2-r0_mips32r2el-24kec-
nf.ipk
tmp/deploy/ipk/mips32r2el-24kec-nf/f/foo/foo_0.2-r0_mips32r2el-24kec-nf.ipk
tmp/deploy/ipk/tgm_r3/f/foo/foo-dbg_0.1-r0_tgm_r3.ipk
tmp/deploy/ipk/tgm_r3/f/foo/foo-dev_0.1-r0_tgm_r3.ipk
tmp/deploy/ipk/tgm_r3/f/foo/foo_0.1-r0_tgm_r3.ipk

So of course do_rootfs will pick the outdated version.

Is that an ipk / OPKG issue? Should something be removing the old packages
from deploy?


Cheers,
Andre'

Re: [PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting

[Robert Yang]

On 11/14/2017 10:02 PM, André Draszik wrote:
> On Tue, 2017-11-14 at 11:41 +0000, Burton, Ross wrote:
>> On 14 November 2017 at 11:34, Burton, Ross <ross.burton@intel.com> wrote:
>>
>>> But don't we remove the old version from the feeds, or does that only
>>> happen if the arch matches?
>>>
>>
>> Well the feed pruning is working correctly:
> 
> Not for me. I created a dummy package foo_0.1.bb, with PACKAGE_ARCH =
> "${MACHINE_ARCH}"
> 
> after bitbake foo:
> $ build.pyro > find tmp/deploy/ipk/ -name foo*ipk
> tmp/deploy/ipk/tgm_r3/f/foo/foo-dbg_0.1-r0_tgm_r3.ipk
> tmp/deploy/ipk/tgm_r3/f/foo/foo-dev_0.1-r0_tgm_r3.ipk
> tmp/deploy/ipk/tgm_r3/f/foo/foo_0.1-r0_tgm_r3.ipk
> 
> Removing the PACKAGE_ARCH line from that recipe and rebuilding foo gives me:
> 
> $ build.pyro > find tmp/deploy/ipk/ -name foo*ipk
> tmp/deploy/ipk/mips32r2el-24kec-nf/f/foo/foo-dbg_0.1-r0_mips32r2el-24kec-
> nf.ipk
> tmp/deploy/ipk/mips32r2el-24kec-nf/f/foo/foo-dev_0.1-r0_mips32r2el-24kec-
> nf.ipk
> tmp/deploy/ipk/mips32r2el-24kec-nf/f/foo/foo_0.1-r0_mips32r2el-24kec-nf.ipk
> tmp/deploy/ipk/tgm_r3/f/foo/foo-dbg_0.1-r0_tgm_r3.ipk
> tmp/deploy/ipk/tgm_r3/f/foo/foo-dev_0.1-r0_tgm_r3.ipk
> tmp/deploy/ipk/tgm_r3/f/foo/foo_0.1-r0_tgm_r3.ipk
> 
> 
> If I bump the version, I am still left with the MACHINE_ARCH packages
> (doesn't matter if I bump the version before or after the change to
> PACKAGE_ARCH):
> $ build.pyro > find tmp/deploy/ipk/ -name foo*ipk
> tmp/deploy/ipk/mips32r2el-24kec-nf/f/foo/foo-dbg_0.2-r0_mips32r2el-24kec-
> nf.ipk
> tmp/deploy/ipk/mips32r2el-24kec-nf/f/foo/foo-dev_0.2-r0_mips32r2el-24kec-
> nf.ipk
> tmp/deploy/ipk/mips32r2el-24kec-nf/f/foo/foo_0.2-r0_mips32r2el-24kec-nf.ipk
> tmp/deploy/ipk/tgm_r3/f/foo/foo-dbg_0.1-r0_tgm_r3.ipk
> tmp/deploy/ipk/tgm_r3/f/foo/foo-dev_0.1-r0_tgm_r3.ipk
> tmp/deploy/ipk/tgm_r3/f/foo/foo_0.1-r0_tgm_r3.ipk
> 
> So of course do_rootfs will pick the outdated version.
> 
> Is that an ipk / OPKG issue? Should something be removing the old packages
> from deploy?

I tried PACKAGE_CLASSES = "package_ipk", I got the same result as ross:

* Before Qi's patch applied:
$ find tmp/deploy/ipk/ | grep autoconf-a
tmp/deploy/ipk/i586/autoconf-archive-dev_2017.09.28-r0_i586.ipk
tmp/deploy/ipk/i586/autoconf-archive_2017.09.28-r0_i586.ipk
tmp/deploy/ipk/i586/autoconf-archive-dbg_2017.09.28-r0_i586.ipk
tmp/deploy/ipk/i586/autoconf-archive-doc_2017.09.28-r0_i586.ipk

* After Qi's patch applied:
$ find tmp/deploy/ipk/ | grep autoconf-a
tmp/deploy/ipk/all/autoconf-archive_2017.09.28-r0_all.ipk
tmp/deploy/ipk/all/autoconf-archive-doc_2017.09.28-r0_all.ipk

I guess something is wrong with your build.

// Robert

> 
> 
> Cheers,
> Andre'
> 
> 

Re: [PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting

[André Draszik]

Hi,

On Wed, 2017-11-15 at 10:23 +0800, Robert Yang wrote:
> 
> On 11/14/2017 10:02 PM, André Draszik wrote:
> > Is that an ipk / OPKG issue? Should something be removing the old
> > packages
> > from deploy?
> 
> I tried PACKAGE_CLASSES = "package_ipk", I got the same result as ross:

OK, so I tried with autoconf-archive myself now. I get the same result as
you two.

But - When moving a recipe from MACHINE_ARCH to TUNE_PKGARCH then the old
package is not removed.

[...]
> I guess something is wrong with your build.

To summarise:
- moving a recipe from TUNE_PKGARCH to allarch removes the old package
- moving a recipe from MACHINE_ARCH to TUNE_PKGARCH *doesn't* remove the old
  package.

Where would I start looking?


Cheers,
Andre'

Re: [PATCH 1/1] autoconf-archive: inherit allarch and fix package splitting

[Robert Yang]

On 11/15/2017 05:06 PM, André Draszik wrote:
> Hi,
> 
> On Wed, 2017-11-15 at 10:23 +0800, Robert Yang wrote:
>>
>> On 11/14/2017 10:02 PM, André Draszik wrote:
>>> Is that an ipk / OPKG issue? Should something be removing the old
>>> packages
>>> from deploy?
>>
>> I tried PACKAGE_CLASSES = "package_ipk", I got the same result as ross:
> 
> OK, so I tried with autoconf-archive myself now. I get the same result as
> you two.
> 
> But - When moving a recipe from MACHINE_ARCH to TUNE_PKGARCH then the old
> package is not removed.
> 
> [...]
>> I guess something is wrong with your build.
> 
> To summarise:
> - moving a recipe from TUNE_PKGARCH to allarch removes the old package
> - moving a recipe from MACHINE_ARCH to TUNE_PKGARCH *doesn't* remove the old
>    package.
> 
> Where would I start looking?

It is removed by sstate.bbclass, take a look at sstate_clean() may help.

// Robert

> 
> 
> Cheers,
> Andre'
> 
>