opkg, opkg-config-base and opkg-collateral

opkg, opkg-config-base and opkg-collateral

[Paul Barker]

Hi all,

Does anyone know why the configuration files for opkg are split into
opkg-config-base (containing just '/etc/opkg/arch.conf') and
opkg-collateral (containing just '/etc/opkg/opkg.conf')? It looks like
the split dates back to openembedded classic.

If there isn't a good reason for this perhaps now would be a good time
to merge all this back into the 'opkg' recipe and package. I'm happy
to put the patch together, just checking if it sounds like a good idea
before I do the work.

Cheers,

-- 
Paul Barker

Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk

Re: opkg, opkg-config-base and opkg-collateral

[Richard Purdie]

On Tue, 2014-11-25 at 20:27 +0000, Paul Barker wrote:
> Hi all,
> 
> Does anyone know why the configuration files for opkg are split into
> opkg-config-base (containing just '/etc/opkg/arch.conf') and
> opkg-collateral (containing just '/etc/opkg/opkg.conf')? It looks like
> the split dates back to openembedded classic.
> 
> If there isn't a good reason for this perhaps now would be a good time
> to merge all this back into the 'opkg' recipe and package. I'm happy
> to put the patch together, just checking if it sounds like a good idea
> before I do the work.

I think at least one of the above was intended to allow distro specific
package feeds to be preconfigured as as such belonged as a standalone
config file.

The architecture file is also machine specific, we wouldn't want opkg
itself rebuilding for every machine so that is probably why its
separate.

Three different things on the other hand seems excessive. We probably
could survive with some merhing with opkg and the remainder being
machine specific.

Cheers,

Richard

Re: opkg, opkg-config-base and opkg-collateral

[Mike Looijmans]

On 11/26/2014 01:14 PM, Richard Purdie wrote:
> On Tue, 2014-11-25 at 20:27 +0000, Paul Barker wrote:
>> Hi all,
>>
>> Does anyone know why the configuration files for opkg are split into
>> opkg-config-base (containing just '/etc/opkg/arch.conf') and
>> opkg-collateral (containing just '/etc/opkg/opkg.conf')? It looks like
>> the split dates back to openembedded classic.
>>
>> If there isn't a good reason for this perhaps now would be a good time
>> to merge all this back into the 'opkg' recipe and package. I'm happy
>> to put the patch together, just checking if it sounds like a good idea
>> before I do the work.
>
> I think at least one of the above was intended to allow distro specific
> package feeds to be preconfigured as as such belonged as a standalone
> config file.
>
> The architecture file is also machine specific, we wouldn't want opkg
> itself rebuilding for every machine so that is probably why its
> separate.
>
> Three different things on the other hand seems excessive. We probably
> could survive with some merhing with opkg and the remainder being
> machine specific.

It's only excessive in its default state. Once you have a proper distribution 
with an online feed, it makes perfect sense to have separate packages:
- the opkg "core" files which are the same for everyone.
- the machine-specific list of supported architectures (maybe extended with 
distro-specific extras)
- the distro specific config files containing the location of the various 
feeds. These may also be machine specific, for example due to a package like 
Opera being only allowed to run on machines from a particular vendor.

I for one would not want to see any of these three merged.


Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Systems
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: (+31) (0) 499 33 69 79
Telefax:  (+31) (0) 499 33 69 70
E-mail: mike.looijmans@topic.nl
Website: www.topic.nl

Please consider the environment before printing this e-mail

Topic zoekt gedreven (embedded) software specialisten!
http://topic.nl/vacatures/topic-zoekt-software-engineers/

Re: opkg, opkg-config-base and opkg-collateral

[Paul Barker]

On 26 November 2014 at 12:14, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Tue, 2014-11-25 at 20:27 +0000, Paul Barker wrote:
> > Hi all,
> >
> > Does anyone know why the configuration files for opkg are split into
> > opkg-config-base (containing just '/etc/opkg/arch.conf') and
> > opkg-collateral (containing just '/etc/opkg/opkg.conf')? It looks like
> > the split dates back to openembedded classic.
> >
> > If there isn't a good reason for this perhaps now would be a good time
> > to merge all this back into the 'opkg' recipe and package. I'm happy
> > to put the patch together, just checking if it sounds like a good idea
> > before I do the work.
>
> I think at least one of the above was intended to allow distro specific
> package feeds to be preconfigured as as such belonged as a standalone
> config file.

That would probably be opkg-collateral as it pulls in a 'src' file
which is empty in oe-core.

However there's also the poky-feed-config-opkg recipe which provides
feed configurations in /etc/opkg/base-feeds.conf. It also fills in
/etc/opkg/arch.conf which means it would not be installable alongside
opkg-config-base.

I suggest the config settings are merged back into the opkg recipe,
removing opkg-collateral completely. We can then improve
poky-feed-config-opkg and rename it to something like opkg-feed-config
so that it's clear it's usable by distros other than poky. This
improved recipe would just contain /etc/opkg/feeds.conf (the 'base-'
filename prefix is unnecessary).

In terms of improving the feed config for opkg, by default I suggest
we just setup feeds for 'all', ${ALL_MULTILIB_PACKAGE_ARCHS} and
${MACHINE_ARCH}. This should avoid creating feed entries for
architectures which the target system supports but for which we never
create packages (for example, qemux86 lists 'all', 'any', 'noarch',
'x85', 'i586' and 'qemux86' as supported architectures but we only
ever create packages for 'all', 'i586' and 'qemux86'). By using a
bbappend a layer should be able to add to these defaults. The URI for
each package feed would be prefixed with ${OPKG_FEED_PREFIX} which
must be set in the distro config or local.conf. If OPKG_FEED_PREFIX is
not set, the recipe would create an empty base-feeds.conf file.

It would be possible with a bit of scripting to set the hold flag on
the opkg-feed-config package when it is installed so that even if a
new version of the package is compiled, the feeds for a target board
will not be automatically upgraded with 'opkg upgrade'. We could then
create a dist-upgrade script which removes the hold flag, upgrades
opkg-feed-config, and re-sets the hold flag. So to provide a new
version of a distro which users could upgrade to but are not
automatically upgraded to, you'd build a new version of
opkg-feed-config (probably by changing OPKG_FEED_PREFIX) and run the
dist-upgrade script on the target.

I'll put together RFC patches to show what I mean, I think that's a
pretty good explanation for now though.

>
> The architecture file is also machine specific, we wouldn't want opkg
> itself rebuilding for every machine so that is probably why its
> separate.
>

I didn't spot this but it does make sense. Perhaps opkg-config-base
should be renamed to opkg-arch-config to avoid future confusion.

> Three different things on the other hand seems excessive. We probably
> could survive with some merhing with opkg and the remainder being
> machine specific.
>

I'd keep three, but refactor them as I've described above:
- opkg: Includes '/etc/opkg/opkg.conf'
- opkg-arch-config: Includes '/etc/opkg/arch.conf', machine specific
- opkg-feed-config: Includes '/etc/opkg/feeds.conf', distro and
possibly machine specific

> Cheers,
>
> Richard
>

Thanks,

-- 
Paul Barker

Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk

Re: opkg, opkg-config-base and opkg-collateral

[Paul Barker]

On 26 November 2014 at 13:43, Mike Looijmans <mike.looijmans@topic.nl> wrote:
> On 11/26/2014 01:14 PM, Richard Purdie wrote:
>>
>> On Tue, 2014-11-25 at 20:27 +0000, Paul Barker wrote:
>>>
>>> Hi all,
>>>
>>> Does anyone know why the configuration files for opkg are split into
>>> opkg-config-base (containing just '/etc/opkg/arch.conf') and
>>> opkg-collateral (containing just '/etc/opkg/opkg.conf')? It looks like
>>> the split dates back to openembedded classic.
>>>
>>> If there isn't a good reason for this perhaps now would be a good time
>>> to merge all this back into the 'opkg' recipe and package. I'm happy
>>> to put the patch together, just checking if it sounds like a good idea
>>> before I do the work.
>
> It's only excessive in its default state. Once you have a proper
> distribution with an online feed, it makes perfect sense to have separate
> packages:
> - the opkg "core" files which are the same for everyone.
> - the machine-specific list of supported architectures (maybe extended with
> distro-specific extras)
> - the distro specific config files containing the location of the various
> feeds. These may also be machine specific, for example due to a package like
> Opera being only allowed to run on machines from a particular vendor.
>
> I for one would not want to see any of these three merged.
>

I saw this while writing my reply to Richard's email and I think I've
accounted for these comments in that reply. I've definitely had second
thoughts about merging all three together due to machine and distro
specific bits, but I think we can improve the split between different
packages and recipes.

-- 
Paul Barker

Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk

Re: opkg, opkg-config-base and opkg-collateral

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 4558 bytes --]

On Wed, Nov 26, 2014 at 01:50:38PM +0000, Paul Barker wrote:
> On 26 November 2014 at 12:14, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> >
> > On Tue, 2014-11-25 at 20:27 +0000, Paul Barker wrote:
> > > Hi all,
> > >
> > > Does anyone know why the configuration files for opkg are split into
> > > opkg-config-base (containing just '/etc/opkg/arch.conf') and
> > > opkg-collateral (containing just '/etc/opkg/opkg.conf')? It looks like
> > > the split dates back to openembedded classic.
> > >
> > > If there isn't a good reason for this perhaps now would be a good time
> > > to merge all this back into the 'opkg' recipe and package. I'm happy
> > > to put the patch together, just checking if it sounds like a good idea
> > > before I do the work.
> >
> > I think at least one of the above was intended to allow distro specific
> > package feeds to be preconfigured as as such belonged as a standalone
> > config file.
> 
> That would probably be opkg-collateral as it pulls in a 'src' file
> which is empty in oe-core.
> 
> However there's also the poky-feed-config-opkg recipe which provides
> feed configurations in /etc/opkg/base-feeds.conf. It also fills in
> /etc/opkg/arch.conf which means it would not be installable alongside
> opkg-config-base.
> 
> I suggest the config settings are merged back into the opkg recipe,
> removing opkg-collateral completely. We can then improve
> poky-feed-config-opkg and rename it to something like opkg-feed-config
> so that it's clear it's usable by distros other than poky. This
> improved recipe would just contain /etc/opkg/feeds.conf (the 'base-'
> filename prefix is unnecessary).
> 
> In terms of improving the feed config for opkg, by default I suggest
> we just setup feeds for 'all', ${ALL_MULTILIB_PACKAGE_ARCHS} and
> ${MACHINE_ARCH}. This should avoid creating feed entries for
> architectures which the target system supports but for which we never
> create packages (for example, qemux86 lists 'all', 'any', 'noarch',
> 'x85', 'i586' and 'qemux86' as supported architectures but we only
> ever create packages for 'all', 'i586' and 'qemux86'). By using a
> bbappend a layer should be able to add to these defaults. The URI for
> each package feed would be prefixed with ${OPKG_FEED_PREFIX} which
> must be set in the distro config or local.conf. If OPKG_FEED_PREFIX is
> not set, the recipe would create an empty base-feeds.conf file.
> 
> It would be possible with a bit of scripting to set the hold flag on
> the opkg-feed-config package when it is installed so that even if a
> new version of the package is compiled, the feeds for a target board
> will not be automatically upgraded with 'opkg upgrade'. We could then
> create a dist-upgrade script which removes the hold flag, upgrades
> opkg-feed-config, and re-sets the hold flag. So to provide a new
> version of a distro which users could upgrade to but are not
> automatically upgraded to, you'd build a new version of
> opkg-feed-config (probably by changing OPKG_FEED_PREFIX) and run the
> dist-upgrade script on the target.
> 
> I'll put together RFC patches to show what I mean, I think that's a
> pretty good explanation for now though.
> 
> >
> > The architecture file is also machine specific, we wouldn't want opkg
> > itself rebuilding for every machine so that is probably why its
> > separate.
> >
> 
> I didn't spot this but it does make sense. Perhaps opkg-config-base
> should be renamed to opkg-arch-config to avoid future confusion.
> 
> > Three different things on the other hand seems excessive. We probably
> > could survive with some merhing with opkg and the remainder being
> > machine specific.
> >
> 
> I'd keep three, but refactor them as I've described above:
> - opkg: Includes '/etc/opkg/opkg.conf'
> - opkg-arch-config: Includes '/etc/opkg/arch.conf', machine specific
> - opkg-feed-config: Includes '/etc/opkg/feeds.conf', distro and
> possibly machine specific

Please also check
http://git.openembedded.org/meta-openembedded/tree/meta-oe/recipes-core/meta/distro-feed-configs.bb

+ filtering the architectures

https://github.com/shr-distribution/meta-smartphone/blob/master/meta-shr-distro/recipes-core/meta/distro-feed-configs.bbappend

+ stricter filter for individual MACHINEs based on selected DEFAULTTUNE

https://github.com/shr-distribution/meta-smartphone/blob/master/meta-shr-distro/conf/distro/include/defaulttunes.inc

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: opkg, opkg-config-base and opkg-collateral

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 1853 bytes --]

On Wed, Nov 26, 2014 at 04:17:08PM +0100, Martin Jansa wrote:
> On Wed, Nov 26, 2014 at 01:50:38PM +0000, Paul Barker wrote:
> > 
> > I'd keep three, but refactor them as I've described above:
> > - opkg: Includes '/etc/opkg/opkg.conf'
> > - opkg-arch-config: Includes '/etc/opkg/arch.conf', machine specific
> > - opkg-feed-config: Includes '/etc/opkg/feeds.conf', distro and
> > possibly machine specific
> 
> Please also check
> http://git.openembedded.org/meta-openembedded/tree/meta-oe/recipes-core/meta/distro-feed-configs.bb
> 
> + filtering the architectures
> 
> https://github.com/shr-distribution/meta-smartphone/blob/master/meta-shr-distro/recipes-core/meta/distro-feed-configs.bbappend
> 
> + stricter filter for individual MACHINEs based on selected DEFAULTTUNE
> 
> https://github.com/shr-distribution/meta-smartphone/blob/master/meta-shr-distro/conf/distro/include/defaulttunes.inc
> 

Thanks for the pointer Martin, I forgot to check outside oe-core. Also, sorry
for the late reply, been rather busy recently!

I suggest we drop poky-feed-config-opkg from oe-core as it isn't really usable
as-is and the Yocto Project manual recommends using distro-feed-config anyway
(http://www.yoctoproject.org/docs/1.7/mega-manual/mega-manual.html#runtime-package-management-build).

I still think the opkg and opkg-collateral recipes should be refactored as I
proposed in my previous email. I'll try to throw together some RFC patches this
week.

We could also consider moving distro-feed-config to oe-core if it is the
recommended way to setup opkg feeds. I do think that a unified way of
configuring ipk/deb/rpm feeds would be good to have in the long term, but we can
at least tidy up the opkg configuration for now.

Thanks,

-- 
Paul Barker

Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 501 bytes --]

Re: opkg, opkg-config-base and opkg-collateral

[Michael Gloff]

[-- Attachment #1: Type: text/plain, Size: 3281 bytes --]

Paul,
Thanks for looking into this further. I agree that poky-feed-config-opkg
does not work for anything other than a reference. I've resorted to
creating my own, but should be maybe even a part of the opkg recipe so as
not to disturb rpm/deb. Below is what I am using. arch.conf and
base-feeds.conf only really need all, architecture and machine.
FEEDURIPREFIX ?= "ftp://MYMACHINE"

do_compile() {
        mkdir -p ${S}/${sysconfdir}/opkg/

        ipkgarchs="all ${MACHINE_ARCH} ${PACKAGE_ARCH}"

        basefeedconf=${S}/${sysconfdir}/opkg/base-feeds.conf

        rm -f $basefeedconf
        touch $basefeedconf

        for arch in $ipkgarchs; do
                echo "src/gz $arch ${FEEDURIPREFIX}/$arch" >> $basefeedconf
        done
}


do_install () {
        install -d ${D}${sysconfdir}/opkg
        install -m 0644  ${S}/${sysconfdir}/opkg/* ${D}${sysconfdir}/opkg/
}

FILES_${PN} = "${sysconfdir}/opkg/ "

CONFFILES_${PN} += "${sysconfdir}/opkg/base-feeds.conf"

Michael Gloff

On Sat, Dec 13, 2014 at 6:54 AM, Paul Barker <paul@paulbarker.me.uk> wrote:
>
> On Wed, Nov 26, 2014 at 04:17:08PM +0100, Martin Jansa wrote:
> > On Wed, Nov 26, 2014 at 01:50:38PM +0000, Paul Barker wrote:
> > >
> > > I'd keep three, but refactor them as I've described above:
> > > - opkg: Includes '/etc/opkg/opkg.conf'
> > > - opkg-arch-config: Includes '/etc/opkg/arch.conf', machine specific
> > > - opkg-feed-config: Includes '/etc/opkg/feeds.conf', distro and
> > > possibly machine specific
> >
> > Please also check
> >
> http://git.openembedded.org/meta-openembedded/tree/meta-oe/recipes-core/meta/distro-feed-configs.bb
> >
> > + filtering the architectures
> >
> >
> https://github.com/shr-distribution/meta-smartphone/blob/master/meta-shr-distro/recipes-core/meta/distro-feed-configs.bbappend
> >
> > + stricter filter for individual MACHINEs based on selected DEFAULTTUNE
> >
> >
> https://github.com/shr-distribution/meta-smartphone/blob/master/meta-shr-distro/conf/distro/include/defaulttunes.inc
> >
>
> Thanks for the pointer Martin, I forgot to check outside oe-core. Also,
> sorry
> for the late reply, been rather busy recently!
>
> I suggest we drop poky-feed-config-opkg from oe-core as it isn't really
> usable
> as-is and the Yocto Project manual recommends using distro-feed-config
> anyway
> (
> http://www.yoctoproject.org/docs/1.7/mega-manual/mega-manual.html#runtime-package-management-build
> ).
>
> I still think the opkg and opkg-collateral recipes should be refactored as
> I
> proposed in my previous email. I'll try to throw together some RFC patches
> this
> week.
>
> We could also consider moving distro-feed-config to oe-core if it is the
> recommended way to setup opkg feeds. I do think that a unified way of
> configuring ipk/deb/rpm feeds would be good to have in the long term, but
> we can
> at least tidy up the opkg configuration for now.
>
> Thanks,
>
> --
> Paul Barker
>
> Email: paul@paulbarker.me.uk
> http://www.paulbarker.me.uk
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>

[-- Attachment #2: Type: text/html, Size: 4876 bytes --]