[PATCH] rootfs_ipk, image: Add debug capture support

[PATCH] rootfs_ipk, image: Add debug capture support

[Phil Blundell]

If ${IMAGE_BUILD_DEBUG} is set, construct a parallel tree containing
the debug data for the packages that have been installed in the
rootfs, then tar it up and deploy it alongside the rootfs images.

Signed-off-by: Phil Blundell <philb@gnu.org>
---
 meta/classes/image.bbclass      |    9 +++++++++
 meta/classes/rootfs_ipk.bbclass |   15 +++++++++++++++
 meta/conf/bitbake.conf          |    1 +
 3 files changed, 25 insertions(+)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index ab212b3..c82fba1 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -445,3 +445,12 @@ do_package_write_deb[noexec] = "1"
 do_package_write_rpm[noexec] = "1"
 
 addtask rootfs before do_build
+
+fakeroot do_capture_debug() {
+	if [ "${IMAGE_BUILD_DEBUG}" = "1" -a -n "${IMAGE_ROOTFS_DBG}" -a -d "${IMAGE_ROOTFS_DBG}" ]; then
+		tar czf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.debug.tar.gz -C ${IMAGE_ROOTFS_DBG} .
+		[ "${IMAGE_NAME}" == "${IMAGE_LINK_NAME}" ] || ln -sf ${IMAGE_NAME}.debug.tar.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.debug.tar.gz
+	fi
+}
+
+addtask capture_debug before do_build after do_rootfs
diff --git a/meta/classes/rootfs_ipk.bbclass b/meta/classes/rootfs_ipk.bbclass
index 46e8d60..e4e74dd 100644
--- a/meta/classes/rootfs_ipk.bbclass
+++ b/meta/classes/rootfs_ipk.bbclass
@@ -9,6 +9,8 @@ EXTRAOPKGCONFIG ?= ""
 ROOTFS_PKGMANAGE = "opkg opkg-collateral ${EXTRAOPKGCONFIG}"
 ROOTFS_PKGMANAGE_BOOTSTRAP  = "run-postinsts"
 
+IMAGE_BUILD_DEBUG ?= "0"
+
 do_rootfs[depends] += "opkg-native:do_populate_sysroot opkg-utils-native:do_populate_sysroot"
 do_rootfs[recrdeptask] += "do_package_write_ipk"
 
@@ -17,6 +19,7 @@ do_rootfs[lockfiles] += "${WORKDIR}/ipk.lock"
 IPKG_ARGS = "-f ${IPKGCONF_TARGET} -o ${IMAGE_ROOTFS} --force-overwrite"
 # The _POST version also works when constructing the matching SDK
 IPKG_ARGS_POST = "-f ${IPKGCONF_TARGET} -o $INSTALL_ROOTFS_IPK --force-overwrite"
+IPKG_ARGS_DBG = "-f ${IPKGCONF_TARGET} -o ${IMAGE_ROOTFS_DBG} --force-overwrite"
 
 OPKG_PREPROCESS_COMMANDS = "package_update_index_ipk; package_generate_ipkg_conf"
 
@@ -88,6 +91,18 @@ fakeroot rootfs_ipk_do_rootfs () {
 		fi
 	fi
 
+	if [ "${IMAGE_BUILD_DEBUG}" = "1" -a -n "${IMAGE_ROOTFS_DBG}" ]; then
+		all_pkgs=`opkg-cl ${IPKG_ARGS} list-installed | awk '{ print $1 }'`
+		[ "${IMAGE_ROOTFS_DBG}" -ef "${IMAGE_ROOTFS}" ] || rm -rf ${IMAGE_ROOTFS_DBG}
+		mkdir -p ${IMAGE_ROOTFS_DBG}${opkglibdir}
+		opkg-cl ${IPKG_ARGS_DBG} update
+		for p in $all_pkgs; do
+	    		if [ "`opkg-cl ${IPKG_ARGS_DBG} info $p-dbg`" != "" ]; then
+	    			opkg-cl ${IPKG_ARGS_DBG} --nodeps install $p-dbg
+			fi
+		done
+	fi
+
 	install -d ${IMAGE_ROOTFS}/${sysconfdir}
 	echo ${BUILDNAME} > ${IMAGE_ROOTFS}/${sysconfdir}/version
 
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 87351e0..66152fe 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -399,6 +399,7 @@ STAGING_KERNEL_DIR = "${STAGING_DIR_HOST}/usr/src/kernel"
 ##################################################################
 
 IMAGE_ROOTFS = "${WORKDIR}/rootfs"
+IMAGE_ROOTFS_DBG = "${IMAGE_ROOTFS}.debug"
 IMAGE_BASENAME = "${PN}"
 IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}-${DATETIME}"
 IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
-- 
1.7.10.4

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Richard Purdie]

On Tue, 2012-10-02 at 10:20 +0100, Phil Blundell wrote:
> If ${IMAGE_BUILD_DEBUG} is set, construct a parallel tree containing
> the debug data for the packages that have been installed in the
> rootfs, then tar it up and deploy it alongside the rootfs images.
> 
> Signed-off-by: Phil Blundell <philb@gnu.org>
> ---
>  meta/classes/image.bbclass      |    9 +++++++++
>  meta/classes/rootfs_ipk.bbclass |   15 +++++++++++++++
>  meta/conf/bitbake.conf          |    1 +
>  3 files changed, 25 insertions(+)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index ab212b3..c82fba1 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -445,3 +445,12 @@ do_package_write_deb[noexec] = "1"
>  do_package_write_rpm[noexec] = "1"
>  
>  addtask rootfs before do_build
> +
> +fakeroot do_capture_debug() {
> +	if [ "${IMAGE_BUILD_DEBUG}" = "1" -a -n "${IMAGE_ROOTFS_DBG}" -a -d "${IMAGE_ROOTFS_DBG}" ]; then
> +		tar czf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.debug.tar.gz -C ${IMAGE_ROOTFS_DBG} .
> +		[ "${IMAGE_NAME}" == "${IMAGE_LINK_NAME}" ] || ln -sf ${IMAGE_NAME}.debug.tar.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.debug.tar.gz
> +	fi
> +}
> +
> +addtask capture_debug before do_build after do_rootfs
> diff --git a/meta/classes/rootfs_ipk.bbclass b/meta/classes/rootfs_ipk.bbclass
> index 46e8d60..e4e74dd 100644
> --- a/meta/classes/rootfs_ipk.bbclass
> +++ b/meta/classes/rootfs_ipk.bbclass
> @@ -9,6 +9,8 @@ EXTRAOPKGCONFIG ?= ""
>  ROOTFS_PKGMANAGE = "opkg opkg-collateral ${EXTRAOPKGCONFIG}"
>  ROOTFS_PKGMANAGE_BOOTSTRAP  = "run-postinsts"
>  
> +IMAGE_BUILD_DEBUG ?= "0"
> +
>  do_rootfs[depends] += "opkg-native:do_populate_sysroot opkg-utils-native:do_populate_sysroot"
>  do_rootfs[recrdeptask] += "do_package_write_ipk"
>  
> @@ -17,6 +19,7 @@ do_rootfs[lockfiles] += "${WORKDIR}/ipk.lock"
>  IPKG_ARGS = "-f ${IPKGCONF_TARGET} -o ${IMAGE_ROOTFS} --force-overwrite"
>  # The _POST version also works when constructing the matching SDK
>  IPKG_ARGS_POST = "-f ${IPKGCONF_TARGET} -o $INSTALL_ROOTFS_IPK --force-overwrite"
> +IPKG_ARGS_DBG = "-f ${IPKGCONF_TARGET} -o ${IMAGE_ROOTFS_DBG} --force-overwrite"
>  
>  OPKG_PREPROCESS_COMMANDS = "package_update_index_ipk; package_generate_ipkg_conf"
>  
> @@ -88,6 +91,18 @@ fakeroot rootfs_ipk_do_rootfs () {
>  		fi
>  	fi
>  
> +	if [ "${IMAGE_BUILD_DEBUG}" = "1" -a -n "${IMAGE_ROOTFS_DBG}" ]; then
> +		all_pkgs=`opkg-cl ${IPKG_ARGS} list-installed | awk '{ print $1 }'`
> +		[ "${IMAGE_ROOTFS_DBG}" -ef "${IMAGE_ROOTFS}" ] || rm -rf ${IMAGE_ROOTFS_DBG}
> +		mkdir -p ${IMAGE_ROOTFS_DBG}${opkglibdir}
> +		opkg-cl ${IPKG_ARGS_DBG} update
> +		for p in $all_pkgs; do
> +	    		if [ "`opkg-cl ${IPKG_ARGS_DBG} info $p-dbg`" != "" ]; then
> +	    			opkg-cl ${IPKG_ARGS_DBG} --nodeps install $p-dbg
> +			fi
> +		done
> +	fi
> +
>  	install -d ${IMAGE_ROOTFS}/${sysconfdir}
>  	echo ${BUILDNAME} > ${IMAGE_ROOTFS}/${sysconfdir}/version
>  

Hasn't Paul added a general mechanism for doing this so we could add
this feature a level higher so that it could be used by all packaging
formats?

I'm a little concerned that the different backends are getting totally
different sets of features at this point. In some cases we have very
good reasons for it but that doesn't appear to be the case here.

Cheers,

Richard

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Phil Blundell]

On Tue, 2012-10-02 at 12:12 +0100, Richard Purdie wrote:
> Hasn't Paul added a general mechanism for doing this so we could add
> this feature a level higher so that it could be used by all packaging
> formats?

Ah, possibly.  I'll have a look.  Can you give me a pointer to the
mechanism you were thinking of?

thanks

p.

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Richard Purdie]

On Tue, 2012-10-02 at 12:15 +0100, Phil Blundell wrote:
> On Tue, 2012-10-02 at 12:12 +0100, Richard Purdie wrote:
> > Hasn't Paul added a general mechanism for doing this so we could add
> > this feature a level higher so that it could be used by all packaging
> > formats?
> 
> Ah, possibly.  I'll have a look.  Can you give me a pointer to the
> mechanism you were thinking of?

Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition
of dbg packages.

Cheers,

Richard

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Otavio Salvador]

On Tue, Oct 2, 2012 at 8:38 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Tue, 2012-10-02 at 12:15 +0100, Phil Blundell wrote:
>> On Tue, 2012-10-02 at 12:12 +0100, Richard Purdie wrote:
>> > Hasn't Paul added a general mechanism for doing this so we could add
>> > this feature a level higher so that it could be used by all packaging
>> > formats?
>>
>> Ah, possibly.  I'll have a look.  Can you give me a pointer to the
>> mechanism you were thinking of?
>
> Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition
> of dbg packages.

Or IMAGE_FEATURES += "dbg-img" ;-)

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Phil Blundell]

On Tue, 2012-10-02 at 12:38 +0100, Richard Purdie wrote:
> On Tue, 2012-10-02 at 12:15 +0100, Phil Blundell wrote:
> > On Tue, 2012-10-02 at 12:12 +0100, Richard Purdie wrote:
> > > Hasn't Paul added a general mechanism for doing this so we could add
> > > this feature a level higher so that it could be used by all packaging
> > > formats?
> > 
> > Ah, possibly.  I'll have a look.  Can you give me a pointer to the
> > mechanism you were thinking of?
> 
> Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition
> of dbg packages.

Ah yes, I see it.  It looks like that will cause the debug packages to
go into the rootfs itself, rather than a parallel tree, but I guess I
can probably arrange to fish them back out again before the image is
built.

thanks

p.

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Phil Blundell]

On Tue, 2012-10-02 at 10:42 -0300, Otavio Salvador wrote:
> Or IMAGE_FEATURES += "dbg-img" ;-)

Where is that implemented?  I couldn't find any obvious reference to
dbg-img in any classes.

p.

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Martin Ertsås]

On 10/02/12 15:53, Otavio Salvador wrote:
> On Tue, Oct 2, 2012 at 10:51 AM, Phil Blundell <philb@gnu.org> wrote:
>> On Tue, 2012-10-02 at 10:42 -0300, Otavio Salvador wrote:
>>> Or IMAGE_FEATURES += "dbg-img" ;-)
>> Where is that implemented?  I couldn't find any obvious reference to
>> dbg-img in any classes.
> Sorry by cause a missunderstanding ... I was just pointing to a
> possible new feature you could add.
>
that feature is already implemented though, in dbg-pkgs. If you don't
mean dbg-img should do something other/more than what dbg-pkgs does?

- Martin

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Otavio Salvador]

On Tue, Oct 2, 2012 at 10:51 AM, Phil Blundell <philb@gnu.org> wrote:
> On Tue, 2012-10-02 at 10:42 -0300, Otavio Salvador wrote:
>> Or IMAGE_FEATURES += "dbg-img" ;-)
>
> Where is that implemented?  I couldn't find any obvious reference to
> dbg-img in any classes.

Sorry by cause a missunderstanding ... I was just pointing to a
possible new feature you could add.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Martin Ertsås]

On 10/02/12 15:59, Otavio Salvador wrote:
> On Tue, Oct 2, 2012 at 10:52 AM, Martin Ertsås <mertsas@cisco.com> wrote:
>> On 10/02/12 15:53, Otavio Salvador wrote:
>>> On Tue, Oct 2, 2012 at 10:51 AM, Phil Blundell <philb@gnu.org> wrote:
>>>> On Tue, 2012-10-02 at 10:42 -0300, Otavio Salvador wrote:
>>>>> Or IMAGE_FEATURES += "dbg-img" ;-)
>>>> Where is that implemented?  I couldn't find any obvious reference to
>>>> dbg-img in any classes.
>>> Sorry by cause a missunderstanding ... I was just pointing to a
>>> possible new feature you could add.
>>>
>> that feature is already implemented though, in dbg-pkgs. If you don't
>> mean dbg-img should do something other/more than what dbg-pkgs does?
> To build a separated tar with the symbols, not on the image itself.
>
With only the symbols? Or a separate image with symbols, side by side
with the stripped one? In the later case you could just add a export
IMAGE_BASENAME = "my-dbg-image" in the image recipe. If you mean with
only the symbols I don't think we have any features for that right now.

- Martin

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Otavio Salvador]

On Tue, Oct 2, 2012 at 10:52 AM, Martin Ertsås <mertsas@cisco.com> wrote:
> On 10/02/12 15:53, Otavio Salvador wrote:
>> On Tue, Oct 2, 2012 at 10:51 AM, Phil Blundell <philb@gnu.org> wrote:
>>> On Tue, 2012-10-02 at 10:42 -0300, Otavio Salvador wrote:
>>>> Or IMAGE_FEATURES += "dbg-img" ;-)
>>> Where is that implemented?  I couldn't find any obvious reference to
>>> dbg-img in any classes.
>> Sorry by cause a missunderstanding ... I was just pointing to a
>> possible new feature you could add.
>>
> that feature is already implemented though, in dbg-pkgs. If you don't
> mean dbg-img should do something other/more than what dbg-pkgs does?

To build a separated tar with the symbols, not on the image itself.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Otavio Salvador]

On Tue, Oct 2, 2012 at 10:57 AM, Martin Ertsås <mertsas@cisco.com> wrote:
> On 10/02/12 15:59, Otavio Salvador wrote:
>> On Tue, Oct 2, 2012 at 10:52 AM, Martin Ertsås <mertsas@cisco.com> wrote:
>>> On 10/02/12 15:53, Otavio Salvador wrote:
>>>> On Tue, Oct 2, 2012 at 10:51 AM, Phil Blundell <philb@gnu.org> wrote:
>>>>> On Tue, 2012-10-02 at 10:42 -0300, Otavio Salvador wrote:
>>>>>> Or IMAGE_FEATURES += "dbg-img" ;-)
>>>>> Where is that implemented?  I couldn't find any obvious reference to
>>>>> dbg-img in any classes.
>>>> Sorry by cause a missunderstanding ... I was just pointing to a
>>>> possible new feature you could add.
>>>>
>>> that feature is already implemented though, in dbg-pkgs. If you don't
>>> mean dbg-img should do something other/more than what dbg-pkgs does?
>> To build a separated tar with the symbols, not on the image itself.
>>
> With only the symbols? Or a separate image with symbols, side by side
> with the stripped one? In the later case you could just add a export
> IMAGE_BASENAME = "my-dbg-image" in the image recipe. If you mean with
> only the symbols I don't think we have any features for that right now.

I thought it being side by side.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Phil Blundell]

On Tue, 2012-10-02 at 12:38 +0100, Richard Purdie wrote:
> On Tue, 2012-10-02 at 12:15 +0100, Phil Blundell wrote:
> > On Tue, 2012-10-02 at 12:12 +0100, Richard Purdie wrote:
> > > Hasn't Paul added a general mechanism for doing this so we could add
> > > this feature a level higher so that it could be used by all packaging
> > > formats?
> > 
> > Ah, possibly.  I'll have a look.  Can you give me a pointer to the
> > mechanism you were thinking of?
> 
> Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition
> of dbg packages.

I've somewhat belatedly gotten around to using this and encountered a
couple of issuettes:

1. There was a spurious-looking RRECOMMENDS_${PN}-dbg in bitbake.conf
which was causing ${PN} to be dragged into the image whenever ${PN}-dbg
was installed.  This is undesirable because it means that, for example,
any image which includes sysvinit-pidof will end up installing sysvinit
and this might cause quite drastic changes to the behaviour of the
resulting image.

2. Having fixed the above I'm now left with a couple of file conflicts
in the -dbg packages.  For example, sysvinit and util-linux both
ship /sbin/sulogin and so sysvinit-dbg and util-linux-dbg both want to
install /sbin/.debug/sulogin.  This is unfortunate if you have an image
which uses, say, sysvinit-pidof and util-linux-blkid.

I'm not quite sure what the right way to fix (2) is.  I suppose in an
ideal world the -dbg packages would be separated in the same way the
parent binary packages are, but that doesn't look entirely
straightforward to arrange.

p.

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1999 bytes --]

On Thu, Apr 25, 2013 at 12:12:54PM +0100, Phil Blundell wrote:
> On Tue, 2012-10-02 at 12:38 +0100, Richard Purdie wrote:
> > On Tue, 2012-10-02 at 12:15 +0100, Phil Blundell wrote:
> > > On Tue, 2012-10-02 at 12:12 +0100, Richard Purdie wrote:
> > > > Hasn't Paul added a general mechanism for doing this so we could add
> > > > this feature a level higher so that it could be used by all packaging
> > > > formats?
> > > 
> > > Ah, possibly.  I'll have a look.  Can you give me a pointer to the
> > > mechanism you were thinking of?
> > 
> > Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition
> > of dbg packages.
> 
> I've somewhat belatedly gotten around to using this and encountered a
> couple of issuettes:
> 
> 1. There was a spurious-looking RRECOMMENDS_${PN}-dbg in bitbake.conf
> which was causing ${PN} to be dragged into the image whenever ${PN}-dbg
> was installed.  This is undesirable because it means that, for example,
> any image which includes sysvinit-pidof will end up installing sysvinit
> and this might cause quite drastic changes to the behaviour of the
> resulting image.
> 
> 2. Having fixed the above I'm now left with a couple of file conflicts
> in the -dbg packages.  For example, sysvinit and util-linux both
> ship /sbin/sulogin and so sysvinit-dbg and util-linux-dbg both want to
> install /sbin/.debug/sulogin.  This is unfortunate if you have an image
> which uses, say, sysvinit-pidof and util-linux-blkid.
> 
> I'm not quite sure what the right way to fix (2) is.  I suppose in an
> ideal world the -dbg packages would be separated in the same way the
> parent binary packages are, but that doesn't look entirely
> straightforward to arrange.

yes there is a lot of conflicts between PN-doc and PN-dbg, I had similar
issues when trying to build world-image:
http://lists.linuxtogo.org/pipermail/openembedded-core/2013-January/034744.html

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Mark Hatle]

On 4/25/13 6:12 AM, Phil Blundell wrote:
> On Tue, 2012-10-02 at 12:38 +0100, Richard Purdie wrote:
>> On Tue, 2012-10-02 at 12:15 +0100, Phil Blundell wrote:
>>> On Tue, 2012-10-02 at 12:12 +0100, Richard Purdie wrote:
>>>> Hasn't Paul added a general mechanism for doing this so we could add
>>>> this feature a level higher so that it could be used by all packaging
>>>> formats?
>>>
>>> Ah, possibly.  I'll have a look.  Can you give me a pointer to the
>>> mechanism you were thinking of?
>>
>> Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition
>> of dbg packages.
>
> I've somewhat belatedly gotten around to using this and encountered a
> couple of issuettes:
>
> 1. There was a spurious-looking RRECOMMENDS_${PN}-dbg in bitbake.conf
> which was causing ${PN} to be dragged into the image whenever ${PN}-dbg
> was installed.  This is undesirable because it means that, for example,
> any image which includes sysvinit-pidof will end up installing sysvinit
> and this might cause quite drastic changes to the behaviour of the
> resulting image.
>
> 2. Having fixed the above I'm now left with a couple of file conflicts
> in the -dbg packages.  For example, sysvinit and util-linux both
> ship /sbin/sulogin and so sysvinit-dbg and util-linux-dbg both want to
> install /sbin/.debug/sulogin.  This is unfortunate if you have an image
> which uses, say, sysvinit-pidof and util-linux-blkid.

Is this a problem that they should have used the update-alternatives for 
sulogin?  (Sounds like it might be a security issue though...)  This would avoid 
the .debug conflict.

--Mark

> I'm not quite sure what the right way to fix (2) is.  I suppose in an
> ideal world the -dbg packages would be separated in the same way the
> parent binary packages are, but that doesn't look entirely
> straightforward to arrange.
>
> p.
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Phil Blundell]

On Thu, 2013-04-25 at 08:47 -0500, Mark Hatle wrote:
> Is this a problem that they should have used the update-alternatives for 
> sulogin?  (Sounds like it might be a security issue though...)  This would avoid 
> the .debug conflict.

I dunno.  It seems a bit sad to force use of update-alternatives just to
deconflict the .debug data (since the binaries in question would never
be installed at the same time anyway), and in general it seems like we
want less update-alternatives rather than more of it.  

But I guess that would indeed solve the problem and it's probably the
lowest-effort thing that does.  So maybe that's the way ahead.

p.

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Mark Hatle]

On 4/26/13 8:57 AM, Phil Blundell wrote:
> On Thu, 2013-04-25 at 08:47 -0500, Mark Hatle wrote:
>> Is this a problem that they should have used the update-alternatives for
>> sulogin?  (Sounds like it might be a security issue though...)  This would avoid
>> the .debug conflict.
>
> I dunno.  It seems a bit sad to force use of update-alternatives just to
> deconflict the .debug data (since the binaries in question would never
> be installed at the same time anyway), and in general it seems like we
> want less update-alternatives rather than more of it.
>
> But I guess that would indeed solve the problem and it's probably the
> lowest-effort thing that does.  So maybe that's the way ahead.

The alternative of course is to crease special -dbg packages for the two 
conflicting items.  I.e. foo-dbg, foo-sulogin-dbg, bar-dbg and bar-sulogin-dbg...

--Mark

> p.
>
>

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Phil Blundell]

On Fri, 2013-04-26 at 09:16 -0500, Mark Hatle wrote:
> The alternative of course is to crease special -dbg packages for the two 
> conflicting items.  I.e. foo-dbg, foo-sulogin-dbg, bar-dbg and bar-sulogin-dbg...

Yeah, indeed, that's what I suggested in my original email.  At the time
I thought that would be hard to arrange (in the general case), but
having given it some further consideration perhaps it isn't all that bad
after all.

p.

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Mark Hatle]

On 4/26/13 9:27 AM, Phil Blundell wrote:
> On Fri, 2013-04-26 at 09:16 -0500, Mark Hatle wrote:
>> The alternative of course is to crease special -dbg packages for the two
>> conflicting items.  I.e. foo-dbg, foo-sulogin-dbg, bar-dbg and bar-sulogin-dbg...
>
> Yeah, indeed, that's what I suggested in my original email.  At the time
> I thought that would be hard to arrange (in the general case), but
> having given it some further consideration perhaps it isn't all that bad
> after all.

I certainly wouldn't be against an enhancement that tries to match up the 
binaries to their debug and create suitably named -dbg packages.  The only 
tricky part is what to do with the associated sources?  Since those are not 
arranged according to binaries, but generally for the whole recipe.

--Mark

> p.
>
>

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Phil Blundell]

On Fri, 2013-04-26 at 09:39 -0500, Mark Hatle wrote:
> On 4/26/13 9:27 AM, Phil Blundell wrote:
> > On Fri, 2013-04-26 at 09:16 -0500, Mark Hatle wrote:
> >> The alternative of course is to crease special -dbg packages for the two
> >> conflicting items.  I.e. foo-dbg, foo-sulogin-dbg, bar-dbg and bar-sulogin-dbg...
> >
> > Yeah, indeed, that's what I suggested in my original email.  At the time
> > I thought that would be hard to arrange (in the general case), but
> > having given it some further consideration perhaps it isn't all that bad
> > after all.
> 
> I certainly wouldn't be against an enhancement that tries to match up the 
> binaries to their debug and create suitably named -dbg packages.  The only 
> tricky part is what to do with the associated sources?  Since those are not 
> arranged according to binaries, but generally for the whole recipe.

The sources can stay where they are, in ${PN}-dbg.  There won't be any
conflict there because the installed files are already namespaced by
recipe.

p.

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Otavio Salvador]

On Fri, Apr 26, 2013 at 11:41 AM, Phil Blundell <pb@pbcl.net> wrote:
> On Fri, 2013-04-26 at 09:39 -0500, Mark Hatle wrote:
>> On 4/26/13 9:27 AM, Phil Blundell wrote:
>> > On Fri, 2013-04-26 at 09:16 -0500, Mark Hatle wrote:
>> >> The alternative of course is to crease special -dbg packages for the two
>> >> conflicting items.  I.e. foo-dbg, foo-sulogin-dbg, bar-dbg and bar-sulogin-dbg...
>> >
>> > Yeah, indeed, that's what I suggested in my original email.  At the time
>> > I thought that would be hard to arrange (in the general case), but
>> > having given it some further consideration perhaps it isn't all that bad
>> > after all.
>>
>> I certainly wouldn't be against an enhancement that tries to match up the
>> binaries to their debug and create suitably named -dbg packages.  The only
>> tricky part is what to do with the associated sources?  Since those are not
>> arranged according to binaries, but generally for the whole recipe.
>
> The sources can stay where they are, in ${PN}-dbg.  There won't be any
> conflict there because the installed files are already namespaced by
> recipe.

Maybe add a ${PN}-source with it? So any -dbg could rdepends on it?

--
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Mark Hatle]

On 4/26/13 9:50 AM, Otavio Salvador wrote:
> On Fri, Apr 26, 2013 at 11:41 AM, Phil Blundell <pb@pbcl.net> wrote:
>> On Fri, 2013-04-26 at 09:39 -0500, Mark Hatle wrote:
>>> On 4/26/13 9:27 AM, Phil Blundell wrote:
>>>> On Fri, 2013-04-26 at 09:16 -0500, Mark Hatle wrote:
>>>>> The alternative of course is to crease special -dbg packages for the two
>>>>> conflicting items.  I.e. foo-dbg, foo-sulogin-dbg, bar-dbg and bar-sulogin-dbg...
>>>>
>>>> Yeah, indeed, that's what I suggested in my original email.  At the time
>>>> I thought that would be hard to arrange (in the general case), but
>>>> having given it some further consideration perhaps it isn't all that bad
>>>> after all.
>>>
>>> I certainly wouldn't be against an enhancement that tries to match up the
>>> binaries to their debug and create suitably named -dbg packages.  The only
>>> tricky part is what to do with the associated sources?  Since those are not
>>> arranged according to binaries, but generally for the whole recipe.
>>
>> The sources can stay where they are, in ${PN}-dbg.  There won't be any
>> conflict there because the installed files are already namespaced by
>> recipe.
>
> Maybe add a ${PN}-source with it? So any -dbg could rdepends on it?

-dbg-source would be better.. but I don't mind that.  I know there have been 
complaints where people want the debug symbols on the target, but don't want the 
debug sources.  This could address that as well.

I'd suggest someone (Phil ?) collect these and add a open an enhancement bug in 
the YP bugzilla.  That way progress on implementation can be tracked.   (Unless 
of course someone just implements it.. but I'm guessing it won't be quite that 
straightforward.)

--Mark

> --
> Otavio Salvador                             O.S. Systems
> E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
> Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br
>

Re: [PATCH] rootfs_ipk, image: Add debug capture support

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 1616 bytes --]

On Thu, Apr 25, 2013 at 7:12 AM, Phil Blundell <pb@pbcl.net> wrote:

> > Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition
> > of dbg packages.
>
> I've somewhat belatedly gotten around to using this and encountered a
> couple of issuettes:
>
> 1. There was a spurious-looking RRECOMMENDS_${PN}-dbg in bitbake.conf
> which was causing ${PN} to be dragged into the image whenever ${PN}-dbg
> was installed.  This is undesirable because it means that, for example,
> any image which includes sysvinit-pidof will end up installing sysvinit
> and this might cause quite drastic changes to the behaviour of the
> resulting image.
>
> 2. Having fixed the above I'm now left with a couple of file conflicts
> in the -dbg packages.  For example, sysvinit and util-linux both
> ship /sbin/sulogin and so sysvinit-dbg and util-linux-dbg both want to
> install /sbin/.debug/sulogin.  This is unfortunate if you have an image
> which uses, say, sysvinit-pidof and util-linux-blkid.
>
> I'm not quite sure what the right way to fix (2) is.  I suppose in an
> ideal world the -dbg packages would be separated in the same way the
> parent binary packages are, but that doesn't look entirely
> straightforward to arrange.


I had implemented something along these lines back in oe classic, when I
was at MontaVista.  See
http://git.openembedded.org/openembedded/tree/classes/package_dbg.bbclass for
that implementation. I haven't touched it or tried using it in some time,
however.
-- 
Christopher Larson
clarson at kergoth dot com
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 2168 bytes --]

Debug Packaging (was: rootfs_ipk, image: Add debug capture support)

[Richard Purdie]

On Sat, 2013-04-27 at 17:15 -0400, Chris Larson wrote:

> On Thu, Apr 25, 2013 at 7:12 AM, Phil Blundell <pb@pbcl.net> wrote:

>         I'm not quite sure what the right way to fix (2) is.  I
>         suppose in an
>         ideal world the -dbg packages would be separated in the same
>         way the
>         parent binary packages are, but that doesn't look entirely
>         straightforward to arrange.
> 
> I had implemented something along these lines back in oe classic, when
> I was at MontaVista.  See
> http://git.openembedded.org/openembedded/tree/classes/package_dbg.bbclass for that implementation. I haven't touched it or tried using it in some time, however.

I have to admit I've been wondering if we shouldn't overhaul the -dbg
part of the system a bit. There are a few things I've been wondering
about:

a) Should we ditch FILES_${PN}-dbg and have it auto constructed from
any .debug directories? This would be a rather nice automation/cleanup.
I can't see a pressing reason not to do this.

b) Consider splitting it to a -dbg package per package where there are
debug files as per above. There are pros and cons to this and I'm torn,
see c).

c) Consider splitting the debug source into a separate package. If we
did do b), the question is where should the sources go?

In the bigger picture we have various dependency chain issues as the
-dbg and -dev dependency changes are horrible. The question has always
been whether X-dbg or X-dev should be usable to pull in sensible
dependencies.

Thinking about the scenarios you might use these in:

a) A binary from package X segfaults. You want to get good gdb data for
why. You therefore install X-dbg. X links against Y. You therefore want
the symbols/code for Y too so you can trace into the problem which may
be in Y.

b) You want to compile against X, you install X-dev. You expect anything
X needs to link with (e.g. through any .pc file) to also be present.

c) You want to rebuild X and hence install X-dev to ensure the build
dependencies are present.

d) There was once an idea that you could do something like install
core-image-minimal-dev to get the equivalent of core-image-minimal with
dev-pkgs. I think we've found better ways to do this kind of thing now?

In a/b/c, the usability fail is if you try to gdb/compile, find a
dependency missing, install it and repeat this cycle several times over.

Given d) is dead, thankfully, does that let us rip some code out and
improve the dependencies?

Cheers,

Richard

Re: Debug Packaging (was: rootfs_ipk, image: Add debug capture support)

[Phil Blundell]

On Mon, 2013-04-29 at 21:10 +0100, Richard Purdie wrote:
> a) Should we ditch FILES_${PN}-dbg and have it auto constructed from
>any .debug directories? This would be a rather nice automation/cleanup.
>I can't see a pressing reason not to do this.
>
> b) Consider splitting it to a -dbg package per package where there are
> debug files as per above. There are pros and cons to this and I'm torn,
> see c).
> 
> c) Consider splitting the debug source into a separate package. If we
> did do b), the question is where should the sources go?

I think (b) would definitely be desirable for the reasons I mentioned
previously.  The obvious answer to (c) is that the sources should go in
a new ${PN}-debug-source kind of package and all the ${PN}-foo-dbg
packages can recommend it.

If we do do (b) then it seems as though (a) would just fall out in the
wash so there's probably no need to consider that explicitly.

I'd also quite like to separate the sources from the debug data since I
have at least some use-cases where I want the latter installed but not
the former.

> a) A binary from package X segfaults. You want to get good gdb data for
> why. You therefore install X-dbg. X links against Y. You therefore want
> the symbols/code for Y too so you can trace into the problem which may
> be in Y.

That's true up to a point, although most people probably don't need/want
the debugging symbols for (say) glibc installed even though their
program might well link against it.  There have been some other
occasions when I've wanted to deliberately exclude the debug data for
library dependencies, most notably webkit which has such a vast amount
of debug info that GDB can take several minutes to start when it's
installed.

Also, the above does apply specifically to the "links against" case, not
necessarily for everything in RDEPENDS.  For example, if I have a
package which includes a bash script and RDEPENDS on bash, that doesn't
mean that I want my X-dbg to start dragging in bash's debug data.

> b) You want to compile against X, you install X-dev. You expect anything
> X needs to link with (e.g. through any .pc file) to also be present.

Correct.  But again, this doesn't necessarily mean installing Y-dev for
every Y that X depends on; ideally the logic wants to be a bit smarter
than that.  Nowadays you probably can get most/all of the necessary bits
from pkgconfig for the majority of packages.

> c) You want to rebuild X and hence install X-dev to ensure the build
> dependencies are present.

I think this one is a separate issue and should be handled differently,
for example by encoding the build dependencies as a Build-Depends: field
in X's control data, or by generating an extra X-build-depends output
package which just depends on all the necessary things, or (for those
people using rpm at least) generating an SRPM which contains the right
bits in its spec file.  

Note that the set of things that X needs installed in order to build
itself might well be larger than the set of things that need installing
under scenario (b) above.  

> d) There was once an idea that you could do something like install
> core-image-minimal-dev to get the equivalent of core-image-minimal with
> dev-pkgs. I think we've found better ways to do this kind of thing now?

Agreed.

p.

Re: Debug Packaging

[Mark Hatle]

On 4/29/13 3:10 PM, Richard Purdie wrote:
> On Sat, 2013-04-27 at 17:15 -0400, Chris Larson wrote:
>
>> On Thu, Apr 25, 2013 at 7:12 AM, Phil Blundell <pb@pbcl.net> wrote:
>
>>          I'm not quite sure what the right way to fix (2) is.  I
>>          suppose in an
>>          ideal world the -dbg packages would be separated in the same
>>          way the
>>          parent binary packages are, but that doesn't look entirely
>>          straightforward to arrange.
>>
>> I had implemented something along these lines back in oe classic, when
>> I was at MontaVista.  See
>> http://git.openembedded.org/openembedded/tree/classes/package_dbg.bbclass for that implementation. I haven't touched it or tried using it in some time, however.
>
> I have to admit I've been wondering if we shouldn't overhaul the -dbg
> part of the system a bit. There are a few things I've been wondering
> about:
>
> a) Should we ditch FILES_${PN}-dbg and have it auto constructed from
> any .debug directories? This would be a rather nice automation/cleanup.
> I can't see a pressing reason not to do this.
>
> b) Consider splitting it to a -dbg package per package where there are
> debug files as per above. There are pros and cons to this and I'm torn,
> see c).
>
> c) Consider splitting the debug source into a separate package. If we
> did do b), the question is where should the sources go?

I think that the proposal for automatically generating a -dbg per regular 
package is a good one.  Then the sources can go into -dbg-source or something 
similar.. (source-dbg so it still ends in dbg).

This will give us the most flexibility to limit the amount of code that HAS to 
be installed onto a target to do basic crash dumping, if not full debug on the 
target.

> In the bigger picture we have various dependency chain issues as the
> -dbg and -dev dependency changes are horrible. The question has always
> been whether X-dbg or X-dev should be usable to pull in sensible
> dependencies.
>
> Thinking about the scenarios you might use these in:
>
> a) A binary from package X segfaults. You want to get good gdb data for
> why. You therefore install X-dbg. X links against Y. You therefore want
> the symbols/code for Y too so you can trace into the problem which may
> be in Y.

This is the one that is sketchy to me.  I think the rrecommend is a good idea 
here.  If you recommend everything that the main package (that you based on) 
needs that you should be able to do this.  By splitting the size of the -dbg 
into smaller units then the -dbg recommends should be limited as well.  I say 
recommend vs depend simply because the there are cases where I only want the 
symbols for one binary, I don't care about it's dependencies.

The question though is should the source also be a recommend or suggest?

> b) You want to compile against X, you install X-dev. You expect anything
> X needs to link with (e.g. through any .pc file) to also be present.
>
> c) You want to rebuild X and hence install X-dev to ensure the build
> dependencies are present.

I think this very much reasonable on the -dev front.  If I install a -dev 
package, I expect the system to be able to compile software without additional 
manual steps.

> d) There was once an idea that you could do something like install
> core-image-minimal-dev to get the equivalent of core-image-minimal with
> dev-pkgs. I think we've found better ways to do this kind of thing now?
>
> In a/b/c, the usability fail is if you try to gdb/compile, find a
> dependency missing, install it and repeat this cycle several times over.

Ya, the issue though is there are more use cases for debug then for the -dev 
case.  So I'm comfortable with slight differences in behavior between the two.

--Mark

> Given d) is dead, thankfully, does that let us rip some code out and
> improve the dependencies?
>
> Cheers,
>
> Richard
>
>
>
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>