Reducing the perl footprint on my image

Reducing the perl footprint on my image

[rustyhowell]

[-- Attachment #1: Type: text/plain, Size: 501 bytes --]

I have an image that is using debian package management  (PACKAGE_CLASSES = "package_deb").  Because apt and dpkg require perl, perl is being installed in the image.   No problem.  Except that the entire perl stack is 669 packages. Most of the packages are super tiny and are probably not needed by apt or dpkg.  Is there a way to strip down the large number of perl packages in the image to just what is needed by apt/dpkg?
I haven't found anything in the MegaManual yet.
Thanks in advance.

[-- Attachment #2: Type: text/html, Size: 533 bytes --]

Re: [yocto] Reducing the perl footprint on my image

[Steve Sakoman]

On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@gmail.com> wrote:
>
> I have an image that is using debian package management  (PACKAGE_CLASSES = "package_deb").  Because apt and dpkg require perl, perl is being installed in the image.   No problem.  Except that the entire perl stack is 669 packages.

I just took a look at the manifest for one of my images that includes
PACKAGE_CLASSES = "package_deb".  I see the perl package plus 43
perl-module packages.  Are you sure that something else in your images
isn't pulling in all of those other perl-module packages?

Steve

> Most of the packages are super tiny and are probably not needed by apt or dpkg.  Is there a way to strip down the large number of perl packages in the image to just what is needed by apt/dpkg?
> I haven't found anything in the MegaManual yet.
> Thanks in advance.
> 
>

Re: [yocto] Reducing the perl footprint on my image

[Steve Sakoman]

On Tue, Mar 2, 2021 at 10:01 AM Steve Sakoman via
lists.yoctoproject.org <steve=sakoman.com@lists.yoctoproject.org>
wrote:
>
> On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@gmail.com> wrote:
> >
> > I have an image that is using debian package management  (PACKAGE_CLASSES = "package_deb").  Because apt and dpkg require perl, perl is being installed in the image.   No problem.  Except that the entire perl stack is 669 packages.
>
> I just took a look at the manifest for one of my images that includes
> PACKAGE_CLASSES = "package_deb".  I see the perl package plus 43
> perl-module packages.  Are you sure that something else in your images
> isn't pulling in all of those other perl-module packages?

It just occurred to me to make sure you are looking in the image
manifest to see which packages are actually installed in your image.
The perl recipe does generate 676 packages (in dunfell) so perhaps you
might be looking at the generated packages rather than the installed
packages??

Steve


> > Most of the packages are super tiny and are probably not needed by apt or dpkg.  Is there a way to strip down the large number of perl packages in the image to just what is needed by apt/dpkg?
> > I haven't found anything in the MegaManual yet.
> > Thanks in advance.
> >
> >
>
> 
>

Re: [yocto] Reducing the perl footprint on my image

[Diego Santa Cruz]

> -----Original Message-----
> From: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> On
> Behalf Of Steve Sakoman via lists.yoctoproject.org
> Sent: 02 March 2021 21:06
> To: Steve Sakoman <steve@sakoman.com>
> Cc: rustyhowell@gmail.com; Yocto (yocto@lists.yoctoproject.org)
> <yocto@lists.yoctoproject.org>
> Subject: Re: [yocto] Reducing the perl footprint on my image
> 
> On Tue, Mar 2, 2021 at 10:01 AM Steve Sakoman via
> lists.yoctoproject.org <steve=sakoman.com@lists.yoctoproject.org>
> wrote:
> >
> > On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@gmail.com> wrote:
> > >
> > > I have an image that is using debian package management
> (PACKAGE_CLASSES = "package_deb").  Because apt and dpkg require perl,
> perl is being installed in the image.   No problem.  Except that the entire perl
> stack is 669 packages.
> >
> > I just took a look at the manifest for one of my images that includes
> > PACKAGE_CLASSES = "package_deb".  I see the perl package plus 43
> > perl-module packages.  Are you sure that something else in your images
> > isn't pulling in all of those other perl-module packages?
> 
> It just occurred to me to make sure you are looking in the image
> manifest to see which packages are actually installed in your image.
> The perl recipe does generate 676 packages (in dunfell) so perhaps you
> might be looking at the generated packages rather than the installed
> packages??
> 

I encountered a similar problem with package management enabled and rpm as package format, where I also just install rpm for package management and not all dnf stack. I get quite a lot of perl and python packages into the image which are pulled by the rpm package, but they are only needed for things like rpm-build, rpm-sign, etc., not for the bare rpm command, which is the only one I need in the image.

So I locally extended the rpm recipe to split those tools into rpm-build, rpm-sign and rpm-archive and skip those packages in the image. I should probably send patches for that to oe-core. Is that something that could be accepted?

-- 
Diego Santa Cruz, PhD
Technology Architect
spinetix.com

Re: [yocto] Reducing the perl footprint on my image

[Richard Purdie]

On Tue, 2021-03-02 at 20:42 +0000, Diego Santa Cruz via lists.yoctoproject.org wrote:
> > -----Original Message-----
> > From: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> On
> > Behalf Of Steve Sakoman via lists.yoctoproject.org
> > Sent: 02 March 2021 21:06
> > To: Steve Sakoman <steve@sakoman.com>
> > Cc: rustyhowell@gmail.com; Yocto (yocto@lists.yoctoproject.org)
> > <yocto@lists.yoctoproject.org>
> > Subject: Re: [yocto] Reducing the perl footprint on my image
> > 
> > On Tue, Mar 2, 2021 at 10:01 AM Steve Sakoman via
> > lists.yoctoproject.org <steve=sakoman.com@lists.yoctoproject.org>
> > wrote:
> > > 
> > > On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@gmail.com> wrote:
> > > > 
> > > > I have an image that is using debian package management
> > (PACKAGE_CLASSES = "package_deb").  Because apt and dpkg require perl,
> > perl is being installed in the image.   No problem.  Except that the entire perl
> > stack is 669 packages.
> > > 
> > > I just took a look at the manifest for one of my images that includes
> > > PACKAGE_CLASSES = "package_deb".  I see the perl package plus 43
> > > perl-module packages.  Are you sure that something else in your images
> > > isn't pulling in all of those other perl-module packages?
> > 
> > It just occurred to me to make sure you are looking in the image
> > manifest to see which packages are actually installed in your image.
> > The perl recipe does generate 676 packages (in dunfell) so perhaps you
> > might be looking at the generated packages rather than the installed
> > packages??
> > 
> 
> I encountered a similar problem with package management enabled and rpm 
> as package format, where I also just install rpm for package management 
> and not all dnf stack. I get quite a lot of perl and python packages 
> into the image which are pulled by the rpm package, but they are only
> needed for things like rpm-build, rpm-sign, etc., not for the bare 
> rpm command, which is the only one I need in the image.
> 
> So I locally extended the rpm recipe to split those tools into rpm-build, 
> rpm-sign and rpm-archive and skip those packages in the image. I should 
> probably send patches for that to oe-core. Is that something that could
> be accepted?

Not sure they need to go to separate packages but moving those 
three to some kind of "build" package would make a lot of sense
to me at least.

Cheers,

Richard

Re: [yocto] Reducing the perl footprint on my image

[rustyhowell]

[-- Attachment #1: Type: text/plain, Size: 3056 bytes --]

Steve, you're right.  My image.bb is adding various lmsensors packages to
the IMAGE_INSTALL, some of which depend on perl-modules.   perl-modules
seems to be the big monster, not perl.  We need lmsensors.  So I guess my
question is really, how can I (or can I at all) reduce the footprint of
perl-modules?  perl-modules doesn't have any Depends, rather it has a long
list (~ 670) of Recommends.

On Tue, Mar 2, 2021 at 1:49 PM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Tue, 2021-03-02 at 20:42 +0000, Diego Santa Cruz via
> lists.yoctoproject.org wrote:
> > > -----Original Message-----
> > > From: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> On
> > > Behalf Of Steve Sakoman via lists.yoctoproject.org
> > > Sent: 02 March 2021 21:06
> > > To: Steve Sakoman <steve@sakoman.com>
> > > Cc: rustyhowell@gmail.com; Yocto (yocto@lists.yoctoproject.org)
> > > <yocto@lists.yoctoproject.org>
> > > Subject: Re: [yocto] Reducing the perl footprint on my image
> > >
> > > On Tue, Mar 2, 2021 at 10:01 AM Steve Sakoman via
> > > lists.yoctoproject.org <steve=sakoman.com@lists.yoctoproject.org>
> > > wrote:
> > > >
> > > > On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@gmail.com> wrote:
> > > > >
> > > > > I have an image that is using debian package management
> > > (PACKAGE_CLASSES = "package_deb").  Because apt and dpkg require perl,
> > > perl is being installed in the image.   No problem.  Except that the
> entire perl
> > > stack is 669 packages.
> > > >
> > > > I just took a look at the manifest for one of my images that includes
> > > > PACKAGE_CLASSES = "package_deb".  I see the perl package plus 43
> > > > perl-module packages.  Are you sure that something else in your
> images
> > > > isn't pulling in all of those other perl-module packages?
> > >
> > > It just occurred to me to make sure you are looking in the image
> > > manifest to see which packages are actually installed in your image.
> > > The perl recipe does generate 676 packages (in dunfell) so perhaps you
> > > might be looking at the generated packages rather than the installed
> > > packages??
> > >
> >
> > I encountered a similar problem with package management enabled and rpm
> > as package format, where I also just install rpm for package management
> > and not all dnf stack. I get quite a lot of perl and python packages
> > into the image which are pulled by the rpm package, but they are only
> > needed for things like rpm-build, rpm-sign, etc., not for the bare
> > rpm command, which is the only one I need in the image.
> >
> > So I locally extended the rpm recipe to split those tools into
> rpm-build,
> > rpm-sign and rpm-archive and skip those packages in the image. I should
> > probably send patches for that to oe-core. Is that something that could
> > be accepted?
>
> Not sure they need to go to separate packages but moving those
> three to some kind of "build" package would make a lot of sense
> to me at least.
>
> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 4583 bytes --]

Re: [yocto] Reducing the perl footprint on my image

[Tim Orling]

[-- Attachment #1: Type: text/plain, Size: 4547 bytes --]

On Wed, Mar 3, 2021 at 7:06 AM Rusty Howell <rustyhowell@gmail.com> wrote:

> Steve, you're right.  My image.bb is adding various lmsensors packages to
> the IMAGE_INSTALL, some of which depend on perl-modules.   perl-modules
> seems to be the big monster, not perl.  We need lmsensors.  So I guess my
> question is really, how can I (or can I at all) reduce the footprint of
> perl-modules?  perl-modules doesn't have any Depends, rather it has a long
> list (~ 670) of Recommends.
>
>
If you look at the lmsensors recipe, the lmsensors-sensorsdetect [1] and
lmsensors-sensorsconfconvert [2] subpackages both have perl-modules in
RDEPENDS.
One way to determine the ACTUAL dependencies is to run the perl.req script
from rpm [3].

$ perl.req sensors-detect
perl >= 0:5.004
perl(Fcntl)
perl(File::Basename)
perl(constant)
perl(strict)
perl(vars)

$ perl.req sensors-conf-convert
perl(strict)
perl(vars)

You then need to translate those into the perl-modules-* subpackages, which
is easiest to see in perl-rdepends.txt [4].

I have not built nor tested this, but using the described approach I have a
patch for lmsensors [5].

Please cherry-pick, test and if it works, add your signed-off-by and submit
to the mailing list.

[1]
https://git.openembedded.org/meta-openembedded/tree/meta-oe/recipes-bsp/lm_sensors/lmsensors_3.6.0.bb#n154
[2]
https://git.openembedded.org/meta-openembedded/tree/meta-oe/recipes-bsp/lm_sensors/lmsensors_3.6.0.bb#n159
[3]
https://github.com/rpm-software-management/rpm/blob/master/scripts/perl.req
[4]
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/perl/files/perl-rdepends.txt
[5]
https://git.openembedded.org/meta-openembedded-contrib/commit/?h=timo/lmsensors-perldeps&id=467a6bdce3877b27253c932d806f853b916a8f9e

On Tue, Mar 2, 2021 at 1:49 PM Richard Purdie <
> richard.purdie@linuxfoundation.org> wrote:
>
>> On Tue, 2021-03-02 at 20:42 +0000, Diego Santa Cruz via
>> lists.yoctoproject.org wrote:
>> > > -----Original Message-----
>> > > From: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> On
>> > > Behalf Of Steve Sakoman via lists.yoctoproject.org
>> > > Sent: 02 March 2021 21:06
>> > > To: Steve Sakoman <steve@sakoman.com>
>> > > Cc: rustyhowell@gmail.com; Yocto (yocto@lists.yoctoproject.org)
>> > > <yocto@lists.yoctoproject.org>
>> > > Subject: Re: [yocto] Reducing the perl footprint on my image
>> > >
>> > > On Tue, Mar 2, 2021 at 10:01 AM Steve Sakoman via
>> > > lists.yoctoproject.org <steve=sakoman.com@lists.yoctoproject.org>
>> > > wrote:
>> > > >
>> > > > On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@gmail.com> wrote:
>> > > > >
>> > > > > I have an image that is using debian package management
>> > > (PACKAGE_CLASSES = "package_deb").  Because apt and dpkg require perl,
>> > > perl is being installed in the image.   No problem.  Except that the
>> entire perl
>> > > stack is 669 packages.
>> > > >
>> > > > I just took a look at the manifest for one of my images that
>> includes
>> > > > PACKAGE_CLASSES = "package_deb".  I see the perl package plus 43
>> > > > perl-module packages.  Are you sure that something else in your
>> images
>> > > > isn't pulling in all of those other perl-module packages?
>> > >
>> > > It just occurred to me to make sure you are looking in the image
>> > > manifest to see which packages are actually installed in your image.
>> > > The perl recipe does generate 676 packages (in dunfell) so perhaps you
>> > > might be looking at the generated packages rather than the installed
>> > > packages??
>> > >
>> >
>> > I encountered a similar problem with package management enabled and rpm
>> > as package format, where I also just install rpm for package management
>> > and not all dnf stack. I get quite a lot of perl and python packages
>> > into the image which are pulled by the rpm package, but they are only
>> > needed for things like rpm-build, rpm-sign, etc., not for the bare
>> > rpm command, which is the only one I need in the image.
>> >
>> > So I locally extended the rpm recipe to split those tools into
>> rpm-build,
>> > rpm-sign and rpm-archive and skip those packages in the image. I should
>> > probably send patches for that to oe-core. Is that something that could
>> > be accepted?
>>
>> Not sure they need to go to separate packages but moving those
>> three to some kind of "build" package would make a lot of sense
>> to me at least.
>>
>> Cheers,
>>
>> Richard
>>
>>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 7356 bytes --]

Re: [yocto] Reducing the perl footprint on my image

[Tim Orling]

[-- Attachment #1: Type: text/plain, Size: 5089 bytes --]

On Wed, Mar 3, 2021 at 12:28 PM Tim Orling via lists.yoctoproject.org
<ticotimo=gmail.com@lists.yoctoproject.org> wrote:

>
>
> On Wed, Mar 3, 2021 at 7:06 AM Rusty Howell <rustyhowell@gmail.com> wrote:
>
>> Steve, you're right.  My image.bb is adding various lmsensors packages
>> to the IMAGE_INSTALL, some of which depend on perl-modules.   perl-modules
>> seems to be the big monster, not perl.  We need lmsensors.  So I guess my
>> question is really, how can I (or can I at all) reduce the footprint of
>> perl-modules?  perl-modules doesn't have any Depends, rather it has a long
>> list (~ 670) of Recommends.
>>
>>
> If you look at the lmsensors recipe, the lmsensors-sensorsdetect [1] and
> lmsensors-sensorsconfconvert [2] subpackages both have perl-modules in
> RDEPENDS.
> One way to determine the ACTUAL dependencies is to run the perl.req script
> from rpm [3].
>

Related bug [6]

>
> $ perl.req sensors-detect
> perl >= 0:5.004
> perl(Fcntl)
> perl(File::Basename)
> perl(constant)
> perl(strict)
> perl(vars)
>
> $ perl.req sensors-conf-convert
> perl(strict)
> perl(vars)
>
> You then need to translate those into the perl-modules-* subpackages,
> which is easiest to see in perl-rdepends.txt [4].
>
> I have not built nor tested this, but using the described approach I have
> a patch for lmsensors [5].
>
> Please cherry-pick, test and if it works, add your signed-off-by and
> submit to the mailing list.
>
> [1]
> https://git.openembedded.org/meta-openembedded/tree/meta-oe/recipes-bsp/lm_sensors/lmsensors_3.6.0.bb#n154
> [2]
> https://git.openembedded.org/meta-openembedded/tree/meta-oe/recipes-bsp/lm_sensors/lmsensors_3.6.0.bb#n159
> [3]
> https://github.com/rpm-software-management/rpm/blob/master/scripts/perl.req
> [4]
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/perl/files/perl-rdepends.txt
> [5]
> https://git.openembedded.org/meta-openembedded-contrib/commit/?h=timo/lmsensors-perldeps&id=467a6bdce3877b27253c932d806f853b916a8f9e
>
[6] https://bugzilla.yoctoproject.org/show_bug.cgi?id=13376

>
> <https://git.openembedded.org/meta-openembedded-contrib/commit/?h=timo/lmsensors-perldeps&id=467a6bdce3877b27253c932d806f853b916a8f9e>
>
> On Tue, Mar 2, 2021 at 1:49 PM Richard Purdie <
>> richard.purdie@linuxfoundation.org> wrote:
>>
>>> On Tue, 2021-03-02 at 20:42 +0000, Diego Santa Cruz via
>>> lists.yoctoproject.org wrote:
>>> > > -----Original Message-----
>>> > > From: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> On
>>> > > Behalf Of Steve Sakoman via lists.yoctoproject.org
>>> > > Sent: 02 March 2021 21:06
>>> > > To: Steve Sakoman <steve@sakoman.com>
>>> > > Cc: rustyhowell@gmail.com; Yocto (yocto@lists.yoctoproject.org)
>>> > > <yocto@lists.yoctoproject.org>
>>> > > Subject: Re: [yocto] Reducing the perl footprint on my image
>>> > >
>>> > > On Tue, Mar 2, 2021 at 10:01 AM Steve Sakoman via
>>> > > lists.yoctoproject.org <steve=sakoman.com@lists.yoctoproject.org>
>>> > > wrote:
>>> > > >
>>> > > > On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@gmail.com> wrote:
>>> > > > >
>>> > > > > I have an image that is using debian package management
>>> > > (PACKAGE_CLASSES = "package_deb").  Because apt and dpkg require
>>> perl,
>>> > > perl is being installed in the image.   No problem.  Except that the
>>> entire perl
>>> > > stack is 669 packages.
>>> > > >
>>> > > > I just took a look at the manifest for one of my images that
>>> includes
>>> > > > PACKAGE_CLASSES = "package_deb".  I see the perl package plus 43
>>> > > > perl-module packages.  Are you sure that something else in your
>>> images
>>> > > > isn't pulling in all of those other perl-module packages?
>>> > >
>>> > > It just occurred to me to make sure you are looking in the image
>>> > > manifest to see which packages are actually installed in your image.
>>> > > The perl recipe does generate 676 packages (in dunfell) so perhaps
>>> you
>>> > > might be looking at the generated packages rather than the installed
>>> > > packages??
>>> > >
>>> >
>>> > I encountered a similar problem with package management enabled and
>>> rpm
>>> > as package format, where I also just install rpm for package
>>> management
>>> > and not all dnf stack. I get quite a lot of perl and python packages
>>> > into the image which are pulled by the rpm package, but they are only
>>> > needed for things like rpm-build, rpm-sign, etc., not for the bare
>>> > rpm command, which is the only one I need in the image.
>>> >
>>> > So I locally extended the rpm recipe to split those tools into
>>> rpm-build,
>>> > rpm-sign and rpm-archive and skip those packages in the image. I
>>> should
>>> > probably send patches for that to oe-core. Is that something that could
>>> > be accepted?
>>>
>>> Not sure they need to go to separate packages but moving those
>>> three to some kind of "build" package would make a lot of sense
>>> to me at least.
>>>
>>> Cheers,
>>>
>>> Richard
>>>
>>>
>>
>>
>>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 9088 bytes --]

Re: [yocto] Reducing the perl footprint on my image

[Diego Santa Cruz]

> -----Original Message-----
> From: Richard Purdie <richard.purdie@linuxfoundation.org>
> Sent: 02 March 2021 21:50
> To: Diego Santa Cruz <Diego.SantaCruz@spinetix.com>;
> steve@sakoman.com
> Cc: rustyhowell@gmail.com; Yocto (yocto@lists.yoctoproject.org)
> <yocto@lists.yoctoproject.org>
> Subject: Re: [yocto] Reducing the perl footprint on my image
> 
[...]
> > So I locally extended the rpm recipe to split those tools into rpm-build,
> > rpm-sign and rpm-archive and skip those packages in the image. I should
> > probably send patches for that to oe-core. Is that something that could
> > be accepted?
> 
> Not sure they need to go to separate packages but moving those
> three to some kind of "build" package would make a lot of sense
> to me at least.

I'll be sending a patch to split it then. Having the sign and archive in separate packages allows to skip their dependencies in images too, and they are not really related to the rpm build part and the complexity delta is minimal; usually there is no need to have rpm-sign in a target image nor rpm2archive if the archive PACKAGECONFIG is enabled. 

Best,
Diego

-- 
Diego Santa Cruz, PhD
Technology Architect
spinetix.com