* Fw: Reducing rootfs size in yocto
[not found] <HK0PR04MB29144DAFD598E96154278145B4CD0@HK0PR04MB2914.apcprd04.prod.outlook.com>
@ 2020-03-28 13:25 ` ajama
2020-03-28 14:08 ` [yocto] " Konrad Weihmann
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: ajama @ 2020-03-28 13:25 UTC (permalink / raw)
To: yocto
[-- Attachment #1: Type: text/plain, Size: 1496 bytes --]
________________________________
Hi All,
I am trying to reduce the image size.
how can i reduce the size of following packages so that i could get a small image size after integrating these packages in yocto?
1) boost-dev 2) docker 3) containerd-opencontainers 4) nodejs 5) nodejs-npm 6) python3-numpy 7) cpp 8) postgresql
Thanks in advance,
Ajam Ali
::DISCLAIMER::
________________________________
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
________________________________
[-- Attachment #2: Type: text/html, Size: 3303 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [yocto] Fw: Reducing rootfs size in yocto
2020-03-28 13:25 ` Fw: Reducing rootfs size in yocto ajama
@ 2020-03-28 14:08 ` Konrad Weihmann
2020-03-28 18:36 ` Ajam Ali
2020-03-29 14:46 ` Oliver Westermann
2020-03-30 6:57 ` Mikko Rapeli
2 siblings, 1 reply; 8+ messages in thread
From: Konrad Weihmann @ 2020-03-28 14:08 UTC (permalink / raw)
To: Ajam Ali; +Cc: yocto
[-- Attachment #1: Type: text/plain, Size: 2985 bytes --]
Pretty general questions, but I try to answer your question.
I don't know what the purpose of your image is, but installing
development tools like boost-dev or gcc (I guess that is what you mean
with cpp) makes your image automatically a lot bigger, I would suggest
to put devtools into a SDK, which could be shipped separately.
Next thing you can do is to analyze the what is actually so big in your
image, e.g. by creating a tar.gz of the image, unpacking it and browsing
by 'du -sh *'.
You can get the package/recipe to a file by using 'oe-pkgdata-util
find-path <path>'.
In the identified recipe you can look for PACKAGECONFIG or OE_EXTRACONF
switches that you don't need and disable them.
As said, I don't know what the purpose/environment of your image will be
like, but you could also consider to use a different fs-format like squash.
In general if you want small images preferring musl over glibc and
systemV over systemd is something that you could consider as well.
TL;DR
1) move to SDK
2) check PACKAGECONFIG switches
3) same as 2 or consider something "smaller" like lxc
4) same as 2
5) same as 2
6) same as 2
7) move to SDK
8) same as 2
On 28.03.20 14:25, Ajam Ali wrote:
>
>
> ------------------------------------------------------------------------
> Hi All,
>
> I am trying to reduce the image size.
>
> how can i reduce the size of following packages so that i could get a
> small image size after integrating these packages in yocto?
>
>
> 1) boost-dev 2) docker 3) containerd-opencontainers 4) nodejs 5)
> nodejs-npm 6) python3-numpy 7) cpp 8) postgresql
>
>
>
> Thanks in advance,
>
> Ajam Ali
>
>
> ::DISCLAIMER::
> ------------------------------------------------------------------------
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only. E-mail transmission is not
> guaranteed to be secure or error-free as information could be
> intercepted, corrupted, lost, destroyed, arrive late or incomplete, or
> may contain viruses in transmission. The e mail and its contents (with
> or without referred errors) shall therefore not attach any liability
> on the originator or HCL or its affiliates. Views or opinions, if any,
> presented in this email are solely those of the author and may not
> necessarily reflect the views or opinions of HCL or its affiliates.
> Any form of reproduction, dissemination, copying, disclosure,
> modification, distribution and / or publication of this message
> without the prior written consent of authorized representative of HCL
> is strictly prohibited. If you have received this email in error
> please delete it and notify the sender immediately. Before opening any
> email and/or attachments, please check them for viruses and other defects.
> ------------------------------------------------------------------------
>
>
[-- Attachment #2: Type: text/html, Size: 6086 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [yocto] Fw: Reducing rootfs size in yocto
2020-03-28 14:08 ` [yocto] " Konrad Weihmann
@ 2020-03-28 18:36 ` Ajam Ali
2020-03-28 19:45 ` Konrad Weihmann
0 siblings, 1 reply; 8+ messages in thread
From: Ajam Ali @ 2020-03-28 18:36 UTC (permalink / raw)
To: Konrad Weihmann; +Cc: yocto
[-- Attachment #1: Type: text/plain, Size: 3437 bytes --]
Hi Konrad,
Actually my current image size is 3.9GB because of heavy size packages required by my project and without project required packages my image size in Yocto is 800MB.
I want to reduce the image size as maximum as possible.
Thanks a lot in advance,
Ajam Ali
________________________________
From: Konrad Weihmann <kweihmann@outlook.com>
Sent: Saturday, March 28, 2020 7:38 PM
To: Ajam Ali <AjamA@hcl.com>
Cc: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org>
Subject: Re: [yocto] Fw: Reducing rootfs size in yocto
[CAUTION: This Email is from outside the Organization. Do not click links or open attachments unless you trust the sender.]
Pretty general questions, but I try to answer your question.
I don't know what the purpose of your image is, but installing development tools like boost-dev or gcc (I guess that is what you mean with cpp) makes your image automatically a lot bigger, I would suggest to put devtools into a SDK, which could be shipped separately.
Next thing you can do is to analyze the what is actually so big in your image, e.g. by creating a tar.gz of the image, unpacking it and browsing by 'du -sh *'.
You can get the package/recipe to a file by using 'oe-pkgdata-util find-path <path>'.
In the identified recipe you can look for PACKAGECONFIG or OE_EXTRACONF switches that you don't need and disable them.
As said, I don't know what the purpose/environment of your image will be like, but you could also consider to use a different fs-format like squash.
In general if you want small images preferring musl over glibc and systemV over systemd is something that you could consider as well.
TL;DR
1) move to SDK
2) check PACKAGECONFIG switches
3) same as 2 or consider something "smaller" like lxc
4) same as 2
5) same as 2
6) same as 2
7) move to SDK
8) same as 2
On 28.03.20 14:25, Ajam Ali wrote:
________________________________
Hi All,
I am trying to reduce the image size.
how can i reduce the size of following packages so that i could get a small image size after integrating these packages in yocto?
1) boost-dev 2) docker 3) containerd-opencontainers 4) nodejs 5) nodejs-npm 6) python3-numpy 7) cpp 8) postgresql
Thanks in advance,
Ajam Ali
::DISCLAIMER::
________________________________
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
________________________________
[-- Attachment #2: Type: text/html, Size: 7453 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [yocto] Fw: Reducing rootfs size in yocto
2020-03-28 18:36 ` Ajam Ali
@ 2020-03-28 19:45 ` Konrad Weihmann
0 siblings, 0 replies; 8+ messages in thread
From: Konrad Weihmann @ 2020-03-28 19:45 UTC (permalink / raw)
To: Ajam Ali; +Cc: yocto
[-- Attachment #1: Type: text/plain, Size: 5055 bytes --]
HI,
800MB sounds fairly much for just the base system - maybe there is some
potential for you to slim it down.
If your project requires all these packages, I' afraid there isn't an
easy magical switch to reduce the size.
What you could do is take the image manifest
(https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#var-IMAGE_MANIFEST)
and browse manually though it, if there
is any obvious package that shouldn't be installed, which could be
removed - but to be honest 3.9GB for all that stuff required according
to your list sounds like a reasonable size for the rootfs.
If you already know what the real "big ones" are in your image you could
try to configure them to include only things really needed - if not you
could also take the
image manifest feed the information into "oe-pkgdata-util package-info
<pkgname>" to get the size of the package.
I guess from here on you can take it.
In the end 3.9GB for a fully fledged development environment doesn't
sounds too much - just remember a vanilla ubuntu for example takes at
least 15GB.
BR
Konrad
On 28.03.20 19:36, Ajam Ali wrote:
> Hi Konrad,
>
> Actually my current image size is 3.9GB because of heavy size packages
> required by my project and without project required packages my image
> size in Yocto is 800MB.
>
> I want to reduce the image size as maximum as possible.
>
>
> Thanks a lot in advance,
> Ajam Ali
> ------------------------------------------------------------------------
> *From:* Konrad Weihmann <kweihmann@outlook.com>
> *Sent:* Saturday, March 28, 2020 7:38 PM
> *To:* Ajam Ali <AjamA@hcl.com>
> *Cc:* yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org>
> *Subject:* Re: [yocto] Fw: Reducing rootfs size in yocto
>
> **[CAUTION: This Email is from outside the Organization. Do not click
> links or open attachments unless you trust the sender.]
>
> Pretty general questions, but I try to answer your question.
>
>
> I don't know what the purpose of your image is, but installing
> development tools like boost-dev or gcc (I guess that is what you mean
> with cpp) makes your image automatically a lot bigger, I would suggest
> to put devtools into a SDK, which could be shipped separately.
>
>
> Next thing you can do is to analyze the what is actually so big in
> your image, e.g. by creating a tar.gz of the image, unpacking it and
> browsing by 'du -sh *'.
> You can get the package/recipe to a file by using 'oe-pkgdata-util
> find-path <path>'.
>
>
> In the identified recipe you can look for PACKAGECONFIG or
> OE_EXTRACONF switches that you don't need and disable them.
>
>
> As said, I don't know what the purpose/environment of your image will
> be like, but you could also consider to use a different fs-format like
> squash.
>
>
> In general if you want small images preferring musl over glibc and
> systemV over systemd is something that you could consider as well.
>
>
> TL;DR
>
>
> 1) move to SDK
>
> 2) check PACKAGECONFIG switches
>
> 3) same as 2 or consider something "smaller" like lxc
>
> 4) same as 2
>
> 5) same as 2
>
> 6) same as 2
>
> 7) move to SDK
>
> 8) same as 2
>
>
> On 28.03.20 14:25, Ajam Ali wrote:
>>
>>
>> ------------------------------------------------------------------------
>> Hi All,
>>
>> I am trying to reduce the image size.
>>
>> how can i reduce the size of following packages so that i could get
>> a small image size after integrating these packages in yocto?
>>
>>
>> 1) boost-dev 2) docker 3) containerd-opencontainers 4) nodejs 5)
>> nodejs-npm 6) python3-numpy 7) cpp 8) postgresql
>>
>>
>>
>> Thanks in advance,
>>
>> Ajam Ali
>>
>>
>> ::DISCLAIMER::
>> ------------------------------------------------------------------------
>> The contents of this e-mail and any attachment(s) are confidential
>> and intended for the named recipient(s) only. E-mail transmission is
>> not guaranteed to be secure or error-free as information could be
>> intercepted, corrupted, lost, destroyed, arrive late or incomplete,
>> or may contain viruses in transmission. The e mail and its contents
>> (with or without referred errors) shall therefore not attach any
>> liability on the originator or HCL or its affiliates. Views or
>> opinions, if any, presented in this email are solely those of the
>> author and may not necessarily reflect the views or opinions of HCL
>> or its affiliates. Any form of reproduction, dissemination, copying,
>> disclosure, modification, distribution and / or publication of this
>> message without the prior written consent of authorized
>> representative of HCL is strictly prohibited. If you have received
>> this email in error please delete it and notify the sender
>> immediately. Before opening any email and/or attachments, please
>> check them for viruses and other defects.
>> ------------------------------------------------------------------------
>>
>>
[-- Attachment #2: Type: text/html, Size: 11956 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fw: Reducing rootfs size in yocto
2020-03-28 13:25 ` Fw: Reducing rootfs size in yocto ajama
2020-03-28 14:08 ` [yocto] " Konrad Weihmann
@ 2020-03-29 14:46 ` Oliver Westermann
2020-03-30 4:59 ` [yocto] " Ajam Ali
2020-03-30 6:57 ` Mikko Rapeli
2 siblings, 1 reply; 8+ messages in thread
From: Oliver Westermann @ 2020-03-29 14:46 UTC (permalink / raw)
To: yocto
[-- Attachment #1: Type: text/plain, Size: 54 bytes --]
Do you need the -dev version of boost on the device?
[-- Attachment #2: Type: text/html, Size: 54 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [yocto] Fw: Reducing rootfs size in yocto
2020-03-29 14:46 ` Oliver Westermann
@ 2020-03-30 4:59 ` Ajam Ali
0 siblings, 0 replies; 8+ messages in thread
From: Ajam Ali @ 2020-03-30 4:59 UTC (permalink / raw)
To: yocto, oliver.westermann
[-- Attachment #1: Type: text/plain, Size: 1875 bytes --]
Hi,
-dev version of boost is a supporting package which is getting installed by default
not by me using local.conf.
Thanks,
Ajam Ali
________________________________
From: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> on behalf of Oliver Westermann via Lists.Yoctoproject.Org <oliver.westermann=cognex.com@lists.yoctoproject.org>
Sent: Sunday, March 29, 2020 8:16 PM
To: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org>
Cc: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org>
Subject: Re: [yocto] Fw: Reducing rootfs size in yocto
[CAUTION: This Email is from outside the Organization. Do not click links or open attachments unless you trust the sender.]
Do you need the -dev version of boost on the device?
::DISCLAIMER::
________________________________
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
________________________________
[-- Attachment #2: Type: text/html, Size: 3653 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [yocto] Fw: Reducing rootfs size in yocto
2020-03-28 13:25 ` Fw: Reducing rootfs size in yocto ajama
2020-03-28 14:08 ` [yocto] " Konrad Weihmann
2020-03-29 14:46 ` Oliver Westermann
@ 2020-03-30 6:57 ` Mikko Rapeli
2020-04-08 22:12 ` Randy MacLeod
2 siblings, 1 reply; 8+ messages in thread
From: Mikko Rapeli @ 2020-03-30 6:57 UTC (permalink / raw)
To: ajama; +Cc: yocto
Hi,
Generic approaches for reducing yocto target image size where buildhistory
is an important source of information:
* review and minimize distro features, enable only what you need
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-features-distro
* review and minimize image recipe, install only those image features and packages
which you need
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-features-image
* review the needed binary packages: traverse the tree of dependencies what you need,
review every single recipe and change PACKAGECONFIG or build flags to remove
features which you do not need. This it may be possible to remove large set
of dependencies from images, e.g. language bindings, debug tools. In some cases
one just needs a single binary executable or shared library but it is bundled
in a larger binary package with more complex dependencies. For these cases
add a bbappend to custom layers to change the packaging to only include what
you need.
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-PACKAGECONFIG
* change compiler flags from default O2 to Os optimization. This can work for
some recipes which are not prerformance critical or to the whole build. Note
that many SW components break this in their own build scripts by adding
O2 or even O3 by default. Check from buildhistory that binary sizes go smaller
after switch from O2 to Os and review every important recipe which did not.
You may be able to save up to 15% in rootfs size this way, but all depends
on the details of SW components and image contents.
see poky/meta/conf/bitbake.conf
I've followed this for multiple products and found out that poky is nice to work with
but BSP layers add or depend on all kinds of extra things that products do not need.
Thus I have ended by BBMASK'ing away large parts of BSP layer recipes in distro
configs. This with multiple x86 and ARM BSP layers from various SoC vendors.
Also, RTFM :)
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#building-a-tiny-system
Hope this helps,
-Mikko
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [yocto] Fw: Reducing rootfs size in yocto
2020-03-30 6:57 ` Mikko Rapeli
@ 2020-04-08 22:12 ` Randy MacLeod
0 siblings, 0 replies; 8+ messages in thread
From: Randy MacLeod @ 2020-04-08 22:12 UTC (permalink / raw)
To: Mikko Rapeli, ajama; +Cc: yocto
On 2020-03-30 2:57 a.m., Mikko Rapeli wrote:
> Hi,
>
> Generic approaches for reducing yocto target image size where buildhistory
> is an important source of information:
>
> * review and minimize distro features, enable only what you need
> https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-features-distro
>
> * review and minimize image recipe, install only those image features and packages
> which you need
> https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-features-image
>
> * review the needed binary packages: traverse the tree of dependencies what you need,
> review every single recipe and change PACKAGECONFIG or build flags to remove
> features which you do not need. This it may be possible to remove large set
> of dependencies from images, e.g. language bindings, debug tools. In some cases
> one just needs a single binary executable or shared library but it is bundled
> in a larger binary package with more complex dependencies. For these cases
> add a bbappend to custom layers to change the packaging to only include what
> you need.
> https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-PACKAGECONFIG
>
> * change compiler flags from default O2 to Os optimization. This can work for
> some recipes which are not prerformance critical or to the whole build. Note
> that many SW components break this in their own build scripts by adding
> O2 or even O3 by default. Check from buildhistory that binary sizes go smaller
> after switch from O2 to Os and review every important recipe which did not.
> You may be able to save up to 15% in rootfs size this way, but all depends
> on the details of SW components and image contents.
> see poky/meta/conf/bitbake.conf
>
> I've followed this for multiple products and found out that poky is nice to work with
> but BSP layers add or depend on all kinds of extra things that products do not need.
> Thus I have ended by BBMASK'ing away large parts of BSP layer recipes in distro
> configs. This with multiple x86 and ARM BSP layers from various SoC vendors.
I've also seen this problem and was frustrated by it.
The BSP vendors/developers want to test the full feature set of the
hardware but they often don't bother stripping out packages after
confirming that everything works. This results in bloat that can
be a problem for some users.
Using BBMASK is a good coping mechanism! :)
It would also be good if BSP developers split the BSP into the
essential elements and extras. Where to draw the line between
two such categories can tricky. One could also have finer-grained
packagegroups but splitting things into essential and extras is
a good start.
In WR Linux, we have a setup tool that let's you add 'features'
This comes from a layer that improves ease of use for some people:
https://github.com/WindRiver-Labs/wr-template/blob/WRLINUX_10_19_BASE/README
and is always used in the WR Linux distro.
And as can be seen here:
https://github.com/WindRiver-Labs/wrlinux/tree/WRLINUX_10_19_BASE/wrlinux-distro/templates/feature/bsp-extras
There is a generic BSP independent feature that you can add to
a setup:
feature/bsp-extras
Adds some useful tools defined by the BSP_EXTRAS_PACKAGES option
in the BSP layer.
To see the list of available tools, run the bitbake -e command in
a sourced terminal, and look for the BSP_EXTRAS_PACKAGES.
Maybe this description will help get BSP developers to agree on
doing something similar regardless of the 'setup/wr-template' layer.
Good luck and keep your BSPs minimal!
../Randy
>
> Also, RTFM :)
>
> https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#building-a-tiny-system
>
> Hope this helps,
>
> -Mikko
>
>
>
>
--
# Randy MacLeod
# Wind River Linux
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2020-04-08 22:13 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <HK0PR04MB29144DAFD598E96154278145B4CD0@HK0PR04MB2914.apcprd04.prod.outlook.com>
2020-03-28 13:25 ` Fw: Reducing rootfs size in yocto ajama
2020-03-28 14:08 ` [yocto] " Konrad Weihmann
2020-03-28 18:36 ` Ajam Ali
2020-03-28 19:45 ` Konrad Weihmann
2020-03-29 14:46 ` Oliver Westermann
2020-03-30 4:59 ` [yocto] " Ajam Ali
2020-03-30 6:57 ` Mikko Rapeli
2020-04-08 22:12 ` Randy MacLeod
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.