* [Buildroot] external, how to achieve our needs? @ 2020-02-25 15:01 Ludovic Desroches 2020-02-26 5:36 ` Baruch Siach ` (2 more replies) 0 siblings, 3 replies; 8+ messages in thread From: Ludovic Desroches @ 2020-02-25 15:01 UTC (permalink / raw) To: buildroot Hi, We are currently using it for linux4sam but it seems it has limitations and I would like to know if you have advice to share or the same issue. I did a quick search in the mailing list, I found a discussion with few answers and patches from Yann for external toolchains, jpeg and openssl. Unfortunately, it doesn't cover all our use cases. At the beginning, we forked buildroot in the buildroot-at91 tree. It was mostly done to put patches we sent upstream. We release a distribution called linux4sam which contains bootloaders, kernel and rootfs. We can't wait for the next release of Buildroot to get our patches. It was also useful to handle additional defconfigs, buildroot bug fixes and sometimes patches that are not accepted upstream. When the external feature was released, we thought that it should be better to rely on Buildroot vanilla and to release only our external. The goal was to remove the buildroot-at91 fork. At least, it was the plan... Now, we still have buildroot-at91 + buildroot-external-microchip trees. We faced several problems which make difficult to only provide an external. One of the latest issue is we are adding support for a GPU in cairo and this work is not upstream yet. It involves patching cairo sources (that's not a big deal) but also to modify cairo.mk to add dependencies and new configure options. I discussed with Thomas who helped me to achieve it with some tricks. I added a cairo_microchip package, I had to patch some libraries to link with this version of cairo and use the external.mk to force extra dependencies for cairo and pango. Honestly, even if it works, I am not pleased with the solution. It will be so much easier and cleaner to do this in buildroot-at91 instead of buildroot-external-microchip. We discussed again, internally, about the external and the fork of buildroot but we have different opinions. I would like to emphasize that recent changes in buildroot make us think that it won't be easy to get rid of buildroot-at91: devmem2 deprecated, rgnd which increases the boot time since it uses the lib jitterentropy, curl library name change and others. My feeling is that the external is not flexible enough to allow us to remove buildroot-at91. Is it planned to offer a way to modify package configs and makefiles or is it totally against Buildroot philosophy? I think this is a strength of the Yocto layers but it's also a weakness as it becomes difficult to know what is built and how. I also wonder if we can have one version of the external to support several versions of buildroot or if we'll need to tag it according to the version of buildroot it has been tested against. Do we put too much hope in the external? If I try to summarize our options with their pros and cons: - buildroot fork: - pros: - one tree - full control, can do whatever we want - cons: - fork - need to rebase patches for new releases - may not be able to keep the pace of buildroot releases - buildroot external: - pros: - one tree - not dependent on buildroot version (in theory) so no rebase to do - cons: - can't modify buildroot package makefiles, only the source code - can't quickly solve bugs in buildroot - may have dependency on the buildroot version - buildroot fork + external: - pros: - full control, can do whatever we want - less patches to rebase (only the ones on top of buildroot) - cons: - fork - two trees - may not be able to keep the pace of buildroot releases - may have buildroot fork with no patches on top of it sometime If you have any suggestions, about improvements for Buildroot or the way we manage our Buildroot offer, I would be happy to hear them. Regards, Ludovic ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] external, how to achieve our needs? 2020-02-25 15:01 [Buildroot] external, how to achieve our needs? Ludovic Desroches @ 2020-02-26 5:36 ` Baruch Siach 2020-02-26 13:20 ` Ludovic Desroches 2020-02-26 12:11 ` Mircea GLIGA 2020-02-26 13:30 ` Mircea GLIGA 2 siblings, 1 reply; 8+ messages in thread From: Baruch Siach @ 2020-02-26 5:36 UTC (permalink / raw) To: buildroot Hi Ludovic, On Tue, Feb 25 2020, Ludovic Desroches wrote: > I would like to emphasize that recent changes in buildroot make us > think that it won't be easy to get rid of buildroot-at91: devmem2 > deprecated, rgnd which increases the boot time since it uses the lib > jitterentropy, curl library name change and others. The curl library name has not changed in the last few year. Do you refer to the curl binary Kconfig symbol name change (commit 2a057339cc12)? How does that affect your use case? Do you need to support several Buildroot versions in a single config? baruch -- http://baruch.siach.name/blog/ ~. .~ Tk Open Systems =}------------------------------------------------ooO--U--Ooo------------{= - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il - ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] external, how to achieve our needs? 2020-02-26 5:36 ` Baruch Siach @ 2020-02-26 13:20 ` Ludovic Desroches 0 siblings, 0 replies; 8+ messages in thread From: Ludovic Desroches @ 2020-02-26 13:20 UTC (permalink / raw) To: buildroot Hi Baruch, On Wed, Feb 26, 2020 at 07:36:00AM +0200, Baruch Siach wrote: > Hi Ludovic, > > On Tue, Feb 25 2020, Ludovic Desroches wrote: > > I would like to emphasize that recent changes in buildroot make us > > think that it won't be easy to get rid of buildroot-at91: devmem2 > > deprecated, rgnd which increases the boot time since it uses the lib > > jitterentropy, curl library name change and others. > > The curl library name has not changed in the last few year. Do you refer > to the curl binary Kconfig symbol name change (commit 2a057339cc12)? How > does that affect your use case? Do you need to support several Buildroot > versions in a single config? Yes, that's the Kconfig symbol. That's one of the pending question, is it possible to support different versions of Buildroot. When we start using the external, one benefit we had in mind was we won't have to check/update the external at each new release of Buildroot. It seems we were wrong, it could only save us to rebase patches in comparison to the buildroot fork solution. Regards, Ludovic > > baruch > > -- > http://baruch.siach.name/blog/ ~. .~ Tk Open Systems > =}------------------------------------------------ooO--U--Ooo------------{= > - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il - > _______________________________________________ > buildroot mailing list > buildroot at busybox.net > http://lists.busybox.net/mailman/listinfo/buildroot ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] external, how to achieve our needs? 2020-02-25 15:01 [Buildroot] external, how to achieve our needs? Ludovic Desroches 2020-02-26 5:36 ` Baruch Siach @ 2020-02-26 12:11 ` Mircea GLIGA 2020-02-26 13:29 ` Ludovic Desroches 2020-02-26 13:30 ` Mircea GLIGA 2 siblings, 1 reply; 8+ messages in thread From: Mircea GLIGA @ 2020-02-26 12:11 UTC (permalink / raw) To: buildroot Hi Ludovic, I also have this kind of problems when implementing our external, sometimes is useful to 'override' or append to a recipe (Yocto style). * I ended up "duplicating" recipes in our layer, e.g. bd-tcpdump in order to build the package differently * where it was possible, I manipulated the original package variables, from a .mk file in the external layer: e.g modify the *_CONF_OPTS or *_CONF_ENV for some package I'm also not that pleased with this 'unofficial' solution. BR Mircea ________________________________ From: buildroot <buildroot-bounces@busybox.net> on behalf of Ludovic Desroches <ludovic.desroches@microchip.com> Sent: Tuesday, February 25, 2020 17:01 To: buildroot at buildroot.org <buildroot@buildroot.org> Cc: Eugen Hristev <Eugen.Hristev@microchip.com>; Joshua Henderson <Joshua.Henderson@microchip.com>; Nicolas Ferre <Nicolas.Ferre@Microchip.com>; Cristian Birsan <Cristian.Birsan@microchip.com>; Tudor Ambarus <Tudor.Ambarus@microchip.com> Subject: [Buildroot] external, how to achieve our needs? Hi, We are currently using it for linux4sam but it seems it has limitations and I would like to know if you have advice to share or the same issue. I did a quick search in the mailing list, I found a discussion with few answers and patches from Yann for external toolchains, jpeg and openssl. Unfortunately, it doesn't cover all our use cases. At the beginning, we forked buildroot in the buildroot-at91 tree. It was mostly done to put patches we sent upstream. We release a distribution called linux4sam which contains bootloaders, kernel and rootfs. We can't wait for the next release of Buildroot to get our patches. It was also useful to handle additional defconfigs, buildroot bug fixes and sometimes patches that are not accepted upstream. When the external feature was released, we thought that it should be better to rely on Buildroot vanilla and to release only our external. The goal was to remove the buildroot-at91 fork. At least, it was the plan... Now, we still have buildroot-at91 + buildroot-external-microchip trees. We faced several problems which make difficult to only provide an external. One of the latest issue is we are adding support for a GPU in cairo and this work is not upstream yet. It involves patching cairo sources (that's not a big deal) but also to modify cairo.mk to add dependencies and new configure options. I discussed with Thomas who helped me to achieve it with some tricks. I added a cairo_microchip package, I had to patch some libraries to link with this version of cairo and use the external.mk to force extra dependencies for cairo and pango. Honestly, even if it works, I am not pleased with the solution. It will be so much easier and cleaner to do this in buildroot-at91 instead of buildroot-external-microchip. We discussed again, internally, about the external and the fork of buildroot but we have different opinions. I would like to emphasize that recent changes in buildroot make us think that it won't be easy to get rid of buildroot-at91: devmem2 deprecated, rgnd which increases the boot time since it uses the lib jitterentropy, curl library name change and others. My feeling is that the external is not flexible enough to allow us to remove buildroot-at91. Is it planned to offer a way to modify package configs and makefiles or is it totally against Buildroot philosophy? I think this is a strength of the Yocto layers but it's also a weakness as it becomes difficult to know what is built and how. I also wonder if we can have one version of the external to support several versions of buildroot or if we'll need to tag it according to the version of buildroot it has been tested against. Do we put too much hope in the external? If I try to summarize our options with their pros and cons: - buildroot fork: - pros: - one tree - full control, can do whatever we want - cons: - fork - need to rebase patches for new releases - may not be able to keep the pace of buildroot releases - buildroot external: - pros: - one tree - not dependent on buildroot version (in theory) so no rebase to do - cons: - can't modify buildroot package makefiles, only the source code - can't quickly solve bugs in buildroot - may have dependency on the buildroot version - buildroot fork + external: - pros: - full control, can do whatever we want - less patches to rebase (only the ones on top of buildroot) - cons: - fork - two trees - may not be able to keep the pace of buildroot releases - may have buildroot fork with no patches on top of it sometime If you have any suggestions, about improvements for Buildroot or the way we manage our Buildroot offer, I would be happy to hear them. Regards, Ludovic _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot ________________________ This email was scanned by Bitdefender -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200226/568519a3/attachment.html> ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] external, how to achieve our needs? 2020-02-26 12:11 ` Mircea GLIGA @ 2020-02-26 13:29 ` Ludovic Desroches 2020-02-26 15:14 ` Peter Korsgaard 0 siblings, 1 reply; 8+ messages in thread From: Ludovic Desroches @ 2020-02-26 13:29 UTC (permalink / raw) To: buildroot Hi Mircea, On Wed, Feb 26, 2020 at 12:11:36PM +0000, Mircea GLIGA wrote: > Hi Ludovic, > > I also have this kind of problems when implementing our external, sometimes is useful to 'override' or append to a recipe (Yocto style). > > * I ended up "duplicating" recipes in our layer, e.g. bd-tcpdump in order to build the package differently From the top of my head, I tried this and I got an error when duplicating the recipes in the external. Did you use other tricks? > * where it was possible, I manipulated the original package variables, from a .mk file in the external layer: e.g modify the *_CONF_OPTS or *_CONF_ENV for some package I also have some hacks in external.mk as suggested by Thomas: $(LIBRSVG_TARGET_CONFIGURE):| cairo_microchip $(PANGO_TARGET_CONFIGURE):| cairo_microchip But it's not perfect, buildroot is not aware of the dependecy when you use target susch as show-depends. Regards, Ludovic > > I'm also not that pleased with this 'unofficial' solution. > > BR > Mircea > ________________________________ > From: buildroot <buildroot-bounces@busybox.net> on behalf of Ludovic Desroches <ludovic.desroches@microchip.com> > Sent: Tuesday, February 25, 2020 17:01 > To: buildroot at buildroot.org <buildroot@buildroot.org> > Cc: Eugen Hristev <Eugen.Hristev@microchip.com>; Joshua Henderson <Joshua.Henderson@microchip.com>; Nicolas Ferre <Nicolas.Ferre@Microchip.com>; Cristian Birsan <Cristian.Birsan@microchip.com>; Tudor Ambarus <Tudor.Ambarus@microchip.com> > Subject: [Buildroot] external, how to achieve our needs? > > Hi, > > We are currently using it for linux4sam but it seems it has limitations > and I would like to know if you have advice to share or the same issue. > I did a quick search in the mailing list, I found a discussion with few > answers and patches from Yann for external toolchains, jpeg and openssl. > Unfortunately, it doesn't cover all our use cases. > > At the beginning, we forked buildroot in the buildroot-at91 tree. It was > mostly done to put patches we sent upstream. We release a distribution > called linux4sam which contains bootloaders, kernel and rootfs. We can't > wait for the next release of Buildroot to get our patches. It was also > useful to handle additional defconfigs, buildroot bug fixes and sometimes > patches that are not accepted upstream. > > When the external feature was released, we thought that it should be better > to rely on Buildroot vanilla and to release only our external. The goal was > to remove the buildroot-at91 fork. At least, it was the plan... > > Now, we still have buildroot-at91 + buildroot-external-microchip trees. We > faced several problems which make difficult to only provide an external. > > One of the latest issue is we are adding support for a GPU in cairo and > this work is not upstream yet. It involves patching cairo sources > (that's not a big deal) but also to modify cairo.mk to add dependencies and > new configure options. I discussed with Thomas who helped me to achieve it > with some tricks. I added a cairo_microchip package, I had to patch some > libraries to link with this version of cairo and use the external.mk to > force extra dependencies for cairo and pango. Honestly, even if it works, > I am not pleased with the solution. It will be so much easier and cleaner > to do this in buildroot-at91 instead of buildroot-external-microchip. We > discussed again, internally, about the external and the fork of buildroot > but we have different opinions. I would like to emphasize that recent > changes in buildroot make us think that it won't be easy to get rid of > buildroot-at91: devmem2 deprecated, rgnd which increases the boot time > since it uses the lib jitterentropy, curl library name change and others. > > My feeling is that the external is not flexible enough to allow us to > remove buildroot-at91. Is it planned to offer a way to modify package > configs and makefiles or is it totally against Buildroot philosophy? I > think this is a strength of the Yocto layers but it's also a weakness as > it becomes difficult to know what is built and how. I also wonder if we can > have one version of the external to support several versions of buildroot > or if we'll need to tag it according to the version of buildroot it has > been tested against. Do we put too much hope in the external? > > If I try to summarize our options with their pros and cons: > > - buildroot fork: > - pros: > - one tree > - full control, can do whatever we want > - cons: > - fork > - need to rebase patches for new releases > - may not be able to keep the pace of buildroot releases > > - buildroot external: > - pros: > - one tree > - not dependent on buildroot version (in theory) so no rebase to do > - cons: > - can't modify buildroot package makefiles, only the source code > - can't quickly solve bugs in buildroot > - may have dependency on the buildroot version > > - buildroot fork + external: > - pros: > - full control, can do whatever we want > - less patches to rebase (only the ones on top of buildroot) > - cons: > - fork > - two trees > - may not be able to keep the pace of buildroot releases > - may have buildroot fork with no patches on top of it sometime > > > If you have any suggestions, about improvements for Buildroot or the way we > manage our Buildroot offer, I would be happy to hear them. > > > Regards, > > Ludovic > _______________________________________________ > buildroot mailing list > buildroot at busybox.net > http://lists.busybox.net/mailman/listinfo/buildroot > > ________________________ > This email was scanned by Bitdefender ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] external, how to achieve our needs? 2020-02-26 13:29 ` Ludovic Desroches @ 2020-02-26 15:14 ` Peter Korsgaard 2020-02-26 21:59 ` Vadim Kochan 0 siblings, 1 reply; 8+ messages in thread From: Peter Korsgaard @ 2020-02-26 15:14 UTC (permalink / raw) To: buildroot >>>>> "Ludovic" == Ludovic Desroches <ludovic.desroches@microchip.com> writes: > Hi Mircea, > On Wed, Feb 26, 2020 at 12:11:36PM +0000, Mircea GLIGA wrote: >> Hi Ludovic, >> >> I also have this kind of problems when implementing our external, >> sometimes is useful to 'override' or append to a recipe (Yocto >> style). >> >> * I ended up "duplicating" recipes in our layer, e.g. bd-tcpdump >> in order to build the package differently > From the top of my head, I tried this and I got an error when duplicating > the recipes in the external. Did you use other tricks? Buildrot uses a global namespace, so you have to rename the package (and the makefile/kconfig symbols) for it to work. It naturally only works for packages without any reverse dependencies as those packages doesn't know about your custom 'foo' package. -- Bye, Peter Korsgaard ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] external, how to achieve our needs? 2020-02-26 15:14 ` Peter Korsgaard @ 2020-02-26 21:59 ` Vadim Kochan 0 siblings, 0 replies; 8+ messages in thread From: Vadim Kochan @ 2020-02-26 21:59 UTC (permalink / raw) To: buildroot Hi All, On Wed, Feb 26, 2020 at 5:14 PM Peter Korsgaard <peter@korsgaard.com> wrote: > > >>>>> "Ludovic" == Ludovic Desroches <ludovic.desroches@microchip.com> writes: > > > Hi Mircea, > > On Wed, Feb 26, 2020 at 12:11:36PM +0000, Mircea GLIGA wrote: > >> Hi Ludovic, > >> > >> I also have this kind of problems when implementing our external, > >> sometimes is useful to 'override' or append to a recipe (Yocto > >> style). > >> > >> * I ended up "duplicating" recipes in our layer, e.g. bd-tcpdump > >> in order to build the package differently > > > From the top of my head, I tried this and I got an error when duplicating > > the recipes in the external. Did you use other tricks? > > Buildrot uses a global namespace, so you have to rename the package (and > the makefile/kconfig symbols) for it to work. > > It naturally only works for packages without any reverse dependencies as > those packages doesn't know about your custom 'foo' package. > For me looks like it can be solved (did not tried yet) by: 1) Including BR2_EXTERNAL_MKS before include all the packages, and filter-out these which has set variable $(PKG)_OVERRIDE=y which means that external tree can contain some special version of package from the main Buildroot, and this custom .mk package can set XXX_OVERRIDE=YES to do not include the original package by main Makefile. 2) The custom Config.in should be not a problem as it should replace original config options. But I am not a Buildroot expert, so I might missed something. Regards, Vadim Kochan ^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] external, how to achieve our needs? 2020-02-25 15:01 [Buildroot] external, how to achieve our needs? Ludovic Desroches 2020-02-26 5:36 ` Baruch Siach 2020-02-26 12:11 ` Mircea GLIGA @ 2020-02-26 13:30 ` Mircea GLIGA 2 siblings, 0 replies; 8+ messages in thread From: Mircea GLIGA @ 2020-02-26 13:30 UTC (permalink / raw) To: buildroot Hi Ludovic, I also have this kind of problems when implementing our external, sometimes is useful to 'override' or append to a recipe (Yocto style). * I ended up "duplicating" recipes in our layer, e.g. bd-tcpdump in order to build the package differently * Where it was possible, I manipulated the original package variables, from a .mk file in the external layer: e.g modify the *_CONF_OPTS or *_CONF_ENV for some package I'm also not that pleased with this 'unofficial' solution. BR Mircea ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2020-02-26 21:59 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-02-25 15:01 [Buildroot] external, how to achieve our needs? Ludovic Desroches 2020-02-26 5:36 ` Baruch Siach 2020-02-26 13:20 ` Ludovic Desroches 2020-02-26 12:11 ` Mircea GLIGA 2020-02-26 13:29 ` Ludovic Desroches 2020-02-26 15:14 ` Peter Korsgaard 2020-02-26 21:59 ` Vadim Kochan 2020-02-26 13:30 ` Mircea GLIGA
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.