follow up: keeping the source with an "externalsrc"-based recipe

follow up: keeping the source with an "externalsrc"-based recipe

[Robert P. J. Day]

  as a short followup to my earlier note about creating a bunch of
externalsrc-based recipes to demonstrate their proper structure, it's
easy to add the recipe files to a layer, but i then realized i needed
to figure out where to put the corresponding source, and how i would
refer to it with the EXTERNALSRC variable, and after some thought, it
occurred that the simplest solution is to simply put the source in a
"src" subdirectory next to the recipe itself, and use:

  inherit externalsrc

  EXTERNALSRC = "${THISDIR}/src"
  EXTERNALSRC_BUILD = "${EXTERNALSRC}"

yes, i know this is hacky, but it demonstrates the process, and there
seems to be nothing magic about having the source right there, and all
of the standard bitbake tasks -- including cleaning -- seem to work
properly.

  aside from looking weird, is there anything about this that wouldn't
work?

rday

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Quentin Schulz]

On Fri, May 28, 2021 at 08:51:51AM -0400, Robert P. J. Day wrote:
> 
>   as a short followup to my earlier note about creating a bunch of
> externalsrc-based recipes to demonstrate their proper structure, it's
> easy to add the recipe files to a layer, but i then realized i needed
> to figure out where to put the corresponding source, and how i would
> refer to it with the EXTERNALSRC variable, and after some thought, it
> occurred that the simplest solution is to simply put the source in a
> "src" subdirectory next to the recipe itself, and use:
> 
>   inherit externalsrc
> 
>   EXTERNALSRC = "${THISDIR}/src"

FWIW, we use multiple ../ in the variable and it works just fine. Might
be a slightly better example than having it next to the recipe, since
this would actually be covered by SRC_URI = "file://whatever". In that
case, you could add the sources in `yocto-docs/externalsrc-src` for
example?

In any case, an example recipe is very much welcomed so thanks for
suggesting one!

Cheers,
Quentin

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Robert P. J. Day]

On Fri, 28 May 2021, Quentin Schulz wrote:

> On Fri, May 28, 2021 at 08:51:51AM -0400, Robert P. J. Day wrote:
> >
> >   as a short followup to my earlier note about creating a bunch of
> > externalsrc-based recipes to demonstrate their proper structure, it's
> > easy to add the recipe files to a layer, but i then realized i needed
> > to figure out where to put the corresponding source, and how i would
> > refer to it with the EXTERNALSRC variable, and after some thought, it
> > occurred that the simplest solution is to simply put the source in a
> > "src" subdirectory next to the recipe itself, and use:
> >
> >   inherit externalsrc
> >
> >   EXTERNALSRC = "${THISDIR}/src"
>
> FWIW, we use multiple ../ in the variable and it works just fine.
> Might be a slightly better example than having it next to the
> recipe, since this would actually be covered by SRC_URI =
> "file://whatever". In that case, you could add the sources in
> `yocto-docs/externalsrc-src` for example?

  i know you could use SRC_URI, but i really want to stress the usage
of "externalsrc" and force the developer to think that the source is
really elsewhere.

rday

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Robert P. J. Day]

On Fri, 28 May 2021, Quentin Schulz wrote:

> On Fri, May 28, 2021 at 08:51:51AM -0400, Robert P. J. Day wrote:
> >
> >   as a short followup to my earlier note about creating a bunch of
> > externalsrc-based recipes to demonstrate their proper structure, it's
> > easy to add the recipe files to a layer, but i then realized i needed
> > to figure out where to put the corresponding source, and how i would
> > refer to it with the EXTERNALSRC variable, and after some thought, it
> > occurred that the simplest solution is to simply put the source in a
> > "src" subdirectory next to the recipe itself, and use:
> >
> >   inherit externalsrc
> >
> >   EXTERNALSRC = "${THISDIR}/src"
>
> FWIW, we use multiple ../ in the variable and it works just fine.
> Might be a slightly better example than having it next to the
> recipe, since this would actually be covered by SRC_URI =
> "file://whatever". In that case, you could add the sources in
> `yocto-docs/externalsrc-src` for example?

  after further cogitation, it seems like using SRC_URI might be the
way to go after all, unless i drastically misunderstand some features
of "externalsrc".

  at the moment, most of this external source has to be compiled in
place; very little of it supports "out of tree" building, which means
that i'm forced to(?):

  EXTERNALSRC = <source location>
  EXTERNALSRC_BUILD = "${EXTERNALSRC}"

however, that obviously(?) means that only one developer can build
that recipe at a time, else they would step on each other's ongoing
builds -- do i understand that correctly?

  using SRC_URI, on the other hand, would copy the source directory
into the WORKDIR, so that solves that problem. am i understanding this
correctly?

rday

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Quentin Schulz]

On Fri, May 28, 2021 at 09:57:26AM -0400, Robert P. J. Day wrote:
> On Fri, 28 May 2021, Quentin Schulz wrote:
> 
> > On Fri, May 28, 2021 at 08:51:51AM -0400, Robert P. J. Day wrote:
> > >
> > >   as a short followup to my earlier note about creating a bunch of
> > > externalsrc-based recipes to demonstrate their proper structure, it's
> > > easy to add the recipe files to a layer, but i then realized i needed
> > > to figure out where to put the corresponding source, and how i would
> > > refer to it with the EXTERNALSRC variable, and after some thought, it
> > > occurred that the simplest solution is to simply put the source in a
> > > "src" subdirectory next to the recipe itself, and use:
> > >
> > >   inherit externalsrc
> > >
> > >   EXTERNALSRC = "${THISDIR}/src"
> >
> > FWIW, we use multiple ../ in the variable and it works just fine.
> > Might be a slightly better example than having it next to the
> > recipe, since this would actually be covered by SRC_URI =
> > "file://whatever". In that case, you could add the sources in
> > `yocto-docs/externalsrc-src` for example?
> 
>   after further cogitation, it seems like using SRC_URI might be the
> way to go after all, unless i drastically misunderstand some features
> of "externalsrc".
> 
>   at the moment, most of this external source has to be compiled in
> place; very little of it supports "out of tree" building, which means
> that i'm forced to(?):

What make you say that? Links, sources? By default it builds in the
usual ${WORKDIR}/${BPN}-${PV}.

> 
>   EXTERNALSRC = <source location>
>   EXTERNALSRC_BUILD = "${EXTERNALSRC}"
> 
> however, that obviously(?) means that only one developer can build
> that recipe at a time, else they would step on each other's ongoing
> builds -- do i understand that correctly?
> 

In the case of EXTERNALSRC_BUILD = "${EXTERNALSRC}" I would say that
this is correct?

Sharing the same externalsrc sources between two distinct Yocto
setups is most likely bound to fail at some point. Anyway, you should
see externalsrc as stable as AUTOREV in SRCREV, at least that's what it
is to me.

externalsrc isn't much more than a stripped down SRC_URI =
"file://src/", the fetch/unpack are different, the way to track if files
is different too.

>   using SRC_URI, on the other hand, would copy the source directory
> into the WORKDIR, so that solves that problem. am i understanding this
> correctly?
> 

SRC_URI by default is looked only in three directories next to the
recipe and therefore is part of the git repo of your Yocto metadata.
externalsrc can be used to have sources anywhere on a filesystem.

I'm not sure I fully understood your confusion therefore I might have
not made anything clearer. Let me know what's confusing in SRC_URI vs
externalsrc or something else, I may be able to help.

Cheers,
Quentin

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Robert P. J. Day]

On Fri, 28 May 2021, Quentin Schulz wrote:

> On Fri, May 28, 2021 at 09:57:26AM -0400, Robert P. J. Day wrote:
> > On Fri, 28 May 2021, Quentin Schulz wrote:
> >
> > > On Fri, May 28, 2021 at 08:51:51AM -0400, Robert P. J. Day wrote:
> > > >
> > > >   as a short followup to my earlier note about creating a bunch of
> > > > externalsrc-based recipes to demonstrate their proper structure, it's
> > > > easy to add the recipe files to a layer, but i then realized i needed
> > > > to figure out where to put the corresponding source, and how i would
> > > > refer to it with the EXTERNALSRC variable, and after some thought, it
> > > > occurred that the simplest solution is to simply put the source in a
> > > > "src" subdirectory next to the recipe itself, and use:
> > > >
> > > >   inherit externalsrc
> > > >
> > > >   EXTERNALSRC = "${THISDIR}/src"
> > >
> > > FWIW, we use multiple ../ in the variable and it works just fine.
> > > Might be a slightly better example than having it next to the
> > > recipe, since this would actually be covered by SRC_URI =
> > > "file://whatever". In that case, you could add the sources in
> > > `yocto-docs/externalsrc-src` for example?
> >
> >   after further cogitation, it seems like using SRC_URI might be the
> > way to go after all, unless i drastically misunderstand some features
> > of "externalsrc".
> >
> >   at the moment, most of this external source has to be compiled in
> > place; very little of it supports "out of tree" building, which means
> > that i'm forced to(?):
>
> What make you say that? Links, sources? By default it builds in the
> usual ${WORKDIR}/${BPN}-${PV}.

  i know, but if i do this with the source example i'm playing with:

EXTERNALSRC = "${THISDIR}/src"
# EXTERNALSRC_BUILD = "${EXTERNALSRC}"

and let EXTERNALSRC_BUILD default to the value you identify, i get:

| make: *** No rule to make target 'install'.  Stop.

i clearly don't understand something fundamental about using
externalsrc -- how do i get that external, remote source to build
under WORKDIR?

rday

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Quentin Schulz]

On Fri, May 28, 2021 at 10:47:30AM -0400, Robert P. J. Day wrote:
> On Fri, 28 May 2021, Quentin Schulz wrote:
> 
> > On Fri, May 28, 2021 at 09:57:26AM -0400, Robert P. J. Day wrote:
> > > On Fri, 28 May 2021, Quentin Schulz wrote:
> > >
> > > > On Fri, May 28, 2021 at 08:51:51AM -0400, Robert P. J. Day wrote:
> > > > >
> > > > >   as a short followup to my earlier note about creating a bunch of
> > > > > externalsrc-based recipes to demonstrate their proper structure, it's
> > > > > easy to add the recipe files to a layer, but i then realized i needed
> > > > > to figure out where to put the corresponding source, and how i would
> > > > > refer to it with the EXTERNALSRC variable, and after some thought, it
> > > > > occurred that the simplest solution is to simply put the source in a
> > > > > "src" subdirectory next to the recipe itself, and use:
> > > > >
> > > > >   inherit externalsrc
> > > > >
> > > > >   EXTERNALSRC = "${THISDIR}/src"
> > > >
> > > > FWIW, we use multiple ../ in the variable and it works just fine.
> > > > Might be a slightly better example than having it next to the
> > > > recipe, since this would actually be covered by SRC_URI =
> > > > "file://whatever". In that case, you could add the sources in
> > > > `yocto-docs/externalsrc-src` for example?
> > >
> > >   after further cogitation, it seems like using SRC_URI might be the
> > > way to go after all, unless i drastically misunderstand some features
> > > of "externalsrc".
> > >
> > >   at the moment, most of this external source has to be compiled in
> > > place; very little of it supports "out of tree" building, which means
> > > that i'm forced to(?):
> >
> > What make you say that? Links, sources? By default it builds in the
> > usual ${WORKDIR}/${BPN}-${PV}.
> 
>   i know, but if i do this with the source example i'm playing with:
> 
> EXTERNALSRC = "${THISDIR}/src"
> # EXTERNALSRC_BUILD = "${EXTERNALSRC}"
> 
> and let EXTERNALSRC_BUILD default to the value you identify, i get:
> 
> | make: *** No rule to make target 'install'.  Stop.
> 
> i clearly don't understand something fundamental about using
> externalsrc -- how do i get that external, remote source to build
> under WORKDIR?
> 

Would you mind sharing your work so far (recipe + source)? So we can try
to reproduce on our side? I have a very complex recipe without
EXTERNALSRC_BUILD set (on thud though, but with some backports from later
versions for the externalsrc class).

I couldn't find tests for externalsrc recipes, so either I'm blind
(can't exclude it) or we're missing such tests, which we should add :)

Cheers,
Quentin

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Robert P. J. Day]

On Fri, 28 May 2021, Quentin Schulz wrote:

> On Fri, May 28, 2021 at 10:47:30AM -0400, Robert P. J. Day wrote:
> > On Fri, 28 May 2021, Quentin Schulz wrote:
> >
> > > On Fri, May 28, 2021 at 09:57:26AM -0400, Robert P. J. Day wrote:
> > > > On Fri, 28 May 2021, Quentin Schulz wrote:
> > > >
> > > > > On Fri, May 28, 2021 at 08:51:51AM -0400, Robert P. J. Day wrote:
> > > > > >
> > > > > >   as a short followup to my earlier note about creating a bunch of
> > > > > > externalsrc-based recipes to demonstrate their proper structure, it's
> > > > > > easy to add the recipe files to a layer, but i then realized i needed
> > > > > > to figure out where to put the corresponding source, and how i would
> > > > > > refer to it with the EXTERNALSRC variable, and after some thought, it
> > > > > > occurred that the simplest solution is to simply put the source in a
> > > > > > "src" subdirectory next to the recipe itself, and use:
> > > > > >
> > > > > >   inherit externalsrc
> > > > > >
> > > > > >   EXTERNALSRC = "${THISDIR}/src"
> > > > >
> > > > > FWIW, we use multiple ../ in the variable and it works just fine.
> > > > > Might be a slightly better example than having it next to the
> > > > > recipe, since this would actually be covered by SRC_URI =
> > > > > "file://whatever". In that case, you could add the sources in
> > > > > `yocto-docs/externalsrc-src` for example?
> > > >
> > > >   after further cogitation, it seems like using SRC_URI might be the
> > > > way to go after all, unless i drastically misunderstand some features
> > > > of "externalsrc".
> > > >
> > > >   at the moment, most of this external source has to be compiled in
> > > > place; very little of it supports "out of tree" building, which means
> > > > that i'm forced to(?):
> > >
> > > What make you say that? Links, sources? By default it builds in the
> > > usual ${WORKDIR}/${BPN}-${PV}.
> >
> >   i know, but if i do this with the source example i'm playing with:
> >
> > EXTERNALSRC = "${THISDIR}/src"
> > # EXTERNALSRC_BUILD = "${EXTERNALSRC}"
> >
> > and let EXTERNALSRC_BUILD default to the value you identify, i get:
> >
> > | make: *** No rule to make target 'install'.  Stop.
> >
> > i clearly don't understand something fundamental about using
> > externalsrc -- how do i get that external, remote source to build
> > under WORKDIR?
> >
>
> Would you mind sharing your work so far (recipe + source)? So we can
> try to reproduce on our side? I have a very complex recipe without
> EXTERNALSRC_BUILD set (on thud though, but with some backports from
> later versions for the externalsrc class).
>
> I couldn't find tests for externalsrc recipes, so either I'm blind
> (can't exclude it) or we're missing such tests, which we should add
> :)

  i'm just using dmidecode as grabbed from here:

http://cgit.openembedded.org/openembedded-core/tree/meta/recipes-devtools/dmidecode/dmidecode_3.3.bb?h=master

but let me ask a fairly basic question that might clear this up.

  AIUI (and lord knows i could be wrong), depending on the structure
of the Makefile for some blob of source, it may not be possible to
build that source "out of tree". unless you've built that sort of
functionality into the Makefile, then you have no choice but to
do the build in the source tree. as in, ${B} = ${S}.

  based on what i see, the dmidecode source must be built in-tree --
there is no functionality in the Makefile to support out-of-tree
building. so if i use externalsrc with dmidecode, then the build
*must* happen in the source directory i identify with EXTERNALSRC.

  am i misunderstanding something really critical here?

rday

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Quentin Schulz]

On Fri, May 28, 2021 at 11:15:40AM -0400, Robert P. J. Day wrote:
> On Fri, 28 May 2021, Quentin Schulz wrote:
> 
> > On Fri, May 28, 2021 at 10:47:30AM -0400, Robert P. J. Day wrote:
> > > On Fri, 28 May 2021, Quentin Schulz wrote:
> > >
> > > > On Fri, May 28, 2021 at 09:57:26AM -0400, Robert P. J. Day wrote:
> > > > > On Fri, 28 May 2021, Quentin Schulz wrote:
> > > > >
> > > > > > On Fri, May 28, 2021 at 08:51:51AM -0400, Robert P. J. Day wrote:
> > > > > > >
> > > > > > >   as a short followup to my earlier note about creating a bunch of
> > > > > > > externalsrc-based recipes to demonstrate their proper structure, it's
> > > > > > > easy to add the recipe files to a layer, but i then realized i needed
> > > > > > > to figure out where to put the corresponding source, and how i would
> > > > > > > refer to it with the EXTERNALSRC variable, and after some thought, it
> > > > > > > occurred that the simplest solution is to simply put the source in a
> > > > > > > "src" subdirectory next to the recipe itself, and use:
> > > > > > >
> > > > > > >   inherit externalsrc
> > > > > > >
> > > > > > >   EXTERNALSRC = "${THISDIR}/src"
> > > > > >
> > > > > > FWIW, we use multiple ../ in the variable and it works just fine.
> > > > > > Might be a slightly better example than having it next to the
> > > > > > recipe, since this would actually be covered by SRC_URI =
> > > > > > "file://whatever". In that case, you could add the sources in
> > > > > > `yocto-docs/externalsrc-src` for example?
> > > > >
> > > > >   after further cogitation, it seems like using SRC_URI might be the
> > > > > way to go after all, unless i drastically misunderstand some features
> > > > > of "externalsrc".
> > > > >
> > > > >   at the moment, most of this external source has to be compiled in
> > > > > place; very little of it supports "out of tree" building, which means
> > > > > that i'm forced to(?):
> > > >
> > > > What make you say that? Links, sources? By default it builds in the
> > > > usual ${WORKDIR}/${BPN}-${PV}.
> > >
> > >   i know, but if i do this with the source example i'm playing with:
> > >
> > > EXTERNALSRC = "${THISDIR}/src"
> > > # EXTERNALSRC_BUILD = "${EXTERNALSRC}"
> > >
> > > and let EXTERNALSRC_BUILD default to the value you identify, i get:
> > >
> > > | make: *** No rule to make target 'install'.  Stop.
> > >
> > > i clearly don't understand something fundamental about using
> > > externalsrc -- how do i get that external, remote source to build
> > > under WORKDIR?
> > >
> >
> > Would you mind sharing your work so far (recipe + source)? So we can
> > try to reproduce on our side? I have a very complex recipe without
> > EXTERNALSRC_BUILD set (on thud though, but with some backports from
> > later versions for the externalsrc class).
> >
> > I couldn't find tests for externalsrc recipes, so either I'm blind
> > (can't exclude it) or we're missing such tests, which we should add
> > :)
> 
>   i'm just using dmidecode as grabbed from here:
> 
> http://cgit.openembedded.org/openembedded-core/tree/meta/recipes-devtools/dmidecode/dmidecode_3.3.bb?h=master
> 
> but let me ask a fairly basic question that might clear this up.
> 
>   AIUI (and lord knows i could be wrong), depending on the structure
> of the Makefile for some blob of source, it may not be possible to
> build that source "out of tree". unless you've built that sort of
> functionality into the Makefile, then you have no choice but to
> do the build in the source tree. as in, ${B} = ${S}.
> 
>   based on what i see, the dmidecode source must be built in-tree --
> there is no functionality in the Makefile to support out-of-tree
> building. so if i use externalsrc with dmidecode, then the build
> *must* happen in the source directory i identify with EXTERNALSRC.
> 

${B} = ${S} by default, so that might explain why it works without
externalsrc being inherited.

You can see in the makefile that this is assumed. Only the install tasks
are installing somewhere else (DESTDIR), c.f.
https://git.savannah.nongnu.org/cgit/dmidecode.git/tree/Makefile

So yes, except if you patch the Makefile to support out-of-tree build,
you would need both EXTERNALSRC and EXTERNALSRC_BUILD set to the same
value.

Maybe a Cmake example would suit better since B is set to "${WORKDIR}/build"
by default?

Otherwise, write a proper Makefile that allows out-of-tree builds for
the example :)

Thanks for asking, I learned something new :)

Cheers,
Quentin

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Robert P. J. Day]

On Fri, 28 May 2021, Quentin Schulz wrote:

> On Fri, May 28, 2021 at 11:15:40AM -0400, Robert P. J. Day wrote:
> > On Fri, 28 May 2021, Quentin Schulz wrote:
> >
> > > On Fri, May 28, 2021 at 10:47:30AM -0400, Robert P. J. Day wrote:
> > > > On Fri, 28 May 2021, Quentin Schulz wrote:
> > > >
> > > > > On Fri, May 28, 2021 at 09:57:26AM -0400, Robert P. J. Day wrote:
> > > > > > On Fri, 28 May 2021, Quentin Schulz wrote:
> > > > > >
> > > > > > > On Fri, May 28, 2021 at 08:51:51AM -0400, Robert P. J. Day wrote:
> > > > > > > >
> > > > > > > >   as a short followup to my earlier note about creating a bunch of
> > > > > > > > externalsrc-based recipes to demonstrate their proper structure, it's
> > > > > > > > easy to add the recipe files to a layer, but i then realized i needed
> > > > > > > > to figure out where to put the corresponding source, and how i would
> > > > > > > > refer to it with the EXTERNALSRC variable, and after some thought, it
> > > > > > > > occurred that the simplest solution is to simply put the source in a
> > > > > > > > "src" subdirectory next to the recipe itself, and use:
> > > > > > > >
> > > > > > > >   inherit externalsrc
> > > > > > > >
> > > > > > > >   EXTERNALSRC = "${THISDIR}/src"
> > > > > > >
> > > > > > > FWIW, we use multiple ../ in the variable and it works just fine.
> > > > > > > Might be a slightly better example than having it next to the
> > > > > > > recipe, since this would actually be covered by SRC_URI =
> > > > > > > "file://whatever". In that case, you could add the sources in
> > > > > > > `yocto-docs/externalsrc-src` for example?
> > > > > >
> > > > > >   after further cogitation, it seems like using SRC_URI might be the
> > > > > > way to go after all, unless i drastically misunderstand some features
> > > > > > of "externalsrc".
> > > > > >
> > > > > >   at the moment, most of this external source has to be compiled in
> > > > > > place; very little of it supports "out of tree" building, which means
> > > > > > that i'm forced to(?):
> > > > >
> > > > > What make you say that? Links, sources? By default it builds in the
> > > > > usual ${WORKDIR}/${BPN}-${PV}.
> > > >
> > > >   i know, but if i do this with the source example i'm playing with:
> > > >
> > > > EXTERNALSRC = "${THISDIR}/src"
> > > > # EXTERNALSRC_BUILD = "${EXTERNALSRC}"
> > > >
> > > > and let EXTERNALSRC_BUILD default to the value you identify, i get:
> > > >
> > > > | make: *** No rule to make target 'install'.  Stop.
> > > >
> > > > i clearly don't understand something fundamental about using
> > > > externalsrc -- how do i get that external, remote source to build
> > > > under WORKDIR?
> > > >
> > >
> > > Would you mind sharing your work so far (recipe + source)? So we can
> > > try to reproduce on our side? I have a very complex recipe without
> > > EXTERNALSRC_BUILD set (on thud though, but with some backports from
> > > later versions for the externalsrc class).
> > >
> > > I couldn't find tests for externalsrc recipes, so either I'm blind
> > > (can't exclude it) or we're missing such tests, which we should add
> > > :)
> >
> >   i'm just using dmidecode as grabbed from here:
> >
> > http://cgit.openembedded.org/openembedded-core/tree/meta/recipes-devtools/dmidecode/dmidecode_3.3.bb?h=master
> >
> > but let me ask a fairly basic question that might clear this up.
> >
> >   AIUI (and lord knows i could be wrong), depending on the
> > structure of the Makefile for some blob of source, it may not be
> > possible to build that source "out of tree". unless you've built
> > that sort of functionality into the Makefile, then you have no
> > choice but to do the build in the source tree. as in, ${B} = ${S}.
> >
> >   based on what i see, the dmidecode source must be built in-tree
> > -- there is no functionality in the Makefile to support
> > out-of-tree building. so if i use externalsrc with dmidecode, then
> > the build *must* happen in the source directory i identify with
> > EXTERNALSRC.
>
> ${B} = ${S} by default, so that might explain why it works without
> externalsrc being inherited.
>
> You can see in the makefile that this is assumed. Only the install
> tasks are installing somewhere else (DESTDIR), c.f.
> https://git.savannah.nongnu.org/cgit/dmidecode.git/tree/Makefile
>
> So yes, except if you patch the Makefile to support out-of-tree
> build, you would need both EXTERNALSRC and EXTERNALSRC_BUILD set to
> the same value.

  exactly, which is why if one is saddled with source directories that
can't build out-of-tree, then using externalsrc for any of those
directories if multiple developers are trying to build at the same
time would crash and burn in short order as they stepped all over each
other; hence, the need to back off and use SRC_URI in cases like that.

> Maybe a Cmake example would suit better since B is set to
> "${WORKDIR}/build" by default?
>
> Otherwise, write a proper Makefile that allows out-of-tree builds
> for the example :)

  nice idea if you have the kind of authority that allows you to
dictate how developers develop. :-) ok, i think we've resolved this.

rday

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Robert P. J. Day]

On Fri, 28 May 2021, Quentin Schulz wrote:

... big snip ...

> ${B} = ${S} by default, so that might explain why it works without
> externalsrc being inherited.
>
> You can see in the makefile that this is assumed. Only the install tasks
> are installing somewhere else (DESTDIR), c.f.
> https://git.savannah.nongnu.org/cgit/dmidecode.git/tree/Makefile
>
> So yes, except if you patch the Makefile to support out-of-tree build,
> you would need both EXTERNALSRC and EXTERNALSRC_BUILD set to the same
> value.
>
> Maybe a Cmake example would suit better since B is set to "${WORKDIR}/build"
> by default?
>
> Otherwise, write a proper Makefile that allows out-of-tree builds for
> the example :)

  in the end, i suspect i was overthinking it, thinking somehow there
was some magic to "externalsrc", only to finally realize it was
totally inappropriate for source that had to be built in-tree. so i
just switched over to using SRC_URI and, for various recipes like
dmidecode, i created a recipe that started with some variation of:

  LICENSE = "CLOSED"
  LIC_FILES_CHKSUM = ""
  FILESEXTRAPATHS_prepend := "${THISDIR}:"
  SRC_URI := "file://src"
  S = "${WORKDIR}/src"

then just unpacked and copied the source to a directory named "src"
right next to the corresponding recipe. yes, it's goofy, but it's for
demonstration purposes only, and seems to do what i want.

  lots of noise for a very simple solution in the end. isn't that just
the way it goes?

rday

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Quentin Schulz]

Hi Robert,

On Fri, May 28, 2021 at 05:02:13PM -0400, Robert P. J. Day wrote:
> On Fri, 28 May 2021, Quentin Schulz wrote:
> 
> ... big snip ...
> 
> > ${B} = ${S} by default, so that might explain why it works without
> > externalsrc being inherited.
> >
> > You can see in the makefile that this is assumed. Only the install tasks
> > are installing somewhere else (DESTDIR), c.f.
> > https://git.savannah.nongnu.org/cgit/dmidecode.git/tree/Makefile
> >
> > So yes, except if you patch the Makefile to support out-of-tree build,
> > you would need both EXTERNALSRC and EXTERNALSRC_BUILD set to the same
> > value.
> >
> > Maybe a Cmake example would suit better since B is set to "${WORKDIR}/build"
> > by default?
> >
> > Otherwise, write a proper Makefile that allows out-of-tree builds for
> > the example :)
> 
>   in the end, i suspect i was overthinking it, thinking somehow there
> was some magic to "externalsrc", only to finally realize it was
> totally inappropriate for source that had to be built in-tree. so i
> just switched over to using SRC_URI and, for various recipes like
> dmidecode, i created a recipe that started with some variation of:
> 
>   LICENSE = "CLOSED"
>   LIC_FILES_CHKSUM = ""
>   FILESEXTRAPATHS_prepend := "${THISDIR}:"
>   SRC_URI := "file://src"

No need for := here, a simple = should suffice?

>   S = "${WORKDIR}/src"
> 

Why not put the sources in:
meta-mylayer/recipes-test/test/my-test-0.1/src/
meta-mylayer/recipes-test/test/my-test_0.1.bb

and have SRC_URI = "file://src/" without needing to add
FILESEXTRAPATHS_prepend := "${THISDIR}:" so that it follows what almost
every recipe does?

S will be as you wrote above.

I thought you were trying to find an example with externalsrc for our
docs, but it seems you have a different thing in mind, can you tell us
so we are able to help you with your actual issue/question :) ?

Cheers,
Quentin

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Robert P. J. Day]

On Mon, 31 May 2021, Quentin Schulz wrote:

> Hi Robert,
>
> On Fri, May 28, 2021 at 05:02:13PM -0400, Robert P. J. Day wrote:
> > On Fri, 28 May 2021, Quentin Schulz wrote:
> >
> > ... big snip ...
> >
> > > ${B} = ${S} by default, so that might explain why it works without
> > > externalsrc being inherited.
> > >
> > > You can see in the makefile that this is assumed. Only the install tasks
> > > are installing somewhere else (DESTDIR), c.f.
> > > https://git.savannah.nongnu.org/cgit/dmidecode.git/tree/Makefile
> > >
> > > So yes, except if you patch the Makefile to support out-of-tree build,
> > > you would need both EXTERNALSRC and EXTERNALSRC_BUILD set to the same
> > > value.
> > >
> > > Maybe a Cmake example would suit better since B is set to "${WORKDIR}/build"
> > > by default?
> > >
> > > Otherwise, write a proper Makefile that allows out-of-tree builds for
> > > the example :)
> >
> >   in the end, i suspect i was overthinking it, thinking somehow there
> > was some magic to "externalsrc", only to finally realize it was
> > totally inappropriate for source that had to be built in-tree. so i
> > just switched over to using SRC_URI and, for various recipes like
> > dmidecode, i created a recipe that started with some variation of:
> >
> >   LICENSE = "CLOSED"
> >   LIC_FILES_CHKSUM = ""
> >   FILESEXTRAPATHS_prepend := "${THISDIR}:"
> >   SRC_URI := "file://src"
>
> No need for := here, a simple = should suffice?

  yup, i already simplified that, even though it doesn't hurt
anything.

> >   S = "${WORKDIR}/src"
> >
>
> Why not put the sources in:
> meta-mylayer/recipes-test/test/my-test-0.1/src/
> meta-mylayer/recipes-test/test/my-test_0.1.bb
>
> and have SRC_URI = "file://src/" without needing to add
> FILESEXTRAPATHS_prepend := "${THISDIR}:" so that it follows what
> almost every recipe does?
>
> S will be as you wrote above.

  i know it looks goofy, but these recipes are just meant to be
templates to show other developers what a normal Makefile-based recipe
will look like, and if they have their source elsewhere, they'll have
to set FILESEXTRAPATHS anyway. i was grappling with how to do this,
but this is not meant to be production-level, it's just for people to
see how to structure their code so it follows the OE model. i might
re-do it.

> I thought you were trying to find an example with externalsrc for
> our docs, but it seems you have a different thing in mind, can you
> tell us so we are able to help you with your actual issue/question
> :) ?

  externalsrc turned out to have the fatal flaw that it doesn't scale
well for source that can't be built out-of-tree.

rday

Re: [OE-core] follow up: keeping the source with an "externalsrc"-based recipe

[Robert P. J. Day]

On Mon, 31 May 2021, Quentin Schulz wrote:

> >   LICENSE = "CLOSED"
> >   LIC_FILES_CHKSUM = ""
> >   FILESEXTRAPATHS_prepend := "${THISDIR}:"
> >   SRC_URI := "file://src"
>
> No need for := here, a simple = should suffice?
>
> >   S = "${WORKDIR}/src"
> >
>
> Why not put the sources in:
> meta-mylayer/recipes-test/test/my-test-0.1/src/
> meta-mylayer/recipes-test/test/my-test_0.1.bb
>
> and have SRC_URI = "file://src/" without needing to add
> FILESEXTRAPATHS_prepend := "${THISDIR}:" so that it follows what almost
> every recipe does?
>
> S will be as you wrote above.

  ah, i misread what you were suggesting here, yes, your idea is
cleaner.

rday