All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] Revert "protobuf: stage protoc binary to sysroot"
@ 2023-09-30 17:30 Clément Péron
  2023-10-01  0:48 ` [oe] " Vyacheslav Yurkov
  0 siblings, 1 reply; 35+ messages in thread
From: Clément Péron @ 2023-09-30 17:30 UTC (permalink / raw)
  To: openembedded-devel; +Cc: Clément Péron, Samuli Piippo, Ross Burton

This reverts commit a0557fe5433620717eeb00d3b16801711337b1a4.

As said by Ross[Ø]:
"Putting the _target_ protoc into the sysroot for executation at _build_
time isn't useful because even if it has the right architecture, the
tune might be incompatible.  Recipes which want protoc should just depend
on protobuf-native."

This has been reverted recently by Samuli[1]:
"If protoc is enabled for the build, recipes using protobuf will
fail when protoc is not available in the recipe sysroot"

Be the revert is incorret as This is an issue coming from qtgrpc
other recipes that use protobuf or gRPC compiler, proplery looks for
the binary in the correct sysroot folder.

Qtgrpc recipe should fix this issue at the recipe level, for example this
is what I've done for "etcd-cpp-apiv3" recipe[2] that doesn't need this
patch to properly compile.

So keeping this hack doesn't seems to be a correct fix.

Note that qtgrpc recipe isn't available on meta-oe nor any other public
layers.

0: https://patchwork.yoctoproject.org/project/oe/patch/20230904161230.377450-1-ross.burton@arm.com/
1: https://patchwork.yoctoproject.org/project/oe/patch/20230927051101.3088498-1-samuli.piippo@qt.io/
2: https://github.com/etcd-cpp-apiv3/etcd-cpp-apiv3/commit/47f0d9e0326f3cc31c801a0ecf7312d1049ece3e

CC: Samuli Piippo <samuli.piippo@gmail.com>
CC: Ross Burton <ross.burton@arm.com>
Signed-off-by: Clément Péron <peron.clem@gmail.com>
---
 meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb b/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
index 06d73d648..1edc21cdf 100644
--- a/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
+++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
@@ -101,9 +101,6 @@ PACKAGE_BEFORE_PN = "${PN}-compiler ${PN}-lite"
 FILES:${PN}-compiler = "${bindir} ${libdir}/libprotoc${SOLIBS}"
 FILES:${PN}-lite = "${libdir}/libprotobuf-lite${SOLIBS}"
 
-# CMake requires binaries to exist in sysroot, even if they have wrong architecture.
-SYSROOT_DIRS += "${bindir}"
-
 RDEPENDS:${PN}-compiler = "${PN}"
 RDEPENDS:${PN}-dev += "${PN}-compiler"
 RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', '', d)}"
-- 
2.39.3 (Apple Git-145)



^ permalink raw reply related	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-09-30 17:30 [PATCH] Revert "protobuf: stage protoc binary to sysroot" Clément Péron
@ 2023-10-01  0:48 ` Vyacheslav Yurkov
  2023-10-01  3:03   ` Khem Raj
  0 siblings, 1 reply; 35+ messages in thread
From: Vyacheslav Yurkov @ 2023-10-01  0:48 UTC (permalink / raw)
  To: Clément Péron, openembedded-devel; +Cc: Samuli Piippo, Ross Burton

I agree, protoc should _not_ be in sysroot, because target binary can't 
run on host system.

Vyacheslav

On 30.09.2023 19:30, Clément Péron wrote:
> This reverts commit a0557fe5433620717eeb00d3b16801711337b1a4.
>
> As said by Ross[Ø]:
> "Putting the _target_ protoc into the sysroot for executation at _build_
> time isn't useful because even if it has the right architecture, the
> tune might be incompatible.  Recipes which want protoc should just depend
> on protobuf-native."
>
> This has been reverted recently by Samuli[1]:
> "If protoc is enabled for the build, recipes using protobuf will
> fail when protoc is not available in the recipe sysroot"
>
> Be the revert is incorret as This is an issue coming from qtgrpc
> other recipes that use protobuf or gRPC compiler, proplery looks for
> the binary in the correct sysroot folder.
>
> Qtgrpc recipe should fix this issue at the recipe level, for example this
> is what I've done for "etcd-cpp-apiv3" recipe[2] that doesn't need this
> patch to properly compile.
>
> So keeping this hack doesn't seems to be a correct fix.
>
> Note that qtgrpc recipe isn't available on meta-oe nor any other public
> layers.
>
> 0: https://patchwork.yoctoproject.org/project/oe/patch/20230904161230.377450-1-ross.burton@arm.com/
> 1: https://patchwork.yoctoproject.org/project/oe/patch/20230927051101.3088498-1-samuli.piippo@qt.io/
> 2: https://github.com/etcd-cpp-apiv3/etcd-cpp-apiv3/commit/47f0d9e0326f3cc31c801a0ecf7312d1049ece3e
>
> CC: Samuli Piippo <samuli.piippo@gmail.com>
> CC: Ross Burton <ross.burton@arm.com>
> Signed-off-by: Clément Péron <peron.clem@gmail.com>
> ---
>   meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb | 3 ---
>   1 file changed, 3 deletions(-)
>
> diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb b/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
> index 06d73d648..1edc21cdf 100644
> --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
> @@ -101,9 +101,6 @@ PACKAGE_BEFORE_PN = "${PN}-compiler ${PN}-lite"
>   FILES:${PN}-compiler = "${bindir} ${libdir}/libprotoc${SOLIBS}"
>   FILES:${PN}-lite = "${libdir}/libprotobuf-lite${SOLIBS}"
>   
> -# CMake requires binaries to exist in sysroot, even if they have wrong architecture.
> -SYSROOT_DIRS += "${bindir}"
> -
>   RDEPENDS:${PN}-compiler = "${PN}"
>   RDEPENDS:${PN}-dev += "${PN}-compiler"
>   RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', '', d)}"
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#105284): https://lists.openembedded.org/g/openembedded-devel/message/105284
> Mute This Topic: https://lists.openembedded.org/mt/101679410/4455192
> Group Owner: openembedded-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [uvv.mail@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>



^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-01  0:48 ` [oe] " Vyacheslav Yurkov
@ 2023-10-01  3:03   ` Khem Raj
  2023-10-01 10:11     ` Martin Jansa
  0 siblings, 1 reply; 35+ messages in thread
From: Khem Raj @ 2023-10-01  3:03 UTC (permalink / raw)
  To: Vyacheslav Yurkov
  Cc: Clément Péron, Ross Burton, Samuli Piippo, openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 3313 bytes --]

On Sat, Sep 30, 2023 at 5:48 PM Vyacheslav Yurkov <uvv.mail@gmail.com>
wrote:

> I agree, protoc should _not_ be in sysroot, because target binary can't
> run on host system.
>

Yeah this seems correct approach to me

>
> Vyacheslav
>
> On 30.09.2023 19:30, Clément Péron wrote:
> > This reverts commit a0557fe5433620717eeb00d3b16801711337b1a4.
> >
> > As said by Ross[Ø]:
> > "Putting the _target_ protoc into the sysroot for executation at _build_
> > time isn't useful because even if it has the right architecture, the
> > tune might be incompatible.  Recipes which want protoc should just depend
> > on protobuf-native."
> >
> > This has been reverted recently by Samuli[1]:
> > "If protoc is enabled for the build, recipes using protobuf will
> > fail when protoc is not available in the recipe sysroot"
> >
> > Be the revert is incorret as This is an issue coming from qtgrpc
> > other recipes that use protobuf or gRPC compiler, proplery looks for
> > the binary in the correct sysroot folder.
> >
> > Qtgrpc recipe should fix this issue at the recipe level, for example this
> > is what I've done for "etcd-cpp-apiv3" recipe[2] that doesn't need this
> > patch to properly compile.
> >
> > So keeping this hack doesn't seems to be a correct fix.
> >
> > Note that qtgrpc recipe isn't available on meta-oe nor any other public
> > layers.
> >
> > 0:
> https://patchwork.yoctoproject.org/project/oe/patch/20230904161230.377450-1-ross.burton@arm.com/
> > 1:
> https://patchwork.yoctoproject.org/project/oe/patch/20230927051101.3088498-1-samuli.piippo@qt.io/
> > 2:
> https://github.com/etcd-cpp-apiv3/etcd-cpp-apiv3/commit/47f0d9e0326f3cc31c801a0ecf7312d1049ece3e
> >
> > CC: Samuli Piippo <samuli.piippo@gmail.com>
> > CC: Ross Burton <ross.burton@arm.com>
> > Signed-off-by: Clément Péron <peron.clem@gmail.com>
> > ---
> >   meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb | 3 ---
> >   1 file changed, 3 deletions(-)
> >
> > diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
> b/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
> > index 06d73d648..1edc21cdf 100644
> > --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
> > +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
> > @@ -101,9 +101,6 @@ PACKAGE_BEFORE_PN = "${PN}-compiler ${PN}-lite"
> >   FILES:${PN}-compiler = "${bindir} ${libdir}/libprotoc${SOLIBS}"
> >   FILES:${PN}-lite = "${libdir}/libprotobuf-lite${SOLIBS}"
> >
> > -# CMake requires binaries to exist in sysroot, even if they have wrong
> architecture.
> > -SYSROOT_DIRS += "${bindir}"
> > -
> >   RDEPENDS:${PN}-compiler = "${PN}"
> >   RDEPENDS:${PN}-dev += "${PN}-compiler"
> >   RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG',
> 'python', 'python3-protobuf', '', d)}"
> >
> >
> >
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#105290):
> https://lists.openembedded.org/g/openembedded-devel/message/105290
> Mute This Topic: https://lists.openembedded.org/mt/101679410/1997914
> Group Owner: openembedded-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [
> raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 5759 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-01  3:03   ` Khem Raj
@ 2023-10-01 10:11     ` Martin Jansa
  2023-10-01 10:31       ` Clément Péron
  0 siblings, 1 reply; 35+ messages in thread
From: Martin Jansa @ 2023-10-01 10:11 UTC (permalink / raw)
  To: Khem Raj
  Cc: Vyacheslav Yurkov, Clément Péron, Ross Burton,
	Samuli Piippo, openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 3848 bytes --]

well, target bindir isn't in default PATH so it wont get executed by default

I have used this work around in some meta-ros recipes where the .cmake file
was checking the binary existence even when this .cmake was used only to
get right list of libraries to link with (and the binary was not called at
all - which imho can happen with protobuf as well)

On Sun, 1 Oct 2023, 05:03 Khem Raj, <raj.khem@gmail.com> wrote:

>
>
> On Sat, Sep 30, 2023 at 5:48 PM Vyacheslav Yurkov <uvv.mail@gmail.com>
> wrote:
>
>> I agree, protoc should _not_ be in sysroot, because target binary can't
>> run on host system.
>>
>
> Yeah this seems correct approach to me
>
>>
>> Vyacheslav
>>
>> On 30.09.2023 19:30, Clément Péron wrote:
>> > This reverts commit a0557fe5433620717eeb00d3b16801711337b1a4.
>> >
>> > As said by Ross[Ø]:
>> > "Putting the _target_ protoc into the sysroot for executation at _build_
>> > time isn't useful because even if it has the right architecture, the
>> > tune might be incompatible.  Recipes which want protoc should just
>> depend
>> > on protobuf-native."
>> >
>> > This has been reverted recently by Samuli[1]:
>> > "If protoc is enabled for the build, recipes using protobuf will
>> > fail when protoc is not available in the recipe sysroot"
>> >
>> > Be the revert is incorret as This is an issue coming from qtgrpc
>> > other recipes that use protobuf or gRPC compiler, proplery looks for
>> > the binary in the correct sysroot folder.
>> >
>> > Qtgrpc recipe should fix this issue at the recipe level, for example
>> this
>> > is what I've done for "etcd-cpp-apiv3" recipe[2] that doesn't need this
>> > patch to properly compile.
>> >
>> > So keeping this hack doesn't seems to be a correct fix.
>> >
>> > Note that qtgrpc recipe isn't available on meta-oe nor any other public
>> > layers.
>> >
>> > 0:
>> https://patchwork.yoctoproject.org/project/oe/patch/20230904161230.377450-1-ross.burton@arm.com/
>> > 1:
>> https://patchwork.yoctoproject.org/project/oe/patch/20230927051101.3088498-1-samuli.piippo@qt.io/
>> > 2:
>> https://github.com/etcd-cpp-apiv3/etcd-cpp-apiv3/commit/47f0d9e0326f3cc31c801a0ecf7312d1049ece3e
>> >
>> > CC: Samuli Piippo <samuli.piippo@gmail.com>
>> > CC: Ross Burton <ross.burton@arm.com>
>> > Signed-off-by: Clément Péron <peron.clem@gmail.com>
>> > ---
>> >   meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb | 3 ---
>> >   1 file changed, 3 deletions(-)
>> >
>> > diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
>> b/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
>> > index 06d73d648..1edc21cdf 100644
>> > --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
>> > +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
>> > @@ -101,9 +101,6 @@ PACKAGE_BEFORE_PN = "${PN}-compiler ${PN}-lite"
>> >   FILES:${PN}-compiler = "${bindir} ${libdir}/libprotoc${SOLIBS}"
>> >   FILES:${PN}-lite = "${libdir}/libprotobuf-lite${SOLIBS}"
>> >
>> > -# CMake requires binaries to exist in sysroot, even if they have wrong
>> architecture.
>> > -SYSROOT_DIRS += "${bindir}"
>> > -
>> >   RDEPENDS:${PN}-compiler = "${PN}"
>> >   RDEPENDS:${PN}-dev += "${PN}-compiler"
>> >   RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG',
>> 'python', 'python3-protobuf', '', d)}"
>> >
>> >
>> >
>>
>>
>>
>>
>>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#105291):
> https://lists.openembedded.org/g/openembedded-devel/message/105291
> Mute This Topic: https://lists.openembedded.org/mt/101679410/3617156
> Group Owner: openembedded-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [
> martin.jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 6753 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-01 10:11     ` Martin Jansa
@ 2023-10-01 10:31       ` Clément Péron
  2023-10-01 18:22         ` Martin Jansa
  0 siblings, 1 reply; 35+ messages in thread
From: Clément Péron @ 2023-10-01 10:31 UTC (permalink / raw)
  To: Martin Jansa
  Cc: Khem Raj, Vyacheslav Yurkov, Ross Burton, Samuli Piippo,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 4487 bytes --]

Hi Martin,

On Sun, 1 Oct 2023 at 12:11, Martin Jansa <martin.jansa@gmail.com> wrote:
>
> well, target bindir isn't in default PATH so it wont get executed by
default

Is it a reason to allow mixing executable arch/tune?

How do you handle it when you want to install protoc/grpc-cpp-plugin for
the target too?

>
> I have used this work around in some meta-ros recipes where the .cmake
file was checking the binary existence even when this .cmake was used only
to get right list of libraries to link with (and the binary was not called
at all - which imho can happen with protobuf as well)

As you say it's a workaround, and it's an issue with the CMake recipe.

If we can properly fix it at the recipe level, then why should we keep this
workaround/hack?

Also if you really need it, you can have a .bbappend for protoc/grpc in the
downstream layer. But I would be in favor to avoid this by default no?

Clement

>
> On Sun, 1 Oct 2023, 05:03 Khem Raj, <raj.khem@gmail.com> wrote:
>>
>>
>>
>> On Sat, Sep 30, 2023 at 5:48 PM Vyacheslav Yurkov <uvv.mail@gmail.com>
wrote:
>>>
>>> I agree, protoc should _not_ be in sysroot, because target binary can't
>>> run on host system.
>>
>>
>> Yeah this seems correct approach to me
>>>
>>>
>>> Vyacheslav
>>>
>>> On 30.09.2023 19:30, Clément Péron wrote:
>>> > This reverts commit a0557fe5433620717eeb00d3b16801711337b1a4.
>>> >
>>> > As said by Ross[Ø]:
>>> > "Putting the _target_ protoc into the sysroot for executation at
_build_
>>> > time isn't useful because even if it has the right architecture, the
>>> > tune might be incompatible.  Recipes which want protoc should just
depend
>>> > on protobuf-native."
>>> >
>>> > This has been reverted recently by Samuli[1]:
>>> > "If protoc is enabled for the build, recipes using protobuf will
>>> > fail when protoc is not available in the recipe sysroot"
>>> >
>>> > Be the revert is incorret as This is an issue coming from qtgrpc
>>> > other recipes that use protobuf or gRPC compiler, proplery looks for
>>> > the binary in the correct sysroot folder.
>>> >
>>> > Qtgrpc recipe should fix this issue at the recipe level, for example
this
>>> > is what I've done for "etcd-cpp-apiv3" recipe[2] that doesn't need
this
>>> > patch to properly compile.
>>> >
>>> > So keeping this hack doesn't seems to be a correct fix.
>>> >
>>> > Note that qtgrpc recipe isn't available on meta-oe nor any other
public
>>> > layers.
>>> >
>>> > 0:
https://patchwork.yoctoproject.org/project/oe/patch/20230904161230.377450-1-ross.burton@arm.com/
>>> > 1:
https://patchwork.yoctoproject.org/project/oe/patch/20230927051101.3088498-1-samuli.piippo@qt.io/
>>> > 2:
https://github.com/etcd-cpp-apiv3/etcd-cpp-apiv3/commit/47f0d9e0326f3cc31c801a0ecf7312d1049ece3e
>>> >
>>> > CC: Samuli Piippo <samuli.piippo@gmail.com>
>>> > CC: Ross Burton <ross.burton@arm.com>
>>> > Signed-off-by: Clément Péron <peron.clem@gmail.com>
>>> > ---
>>> >   meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb | 3 ---
>>> >   1 file changed, 3 deletions(-)
>>> >
>>> > diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
b/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
>>> > index 06d73d648..1edc21cdf 100644
>>> > --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
>>> > +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
>>> > @@ -101,9 +101,6 @@ PACKAGE_BEFORE_PN = "${PN}-compiler ${PN}-lite"
>>> >   FILES:${PN}-compiler = "${bindir} ${libdir}/libprotoc${SOLIBS}"
>>> >   FILES:${PN}-lite = "${libdir}/libprotobuf-lite${SOLIBS}"
>>> >
>>> > -# CMake requires binaries to exist in sysroot, even if they have
wrong architecture.
>>> > -SYSROOT_DIRS += "${bindir}"
>>> > -
>>> >   RDEPENDS:${PN}-compiler = "${PN}"
>>> >   RDEPENDS:${PN}-dev += "${PN}-compiler"
>>> >   RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG',
'python', 'python3-protobuf', '', d)}"
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#105291):
https://lists.openembedded.org/g/openembedded-devel/message/105291
>> Mute This Topic: https://lists.openembedded.org/mt/101679410/3617156
>> Group Owner: openembedded-devel+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [
martin.jansa@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>

[-- Attachment #2: Type: text/html, Size: 7436 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-01 10:31       ` Clément Péron
@ 2023-10-01 18:22         ` Martin Jansa
  2023-10-02 16:25           ` Vyacheslav Yurkov
       [not found]           ` <178A5713556C6E82.9230@lists.openembedded.org>
  0 siblings, 2 replies; 35+ messages in thread
From: Martin Jansa @ 2023-10-01 18:22 UTC (permalink / raw)
  To: Clément Péron
  Cc: Khem Raj, Vyacheslav Yurkov, Ross Burton, Samuli Piippo,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 2331 bytes --]

On Sun, Oct 1, 2023 at 12:31 PM Clément Péron <peron.clem@gmail.com> wrote:

> Hi Martin,
>
> On Sun, 1 Oct 2023 at 12:11, Martin Jansa <martin.jansa@gmail.com> wrote:
> >
> > well, target bindir isn't in default PATH so it wont get executed by
> default
>
> Is it a reason to allow mixing executable arch/tune?
>

What is getting mixed? target sysroot is still for target and native
sysroot for native.

How do you handle it when you want to install protoc/grpc-cpp-plugin for
> the target too?
>

SYSROOT_DIRS variable doesn't affect how it's packaged for target, does it?
It just allows the binary from bindir to be available in RSS next to
binaries from libdir, both still separate from native RSS as it should.

> I have used this work around in some meta-ros recipes where the .cmake
> file was checking the binary existence even when this .cmake was used only
> to get right list of libraries to link with (and the binary was not called
> at all - which imho can happen with protobuf as well)
>
> As you say it's a workaround, and it's an issue with the CMake recipe.
>

Issue with .cmake file from protobuf, right?


> If we can properly fix it at the recipe level, then why should we keep
> this workaround/hack?
>
> Also if you really need it, you can have a .bbappend for protoc/grpc in
> the downstream layer. But I would be in favor to avoid this by default no?
>

I haven't seen any proper fix for this exact issue. Yes you can patch
.cmake generation not to include bindir when building for target (while it
needs it for native build), but then that change will break possible use of
protobuf on target, so the SYSROOT_DIRS still seems like lesser evil here.

Here are some examples from meta-ros:
https://github.com/ros/meta-ros/commit/875ceed4b1c693c2eb0f2b7ea85385be576a62cc
https://github.com/ros/meta-ros/commit/67c03755543655ad0634e078bc30d8ab9969a16c
https://github.com/ros/meta-ros/commit/5e61eb0520d2841c0da7e14eecc833b3c4104533
https://github.com/ros/meta-ros/commit/a53bb7cc8b45a72b704e75311d68b37a6db50cdf

I don't care about protobuf, but the explanation from Samuli looks like the
same issue as what I remember from above and I don't see how SYSROOT_DIRS
causes mixing executables or any issues for installing them for target.

Cheers,

>

[-- Attachment #2: Type: text/html, Size: 4123 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-01 18:22         ` Martin Jansa
@ 2023-10-02 16:25           ` Vyacheslav Yurkov
       [not found]           ` <178A5713556C6E82.9230@lists.openembedded.org>
  1 sibling, 0 replies; 35+ messages in thread
From: Vyacheslav Yurkov @ 2023-10-02 16:25 UTC (permalink / raw)
  To: Martin Jansa, Clément Péron
  Cc: Khem Raj, Ross Burton, Samuli Piippo, openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 1921 bytes --]

On 01.10.2023 20:22, Martin Jansa wrote:
>
> Issue with .cmake file from protobuf, right?

No, it's the issue with the cmake of the project that uses protobuf. But 
that depends on a user case, perhaps I didn't get it properly.

>     If we can properly fix it at the recipe level, then why should we
>     keep this workaround/hack?
>
>     Also if you really need it, you can have a .bbappend for
>     protoc/grpc in the downstream layer. But I would be in favor to
>     avoid this by default no?
>
>
> I haven't seen any proper fix for this exact issue. Yes you can patch 
> .cmake generation not to include bindir when building for target 
> (while it needs it for native build), but then that change will break 
> possible use of protobuf on target, so the SYSROOT_DIRS still seems 
> like lesser evil here.
>
> Here are some examples from meta-ros:
> https://github.com/ros/meta-ros/commit/875ceed4b1c693c2eb0f2b7ea85385be576a62cc
> https://github.com/ros/meta-ros/commit/67c03755543655ad0634e078bc30d8ab9969a16c
> https://github.com/ros/meta-ros/commit/5e61eb0520d2841c0da7e14eecc833b3c4104533
> https://github.com/ros/meta-ros/commit/a53bb7cc8b45a72b704e75311d68b37a6db50cdf
>
> I don't care about protobuf, but the explanation from Samuli looks 
> like the same issue as what I remember from above and I don't see how 
> SYSROOT_DIRS causes mixing executables or any issues for installing 
> them for target.

I didn't look at the original issue of Samuli, but your issue is also 
fixed by a workaround. IMHO, the proper fix would be not to make CMake 
"happy" by providing a binary it can't use, but a proper export of 
cross-compiled tools for 
https://github.com/geographiclib/geographiclib/blob/main/CMakeLists.txt 
library. I'd say the workflow should be similar to protobuf/grpc 
recipes, i.e. a native recipe builds the libraries only, and the target 
recipe builds tools and libraries.

Regards,
Slava

[-- Attachment #2: Type: text/html, Size: 4370 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
       [not found]           ` <178A5713556C6E82.9230@lists.openembedded.org>
@ 2023-10-12 12:06             ` Vyacheslav Yurkov
  2023-10-18 12:28               ` Samuli Piippo
  0 siblings, 1 reply; 35+ messages in thread
From: Vyacheslav Yurkov @ 2023-10-12 12:06 UTC (permalink / raw)
  To: Martin Jansa, Clément Péron
  Cc: Khem Raj, Ross Burton, Samuli Piippo, openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 3805 bytes --]

I'd like to follow-up on this and say that I see this issue now with 
nativesdk build, in particular nativesdk-grpc recipe fails in master 
with the same error. I hope that's partially related to the issue 
everybody is confused about, but I'd like understand how to properly fix it.

|   The imported target "protobuf::protoc" references the file
|
| 
"/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
|
|   but this file does not exist.  Possible reasons include:
|
|   * The file was deleted, renamed, or moved to another location.

The protoc-23.4.0 file already exists in the recipe-sysroot-native.

And we do want protoc to be present in the SDK, but nativesdk build is a 
cross-compilation, because SDKMACHINE and HOST might be different.
In this particular grpc case there's an option to say where the actual 
protoc executable is ( 
-D_gRPC_PROTOBUF_PROTOC_EXECUTABLE=/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot-native/usr/bin/protoc 
is passed to grpc).

So there shouldn't be any issues here, but nativesdk build still fails.

Looking into it further I see that protoc is actually packaged by 
protobuf-compiler package, but when I try to add a package dependency to 
grpc recipe I get this:

nativesdk-protobuf RPROVIDES nativesdk-protobuf-compiler

Moreover, if I go into devshell and forcefully install 
nativesdk-libprotobuf-compiler, then `bitbake nativesdk-grpc` is happy 
and compiles!

So something wrong with the dependencies in protobuf recipe, but I can't 
figure out yet what exactly. Any ideas?

Regards,
Slava

On 02.10.2023 18:25, Vyacheslav Yurkov via lists.openembedded.org wrote:
> On 01.10.2023 20:22, Martin Jansa wrote:
>>
>> Issue with .cmake file from protobuf, right?
>
> No, it's the issue with the cmake of the project that uses protobuf. 
> But that depends on a user case, perhaps I didn't get it properly.
>
>>     If we can properly fix it at the recipe level, then why should we
>>     keep this workaround/hack?
>>
>>     Also if you really need it, you can have a .bbappend for
>>     protoc/grpc in the downstream layer. But I would be in favor to
>>     avoid this by default no?
>>
>>
>> I haven't seen any proper fix for this exact issue. Yes you can patch 
>> .cmake generation not to include bindir when building for target 
>> (while it needs it for native build), but then that change will break 
>> possible use of protobuf on target, so the SYSROOT_DIRS still seems 
>> like lesser evil here.
>>
>> Here are some examples from meta-ros:
>> https://github.com/ros/meta-ros/commit/875ceed4b1c693c2eb0f2b7ea85385be576a62cc
>> https://github.com/ros/meta-ros/commit/67c03755543655ad0634e078bc30d8ab9969a16c
>> https://github.com/ros/meta-ros/commit/5e61eb0520d2841c0da7e14eecc833b3c4104533
>> https://github.com/ros/meta-ros/commit/a53bb7cc8b45a72b704e75311d68b37a6db50cdf
>>
>> I don't care about protobuf, but the explanation from Samuli looks 
>> like the same issue as what I remember from above and I don't see how 
>> SYSROOT_DIRS causes mixing executables or any issues for installing 
>> them for target.
>
> I didn't look at the original issue of Samuli, but your issue is also 
> fixed by a workaround. IMHO, the proper fix would be not to make CMake 
> "happy" by providing a binary it can't use, but a proper export of 
> cross-compiled tools for 
> https://github.com/geographiclib/geographiclib/blob/main/CMakeLists.txt 
> library. I'd say the workflow should be similar to protobuf/grpc 
> recipes, i.e. a native recipe builds the libraries only, and the 
> target recipe builds tools and libraries.
>
> Regards,
> Slava

[-- Attachment #2: Type: text/html, Size: 7000 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-12 12:06             ` Vyacheslav Yurkov
@ 2023-10-18 12:28               ` Samuli Piippo
  2023-10-18 15:27                 ` Ross Burton
  2023-10-20  6:17                 ` Vyacheslav Yurkov
  0 siblings, 2 replies; 35+ messages in thread
From: Samuli Piippo @ 2023-10-18 12:28 UTC (permalink / raw)
  To: Vyacheslav Yurkov
  Cc: Martin Jansa, Clément Péron, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 4952 bytes --]

Following simple test recipe will fail now when trying to use Protobuf with
CMake.

inherit cmake
DEPENDS += "protobuf"

do_configure:prepend() {
    echo "find_package(Protobuf CONFIG)" > ${S}/CMakeLists.txt
}

BBCLASSEXTEND = "nativesdk"

---
CMake Error at
tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake:107
(message):
  The imported target "protobuf::protoc" references the file


 "tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"

  but this file does not exist.
---

I don't quite see why every project/recipe should fix this independently
when the simple workaround for this yocto-specific issue is available
(namely the use of SYSROOT_DIRS).

-samuli

ps. qtgrpc recipe and other Qt6 recipes are now visible in the layerindex
https://layers.openembedded.org/layerindex/recipe/348210/


On Thu, 12 Oct 2023 at 15:06, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:

> I'd like to follow-up on this and say that I see this issue now with
> nativesdk build, in particular nativesdk-grpc recipe fails in master with
> the same error. I hope that's partially related to the issue everybody is
> confused about, but I'd like understand how to properly fix it.
>
> |   The imported target "protobuf::protoc" references the file
> |
> |
> "/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
> |
> |   but this file does not exist.  Possible reasons include:
> |
> |   * The file was deleted, renamed, or moved to another location.
>
> The protoc-23.4.0 file already exists in the recipe-sysroot-native.
>
> And we do want protoc to be present in the SDK, but nativesdk build is a
> cross-compilation, because SDKMACHINE and HOST might be different.
> In this particular grpc case there's an option to say where the actual
> protoc executable is (
> -D_gRPC_PROTOBUF_PROTOC_EXECUTABLE=/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot-native/usr/bin/protoc
> is passed to grpc).
>
> So there shouldn't be any issues here, but nativesdk build still fails.
>
> Looking into it further I see that protoc is actually packaged by
> protobuf-compiler package, but when I try to add a package dependency to
> grpc recipe I get this:
>
> nativesdk-protobuf RPROVIDES nativesdk-protobuf-compiler
>
> Moreover, if I go into devshell and forcefully install
> nativesdk-libprotobuf-compiler, then `bitbake nativesdk-grpc` is happy and
> compiles!
>
> So something wrong with the dependencies in protobuf recipe, but I can't
> figure out yet what exactly. Any ideas?
>
> Regards,
> Slava
>
> On 02.10.2023 18:25, Vyacheslav Yurkov via lists.openembedded.org wrote:
>
> On 01.10.2023 20:22, Martin Jansa wrote:
>
>
> Issue with .cmake file from protobuf, right?
>
>
> No, it's the issue with the cmake of the project that uses protobuf. But
> that depends on a user case, perhaps I didn't get it properly.
>
>
>
>> If we can properly fix it at the recipe level, then why should we keep
>> this workaround/hack?
>>
>> Also if you really need it, you can have a .bbappend for protoc/grpc in
>> the downstream layer. But I would be in favor to avoid this by default no?
>>
>
> I haven't seen any proper fix for this exact issue. Yes you can patch
> .cmake generation not to include bindir when building for target (while it
> needs it for native build), but then that change will break possible use of
> protobuf on target, so the SYSROOT_DIRS still seems like lesser evil here.
>
> Here are some examples from meta-ros:
>
> https://github.com/ros/meta-ros/commit/875ceed4b1c693c2eb0f2b7ea85385be576a62cc
>
> https://github.com/ros/meta-ros/commit/67c03755543655ad0634e078bc30d8ab9969a16c
>
> https://github.com/ros/meta-ros/commit/5e61eb0520d2841c0da7e14eecc833b3c4104533
>
> https://github.com/ros/meta-ros/commit/a53bb7cc8b45a72b704e75311d68b37a6db50cdf
>
> I don't care about protobuf, but the explanation from Samuli looks like
> the same issue as what I remember from above and I don't see how
> SYSROOT_DIRS causes mixing executables or any issues for installing them
> for target.
>
>
> I didn't look at the original issue of Samuli, but your issue is also
> fixed by a workaround. IMHO, the proper fix would be not to make CMake
> "happy" by providing a binary it can't use, but a proper export of
> cross-compiled tools for
> https://github.com/geographiclib/geographiclib/blob/main/CMakeLists.txt
> library. I'd say the workflow should be similar to protobuf/grpc recipes,
> i.e. a native recipe builds the libraries only, and the target recipe
> builds tools and libraries.
>
> Regards,
> Slava
>
>
>

[-- Attachment #2: Type: text/html, Size: 8290 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-18 12:28               ` Samuli Piippo
@ 2023-10-18 15:27                 ` Ross Burton
  2023-10-18 15:34                   ` Martin Jansa
  2023-10-20  6:17                 ` Vyacheslav Yurkov
  1 sibling, 1 reply; 35+ messages in thread
From: Ross Burton @ 2023-10-18 15:27 UTC (permalink / raw)
  To: Samuli Piippo
  Cc: Vyacheslav Yurkov, Martin Jansa, Clément Péron,
	Khem Raj, openembedded-devel

On 18 Oct 2023, at 13:28, Samuli Piippo <samuli.piippo@gmail.com> wrote:
> I don't quite see why every project/recipe should fix this independently when the simple workaround for this yocto-specific issue is available (namely the use of SYSROOT_DIRS).

I’d say the one fix should be done in the protobuf cmake fragments.  It’s trying to find a binary _it can’t run_ which is wrong.

Ross

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-18 15:27                 ` Ross Burton
@ 2023-10-18 15:34                   ` Martin Jansa
  0 siblings, 0 replies; 35+ messages in thread
From: Martin Jansa @ 2023-10-18 15:34 UTC (permalink / raw)
  To: Ross Burton
  Cc: Samuli Piippo, Vyacheslav Yurkov, Clément Péron,
	Khem Raj, openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 924 bytes --]

On Wed, Oct 18, 2023 at 5:27 PM Ross Burton <ross.burton@arm.com> wrote:

> On 18 Oct 2023, at 13:28, Samuli Piippo <samuli.piippo@gmail.com> wrote:
> > I don't quite see why every project/recipe should fix this independently
> when the simple workaround for this yocto-specific issue is available
> (namely the use of SYSROOT_DIRS).
>
> I’d say the one fix should be done in the protobuf cmake fragments.  It’s
> trying to find a binary _it can’t run_ which is wrong.
>

It's not that bad as long as it doesn't really plan to run it (only check
that it exists and that its fragment doesn't reference missing files).

The same fragment will be used from PN-dev if someone tries to use protobuf
on target and in that case the binary should exist and run. Good luck
trying to provide different cmake fragments to be used from target protobuf
in RSS and from target protobuf-dev on target.

Cheers,

[-- Attachment #2: Type: text/html, Size: 1364 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-18 12:28               ` Samuli Piippo
  2023-10-18 15:27                 ` Ross Burton
@ 2023-10-20  6:17                 ` Vyacheslav Yurkov
  2023-10-20  7:07                   ` Martin Jansa
  2023-10-20  7:11                   ` Samuli Piippo
  1 sibling, 2 replies; 35+ messages in thread
From: Vyacheslav Yurkov @ 2023-10-20  6:17 UTC (permalink / raw)
  To: Samuli Piippo
  Cc: Martin Jansa, Clément Péron, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 4019 bytes --]

I think the problem goes down to how dependencies are populated in the 
sysroot. I raised this question in oe-core mailing list 
https://lists.openembedded.org/g/openembedded-core/message/189324 , but 
didn't get an answer yet.

DEPENDS += "protobuf" will only install the main package of protobuf to 
the sysroot, because it assumes that nativesdk-protobuf-compiler is also 
part of the main package (which is not). I guess the part of the system 
populating sysroot deals with recipes only, and not with packages. I'm 
still trying to understand how that should be addressed properly.
My understanding is that either all packages should be installed to 
sysroot (which recipe claims it provides), or we need a way to indicate 
a package level dependency (i.e. nativesdk-protobuf-compiler in this case)

Any ideas?

Slava

On 18.10.2023 14:28, Samuli Piippo wrote:
> Following simple test recipe will fail now when trying to use Protobuf 
> with CMake.
>
> inherit cmake
> DEPENDS += "protobuf"
>
> do_configure:prepend() {
>     echo "find_package(Protobuf CONFIG)" > ${S}/CMakeLists.txt
> }
>
> BBCLASSEXTEND = "nativesdk"
>
> ---
> CMake Error at 
> tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake:107 
> (message):
>   The imported target "protobuf::protoc" references the file
>
>  "tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>
>   but this file does not exist.
> ---
>
> I don't quite see why every project/recipe should fix this 
> independently when the simple workaround for this yocto-specific issue 
> is available (namely the use of SYSROOT_DIRS).
>
> -samuli
>
> ps. qtgrpc recipe and other Qt6 recipes are now visible in the 
> layerindex https://layers.openembedded.org/layerindex/recipe/348210/
>
>
> On Thu, 12 Oct 2023 at 15:06, Vyacheslav Yurkov <uvv.mail@gmail.com> 
> wrote:
>
>     I'd like to follow-up on this and say that I see this issue now
>     with nativesdk build, in particular nativesdk-grpc recipe fails in
>     master with the same error. I hope that's partially related to the
>     issue everybody is confused about, but I'd like understand how to
>     properly fix it.
>
>     |   The imported target "protobuf::protoc" references the file
>     |
>     |
>     "/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>     |
>     |   but this file does not exist.  Possible reasons include:
>     |
>     |   * The file was deleted, renamed, or moved to another location.
>
>     The protoc-23.4.0 file already exists in the recipe-sysroot-native.
>
>     And we do want protoc to be present in the SDK, but nativesdk
>     build is a cross-compilation, because SDKMACHINE and HOST might be
>     different.
>     In this particular grpc case there's an option to say where the
>     actual protoc executable is (
>     -D_gRPC_PROTOBUF_PROTOC_EXECUTABLE=/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot-native/usr/bin/protoc
>     is passed to grpc).
>
>     So there shouldn't be any issues here, but nativesdk build still
>     fails.
>
>     Looking into it further I see that protoc is actually packaged by
>     protobuf-compiler package, but when I try to add a package
>     dependency to grpc recipe I get this:
>
>     nativesdk-protobuf RPROVIDES nativesdk-protobuf-compiler
>
>     Moreover, if I go into devshell and forcefully install
>     nativesdk-libprotobuf-compiler, then `bitbake nativesdk-grpc` is
>     happy and compiles!
>
>     So something wrong with the dependencies in protobuf recipe, but I
>     can't figure out yet what exactly. Any ideas?
>
>     Regards,
>     Slava
>

[-- Attachment #2: Type: text/html, Size: 6468 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-20  6:17                 ` Vyacheslav Yurkov
@ 2023-10-20  7:07                   ` Martin Jansa
  2023-10-20  7:11                   ` Samuli Piippo
  1 sibling, 0 replies; 35+ messages in thread
From: Martin Jansa @ 2023-10-20  7:07 UTC (permalink / raw)
  To: Vyacheslav Yurkov
  Cc: Samuli Piippo, Clément Péron, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 4399 bytes --]

No, it installs whatever recipe stages in do_populate_sysroot which is
controlled by SYSROOT_DIRS, so the solution still makes sense to me as
explained before and IMHO is better than the alternatives (changing the
protobuf .cmake files to be different in sysroot and in PN-dev).

On Fri, Oct 20, 2023 at 8:17 AM Vyacheslav Yurkov <uvv.mail@gmail.com>
wrote:

> I think the problem goes down to how dependencies are populated in the
> sysroot. I raised this question in oe-core mailing list
> https://lists.openembedded.org/g/openembedded-core/message/189324 , but
> didn't get an answer yet.
>
> DEPENDS += "protobuf" will only install the main package of protobuf to
> the sysroot, because it assumes that nativesdk-protobuf-compiler is also
> part of the main package (which is not). I guess the part of the system
> populating sysroot deals with recipes only, and not with packages. I'm
> still trying to understand how that should be addressed properly.
> My understanding is that either all packages should be installed to
> sysroot (which recipe claims it provides), or we need a way to indicate a
> package level dependency (i.e. nativesdk-protobuf-compiler in this case)
>
> Any ideas?
>
> Slava
>
> On 18.10.2023 14:28, Samuli Piippo wrote:
>
> Following simple test recipe will fail now when trying to use Protobuf
> with CMake.
>
> inherit cmake
> DEPENDS += "protobuf"
>
> do_configure:prepend() {
>     echo "find_package(Protobuf CONFIG)" > ${S}/CMakeLists.txt
> }
>
> BBCLASSEXTEND = "nativesdk"
>
> ---
> CMake Error at
> tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake:107
> (message):
>   The imported target "protobuf::protoc" references the file
>
>
>  "tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>
>   but this file does not exist.
> ---
>
> I don't quite see why every project/recipe should fix this independently
> when the simple workaround for this yocto-specific issue is available
> (namely the use of SYSROOT_DIRS).
>
> -samuli
>
> ps. qtgrpc recipe and other Qt6 recipes are now visible in the layerindex
> https://layers.openembedded.org/layerindex/recipe/348210/
>
>
> On Thu, 12 Oct 2023 at 15:06, Vyacheslav Yurkov <uvv.mail@gmail.com>
> wrote:
>
>> I'd like to follow-up on this and say that I see this issue now with
>> nativesdk build, in particular nativesdk-grpc recipe fails in master with
>> the same error. I hope that's partially related to the issue everybody is
>> confused about, but I'd like understand how to properly fix it.
>>
>> |   The imported target "protobuf::protoc" references the file
>> |
>> |
>> "/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>> |
>> |   but this file does not exist.  Possible reasons include:
>> |
>> |   * The file was deleted, renamed, or moved to another location.
>>
>> The protoc-23.4.0 file already exists in the recipe-sysroot-native.
>>
>> And we do want protoc to be present in the SDK, but nativesdk build is a
>> cross-compilation, because SDKMACHINE and HOST might be different.
>> In this particular grpc case there's an option to say where the actual
>> protoc executable is (
>> -D_gRPC_PROTOBUF_PROTOC_EXECUTABLE=/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot-native/usr/bin/protoc
>> is passed to grpc).
>>
>> So there shouldn't be any issues here, but nativesdk build still fails.
>>
>> Looking into it further I see that protoc is actually packaged by
>> protobuf-compiler package, but when I try to add a package dependency to
>> grpc recipe I get this:
>>
>> nativesdk-protobuf RPROVIDES nativesdk-protobuf-compiler
>>
>> Moreover, if I go into devshell and forcefully install
>> nativesdk-libprotobuf-compiler, then `bitbake nativesdk-grpc` is happy and
>> compiles!
>>
>> So something wrong with the dependencies in protobuf recipe, but I can't
>> figure out yet what exactly. Any ideas?
>>
>> Regards,
>> Slava
>>
>>

[-- Attachment #2: Type: text/html, Size: 6958 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-20  6:17                 ` Vyacheslav Yurkov
  2023-10-20  7:07                   ` Martin Jansa
@ 2023-10-20  7:11                   ` Samuli Piippo
  2023-10-20  7:35                     ` Martin Jansa
  1 sibling, 1 reply; 35+ messages in thread
From: Samuli Piippo @ 2023-10-20  7:11 UTC (permalink / raw)
  To: Vyacheslav Yurkov
  Cc: Martin Jansa, Clément Péron, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 4451 bytes --]

Dependency to the recipe name will always install all packages to RSS.
You can verify this by checking that the libprotoc.so from the
nativesdk-protobuf-compile package is in the sysroot.
The problem you are facing is the fact that binaries are not installed into
the sysroot (for nativesdk builds), so you are only missing the
bin/protoc-23.4.0 file.
The proposed and twice reverted fix for this is the aforementioned
SYSROOT_DIRS.

-samuli

On Fri, 20 Oct 2023 at 09:17, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:

> I think the problem goes down to how dependencies are populated in the
> sysroot. I raised this question in oe-core mailing list
> https://lists.openembedded.org/g/openembedded-core/message/189324 , but
> didn't get an answer yet.
>
> DEPENDS += "protobuf" will only install the main package of protobuf to
> the sysroot, because it assumes that nativesdk-protobuf-compiler is also
> part of the main package (which is not). I guess the part of the system
> populating sysroot deals with recipes only, and not with packages. I'm
> still trying to understand how that should be addressed properly.
> My understanding is that either all packages should be installed to
> sysroot (which recipe claims it provides), or we need a way to indicate a
> package level dependency (i.e. nativesdk-protobuf-compiler in this case)
>
> Any ideas?
>
> Slava
>
> On 18.10.2023 14:28, Samuli Piippo wrote:
>
> Following simple test recipe will fail now when trying to use Protobuf
> with CMake.
>
> inherit cmake
> DEPENDS += "protobuf"
>
> do_configure:prepend() {
>     echo "find_package(Protobuf CONFIG)" > ${S}/CMakeLists.txt
> }
>
> BBCLASSEXTEND = "nativesdk"
>
> ---
> CMake Error at
> tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake:107
> (message):
>   The imported target "protobuf::protoc" references the file
>
>
>  "tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>
>   but this file does not exist.
> ---
>
> I don't quite see why every project/recipe should fix this independently
> when the simple workaround for this yocto-specific issue is available
> (namely the use of SYSROOT_DIRS).
>
> -samuli
>
> ps. qtgrpc recipe and other Qt6 recipes are now visible in the layerindex
> https://layers.openembedded.org/layerindex/recipe/348210/
>
>
> On Thu, 12 Oct 2023 at 15:06, Vyacheslav Yurkov <uvv.mail@gmail.com>
> wrote:
>
>> I'd like to follow-up on this and say that I see this issue now with
>> nativesdk build, in particular nativesdk-grpc recipe fails in master with
>> the same error. I hope that's partially related to the issue everybody is
>> confused about, but I'd like understand how to properly fix it.
>>
>> |   The imported target "protobuf::protoc" references the file
>> |
>> |
>> "/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>> |
>> |   but this file does not exist.  Possible reasons include:
>> |
>> |   * The file was deleted, renamed, or moved to another location.
>>
>> The protoc-23.4.0 file already exists in the recipe-sysroot-native.
>>
>> And we do want protoc to be present in the SDK, but nativesdk build is a
>> cross-compilation, because SDKMACHINE and HOST might be different.
>> In this particular grpc case there's an option to say where the actual
>> protoc executable is (
>> -D_gRPC_PROTOBUF_PROTOC_EXECUTABLE=/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot-native/usr/bin/protoc
>> is passed to grpc).
>>
>> So there shouldn't be any issues here, but nativesdk build still fails.
>>
>> Looking into it further I see that protoc is actually packaged by
>> protobuf-compiler package, but when I try to add a package dependency to
>> grpc recipe I get this:
>>
>> nativesdk-protobuf RPROVIDES nativesdk-protobuf-compiler
>>
>> Moreover, if I go into devshell and forcefully install
>> nativesdk-libprotobuf-compiler, then `bitbake nativesdk-grpc` is happy and
>> compiles!
>>
>> So something wrong with the dependencies in protobuf recipe, but I can't
>> figure out yet what exactly. Any ideas?
>>
>> Regards,
>> Slava
>>
>>

[-- Attachment #2: Type: text/html, Size: 7189 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-20  7:11                   ` Samuli Piippo
@ 2023-10-20  7:35                     ` Martin Jansa
  2023-10-22 18:45                       ` Vyacheslav Yurkov
  0 siblings, 1 reply; 35+ messages in thread
From: Martin Jansa @ 2023-10-20  7:35 UTC (permalink / raw)
  To: Samuli Piippo
  Cc: Vyacheslav Yurkov, Clément Péron, Khem Raj,
	Ross Burton, openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 4966 bytes --]

do_prepare_recipe_sysroot doesn't install any packages to RSS. Only package
manager on target or in do_rootfs uses packages.

It installs the files from dependency populate_sysroot task.

On Fri, Oct 20, 2023 at 9:11 AM Samuli Piippo <samuli.piippo@gmail.com>
wrote:

> Dependency to the recipe name will always install all packages to RSS.
> You can verify this by checking that the libprotoc.so from the
> nativesdk-protobuf-compile package is in the sysroot.
> The problem you are facing is the fact that binaries are not installed
> into the sysroot (for nativesdk builds), so you are only missing the
> bin/protoc-23.4.0 file.
> The proposed and twice reverted fix for this is the aforementioned
> SYSROOT_DIRS.
>
> -samuli
>
> On Fri, 20 Oct 2023 at 09:17, Vyacheslav Yurkov <uvv.mail@gmail.com>
> wrote:
>
>> I think the problem goes down to how dependencies are populated in the
>> sysroot. I raised this question in oe-core mailing list
>> https://lists.openembedded.org/g/openembedded-core/message/189324 , but
>> didn't get an answer yet.
>>
>> DEPENDS += "protobuf" will only install the main package of protobuf to
>> the sysroot, because it assumes that nativesdk-protobuf-compiler is also
>> part of the main package (which is not). I guess the part of the system
>> populating sysroot deals with recipes only, and not with packages. I'm
>> still trying to understand how that should be addressed properly.
>> My understanding is that either all packages should be installed to
>> sysroot (which recipe claims it provides), or we need a way to indicate a
>> package level dependency (i.e. nativesdk-protobuf-compiler in this case)
>>
>> Any ideas?
>>
>> Slava
>>
>> On 18.10.2023 14:28, Samuli Piippo wrote:
>>
>> Following simple test recipe will fail now when trying to use Protobuf
>> with CMake.
>>
>> inherit cmake
>> DEPENDS += "protobuf"
>>
>> do_configure:prepend() {
>>     echo "find_package(Protobuf CONFIG)" > ${S}/CMakeLists.txt
>> }
>>
>> BBCLASSEXTEND = "nativesdk"
>>
>> ---
>> CMake Error at
>> tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake:107
>> (message):
>>   The imported target "protobuf::protoc" references the file
>>
>>
>>  "tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>>
>>   but this file does not exist.
>> ---
>>
>> I don't quite see why every project/recipe should fix this independently
>> when the simple workaround for this yocto-specific issue is available
>> (namely the use of SYSROOT_DIRS).
>>
>> -samuli
>>
>> ps. qtgrpc recipe and other Qt6 recipes are now visible in the layerindex
>> https://layers.openembedded.org/layerindex/recipe/348210/
>>
>>
>> On Thu, 12 Oct 2023 at 15:06, Vyacheslav Yurkov <uvv.mail@gmail.com>
>> wrote:
>>
>>> I'd like to follow-up on this and say that I see this issue now with
>>> nativesdk build, in particular nativesdk-grpc recipe fails in master with
>>> the same error. I hope that's partially related to the issue everybody is
>>> confused about, but I'd like understand how to properly fix it.
>>>
>>> |   The imported target "protobuf::protoc" references the file
>>> |
>>> |
>>> "/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>>> |
>>> |   but this file does not exist.  Possible reasons include:
>>> |
>>> |   * The file was deleted, renamed, or moved to another location.
>>>
>>> The protoc-23.4.0 file already exists in the recipe-sysroot-native.
>>>
>>> And we do want protoc to be present in the SDK, but nativesdk build is a
>>> cross-compilation, because SDKMACHINE and HOST might be different.
>>> In this particular grpc case there's an option to say where the actual
>>> protoc executable is (
>>> -D_gRPC_PROTOBUF_PROTOC_EXECUTABLE=/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot-native/usr/bin/protoc
>>> is passed to grpc).
>>>
>>> So there shouldn't be any issues here, but nativesdk build still fails.
>>>
>>> Looking into it further I see that protoc is actually packaged by
>>> protobuf-compiler package, but when I try to add a package dependency to
>>> grpc recipe I get this:
>>>
>>> nativesdk-protobuf RPROVIDES nativesdk-protobuf-compiler
>>>
>>> Moreover, if I go into devshell and forcefully install
>>> nativesdk-libprotobuf-compiler, then `bitbake nativesdk-grpc` is happy and
>>> compiles!
>>>
>>> So something wrong with the dependencies in protobuf recipe, but I can't
>>> figure out yet what exactly. Any ideas?
>>>
>>> Regards,
>>> Slava
>>>
>>>

[-- Attachment #2: Type: text/html, Size: 7786 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-20  7:35                     ` Martin Jansa
@ 2023-10-22 18:45                       ` Vyacheslav Yurkov
  2023-10-24 11:52                         ` Clément Péron
  0 siblings, 1 reply; 35+ messages in thread
From: Vyacheslav Yurkov @ 2023-10-22 18:45 UTC (permalink / raw)
  To: Martin Jansa, Samuli Piippo
  Cc: Clément Péron, Khem Raj, Ross Burton, openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 6056 bytes --]

I'm a bit confused here.

I just did 'find 
tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/ -name 
"protoc*"' and I see protoc binary only in recipe-sysroot-native. That 
tells me that protoc, which is a part of nativesdk-protobuf-compiler, is 
not installed to recipe-sysroot.

What am I missing here?

Slava

On 20.10.2023 09:35, Martin Jansa wrote:
> do_prepare_recipe_sysroot doesn't install any packages to RSS. Only 
> package manager on target or in do_rootfs uses packages.
>
> It installs the files from dependency populate_sysroot task.
>
> On Fri, Oct 20, 2023 at 9:11 AM Samuli Piippo 
> <samuli.piippo@gmail.com> wrote:
>
>     Dependency to the recipe name will always install all packages to RSS.
>     You can verify this by checking that the libprotoc.so from the
>     nativesdk-protobuf-compile package is in the sysroot.
>     The problem you are facing is the fact that binaries are not
>     installed into the sysroot (for nativesdk builds), so you are only
>     missing the bin/protoc-23.4.0 file.
>     The proposed and twice reverted fix for this is the aforementioned
>     SYSROOT_DIRS.
>
>     -samuli
>
>     On Fri, 20 Oct 2023 at 09:17, Vyacheslav Yurkov
>     <uvv.mail@gmail.com> wrote:
>
>         I think the problem goes down to how dependencies are
>         populated in the sysroot. I raised this question in oe-core
>         mailing list
>         https://lists.openembedded.org/g/openembedded-core/message/189324
>         , but didn't get an answer yet.
>
>         DEPENDS += "protobuf" will only install the main package of
>         protobuf to the sysroot, because it assumes that
>         nativesdk-protobuf-compiler is also part of the main package
>         (which is not). I guess the part of the system populating
>         sysroot deals with recipes only, and not with packages. I'm
>         still trying to understand how that should be addressed properly.
>         My understanding is that either all packages should be
>         installed to sysroot (which recipe claims it provides), or we
>         need a way to indicate a package level dependency (i.e.
>         nativesdk-protobuf-compiler in this case)
>
>         Any ideas?
>
>         Slava
>
>         On 18.10.2023 14:28, Samuli Piippo wrote:
>>         Following simple test recipe will fail now when trying to use
>>         Protobuf with CMake.
>>
>>         inherit cmake
>>         DEPENDS += "protobuf"
>>
>>         do_configure:prepend() {
>>             echo "find_package(Protobuf CONFIG)" > ${S}/CMakeLists.txt
>>         }
>>
>>         BBCLASSEXTEND = "nativesdk"
>>
>>         ---
>>         CMake Error at
>>         tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake:107
>>         (message):
>>           The imported target "protobuf::protoc" references the file
>>
>>          "tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>>
>>           but this file does not exist.
>>         ---
>>
>>         I don't quite see why every project/recipe should fix this
>>         independently when the simple workaround for this
>>         yocto-specific issue is available (namely the use
>>         of SYSROOT_DIRS).
>>
>>         -samuli
>>
>>         ps. qtgrpc recipe and other Qt6 recipes are now visible in
>>         the layerindex
>>         https://layers.openembedded.org/layerindex/recipe/348210/
>>
>>
>>         On Thu, 12 Oct 2023 at 15:06, Vyacheslav Yurkov
>>         <uvv.mail@gmail.com> wrote:
>>
>>             I'd like to follow-up on this and say that I see this
>>             issue now with nativesdk build, in particular
>>             nativesdk-grpc recipe fails in master with the same
>>             error. I hope that's partially related to the issue
>>             everybody is confused about, but I'd like understand how
>>             to properly fix it.
>>
>>             |   The imported target "protobuf::protoc" references the
>>             file
>>             |
>>             |
>>             "/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>>             |
>>             |   but this file does not exist.  Possible reasons include:
>>             |
>>             |   * The file was deleted, renamed, or moved to another
>>             location.
>>
>>             The protoc-23.4.0 file already exists in the
>>             recipe-sysroot-native.
>>
>>             And we do want protoc to be present in the SDK, but
>>             nativesdk build is a cross-compilation, because
>>             SDKMACHINE and HOST might be different.
>>             In this particular grpc case there's an option to say
>>             where the actual protoc executable is (
>>             -D_gRPC_PROTOBUF_PROTOC_EXECUTABLE=/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot-native/usr/bin/protoc
>>             is passed to grpc).
>>
>>             So there shouldn't be any issues here, but nativesdk
>>             build still fails.
>>
>>             Looking into it further I see that protoc is actually
>>             packaged by protobuf-compiler package, but when I try to
>>             add a package dependency to grpc recipe I get this:
>>
>>             nativesdk-protobuf RPROVIDES nativesdk-protobuf-compiler
>>
>>             Moreover, if I go into devshell and forcefully install
>>             nativesdk-libprotobuf-compiler, then `bitbake
>>             nativesdk-grpc` is happy and compiles!
>>
>>             So something wrong with the dependencies in protobuf
>>             recipe, but I can't figure out yet what exactly. Any ideas?
>>
>>             Regards,
>>             Slava
>>


[-- Attachment #2: Type: text/html, Size: 11025 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-22 18:45                       ` Vyacheslav Yurkov
@ 2023-10-24 11:52                         ` Clément Péron
  2023-10-24 13:25                           ` Samuli Piippo
       [not found]                           ` <17910E05B5C0179B.11247@lists.openembedded.org>
  0 siblings, 2 replies; 35+ messages in thread
From: Clément Péron @ 2023-10-24 11:52 UTC (permalink / raw)
  To: Vyacheslav Yurkov
  Cc: Martin Jansa, Samuli Piippo, Khem Raj, Ross Burton, openembedded-devel

Hi,

I have pushed a branch on my repo:
https://github.com/clementperon/meta-openembedded/tree/test_protobuf

and tried to reproduce the issue on my setup with the following kas yaml file.

But I don't see any issue at the configure stage.

cat XXXX/cortexa57-poky-linux-musl/test-protobuf/0.3/temp/log.do_configure

Show that -- Version:23.4.0 of Protobuf is found without any issue

Cmake host version is
$> cmake --version
cmake version 3.27.7

I'm missing something ?

Regards,
Clement

kas.yml
====================
header:
  version: 8

machine: qemuarm64
distro: poky
target:
  - test-protobuf

repos:
  poky:
    url: https://git.yoctoproject.org/git/poky
    path: layers/poky
    refspec: master
    layers:
      meta:
      meta-poky:
      meta-yocto-bsp:

  meta-openembedded:
    url: https://github.com/clementperon/meta-openembedded
    path: layers/meta-openembedded
    refspec: test_protobuf
    layers:
      meta-oe:
      meta-networking:
      meta-python:

bblayers_conf_header:
  standard: |
    POKY_BBLAYERS_CONF_VERSION = "2"
    BBPATH = "${TOPDIR}"
    BBFILES ?= ""

local_conf_header:
  libc: |
    TCLIBC = "musl"
  standard: |
    CONF_VERSION = "2"
    PACKAGE_CLASSES = "package_ipk"
  debug-tweaks: |
    EXTRA_IMAGE_FEATURES ?= "debug-tweaks"

  diskmon: |
    BB_DISKMON_DIRS = "\
        STOPTASKS,${TMPDIR},1G,100K \
        STOPTASKS,${DL_DIR},1G,100K \
        STOPTASKS,${SSTATE_DIR},1G,100K \
        STOPTASKS,/tmp,100M,100K \
        ABORT,${TMPDIR},100M,1K \
        ABORT,${DL_DIR},100M,1K \
        ABORT,${SSTATE_DIR},100M,1K \
        ABORT,/tmp,10M,1K"
======================

On Sun, 22 Oct 2023 at 20:45, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:
>
> I'm a bit confused here.
>
> I just did 'find tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/ -name "protoc*"' and I see protoc binary only in recipe-sysroot-native. That tells me that protoc, which is a part of nativesdk-protobuf-compiler, is not installed to recipe-sysroot.
>
> What am I missing here?
>
> Slava
>
> On 20.10.2023 09:35, Martin Jansa wrote:
>
> do_prepare_recipe_sysroot doesn't install any packages to RSS. Only package manager on target or in do_rootfs uses packages.
>
> It installs the files from dependency populate_sysroot task.
>
> On Fri, Oct 20, 2023 at 9:11 AM Samuli Piippo <samuli.piippo@gmail.com> wrote:
>>
>> Dependency to the recipe name will always install all packages to RSS.
>> You can verify this by checking that the libprotoc.so from the nativesdk-protobuf-compile package is in the sysroot.
>> The problem you are facing is the fact that binaries are not installed into the sysroot (for nativesdk builds), so you are only missing the bin/protoc-23.4.0 file.
>> The proposed and twice reverted fix for this is the aforementioned SYSROOT_DIRS.
>>
>> -samuli
>>
>> On Fri, 20 Oct 2023 at 09:17, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:
>>>
>>> I think the problem goes down to how dependencies are populated in the sysroot. I raised this question in oe-core mailing list https://lists.openembedded.org/g/openembedded-core/message/189324 , but didn't get an answer yet.
>>>
>>> DEPENDS += "protobuf" will only install the main package of protobuf to the sysroot, because it assumes that nativesdk-protobuf-compiler is also part of the main package (which is not). I guess the part of the system populating sysroot deals with recipes only, and not with packages. I'm still trying to understand how that should be addressed properly.
>>> My understanding is that either all packages should be installed to sysroot (which recipe claims it provides), or we need a way to indicate a package level dependency (i.e. nativesdk-protobuf-compiler in this case)
>>>
>>> Any ideas?
>>>
>>> Slava
>>>
>>> On 18.10.2023 14:28, Samuli Piippo wrote:
>>>
>>> Following simple test recipe will fail now when trying to use Protobuf with CMake.
>>>
>>> inherit cmake
>>> DEPENDS += "protobuf"
>>>
>>> do_configure:prepend() {
>>>     echo "find_package(Protobuf CONFIG)" > ${S}/CMakeLists.txt
>>> }
>>>
>>> BBCLASSEXTEND = "nativesdk"
>>>
>>> ---
>>> CMake Error at tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake:107 (message):
>>>   The imported target "protobuf::protoc" references the file
>>>
>>>      "tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>>>
>>>   but this file does not exist.
>>> ---
>>>
>>> I don't quite see why every project/recipe should fix this independently when the simple workaround for this yocto-specific issue is available (namely the use of SYSROOT_DIRS).
>>>
>>> -samuli
>>>
>>> ps. qtgrpc recipe and other Qt6 recipes are now visible in the layerindex https://layers.openembedded.org/layerindex/recipe/348210/
>>>
>>>
>>> On Thu, 12 Oct 2023 at 15:06, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:
>>>>
>>>> I'd like to follow-up on this and say that I see this issue now with nativesdk build, in particular nativesdk-grpc recipe fails in master with the same error. I hope that's partially related to the issue everybody is confused about, but I'd like understand how to properly fix it.
>>>>
>>>> |   The imported target "protobuf::protoc" references the file
>>>> |
>>>> |      "/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>>>> |
>>>> |   but this file does not exist.  Possible reasons include:
>>>> |
>>>> |   * The file was deleted, renamed, or moved to another location.
>>>>
>>>> The protoc-23.4.0 file already exists in the recipe-sysroot-native.
>>>>
>>>> And we do want protoc to be present in the SDK, but nativesdk build is a cross-compilation, because SDKMACHINE and HOST might be different.
>>>> In this particular grpc case there's an option to say where the actual protoc executable is ( -D_gRPC_PROTOBUF_PROTOC_EXECUTABLE=/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot-native/usr/bin/protoc is passed to grpc).
>>>>
>>>> So there shouldn't be any issues here, but nativesdk build still fails.
>>>>
>>>> Looking into it further I see that protoc is actually packaged by protobuf-compiler package, but when I try to add a package dependency to grpc recipe I get this:
>>>>
>>>> nativesdk-protobuf RPROVIDES nativesdk-protobuf-compiler
>>>>
>>>> Moreover, if I go into devshell and forcefully install nativesdk-libprotobuf-compiler, then `bitbake nativesdk-grpc` is happy and compiles!
>>>>
>>>> So something wrong with the dependencies in protobuf recipe, but I can't figure out yet what exactly. Any ideas?
>>>>
>>>> Regards,
>>>> Slava
>>>>
>
>


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-10-24 11:52                         ` Clément Péron
@ 2023-10-24 13:25                           ` Samuli Piippo
       [not found]                           ` <17910E05B5C0179B.11247@lists.openembedded.org>
  1 sibling, 0 replies; 35+ messages in thread
From: Samuli Piippo @ 2023-10-24 13:25 UTC (permalink / raw)
  To: Clément Péron
  Cc: Vyacheslav Yurkov, Martin Jansa, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 7742 bytes --]

The protoc binary is not enabled for target builds. Try nativesdk build and
it should then fail to find it.

On Tue, 24 Oct 2023 at 14:53, Clément Péron <peron.clem@gmail.com> wrote:

> Hi,
>
> I have pushed a branch on my repo:
> https://github.com/clementperon/meta-openembedded/tree/test_protobuf
>
> and tried to reproduce the issue on my setup with the following kas yaml
> file.
>
> But I don't see any issue at the configure stage.
>
> cat XXXX/cortexa57-poky-linux-musl/test-protobuf/0.3/temp/log.do_configure
>
> Show that -- Version:23.4.0 of Protobuf is found without any issue
>
> Cmake host version is
> $> cmake --version
> cmake version 3.27.7
>
> I'm missing something ?
>
> Regards,
> Clement
>
> kas.yml
> ====================
> header:
>   version: 8
>
> machine: qemuarm64
> distro: poky
> target:
>   - test-protobuf
>
> repos:
>   poky:
>     url: https://git.yoctoproject.org/git/poky
>     path: layers/poky
>     refspec: master
>     layers:
>       meta:
>       meta-poky:
>       meta-yocto-bsp:
>
>   meta-openembedded:
>     url: https://github.com/clementperon/meta-openembedded
>     path: layers/meta-openembedded
>     refspec: test_protobuf
>     layers:
>       meta-oe:
>       meta-networking:
>       meta-python:
>
> bblayers_conf_header:
>   standard: |
>     POKY_BBLAYERS_CONF_VERSION = "2"
>     BBPATH = "${TOPDIR}"
>     BBFILES ?= ""
>
> local_conf_header:
>   libc: |
>     TCLIBC = "musl"
>   standard: |
>     CONF_VERSION = "2"
>     PACKAGE_CLASSES = "package_ipk"
>   debug-tweaks: |
>     EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
>
>   diskmon: |
>     BB_DISKMON_DIRS = "\
>         STOPTASKS,${TMPDIR},1G,100K \
>         STOPTASKS,${DL_DIR},1G,100K \
>         STOPTASKS,${SSTATE_DIR},1G,100K \
>         STOPTASKS,/tmp,100M,100K \
>         ABORT,${TMPDIR},100M,1K \
>         ABORT,${DL_DIR},100M,1K \
>         ABORT,${SSTATE_DIR},100M,1K \
>         ABORT,/tmp,10M,1K"
> ======================
>
> On Sun, 22 Oct 2023 at 20:45, Vyacheslav Yurkov <uvv.mail@gmail.com>
> wrote:
> >
> > I'm a bit confused here.
> >
> > I just did 'find
> tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/ -name
> "protoc*"' and I see protoc binary only in recipe-sysroot-native. That
> tells me that protoc, which is a part of nativesdk-protobuf-compiler, is
> not installed to recipe-sysroot.
> >
> > What am I missing here?
> >
> > Slava
> >
> > On 20.10.2023 09:35, Martin Jansa wrote:
> >
> > do_prepare_recipe_sysroot doesn't install any packages to RSS. Only
> package manager on target or in do_rootfs uses packages.
> >
> > It installs the files from dependency populate_sysroot task.
> >
> > On Fri, Oct 20, 2023 at 9:11 AM Samuli Piippo <samuli.piippo@gmail.com>
> wrote:
> >>
> >> Dependency to the recipe name will always install all packages to RSS.
> >> You can verify this by checking that the libprotoc.so from the
> nativesdk-protobuf-compile package is in the sysroot.
> >> The problem you are facing is the fact that binaries are not installed
> into the sysroot (for nativesdk builds), so you are only missing the
> bin/protoc-23.4.0 file.
> >> The proposed and twice reverted fix for this is the aforementioned
> SYSROOT_DIRS.
> >>
> >> -samuli
> >>
> >> On Fri, 20 Oct 2023 at 09:17, Vyacheslav Yurkov <uvv.mail@gmail.com>
> wrote:
> >>>
> >>> I think the problem goes down to how dependencies are populated in the
> sysroot. I raised this question in oe-core mailing list
> https://lists.openembedded.org/g/openembedded-core/message/189324 , but
> didn't get an answer yet.
> >>>
> >>> DEPENDS += "protobuf" will only install the main package of protobuf
> to the sysroot, because it assumes that nativesdk-protobuf-compiler is also
> part of the main package (which is not). I guess the part of the system
> populating sysroot deals with recipes only, and not with packages. I'm
> still trying to understand how that should be addressed properly.
> >>> My understanding is that either all packages should be installed to
> sysroot (which recipe claims it provides), or we need a way to indicate a
> package level dependency (i.e. nativesdk-protobuf-compiler in this case)
> >>>
> >>> Any ideas?
> >>>
> >>> Slava
> >>>
> >>> On 18.10.2023 14:28, Samuli Piippo wrote:
> >>>
> >>> Following simple test recipe will fail now when trying to use Protobuf
> with CMake.
> >>>
> >>> inherit cmake
> >>> DEPENDS += "protobuf"
> >>>
> >>> do_configure:prepend() {
> >>>     echo "find_package(Protobuf CONFIG)" > ${S}/CMakeLists.txt
> >>> }
> >>>
> >>> BBCLASSEXTEND = "nativesdk"
> >>>
> >>> ---
> >>> CMake Error at
> tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake:107
> (message):
> >>>   The imported target "protobuf::protoc" references the file
> >>>
> >>>
> "tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
> >>>
> >>>   but this file does not exist.
> >>> ---
> >>>
> >>> I don't quite see why every project/recipe should fix this
> independently when the simple workaround for this yocto-specific issue is
> available (namely the use of SYSROOT_DIRS).
> >>>
> >>> -samuli
> >>>
> >>> ps. qtgrpc recipe and other Qt6 recipes are now visible in the
> layerindex https://layers.openembedded.org/layerindex/recipe/348210/
> >>>
> >>>
> >>> On Thu, 12 Oct 2023 at 15:06, Vyacheslav Yurkov <uvv.mail@gmail.com>
> wrote:
> >>>>
> >>>> I'd like to follow-up on this and say that I see this issue now with
> nativesdk build, in particular nativesdk-grpc recipe fails in master with
> the same error. I hope that's partially related to the issue everybody is
> confused about, but I'd like understand how to properly fix it.
> >>>>
> >>>> |   The imported target "protobuf::protoc" references the file
> >>>> |
> >>>> |
> "/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
> >>>> |
> >>>> |   but this file does not exist.  Possible reasons include:
> >>>> |
> >>>> |   * The file was deleted, renamed, or moved to another location.
> >>>>
> >>>> The protoc-23.4.0 file already exists in the recipe-sysroot-native.
> >>>>
> >>>> And we do want protoc to be present in the SDK, but nativesdk build
> is a cross-compilation, because SDKMACHINE and HOST might be different.
> >>>> In this particular grpc case there's an option to say where the
> actual protoc executable is (
> -D_gRPC_PROTOBUF_PROTOC_EXECUTABLE=/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot-native/usr/bin/protoc
> is passed to grpc).
> >>>>
> >>>> So there shouldn't be any issues here, but nativesdk build still
> fails.
> >>>>
> >>>> Looking into it further I see that protoc is actually packaged by
> protobuf-compiler package, but when I try to add a package dependency to
> grpc recipe I get this:
> >>>>
> >>>> nativesdk-protobuf RPROVIDES nativesdk-protobuf-compiler
> >>>>
> >>>> Moreover, if I go into devshell and forcefully install
> nativesdk-libprotobuf-compiler, then `bitbake nativesdk-grpc` is happy and
> compiles!
> >>>>
> >>>> So something wrong with the dependencies in protobuf recipe, but I
> can't figure out yet what exactly. Any ideas?
> >>>>
> >>>> Regards,
> >>>> Slava
> >>>>
> >
> >
>

[-- Attachment #2: Type: text/html, Size: 10194 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
       [not found]                           ` <17910E05B5C0179B.11247@lists.openembedded.org>
@ 2023-11-08 13:30                             ` Samuli Piippo
  2023-11-09  0:13                               ` Clément Péron
  0 siblings, 1 reply; 35+ messages in thread
From: Samuli Piippo @ 2023-11-08 13:30 UTC (permalink / raw)
  To: samuli.piippo
  Cc: Clément Péron, Vyacheslav Yurkov, Martin Jansa,
	Khem Raj, Ross Burton, openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 8618 bytes --]

Hi Clément,

Were you able to reproduce the issue?

On Tue, 24 Oct 2023 at 16:33, Samuli Piippo via lists.openembedded.org
<samuli.piippo=gmail.com@lists.openembedded.org> wrote:

> The protoc binary is not enabled for target builds. Try nativesdk build
> and it should then fail to find it.
>
> On Tue, 24 Oct 2023 at 14:53, Clément Péron <peron.clem@gmail.com> wrote:
>
>> Hi,
>>
>> I have pushed a branch on my repo:
>> https://github.com/clementperon/meta-openembedded/tree/test_protobuf
>>
>> and tried to reproduce the issue on my setup with the following kas yaml
>> file.
>>
>> But I don't see any issue at the configure stage.
>>
>> cat XXXX/cortexa57-poky-linux-musl/test-protobuf/0.3/temp/log.do_configure
>>
>> Show that -- Version:23.4.0 of Protobuf is found without any issue
>>
>> Cmake host version is
>> $> cmake --version
>> cmake version 3.27.7
>>
>> I'm missing something ?
>>
>> Regards,
>> Clement
>>
>> kas.yml
>> ====================
>> header:
>>   version: 8
>>
>> machine: qemuarm64
>> distro: poky
>> target:
>>   - test-protobuf
>>
>> repos:
>>   poky:
>>     url: https://git.yoctoproject.org/git/poky
>>     path: layers/poky
>>     refspec: master
>>     layers:
>>       meta:
>>       meta-poky:
>>       meta-yocto-bsp:
>>
>>   meta-openembedded:
>>     url: https://github.com/clementperon/meta-openembedded
>>     path: layers/meta-openembedded
>>     refspec: test_protobuf
>>     layers:
>>       meta-oe:
>>       meta-networking:
>>       meta-python:
>>
>> bblayers_conf_header:
>>   standard: |
>>     POKY_BBLAYERS_CONF_VERSION = "2"
>>     BBPATH = "${TOPDIR}"
>>     BBFILES ?= ""
>>
>> local_conf_header:
>>   libc: |
>>     TCLIBC = "musl"
>>   standard: |
>>     CONF_VERSION = "2"
>>     PACKAGE_CLASSES = "package_ipk"
>>   debug-tweaks: |
>>     EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
>>
>>   diskmon: |
>>     BB_DISKMON_DIRS = "\
>>         STOPTASKS,${TMPDIR},1G,100K \
>>         STOPTASKS,${DL_DIR},1G,100K \
>>         STOPTASKS,${SSTATE_DIR},1G,100K \
>>         STOPTASKS,/tmp,100M,100K \
>>         ABORT,${TMPDIR},100M,1K \
>>         ABORT,${DL_DIR},100M,1K \
>>         ABORT,${SSTATE_DIR},100M,1K \
>>         ABORT,/tmp,10M,1K"
>> ======================
>>
>> On Sun, 22 Oct 2023 at 20:45, Vyacheslav Yurkov <uvv.mail@gmail.com>
>> wrote:
>> >
>> > I'm a bit confused here.
>> >
>> > I just did 'find
>> tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/ -name
>> "protoc*"' and I see protoc binary only in recipe-sysroot-native. That
>> tells me that protoc, which is a part of nativesdk-protobuf-compiler, is
>> not installed to recipe-sysroot.
>> >
>> > What am I missing here?
>> >
>> > Slava
>> >
>> > On 20.10.2023 09:35, Martin Jansa wrote:
>> >
>> > do_prepare_recipe_sysroot doesn't install any packages to RSS. Only
>> package manager on target or in do_rootfs uses packages.
>> >
>> > It installs the files from dependency populate_sysroot task.
>> >
>> > On Fri, Oct 20, 2023 at 9:11 AM Samuli Piippo <samuli.piippo@gmail.com>
>> wrote:
>> >>
>> >> Dependency to the recipe name will always install all packages to RSS.
>> >> You can verify this by checking that the libprotoc.so from the
>> nativesdk-protobuf-compile package is in the sysroot.
>> >> The problem you are facing is the fact that binaries are not installed
>> into the sysroot (for nativesdk builds), so you are only missing the
>> bin/protoc-23.4.0 file.
>> >> The proposed and twice reverted fix for this is the aforementioned
>> SYSROOT_DIRS.
>> >>
>> >> -samuli
>> >>
>> >> On Fri, 20 Oct 2023 at 09:17, Vyacheslav Yurkov <uvv.mail@gmail.com>
>> wrote:
>> >>>
>> >>> I think the problem goes down to how dependencies are populated in
>> the sysroot. I raised this question in oe-core mailing list
>> https://lists.openembedded.org/g/openembedded-core/message/189324 , but
>> didn't get an answer yet.
>> >>>
>> >>> DEPENDS += "protobuf" will only install the main package of protobuf
>> to the sysroot, because it assumes that nativesdk-protobuf-compiler is also
>> part of the main package (which is not). I guess the part of the system
>> populating sysroot deals with recipes only, and not with packages. I'm
>> still trying to understand how that should be addressed properly.
>> >>> My understanding is that either all packages should be installed to
>> sysroot (which recipe claims it provides), or we need a way to indicate a
>> package level dependency (i.e. nativesdk-protobuf-compiler in this case)
>> >>>
>> >>> Any ideas?
>> >>>
>> >>> Slava
>> >>>
>> >>> On 18.10.2023 14:28, Samuli Piippo wrote:
>> >>>
>> >>> Following simple test recipe will fail now when trying to use
>> Protobuf with CMake.
>> >>>
>> >>> inherit cmake
>> >>> DEPENDS += "protobuf"
>> >>>
>> >>> do_configure:prepend() {
>> >>>     echo "find_package(Protobuf CONFIG)" > ${S}/CMakeLists.txt
>> >>> }
>> >>>
>> >>> BBCLASSEXTEND = "nativesdk"
>> >>>
>> >>> ---
>> >>> CMake Error at
>> tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake:107
>> (message):
>> >>>   The imported target "protobuf::protoc" references the file
>> >>>
>> >>>
>> "tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>> >>>
>> >>>   but this file does not exist.
>> >>> ---
>> >>>
>> >>> I don't quite see why every project/recipe should fix this
>> independently when the simple workaround for this yocto-specific issue is
>> available (namely the use of SYSROOT_DIRS).
>> >>>
>> >>> -samuli
>> >>>
>> >>> ps. qtgrpc recipe and other Qt6 recipes are now visible in the
>> layerindex https://layers.openembedded.org/layerindex/recipe/348210/
>> >>>
>> >>>
>> >>> On Thu, 12 Oct 2023 at 15:06, Vyacheslav Yurkov <uvv.mail@gmail.com>
>> wrote:
>> >>>>
>> >>>> I'd like to follow-up on this and say that I see this issue now with
>> nativesdk build, in particular nativesdk-grpc recipe fails in master with
>> the same error. I hope that's partially related to the issue everybody is
>> confused about, but I'd like understand how to properly fix it.
>> >>>>
>> >>>> |   The imported target "protobuf::protoc" references the file
>> >>>> |
>> >>>> |
>> "/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>> >>>> |
>> >>>> |   but this file does not exist.  Possible reasons include:
>> >>>> |
>> >>>> |   * The file was deleted, renamed, or moved to another location.
>> >>>>
>> >>>> The protoc-23.4.0 file already exists in the recipe-sysroot-native.
>> >>>>
>> >>>> And we do want protoc to be present in the SDK, but nativesdk build
>> is a cross-compilation, because SDKMACHINE and HOST might be different.
>> >>>> In this particular grpc case there's an option to say where the
>> actual protoc executable is (
>> -D_gRPC_PROTOBUF_PROTOC_EXECUTABLE=/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot-native/usr/bin/protoc
>> is passed to grpc).
>> >>>>
>> >>>> So there shouldn't be any issues here, but nativesdk build still
>> fails.
>> >>>>
>> >>>> Looking into it further I see that protoc is actually packaged by
>> protobuf-compiler package, but when I try to add a package dependency to
>> grpc recipe I get this:
>> >>>>
>> >>>> nativesdk-protobuf RPROVIDES nativesdk-protobuf-compiler
>> >>>>
>> >>>> Moreover, if I go into devshell and forcefully install
>> nativesdk-libprotobuf-compiler, then `bitbake nativesdk-grpc` is happy and
>> compiles!
>> >>>>
>> >>>> So something wrong with the dependencies in protobuf recipe, but I
>> can't figure out yet what exactly. Any ideas?
>> >>>>
>> >>>> Regards,
>> >>>> Slava
>> >>>>
>> >
>> >
>>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#105722):
> https://lists.openembedded.org/g/openembedded-devel/message/105722
> Mute This Topic: https://lists.openembedded.org/mt/101679410/3617605
> Group Owner: openembedded-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [
> samuli.piippo@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 11732 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-11-08 13:30                             ` Samuli Piippo
@ 2023-11-09  0:13                               ` Clément Péron
  2023-11-09 10:19                                 ` Vyacheslav Yurkov
  0 siblings, 1 reply; 35+ messages in thread
From: Clément Péron @ 2023-11-09  0:13 UTC (permalink / raw)
  To: Samuli Piippo
  Cc: Vyacheslav Yurkov, Martin Jansa, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 9305 bytes --]

Hi Samuli,

Le mer. 8 nov. 2023, 22:30, Samuli Piippo <samuli.piippo@gmail.com> a
écrit :

> Hi Clément,
>
> Were you able to reproduce the issue?
>

Sorry I'm on vacation without access to a laptop at the moment.

But if I understand the issue is not related to a target or native recipes
issue, right?

It's only when building the SDK and then try to build an application with
it right?

Maybe there is a missing dependency when building the SDK.

Regards,
Clement



> On Tue, 24 Oct 2023 at 16:33, Samuli Piippo via lists.openembedded.org
> <samuli.piippo=gmail.com@lists.openembedded.org> wrote:
>
>> The protoc binary is not enabled for target builds. Try nativesdk build
>> and it should then fail to find it.
>>
>> On Tue, 24 Oct 2023 at 14:53, Clément Péron <peron.clem@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I have pushed a branch on my repo:
>>> https://github.com/clementperon/meta-openembedded/tree/test_protobuf
>>>
>>> and tried to reproduce the issue on my setup with the following kas yaml
>>> file.
>>>
>>> But I don't see any issue at the configure stage.
>>>
>>> cat
>>> XXXX/cortexa57-poky-linux-musl/test-protobuf/0.3/temp/log.do_configure
>>>
>>> Show that -- Version:23.4.0 of Protobuf is found without any issue
>>>
>>> Cmake host version is
>>> $> cmake --version
>>> cmake version 3.27.7
>>>
>>> I'm missing something ?
>>>
>>> Regards,
>>> Clement
>>>
>>> kas.yml
>>> ====================
>>> header:
>>>   version: 8
>>>
>>> machine: qemuarm64
>>> distro: poky
>>> target:
>>>   - test-protobuf
>>>
>>> repos:
>>>   poky:
>>>     url: https://git.yoctoproject.org/git/poky
>>>     path: layers/poky
>>>     refspec: master
>>>     layers:
>>>       meta:
>>>       meta-poky:
>>>       meta-yocto-bsp:
>>>
>>>   meta-openembedded:
>>>     url: https://github.com/clementperon/meta-openembedded
>>>     path: layers/meta-openembedded
>>>     refspec: test_protobuf
>>>     layers:
>>>       meta-oe:
>>>       meta-networking:
>>>       meta-python:
>>>
>>> bblayers_conf_header:
>>>   standard: |
>>>     POKY_BBLAYERS_CONF_VERSION = "2"
>>>     BBPATH = "${TOPDIR}"
>>>     BBFILES ?= ""
>>>
>>> local_conf_header:
>>>   libc: |
>>>     TCLIBC = "musl"
>>>   standard: |
>>>     CONF_VERSION = "2"
>>>     PACKAGE_CLASSES = "package_ipk"
>>>   debug-tweaks: |
>>>     EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
>>>
>>>   diskmon: |
>>>     BB_DISKMON_DIRS = "\
>>>         STOPTASKS,${TMPDIR},1G,100K \
>>>         STOPTASKS,${DL_DIR},1G,100K \
>>>         STOPTASKS,${SSTATE_DIR},1G,100K \
>>>         STOPTASKS,/tmp,100M,100K \
>>>         ABORT,${TMPDIR},100M,1K \
>>>         ABORT,${DL_DIR},100M,1K \
>>>         ABORT,${SSTATE_DIR},100M,1K \
>>>         ABORT,/tmp,10M,1K"
>>> ======================
>>>
>>> On Sun, 22 Oct 2023 at 20:45, Vyacheslav Yurkov <uvv.mail@gmail.com>
>>> wrote:
>>> >
>>> > I'm a bit confused here.
>>> >
>>> > I just did 'find
>>> tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/ -name
>>> "protoc*"' and I see protoc binary only in recipe-sysroot-native. That
>>> tells me that protoc, which is a part of nativesdk-protobuf-compiler, is
>>> not installed to recipe-sysroot.
>>> >
>>> > What am I missing here?
>>> >
>>> > Slava
>>> >
>>> > On 20.10.2023 09:35, Martin Jansa wrote:
>>> >
>>> > do_prepare_recipe_sysroot doesn't install any packages to RSS. Only
>>> package manager on target or in do_rootfs uses packages.
>>> >
>>> > It installs the files from dependency populate_sysroot task.
>>> >
>>> > On Fri, Oct 20, 2023 at 9:11 AM Samuli Piippo <samuli.piippo@gmail.com>
>>> wrote:
>>> >>
>>> >> Dependency to the recipe name will always install all packages to RSS.
>>> >> You can verify this by checking that the libprotoc.so from the
>>> nativesdk-protobuf-compile package is in the sysroot.
>>> >> The problem you are facing is the fact that binaries are not
>>> installed into the sysroot (for nativesdk builds), so you are only missing
>>> the bin/protoc-23.4.0 file.
>>> >> The proposed and twice reverted fix for this is the aforementioned
>>> SYSROOT_DIRS.
>>> >>
>>> >> -samuli
>>> >>
>>> >> On Fri, 20 Oct 2023 at 09:17, Vyacheslav Yurkov <uvv.mail@gmail.com>
>>> wrote:
>>> >>>
>>> >>> I think the problem goes down to how dependencies are populated in
>>> the sysroot. I raised this question in oe-core mailing list
>>> https://lists.openembedded.org/g/openembedded-core/message/189324 , but
>>> didn't get an answer yet.
>>> >>>
>>> >>> DEPENDS += "protobuf" will only install the main package of protobuf
>>> to the sysroot, because it assumes that nativesdk-protobuf-compiler is also
>>> part of the main package (which is not). I guess the part of the system
>>> populating sysroot deals with recipes only, and not with packages. I'm
>>> still trying to understand how that should be addressed properly.
>>> >>> My understanding is that either all packages should be installed to
>>> sysroot (which recipe claims it provides), or we need a way to indicate a
>>> package level dependency (i.e. nativesdk-protobuf-compiler in this case)
>>> >>>
>>> >>> Any ideas?
>>> >>>
>>> >>> Slava
>>> >>>
>>> >>> On 18.10.2023 14:28, Samuli Piippo wrote:
>>> >>>
>>> >>> Following simple test recipe will fail now when trying to use
>>> Protobuf with CMake.
>>> >>>
>>> >>> inherit cmake
>>> >>> DEPENDS += "protobuf"
>>> >>>
>>> >>> do_configure:prepend() {
>>> >>>     echo "find_package(Protobuf CONFIG)" > ${S}/CMakeLists.txt
>>> >>> }
>>> >>>
>>> >>> BBCLASSEXTEND = "nativesdk"
>>> >>>
>>> >>> ---
>>> >>> CMake Error at
>>> tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake:107
>>> (message):
>>> >>>   The imported target "protobuf::protoc" references the file
>>> >>>
>>> >>>
>>> "tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-test/1.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>>> >>>
>>> >>>   but this file does not exist.
>>> >>> ---
>>> >>>
>>> >>> I don't quite see why every project/recipe should fix this
>>> independently when the simple workaround for this yocto-specific issue is
>>> available (namely the use of SYSROOT_DIRS).
>>> >>>
>>> >>> -samuli
>>> >>>
>>> >>> ps. qtgrpc recipe and other Qt6 recipes are now visible in the
>>> layerindex https://layers.openembedded.org/layerindex/recipe/348210/
>>> >>>
>>> >>>
>>> >>> On Thu, 12 Oct 2023 at 15:06, Vyacheslav Yurkov <uvv.mail@gmail.com>
>>> wrote:
>>> >>>>
>>> >>>> I'd like to follow-up on this and say that I see this issue now
>>> with nativesdk build, in particular nativesdk-grpc recipe fails in master
>>> with the same error. I hope that's partially related to the issue everybody
>>> is confused about, but I'd like understand how to properly fix it.
>>> >>>>
>>> >>>> |   The imported target "protobuf::protoc" references the file
>>> >>>> |
>>> >>>> |
>>> "/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-23.4.0"
>>> >>>> |
>>> >>>> |   but this file does not exist.  Possible reasons include:
>>> >>>> |
>>> >>>> |   * The file was deleted, renamed, or moved to another location.
>>> >>>>
>>> >>>> The protoc-23.4.0 file already exists in the recipe-sysroot-native.
>>> >>>>
>>> >>>> And we do want protoc to be present in the SDK, but nativesdk build
>>> is a cross-compilation, because SDKMACHINE and HOST might be different.
>>> >>>> In this particular grpc case there's an option to say where the
>>> actual protoc executable is (
>>> -D_gRPC_PROTOBUF_PROTOC_EXECUTABLE=/home/uvv/projects/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-grpc/1.56.2/recipe-sysroot-native/usr/bin/protoc
>>> is passed to grpc).
>>> >>>>
>>> >>>> So there shouldn't be any issues here, but nativesdk build still
>>> fails.
>>> >>>>
>>> >>>> Looking into it further I see that protoc is actually packaged by
>>> protobuf-compiler package, but when I try to add a package dependency to
>>> grpc recipe I get this:
>>> >>>>
>>> >>>> nativesdk-protobuf RPROVIDES nativesdk-protobuf-compiler
>>> >>>>
>>> >>>> Moreover, if I go into devshell and forcefully install
>>> nativesdk-libprotobuf-compiler, then `bitbake nativesdk-grpc` is happy and
>>> compiles!
>>> >>>>
>>> >>>> So something wrong with the dependencies in protobuf recipe, but I
>>> can't figure out yet what exactly. Any ideas?
>>> >>>>
>>> >>>> Regards,
>>> >>>> Slava
>>> >>>>
>>> >
>>> >
>>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#105722):
>> https://lists.openembedded.org/g/openembedded-devel/message/105722
>> Mute This Topic: https://lists.openembedded.org/mt/101679410/3617605
>> Group Owner: openembedded-devel+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [
>> samuli.piippo@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>>

[-- Attachment #2: Type: text/html, Size: 13412 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-11-09  0:13                               ` Clément Péron
@ 2023-11-09 10:19                                 ` Vyacheslav Yurkov
  2023-11-09 13:35                                   ` Samuli Piippo
  0 siblings, 1 reply; 35+ messages in thread
From: Vyacheslav Yurkov @ 2023-11-09 10:19 UTC (permalink / raw)
  To: Clément Péron, Samuli Piippo
  Cc: Martin Jansa, Khem Raj, Ross Burton, openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

Hi Clément,
No, the issue is when you just build nativesdk recipe (we are not at the 
stage to use the SDK yet).

And that was exactly my question how to propagate the dependency 
correctly for nativesdk build.

Regards,
Slava

On 09.11.2023 01:13, Clément Péron wrote:
> Hi Samuli,
>
> Le mer. 8 nov. 2023, 22:30, Samuli Piippo <samuli.piippo@gmail.com> a 
> écrit :
>
>     Hi Clément,
>
>     Were you able to reproduce the issue?
>
>
> Sorry I'm on vacation without access to a laptop at the moment.
>
> But if I understand the issue is not related to a target or native 
> recipes issue, right?
>
> It's only when building the SDK and then try to build an application 
> with it right?
>
> Maybe there is a missing dependency when building the SDK.
>
> Regards,
> Clement
>

[-- Attachment #2: Type: text/html, Size: 2477 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-11-09 10:19                                 ` Vyacheslav Yurkov
@ 2023-11-09 13:35                                   ` Samuli Piippo
  2023-11-09 15:05                                     ` Vyacheslav Yurkov
  0 siblings, 1 reply; 35+ messages in thread
From: Samuli Piippo @ 2023-11-09 13:35 UTC (permalink / raw)
  To: Vyacheslav Yurkov
  Cc: Clément Péron, Martin Jansa, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 1245 bytes --]

DEPENDS += "protobuf" is enough to have dependencies resolved.

Problem is that bitbake's do_populate_sysroot does not take anything from
bindir.
This leads to nativesdk-(qt)grpc's do_prepare_recipe_sysroot to not get the
protoc binary into the recipe_sysroot as expected by CMake.

This can be fixed with SYSROOT_DIRS.

On Thu, 9 Nov 2023 at 12:19, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:

> Hi Clément,
> No, the issue is when you just build nativesdk recipe (we are not at the
> stage to use the SDK yet).
>
> And that was exactly my question how to propagate the dependency correctly
> for nativesdk build.
>
> Regards,
> Slava
>
> On 09.11.2023 01:13, Clément Péron wrote:
>
> Hi Samuli,
>
> Le mer. 8 nov. 2023, 22:30, Samuli Piippo <samuli.piippo@gmail.com> a
> écrit :
>
>> Hi Clément,
>>
>> Were you able to reproduce the issue?
>>
>
> Sorry I'm on vacation without access to a laptop at the moment.
>
> But if I understand the issue is not related to a target or native recipes
> issue, right?
>
> It's only when building the SDK and then try to build an application with
> it right?
>
> Maybe there is a missing dependency when building the SDK.
>
> Regards,
> Clement
>
>
>

[-- Attachment #2: Type: text/html, Size: 2968 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-11-09 13:35                                   ` Samuli Piippo
@ 2023-11-09 15:05                                     ` Vyacheslav Yurkov
  2023-11-10  1:23                                       ` Clément Péron
  0 siblings, 1 reply; 35+ messages in thread
From: Vyacheslav Yurkov @ 2023-11-09 15:05 UTC (permalink / raw)
  To: Samuli Piippo
  Cc: Clément Péron, Martin Jansa, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 1828 bytes --]

Thanks for the explanation, that was the missing piece for me to get an 
understanding of the problem :)

I think it's not only me, but also others, were confused by the fact 
that we are talking about native(sdk), but try to fix something with a 
non-native variable.

I've just gave it a try and have two suggestions how this can be addressed.

1. Recipe based solution:
SYSROOT_DIRS:append:class-nativesdk = " $bindir}"

2. oe-core based solution:
--- a/meta/classes-global/staging.bbclass
+++ b/meta/classes-global/staging.bbclass
@@ -26,6 +26,7 @@ SYSROOT_DIRS_NATIVE = " \
      ${localstatedir} \
  "
  SYSROOT_DIRS:append:class-native = " ${SYSROOT_DIRS_NATIVE}"
+SYSROOT_DIRS:append:class-nativesdk = " ${SYSROOT_DIRS_NATIVE}"
  SYSROOT_DIRS:append:class-cross = " ${SYSROOT_DIRS_NATIVE}"
  SYSROOT_DIRS:append:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"

The second one will invalidate nativesdk sstate, but might cover not 
only grpc issue (I would assume gdbus-codegen or any other generator has 
the same issue when used for SDK build).

What are your thoughs?

Thanks,
Slava

On 09.11.2023 14:35, Samuli Piippo wrote:
> DEPENDS += "protobuf" is enough to have dependencies resolved.
>
> Problem is that bitbake's do_populate_sysroot does not take anything 
> from bindir.
> This leads to nativesdk-(qt)grpc's do_prepare_recipe_sysroot to not 
> get the protoc binary into the recipe_sysroot as expected by CMake.
>
> This can be fixed with SYSROOT_DIRS.
>
> On Thu, 9 Nov 2023 at 12:19, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:
>
>     Hi Clément,
>     No, the issue is when you just build nativesdk recipe (we are not
>     at the stage to use the SDK yet).
>
>     And that was exactly my question how to propagate the dependency
>     correctly for nativesdk build.
>
>     Regards,
>     Slava
>

[-- Attachment #2: Type: text/html, Size: 3334 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-11-09 15:05                                     ` Vyacheslav Yurkov
@ 2023-11-10  1:23                                       ` Clément Péron
  2023-11-10  6:19                                         ` Vyacheslav Yurkov
       [not found]                                         ` <17962EA21B11FB58.7380@lists.openembedded.org>
  0 siblings, 2 replies; 35+ messages in thread
From: Clément Péron @ 2023-11-10  1:23 UTC (permalink / raw)
  To: Vyacheslav Yurkov
  Cc: Samuli Piippo, Martin Jansa, Khem Raj, Ross Burton, openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 2137 bytes --]

Hi Samuli,

Le ven. 10 nov. 2023, 00:05, Vyacheslav Yurkov <uvv.mail@gmail.com> a
écrit :

> Thanks for the explanation, that was the missing piece for me to get an
> understanding of the problem :)
>
> I think it's not only me, but also others, were confused by the fact that
> we are talking about native(sdk), but try to fix something with a
> non-native variable.
>
> I've just gave it a try and have two suggestions how this can be addressed.
>
> 1. Recipe based solution:
> SYSROOT_DIRS:append:class-nativesdk = " $bindir}"
>

Is it not just missing something like

DEPENDS:append:class-nativesdk = " protobuf-native "


Regards

Clement




> 2. oe-core based solution:
> --- a/meta/classes-global/staging.bbclass
> +++ b/meta/classes-global/staging.bbclass
> @@ -26,6 +26,7 @@ SYSROOT_DIRS_NATIVE = " \
>      ${localstatedir} \
>  "
>  SYSROOT_DIRS:append:class-native = " ${SYSROOT_DIRS_NATIVE}"
> +SYSROOT_DIRS:append:class-nativesdk = " ${SYSROOT_DIRS_NATIVE}"
>  SYSROOT_DIRS:append:class-cross = " ${SYSROOT_DIRS_NATIVE}"
>  SYSROOT_DIRS:append:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"
>
> The second one will invalidate nativesdk sstate, but might cover not only
> grpc issue (I would assume gdbus-codegen or any other generator has the
> same issue when used for SDK build).
>
> What are your thoughs?
>
> Thanks,
> Slava
>
> On 09.11.2023 14:35, Samuli Piippo wrote:
>
> DEPENDS += "protobuf" is enough to have dependencies resolved.
>
> Problem is that bitbake's do_populate_sysroot does not take anything from
> bindir.
> This leads to nativesdk-(qt)grpc's do_prepare_recipe_sysroot to not get
> the protoc binary into the recipe_sysroot as expected by CMake.
>
> This can be fixed with SYSROOT_DIRS.
>
> On Thu, 9 Nov 2023 at 12:19, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:
>
>> Hi Clément,
>> No, the issue is when you just build nativesdk recipe (we are not at the
>> stage to use the SDK yet).
>>
>> And that was exactly my question how to propagate the dependency
>> correctly for nativesdk build.
>>
>> Regards,
>> Slava
>>
>
>

[-- Attachment #2: Type: text/html, Size: 5386 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-11-10  1:23                                       ` Clément Péron
@ 2023-11-10  6:19                                         ` Vyacheslav Yurkov
       [not found]                                         ` <17962EA21B11FB58.7380@lists.openembedded.org>
  1 sibling, 0 replies; 35+ messages in thread
From: Vyacheslav Yurkov @ 2023-11-10  6:19 UTC (permalink / raw)
  To: Clément Péron
  Cc: Samuli Piippo, Martin Jansa, Khem Raj, Ross Burton, openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 2512 bytes --]

On 10.11.2023 02:23, Clément Péron wrote:
> Hi Samuli,
>
> Le ven. 10 nov. 2023, 00:05, Vyacheslav Yurkov <uvv.mail@gmail.com> a 
> écrit :
>
>     Thanks for the explanation, that was the missing piece for me to
>     get an understanding of the problem :)
>
>     I think it's not only me, but also others, were confused by the
>     fact that we are talking about native(sdk), but try to fix
>     something with a non-native variable.
>
>     I've just gave it a try and have two suggestions how this can be
>     addressed.
>
>     1. Recipe based solution:
>     SYSROOT_DIRS:append:class-nativesdk = " $bindir}"
>
>
> Is it not just missing something like
>
> DEPENDS:append:class-nativesdk= " protobuf-native "
> Regards
> Clement

This is already implicit by DEPENDS += "protobuf". Take a look at my 
second suggestion. You can see that ${bindir} is staged for native 
recipes, but not for nativesdk.

>
>     2. oe-core based solution:
>     --- a/meta/classes-global/staging.bbclass
>     +++ b/meta/classes-global/staging.bbclass
>     @@ -26,6 +26,7 @@ SYSROOT_DIRS_NATIVE = " \
>          ${localstatedir} \
>      "
>      SYSROOT_DIRS:append:class-native = " ${SYSROOT_DIRS_NATIVE}"
>     +SYSROOT_DIRS:append:class-nativesdk = " ${SYSROOT_DIRS_NATIVE}"
>      SYSROOT_DIRS:append:class-cross = " ${SYSROOT_DIRS_NATIVE}"
>      SYSROOT_DIRS:append:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"
>
>     The second one will invalidate nativesdk sstate, but might cover
>     not only grpc issue (I would assume gdbus-codegen or any other
>     generator has the same issue when used for SDK build).
>
>     What are your thoughs?
>
>     Thanks,
>     Slava
>
>     On 09.11.2023 14:35, Samuli Piippo wrote:
>>     DEPENDS += "protobuf" is enough to have dependencies resolved.
>>
>>     Problem is that bitbake's do_populate_sysroot does not take
>>     anything from bindir.
>>     This leads to nativesdk-(qt)grpc's do_prepare_recipe_sysroot to
>>     not get the protoc binary into the recipe_sysroot as expected by
>>     CMake.
>>
>>     This can be fixed with SYSROOT_DIRS.
>>
>>     On Thu, 9 Nov 2023 at 12:19, Vyacheslav Yurkov
>>     <uvv.mail@gmail.com> wrote:
>>
>>         Hi Clément,
>>         No, the issue is when you just build nativesdk recipe (we are
>>         not at the stage to use the SDK yet).
>>
>>         And that was exactly my question how to propagate the
>>         dependency correctly for nativesdk build.
>>
>>         Regards,
>>         Slava
>>
>


[-- Attachment #2: Type: text/html, Size: 7902 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
       [not found]                                         ` <17962EA21B11FB58.7380@lists.openembedded.org>
@ 2023-11-10  6:40                                           ` Vyacheslav Yurkov
  2023-11-10  7:12                                             ` Samuli Piippo
  0 siblings, 1 reply; 35+ messages in thread
From: Vyacheslav Yurkov @ 2023-11-10  6:40 UTC (permalink / raw)
  To: Clément Péron
  Cc: Samuli Piippo, Martin Jansa, Khem Raj, Ross Burton, openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 2741 bytes --]

FTR, I suggested oe-core solution in the corresponding mailing list.

Slava

On 10.11.2023 07:19, Vyacheslav Yurkov via lists.openembedded.org wrote:
> On 10.11.2023 02:23, Clément Péron wrote:
>> Hi Samuli,
>>
>> Le ven. 10 nov. 2023, 00:05, Vyacheslav Yurkov <uvv.mail@gmail.com> a 
>> écrit :
>>
>>     Thanks for the explanation, that was the missing piece for me to
>>     get an understanding of the problem :)
>>
>>     I think it's not only me, but also others, were confused by the
>>     fact that we are talking about native(sdk), but try to fix
>>     something with a non-native variable.
>>
>>     I've just gave it a try and have two suggestions how this can be
>>     addressed.
>>
>>     1. Recipe based solution:
>>     SYSROOT_DIRS:append:class-nativesdk = " $bindir}"
>>
>>
>> Is it not just missing something like
>>
>> DEPENDS:append:class-nativesdk= " protobuf-native "
>> Regards
>> Clement
>
> This is already implicit by DEPENDS += "protobuf". Take a look at my 
> second suggestion. You can see that ${bindir} is staged for native 
> recipes, but not for nativesdk.
>
>>
>>     2. oe-core based solution:
>>     --- a/meta/classes-global/staging.bbclass
>>     +++ b/meta/classes-global/staging.bbclass
>>     @@ -26,6 +26,7 @@ SYSROOT_DIRS_NATIVE = " \
>>          ${localstatedir} \
>>      "
>>      SYSROOT_DIRS:append:class-native = " ${SYSROOT_DIRS_NATIVE}"
>>     +SYSROOT_DIRS:append:class-nativesdk = " ${SYSROOT_DIRS_NATIVE}"
>>      SYSROOT_DIRS:append:class-cross = " ${SYSROOT_DIRS_NATIVE}"
>>      SYSROOT_DIRS:append:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"
>>
>>     The second one will invalidate nativesdk sstate, but might cover
>>     not only grpc issue (I would assume gdbus-codegen or any other
>>     generator has the same issue when used for SDK build).
>>
>>     What are your thoughs?
>>
>>     Thanks,
>>     Slava
>>
>>     On 09.11.2023 14:35, Samuli Piippo wrote:
>>>     DEPENDS += "protobuf" is enough to have dependencies resolved.
>>>
>>>     Problem is that bitbake's do_populate_sysroot does not take
>>>     anything from bindir.
>>>     This leads to nativesdk-(qt)grpc's do_prepare_recipe_sysroot to
>>>     not get the protoc binary into the recipe_sysroot as expected by
>>>     CMake.
>>>
>>>     This can be fixed with SYSROOT_DIRS.
>>>
>>>     On Thu, 9 Nov 2023 at 12:19, Vyacheslav Yurkov
>>>     <uvv.mail@gmail.com> wrote:
>>>
>>>         Hi Clément,
>>>         No, the issue is when you just build nativesdk recipe (we
>>>         are not at the stage to use the SDK yet).
>>>
>>>         And that was exactly my question how to propagate the
>>>         dependency correctly for nativesdk build.
>>>
>>>         Regards,
>>>         Slava
>>>
>>

[-- Attachment #2: Type: text/html, Size: 8690 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-11-10  6:40                                           ` Vyacheslav Yurkov
@ 2023-11-10  7:12                                             ` Samuli Piippo
  2023-11-10  7:38                                               ` Vyacheslav Yurkov
  0 siblings, 1 reply; 35+ messages in thread
From: Samuli Piippo @ 2023-11-10  7:12 UTC (permalink / raw)
  To: Vyacheslav Yurkov
  Cc: Clément Péron, Martin Jansa, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 2927 bytes --]

This is not strictly a nativesdk issue.
If you enable PACKAGECONFIG = "compiler" for protobuf, it will then build
protoc for the target.
This will then cause grpc target build to fail in the same way as nativesdk
build.

On Fri, 10 Nov 2023 at 08:40, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:

> FTR, I suggested oe-core solution in the corresponding mailing list.
>
> Slava
>
> On 10.11.2023 07:19, Vyacheslav Yurkov via lists.openembedded.org wrote:
>
> On 10.11.2023 02:23, Clément Péron wrote:
>
> Hi Samuli,
>
> Le ven. 10 nov. 2023, 00:05, Vyacheslav Yurkov <uvv.mail@gmail.com> a
> écrit :
>
>> Thanks for the explanation, that was the missing piece for me to get an
>> understanding of the problem :)
>>
>> I think it's not only me, but also others, were confused by the fact that
>> we are talking about native(sdk), but try to fix something with a
>> non-native variable.
>>
>> I've just gave it a try and have two suggestions how this can be
>> addressed.
>>
>> 1. Recipe based solution:
>> SYSROOT_DIRS:append:class-nativesdk = " $bindir}"
>>
>
> Is it not just missing something like
>
> DEPENDS:append:class-nativesdk = " protobuf-native "
>
>  Regards
>
> Clement
>
>
> This is already implicit by DEPENDS += "protobuf". Take a look at my
> second suggestion. You can see that ${bindir} is staged for native recipes,
> but not for nativesdk.
>
>
>> 2. oe-core based solution:
>> --- a/meta/classes-global/staging.bbclass
>> +++ b/meta/classes-global/staging.bbclass
>> @@ -26,6 +26,7 @@ SYSROOT_DIRS_NATIVE = " \
>>      ${localstatedir} \
>>  "
>>  SYSROOT_DIRS:append:class-native = " ${SYSROOT_DIRS_NATIVE}"
>> +SYSROOT_DIRS:append:class-nativesdk = " ${SYSROOT_DIRS_NATIVE}"
>>  SYSROOT_DIRS:append:class-cross = " ${SYSROOT_DIRS_NATIVE}"
>>  SYSROOT_DIRS:append:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"
>>
>> The second one will invalidate nativesdk sstate, but might cover not only
>> grpc issue (I would assume gdbus-codegen or any other generator has the
>> same issue when used for SDK build).
>>
>> What are your thoughs?
>>
>> Thanks,
>> Slava
>>
>> On 09.11.2023 14:35, Samuli Piippo wrote:
>>
>> DEPENDS += "protobuf" is enough to have dependencies resolved.
>>
>> Problem is that bitbake's do_populate_sysroot does not take anything from
>> bindir.
>> This leads to nativesdk-(qt)grpc's do_prepare_recipe_sysroot to not get
>> the protoc binary into the recipe_sysroot as expected by CMake.
>>
>> This can be fixed with SYSROOT_DIRS.
>>
>> On Thu, 9 Nov 2023 at 12:19, Vyacheslav Yurkov <uvv.mail@gmail.com>
>> wrote:
>>
>>> Hi Clément,
>>> No, the issue is when you just build nativesdk recipe (we are not at the
>>> stage to use the SDK yet).
>>>
>>> And that was exactly my question how to propagate the dependency
>>> correctly for nativesdk build.
>>>
>>> Regards,
>>> Slava
>>>
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 8742 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-11-10  7:12                                             ` Samuli Piippo
@ 2023-11-10  7:38                                               ` Vyacheslav Yurkov
  2023-11-10  7:50                                                 ` Samuli Piippo
  0 siblings, 1 reply; 35+ messages in thread
From: Vyacheslav Yurkov @ 2023-11-10  7:38 UTC (permalink / raw)
  To: Samuli Piippo
  Cc: Clément Péron, Martin Jansa, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 3547 bytes --]

You sure? Because I remember the test in grpc to check for 
CROSS_COMPILING in cmake to avoid that.

Slava

On 10.11.2023 08:12, Samuli Piippo wrote:
> This is not strictly a nativesdk issue.
> If you enable PACKAGECONFIG = "compiler" for protobuf, it will then 
> build protoc for the target.
> This will then cause grpc target build to fail in the same way as 
> nativesdk build.
>
> On Fri, 10 Nov 2023 at 08:40, Vyacheslav Yurkov <uvv.mail@gmail.com> 
> wrote:
>
>     FTR, I suggested oe-core solution in the corresponding mailing list.
>
>     Slava
>
>     On 10.11.2023 07:19, Vyacheslav Yurkov via lists.openembedded.org
>     <http://lists.openembedded.org> wrote:
>>     On 10.11.2023 02:23, Clément Péron wrote:
>>>     Hi Samuli,
>>>
>>>     Le ven. 10 nov. 2023, 00:05, Vyacheslav Yurkov
>>>     <uvv.mail@gmail.com> a écrit :
>>>
>>>         Thanks for the explanation, that was the missing piece for
>>>         me to get an understanding of the problem :)
>>>
>>>         I think it's not only me, but also others, were confused by
>>>         the fact that we are talking about native(sdk), but try to
>>>         fix something with a non-native variable.
>>>
>>>         I've just gave it a try and have two suggestions how this
>>>         can be addressed.
>>>
>>>         1. Recipe based solution:
>>>         SYSROOT_DIRS:append:class-nativesdk = " $bindir}"
>>>
>>>
>>>     Is it not just missing something like
>>>
>>>     DEPENDS:append:class-nativesdk= " protobuf-native "
>>>     Regards
>>>     Clement
>>
>>     This is already implicit by DEPENDS += "protobuf". Take a look at
>>     my second suggestion. You can see that ${bindir} is staged for
>>     native recipes, but not for nativesdk.
>>
>>>
>>>         2. oe-core based solution:
>>>         --- a/meta/classes-global/staging.bbclass
>>>         +++ b/meta/classes-global/staging.bbclass
>>>         @@ -26,6 +26,7 @@ SYSROOT_DIRS_NATIVE = " \
>>>              ${localstatedir} \
>>>          "
>>>          SYSROOT_DIRS:append:class-native = " ${SYSROOT_DIRS_NATIVE}"
>>>         +SYSROOT_DIRS:append:class-nativesdk = " ${SYSROOT_DIRS_NATIVE}"
>>>          SYSROOT_DIRS:append:class-cross = " ${SYSROOT_DIRS_NATIVE}"
>>>          SYSROOT_DIRS:append:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"
>>>
>>>         The second one will invalidate nativesdk sstate, but might
>>>         cover not only grpc issue (I would assume gdbus-codegen or
>>>         any other generator has the same issue when used for SDK build).
>>>
>>>         What are your thoughs?
>>>
>>>         Thanks,
>>>         Slava
>>>
>>>         On 09.11.2023 14:35, Samuli Piippo wrote:
>>>>         DEPENDS += "protobuf" is enough to have dependencies resolved.
>>>>
>>>>         Problem is that bitbake's do_populate_sysroot does not take
>>>>         anything from bindir.
>>>>         This leads to nativesdk-(qt)grpc's
>>>>         do_prepare_recipe_sysroot to not get the protoc binary into
>>>>         the recipe_sysroot as expected by CMake.
>>>>
>>>>         This can be fixed with SYSROOT_DIRS.
>>>>
>>>>         On Thu, 9 Nov 2023 at 12:19, Vyacheslav Yurkov
>>>>         <uvv.mail@gmail.com> wrote:
>>>>
>>>>             Hi Clément,
>>>>             No, the issue is when you just build nativesdk recipe
>>>>             (we are not at the stage to use the SDK yet).
>>>>
>>>>             And that was exactly my question how to propagate the
>>>>             dependency correctly for nativesdk build.
>>>>
>>>>             Regards,
>>>>             Slava
>>>>

[-- Attachment #2: Type: text/html, Size: 11140 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-11-10  7:38                                               ` Vyacheslav Yurkov
@ 2023-11-10  7:50                                                 ` Samuli Piippo
  2023-11-10  8:05                                                   ` Vyacheslav Yurkov
  0 siblings, 1 reply; 35+ messages in thread
From: Samuli Piippo @ 2023-11-10  7:50 UTC (permalink / raw)
  To: Vyacheslav Yurkov
  Cc: Clément Péron, Martin Jansa, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 3685 bytes --]

| CMake Error at
/home/sapiippo/qt6/build/tmp/work/cortexa57-poky-linux/grpc/1.59.2/recipe-sysroot/usr/lib/cmake/protobuf/protobuf-targets.cmake:116
(message):
|   The imported target "protobuf::protoc" references the file
|
|
     "/home/sapiippo/qt6/build/tmp/work/cortexa57-poky-linux/grpc/1.59.2/recipe-sysroot/usr/bin/protoc-23.4.0"
|
|   but this file does not exist.  Possible reasons include:

On Fri, 10 Nov 2023 at 09:38, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:

> You sure? Because I remember the test in grpc to check for CROSS_COMPILING
> in cmake to avoid that.
>
> Slava
>
> On 10.11.2023 08:12, Samuli Piippo wrote:
>
> This is not strictly a nativesdk issue.
> If you enable PACKAGECONFIG = "compiler" for protobuf, it will then build
> protoc for the target.
> This will then cause grpc target build to fail in the same way as
> nativesdk build.
>
> On Fri, 10 Nov 2023 at 08:40, Vyacheslav Yurkov <uvv.mail@gmail.com>
> wrote:
>
>> FTR, I suggested oe-core solution in the corresponding mailing list.
>>
>> Slava
>>
>> On 10.11.2023 07:19, Vyacheslav Yurkov via lists.openembedded.org wrote:
>>
>> On 10.11.2023 02:23, Clément Péron wrote:
>>
>> Hi Samuli,
>>
>> Le ven. 10 nov. 2023, 00:05, Vyacheslav Yurkov <uvv.mail@gmail.com> a
>> écrit :
>>
>>> Thanks for the explanation, that was the missing piece for me to get an
>>> understanding of the problem :)
>>>
>>> I think it's not only me, but also others, were confused by the fact
>>> that we are talking about native(sdk), but try to fix something with a
>>> non-native variable.
>>>
>>> I've just gave it a try and have two suggestions how this can be
>>> addressed.
>>>
>>> 1. Recipe based solution:
>>> SYSROOT_DIRS:append:class-nativesdk = " $bindir}"
>>>
>>
>> Is it not just missing something like
>>
>> DEPENDS:append:class-nativesdk = " protobuf-native "
>>
>>  Regards
>>
>> Clement
>>
>>
>> This is already implicit by DEPENDS += "protobuf". Take a look at my
>> second suggestion. You can see that ${bindir} is staged for native recipes,
>> but not for nativesdk.
>>
>>
>>> 2. oe-core based solution:
>>> --- a/meta/classes-global/staging.bbclass
>>> +++ b/meta/classes-global/staging.bbclass
>>> @@ -26,6 +26,7 @@ SYSROOT_DIRS_NATIVE = " \
>>>      ${localstatedir} \
>>>  "
>>>  SYSROOT_DIRS:append:class-native = " ${SYSROOT_DIRS_NATIVE}"
>>> +SYSROOT_DIRS:append:class-nativesdk = " ${SYSROOT_DIRS_NATIVE}"
>>>  SYSROOT_DIRS:append:class-cross = " ${SYSROOT_DIRS_NATIVE}"
>>>  SYSROOT_DIRS:append:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"
>>>
>>> The second one will invalidate nativesdk sstate, but might cover not
>>> only grpc issue (I would assume gdbus-codegen or any other generator has
>>> the same issue when used for SDK build).
>>>
>>> What are your thoughs?
>>>
>>> Thanks,
>>> Slava
>>>
>>> On 09.11.2023 14:35, Samuli Piippo wrote:
>>>
>>> DEPENDS += "protobuf" is enough to have dependencies resolved.
>>>
>>> Problem is that bitbake's do_populate_sysroot does not take anything
>>> from bindir.
>>> This leads to nativesdk-(qt)grpc's do_prepare_recipe_sysroot to not get
>>> the protoc binary into the recipe_sysroot as expected by CMake.
>>>
>>> This can be fixed with SYSROOT_DIRS.
>>>
>>> On Thu, 9 Nov 2023 at 12:19, Vyacheslav Yurkov <uvv.mail@gmail.com>
>>> wrote:
>>>
>>>> Hi Clément,
>>>> No, the issue is when you just build nativesdk recipe (we are not at
>>>> the stage to use the SDK yet).
>>>>
>>>> And that was exactly my question how to propagate the dependency
>>>> correctly for nativesdk build.
>>>>
>>>> Regards,
>>>> Slava
>>>>
>>>
>

[-- Attachment #2: Type: text/html, Size: 11411 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-11-10  7:50                                                 ` Samuli Piippo
@ 2023-11-10  8:05                                                   ` Vyacheslav Yurkov
  2023-11-10  8:44                                                     ` Samuli Piippo
  0 siblings, 1 reply; 35+ messages in thread
From: Vyacheslav Yurkov @ 2023-11-10  8:05 UTC (permalink / raw)
  To: Samuli Piippo
  Cc: Clément Péron, Martin Jansa, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 4954 bytes --]

OK, but that seems to be similar issue, although a different one :)
I recall I disabled compiler by default for target, because it's rare 
use case.

So compiler can be enabled (if needed), but then perhaps another tweak 
with sysroot will be necessary in the recipe? I suggest to keep it as a 
comment in the recipe. What do you think?

Slava

On 10.11.2023 08:50, Samuli Piippo wrote:
>
> | CMake Error at 
> /home/sapiippo/qt6/build/tmp/work/cortexa57-poky-linux/grpc/1.59.2/recipe-sysroot/usr/lib/cmake/protobuf/protobuf-targets.cmake:116 
> (message):
> |   The imported target "protobuf::protoc" references the file
> |
> | 
>      "/home/sapiippo/qt6/build/tmp/work/cortexa57-poky-linux/grpc/1.59.2/recipe-sysroot/usr/bin/protoc-23.4.0"
> |
> |   but this file does not exist.  Possible reasons include:
>
> On Fri, 10 Nov 2023 at 09:38, Vyacheslav Yurkov <uvv.mail@gmail.com> 
> wrote:
>
>     You sure? Because I remember the test in grpc to check for
>     CROSS_COMPILING in cmake to avoid that.
>
>     Slava
>
>     On 10.11.2023 08:12, Samuli Piippo wrote:
>>     This is not strictly a nativesdk issue.
>>     If you enable PACKAGECONFIG = "compiler" for protobuf, it will
>>     then build protoc for the target.
>>     This will then cause grpc target build to fail in the same way as
>>     nativesdk build.
>>
>>     On Fri, 10 Nov 2023 at 08:40, Vyacheslav Yurkov
>>     <uvv.mail@gmail.com> wrote:
>>
>>         FTR, I suggested oe-core solution in the corresponding
>>         mailing list.
>>
>>         Slava
>>
>>         On 10.11.2023 07:19, Vyacheslav Yurkov via
>>         lists.openembedded.org <http://lists.openembedded.org> wrote:
>>>         On 10.11.2023 02:23, Clément Péron wrote:
>>>>         Hi Samuli,
>>>>
>>>>         Le ven. 10 nov. 2023, 00:05, Vyacheslav Yurkov
>>>>         <uvv.mail@gmail.com> a écrit :
>>>>
>>>>             Thanks for the explanation, that was the missing piece
>>>>             for me to get an understanding of the problem :)
>>>>
>>>>             I think it's not only me, but also others, were
>>>>             confused by the fact that we are talking about
>>>>             native(sdk), but try to fix something with a non-native
>>>>             variable.
>>>>
>>>>             I've just gave it a try and have two suggestions how
>>>>             this can be addressed.
>>>>
>>>>             1. Recipe based solution:
>>>>             SYSROOT_DIRS:append:class-nativesdk = " $bindir}"
>>>>
>>>>
>>>>         Is it not just missing something like
>>>>
>>>>         DEPENDS:append:class-nativesdk= " protobuf-native "
>>>>         Regards
>>>>         Clement
>>>
>>>         This is already implicit by DEPENDS += "protobuf". Take a
>>>         look at my second suggestion. You can see that ${bindir} is
>>>         staged for native recipes, but not for nativesdk.
>>>
>>>>
>>>>             2. oe-core based solution:
>>>>             --- a/meta/classes-global/staging.bbclass
>>>>             +++ b/meta/classes-global/staging.bbclass
>>>>             @@ -26,6 +26,7 @@ SYSROOT_DIRS_NATIVE = " \
>>>>                  ${localstatedir} \
>>>>              "
>>>>              SYSROOT_DIRS:append:class-native = "
>>>>             ${SYSROOT_DIRS_NATIVE}"
>>>>             +SYSROOT_DIRS:append:class-nativesdk = "
>>>>             ${SYSROOT_DIRS_NATIVE}"
>>>>              SYSROOT_DIRS:append:class-cross = "
>>>>             ${SYSROOT_DIRS_NATIVE}"
>>>>              SYSROOT_DIRS:append:class-crosssdk = "
>>>>             ${SYSROOT_DIRS_NATIVE}"
>>>>
>>>>             The second one will invalidate nativesdk sstate, but
>>>>             might cover not only grpc issue (I would assume
>>>>             gdbus-codegen or any other generator has the same issue
>>>>             when used for SDK build).
>>>>
>>>>             What are your thoughs?
>>>>
>>>>             Thanks,
>>>>             Slava
>>>>
>>>>             On 09.11.2023 14:35, Samuli Piippo wrote:
>>>>>             DEPENDS += "protobuf" is enough to have dependencies
>>>>>             resolved.
>>>>>
>>>>>             Problem is that bitbake's do_populate_sysroot does not
>>>>>             take anything from bindir.
>>>>>             This leads to nativesdk-(qt)grpc's
>>>>>             do_prepare_recipe_sysroot to not get the protoc binary
>>>>>             into the recipe_sysroot as expected by CMake.
>>>>>
>>>>>             This can be fixed with SYSROOT_DIRS.
>>>>>
>>>>>             On Thu, 9 Nov 2023 at 12:19, Vyacheslav Yurkov
>>>>>             <uvv.mail@gmail.com> wrote:
>>>>>
>>>>>                 Hi Clément,
>>>>>                 No, the issue is when you just build nativesdk
>>>>>                 recipe (we are not at the stage to use the SDK yet).
>>>>>
>>>>>                 And that was exactly my question how to propagate
>>>>>                 the dependency correctly for nativesdk build.
>>>>>
>>>>>                 Regards,
>>>>>                 Slava
>>>>>
>

[-- Attachment #2: Type: text/html, Size: 15197 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-11-10  8:05                                                   ` Vyacheslav Yurkov
@ 2023-11-10  8:44                                                     ` Samuli Piippo
  2023-11-10  9:00                                                       ` Vyacheslav Yurkov
  0 siblings, 1 reply; 35+ messages in thread
From: Samuli Piippo @ 2023-11-10  8:44 UTC (permalink / raw)
  To: Vyacheslav Yurkov
  Cc: Clément Péron, Martin Jansa, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 4595 bytes --]

Agreed that enabling compiler in target would be rarely used.

Having
SYSROOT_DIRS += "${bindir}"
would, however, cover both builds without additional tweaks.
Since target build doesn't have any other binaries, this wouldn't do
anything extra with default PACKAGECONFIG.

On Fri, 10 Nov 2023 at 10:06, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:

> OK, but that seems to be similar issue, although a different one :)
> I recall I disabled compiler by default for target, because it's rare use
> case.
>
> So compiler can be enabled (if needed), but then perhaps another tweak
> with sysroot will be necessary in the recipe? I suggest to keep it as a
> comment in the recipe. What do you think?
>
> Slava
>
> On 10.11.2023 08:50, Samuli Piippo wrote:
>
>
> | CMake Error at
> /home/sapiippo/qt6/build/tmp/work/cortexa57-poky-linux/grpc/1.59.2/recipe-sysroot/usr/lib/cmake/protobuf/protobuf-targets.cmake:116
> (message):
> |   The imported target "protobuf::protoc" references the file
> |
> |
>      "/home/sapiippo/qt6/build/tmp/work/cortexa57-poky-linux/grpc/1.59.2/recipe-sysroot/usr/bin/protoc-23.4.0"
> |
> |   but this file does not exist.  Possible reasons include:
>
> On Fri, 10 Nov 2023 at 09:38, Vyacheslav Yurkov <uvv.mail@gmail.com>
> wrote:
>
>> You sure? Because I remember the test in grpc to check for
>> CROSS_COMPILING in cmake to avoid that.
>>
>> Slava
>>
>> On 10.11.2023 08:12, Samuli Piippo wrote:
>>
>> This is not strictly a nativesdk issue.
>> If you enable PACKAGECONFIG = "compiler" for protobuf, it will then build
>> protoc for the target.
>> This will then cause grpc target build to fail in the same way as
>> nativesdk build.
>>
>> On Fri, 10 Nov 2023 at 08:40, Vyacheslav Yurkov <uvv.mail@gmail.com>
>> wrote:
>>
>>> FTR, I suggested oe-core solution in the corresponding mailing list.
>>>
>>> Slava
>>>
>>> On 10.11.2023 07:19, Vyacheslav Yurkov via lists.openembedded.org wrote:
>>>
>>> On 10.11.2023 02:23, Clément Péron wrote:
>>>
>>> Hi Samuli,
>>>
>>> Le ven. 10 nov. 2023, 00:05, Vyacheslav Yurkov <uvv.mail@gmail.com> a
>>> écrit :
>>>
>>>> Thanks for the explanation, that was the missing piece for me to get an
>>>> understanding of the problem :)
>>>>
>>>> I think it's not only me, but also others, were confused by the fact
>>>> that we are talking about native(sdk), but try to fix something with a
>>>> non-native variable.
>>>>
>>>> I've just gave it a try and have two suggestions how this can be
>>>> addressed.
>>>>
>>>> 1. Recipe based solution:
>>>> SYSROOT_DIRS:append:class-nativesdk = " $bindir}"
>>>>
>>>
>>> Is it not just missing something like
>>>
>>> DEPENDS:append:class-nativesdk = " protobuf-native "
>>>
>>>  Regards
>>>
>>> Clement
>>>
>>>
>>> This is already implicit by DEPENDS += "protobuf". Take a look at my
>>> second suggestion. You can see that ${bindir} is staged for native recipes,
>>> but not for nativesdk.
>>>
>>>
>>>> 2. oe-core based solution:
>>>> --- a/meta/classes-global/staging.bbclass
>>>> +++ b/meta/classes-global/staging.bbclass
>>>> @@ -26,6 +26,7 @@ SYSROOT_DIRS_NATIVE = " \
>>>>      ${localstatedir} \
>>>>  "
>>>>  SYSROOT_DIRS:append:class-native = " ${SYSROOT_DIRS_NATIVE}"
>>>> +SYSROOT_DIRS:append:class-nativesdk = " ${SYSROOT_DIRS_NATIVE}"
>>>>  SYSROOT_DIRS:append:class-cross = " ${SYSROOT_DIRS_NATIVE}"
>>>>  SYSROOT_DIRS:append:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"
>>>>
>>>> The second one will invalidate nativesdk sstate, but might cover not
>>>> only grpc issue (I would assume gdbus-codegen or any other generator has
>>>> the same issue when used for SDK build).
>>>>
>>>> What are your thoughs?
>>>>
>>>> Thanks,
>>>> Slava
>>>>
>>>> On 09.11.2023 14:35, Samuli Piippo wrote:
>>>>
>>>> DEPENDS += "protobuf" is enough to have dependencies resolved.
>>>>
>>>> Problem is that bitbake's do_populate_sysroot does not take anything
>>>> from bindir.
>>>> This leads to nativesdk-(qt)grpc's do_prepare_recipe_sysroot to not get
>>>> the protoc binary into the recipe_sysroot as expected by CMake.
>>>>
>>>> This can be fixed with SYSROOT_DIRS.
>>>>
>>>> On Thu, 9 Nov 2023 at 12:19, Vyacheslav Yurkov <uvv.mail@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Clément,
>>>>> No, the issue is when you just build nativesdk recipe (we are not at
>>>>> the stage to use the SDK yet).
>>>>>
>>>>> And that was exactly my question how to propagate the dependency
>>>>> correctly for nativesdk build.
>>>>>
>>>>> Regards,
>>>>> Slava
>>>>>
>>>>
>>
>

[-- Attachment #2: Type: text/html, Size: 15098 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-11-10  8:44                                                     ` Samuli Piippo
@ 2023-11-10  9:00                                                       ` Vyacheslav Yurkov
  2023-11-13  8:42                                                         ` [meta-oe][PATCH] protobuf: stage protoc binary to sysroot Samuli Piippo
  0 siblings, 1 reply; 35+ messages in thread
From: Vyacheslav Yurkov @ 2023-11-10  9:00 UTC (permalink / raw)
  To: Samuli Piippo
  Cc: Clément Péron, Martin Jansa, Khem Raj, Ross Burton,
	openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 1163 bytes --]

Shall we go for round 3 of the patch then? :)

I'd wait for Clément feedback and then all 3 of us can sign it off.

I would also like to have a short comment/explanation before the 
SYSROOT_DIRS as to why it's required in this particular recipe.

FTR, as expected Richard didn't like the other suggestion, because the 
size of sstate is affected by that.

Slava

On 10.11.2023 09:44, Samuli Piippo wrote:
> Agreed that enabling compiler in target would be rarely used.
>
> Having
> SYSROOT_DIRS += "${bindir}"
> would, however, cover both builds without additional tweaks.
> Since target build doesn't have any other binaries, this wouldn't do 
> anything extra with default PACKAGECONFIG.
>
> On Fri, 10 Nov 2023 at 10:06, Vyacheslav Yurkov <uvv.mail@gmail.com> 
> wrote:
>
>     OK, but that seems to be similar issue, although a different one :)
>     I recall I disabled compiler by default for target, because it's
>     rare use case.
>
>     So compiler can be enabled (if needed), but then perhaps another
>     tweak with sysroot will be necessary in the recipe? I suggest to
>     keep it as a comment in the recipe. What do you think?
>
>     Slava
>

[-- Attachment #2: Type: text/html, Size: 2451 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* [meta-oe][PATCH] protobuf: stage protoc binary to sysroot
  2023-11-10  9:00                                                       ` Vyacheslav Yurkov
@ 2023-11-13  8:42                                                         ` Samuli Piippo
  2023-11-15  7:19                                                           ` Khem Raj
  0 siblings, 1 reply; 35+ messages in thread
From: Samuli Piippo @ 2023-11-13  8:42 UTC (permalink / raw)
  To: openembedded-devel

If protoc is enabled for the build, recipes using protobuf will
fail when protoc is not available in the recipe sysroot:

|   The imported target "protobuf::protoc" references the file
|
|      ".../recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-3.21.5.0"
|
|   but this file does not exist.  Possible reasons include:
|
|   * The file was deleted, renamed, or moved to another location.
|
|   * An install or uninstall procedure did not complete successfully.
|
|   * The installation package was faulty and contained
|
|      ".../recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake"
|
|   but not all the files it references.

Use SYSROOT_DIRS to stage the binary to sysroot so it's always
available for other recipes.

Signed-off-by: Samuli Piippo <samuli.piippo@qt.io>
---
 meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb b/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
index 1edc21cdf..7d37ee3d2 100644
--- a/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
+++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.23.4.bb
@@ -101,6 +101,9 @@ PACKAGE_BEFORE_PN = "${PN}-compiler ${PN}-lite"
 FILES:${PN}-compiler = "${bindir} ${libdir}/libprotoc${SOLIBS}"
 FILES:${PN}-lite = "${libdir}/libprotobuf-lite${SOLIBS}"
 
+# CMake requires protoc binary to exist in sysroot, even if it has wrong architecture.
+SYSROOT_DIRS += "${bindir}"
+
 RDEPENDS:${PN}-compiler = "${PN}"
 RDEPENDS:${PN}-dev += "${PN}-compiler"
 RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-protobuf', '', d)}"
-- 
2.25.1



^ permalink raw reply related	[flat|nested] 35+ messages in thread

* Re: [meta-oe][PATCH] protobuf: stage protoc binary to sysroot
  2023-11-13  8:42                                                         ` [meta-oe][PATCH] protobuf: stage protoc binary to sysroot Samuli Piippo
@ 2023-11-15  7:19                                                           ` Khem Raj
  0 siblings, 0 replies; 35+ messages in thread
From: Khem Raj @ 2023-11-15  7:19 UTC (permalink / raw)
  To: openembedded-devel, Samuli Piippo


On Mon, 13 Nov 2023 08:42:45 +0000, Samuli Piippo wrote:
> If protoc is enabled for the build, recipes using protobuf will
> fail when protoc is not available in the recipe sysroot:
> 
> |   The imported target "protobuf::protoc" references the file
> |
> |      ".../recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-3.21.5.0"
> |
> |   but this file does not exist.  Possible reasons include:
> |
> |   * The file was deleted, renamed, or moved to another location.
> |
> |   * An install or uninstall procedure did not complete successfully.
> |
> |   * The installation package was faulty and contained
> |
> |      ".../recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake"
> |
> |   but not all the files it references.
> 
> [...]

Applied, thanks!

[1/1] protobuf: stage protoc binary to sysroot
      commit: b1deec0de476d8f0a0c6427919f57ee5f681ea0e

Best regards,
-- 
Khem Raj <raj.khem@gmail.com>



^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [oe] [PATCH] Revert "protobuf: stage protoc binary to sysroot"
  2023-09-04 16:12 [PATCH] Revert "protobuf: stage protoc binary to sysroot" ross.burton
@ 2023-09-11  9:50 ` Samuli Piippo
  0 siblings, 0 replies; 35+ messages in thread
From: Samuli Piippo @ 2023-09-11  9:50 UTC (permalink / raw)
  To: Ross Burton; +Cc: openembedded-devel, nd

[-- Attachment #1: Type: text/plain, Size: 3147 bytes --]

Hi Ross,

The binaries are surely not useful for execution, but unfortunately they
are needed for CMake.
With this revert, Qt is now again experiencing build failures with protobuf:

| CMake Error at
/home/qt/qt6/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-qtgrpc/6.7.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/lib/cmake/protobuf/protobuf-targets.cmake:100
(message):

|   The imported target "protobuf::protoc" references the file

|

|
     "/home/qt/qt6/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-qtgrpc/6.7.0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/bin/protoc-3.21.12.0"

|

|   but this file does not exist.  Possible reasons include:



and in mingw builds:

| CMake Error at
/home/qt/work/build/build/tmp/work/x86_64-nativesdk-mingw32-w64-mingw32/nativesdk-qtgrpc/6.5.3/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-w64-mingw32/usr/lib/cmake/protobuf/protobuf-targets.cmake:100
(message):
|   The imported target "protobuf::libprotobuf-lite" references the file
|
|      "/home/qt/work/build/build/tmp/work/x86_64-nativesdk-mingw32-w64-mingw32/nativesdk-qtgrpc/6.5.3/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-w64-mingw32/usr/bin/libprotobuf-lite.dll"
|
|   but this file does not exist.  Possible reasons include:



On Mon, 4 Sept 2023 at 19:12, Ross Burton <ross.burton@arm.com> wrote:

> From: Ross Burton <ross.burton@arm.com>
>
> Putting the _target_ protoc into the sysroot for executation at _build_
> time isn't useful because even if it has the right architecture, the
> tune might be incompatible.  Recipes which want protoc should just depend
> on protobuf-native.
>
> This reverts commit d7f46fa816964e30edb2ccfecc57a26251cc351c.
>
> Signed-off-by: Ross Burton <ross.burton@arm.com>
> ---
>  meta-oe/recipes-devtools/protobuf/protobuf_3.21.12.bb | 2 --
>  1 file changed, 2 deletions(-)
>
> diff --git a/meta-oe/recipes-devtools/protobuf/protobuf_3.21.12.bb
> b/meta-oe/recipes-devtools/protobuf/protobuf_3.21.12.bb
> index 343933033b..d95e1c20f3 100644
> --- a/meta-oe/recipes-devtools/protobuf/protobuf_3.21.12.bb
> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_3.21.12.bb
> @@ -92,8 +92,6 @@ PACKAGE_BEFORE_PN = "${PN}-compiler ${PN}-lite"
>  FILES:${PN}-compiler = "${bindir} ${libdir}/libprotoc${SOLIBS}"
>  FILES:${PN}-lite = "${libdir}/libprotobuf-lite${SOLIBS}"
>
> -SYSROOT_DIRS += "${bindir}"
> -
>  RDEPENDS:${PN}-compiler = "${PN}"
>  RDEPENDS:${PN}-dev += "${PN}-compiler"
>  RDEPENDS:${PN}-ptest = "bash ${@bb.utils.contains('PACKAGECONFIG',
> 'python', 'python3-protobuf', '', d)}"
> --
> 2.34.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#104716):
> https://lists.openembedded.org/g/openembedded-devel/message/104716
> Mute This Topic: https://lists.openembedded.org/mt/101151552/3617605
> Group Owner: openembedded-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [
> samuli.piippo@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 7589 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

end of thread, other threads:[~2023-11-15  7:19 UTC | newest]

Thread overview: 35+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-09-30 17:30 [PATCH] Revert "protobuf: stage protoc binary to sysroot" Clément Péron
2023-10-01  0:48 ` [oe] " Vyacheslav Yurkov
2023-10-01  3:03   ` Khem Raj
2023-10-01 10:11     ` Martin Jansa
2023-10-01 10:31       ` Clément Péron
2023-10-01 18:22         ` Martin Jansa
2023-10-02 16:25           ` Vyacheslav Yurkov
     [not found]           ` <178A5713556C6E82.9230@lists.openembedded.org>
2023-10-12 12:06             ` Vyacheslav Yurkov
2023-10-18 12:28               ` Samuli Piippo
2023-10-18 15:27                 ` Ross Burton
2023-10-18 15:34                   ` Martin Jansa
2023-10-20  6:17                 ` Vyacheslav Yurkov
2023-10-20  7:07                   ` Martin Jansa
2023-10-20  7:11                   ` Samuli Piippo
2023-10-20  7:35                     ` Martin Jansa
2023-10-22 18:45                       ` Vyacheslav Yurkov
2023-10-24 11:52                         ` Clément Péron
2023-10-24 13:25                           ` Samuli Piippo
     [not found]                           ` <17910E05B5C0179B.11247@lists.openembedded.org>
2023-11-08 13:30                             ` Samuli Piippo
2023-11-09  0:13                               ` Clément Péron
2023-11-09 10:19                                 ` Vyacheslav Yurkov
2023-11-09 13:35                                   ` Samuli Piippo
2023-11-09 15:05                                     ` Vyacheslav Yurkov
2023-11-10  1:23                                       ` Clément Péron
2023-11-10  6:19                                         ` Vyacheslav Yurkov
     [not found]                                         ` <17962EA21B11FB58.7380@lists.openembedded.org>
2023-11-10  6:40                                           ` Vyacheslav Yurkov
2023-11-10  7:12                                             ` Samuli Piippo
2023-11-10  7:38                                               ` Vyacheslav Yurkov
2023-11-10  7:50                                                 ` Samuli Piippo
2023-11-10  8:05                                                   ` Vyacheslav Yurkov
2023-11-10  8:44                                                     ` Samuli Piippo
2023-11-10  9:00                                                       ` Vyacheslav Yurkov
2023-11-13  8:42                                                         ` [meta-oe][PATCH] protobuf: stage protoc binary to sysroot Samuli Piippo
2023-11-15  7:19                                                           ` Khem Raj
  -- strict thread matches above, loose matches on Subject: below --
2023-09-04 16:12 [PATCH] Revert "protobuf: stage protoc binary to sysroot" ross.burton
2023-09-11  9:50 ` [oe] " Samuli Piippo

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.