[Question] upx: do_fetch: Bitbake Fetcher Error:,FetchError('Submodule refers to the parent repository.

[Question] upx: do_fetch: Bitbake Fetcher Error:,FetchError('Submodule refers to the parent repository.

[Joakim Roubert]

Hi!

On Kirkstone, since Kirkstone 4.0.5 (commit 
189a6d452e3037c9e94ccdf6af38359fc6058064, 
https://git.yoctoproject.org/poky/commit/?h=kirkstone&id=189a6d452e3037c9e94ccdf6af38359fc6058064) 
the added check gives me

ERROR: upx-3.96+AUTOINC+4e1ae22a1a-r0 do_fetch: Bitbake Fetcher Error:
FetchError('Submodule refers to the parent repository. This will cause
deadlock situation in current version of Bitbake.Consider using git
fetcher instead.', None)

(I noticed that latest master is identical with latest kirkstone.)
I'm experimenting with using git:// instead of gitsm:// and adding a 
suggested solution if found searching for documentation on how to 
migitate this

do_configure:prepend() {
     cd ${WORKDIR}/git
     git submodule update --init --recursive
}

but I've not yet been able to make things work. Since I'm no power user 
when it comes to Yocto magic perhaps someone already knows what would be 
the best way forward on this (if I'm on the wrong path)?

BR,

/Joakim

Re: [meta-virtualization] [Question] upx: do_fetch: Bitbake Fetcher Error:,FetchError('Submodule refers to the parent repository.

[Bruce Ashfield]

On Thu, Dec 1, 2022 at 8:46 AM Joakim Roubert <joakim.roubert@axis.com> wrote:
>
> Hi!
>
> On Kirkstone, since Kirkstone 4.0.5 (commit
> 189a6d452e3037c9e94ccdf6af38359fc6058064,
> https://git.yoctoproject.org/poky/commit/?h=kirkstone&id=189a6d452e3037c9e94ccdf6af38359fc6058064)
> the added check gives me
>
> ERROR: upx-3.96+AUTOINC+4e1ae22a1a-r0 do_fetch: Bitbake Fetcher Error:
> FetchError('Submodule refers to the parent repository. This will cause
> deadlock situation in current version of Bitbake.Consider using git
> fetcher instead.', None)
>
> (I noticed that latest master is identical with latest kirkstone.)
> I'm experimenting with using git:// instead of gitsm:// and adding a
> suggested solution if found searching for documentation on how to
> migitate this

There's been some serious re-writing of the repository, which is
generally speaking a no-no.

The hash we were previously using is completely gone. But then again,
we are using a branch called "devel", so history re-writing could be
expected.

The fix is not really about the fetcher,  but using a different git hash.

I've bumped the recipe locally, and our one patch can also be dropped,
as it has been merged.

That being said, I'm now running into cmake build issues, which I'm
working through.

I'll continue to plug away at it in the background today, and with any
luck can have it working again shortly (I'm terrible with cmake).

Bruce

>
> do_configure:prepend() {
>      cd ${WORKDIR}/git
>      git submodule update --init --recursive
> }
>
> but I've not yet been able to make things work. Since I'm no power user
> when it comes to Yocto magic perhaps someone already knows what would be
> the best way forward on this (if I'm on the wrong path)?
>
> BR,
>
> /Joakim
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#7720): https://lists.yoctoproject.org/g/meta-virtualization/message/7720
> Mute This Topic: https://lists.yoctoproject.org/mt/95382316/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] [Question] upx: do_fetch: Bitbake Fetcher Error:,FetchError('Submodule refers to the parent repository.

[Bruce Ashfield]

On Thu, Dec 1, 2022 at 9:50 AM Bruce Ashfield via
lists.yoctoproject.org
<bruce.ashfield=gmail.com@lists.yoctoproject.org> wrote:
>
> On Thu, Dec 1, 2022 at 8:46 AM Joakim Roubert <joakim.roubert@axis.com> wrote:
> >
> > Hi!
> >
> > On Kirkstone, since Kirkstone 4.0.5 (commit
> > 189a6d452e3037c9e94ccdf6af38359fc6058064,
> > https://git.yoctoproject.org/poky/commit/?h=kirkstone&id=189a6d452e3037c9e94ccdf6af38359fc6058064)
> > the added check gives me
> >
> > ERROR: upx-3.96+AUTOINC+4e1ae22a1a-r0 do_fetch: Bitbake Fetcher Error:
> > FetchError('Submodule refers to the parent repository. This will cause
> > deadlock situation in current version of Bitbake.Consider using git
> > fetcher instead.', None)
> >
> > (I noticed that latest master is identical with latest kirkstone.)
> > I'm experimenting with using git:// instead of gitsm:// and adding a
> > suggested solution if found searching for documentation on how to
> > migitate this
>
> There's been some serious re-writing of the repository, which is
> generally speaking a no-no.
>
> The hash we were previously using is completely gone. But then again,
> we are using a branch called "devel", so history re-writing could be
> expected.
>
> The fix is not really about the fetcher,  but using a different git hash.
>
> I've bumped the recipe locally, and our one patch can also be dropped,
> as it has been merged.
>
> That being said, I'm now running into cmake build issues, which I'm
> working through.
>
> I'll continue to plug away at it in the background today, and with any
> luck can have it working again shortly (I'm terrible with cmake).
>

I did have some luck, and I pushed a proposed fix to master-next.

Have a look, and see if it fixes the build for you.

Bruce

> Bruce
>
> >
> > do_configure:prepend() {
> >      cd ${WORKDIR}/git
> >      git submodule update --init --recursive
> > }
> >
> > but I've not yet been able to make things work. Since I'm no power user
> > when it comes to Yocto magic perhaps someone already knows what would be
> > the best way forward on this (if I'm on the wrong path)?
> >
> > BR,
> >
> > /Joakim
> >
> >
> >
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#7721): https://lists.yoctoproject.org/g/meta-virtualization/message/7721
> Mute This Topic: https://lists.yoctoproject.org/mt/95382316/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] [Question] upx: do_fetch: Bitbake Fetcher Error:,FetchError('Submodule refers to the parent repository.

[Joakim Roubert]

On 2022-12-01 16:08, Bruce Ashfield wrote:
> 
> I did have some luck, and I pushed a proposed fix to master-next.
> 
> Have a look, and see if it fixes the build for you.

Thanks a lot; I tried it out immediately.
It will, however, give me the same gitsm error:

ERROR: upx-3.96+AUTOINC+8d1a98e03b-r0 do_fetch: Fetcher failure: 
Submodule refers to the parent repository. This will cause deadlock 
situation in current version of Bitbake.Consider using git fetcher instead.

BR,

/Joakim

Re: [meta-virtualization] [Question] upx: do_fetch: Bitbake Fetcher Error:,FetchError('Submodule refers to the parent repository.

[Joakim Roubert]

[-- Attachment #1: Type: text/plain, Size: 409 bytes --]

On 2022-12-02 09:46, Joakim Roubert wrote:
> 
> Thanks a lot; I tried it out immediately.
> It will, however, give me the same gitsm error:
[...]

By using the git fetcher instead of the gitsm one and explicitly add the 
submodules (whth the corresponding commits) in the recipe as per the 
attached patch (on latest master-next that has your latest fixes), upx 
will build properly here again.

BR,

/Joakim

[-- Attachment #2: 0001-upx-Use-git-fetcher-instead-of-gitsm.patch --]
[-- Type: text/x-patch, Size: 2040 bytes --]

From a49e1cd59e05e23db65f2846feeafcc69cba4543 Mon Sep 17 00:00:00 2001
From: Joakim Roubert <joakimr@axis.com>
Date: Fri, 2 Dec 2022 11:25:15 +0100
Subject: [PATCH] upx: Use git fetcher instead of gitsm

The current upx will trigger the Bitbake Fetcher Error added in
Kirkstone 4.0.5 189a6d452e3037c9e94ccdf6af38359fc6058064 commit. This
patch replaces gitsm with git and explicitly adds the git submodules
with the corresponding commits.

Change-Id: I1b1231f06481f7a1e25dc35277d0f1725c7631e0
Signed-off-by: Joakim Roubert <joakimr@axis.com>
---
 recipes-extended/upx/upx_git.bb | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/recipes-extended/upx/upx_git.bb b/recipes-extended/upx/upx_git.bb
index 989b277..bb8004c 100644
--- a/recipes-extended/upx/upx_git.bb
+++ b/recipes-extended/upx/upx_git.bb
@@ -1,8 +1,17 @@
 HOMEPAGE = "http://upx.sourceforge.net"
 SUMMARY = "Ultimate executable compressor."
 
-SRCREV = "8d1a98e03bf281b2cee459b6c27347e56d13c6a8"
-SRC_URI = "gitsm://github.com/upx/upx;branch=devel;protocol=https \
+SRCREV_upx = "8d1a98e03bf281b2cee459b6c27347e56d13c6a8"
+SRCREV_vendor_doctest = "666e648b68fda2deb141a1fe93e3fd1e2795dd0f"
+SRCREV_vendor_lzma_sdk = "9ebf8f468c689d83504e6c08c6bc26c4a1cf180f"
+SRCREV_vendor_ucl = "4b58d592199dc1e5db691e1a54fb0e5e9af0ecaf"
+SRCREV_vendor_zlib = "2a5b338eb173a701ed179e951d4c390e75e8d4c7"
+SRCREV_FORMAT = "upx"
+SRC_URI = "git://github.com/upx/upx;name=upx;branch=devel;protocol=https \
+           git://github.com/upx/upx-vendor-doctest;name=vendor_doctest;subdir=git/vendor/doctest;branch=upx-vendor;protocol=https \
+           git://github.com/upx/upx-vendor-lzma-sdk;name=vendor_lzma_sdk;subdir=git/vendor/lzma-sdk;branch=upx-vendor;protocol=https \
+           git://github.com/upx/upx-vendor-ucl;name=vendor_ucl;subdir=git/vendor/ucl;branch=upx-vendor;protocol=https \
+           git://github.com/upx/upx-vendor-zlib;name=vendor_zlib;subdir=git/vendor/zlib;branch=upx-vendor;protocol=https \
 "
 
 LICENSE = "GPL-2.0-only"
-- 
2.30.2

Re: [meta-virtualization] [Question] upx: do_fetch: Bitbake Fetcher Error:,FetchError('Submodule refers to the parent repository.

[Bruce Ashfield]

On Fri, Dec 2, 2022 at 3:46 AM Joakim Roubert <joakim.roubert@axis.com> wrote:
>
> On 2022-12-01 16:08, Bruce Ashfield wrote:
> >
> > I did have some luck, and I pushed a proposed fix to master-next.
> >
> > Have a look, and see if it fixes the build for you.
>
> Thanks a lot; I tried it out immediately.
> It will, however, give me the same gitsm error:
>
> ERROR: upx-3.96+AUTOINC+8d1a98e03b-r0 do_fetch: Fetcher failure:
> Submodule refers to the parent repository. This will cause deadlock
> situation in current version of Bitbake.Consider using git fetcher instead.
>

I don't see this on my up to date master build (I'm always following tip).

Let me dig in a bit more.

That being said, I've written multiple recipes like your follow up
patch, and I do prefer the clarity of everything being directly in the
SRC_URI.

Bruce

> BR,
>
> /Joakim
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] [Question] upx: do_fetch: Bitbake Fetcher Error:,FetchError('Submodule refers to the parent repository.

[Bruce Ashfield]

On Fri, Dec 2, 2022 at 8:12 AM Bruce Ashfield via
lists.yoctoproject.org
<bruce.ashfield=gmail.com@lists.yoctoproject.org> wrote:
>
> On Fri, Dec 2, 2022 at 3:46 AM Joakim Roubert <joakim.roubert@axis.com> wrote:
> >
> > On 2022-12-01 16:08, Bruce Ashfield wrote:
> > >
> > > I did have some luck, and I pushed a proposed fix to master-next.
> > >
> > > Have a look, and see if it fixes the build for you.
> >
> > Thanks a lot; I tried it out immediately.
> > It will, however, give me the same gitsm error:
> >
> > ERROR: upx-3.96+AUTOINC+8d1a98e03b-r0 do_fetch: Fetcher failure:
> > Submodule refers to the parent repository. This will cause deadlock
> > situation in current version of Bitbake.Consider using git fetcher instead.
> >
>
> I don't see this on my up to date master build (I'm always following tip).
>
> Let me dig in a bit more.
>
> That being said, I've written multiple recipes like your follow up
> patch, and I do prefer the clarity of everything being directly in the
> SRC_URI.
>

I still can't reproduce the issue here, but everything still works
with your patch on top .. so I've added it to the queue on
master-next.

Bruce

> Bruce
>
> > BR,
> >
> > /Joakim
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#7731): https://lists.yoctoproject.org/g/meta-virtualization/message/7731
> Mute This Topic: https://lists.yoctoproject.org/mt/95382316/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] [Question] upx: do_fetch: Bitbake Fetcher Error:,FetchError('Submodule refers to the parent repository.

[Joakim Roubert]

On 2022-12-02 14:11, Bruce Ashfield wrote:
> 
> That being said, I've written multiple recipes like your follow up
> patch, and I do prefer the clarity of everything being directly in the
> SRC_URI.

Excellent!

On 2022-12-02 15:58, Bruce Ashfield wrote:
 >
 > I still can't reproduce the issue here, but everything still works
 > with your patch on top .. so I've added it to the queue on
 > master-next.

Do you think it could be allowed to reach the kirkstone branch too?

BR,

/Joakim
-- 
Joakim Roubert
Senior Engineer

Axis Communications AB
Emdalavägen 14, SE-223 69 Lund, Sweden
Tel: +46 46 272 18 00, Tel (direct): +46 46 272 27 48
Fax: +46 46 13 61 30, www.axis.com

Re: [meta-virtualization] [Question] upx: do_fetch: Bitbake Fetcher Error:,FetchError('Submodule refers to the parent repository.

[Bruce Ashfield]

On Mon, Dec 5, 2022 at 7:02 AM Joakim Roubert <joakim.roubert@axis.com> wrote:
>
> On 2022-12-02 14:11, Bruce Ashfield wrote:
> >
> > That being said, I've written multiple recipes like your follow up
> > patch, and I do prefer the clarity of everything being directly in the
> > SRC_URI.
>
> Excellent!
>
> On 2022-12-02 15:58, Bruce Ashfield wrote:
>  >
>  > I still can't reproduce the issue here, but everything still works
>  > with your patch on top .. so I've added it to the queue on
>  > master-next.
>
> Do you think it could be allowed to reach the kirkstone branch too?
>

I can do the cherry picks once I get master-next -> master ... which
is hopefully very soon.

Bruce

> BR,
>
> /Joakim
> --
> Joakim Roubert
> Senior Engineer
>
> Axis Communications AB
> Emdalavägen 14, SE-223 69 Lund, Sweden
> Tel: +46 46 272 18 00, Tel (direct): +46 46 272 27 48
> Fax: +46 46 13 61 30, www.axis.com
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#7737): https://lists.yoctoproject.org/g/meta-virtualization/message/7737
> Mute This Topic: https://lists.yoctoproject.org/mt/95382316/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] [Question] upx: do_fetch: Bitbake Fetcher Error:,FetchError('Submodule refers to the parent repository.

[Joakim Roubert]

On 2022-12-07 22:52, Bruce Ashfield wrote:
> 
> I can do the cherry picks once I get master-next -> master ... which
> is hopefully very soon.

Excellent, thanks a lot!

BR,

/Joakim