Am 01.10.2018 um 15:34 schrieb Mark Hatle <mark.hatle@windriver.com>:

On 10/1/18 3:52 AM, Jens Rehsack | External Miltenyi Biotec wrote:
Hi Mark,

after your last update to gitsm fetcher, it seems it cannot handle submodules not using the master branch correctly:

I'm not sure I understand below what is going wrong.

Does it not checkout the primary repository in the correct branch, or does this
get checked out -- and it's the submodules that get 'confused' between the
pointed to commit and the branches?

The primary repository is - as far as I can see from the logs - checked out correctly.

It just tries to apply the branch 'master' to the submodule. The primary repository has

the branch 'develop' - what makes me a bit curios.

The system recursively processes the git submodules, while interrogating it's
way through the system. If the failure is on a submode (not the main
repository) it would be something in this block...

http://git.openembedded.org/bitbake/tree/lib/bb/fetch2/gitsm.py

47 - 108

Looking at the code, line 81.

           # Build new SRC_URI
           proto = uris[module].split(':', 1)[0]
           url = uris[module].replace('%s:' % proto, 'gitsm:', 1)
           url += ';protocol=%s' % proto
           url += ";name=%s" % module
           url += ";qbareclone=1;nocheckout=1"

It looks like a 'q' accidentally made it into the patch. Can you remove this
and try it?   This is where the system constructs SRC_URI for the recursive
(submodule) download. The next block defines the SRCREV and related items and
re-calls fetch.

Will do tomorrow when I'm back in company and report back.

Nowhere in there do we call or set anything related to the branches. Maybe
"nobranch=1" needs to be added to the generated SRC_URI?

I tried that and this ended up in wrong names for the archive files created

for the download cache (see below).

If you can give me a recipe that used to work through 'do_unpack', then I should
be able to attempt to replicate it locally and work through the problems if the
above isn't enough of a place to look.

Unfortunately we have only proprietary recipes using that. I can try create

one for a perl module I maintain, if you like.

You will need meta-cpan for that - can you confirm that meta-cpan works

for you in general (https://github.com/meta-cpan)?

(It also shows a missing test case, one that we need to add -- once we
understand what part and why it's failing.)

--Mark

Best regards,

Jens

--- BEGIN INITIAL ERROR
WARNING: automacs-ui-git-r0 do_fetch: Failed to fetch URL gitsm://mgegit.miltenyibiotec.de:7999/umsa/automacs-ui.git;protocol=ssh;branch=develop;rev=fddb26a33d3f949d0863cffc5306146ab8e5a9a8, attempting MIRRORS if available
ERROR: automacs-ui-git-r0 do_fetch: Fetcher failure for URL: 'gitsm://git@mgegit.miltenyibiotec.de:7999/cpsm/cmake-modules.git;protocol=ssh;name=cmake/modules;qbareclone=1;nocheckout=1'. Unable to fetch URL from any source.
ERROR: automacs-ui-git-r0 do_fetch: Fetcher failure for URL: 'gitsm://mgegit.miltenyibiotec.de:7999/umsa/automacs-ui.git;protocol=ssh;branch=develop;rev=fddb26a33d3f949d0863cffc5306146ab8e5a9a8'. Unable to fetch URL from any source.
ERROR: automacs-ui-git-r0 do_fetch: Function failed: base_do_fetch
ERROR: Logfile of failure stored in: /home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/temp/log.do_fetch.5452
ERROR: Task (/home/sno/gpw-community-bsp/sources/meta-miltenyi/recipes-ui/automacs-ui/automacs-ui_git.bb:do_fetch) failed with exit code '1'

DEBUG: Running export PSEUDO_DISABLED=1; export GIT_PROXY_COMMAND="oe-git-proxy"; export NO_PROXY="*.miltenyibiotec.de,*.miltenyibiotec.com"; export https_proxy="http://proxy2:8080/"; export http_proxy="http://proxy2:8080/"; export PATH="/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/sno/gpw-community-bsp/sources/poky/scripts:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/recipe-sysroot/opt/miltenyi/bin/crossscripts:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/recipe-sysroot-native/usr/sbin:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/recipe-sysroot-native/usr/bin:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/recipe-sysroot-native/sbin:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/recipe-sysroot-native/bin:/home/sno/gpw-community-bsp/sources/poky/bitbake/bin:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/hosttools"; export HOME="/home/sno"; git -c core.fsyncobjectfiles=0 branch --contains c6c052603e2e3ea334c5a71e3584855c5535995f --list master 2> /dev/null | wc -l
--- END INITIAL ERROR

I cloned the appropriate project locally to see what's going on:

sno@yoctopoc:~/prj/automacs-ui/cmake/modules$ git -c core.fsyncobjectfiles=0 branch --contains c6c052603e2e3ea334c5a71e3584855c5535995f --list
* (HEAD detached at c6c0526)
develop
sno@yoctopoc:~/prj/automacs-ui/cmake/modules$ git -c core.fsyncobjectfiles=0 branch --contains c6c052603e2e3ea334c5a71e3584855c5535995f --list master
sno@yoctopoc:~/prj/automacs-ui/cmake/modules$

The I tried to reverse engineer the gitsm fetcher and added:

poky$ git diff
diff --git a/bitbake/lib/bb/fetch2/gitsm.py b/bitbake/lib/bb/fetch2/gitsm.py
index 51f8d0e68c..221af934c3 100644
--- a/bitbake/lib/bb/fetch2/gitsm.py
+++ b/bitbake/lib/bb/fetch2/gitsm.py
@@ -79,6 +79,7 @@ class GitSM(Git):
            url += ';protocol=%s' % proto
            url += ";name=%s" % module
            url += ";qbareclone=1;nocheckout=1"
+            url += ";nobranch=1"

            ld = d.createCopy()
            # Not necessary to set SRC_URI, since we're passing the URI to

which results into

--- BEGIN FOLLOW UP ERROR
ERROR: automacs-ui-git-r0 do_unpack: Fetcher failure: Submodule cmake/modules does not exist in /home/sno/gpw-community-bsp/mops-yocto-platform/downloads/git2/mgegit.miltenyibiotec.de.7999.umsa.automacs-ui.git/modules/cmake/modules or /home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/git/.git/modules/cmake/modules.
ERROR: automacs-ui-git-r0 do_unpack: Function failed: base_do_unpack
ERROR: Logfile of failure stored in: /home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/temp/log.do_unpack.6551
ERROR: Task (/home/sno/gpw-community-bsp/sources/meta-miltenyi/recipes-ui/automacs-ui/automacs-ui_git.bb:do_unpack) failed with exit code '1'
--- END FOLLOW UP ERROR

because it uses the submodule names as archive name:

$ tar xzf downloads/git2_mgegit.miltenyibiotec.de.7999.<TAB>
git2_mgegit.miltenyibiotec.de.7999.cpsm.cmake-modules.git.tar.gz
git2_mgegit.miltenyibiotec.de.7999.cpsm.mcs-lib.git.tar.gz
git2_mgegit.miltenyibiotec.de.7999.umsa.corert.git.tar.gz
git2_mgegit.miltenyibiotec.de.7999.umsa.protos.git.tar.gz

Cheers,
Jens