> Am 01.10.2018 um 15:34 schrieb Mark Hatle : > > On 10/1/18 3:52 AM, Jens Rehsack | External Miltenyi Biotec wrote: >> Hi Mark, >> >> after your last update to gitsm fetcher, it seems it cannot handle submodules not using the master branch correctly: > > I'm not sure I understand below what is going wrong. > > Does it not checkout the primary repository in the correct branch, or does this > get checked out -- and it's the submodules that get 'confused' between the > pointed to commit and the branches? The primary repository is - as far as I can see from the logs - checked out correctly. It just tries to apply the branch 'master' to the submodule. The primary repository has the branch 'develop' - what makes me a bit curios. > The system recursively processes the git submodules, while interrogating it's > way through the system. If the failure is on a submode (not the main > repository) it would be something in this block... > > http://git.openembedded.org/bitbake/tree/lib/bb/fetch2/gitsm.py > > 47 - 108 > > Looking at the code, line 81. > > # Build new SRC_URI > proto = uris[module].split(':', 1)[0] > url = uris[module].replace('%s:' % proto, 'gitsm:', 1) > url += ';protocol=%s' % proto > url += ";name=%s" % module > url += ";qbareclone=1;nocheckout=1" > > > It looks like a 'q' accidentally made it into the patch. Can you remove this > and try it? This is where the system constructs SRC_URI for the recursive > (submodule) download. The next block defines the SRCREV and related items and > re-calls fetch. Will do tomorrow when I'm back in company and report back. > Nowhere in there do we call or set anything related to the branches. Maybe > "nobranch=1" needs to be added to the generated SRC_URI? I tried that and this ended up in wrong names for the archive files created for the download cache (see below). > If you can give me a recipe that used to work through 'do_unpack', then I should > be able to attempt to replicate it locally and work through the problems if the > above isn't enough of a place to look. Unfortunately we have only proprietary recipes using that. I can try create one for a perl module I maintain, if you like. You will need meta-cpan for that - can you confirm that meta-cpan works for you in general (https://github.com/meta-cpan )? > (It also shows a missing test case, one that we need to add -- once we > understand what part and why it's failing.) > > --Mark Best regards, Jens >> --- BEGIN INITIAL ERROR >> WARNING: automacs-ui-git-r0 do_fetch: Failed to fetch URL gitsm://mgegit.miltenyibiotec.de:7999/umsa/automacs-ui.git;protocol=ssh;branch=develop;rev=fddb26a33d3f949d0863cffc5306146ab8e5a9a8, attempting MIRRORS if available >> ERROR: automacs-ui-git-r0 do_fetch: Fetcher failure for URL: 'gitsm://git@mgegit.miltenyibiotec.de:7999/cpsm/cmake-modules.git;protocol=ssh;name=cmake/modules;qbareclone=1;nocheckout=1'. Unable to fetch URL from any source. >> ERROR: automacs-ui-git-r0 do_fetch: Fetcher failure for URL: 'gitsm://mgegit.miltenyibiotec.de:7999/umsa/automacs-ui.git;protocol=ssh;branch=develop;rev=fddb26a33d3f949d0863cffc5306146ab8e5a9a8'. Unable to fetch URL from any source. >> ERROR: automacs-ui-git-r0 do_fetch: Function failed: base_do_fetch >> ERROR: Logfile of failure stored in: /home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/temp/log.do_fetch.5452 >> ERROR: Task (/home/sno/gpw-community-bsp/sources/meta-miltenyi/recipes-ui/automacs-ui/automacs-ui_git.bb:do_fetch) failed with exit code '1' >> >> DEBUG: Running export PSEUDO_DISABLED=1; export GIT_PROXY_COMMAND="oe-git-proxy"; export NO_PROXY="*.miltenyibiotec.de,*.miltenyibiotec.com"; export https_proxy="http://proxy2:8080/"; export http_proxy="http://proxy2:8080/"; export PATH="/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/sno/gpw-community-bsp/sources/poky/scripts:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/recipe-sysroot/opt/miltenyi/bin/crossscripts:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/recipe-sysroot-native/usr/sbin:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/recipe-sysroot-native/usr/bin:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/recipe-sysroot-native/sbin:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/recipe-sysroot-native/bin:/home/sno/gpw-community-bsp/sources/poky/bitbake/bin:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/hosttools"; export HOME="/home/sno"; git -c core.fsyncobjectfiles=0 branch --contains c6c052603e2e3ea334c5a71e3584855c5535995f --list master 2> /dev/null | wc -l >> --- END INITIAL ERROR >> >> I cloned the appropriate project locally to see what's going on: >> >> sno@yoctopoc:~/prj/automacs-ui/cmake/modules$ git -c core.fsyncobjectfiles=0 branch --contains c6c052603e2e3ea334c5a71e3584855c5535995f --list >> * (HEAD detached at c6c0526) >> develop >> sno@yoctopoc:~/prj/automacs-ui/cmake/modules$ git -c core.fsyncobjectfiles=0 branch --contains c6c052603e2e3ea334c5a71e3584855c5535995f --list master >> sno@yoctopoc:~/prj/automacs-ui/cmake/modules$ >> >> The I tried to reverse engineer the gitsm fetcher and added: >> >> poky$ git diff >> diff --git a/bitbake/lib/bb/fetch2/gitsm.py b/bitbake/lib/bb/fetch2/gitsm.py >> index 51f8d0e68c..221af934c3 100644 >> --- a/bitbake/lib/bb/fetch2/gitsm.py >> +++ b/bitbake/lib/bb/fetch2/gitsm.py >> @@ -79,6 +79,7 @@ class GitSM(Git): >> url += ';protocol=%s' % proto >> url += ";name=%s" % module >> url += ";qbareclone=1;nocheckout=1" >> + url += ";nobranch=1" >> >> ld = d.createCopy() >> # Not necessary to set SRC_URI, since we're passing the URI to >> >> which results into >> >> --- BEGIN FOLLOW UP ERROR >> ERROR: automacs-ui-git-r0 do_unpack: Fetcher failure: Submodule cmake/modules does not exist in /home/sno/gpw-community-bsp/mops-yocto-platform/downloads/git2/mgegit.miltenyibiotec.de.7999.umsa.automacs-ui.git/modules/cmake/modules or /home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/git/.git/modules/cmake/modules. >> ERROR: automacs-ui-git-r0 do_unpack: Function failed: base_do_unpack >> ERROR: Logfile of failure stored in: /home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/core2-64-poky-linux/automacs-ui/git-r0/temp/log.do_unpack.6551 >> ERROR: Task (/home/sno/gpw-community-bsp/sources/meta-miltenyi/recipes-ui/automacs-ui/automacs-ui_git.bb:do_unpack) failed with exit code '1' >> --- END FOLLOW UP ERROR >> >> because it uses the submodule names as archive name: >> >> $ tar xzf downloads/git2_mgegit.miltenyibiotec.de.7999. >> git2_mgegit.miltenyibiotec.de.7999.cpsm.cmake-modules.git.tar.gz >> git2_mgegit.miltenyibiotec.de.7999.cpsm.mcs-lib.git.tar.gz >> git2_mgegit.miltenyibiotec.de.7999.umsa.corert.git.tar.gz >> git2_mgegit.miltenyibiotec.de.7999.umsa.protos.git.tar.gz >> >> Cheers, >> Jens >> > -- Jens Rehsack - rehsack@gmail.com