All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Urs Fässler" <urs.fassler@bbv.ch>
To: "bitbake-devel@lists.openembedded.org"
	<bitbake-devel@lists.openembedded.org>
Subject: [PATCH 7/7] fetch2: add symlink for shallow mirror tarballs
Date: Mon, 15 Oct 2018 13:43:22 +0000	[thread overview]
Message-ID: <20181015134255.4634-8-urs.fassler@bbv.ch> (raw)
In-Reply-To: <20181015134255.4634-1-urs.fassler@bbv.ch>

In the unpack step the shallow mirror tarball is not found when the
mirrored tarball is named differently than the original tarball. This
happens due to mirror rewrite rules. To solve the problem we add
symlinks for the differently named tarballs.

Signed-off-by: Urs Fässler <urs.fassler@bbv.ch>
Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
---
 lib/bb/fetch2/__init__.py |  2 ++
 lib/bb/tests/fetch.py     | 44 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)

diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index a1b03a95..6cbaf413 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -1020,6 +1020,8 @@ def try_mirror_url(fetch, origud, ud, ld, check = False):
 
         if os.path.exists(ud.fullmirror):
             ensure_symlink(ud.fullmirror, origud.fullmirror)
+        if ud.shallow and os.path.exists(ud.fullshallow):
+            ensure_symlink(ud.fullshallow, origud.fullshallow)
 
         update_stamp(origud, ld)
         return ud.localpath
diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py
index 1cd4c65c..bad497b4 100644
--- a/lib/bb/tests/fetch.py
+++ b/lib/bb/tests/fetch.py
@@ -595,6 +595,21 @@ class GitShallowTarballNamingTest(FetcherTest):
         dir = os.listdir(self.dldir)
         self.assertIn(self.mirror_tarball, dir)
 
+    @skipIfNoNetwork()
+    def test_that_tarball_exists_for_mirrored_url_and_recipe_url_when_mirroring_is_used_and_the_mirrored_tarball_already_exists(self):
+        self.setup_mirror_rewrite()
+        fetcher = bb.fetch.Fetch([self.mirror_url], self.d)
+        fetcher.download()
+        bb.utils.prunedir(self.dldir + '/git2')
+        bb.utils.prunedir(self.unpackdir)
+        fetcher = bb.fetch.Fetch([self.recipe_url], self.d)
+
+        fetcher.download()
+
+        dir = os.listdir(self.dldir)
+        self.assertIn(self.mirror_tarball, dir)
+        self.assertIn(self.recipe_tarball, dir)
+
 
 class FetcherLocalTest(FetcherTest):
     def setUp(self):
@@ -1763,3 +1778,32 @@ class GitShallowTest(FetcherTest):
 
         dir = os.listdir(self.unpackdir + "/git/")
         self.assertIn("fstests.doap", dir)
+
+    @skipIfNoNetwork()
+    def test_that_unpack_uses_the_git_shallow_tarball_when_using_mirror_rewrite_rules_and_the_git_clone_does_not_exist(self):
+        self.d.setVar('SRCREV', 'e5939ff608b95cdd4d0ab0e1935781ab9a276ac0')
+        self.d.setVar('BB_GIT_SHALLOW', '1')
+        self.d.setVar('BB_GENERATE_SHALLOW_TARBALLS', '1')
+        self.d.setVar('PREMIRRORS', 'git://git.yoctoproject.org/.* git://git.yoctoproject.org/git/PATH;protocol=https \n')
+        fetcher = bb.fetch.Fetch(["git://git.yoctoproject.org/fstests"], self.d)
+        fetcher.download()
+        bb.utils.remove(self.dldir + '/git2', recurse=True)
+
+        fetcher.unpack(self.unpackdir)
+
+        dir = os.listdir(self.unpackdir + "/git/")
+        self.assertIn("fstests.doap", dir)
+
+    @skipIfNoNetwork()
+    def test_that_no_broken_symlinks_exist_when_using_mirror_rewrite_rules_and_mirror_tarball_is_configured(self):
+        self.d.setVar('SRCREV', 'e5939ff608b95cdd4d0ab0e1935781ab9a276ac0')
+        self.d.setVar('BB_GIT_SHALLOW', '1')
+        self.d.setVar('BB_GENERATE_MIRROR_TARBALLS', '1')
+        self.d.delVar('BB_GENERATE_SHALLOW_TARBALLS')
+        self.d.setVar('PREMIRRORS', 'git://git.yoctoproject.org/.* git://git.yoctoproject.org/git/PATH;protocol=https \n')
+        fetcher = bb.fetch.Fetch(["git://git.yoctoproject.org/fstests"], self.d)
+
+        fetcher.download()
+
+        broken_symlinks = get_broken_symlinks(self.dldir)
+        self.assertEqual([], broken_symlinks)
-- 
2.19.1


  parent reply	other threads:[~2018-10-15 19:16 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-15 13:43 [PATCH 0/7] Add symlinks for git (shallow) tarballs when name differs due to mirror rewrite rules Urs Fässler
2018-10-15 13:43 ` [PATCH 1/7] fetch2/git: add tests to capture existing behavior wrt. naming of clone directories Urs Fässler
2018-10-15 13:43 ` [PATCH 3/7] fetch2/git: add tests to capture existing behavior wrt. naming of git shallow tarball Urs Fässler
2018-10-15 13:43 ` [PATCH 2/7] fetch2/git: add tests to capture existing behavior wrt. naming of mirror tarball Urs Fässler
2018-10-15 13:43 ` [PATCH 4/7] fetch2/git: use intention revealing names for premirror tests Urs Fässler
2018-10-15 13:43 ` [PATCH 5/7] fetch2: extract the function which ensures that a valid symlink exists Urs Fässler
2018-10-15 13:43 ` [PATCH 6/7] fetch2: add symlinks for mirror tarballs Urs Fässler
2018-10-15 13:43 ` Urs Fässler [this message]
2018-10-16 20:19 ` [PATCH 0/7] Add symlinks for git (shallow) tarballs when name differs due to mirror rewrite rules Richard Purdie

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181015134255.4634-8-urs.fassler@bbv.ch \
    --to=urs.fassler@bbv.ch \
    --cc=bitbake-devel@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.