* [bitbake-devel][PATCH V2 1/3] fetch2: fix downloadfilename issue with premirror
@ 2021-10-18 3:45 Chen Qi
2021-10-18 3:45 ` [bitbake-devel][PATCH V2 2/3] tests/fetch.py: fix premirror test cases Chen Qi
2021-10-18 3:45 ` [bitbake-devel][PATCH V2 3/3] tests/fetch.py: add test case to ensure downloadfilename is used for premirror Chen Qi
0 siblings, 2 replies; 3+ messages in thread
From: Chen Qi @ 2021-10-18 3:45 UTC (permalink / raw)
To: bitbake-devel
The following commit to fix [Yocto #13039] causes regression of
the behavior of PREMIRRORS.
"bitbake: fetch2: fix premirror URI when downloadfilename defined"
Take meta-openembedded/meta-networking/recipes-protocols/freediameter/freediameter_1.4.0.bb
as an example.
SRC_URI = "\
http://www.freediameter.net/hg/${fd_pkgname}/archive/${PV}.tar.gz;downloadfilename=${fd_pkgname}-${PV}.tar.gz \
...
"
With the above commit, it now tries to fetch 1.4.0.tar.gz instead of
freeDiameter-1.4.0.tar.gz. This makes https://downloads.yoctoproject.org/mirror/sources
not work for freediameter, as it holds freeDiameter-1.4.0.tar.gz.
The commit above tries to avoid fetching from invalid url such as:
https://<some_mirror>/1.4.0.tar.gz/freeDiameter-1.4.0.tar.gz.
And its solution is to make basename to be 1.4.0.tar.gz, thus causing the
regression.
This patch fixes the above regression. For Yocto #13039, it now tries
to fetch from url: https://<some_mirror>/freeDiameter-1.4.0.tar.gz.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
bitbake/lib/bb/fetch2/__init__.py | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index 666cc1306a..000b49a500 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -466,9 +466,13 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
# Kill parameters, they make no sense for mirror tarballs
uri_decoded[5] = {}
elif ud.localpath and ud.method.supports_checksum(ud):
- basename = os.path.basename(uri_decoded[loc])
- if basename and not result_decoded[loc].endswith(basename):
- result_decoded[loc] = os.path.join(result_decoded[loc], basename)
+ basename = os.path.basename(ud.localpath)
+ if basename:
+ uri_basename = os.path.basename(uri_decoded[loc])
+ if basename != uri_basename and result_decoded[loc].endswith(uri_basename):
+ result_decoded[loc] = result_decoded[loc].replace(uri_basename, basename)
+ elif not result_decoded[loc].endswith(basename):
+ result_decoded[loc] = os.path.join(result_decoded[loc], basename)
else:
return None
result = encodeurl(result_decoded)
--
2.33.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [bitbake-devel][PATCH V2 2/3] tests/fetch.py: fix premirror test cases
2021-10-18 3:45 [bitbake-devel][PATCH V2 1/3] fetch2: fix downloadfilename issue with premirror Chen Qi
@ 2021-10-18 3:45 ` Chen Qi
2021-10-18 3:45 ` [bitbake-devel][PATCH V2 3/3] tests/fetch.py: add test case to ensure downloadfilename is used for premirror Chen Qi
1 sibling, 0 replies; 3+ messages in thread
From: Chen Qi @ 2021-10-18 3:45 UTC (permalink / raw)
To: bitbake-devel
When downloadfilename is specified, it is used to fetch from premirror.
So fix the test cases accordingly.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
bitbake/lib/bb/tests/fetch.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py
index 8ad1c85990..d06580db88 100644
--- a/bitbake/lib/bb/tests/fetch.py
+++ b/bitbake/lib/bb/tests/fetch.py
@@ -875,17 +875,17 @@ class FetcherNetworkTest(FetcherTest):
@skipIfNoNetwork()
def test_fetch_premirror_specify_downloadfilename_regex_uri(self):
self.d.setVar("PREMIRRORS", "http://.*/.* https://downloads.yoctoproject.org/releases/bitbake/")
- fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz;downloadfilename=bitbake-v1.0.0.tar.gz"], self.d)
+ fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/1.0.tar.gz;downloadfilename=bitbake-1.0.tar.gz"], self.d)
fetcher.download()
- self.assertEqual(os.path.getsize(self.dldir + "/bitbake-v1.0.0.tar.gz"), 57749)
+ self.assertEqual(os.path.getsize(self.dldir + "/bitbake-1.0.tar.gz"), 57749)
@skipIfNoNetwork()
# BZ13039
def test_fetch_premirror_specify_downloadfilename_specific_uri(self):
self.d.setVar("PREMIRRORS", "http://invalid.yoctoproject.org/releases/bitbake https://downloads.yoctoproject.org/releases/bitbake")
- fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz;downloadfilename=bitbake-v1.0.0.tar.gz"], self.d)
+ fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/1.0.tar.gz;downloadfilename=bitbake-1.0.tar.gz"], self.d)
fetcher.download()
- self.assertEqual(os.path.getsize(self.dldir + "/bitbake-v1.0.0.tar.gz"), 57749)
+ self.assertEqual(os.path.getsize(self.dldir + "/bitbake-1.0.tar.gz"), 57749)
@skipIfNoNetwork()
def gitfetcher(self, url1, url2):
--
2.33.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [bitbake-devel][PATCH V2 3/3] tests/fetch.py: add test case to ensure downloadfilename is used for premirror
2021-10-18 3:45 [bitbake-devel][PATCH V2 1/3] fetch2: fix downloadfilename issue with premirror Chen Qi
2021-10-18 3:45 ` [bitbake-devel][PATCH V2 2/3] tests/fetch.py: fix premirror test cases Chen Qi
@ 2021-10-18 3:45 ` Chen Qi
1 sibling, 0 replies; 3+ messages in thread
From: Chen Qi @ 2021-10-18 3:45 UTC (permalink / raw)
To: bitbake-devel
Add a test case test_fetch_premirror_use_downloadfilename_to_fetch to ensure
that 'downloadfilename' is used when fetching from premirror.
Although the other two previous test cases, test_fetch_premirror_specify_downloadfilename_regex_uri
and test_fetch_premirror_specify_downloadfilename_specific_uri already
implicitly contain such verification, we still need to add a very clear
case to ensure no regression.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
bitbake/lib/bb/tests/fetch.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py
index d06580db88..a008e7cc5b 100644
--- a/bitbake/lib/bb/tests/fetch.py
+++ b/bitbake/lib/bb/tests/fetch.py
@@ -887,6 +887,14 @@ class FetcherNetworkTest(FetcherTest):
fetcher.download()
self.assertEqual(os.path.getsize(self.dldir + "/bitbake-1.0.tar.gz"), 57749)
+ @skipIfNoNetwork()
+ def test_fetch_premirror_use_downloadfilename_to_fetch(self):
+ # Ensure downloadfilename is used when fetching from premirror.
+ self.d.setVar("PREMIRRORS", "http://.*/.* https://downloads.yoctoproject.org/releases/bitbake")
+ fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/bitbake-1.1.tar.gz;downloadfilename=bitbake-1.0.tar.gz"], self.d)
+ fetcher.download()
+ self.assertEqual(os.path.getsize(self.dldir + "/bitbake-1.0.tar.gz"), 57749)
+
@skipIfNoNetwork()
def gitfetcher(self, url1, url2):
def checkrevision(self, fetcher):
--
2.33.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-10-18 3:45 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-18 3:45 [bitbake-devel][PATCH V2 1/3] fetch2: fix downloadfilename issue with premirror Chen Qi
2021-10-18 3:45 ` [bitbake-devel][PATCH V2 2/3] tests/fetch.py: fix premirror test cases Chen Qi
2021-10-18 3:45 ` [bitbake-devel][PATCH V2 3/3] tests/fetch.py: add test case to ensure downloadfilename is used for premirror Chen Qi
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.