All of lore.kernel.org
 help / color / mirror / Atom feed
* [bitbake-devel][0/3] BZ#13039: updates to fetch2 and tests/fetch
@ 2021-09-05 22:27 Scott Weaver
  2021-09-05 22:27 ` [bitbake-devel][ 1/3] bitbake: fetch2: fix premirror URI when downloadfilename defined Scott Weaver
                   ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: Scott Weaver @ 2021-09-05 22:27 UTC (permalink / raw)
  To: bitbake-devel; +Cc: Scott Weaver

In this patch set, I address BZ#13039 and add fetcher tests specifically
for downloadfilename use cases along with an update to the npm fetcher tests.

https://bugzilla.yoctoproject.org/show_bug.cgi?id=13039

Scott Weaver (3):
  bitbake: fetch2: fix premirror URI when downloadfilename defined
  bitbake: tests/fetch: add downloadfilename tests
  bitbake: tests/fetch: add and fix npm tests

 bitbake/lib/bb/fetch2/__init__.py |  2 +-
 bitbake/lib/bb/tests/fetch.py     | 47 +++++++++++++++++++++++++++++--
 2 files changed, 46 insertions(+), 3 deletions(-)

-- 
2.25.1


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [bitbake-devel][ 1/3] bitbake: fetch2: fix premirror URI when downloadfilename defined
  2021-09-05 22:27 [bitbake-devel][0/3] BZ#13039: updates to fetch2 and tests/fetch Scott Weaver
@ 2021-09-05 22:27 ` Scott Weaver
  2021-09-22  1:58   ` kai
                     ` (2 more replies)
  2021-09-05 22:27 ` [bitbake-devel][ 2/3] bitbake: tests/fetch: add downloadfilename tests Scott Weaver
  2021-09-05 22:27 ` [bitbake-devel][ 3/3] bitbake: tests/fetch: add and fix npm tests Scott Weaver
  2 siblings, 3 replies; 9+ messages in thread
From: Scott Weaver @ 2021-09-05 22:27 UTC (permalink / raw)
  To: bitbake-devel; +Cc: Scott Weaver

When downloadfilename is defined in a recipe's SRC_URI and PREMIRRORS is also
defined using the same URI, the downloadfilename is appended to the mirror
URI and it should not be.

[YOCTO #13039]

Signed-off-by: Scott Weaver <weaverjs@gmail.com>
---
 bitbake/lib/bb/fetch2/__init__.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index 914fa5c024..47a4943369 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -466,7 +466,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
                     # Kill parameters, they make no sense for mirror tarballs
                     uri_decoded[5] = {}
                 elif ud.localpath and ud.method.supports_checksum(ud):
-                    basename = os.path.basename(ud.localpath)
+                    basename = os.path.basename(uri_decoded[loc])
                 if basename and not result_decoded[loc].endswith(basename):
                     result_decoded[loc] = os.path.join(result_decoded[loc], basename)
         else:
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [bitbake-devel][ 2/3] bitbake: tests/fetch: add downloadfilename tests
  2021-09-05 22:27 [bitbake-devel][0/3] BZ#13039: updates to fetch2 and tests/fetch Scott Weaver
  2021-09-05 22:27 ` [bitbake-devel][ 1/3] bitbake: fetch2: fix premirror URI when downloadfilename defined Scott Weaver
@ 2021-09-05 22:27 ` Scott Weaver
  2021-09-05 22:27 ` [bitbake-devel][ 3/3] bitbake: tests/fetch: add and fix npm tests Scott Weaver
  2 siblings, 0 replies; 9+ messages in thread
From: Scott Weaver @ 2021-09-05 22:27 UTC (permalink / raw)
  To: bitbake-devel; +Cc: Scott Weaver

This adds three new tests which evaluate different use cases of the
downloadfilename property.

bb.tests.fetch.FetcherNetworkTest:
	- test_fetch_specify_downloadfilename
	- test_fetch_premirror_specify_downloadfilename_regex_uri
	- test_fetch_premirror_specify_downloadfilename_specific_uri

Signed-off-by: Scott Weaver <weaverjs@gmail.com>
---
 bitbake/lib/bb/tests/fetch.py | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py
index 9291ce4a06..1735d0b071 100644
--- a/bitbake/lib/bb/tests/fetch.py
+++ b/bitbake/lib/bb/tests/fetch.py
@@ -866,6 +866,27 @@ class FetcherNetworkTest(FetcherTest):
         fetcher.download()
         self.assertEqual(os.path.getsize(self.dldir + "/bitbake-1.0.tar.gz"), 57749)
 
+    @skipIfNoNetwork()
+    def test_fetch_specify_downloadfilename(self):
+        fetcher = bb.fetch.Fetch(["http://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz;downloadfilename=bitbake-v1.0.0.tar.gz"], self.d)
+        fetcher.download()
+        self.assertEqual(os.path.getsize(self.dldir + "/bitbake-v1.0.0.tar.gz"), 57749)
+
+    @skipIfNoNetwork()
+    def test_fetch_premirror_specify_downloadfilename_regex_uri(self):
+        self.d.setVar("PREMIRRORS", "http://.*/.* http://downloads.yoctoproject.org/releases/bitbake/")
+        fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz;downloadfilename=bitbake-v1.0.0.tar.gz"], self.d)
+        fetcher.download()
+        self.assertEqual(os.path.getsize(self.dldir + "/bitbake-v1.0.0.tar.gz"), 57749)
+
+    @skipIfNoNetwork()
+    # BZ13039
+    def test_fetch_premirror_specify_downloadfilename_specific_uri(self):
+        self.d.setVar("PREMIRRORS", "http://invalid.yoctoproject.org/releases/bitbake http://downloads.yoctoproject.org/releases/bitbake")
+        fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz;downloadfilename=bitbake-v1.0.0.tar.gz"], self.d)
+        fetcher.download()
+        self.assertEqual(os.path.getsize(self.dldir + "/bitbake-v1.0.0.tar.gz"), 57749)
+
     @skipIfNoNetwork()
     def gitfetcher(self, url1, url2):
         def checkrevision(self, fetcher):
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [bitbake-devel][ 3/3] bitbake: tests/fetch: add and fix npm tests
  2021-09-05 22:27 [bitbake-devel][0/3] BZ#13039: updates to fetch2 and tests/fetch Scott Weaver
  2021-09-05 22:27 ` [bitbake-devel][ 1/3] bitbake: fetch2: fix premirror URI when downloadfilename defined Scott Weaver
  2021-09-05 22:27 ` [bitbake-devel][ 2/3] bitbake: tests/fetch: add downloadfilename tests Scott Weaver
@ 2021-09-05 22:27 ` Scott Weaver
  2 siblings, 0 replies; 9+ messages in thread
From: Scott Weaver @ 2021-09-05 22:27 UTC (permalink / raw)
  To: bitbake-devel; +Cc: Scott Weaver

This adds one new test that verifies the use of a specific filename
when defined in PREMIRRORS.

bb.tests.fetch.NPMTest:
	- test_npm_premirrors_with_specified_filename

While testing bz#13039, it was found that test_npm_registry_alternate
fails with ENOTFOUND. This was corrected by using npmjs's public mirror.

The change to fetch2 for bz#13039 highlighted an issue with the
test_npm_premirrors test where the created file:// mirror was using the
downloadfilename rather than the tarball that is defined by the npm url.

Signed-off-by: Scott Weaver <weaverjs@gmail.com>
---
 bitbake/lib/bb/tests/fetch.py | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py
index 1735d0b071..242be36891 100644
--- a/bitbake/lib/bb/tests/fetch.py
+++ b/bitbake/lib/bb/tests/fetch.py
@@ -2278,9 +2278,10 @@ class NPMTest(FetcherTest):
         fetcher.download()
         self.assertTrue(os.path.exists(ud.localpath))
         # Setup the mirror
+        pkgname = os.path.basename(ud.proxy.urls[0].split(';')[0])
         mirrordir = os.path.join(self.tempdir, 'mirror')
         bb.utils.mkdirhier(mirrordir)
-        os.replace(ud.localpath, os.path.join(mirrordir, os.path.basename(ud.localpath)))
+        os.replace(ud.localpath, os.path.join(mirrordir, pkgname))
         self.d.setVar('PREMIRRORS', 'https?$://.*/.* file://%s/\n' % mirrordir)
         self.d.setVar('BB_FETCH_PREMIRRORONLY', '1')
         # Fetch again
@@ -2288,6 +2289,27 @@ class NPMTest(FetcherTest):
         fetcher.download()
         self.assertTrue(os.path.exists(ud.localpath))
 
+    @skipIfNoNpm()
+    @skipIfNoNetwork()
+    def test_npm_premirrors_with_specified_filename(self):
+        url = 'npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0'
+        # Fetch once to get a tarball
+        fetcher = bb.fetch.Fetch([url], self.d)
+        ud = fetcher.ud[fetcher.urls[0]]
+        fetcher.download()
+        self.assertTrue(os.path.exists(ud.localpath))
+        # Setup the mirror
+        mirrordir = os.path.join(self.tempdir, 'mirror')
+        bb.utils.mkdirhier(mirrordir)
+        mirrorfilename = os.path.join(mirrordir, os.path.basename(ud.localpath))
+        os.replace(ud.localpath, mirrorfilename)
+        self.d.setVar('PREMIRRORS', 'https?$://.*/.* file://%s\n' % mirrorfilename)
+        self.d.setVar('BB_FETCH_PREMIRRORONLY', '1')
+        # Fetch again
+        self.assertFalse(os.path.exists(ud.localpath))
+        fetcher.download()
+        self.assertTrue(os.path.exists(ud.localpath))
+
     @skipIfNoNpm()
     @skipIfNoNetwork()
     def test_npm_mirrors(self):
@@ -2350,7 +2372,7 @@ class NPMTest(FetcherTest):
     @skipIfNoNpm()
     @skipIfNoNetwork()
     def test_npm_registry_alternate(self):
-        url = 'npm://registry.freajs.org;package=@savoirfairelinux/node-server-example;version=1.0.0'
+        url = 'npm://skimdb.npmjs.com;package=@savoirfairelinux/node-server-example;version=1.0.0'
         fetcher = bb.fetch.Fetch([url], self.d)
         fetcher.download()
         fetcher.unpack(self.unpackdir)
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [bitbake-devel][ 1/3] bitbake: fetch2: fix premirror URI when downloadfilename defined
  2021-09-05 22:27 ` [bitbake-devel][ 1/3] bitbake: fetch2: fix premirror URI when downloadfilename defined Scott Weaver
@ 2021-09-22  1:58   ` kai
  2021-09-28  2:00     ` Scott Weaver
  2021-10-15  6:37   ` ChenQi
       [not found]   ` <16AE2029FEDEE475.7161@lists.openembedded.org>
  2 siblings, 1 reply; 9+ messages in thread
From: kai @ 2021-09-22  1:58 UTC (permalink / raw)
  To: Scott Weaver, bitbake-devel

[-- Attachment #1: Type: text/plain, Size: 1953 bytes --]

On 9/6/21 6:27 AM, Scott Weaver wrote:
> When downloadfilename is defined in a recipe's SRC_URI and PREMIRRORS is also
> defined using the same URI, the downloadfilename is appended to the mirror
> URI and it should not be.
>
> [YOCTO #13039]

As I commented in the bugzilla, I suppose it is an invalid bug.

It save the tarball with the name of the value of downloadfilename 
rather than the basename of the url.
In this case, the name of tarball in the premirror downloads/ directory 
is uthash-2.0.2.tar.gz rather than
2.0.2.tar.gz if the premirror is created by run bitbake do_fetch. So 
when premirror is used, it checks
uthash-2.0.2.tar.gz in the premirrors but not the original value 
'2.0.2.tar.gz'.

With this patch, it fails to fetch the tarball from premirrors. It could 
be reproduced by config:

BB_NO_NETWORK = "1"
BB_FETCH_PREMIRRORONLY = "1"

Regards,
Kai
>
> Signed-off-by: Scott Weaver <weaverjs@gmail.com>
> ---
>   bitbake/lib/bb/fetch2/__init__.py | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
> index 914fa5c024..47a4943369 100644
> --- a/bitbake/lib/bb/fetch2/__init__.py
> +++ b/bitbake/lib/bb/fetch2/__init__.py
> @@ -466,7 +466,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
>                       # Kill parameters, they make no sense for mirror tarballs
>                       uri_decoded[5] = {}
>                   elif ud.localpath and ud.method.supports_checksum(ud):
> -                    basename = os.path.basename(ud.localpath)
> +                    basename = os.path.basename(uri_decoded[loc])
>                   if basename and not result_decoded[loc].endswith(basename):
>                       result_decoded[loc] = os.path.join(result_decoded[loc], basename)
>           else:
>
> 
>

-- 
Kai Kang
Wind River Linux


[-- Attachment #2: Type: text/html, Size: 2792 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [bitbake-devel][ 1/3] bitbake: fetch2: fix premirror URI when downloadfilename defined
  2021-09-22  1:58   ` kai
@ 2021-09-28  2:00     ` Scott Weaver
  2021-10-14  9:01       ` Robert Yang
  0 siblings, 1 reply; 9+ messages in thread
From: Scott Weaver @ 2021-09-28  2:00 UTC (permalink / raw)
  To: Kai; +Cc: bitbake-devel

On 21-09-22 09:58:13, Kai wrote:
> On 9/6/21 6:27 AM, Scott Weaver wrote:
> > When downloadfilename is defined in a recipe's SRC_URI and PREMIRRORS is also
> > defined using the same URI, the downloadfilename is appended to the mirror
> > URI and it should not be.
> > 
> > [YOCTO #13039]
> 
> As I commented in the bugzilla, I suppose it is an invalid bug.
> 
> It save the tarball with the name of the value of downloadfilename rather
> than the basename of the url.
> In this case, the name of tarball in the premirror downloads/ directory is
> uthash-2.0.2.tar.gz rather than
> 2.0.2.tar.gz if the premirror is created by run bitbake do_fetch. So when
> premirror is used, it checks
> uthash-2.0.2.tar.gz in the premirrors but not the original value
> '2.0.2.tar.gz'.
> 
> With this patch, it fails to fetch the tarball from premirrors. It could be
> reproduced by config:
> 
> BB_NO_NETWORK = "1"
> BB_FETCH_PREMIRRORONLY = "1"
> 
> Regards,
> Kai

Sorry, Kai. I didn't see your email until just now.
I'll look into it. IMO, I would treat this like a new/different bug
than the one I addressed with this patch but regardless it is a
related use case to this BZ.

I'll update the BZ after I've dug into this.

 - Scott


> > 
> > Signed-off-by: Scott Weaver <weaverjs@gmail.com>
> > ---
> >   bitbake/lib/bb/fetch2/__init__.py | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
> > index 914fa5c024..47a4943369 100644
> > --- a/bitbake/lib/bb/fetch2/__init__.py
> > +++ b/bitbake/lib/bb/fetch2/__init__.py
> > @@ -466,7 +466,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
> >                       # Kill parameters, they make no sense for mirror tarballs
> >                       uri_decoded[5] = {}
> >                   elif ud.localpath and ud.method.supports_checksum(ud):
> > -                    basename = os.path.basename(ud.localpath)
> > +                    basename = os.path.basename(uri_decoded[loc])
> >                   if basename and not result_decoded[loc].endswith(basename):
> >                       result_decoded[loc] = os.path.join(result_decoded[loc], basename)
> >           else:
> > 
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#12592): https://lists.openembedded.org/g/bitbake-devel/message/12592
> > Mute This Topic: https://lists.openembedded.org/mt/85401309/3616933
> > Group Owner: bitbake-devel+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [kai.kang@windriver.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> > 
> 
> -- 
> Kai Kang
> Wind River Linux
> 

-- 
  - Scott



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [bitbake-devel][ 1/3] bitbake: fetch2: fix premirror URI when downloadfilename defined
  2021-09-28  2:00     ` Scott Weaver
@ 2021-10-14  9:01       ` Robert Yang
  0 siblings, 0 replies; 9+ messages in thread
From: Robert Yang @ 2021-10-14  9:01 UTC (permalink / raw)
  To: Scott Weaver; +Cc: bitbake-devel, Kai

Hi Scott,

On 9/28/21 10:00 AM, Scott Weaver wrote:
> On 21-09-22 09:58:13, Kai wrote:
>> On 9/6/21 6:27 AM, Scott Weaver wrote:
>>> When downloadfilename is defined in a recipe's SRC_URI and PREMIRRORS is also
>>> defined using the same URI, the downloadfilename is appended to the mirror
>>> URI and it should not be.
>>>
>>> [YOCTO #13039]
>>
>> As I commented in the bugzilla, I suppose it is an invalid bug.
>>
>> It save the tarball with the name of the value of downloadfilename rather
>> than the basename of the url.
>> In this case, the name of tarball in the premirror downloads/ directory is
>> uthash-2.0.2.tar.gz rather than
>> 2.0.2.tar.gz if the premirror is created by run bitbake do_fetch. So when
>> premirror is used, it checks
>> uthash-2.0.2.tar.gz in the premirrors but not the original value
>> '2.0.2.tar.gz'.
>>
>> With this patch, it fails to fetch the tarball from premirrors. It could be
>> reproduced by config:
>>
>> BB_NO_NETWORK = "1"
>> BB_FETCH_PREMIRRORONLY = "1"
>>
>> Regards,
>> Kai
> 
> Sorry, Kai. I didn't see your email until just now.
> I'll look into it. IMO, I would treat this like a new/different bug
> than the one I addressed with this patch but regardless it is a
> related use case to this BZ.
> 
> I'll update the BZ after I've dug into this.
> 
>   - Scott
> 
> 
>>>
>>> Signed-off-by: Scott Weaver <weaverjs@gmail.com>
>>> ---
>>>    bitbake/lib/bb/fetch2/__init__.py | 2 +-
>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
>>> index 914fa5c024..47a4943369 100644
>>> --- a/bitbake/lib/bb/fetch2/__init__.py
>>> +++ b/bitbake/lib/bb/fetch2/__init__.py
>>> @@ -466,7 +466,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
>>>                        # Kill parameters, they make no sense for mirror tarballs
>>>                        uri_decoded[5] = {}
>>>                    elif ud.localpath and ud.method.supports_checksum(ud):
>>> -                    basename = os.path.basename(ud.localpath)
>>> +                    basename = os.path.basename(uri_decoded[loc])

Use 
meta-openembedded/meta-networking/recipes-connectivity/dhcp/dhcp-relay_4.4.2p1.bb as 
an example:

* Before the patch, basename is bind.tar.gz
* After the patch, basename is bind-9.11.32.tar.gz

The patch changes basename from bind-9.11.32.tar.gz to bind.tar.gz, but didn't
remove downloadfilename=bind.tar.gz from the mirror URI. The commit message
says:

"the downloadfilename is appended to the mirror URI and it should not be."

But the downloadfilename is still in the mirror URI after the patch.

And the patch has a side effect is that it makes PREMIRROR or MIRRORS harder to
use, the saved filename in DL_DIR is bind.tar.gz, but the required name
in the mirror is bind-9.11.32.tar.gz since bind.tar.gz doesn't work any more,
which isn't as straightforward as before, people who uses DL_DIR as PREMIRRORS
doesn't work any more.

BTW, leave downloadfilename in the mirror URI won't affect anything AFAIK.

// Robert

>>>                    if basename and not result_decoded[loc].endswith(basename):
>>>                        result_decoded[loc] = os.path.join(result_decoded[loc], basename)
>>>            else:
>>>
>>>
>>>
>>
>> -- 
>> Kai Kang
>> Wind River Linux
>>
> 
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12711): https://lists.openembedded.org/g/bitbake-devel/message/12711
> Mute This Topic: https://lists.openembedded.org/mt/85401309/3616940
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [liezhi.yang@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [bitbake-devel][ 1/3] bitbake: fetch2: fix premirror URI when downloadfilename defined
  2021-09-05 22:27 ` [bitbake-devel][ 1/3] bitbake: fetch2: fix premirror URI when downloadfilename defined Scott Weaver
  2021-09-22  1:58   ` kai
@ 2021-10-15  6:37   ` ChenQi
       [not found]   ` <16AE2029FEDEE475.7161@lists.openembedded.org>
  2 siblings, 0 replies; 9+ messages in thread
From: ChenQi @ 2021-10-15  6:37 UTC (permalink / raw)
  To: Scott Weaver, bitbake-devel, Richard Purdie, Kang Kai, Robert Yang

[-- Attachment #1: Type: text/plain, Size: 2299 bytes --]

Hi Scott,

I think the key problem here is which tarball name to be used for premirror.
Previously, it's using 'downloadfilename'. So instead of holding 
1.4.0.tar.gz on yocto's mirror, it holds freeDiameter-1.4.0.tar.gz.
Now, with this patch, the basename of the URL (e.g. 1.4.0.tar.gz) is used.

This at least causes the regression that a local download/ directory 
could be serve as a premirror.

I've sent out a patch to restore the previous behavior, it should also 
solve yocto#13039.
The title is: [bitbake-devel][PATCH] fetch2: fix downloadfilename issue 
with premirror

Could you please check if it works for you?

Regards,
Qi

On 09/06/2021 06:27 AM, Scott Weaver wrote:
> When downloadfilename is defined in a recipe's SRC_URI and PREMIRRORS is also
> defined using the same URI, the downloadfilename is appended to the mirror
> URI and it should not be.
>
> [YOCTO #13039]
>
> Signed-off-by: Scott Weaver <weaverjs@gmail.com>
> ---
>   bitbake/lib/bb/fetch2/__init__.py | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
> index 914fa5c024..47a4943369 100644
> --- a/bitbake/lib/bb/fetch2/__init__.py
> +++ b/bitbake/lib/bb/fetch2/__init__.py
> @@ -466,7 +466,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
>                       # Kill parameters, they make no sense for mirror tarballs
>                       uri_decoded[5] = {}
>                   elif ud.localpath and ud.method.supports_checksum(ud):
> -                    basename = os.path.basename(ud.localpath)
> +                    basename = os.path.basename(uri_decoded[loc])
>                   if basename and not result_decoded[loc].endswith(basename):
>                       result_decoded[loc] = os.path.join(result_decoded[loc], basename)
>           else:
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12592): https://lists.openembedded.org/g/bitbake-devel/message/12592
> Mute This Topic: https://lists.openembedded.org/mt/85401309/3618072
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [Qi.Chen@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


[-- Attachment #2: Type: text/html, Size: 3424 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [bitbake-devel][ 1/3] bitbake: fetch2: fix premirror URI when downloadfilename defined
       [not found]   ` <16AE2029FEDEE475.7161@lists.openembedded.org>
@ 2021-10-15  6:40     ` ChenQi
  0 siblings, 0 replies; 9+ messages in thread
From: ChenQi @ 2021-10-15  6:40 UTC (permalink / raw)
  To: Scott Weaver, bitbake-devel, Richard Purdie, Kang Kai, Robert Yang

[-- Attachment #1: Type: text/plain, Size: 3257 bytes --]

On 10/15/2021 02:37 PM, Chen Qi wrote:
> Hi Scott,
>
> I think the key problem here is which tarball name to be used for 
> premirror.
> Previously, it's using 'downloadfilename'. So instead of holding 
> 1.4.0.tar.gz on yocto's mirror, it holds freeDiameter-1.4.0.tar.gz.
> Now, with this patch, the basename of the URL (e.g. 1.4.0.tar.gz) is used.
>
> This at least causes the regression that a local download/ directory 
> could be serve as a premirror.

Sorry, typo. Should be:

This at least causes the regression that a local download/ directory 
could not serve as a premirror.

>
> I've sent out a patch to restore the previous behavior, it should also 
> solve yocto#13039.
> The title is: [bitbake-devel][PATCH] fetch2: fix downloadfilename 
> issue with premirror
>
> Could you please check if it works for you?
>
> Regards,
> Qi
>
> On 09/06/2021 06:27 AM, Scott Weaver wrote:
>> When downloadfilename is defined in a recipe's SRC_URI and PREMIRRORS is also
>> defined using the same URI, the downloadfilename is appended to the mirror
>> URI and it should not be.
>>
>> [YOCTO #13039]
>>
>> Signed-off-by: Scott Weaver<weaverjs@gmail.com>
>> ---
>>   bitbake/lib/bb/fetch2/__init__.py 
>> <https://urldefense.com/v3/__http://__init__.py__;%21%21AjveYdw8EvQ%21NcbTHox-6oqYMaVTv6H3RVVTYA06zLNBPvkC6ZLg3Y6PRXUmmAyBKwEPXzGAFvB8Gw$>  | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/bitbake/lib/bb/fetch2/__init__.py 
>> <https://urldefense.com/v3/__http://__init__.py__;%21%21AjveYdw8EvQ%21NcbTHox-6oqYMaVTv6H3RVVTYA06zLNBPvkC6ZLg3Y6PRXUmmAyBKwEPXzGAFvB8Gw$>  b/bitbake/lib/bb/fetch2/__init__.py 
>> <https://urldefense.com/v3/__http://__init__.py__;%21%21AjveYdw8EvQ%21NcbTHox-6oqYMaVTv6H3RVVTYA06zLNBPvkC6ZLg3Y6PRXUmmAyBKwEPXzGAFvB8Gw$>
>> index 914fa5c024..47a4943369 100644
>> --- a/bitbake/lib/bb/fetch2/__init__.py 
>> <https://urldefense.com/v3/__http://__init__.py__;%21%21AjveYdw8EvQ%21NcbTHox-6oqYMaVTv6H3RVVTYA06zLNBPvkC6ZLg3Y6PRXUmmAyBKwEPXzGAFvB8Gw$>
>> +++ b/bitbake/lib/bb/fetch2/__init__.py 
>> <https://urldefense.com/v3/__http://__init__.py__;%21%21AjveYdw8EvQ%21NcbTHox-6oqYMaVTv6H3RVVTYA06zLNBPvkC6ZLg3Y6PRXUmmAyBKwEPXzGAFvB8Gw$>
>> @@ -466,7 +466,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
>>                       # Kill parameters, they make no sense for mirror tarballs
>>                       uri_decoded[5] = {}
>>                   elif ud.localpath and ud.method.supports_checksum(ud):
>> -                    basename = os.path.basename(ud.localpath)
>> +                    basename = os.path.basename(uri_decoded[loc])
>>                   if basename and not result_decoded[loc].endswith(basename):
>>                       result_decoded[loc] = os.path.join(result_decoded[loc], basename)
>>           else:
>>
>>
>>
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12780): https://lists.openembedded.org/g/bitbake-devel/message/12780
> Mute This Topic: https://lists.openembedded.org/mt/85401309/3618072
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [Qi.Chen@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


[-- Attachment #2: Type: text/html, Size: 5026 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2021-10-15  6:27 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-05 22:27 [bitbake-devel][0/3] BZ#13039: updates to fetch2 and tests/fetch Scott Weaver
2021-09-05 22:27 ` [bitbake-devel][ 1/3] bitbake: fetch2: fix premirror URI when downloadfilename defined Scott Weaver
2021-09-22  1:58   ` kai
2021-09-28  2:00     ` Scott Weaver
2021-10-14  9:01       ` Robert Yang
2021-10-15  6:37   ` ChenQi
     [not found]   ` <16AE2029FEDEE475.7161@lists.openembedded.org>
2021-10-15  6:40     ` ChenQi
2021-09-05 22:27 ` [bitbake-devel][ 2/3] bitbake: tests/fetch: add downloadfilename tests Scott Weaver
2021-09-05 22:27 ` [bitbake-devel][ 3/3] bitbake: tests/fetch: add and fix npm tests Scott Weaver

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.