Re: [bitbake-devel][PATCH V2 1/3] fetch2: fix downloadfilename issue with premirror

Re: [bitbake-devel][PATCH V2 1/3] fetch2: fix downloadfilename issue with premirror

[ChenQi]

[-- Attachment #1: Type: text/plain, Size: 3073 bytes --]

Gentle ping on this series.

Regards,
Qi

On 10/18/2021 11:45 AM, Chen Qi wrote:
> The following commit to fix [Yocto #13039] causes regression of
> the behavior of PREMIRRORS.
>
>    "bitbake: fetch2: fix premirror URI when downloadfilename defined"
>
> Take meta-openembedded/meta-networking/recipes-protocols/freediameter/freediameter_1.4.0.bb
> as an example.
> SRC_URI = "\
>      http://www.freediameter.net/hg/${fd_pkgname}/archive/${PV}.tar.gz;downloadfilename=${fd_pkgname}-${PV}.tar.gz \
>      ...
> "
> With the above commit, it now tries to fetch 1.4.0.tar.gz instead of
> freeDiameter-1.4.0.tar.gz. This makes https://downloads.yoctoproject.org/mirror/sources
> not work for freediameter, as it holds freeDiameter-1.4.0.tar.gz.
>
> The commit above tries to avoid fetching from invalid url such as:
> https://<some_mirror>/1.4.0.tar.gz/freeDiameter-1.4.0.tar.gz.
> And its solution is to make basename to be 1.4.0.tar.gz, thus causing the
> regression.
>
> This patch fixes the above regression. For Yocto #13039, it now tries
> to fetch from url: https://<some_mirror>/freeDiameter-1.4.0.tar.gz.
>
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
>   bitbake/lib/bb/fetch2/__init__.py | 10 +++++++---
>   1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
> index 666cc1306a..000b49a500 100644
> --- a/bitbake/lib/bb/fetch2/__init__.py
> +++ b/bitbake/lib/bb/fetch2/__init__.py
> @@ -466,9 +466,13 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
>                       # Kill parameters, they make no sense for mirror tarballs
>                       uri_decoded[5] = {}
>                   elif ud.localpath and ud.method.supports_checksum(ud):
> -                    basename = os.path.basename(uri_decoded[loc])
> -                if basename and not result_decoded[loc].endswith(basename):
> -                    result_decoded[loc] = os.path.join(result_decoded[loc], basename)
> +                    basename = os.path.basename(ud.localpath)
> +                if basename:
> +                    uri_basename = os.path.basename(uri_decoded[loc])
> +                    if basename != uri_basename and result_decoded[loc].endswith(uri_basename):
> +                        result_decoded[loc] = result_decoded[loc].replace(uri_basename, basename)
> +                    elif not result_decoded[loc].endswith(basename):
> +                        result_decoded[loc] = os.path.join(result_decoded[loc], basename)
>           else:
>               return None
>       result = encodeurl(result_decoded)
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12790): https://lists.openembedded.org/g/bitbake-devel/message/12790
> Mute This Topic: https://lists.openembedded.org/mt/86406437/3618072
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [Qi.Chen@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


[-- Attachment #2: Type: text/html, Size: 4314 bytes --]

Re: [bitbake-devel][PATCH V2 1/3] fetch2: fix downloadfilename issue with premirror

[Alexandre Belloni]

Hi,

On 27/10/2021 09:59:54+0800, Chen Qi wrote:
> Gentle ping on this series.
> 

It is currently under testing and will be in the next queue for Richard
to review and apply.

> Regards,
> Qi
> 
> On 10/18/2021 11:45 AM, Chen Qi wrote:
> > The following commit to fix [Yocto #13039] causes regression of
> > the behavior of PREMIRRORS.
> > 
> >    "bitbake: fetch2: fix premirror URI when downloadfilename defined"
> > 
> > Take meta-openembedded/meta-networking/recipes-protocols/freediameter/freediameter_1.4.0.bb
> > as an example.
> > SRC_URI = "\
> >      http://www.freediameter.net/hg/${fd_pkgname}/archive/${PV}.tar.gz;downloadfilename=${fd_pkgname}-${PV}.tar.gz \
> >      ...
> > "
> > With the above commit, it now tries to fetch 1.4.0.tar.gz instead of
> > freeDiameter-1.4.0.tar.gz. This makes https://downloads.yoctoproject.org/mirror/sources
> > not work for freediameter, as it holds freeDiameter-1.4.0.tar.gz.
> > 
> > The commit above tries to avoid fetching from invalid url such as:
> > https://<some_mirror>/1.4.0.tar.gz/freeDiameter-1.4.0.tar.gz.
> > And its solution is to make basename to be 1.4.0.tar.gz, thus causing the
> > regression.
> > 
> > This patch fixes the above regression. For Yocto #13039, it now tries
> > to fetch from url: https://<some_mirror>/freeDiameter-1.4.0.tar.gz.
> > 
> > Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> > ---
> >   bitbake/lib/bb/fetch2/__init__.py | 10 +++++++---
> >   1 file changed, 7 insertions(+), 3 deletions(-)
> > 
> > diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
> > index 666cc1306a..000b49a500 100644
> > --- a/bitbake/lib/bb/fetch2/__init__.py
> > +++ b/bitbake/lib/bb/fetch2/__init__.py
> > @@ -466,9 +466,13 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
> >                       # Kill parameters, they make no sense for mirror tarballs
> >                       uri_decoded[5] = {}
> >                   elif ud.localpath and ud.method.supports_checksum(ud):
> > -                    basename = os.path.basename(uri_decoded[loc])
> > -                if basename and not result_decoded[loc].endswith(basename):
> > -                    result_decoded[loc] = os.path.join(result_decoded[loc], basename)
> > +                    basename = os.path.basename(ud.localpath)
> > +                if basename:
> > +                    uri_basename = os.path.basename(uri_decoded[loc])
> > +                    if basename != uri_basename and result_decoded[loc].endswith(uri_basename):
> > +                        result_decoded[loc] = result_decoded[loc].replace(uri_basename, basename)
> > +                    elif not result_decoded[loc].endswith(basename):
> > +                        result_decoded[loc] = os.path.join(result_decoded[loc], basename)
> >           else:
> >               return None
> >       result = encodeurl(result_decoded)
> > 
> > 
> > 
> > 
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#12840): https://lists.openembedded.org/g/bitbake-devel/message/12840
> Mute This Topic: https://lists.openembedded.org/mt/86406437/3617179
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[bitbake-devel][PATCH V2 1/3] fetch2: fix downloadfilename issue with premirror

[Chen Qi]

The following commit to fix [Yocto #13039] causes regression of
the behavior of PREMIRRORS.

  "bitbake: fetch2: fix premirror URI when downloadfilename defined"

Take meta-openembedded/meta-networking/recipes-protocols/freediameter/freediameter_1.4.0.bb
as an example.
SRC_URI = "\
    http://www.freediameter.net/hg/${fd_pkgname}/archive/${PV}.tar.gz;downloadfilename=${fd_pkgname}-${PV}.tar.gz \
    ...
"
With the above commit, it now tries to fetch 1.4.0.tar.gz instead of
freeDiameter-1.4.0.tar.gz. This makes https://downloads.yoctoproject.org/mirror/sources
not work for freediameter, as it holds freeDiameter-1.4.0.tar.gz.

The commit above tries to avoid fetching from invalid url such as:
https://<some_mirror>/1.4.0.tar.gz/freeDiameter-1.4.0.tar.gz.
And its solution is to make basename to be 1.4.0.tar.gz, thus causing the
regression.

This patch fixes the above regression. For Yocto #13039, it now tries
to fetch from url: https://<some_mirror>/freeDiameter-1.4.0.tar.gz.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 bitbake/lib/bb/fetch2/__init__.py | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index 666cc1306a..000b49a500 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -466,9 +466,13 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
                     # Kill parameters, they make no sense for mirror tarballs
                     uri_decoded[5] = {}
                 elif ud.localpath and ud.method.supports_checksum(ud):
-                    basename = os.path.basename(uri_decoded[loc])
-                if basename and not result_decoded[loc].endswith(basename):
-                    result_decoded[loc] = os.path.join(result_decoded[loc], basename)
+                    basename = os.path.basename(ud.localpath)
+                if basename:
+                    uri_basename = os.path.basename(uri_decoded[loc])
+                    if basename != uri_basename and result_decoded[loc].endswith(uri_basename):
+                        result_decoded[loc] = result_decoded[loc].replace(uri_basename, basename)
+                    elif not result_decoded[loc].endswith(basename):
+                        result_decoded[loc] = os.path.join(result_decoded[loc], basename)
         else:
             return None
     result = encodeurl(result_decoded)
-- 
2.33.0