* [PATCH v3] fetch2: make sure the uri added as expected [not found] <168714731971476B.24733@lists.openembedded.org> @ 2021-09-13 5:48 ` Yu, Mingli 2021-10-26 23:24 ` [bitbake-devel] " Alexandre Belloni 0 siblings, 1 reply; 3+ messages in thread From: Yu, Mingli @ 2021-09-13 5:48 UTC (permalink / raw) To: bitbake-devel From: Mingli Yu <mingli.yu@windriver.com> When call uri_replace to get the new uris, we should guarantee the new uri is added when the replace uri begins with file if the find uri begin with wildcard which mapped as below. PREMIRRORS_append = " \ .*://.*/.* git://somewhere.org/somedir/MIRRORNAME;protocol=http \n \ " | | | | | | find replace Before the patch: the original uri gitsm://github.com/tianocore/edk2.git;branch=master;protocol=https PREMIRRORS_append = " \ .*://.*/.* git://somewhere.org/somedir/MIRRORNAME;protocol=http \n \ " The new uri is git://somewhere.org/somedir/github.com.tianocore.edk2.git/git2_github.com.tianocore.edk2.git.tar.gz;protocol=http After the patch: the original uri gitsm://github.com/tianocore/edk2.git;branch=master;protocol=https PREMIRRORS_append = " \ .*://.*/.* git://somewhere.org/somedir/MIRRORNAME;protocol=http \n \ " The new uri is None as the find begins with wildcard but the replace doesn't begin with file. Signed-off-by: Mingli Yu <mingli.yu@windriver.com> --- lib/bb/fetch2/__init__.py | 2 ++ lib/bb/tests/fetch.py | 8 +++++++- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py index d9e1599a..1c5242c7 100644 --- a/lib/bb/fetch2/__init__.py +++ b/lib/bb/fetch2/__init__.py @@ -430,6 +430,8 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None): uri_replace_decoded = list(decodeurl(uri_replace)) logger.debug2("For url %s comparing %s to %s" % (uri_decoded, uri_find_decoded, uri_replace_decoded)) result_decoded = ['', '', '', '', '', {}] + if uri_find_decoded[0] == '.*' and uri_replace_decoded[0] != 'file': + uri_find_decoded[0] = uri_replace_decoded[0] for loc, i in enumerate(uri_find_decoded): result_decoded[loc] = uri_decoded[loc] regexp = i diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index 242be368..25893a34 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -406,6 +406,8 @@ class MirrorUriTest(FetcherTest): : "git://somewhere.org/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", ("git://someserver.org/bitbake;tag=1234567890123456789012345678901234567890", "git://someserver.org/bitbake", "git://git.openembedded.org/bitbake") : "git://git.openembedded.org/bitbake;tag=1234567890123456789012345678901234567890", + ("gitsm://github.com/tianocore/edk2.git;branch=master;protocol=https", ".*://.*/.*", "git://somewhere.org/somedir/MIRRORNAME;protocol=http") + : None, ("file://sstate-xyz.tgz", "file://.*", "file:///somewhere/1234/sstate-cache") : "file:///somewhere/1234/sstate-cache/sstate-xyz.tgz", ("file://sstate-xyz.tgz", "file://.*", "file:///somewhere/1234/sstate-cache/") @@ -447,7 +449,11 @@ class MirrorUriTest(FetcherTest): ud.setup_localpath(self.d) mirrors = bb.fetch2.mirror_from_string("%s %s" % (k[1], k[2])) newuris, uds = bb.fetch2.build_mirroruris(ud, mirrors, self.d) - self.assertEqual([v], newuris) + if not v: + self.assertEqual([v], newuris) + else: + self.assertEqual([], newuris) + def test_urilist1(self): fetcher = bb.fetch.FetchData("http://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz", self.d) -- 2.32.0 ^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [bitbake-devel] [PATCH v3] fetch2: make sure the uri added as expected 2021-09-13 5:48 ` [PATCH v3] fetch2: make sure the uri added as expected Yu, Mingli @ 2021-10-26 23:24 ` Alexandre Belloni 2021-10-27 6:45 ` Yu, Mingli 0 siblings, 1 reply; 3+ messages in thread From: Alexandre Belloni @ 2021-10-26 23:24 UTC (permalink / raw) To: Yu, Mingli; +Cc: bitbake-devel Hello, I'm sorry, it took me a while to properly test this patch but I'm pretty sure it is the cause of: https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/2773/steps/11/logs/stdio On 13/09/2021 13:48:10+0800, Yu, Mingli wrote: > From: Mingli Yu <mingli.yu@windriver.com> > > When call uri_replace to get the new uris, we should guarantee the > new uri is added when the replace uri begins with file if the find > uri begin with wildcard which mapped as below. > PREMIRRORS_append = " \ > .*://.*/.* git://somewhere.org/somedir/MIRRORNAME;protocol=http \n \ > " | | > | | > | | > find replace > > Before the patch: > the original uri > gitsm://github.com/tianocore/edk2.git;branch=master;protocol=https > > PREMIRRORS_append = " \ > .*://.*/.* git://somewhere.org/somedir/MIRRORNAME;protocol=http \n \ > " > The new uri is git://somewhere.org/somedir/github.com.tianocore.edk2.git/git2_github.com.tianocore.edk2.git.tar.gz;protocol=http > > After the patch: > the original uri > gitsm://github.com/tianocore/edk2.git;branch=master;protocol=https > > PREMIRRORS_append = " \ > .*://.*/.* git://somewhere.org/somedir/MIRRORNAME;protocol=http \n \ > " > The new uri is None as the find begins with wildcard but the replace > doesn't begin with file. > > Signed-off-by: Mingli Yu <mingli.yu@windriver.com> > --- > lib/bb/fetch2/__init__.py | 2 ++ > lib/bb/tests/fetch.py | 8 +++++++- > 2 files changed, 9 insertions(+), 1 deletion(-) > > diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py > index d9e1599a..1c5242c7 100644 > --- a/lib/bb/fetch2/__init__.py > +++ b/lib/bb/fetch2/__init__.py > @@ -430,6 +430,8 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None): > uri_replace_decoded = list(decodeurl(uri_replace)) > logger.debug2("For url %s comparing %s to %s" % (uri_decoded, uri_find_decoded, uri_replace_decoded)) > result_decoded = ['', '', '', '', '', {}] > + if uri_find_decoded[0] == '.*' and uri_replace_decoded[0] != 'file': > + uri_find_decoded[0] = uri_replace_decoded[0] > for loc, i in enumerate(uri_find_decoded): > result_decoded[loc] = uri_decoded[loc] > regexp = i > diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py > index 242be368..25893a34 100644 > --- a/lib/bb/tests/fetch.py > +++ b/lib/bb/tests/fetch.py > @@ -406,6 +406,8 @@ class MirrorUriTest(FetcherTest): > : "git://somewhere.org/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", > ("git://someserver.org/bitbake;tag=1234567890123456789012345678901234567890", "git://someserver.org/bitbake", "git://git.openembedded.org/bitbake") > : "git://git.openembedded.org/bitbake;tag=1234567890123456789012345678901234567890", > + ("gitsm://github.com/tianocore/edk2.git;branch=master;protocol=https", ".*://.*/.*", "git://somewhere.org/somedir/MIRRORNAME;protocol=http") > + : None, > ("file://sstate-xyz.tgz", "file://.*", "file:///somewhere/1234/sstate-cache") > : "file:///somewhere/1234/sstate-cache/sstate-xyz.tgz", > ("file://sstate-xyz.tgz", "file://.*", "file:///somewhere/1234/sstate-cache/") > @@ -447,7 +449,11 @@ class MirrorUriTest(FetcherTest): > ud.setup_localpath(self.d) > mirrors = bb.fetch2.mirror_from_string("%s %s" % (k[1], k[2])) > newuris, uds = bb.fetch2.build_mirroruris(ud, mirrors, self.d) > - self.assertEqual([v], newuris) > + if not v: > + self.assertEqual([v], newuris) > + else: > + self.assertEqual([], newuris) > + > > def test_urilist1(self): > fetcher = bb.fetch.FetchData("http://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz", self.d) > -- > 2.32.0 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#12637): https://lists.openembedded.org/g/bitbake-devel/message/12637 > Mute This Topic: https://lists.openembedded.org/mt/85568847/3617179 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [alexandre.belloni@bootlin.com] > -=-=-=-=-=-=-=-=-=-=-=- > -- Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [bitbake-devel] [PATCH v3] fetch2: make sure the uri added as expected 2021-10-26 23:24 ` [bitbake-devel] " Alexandre Belloni @ 2021-10-27 6:45 ` Yu, Mingli 0 siblings, 0 replies; 3+ messages in thread From: Yu, Mingli @ 2021-10-27 6:45 UTC (permalink / raw) To: Alexandre Belloni; +Cc: bitbake-devel On 10/27/21 7:24 AM, Alexandre Belloni wrote: > [Please note: This e-mail is from an EXTERNAL e-mail address] > > Hello, > > I'm sorry, it took me a while to properly test this patch but I'm pretty > sure it is the cause of: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/2773/steps/11/logs/stdio Thanks for your following up! I will look into this. Thanks, > > On 13/09/2021 13:48:10+0800, Yu, Mingli wrote: >> From: Mingli Yu <mingli.yu@windriver.com> >> >> When call uri_replace to get the new uris, we should guarantee the >> new uri is added when the replace uri begins with file if the find >> uri begin with wildcard which mapped as below. >> PREMIRRORS_append = " \ >> .*://.*/.* git://somewhere.org/somedir/MIRRORNAME;protocol=http \n \ >> " | | >> | | >> | | >> find replace >> >> Before the patch: >> the original uri >> gitsm://github.com/tianocore/edk2.git;branch=master;protocol=https >> >> PREMIRRORS_append = " \ >> .*://.*/.* git://somewhere.org/somedir/MIRRORNAME;protocol=http \n \ >> " >> The new uri is git://somewhere.org/somedir/github.com.tianocore.edk2.git/git2_github.com.tianocore.edk2.git.tar.gz;protocol=http >> >> After the patch: >> the original uri >> gitsm://github.com/tianocore/edk2.git;branch=master;protocol=https >> >> PREMIRRORS_append = " \ >> .*://.*/.* git://somewhere.org/somedir/MIRRORNAME;protocol=http \n \ >> " >> The new uri is None as the find begins with wildcard but the replace >> doesn't begin with file. >> >> Signed-off-by: Mingli Yu <mingli.yu@windriver.com> >> --- >> lib/bb/fetch2/__init__.py | 2 ++ >> lib/bb/tests/fetch.py | 8 +++++++- >> 2 files changed, 9 insertions(+), 1 deletion(-) >> >> diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py >> index d9e1599a..1c5242c7 100644 >> --- a/lib/bb/fetch2/__init__.py >> +++ b/lib/bb/fetch2/__init__.py >> @@ -430,6 +430,8 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None): >> uri_replace_decoded = list(decodeurl(uri_replace)) >> logger.debug2("For url %s comparing %s to %s" % (uri_decoded, uri_find_decoded, uri_replace_decoded)) >> result_decoded = ['', '', '', '', '', {}] >> + if uri_find_decoded[0] == '.*' and uri_replace_decoded[0] != 'file': >> + uri_find_decoded[0] = uri_replace_decoded[0] >> for loc, i in enumerate(uri_find_decoded): >> result_decoded[loc] = uri_decoded[loc] >> regexp = i >> diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py >> index 242be368..25893a34 100644 >> --- a/lib/bb/tests/fetch.py >> +++ b/lib/bb/tests/fetch.py >> @@ -406,6 +406,8 @@ class MirrorUriTest(FetcherTest): >> : "git://somewhere.org/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", >> ("git://someserver.org/bitbake;tag=1234567890123456789012345678901234567890", "git://someserver.org/bitbake", "git://git.openembedded.org/bitbake") >> : "git://git.openembedded.org/bitbake;tag=1234567890123456789012345678901234567890", >> + ("gitsm://github.com/tianocore/edk2.git;branch=master;protocol=https", ".*://.*/.*", "git://somewhere.org/somedir/MIRRORNAME;protocol=http") >> + : None, >> ("file://sstate-xyz.tgz", "file://.*", "file:///somewhere/1234/sstate-cache") >> : "file:///somewhere/1234/sstate-cache/sstate-xyz.tgz", >> ("file://sstate-xyz.tgz", "file://.*", "file:///somewhere/1234/sstate-cache/") >> @@ -447,7 +449,11 @@ class MirrorUriTest(FetcherTest): >> ud.setup_localpath(self.d) >> mirrors = bb.fetch2.mirror_from_string("%s %s" % (k[1], k[2])) >> newuris, uds = bb.fetch2.build_mirroruris(ud, mirrors, self.d) >> - self.assertEqual([v], newuris) >> + if not v: >> + self.assertEqual([v], newuris) >> + else: >> + self.assertEqual([], newuris) >> + >> >> def test_urilist1(self): >> fetcher = bb.fetch.FetchData("http://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz", self.d) >> -- >> 2.32.0 >> > >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#12637): https://lists.openembedded.org/g/bitbake-devel/message/12637 >> Mute This Topic: https://lists.openembedded.org/mt/85568847/3617179 >> Group Owner: bitbake-devel+owner@lists.openembedded.org >> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [alexandre.belloni@bootlin.com] >> -=-=-=-=-=-=-=-=-=-=-=- >> > > > -- > Alexandre Belloni, co-owner and COO, Bootlin > Embedded Linux and Kernel engineering > https://bootlin.com > ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-10-27 6:39 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <168714731971476B.24733@lists.openembedded.org>
2021-09-13 5:48 ` [PATCH v3] fetch2: make sure the uri added as expected Yu, Mingli
2021-10-26 23:24 ` [bitbake-devel] " Alexandre Belloni
2021-10-27 6:45 ` Yu, Mingli
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.