* Fetch problems with PREMIRRORS @ 2011-02-09 13:40 Gary Thomas 2011-02-10 14:03 ` Richard Purdie 0 siblings, 1 reply; 4+ messages in thread From: Gary Thomas @ 2011-02-09 13:40 UTC (permalink / raw) To: Poky Per Richard's suggestion, I have added this to my DISTRO.conf: # Provide pre-staged sources PREMIRRORS = "\ http://.*/.* file:///work/misc/Poky/sources/ \n \ https://.*/.* file:///work/misc/Poky/sources/ \n \ ftp://.*/.* file:///work/misc/Poky/sources/ \n \ bzr://.*/.* file:///work/misc/Poky/sources/ \n \ cvs://.*/.* file:///work/misc/Poky/sources/ \n \ git://.*/.* file:///work/misc/Poky/sources/ \n \ hg://.*/.* file:///work/misc/Poky/sources/ \n \ osc://.*/.* file:///work/misc/Poky/sources/ \n \ p4://.*/.* file:///work/misc/Poky/sources/ \n \ svk://.*/.* file:///work/misc/Poky/sources/ \n \ svn://.*/.* file:///work/misc/Poky/sources/ \n" The tree '/work/misc/Poky/sources' is a fully populated source cache. This all works great as long as the needed sources are in the cache. However, there seems to be no fallback if they are not present. I get errors like this: | NOTE: fetch http://pokylinux.org/sources/git2_github.com.wrpseudo.pseudo.git.tar.gz | NOTE: fetch http://autobuilder.pokylinux.org/sources/git2_github.com.wrpseudo.pseudo.git.tar.gz | ERROR: Function 'Fetcher failure for URL: 'git://github.com/wrpseudo/pseudo.git;protocol=git'. Unable to fetch URL git://github.com/wrpseudo/pseudo.git;protocol=git from any source.' failed NOTE: package pseudo-native-0.0+git1+5434325fc887817ebb2bad36313d8277992cef1d-r18: task Fetcher failure for URL: 'git://github.com/wrpseudo/pseudo.git;protocol=git'. Unable to fetch URL git://github.com/wrpseudo/pseudo.git;protocol=git from any source.: Failed In order to actually fetch, I have to remove the PREMIRRORS from my conf. Is there some way to get it to use the source cache but still fetch when needed? Thanks Note: I also normally have BB_NO_NETWORK="1", but that is not applicable here (no message about it) and setting it false does not change the behaviour. -- ------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------ ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Fetch problems with PREMIRRORS 2011-02-09 13:40 Fetch problems with PREMIRRORS Gary Thomas @ 2011-02-10 14:03 ` Richard Purdie 2011-02-10 14:51 ` Gary Thomas 0 siblings, 1 reply; 4+ messages in thread From: Richard Purdie @ 2011-02-10 14:03 UTC (permalink / raw) To: Gary Thomas; +Cc: Poky On Wed, 2011-02-09 at 06:40 -0700, Gary Thomas wrote: > Per Richard's suggestion, I have added this to my DISTRO.conf: > > # Provide pre-staged sources > PREMIRRORS = "\ > http://.*/.* file:///work/misc/Poky/sources/ \n \ > https://.*/.* file:///work/misc/Poky/sources/ \n \ > ftp://.*/.* file:///work/misc/Poky/sources/ \n \ > bzr://.*/.* file:///work/misc/Poky/sources/ \n \ > cvs://.*/.* file:///work/misc/Poky/sources/ \n \ > git://.*/.* file:///work/misc/Poky/sources/ \n \ > hg://.*/.* file:///work/misc/Poky/sources/ \n \ > osc://.*/.* file:///work/misc/Poky/sources/ \n \ > p4://.*/.* file:///work/misc/Poky/sources/ \n \ > svk://.*/.* file:///work/misc/Poky/sources/ \n \ > svn://.*/.* file:///work/misc/Poky/sources/ \n" > > The tree '/work/misc/Poky/sources' is a fully populated source > cache. > > This all works great as long as the needed sources are in > the cache. However, there seems to be no fallback if they > are not present. I get errors like this: > | NOTE: fetch http://pokylinux.org/sources/git2_github.com.wrpseudo.pseudo.git.tar.gz > | NOTE: fetch http://autobuilder.pokylinux.org/sources/git2_github.com.wrpseudo.pseudo.git.tar.gz > | ERROR: Function 'Fetcher failure for URL: 'git://github.com/wrpseudo/pseudo.git;protocol=git'. Unable to fetch URL git://github.com/wrpseudo/pseudo.git;protocol=git from any > source.' failed > NOTE: package pseudo-native-0.0+git1+5434325fc887817ebb2bad36313d8277992cef1d-r18: task Fetcher failure for URL: 'git://github.com/wrpseudo/pseudo.git;protocol=git'. Unable to > fetch URL git://github.com/wrpseudo/pseudo.git;protocol=git from any source.: Failed > > In order to actually fetch, I have to remove the PREMIRRORS > from my conf. > > Is there some way to get it to use the source cache but > still fetch when needed? > > Thanks > > Note: I also normally have BB_NO_NETWORK="1", but that is not > applicable here (no message about it) and setting it false does > not change the behaviour. It is applicable. Try not setting that variable at all, not setting it false. I suspect there are two issues here: a) BB_NO_NETWORK is having an effect but the log messages are being swallowed... b) "false" handling for the variable might not be working properly so leave it unset. We can fix those things but I'd like to confirm these are the problems first. Cheers, Richard ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Fetch problems with PREMIRRORS 2011-02-10 14:03 ` Richard Purdie @ 2011-02-10 14:51 ` Gary Thomas 2011-02-11 21:54 ` Richard Purdie 0 siblings, 1 reply; 4+ messages in thread From: Gary Thomas @ 2011-02-10 14:51 UTC (permalink / raw) To: Richard Purdie; +Cc: Poky On 02/10/2011 07:03 AM, Richard Purdie wrote: > On Wed, 2011-02-09 at 06:40 -0700, Gary Thomas wrote: >> Per Richard's suggestion, I have added this to my DISTRO.conf: >> >> # Provide pre-staged sources >> PREMIRRORS = "\ >> http://.*/.* file:///work/misc/Poky/sources/ \n \ >> https://.*/.* file:///work/misc/Poky/sources/ \n \ >> ftp://.*/.* file:///work/misc/Poky/sources/ \n \ >> bzr://.*/.* file:///work/misc/Poky/sources/ \n \ >> cvs://.*/.* file:///work/misc/Poky/sources/ \n \ >> git://.*/.* file:///work/misc/Poky/sources/ \n \ >> hg://.*/.* file:///work/misc/Poky/sources/ \n \ >> osc://.*/.* file:///work/misc/Poky/sources/ \n \ >> p4://.*/.* file:///work/misc/Poky/sources/ \n \ >> svk://.*/.* file:///work/misc/Poky/sources/ \n \ >> svn://.*/.* file:///work/misc/Poky/sources/ \n" >> >> The tree '/work/misc/Poky/sources' is a fully populated source >> cache. >> >> This all works great as long as the needed sources are in >> the cache. However, there seems to be no fallback if they >> are not present. I get errors like this: >> | NOTE: fetch http://pokylinux.org/sources/git2_github.com.wrpseudo.pseudo.git.tar.gz >> | NOTE: fetch http://autobuilder.pokylinux.org/sources/git2_github.com.wrpseudo.pseudo.git.tar.gz >> | ERROR: Function 'Fetcher failure for URL: 'git://github.com/wrpseudo/pseudo.git;protocol=git'. Unable to fetch URL git://github.com/wrpseudo/pseudo.git;protocol=git from any >> source.' failed >> NOTE: package pseudo-native-0.0+git1+5434325fc887817ebb2bad36313d8277992cef1d-r18: task Fetcher failure for URL: 'git://github.com/wrpseudo/pseudo.git;protocol=git'. Unable to >> fetch URL git://github.com/wrpseudo/pseudo.git;protocol=git from any source.: Failed >> >> In order to actually fetch, I have to remove the PREMIRRORS >> from my conf. >> >> Is there some way to get it to use the source cache but >> still fetch when needed? >> >> Thanks >> >> Note: I also normally have BB_NO_NETWORK="1", but that is not >> applicable here (no message about it) and setting it false does >> not change the behaviour. > > It is applicable. > > Try not setting that variable at all, not setting it false. I suspect > there are two issues here: > > a) BB_NO_NETWORK is having an effect but the log messages are being > swallowed... > b) "false" handling for the variable might not be working properly so > leave it unset. > > We can fix those things but I'd like to confirm these are the problems > first. Correct, it does seem to be the case. When BB_NO_NETWORK="0", the fetcher works properly. When set to "1", it fails, but no explicit error about why is generated, neither on the console nor in the log.do_fetch file. I made this small change to check it out: diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py index ef9d75f..57c4f20 100644 --- a/bitbake/lib/bb/fetch2/__init__.py +++ b/bitbake/lib/bb/fetch2/__init__.py @@ -395,6 +395,7 @@ def check_network_access(d, info = ""): log remote network access, and error if BB_NO_NETWORK is set """ if bb.data.getVar("BB_NO_NETWORK", d, True) == "1": + bb.error("BB_NO_NETWORK is set, but the fetcher code attempted network access with the command %s" % info) raise FetchError("BB_NO_NETWORK is set, but the fetcher code attempted network access with the command %s" % inf else: logger.debug(1, "Fetcher accessed the network with the command %s" % info) This shows that this _is_ the error case, with the messages making it to the console and log file. I can see that this exception is being thrown on line 886, but the reason is lost in the exception data & I'm unclear how to retrieve it and present it to the user. -- ------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------ ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: Fetch problems with PREMIRRORS 2011-02-10 14:51 ` Gary Thomas @ 2011-02-11 21:54 ` Richard Purdie 0 siblings, 0 replies; 4+ messages in thread From: Richard Purdie @ 2011-02-11 21:54 UTC (permalink / raw) To: Gary Thomas; +Cc: Poky On Thu, 2011-02-10 at 07:51 -0700, Gary Thomas wrote: > > Try not setting that variable at all, not setting it false. I suspect > > there are two issues here: > > > > a) BB_NO_NETWORK is having an effect but the log messages are being > > swallowed... > > b) "false" handling for the variable might not be working properly so > > leave it unset. > > > > We can fix those things but I'd like to confirm these are the problems > > first. > > Correct, it does seem to be the case. When BB_NO_NETWORK="0", > the fetcher works properly. When set to "1", it fails, but > no explicit error about why is generated, neither on the console > nor in the log.do_fetch file. > > I made this small change to check it out: > diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py > index ef9d75f..57c4f20 100644 > --- a/bitbake/lib/bb/fetch2/__init__.py > +++ b/bitbake/lib/bb/fetch2/__init__.py > @@ -395,6 +395,7 @@ def check_network_access(d, info = ""): > log remote network access, and error if BB_NO_NETWORK is set > """ > if bb.data.getVar("BB_NO_NETWORK", d, True) == "1": > + bb.error("BB_NO_NETWORK is set, but the fetcher code attempted network access with the command %s" % info) > raise FetchError("BB_NO_NETWORK is set, but the fetcher code attempted network access with the command %s" % inf > else: > logger.debug(1, "Fetcher accessed the network with the command %s" % info) > > This shows that this _is_ the error case, with the messages making it to > the console and log file. I can see that this exception is being thrown > on line 886, but the reason is lost in the exception data & I'm unclear > how to retrieve it and present it to the user. This should be at least partly addressed with some of the recent commits such as: http://git.pokylinux.org/cgit.cgi/poky/commit/?id=ea70c4362fdb81bc9467975c145c48196c45e3af so the debug logs (with -D) at least show whats happening. Not perfect but better at least... Cheers, Richard ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2011-02-11 21:54 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-02-09 13:40 Fetch problems with PREMIRRORS Gary Thomas 2011-02-10 14:03 ` Richard Purdie 2011-02-10 14:51 ` Gary Thomas 2011-02-11 21:54 ` Richard Purdie
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.