* [PATCH 1/2] sstate: add a LockedSet class to be used on the mirror thread pool
@ 2022-04-19 9:46 Jose Quaresma
2022-04-19 9:46 ` [PATCH 2/2] sstate: reuse the localdata copy " Jose Quaresma
2022-04-28 12:30 ` [OE-core] [PATCH 1/2] sstate: add a LockedSet class to be used " Richard Purdie
0 siblings, 2 replies; 7+ messages in thread
From: Jose Quaresma @ 2022-04-19 9:46 UTC (permalink / raw)
To: openembedded-core; +Cc: Jose Quaresma
The python set() is not thread safe and we use it on the ThreadedPool.
With this LockedSet python class we can call the 'add' and 'remove'
safely inside the ThreadedPool.
This piece of code is taken from the stackoverflow
https://stackoverflow.com/questions/13610654/how-to-make-built-in-containers-sets-dicts-lists-thread-safe
May be related with [YOCTO #14775] -- https://bugzilla.yoctoproject.org/show_bug.cgi?id=14775
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
---
meta/classes/sstate.bbclass | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 1c0cae4893..ebc336b497 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -1008,6 +1008,21 @@ def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True,
if progress:
bb.event.fire(bb.event.ProcessProgress(msg, len(tasklist) - thread_worker.tasks.qsize()), d)
+ import threading
+ class LockedSet(set):
+ """A set where add() and remove() operator are thread-safe"""
+ def __init__(self, *args, **kwargs):
+ self._lock = threading.Lock()
+ super(LockedSet, self).__init__(*args, **kwargs)
+
+ def add(self, elem):
+ with self._lock:
+ super(LockedSet, self).add(elem)
+
+ def remove(self, elem):
+ with self._lock:
+ super(LockedSet, self).remove(elem)
+
tasklist = []
for tid in missed:
sstatefile = d.expand(getsstatefile(tid, siginfo, d))
@@ -1016,6 +1031,10 @@ def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True,
if tasklist:
nproc = min(int(d.getVar("BB_NUMBER_THREADS")), len(tasklist))
+ # this collections will be used on the thread pool so let's do it safely
+ found = LockedSet(found)
+ missed = LockedSet(missed)
+
progress = len(tasklist) >= 100
if progress:
msg = "Checking sstate mirror object availability"
--
2.35.3
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/2] sstate: reuse the localdata copy on the mirror thread pool
2022-04-19 9:46 [PATCH 1/2] sstate: add a LockedSet class to be used on the mirror thread pool Jose Quaresma
@ 2022-04-19 9:46 ` Jose Quaresma
2022-04-19 10:57 ` [OE-core] " Richard Purdie
2022-04-28 12:30 ` [OE-core] [PATCH 1/2] sstate: add a LockedSet class to be used " Richard Purdie
1 sibling, 1 reply; 7+ messages in thread
From: Jose Quaresma @ 2022-04-19 9:46 UTC (permalink / raw)
To: openembedded-core; +Cc: Jose Quaresma
We don't need a new copy of the bitbake data [localdata2] in each running
thread of the pool.
We can do the copy on the startup of the thread pool in each thread worker
and reuse it in each running thread, this is the same we did for the
connection_cache which is reused by fetcher.
May be related with [YOCTO #14775] -- https://bugzilla.yoctoproject.org/show_bug.cgi?id=14775
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
---
meta/classes/sstate.bbclass | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index ebc336b497..7526ec7562 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -979,6 +979,7 @@ def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True,
from bb.fetch2 import FetchConnectionCache
def checkstatus_init(thread_worker):
thread_worker.connection_cache = FetchConnectionCache()
+ thread_worker.localdata2 = bb.data.createCopy(localdata)
def checkstatus_end(thread_worker):
thread_worker.connection_cache.close_connections()
@@ -986,15 +987,14 @@ def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True,
def checkstatus(thread_worker, arg):
(tid, sstatefile) = arg
- localdata2 = bb.data.createCopy(localdata)
srcuri = "file://" + sstatefile
- localdata2.setVar('SRC_URI', srcuri)
+ thread_worker.localdata2.setVar('SRC_URI', srcuri)
bb.debug(2, "SState: Attempting to fetch %s" % srcuri)
import traceback
try:
- fetcher = bb.fetch2.Fetch(srcuri.split(), localdata2,
+ fetcher = bb.fetch2.Fetch(srcuri.split(), thread_worker.localdata2,
connection_cache=thread_worker.connection_cache)
fetcher.checkstatus()
bb.debug(2, "SState: Successful fetch test for %s" % srcuri)
--
2.35.3
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [OE-core] [PATCH 2/2] sstate: reuse the localdata copy on the mirror thread pool
2022-04-19 9:46 ` [PATCH 2/2] sstate: reuse the localdata copy " Jose Quaresma
@ 2022-04-19 10:57 ` Richard Purdie
2022-04-19 11:55 ` Jose Quaresma
0 siblings, 1 reply; 7+ messages in thread
From: Richard Purdie @ 2022-04-19 10:57 UTC (permalink / raw)
To: Jose Quaresma, openembedded-core
On Tue, 2022-04-19 at 10:46 +0100, Jose Quaresma wrote:
> We don't need a new copy of the bitbake data [localdata2] in each running
> thread of the pool.
>
> We can do the copy on the startup of the thread pool in each thread worker
> and reuse it in each running thread, this is the same we did for the
> connection_cache which is reused by fetcher.
>
> May be related with [YOCTO #14775] -- https://bugzilla.yoctoproject.org/show_bug.cgi?id=14775
>
> Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
> ---
> meta/classes/sstate.bbclass | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
It is a standard coding practise where you're changing the data store and don't
want changes to persist, to use createCopy() on the datastore. It is fast and
cheap and I'm not sure we need to make this thread specific?
Was there a reason you thought we should change that to fix some specific issue?
Cheers,
Richard
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] [PATCH 2/2] sstate: reuse the localdata copy on the mirror thread pool
2022-04-19 10:57 ` [OE-core] " Richard Purdie
@ 2022-04-19 11:55 ` Jose Quaresma
0 siblings, 0 replies; 7+ messages in thread
From: Jose Quaresma @ 2022-04-19 11:55 UTC (permalink / raw)
To: Richard Purdie; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 1843 bytes --]
Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia terça,
19/04/2022 à(s) 11:57:
> On Tue, 2022-04-19 at 10:46 +0100, Jose Quaresma wrote:
> > We don't need a new copy of the bitbake data [localdata2] in each running
> > thread of the pool.
> >
> > We can do the copy on the startup of the thread pool in each thread
> worker
> > and reuse it in each running thread, this is the same we did for the
> > connection_cache which is reused by fetcher.
> >
> > May be related with [YOCTO #14775] --
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14775
> >
> > Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
> > ---
> > meta/classes/sstate.bbclass | 6 +++---
> > 1 file changed, 3 insertions(+), 3 deletions(-)
>
> It is a standard coding practise where you're changing the data store and
> don't
> want changes to persist, to use createCopy() on the datastore. It is fast
> and
> cheap and I'm not sure we need to make this thread specific?
>
My interpretation of the use of the createCopy was so that the existing
one localdata, would not be used at the same time in the thread pool.
i.e: each thread has its own localdata2 copied from localdata.
In my patch the behavior is the same, each thread has its own localdata2
but the call of createCopy is done in a safe place.
>
> Was there a reason you thought we should change that to fix some specific
> issue?
>
I suspect that there may be some race condition when calling the createCopy
on the thread pool. If I understand well, the createCopy on the
bitbake/lib/bb/data_smart.py copies a couple of structures.
So I don't know if the call createCopy is safe to do inside the tread pool,
if you think so please drop this patch.
Jose
> Cheers,
>
> Richard
>
>
--
Best regards,
José Quaresma
[-- Attachment #2: Type: text/html, Size: 2985 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] [PATCH 1/2] sstate: add a LockedSet class to be used on the mirror thread pool
2022-04-19 9:46 [PATCH 1/2] sstate: add a LockedSet class to be used on the mirror thread pool Jose Quaresma
2022-04-19 9:46 ` [PATCH 2/2] sstate: reuse the localdata copy " Jose Quaresma
@ 2022-04-28 12:30 ` Richard Purdie
2022-04-29 13:32 ` Jose Quaresma
[not found] ` <16EA614949E42DAD.4798@lists.openembedded.org>
1 sibling, 2 replies; 7+ messages in thread
From: Richard Purdie @ 2022-04-28 12:30 UTC (permalink / raw)
To: Jose Quaresma, openembedded-core
On Tue, 2022-04-19 at 10:46 +0100, Jose Quaresma wrote:
> The python set() is not thread safe and we use it on the ThreadedPool.
> With this LockedSet python class we can call the 'add' and 'remove'
> safely inside the ThreadedPool.
>
> This piece of code is taken from the stackoverflow
> https://stackoverflow.com/questions/13610654/how-to-make-built-in-containers-sets-dicts-lists-thread-safe
>
> May be related with [YOCTO #14775] -- https://bugzilla.yoctoproject.org/show_bug.cgi?id=14775
>
> Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
For info, https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/5083
was a failure with master-next which included the changes in this discussion. It
doesn't seem to address that issue :(
Cheers,
Richard
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] [PATCH 1/2] sstate: add a LockedSet class to be used on the mirror thread pool
2022-04-28 12:30 ` [OE-core] [PATCH 1/2] sstate: add a LockedSet class to be used " Richard Purdie
@ 2022-04-29 13:32 ` Jose Quaresma
[not found] ` <16EA614949E42DAD.4798@lists.openembedded.org>
1 sibling, 0 replies; 7+ messages in thread
From: Jose Quaresma @ 2022-04-29 13:32 UTC (permalink / raw)
To: Richard Purdie; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 5569 bytes --]
Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia quinta,
28/04/2022 à(s) 13:30:
> On Tue, 2022-04-19 at 10:46 +0100, Jose Quaresma wrote:
> > The python set() is not thread safe and we use it on the ThreadedPool.
> > With this LockedSet python class we can call the 'add' and 'remove'
> > safely inside the ThreadedPool.
> >
> > This piece of code is taken from the stackoverflow
> >
> https://stackoverflow.com/questions/13610654/how-to-make-built-in-containers-sets-dicts-lists-thread-safe
> >
> > May be related with [YOCTO #14775] --
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14775
> >
> > Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
>
> For info,
> https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/5083
> was a failure with master-next which included the changes in this
> discussion. It
> doesn't seem to address that issue :(
>
> Cheers,
>
> Richard
>
>
Unfortunately this solution still doesn't solve the problem.
I'm going to analyze the log to try to understand what's going on,
however I leave here one of the tracebacks in case anyone can see an
anomaly here.
The mirror thread pool is considerably different in this patch series
which may imply that the problem may be on the server side but it is still
too early to draw conclusions.
I still haven't had time to think about how to do a test for this case.
Jose
ERROR: SState: cannot test
file://universal/f3/69/sstate:python3-wheel-native:x86_64-linux:0.37.1:r0:x86_64:8:f36956da8ee13cebb1fb6df615d0eb548a4bf637c50021c213d8c6f0973674bf_deploy_source_date_epoch.tar.zst:
TimeoutError('timed out')
Traceback (most recent call last):
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/meta/classes/sstate.bbclass",
line 1023, in checkstatus
fetcher.checkstatus()
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/__init__.py",
line 1829, in checkstatus
ret = m.try_mirrors(self, ud, self.d, mirrors, True)
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/__init__.py",
line 1602, in try_mirrors
return bool(try_mirrors(fetch, d, urldata, mirrors, check))
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/__init__.py",
line 1128, in try_mirrors
ret = try_mirror_url(fetch, origud, uds[index], ld, check)
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/__init__.py",
line 1023, in try_mirror_url
found = ud.method.checkstatus(fetch, ud, ld)
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/wget.py",
line 361, in checkstatus
with opener.open(r, timeout=30) as response:
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/urllib/request.py",
line 519, in open
response = self._open(req, data)
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/urllib/request.py",
line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/urllib/request.py",
line 496, in _call_chain
result = func(*args)
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/wget.py",
line 166, in http_open
return self.do_open(HTTPConnectionCache, req)
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/wget.py",
line 236, in do_open
r = h.getresponse()
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/http/client.py",
line 1374, in getresponse
response.begin()
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/http/client.py",
line 318, in begin
version, status, reason = self._read_status()
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/http/client.py",
line 279, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/socket.py",
line 705, in readinto
return self._sock.recv_into(b)
TimeoutError: timed out
--
Best regards,
José Quaresma
[-- Attachment #2: Type: text/html, Size: 18577 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] [PATCH 1/2] sstate: add a LockedSet class to be used on the mirror thread pool
[not found] ` <16EA614949E42DAD.4798@lists.openembedded.org>
@ 2022-04-29 13:37 ` Jose Quaresma
0 siblings, 0 replies; 7+ messages in thread
From: Jose Quaresma @ 2022-04-29 13:37 UTC (permalink / raw)
To: Jose Quaresma; +Cc: Richard Purdie, OE-core
[-- Attachment #1: Type: text/plain, Size: 10661 bytes --]
Jose Quaresma via lists.openembedded.org <quaresma.jose=
gmail.com@lists.openembedded.org> escreveu no dia sexta, 29/04/2022 à(s)
14:33:
>
>
> Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia
> quinta, 28/04/2022 à(s) 13:30:
>
>> On Tue, 2022-04-19 at 10:46 +0100, Jose Quaresma wrote:
>> > The python set() is not thread safe and we use it on the ThreadedPool.
>> > With this LockedSet python class we can call the 'add' and 'remove'
>> > safely inside the ThreadedPool.
>> >
>> > This piece of code is taken from the stackoverflow
>> >
>> https://stackoverflow.com/questions/13610654/how-to-make-built-in-containers-sets-dicts-lists-thread-safe
>> >
>> > May be related with [YOCTO #14775] --
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14775
>> >
>> > Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
>>
>> For info,
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/5083
>> was a failure with master-next which included the changes in this
>> discussion. It
>> doesn't seem to address that issue :(
>>
>> Cheers,
>>
>> Richard
>>
>>
> Unfortunately this solution still doesn't solve the problem.
>
> I'm going to analyze the log to try to understand what's going on,
> however I leave here one of the tracebacks in case anyone can see an
> anomaly here.
>
> The mirror thread pool is considerably different in this patch series
> which may imply that the problem may be on the server side but it is still
> too early to draw conclusions.
>
> I still haven't had time to think about how to do a test for this case.
>
> Jose
>
> ERROR: SState: cannot test
> file://universal/f3/69/sstate:python3-wheel-native:x86_64-linux:0.37.1:r0:x86_64:8:f36956da8ee13cebb1fb6df615d0eb548a4bf637c50021c213d8c6f0973674bf_deploy_source_date_epoch.tar.zst:
> TimeoutError('timed out')
> Traceback (most recent call last):
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/meta/classes/sstate.bbclass",
> line 1023, in checkstatus
> fetcher.checkstatus()
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/__init__.py",
> line 1829, in checkstatus
> ret = m.try_mirrors(self, ud, self.d, mirrors, True)
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/__init__.py",
> line 1602, in try_mirrors
> return bool(try_mirrors(fetch, d, urldata, mirrors, check))
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/__init__.py",
> line 1128, in try_mirrors
> ret = try_mirror_url(fetch, origud, uds[index], ld, check)
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/__init__.py",
> line 1023, in try_mirror_url
> found = ud.method.checkstatus(fetch, ud, ld)
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/wget.py",
> line 361, in checkstatus
> with opener.open(r, timeout=30) as response:
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/urllib/request.py",
> line 519, in open
> response = self._open(req, data)
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/urllib/request.py",
> line 536, in _open
> result = self._call_chain(self.handle_open, protocol, protocol +
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/urllib/request.py",
> line 496, in _call_chain
> result = func(*args)
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/wget.py",
> line 166, in http_open
> return self.do_open(HTTPConnectionCache, req)
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/wget.py",
> line 236, in do_open
> r = h.getresponse()
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/http/client.py",
> line 1374, in getresponse
> response.begin()
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/http/client.py",
> line 318, in begin
> version, status, reason = self._read_status()
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/http/client.py",
> line 279, in _read_status
> line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
> File
> "/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/socket.py",
> line 705, in readinto
> return self._sock.recv_into(b)
> TimeoutError: timed out
>
>
Latest tracebacks are messed up so here is a copy of it:
ERROR: SState: cannot test
file://universal/f3/69/sstate:python3-wheel-native:x86_64-linux:0.37.1:r0:x86_64:8:f36956da8ee13cebb1fb6df615d0eb548a4bf637c50021c213d8c6f0973674bf_deploy_source_date_epoch.tar.zst:
TimeoutError('timed out')
Traceback (most recent call last):
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/meta/classes/sstate.bbclass",
line 1023, in checkstatus
fetcher.checkstatus()
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/__init__.py",
line 1829, in checkstatus
ret = m.try_mirrors(self, ud, self.d, mirrors, True)
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/__init__.py",
line 1602, in try_mirrors
return bool(try_mirrors(fetch, d, urldata, mirrors, check))
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/__init__.py",
line 1128, in try_mirrors
ret = try_mirror_url(fetch, origud, uds[index], ld, check)
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/__init__.py",
line 1023, in try_mirror_url
found = ud.method.checkstatus(fetch, ud, ld)
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/wget.py",
line 361, in checkstatus
with opener.open(r, timeout=30) as response:
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/urllib/request.py",
line 519, in open
response = self._open(req, data)
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/urllib/request.py",
line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/urllib/request.py",
line 496, in _call_chain
result = func(*args)
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/wget.py",
line 166, in http_open
return self.do_open(HTTPConnectionCache, req)
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/layers/build/bitbake/lib/bb/fetch2/wget.py",
line 236, in do_open
r = h.getresponse()
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/http/client.py",
line 1374, in getresponse
response.begin()
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/http/client.py",
line 318, in begin
version, status, reason = self._read_status()
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/http/client.py",
line 279, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File
"/home/pokybuild/yocto-worker/qemumips64/build/build/tmp/work/qemumips64-poky-linux/core-image-sato/1.0-r0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.10/socket.py",
line 705, in readinto
return self._sock.recv_into(b)
TimeoutError: timed out
> --
> Best regards,
>
> José Quaresma
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#165013):
> https://lists.openembedded.org/g/openembedded-core/message/165013
> Mute This Topic: https://lists.openembedded.org/mt/90558156/5052612
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
--
Best regards,
José Quaresma
[-- Attachment #2: Type: text/html, Size: 22883 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-04-29 13:38 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-19 9:46 [PATCH 1/2] sstate: add a LockedSet class to be used on the mirror thread pool Jose Quaresma
2022-04-19 9:46 ` [PATCH 2/2] sstate: reuse the localdata copy " Jose Quaresma
2022-04-19 10:57 ` [OE-core] " Richard Purdie
2022-04-19 11:55 ` Jose Quaresma
2022-04-28 12:30 ` [OE-core] [PATCH 1/2] sstate: add a LockedSet class to be used " Richard Purdie
2022-04-29 13:32 ` Jose Quaresma
[not found] ` <16EA614949E42DAD.4798@lists.openembedded.org>
2022-04-29 13:37 ` Jose Quaresma
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.