From: Qi.Chen@windriver.com
To: bitbake-devel@lists.openembedded.org
Subject: [bitbake-devel][PATCH] bitbake: add --noreply-timeout option
Date: Wed, 10 May 2023 11:16:31 +0800 [thread overview]
Message-ID: <20230510031631.1813338-1-Qi.Chen@windriver.com> (raw)
From: Chen Qi <Qi.Chen@windriver.com>
For now, if the client gets no reply from server when running a
command, it exits after a period of time. The value is currently
60s. Looking at the history, this value was increased from 8s to
30s, and then it was increased again to 60s.
For now, what I can see is that when running one world build on a
128 core, 512G server, starting a second build has a chance to fail
at updateConfig.
Instead of increasing this value again and again, let's add an option
for easier customization of this value.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
bitbake/lib/bb/main.py | 7 ++++++-
bitbake/lib/bb/server/process.py | 19 ++++++++++---------
2 files changed, 16 insertions(+), 10 deletions(-)
diff --git a/bitbake/lib/bb/main.py b/bitbake/lib/bb/main.py
index 92d8dc0293..47c000b03e 100755
--- a/bitbake/lib/bb/main.py
+++ b/bitbake/lib/bb/main.py
@@ -267,6 +267,11 @@ def create_bitbake_parser():
"set to -1 means no unload, "
"default: Environment variable BB_SERVER_TIMEOUT.")
+ server_group.add_argument("--noreply-timeout", type=int, dest="noreply_timeout",
+ default=60,
+ help="Set timeout to exit bitbake client due to no reply from bitbake server, "
+ "default: 60s.")
+
server_group.add_argument("--remote-server",
default=os.environ.get("BBSERVER"),
help="Connect to the specified server.")
@@ -484,7 +489,7 @@ def setup_bitbake(configParams, extrafeatures=None):
bb.utils.unlockfile(lock)
raise bb.server.process.ProcessTimeout("Bitbake still shutting down as socket exists but no lock?")
if not configParams.server_only:
- server_connection = bb.server.process.connectProcessServer(sockname, featureset)
+ server_connection = bb.server.process.connectProcessServer(sockname, featureset, configParams.noreply_timeout)
if server_connection or configParams.server_only:
break
diff --git a/bitbake/lib/bb/server/process.py b/bitbake/lib/bb/server/process.py
index db417c8428..8c7b6da64a 100644
--- a/bitbake/lib/bb/server/process.py
+++ b/bitbake/lib/bb/server/process.py
@@ -495,16 +495,17 @@ class ProcessServer():
class ServerCommunicator():
- def __init__(self, connection, recv):
+ def __init__(self, connection, recv, timeout):
self.connection = connection
self.recv = recv
+ self.timeout = timeout
def runCommand(self, command):
self.connection.send(command)
- if not self.recv.poll(30):
- logger.info("No reply from server in 30s")
- if not self.recv.poll(30):
- raise ProcessTimeout("Timeout while waiting for a reply from the bitbake server (60s)")
+ if not self.recv.poll(self.timeout / 2):
+ logger.info("No reply from server in %ss" % (self.timeout / 2))
+ if not self.recv.poll(self.timeout):
+ raise ProcessTimeout("Timeout while waiting for a reply from the bitbake server (%ss)" % self.timeout)
ret, exc = self.recv.get()
# Should probably turn all exceptions in exc back into exceptions?
# For now, at least handle BBHandledException
@@ -531,8 +532,8 @@ class ServerCommunicator():
return
class BitBakeProcessServerConnection(object):
- def __init__(self, ui_channel, recv, eq, sock):
- self.connection = ServerCommunicator(ui_channel, recv)
+ def __init__(self, ui_channel, recv, eq, sock, timeout):
+ self.connection = ServerCommunicator(ui_channel, recv, timeout)
self.events = eq
# Save sock so it doesn't get gc'd for the life of our connection
self.socket_connection = sock
@@ -666,7 +667,7 @@ def execServer(lockfd, readypipeinfd, lockname, sockname, server_timeout, xmlrpc
sys.stdout.flush()
sys.stderr.flush()
-def connectProcessServer(sockname, featureset):
+def connectProcessServer(sockname, featureset, timeout):
# Connect to socket
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
# AF_UNIX has path length issues so chdir here to workaround
@@ -704,7 +705,7 @@ def connectProcessServer(sockname, featureset):
sendfds(sock, [writefd, readfd1, writefd2])
- server_connection = BitBakeProcessServerConnection(command_chan, command_chan_recv, eq, sock)
+ server_connection = BitBakeProcessServerConnection(command_chan, command_chan_recv, eq, sock, timeout)
# Close the ends of the pipes we won't use
for i in [writefd, readfd1, writefd2]:
--
2.34.1
next reply other threads:[~2023-05-10 3:16 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-10 3:16 Qi.Chen [this message]
2023-05-10 7:52 ` [bitbake-devel][PATCH] bitbake: add --noreply-timeout option Richard Purdie
2023-05-10 9:43 ` Chen, Qi
2023-05-10 10:35 ` Richard Purdie
2023-05-10 11:40 ` Chen, Qi
[not found] ` <175DC558A674BAA1.32438@lists.openembedded.org>
2023-05-10 15:05 ` Chen, Qi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230510031631.1813338-1-Qi.Chen@windriver.com \
--to=qi.chen@windriver.com \
--cc=bitbake-devel@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).