bitbake-devel.lists.openembedded.org archive mirror
 help / color / mirror / Atom feed
* [bitbake-devel][PATCH] bitbake: add --noreply-timeout option
@ 2023-05-10  3:16 Qi.Chen
  2023-05-10  7:52 ` Richard Purdie
  0 siblings, 1 reply; 6+ messages in thread
From: Qi.Chen @ 2023-05-10  3:16 UTC (permalink / raw)
  To: bitbake-devel

From: Chen Qi <Qi.Chen@windriver.com>

For now, if the client gets no reply from server when running a
command, it exits after a period of time. The value is currently
60s. Looking at the history, this value was increased from 8s to
30s, and then it was increased again to 60s.

For now, what I can see is that when running one world build on a
128 core, 512G server, starting a second build has a chance to fail
at updateConfig.

Instead of increasing this value again and again, let's add an option
for easier customization of this value.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 bitbake/lib/bb/main.py           |  7 ++++++-
 bitbake/lib/bb/server/process.py | 19 ++++++++++---------
 2 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/bitbake/lib/bb/main.py b/bitbake/lib/bb/main.py
index 92d8dc0293..47c000b03e 100755
--- a/bitbake/lib/bb/main.py
+++ b/bitbake/lib/bb/main.py
@@ -267,6 +267,11 @@ def create_bitbake_parser():
                              "set to -1 means no unload, "
                              "default: Environment variable BB_SERVER_TIMEOUT.")
 
+    server_group.add_argument("--noreply-timeout", type=int, dest="noreply_timeout",
+                        default=60,
+                        help="Set timeout to exit bitbake client due to no reply from bitbake server, "
+                             "default: 60s.")
+
     server_group.add_argument("--remote-server",
                         default=os.environ.get("BBSERVER"),
                         help="Connect to the specified server.")
@@ -484,7 +489,7 @@ def setup_bitbake(configParams, extrafeatures=None):
                             bb.utils.unlockfile(lock)
                         raise bb.server.process.ProcessTimeout("Bitbake still shutting down as socket exists but no lock?")
                 if not configParams.server_only:
-                    server_connection = bb.server.process.connectProcessServer(sockname, featureset)
+                    server_connection = bb.server.process.connectProcessServer(sockname, featureset, configParams.noreply_timeout)
 
                 if server_connection or configParams.server_only:
                     break
diff --git a/bitbake/lib/bb/server/process.py b/bitbake/lib/bb/server/process.py
index db417c8428..8c7b6da64a 100644
--- a/bitbake/lib/bb/server/process.py
+++ b/bitbake/lib/bb/server/process.py
@@ -495,16 +495,17 @@ class ProcessServer():
 
 
 class ServerCommunicator():
-    def __init__(self, connection, recv):
+    def __init__(self, connection, recv, timeout):
         self.connection = connection
         self.recv = recv
+        self.timeout = timeout
 
     def runCommand(self, command):
         self.connection.send(command)
-        if not self.recv.poll(30):
-            logger.info("No reply from server in 30s")
-            if not self.recv.poll(30):
-                raise ProcessTimeout("Timeout while waiting for a reply from the bitbake server (60s)")
+        if not self.recv.poll(self.timeout / 2):
+            logger.info("No reply from server in %ss" % (self.timeout / 2))
+            if not self.recv.poll(self.timeout):
+                raise ProcessTimeout("Timeout while waiting for a reply from the bitbake server (%ss)" % self.timeout)
         ret, exc = self.recv.get()
         # Should probably turn all exceptions in exc back into exceptions?
         # For now, at least handle BBHandledException
@@ -531,8 +532,8 @@ class ServerCommunicator():
         return
 
 class BitBakeProcessServerConnection(object):
-    def __init__(self, ui_channel, recv, eq, sock):
-        self.connection = ServerCommunicator(ui_channel, recv)
+    def __init__(self, ui_channel, recv, eq, sock, timeout):
+        self.connection = ServerCommunicator(ui_channel, recv, timeout)
         self.events = eq
         # Save sock so it doesn't get gc'd for the life of our connection
         self.socket_connection = sock
@@ -666,7 +667,7 @@ def execServer(lockfd, readypipeinfd, lockname, sockname, server_timeout, xmlrpc
         sys.stdout.flush()
         sys.stderr.flush()
 
-def connectProcessServer(sockname, featureset):
+def connectProcessServer(sockname, featureset, timeout):
     # Connect to socket
     sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
     # AF_UNIX has path length issues so chdir here to workaround
@@ -704,7 +705,7 @@ def connectProcessServer(sockname, featureset):
 
         sendfds(sock, [writefd, readfd1, writefd2])
 
-        server_connection = BitBakeProcessServerConnection(command_chan, command_chan_recv, eq, sock)
+        server_connection = BitBakeProcessServerConnection(command_chan, command_chan_recv, eq, sock, timeout)
 
         # Close the ends of the pipes we won't use
         for i in [writefd, readfd1, writefd2]:
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-05-10 15:06 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-05-10  3:16 [bitbake-devel][PATCH] bitbake: add --noreply-timeout option Qi.Chen
2023-05-10  7:52 ` Richard Purdie
2023-05-10  9:43   ` Chen, Qi
2023-05-10 10:35     ` Richard Purdie
2023-05-10 11:40       ` Chen, Qi
     [not found]       ` <175DC558A674BAA1.32438@lists.openembedded.org>
2023-05-10 15:05         ` Chen, Qi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).